Xenharmonic Wiki

Harmonic entropy: Difference between revisions

← Older editNewer edit →
VisualWikitext
Mike Battaglia (talk | contribs)
autopatrolled, Bureaucrats, confirmaccount-notify, emailconfirmed, Interface administrators, Sysops
1,399 edits
→Examples: formatting
Mike Battaglia (talk | contribs)
autopatrolled, Bureaucrats, confirmaccount-notify, emailconfirmed, Interface administrators, Sysops
1,399 edits
Line 581: Line 581:


=== Apparent Equivalence of exp-UHE and UHE for <math>a \leq 2</math> ===
=== Apparent Equivalence of exp-UHE and UHE for <math>a \leq 2</math> ===
''Note: this section is for future research; some of it needs to be put on more rigorous footing, but we've left it as it's certainly interesting.''


Let's go back to our original convolution expression for finite-<math>N</math> UHE:
Let's go back to our original convolution expression for finite-<math>N</math> UHE:


$$\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log \left( S^a \ast K^a \right)(-c)$$
$$\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log \left(\left( S^a \ast K^a \right)(-c)\right)$$


We will also, for now, use the notation
We will also, for now, use the notation
Line 598: Line 600:
$$\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log \left(U(0) + \tilde{U}(c) \right)$$
$$\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log \left(U(0) + \tilde{U}(c) \right)$$


We can expand the above into a Taylor series as follows:
Lastly, suppose we only care about the entropy function up to a vertical shift and scaling: in other words, we want to declare two functions <math>f(x), g(x)</math> to be '''linearly equivalent''', and write <math>f(x) \approx g(x)</math>, if for some <math>a, b</math> that don't depend on <math>x</math>, we have <math>f(x) = a\cdot g(x) + b</math>. This means we want to view two entropy functions as equivalent if one is just a scaled and shifted version of the other, so that when "normalizing" them (so that the entropy goes from 0 to 1), we get identical functions. Then we have all of the following relationships:
 
$$\displaystyle \text{UHE}_a(c) \approx \log U(c) \approx \log \left( U(c)^{\frac{1}{1-a}} \right)$$
$$U(c) \approx \tilde{U}(c)$$
 
where we have just dropped the constants of <math>\frac{1}{1-a}</math> and the constant vertical shift of <math>U(0)</math> which doesn't depend on <math>c</math>.
 
Now, the main thing is that, if we are in the region where <math>a ≤ 2</math>, then this is also the region where the <math>U(0)</math> term goes to infinity as <math>N</math> increases: the entropy doesn't converge. And in general, we have the asymptotic expansion
 
$$
\displaystyle
\log(k + x) \sim \log(k) + x/k + \ldots
$$
 
and, for large <math>k</math>, '''as long as''' <math>x \ll k</math>, the higher-order terms become negligible. This means, for all <math>c</math>, we would need to show that <math>\tilde{U}(c) \ll U(0)</math> as <math>N \to \infty</math>. We would then be able to rewrite the above as
 
$$\displaystyle \log U(c) \sim \frac{1}{1-a} \left (\log (U(0)) + \frac{\tilde{U}(c)}{U(0)} \right)$$


$$\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \left(\log(U(0)) + \frac{\tilde{U}(c)}{U(0)} - \frac{\tilde{U}(c)^2}{2 U(0)^2} + \frac{\tilde{U}(c)^3}{3 U(0)^3} - ...\right)$$
This means that, as <math>N \to \infty</math>, we would also get the following linear equivalence:


Now, suppose we only care about the behavior of this function up to a constant vertical shift and scaling. Then we can drop the <math>\frac{1}{1-a}</math> term, since it's just a constant scaling, and also get rid of the <math>\log(U(0))</math> term, which simply subtracts a constant offset. This leaves us with
$$\displaystyle \text{UHE}_a(c) \approx \tilde{U}(c)$$


$$\displaystyle \text{UHE}_a(c) \sim \left(\frac{\tilde{U}(c)}{U(0)} - \frac{\tilde{U}(c)^2}{2 U(0)^2} + \frac{\tilde{U}(c)^3}{3 U(0)^3} - ...\right)$$
Putting this with our earlier result that <math>U(c) \approx \tilde{U}(c)</math>, we get


where <math>\sim</math> denotes the two sides are now "equivalent" up to a constant shifting and scaling.
$$\displaystyle \text{UHE}_a(c) \approx U(c) \approx \log U(c)$$


Finally, we can go one step further and multiply the above by the constant <math>U(0)</math>. Doing so, we get
Now, to get the exp-UHE directly, we substitute in from our earlier definition to note


$$\displaystyle \text{UHE}_a(c) \sim \left(\tilde{U}(c) - \frac{\tilde{U}(c)^2}{2 U(0)} + \frac{\tilde{U}(c)^3}{3 U(0)^2} - ...\right)$$
$$
\displaystyle
\exp(\text{UHE}_a(c)) = U(c)^{\frac{1}{1-a}}
$$


And we now have a function that is equivalent to our original, up to a constant shift and scaling, but which is fairly easy to analyze asymptotically.
We can perform the same substitution of <math>U(c) = U(0) + \tilde{U}(c)</math> again to get


Now, for the Shannon entropy, we know that the above doesn't converge as <math>N \to \infty</math>. Furthermore, after our above analysis, we have seen that it doesn't converge for <math>a \leq 2</math>, corresponding to the region of the Riemann zeta function where <math>\Re(s) < 1</math>.
$$
\displaystyle
\exp(\text{UHE}_a(c)) = \left( U(0) + \tilde{U}(c) \right)^{\frac{1}{1-a}}
$$


However, in the case of <math>a \leq 2</math>, we note empirically that while the function diverges, it diverges in a certain "uniform" sense. That is, as <math>N</math> increases, a constant vertical offset is added to <math>U(c)</math>, so that the function blows up to infinity. However, if this vertical offset is corrected for, for example by subtracting U(0), the resulting curve doesn't seem to grow at all, but rather shrinks in height slightly until it seems to converge. We can see this from the following plot:
Given that <math>a \neq 0</math>, we have a Taylor expansion of <math>(k+x)^{\frac{1}{1-a}}</math> around <math>x = 0</math> of the form


[[File:ExpUHE-asymptotic-growth.png|800px]]
$$
\displaystyle
k^{\frac{1}{1-a}}\left(1 + \frac{x}{k(1-a)} + \frac{x^2}{2k^2(1-a)^2} + \ldots\right)
$$
 
where we have factored out the leading term for clarity. Now, again, we have that if <math>x \ll k</math> -- meaning that <math>\tilde U(c) \ll U(0)</math> -- then the higher-order terms become negligible, and we simply have the asymptotic relationship of


In other words, we can see that as <math>N</math> increases, the growth rate of <math>U(0)</math> dwarfs that of <math>\tilde{U}(c)</math>, which does not seem to grow at all.
$$
\displaystyle
\exp(\text{UHE}_a(c)) \sim U(0)^{\frac{1}{1-a}}\left(1 + \frac{\tilde U(c)}{U(0)(1-a)} \right)
$$


Assuming this is true (which we will not prove here, but which seems self-evident given all of the other results), we can go back to our last expression for exp-UHE:
Lastly, we note that for any particular choice of <math>a</math> and <math>N</math>, the above is simply linearly equivalent to


$$\displaystyle \text{UHE}_a(c) \sim \left(\tilde{U}(c) - \frac{\tilde{U}(c)^2}{U(0)} + \frac{\tilde{U}(c)^3}{U(0)} - ...\right)$$
$$
\displaystyle
U(0)^{\frac{1}{1-a}}\left(1 + \frac{\tilde U(c)}{U(0)(1-a)}\right) \approx \tilde U(c)
$$


and see that as <math>N \to \infty</math>, the <math>U(0)</math> terms blow up, whereas the <math>\tilde{U}(c)</math> terms do not. As a result, all of the terms with <math>U(0)</math> in the denominator disappear, and we are left with:
So that, overall, as <math>N \to \infty</math>, the following equivalence holds:


$$\displaystyle \text{UHE}_a(c) \sim \tilde{U}(c)$$
$$
\displaystyle
\exp(\text{UHE}_a(c)) \approx \tilde{U}(c)
$$


Putting this together from our prior realization that <math>\text{UHE}_a(c) \sim \log \tilde{U}(c)</math>, we get
And since we have already shown that <math>\text{UHE}_a(c) \approx \tilde{U}(c)</math>, we have


before, we get
$$
\displaystyle
\exp(\text{UHE}_a(c)) \approx \text{UHE}_a(c)
$$


$$\displaystyle \tilde{U}(c) \sim \log \tilde{U}(c)$$
Now, the only missing piece needed for all of this is to show that we really do have <math>\tilde{U}(c) \ll U(0)</math> in the region of interest. For now, absent mathematical proof, we will simply plot the behavior for the Shannon entropy as <math>N \to \infty</math>.


additionally, noting that <math>\log \tilde{U}(c) \sim \frac{1}{1-a} \log \tilde{U}(c) = \log \tilde{U}(c)^{\frac{1}{1-a}} \sim \tilde{U}(c)^{\frac{1}{1-a}} = \exp(\text{UHE}_a(c))</math>, we get
What we see is that, while the function diverges, it diverges in a certain "uniform" sense. That is, as <math>N</math> increases, a constant vertical offset is added to <math>U(c)</math>, so that the function blows up to infinity. However, if this vertical offset is corrected for, for example by subtracting U(0), the resulting curve doesn't seem to grow at all, but rather shrinks in height slightly until it seems to converge. We would like to prove this formally, but for now, we can at least see this from the following plot:


$$\displaystyle \text{UHE}_a(c) \sim \exp(\text{UHE}_a(c))$$
[[File:ExpUHE-asymptotic-growth.png|800px]]


as <math>N \to \infty</math>, for <math>a \leq 2</math>.
In other words, we can see that as <math>N</math> increases, the growth rate of <math>U(0)</math> dwarfs that of <math>\tilde{U}(c)</math>, which does not seem to grow at all.


We note that this proof is entirely dependent on the growth rate of <math>\tilde{U}(c)</math> being dwarfed by that of <math>U(0)</math>. This is a fairly weak conjecture to make, given that empirical evidence suggests something much stronger - that not only does it grow more slowly, but that it seems to converge. In particular, it seems to converge on our analytic continuation from before. However, a strict proof of any of these things would be nice.
So, this is a fairly weak conjecture to make, given that empirical evidence suggests something much stronger - that not only does it grow more slowly, but that it seems to not grow at all - it converges! In particular, it seems to converge on our analytic continuation from before. However, a strict proof of any of these things would be nice.


Note again that this does not hold for <math>a \gt 2</math>, where the graph does display a very large difference between UHE and exp-UHE.
Note again that this does not hold for <math>a \gt 2</math>, where the graph does display a very large difference between UHE and exp-UHE.
Retrieved from "https://en.xen.wiki/w/Harmonic_entropy"