Harmonic entropy: Difference between revisions

Line 578:

Note that you have to be careful if you choose to compute the analytically continued <math>a=2</math> numerically: this corresponds to the vertical slice of zeta function at <math>\Re(z) = 1</math>, where there is a pole.

==Equivalence of exp-UHE and UHE for <math>a \leq 2</math>==

Let's go back to our original convolution expression for finite-<math>N</math> UHE:

<math>\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log \left( S^a \ast K^a \right)(-c)</math>

We will also, for now, use the notation

<math>U(c) = \exp((1-a) \text{UHE}_a(c)) = \left( S^a \ast K^a \right)(-c)</math>

to get

<math>\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log U(c)</math>

Now let's consider the auxiliary function <math>\tilde{U}(c) = U(c) - U(0)</math>, which gives us a "shifted" version of the UHE which has the UHE of 1/1 normalized to 0. Then we can re-express the UHE as follows:

<math>\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log \left(U(0) + \tilde{U}(c) \right)</math>

We can expand the above into a Taylor series as follows:

<math>\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \left(\log(U(0)) + \frac{\tilde{U}(c)}{U(0)} - \frac{\tilde{U}(c)^2}{U(0)^2} + \frac{\tilde{U}(c)^3}{U(0)^3} - ...\right)</math>

Now, suppose we only care about the behavior of this function up to a constant vertical shift and scaling. Then we can drop the <math>\frac{1}{1-a}</math> term, since it's just a constant scaling, and also get rid of the <math>\log(U(0))</math> term, which simply subtracts a constant offset. This leaves us with

<math>\displaystyle \text{UHE}_a(c) \sim \left(\frac{\tilde{U}(c)}{U(0)} - \frac{\tilde{U}(c)^2}{U(0)^2} + \frac{\tilde{U}(c)^3}{U(0)^3} - ...\right)</math>

where <math>\sim</math> denotes the two sides are now "equivalent" up to a constant shifting and scaling.

Finally, we can go one step further and multiply the above by the constant <math>U(0)</math>. Doing so, we get

<math>\displaystyle \text{UHE}_a(c) \sim \left(\tilde{U}(c) - \frac{\tilde{U}(c)^2}{U(0)} + \frac{\tilde{U}(c)^3}{U(0)} - ...\right)</math>

And we now have a function that is equivalent to our original, up to a constant shift and scaling, but which is fairly easy to analyze asymptotically.

Now, for the Shannon entropy, we know that the above doesn't converge as <math>N \to \infty</math>. Furthermore, after our above analysis, we have seen that it doesn't converge for <math>a \leq 2</math>, corresponding to the region of the Riemann zeta function where <math>\Re(s) < 1</math>.

However, in the case of <math>a \leq 2</math>, we note empirically that while the function diverges, it diverges in a certain "uniform" sense. That is, as <math>N</math> increases, a constant vertical offset is added to <math>U(c)</math>, so that the function blows up to infinity. However, if this vertical offset is corrected for, for example by subtracting U(0), the resulting curve doesn't seem to grow at all, but rather shrinks in height slightly until it seems to converge. We can see this from the following plot:

[[File:ExpUHE-asymptotic-growth.png|800px]]

In other words, we can see that as <math>N</math> increases, the growth rate of <math>U(0)</math> dwarfs that of <math>\tilde{U}(c)</math>, which does not seem to grow at all.

Assuming this is true (which we will not prove here, but which seems self-evident given all of the other results), we can go back to our last expression for exp-UHE:

<math>\displaystyle \text{UHE}_a(c) \sim \left(\tilde{U}(c) - \frac{\tilde{U}(c)^2}{U(0)} + \frac{\tilde{U}(c)^3}{U(0)} - ...\right)</math>

and see that as <math>N \to \infty</math>, the <math>U(0)</math> terms blow up, whereas the <math>\tilde{U}(c)</math> terms do not. As a result, all of the terms with <math>U(0)</math> in the denominator disappear, and we are left with:

<math>\displaystyle \text{UHE}_a(c) \sim \tilde{U}(c)</math>

Putting this together from our prior realization that <math>\text{UHE}_a(c) \sim \log \tilde{U}(c)</math>, we get

before, we get

<math>\displaystyle \tilde{U}(c) \sim \log \tilde{U}(c)</math>

additionally, noting that <math>\log \tilde{U}(c) \sim \frac{1}{1-a} \log \tilde{U}(c) = \log \tilde{U}(c)^{\frac{1}{1-a}} \sim \tilde{U}(c)^{\frac{1}{1-a}} = \exp(\text{UHE_a}(c))</math>, we get

<math>\displaystyle \text{UHE}_a(c) \sim \exp(\text{UHE_a}(c))</math>

as <math>N \to \infty</math>, for <math>a \leq 2</math>.

We note that this proof is entirely dependent on the growth rate of <math>\tilde{U}(c)</math> being dwarfed by that of <math>U(0)</math>. This is a fairly weak conjecture to make, given that empirical evidence suggests something much stronger - that not only does it grow more slowly, but that it seems to converge. In particular, it seems to converge on our analytic continuation from before. However, a strict proof of any of these things would be nice.

Note again that this does not hold for <math>a \gt 2</math>, where the graph does display a very large difference between UHE and exp-UHE.

==Why not Normalized HE?==

@@ Line 578: / Line 578: @@
 Note that you have to be careful if you choose to compute the analytically continued <math>a=2</math> numerically: this corresponds to the vertical slice of zeta function at <math>\Re(z) = 1</math>, where there is a pole.
+==Equivalence of exp-UHE and UHE for <math>a \leq 2</math>==
+Let's go back to our original convolution expression for finite-<math>N</math> UHE:
+<math>\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log \left( S^a \ast K^a \right)(-c)</math>
+We will also, for now, use the notation
+<math>U(c) = \exp((1-a) \text{UHE}_a(c)) = \left( S^a \ast K^a \right)(-c)</math>
+to get
+<math>\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log U(c)</math>
+Now let's consider the auxiliary function <math>\tilde{U}(c) = U(c) - U(0)</math>, which gives us a "shifted" version of the UHE which has the UHE of 1/1 normalized to 0. Then we can re-express the UHE as follows:
+<math>\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \log \left(U(0) + \tilde{U}(c) \right)</math>
+We can expand the above into a Taylor series as follows:
+<math>\displaystyle \text{UHE}_a(c) = \frac{1}{1-a} \left(\log(U(0)) + \frac{\tilde{U}(c)}{U(0)} - \frac{\tilde{U}(c)^2}{U(0)^2} + \frac{\tilde{U}(c)^3}{U(0)^3} - ...\right)</math>
+Now, suppose we only care about the behavior of this function up to a constant vertical shift and scaling. Then we can drop the <math>\frac{1}{1-a}</math> term, since it's just a constant scaling, and also get rid of the <math>\log(U(0))</math> term, which simply subtracts a constant offset. This leaves us with
+<math>\displaystyle \text{UHE}_a(c) \sim \left(\frac{\tilde{U}(c)}{U(0)} - \frac{\tilde{U}(c)^2}{U(0)^2} + \frac{\tilde{U}(c)^3}{U(0)^3} - ...\right)</math>
+where <math>\sim</math> denotes the two sides are now "equivalent" up to a constant shifting and scaling.
+Finally, we can go one step further and multiply the above by the constant <math>U(0)</math>. Doing so, we get
+<math>\displaystyle \text{UHE}_a(c) \sim \left(\tilde{U}(c) - \frac{\tilde{U}(c)^2}{U(0)} + \frac{\tilde{U}(c)^3}{U(0)} - ...\right)</math>
+And we now have a function that is equivalent to our original, up to a constant shift and scaling, but which is fairly easy to analyze asymptotically.
+Now, for the Shannon entropy, we know that the above doesn't converge as <math>N \to \infty</math>. Furthermore, after our above analysis, we have seen that it doesn't converge for <math>a \leq 2</math>, corresponding to the region of the Riemann zeta function where <math>\Re(s) < 1</math>.
+However, in the case of <math>a \leq 2</math>, we note empirically that while the function diverges, it diverges in a certain "uniform" sense. That is, as <math>N</math> increases, a constant vertical offset is added to <math>U(c)</math>, so that the function blows up to infinity. However, if this vertical offset is corrected for, for example by subtracting U(0), the resulting curve doesn't seem to grow at all, but rather shrinks in height slightly until it seems to converge. We can see this from the following plot:
+[[File:ExpUHE-asymptotic-growth.png|800px]]
+In other words, we can see that as <math>N</math> increases, the growth rate of <math>U(0)</math> dwarfs that of <math>\tilde{U}(c)</math>, which does not seem to grow at all.
+Assuming this is true (which we will not prove here, but which seems self-evident given all of the other results), we can go back to our last expression for exp-UHE:
+<math>\displaystyle \text{UHE}_a(c) \sim \left(\tilde{U}(c) - \frac{\tilde{U}(c)^2}{U(0)} + \frac{\tilde{U}(c)^3}{U(0)} - ...\right)</math>
+and see that as <math>N \to \infty</math>, the <math>U(0)</math> terms blow up, whereas the <math>\tilde{U}(c)</math> terms do not. As a result, all of the terms with <math>U(0)</math> in the denominator disappear, and we are left with:
+<math>\displaystyle \text{UHE}_a(c) \sim \tilde{U}(c)</math>
+Putting this together from our prior realization that <math>\text{UHE}_a(c) \sim \log \tilde{U}(c)</math>, we get
+before, we get
+<math>\displaystyle \tilde{U}(c) \sim \log \tilde{U}(c)</math>
+additionally, noting that <math>\log \tilde{U}(c) \sim \frac{1}{1-a} \log \tilde{U}(c) = \log \tilde{U}(c)^{\frac{1}{1-a}} \sim \tilde{U}(c)^{\frac{1}{1-a}} = \exp(\text{UHE_a}(c))</math>, we get
+<math>\displaystyle \text{UHE}_a(c) \sim \exp(\text{UHE_a}(c))</math>
+as <math>N \to \infty</math>, for <math>a \leq 2</math>.
+We note that this proof is entirely dependent on the growth rate of <math>\tilde{U}(c)</math> being dwarfed by that of <math>U(0)</math>. This is a fairly weak conjecture to make, given that empirical evidence suggests something much stronger - that not only does it grow more slowly, but that it seems to converge. In particular, it seems to converge on our analytic continuation from before. However, a strict proof of any of these things would be nice.
+Note again that this does not hold for <math>a \gt 2</math>, where the graph does display a very large difference between UHE and exp-UHE.
 ==Why not Normalized HE?==