Harmonic entropy: Difference between revisions

Line 76:

* The spreading function is typically a Gaussian distribution with a frequency deviation of 1% either way, or about s=~17 cents.

Other spreading functions have also been explored, such as the use of the heavy-tailed [https://en.wikipedia.org/wiki/Laplace_distribution Laplace distribution], sometimes described as the "Vos function" in Paul's writings. We will assume the Gaussian distribution as the spreading function for the remainder of this article, so that the spreading function for an incoming dyad <math>c</math> can be written as follows:

Other spreading functions have also been explored, such as the use of the heavy-tailed [https://en.wikipedia.org/wiki/Laplace_distribution Laplace distribution], sometimes described as the "Vos function" in Paul's writings. These two functions are part of the [https://en.wikipedia.org/wiki/Generalized_normal_distribution Generalized normal distribution] family, which has a parameter not only for the variance but for the kurtosis. However, for simplicity, we will assume the Gaussian distribution as the spreading function for the remainder of this article, so that the spreading function for an incoming dyad <math>c</math> can be written as follows:

Line 217:

''Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=''∞''). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the complexity function <math>\sqrt{nd}</math>.''

==Convolution-Based Expression For Quickly Computing ~~Renyi~~ Entropy==

==Convolution-Based Expression For Quickly Computing Rényi Entropy==

Below is given an derivation that expresses Harmonic ~~Renyi~~ Entropy in terms of two simpler functions, each of which is a convolution product and hence can be computed quickly using the Fast Fourier Transform.

Below is given an derivation that expresses Harmonic Rényi Entropy in terms of two simpler functions, each of which is a convolution product and hence can be computed quickly using the Fast Fourier Transform.

The below derivation depends on the use of complexity-normalization probabilities, although it may be possible to extend to domain-integral probabilities instead.

===Preliminaries===

The Harmonic ~~Renyi~~ Entropy is defined as

The Harmonic Rényi Entropy is defined as

Line 254:

Finally, we put this all together to obtain a simplified version of the Harmonic ~~Renyi~~ Entropy equation:

Finally, we put this all together to obtain a simplified version of the Harmonic Rényi Entropy equation:

<math>\text{HE}_a(c) = H_a(J=j|C=c) = \frac{1}{1-a} \log \left( \frac{\rho_a(c)}{\psi(c)^a} \right)</math>

Line 289:

We can clean up this notation by defining the auxiliary distribution K:

<math>K(c) = ~~\left(~~\sum_{j \in J} \frac{\delta_{-\cent(j)}}{\|j\|}~~\right)~~</math>

<math>K(c) = \sum_{j \in J} \frac{\delta_{-\cent(j)}}{\|j\|}</math>

Line 319:

And again we clean up notation by defining the auxiliary distribution

<math>K^a(c) = ~~\left(~~\sum_{j \in J} \frac{\delta_{-\cent(j)}}{\|j\|^a}~~\right)~~</math>

<math>K^a(c) = \sum_{j \in J} \frac{\delta_{-\cent(j)}}{\|j\|^a}</math>

so that

Line 331:

===Round-up===

Taking all of this, we can rewrite the original expression for Harmonic ~~Renyi~~ Entropy as follows:

Taking all of this, we can rewrite the original expression for Harmonic Rényi Entropy as follows:

<math>\text{HE}_a(c) = H_a(J=j|C=c) = \frac{1}{1-a} \log \left( \frac{\left[S^a \ast K^a\right](-c)}{\left[S \ast K\right]^a(-c)} \right)</math>

Line 343:

<math>\left[S \ast K\right]^a(-c) = \left[S \ast K\right]^a(c)</math>

We have succeeded in representing Harmonic ~~Renyi~~ Entropy in simple terms of two convolution products, each of which can be computed in <math>O(N log N)</math> time.

We have succeeded in representing Harmonic Rényi Entropy in simple terms of two convolution products, each of which can be computed in <math>O(N log N)</math> time.

=Extending HE to N=∞=

All of the models described above involve a finite set of rational numbers, bounded by some complexity function, and where the complexity is less than some max value <math>N</math>.

It so happens that we are able to analytically continue this definition to the situation where <math>N=\infty</math>, with one mathematical caveat. This enables us to speak cognizantly of the harmonic entropy of an interval as measured against ''all'' rational numbers.

==Background: Unnormalized Entropy==

The caveat is that our derivation only analytically continues the entropy function for the "unnormalized" set of probabilities, which we previously wrote as <math>\hat P(J=j|C=c)</math>. For this definition to be philosophically perfect, we would want to analytically continue the entropy function for the normalized sense of probabilities, previously written as <math>P(J=j|C=c)</math>.

However, in practice, the "unnormalized entropy" appears to be an extremely good approximation to the normalized entropy for large values of <math>N</math>. The resulting curve has the same minima and maxima as HE, the same general shape, and for all intents and purposes looks exactly like HE, just shifted on the y-axis.

It would be nice to show that the unnormalized entropy converges to the normalized entropy, up to a constant additive shift or multiplicative scaling, in the limit of large <math>N</math>. However, we will leave this for future research, as well as the question of how to do an exact derivation of the normalized entropy function.

For now, we will start with a derivation of the unnormalized entropy for <math>N=\infty</math>, as an interesting function worthy of study in its own right - not only because it looks exactly like HE, but because it leads to an expression for unnormalized HE in terms of the [[The_Riemann_Zeta_Function_and_Tuning|Riemann Zeta function]].

==Derivation==

For now, our derivation is limited to the case of <math>\sqrt{nd}</math> Tenney-weighted rationals, although it may be possible to derive a similar result for <math>\max(n,d)</math> weighting as well.

===Analytic Continuation of the Convolution Kernel===

Let's start by recalling the convolution-based expression for Harmonic Rényi Entropy, using complexity-normalization probabilities:

<math>\text{HE}_a(c) = H_a(J=j|C=c) = \frac{1}{1-a} \log \left( \frac{\left[S^a \ast K^a\right](-c)}{\left[S \ast K\right]^a(-c)} \right)</math>

To recall again, <math>S</math> is our spreading function, and <math>K</math> is our convolution kernel as defined above.

The definition for <math>K</math> is:

<math>\displaystyle K(c) = \sum_{j \in J} \dfrac{\delta_{-\cent(j)}}{\|j\|}</math>

where <math>\|j\|</math> represents the "complexity" of the JI basis ratio <math>j</math>. In the particular case of Tenney weighting, we get:

<math>\displaystyle K(c) = \sum_{j \in J} \dfrac{\delta_{-\cent(j)}}{\sqrt{j_n \cdot j_d}}</math>

where <math>j_n</math> and <math>j_d</math> are the numerator and denominator of <math>j</math>, respectively.

Although it may seem odd, we can take the Fourier transform of the above to obtain the following expression:

<math>\displaystyle \mathcal{F}\left\{K(c)\right\}(\omega) = \sum_{j \in J} \dfrac{e^{i \omega \cent(j)}}{\sqrt{j_n \cdot j_d}}</math>

Furthermore, for simplicity, we can change the units, so that rather than the argument being given in cents, it is given in "natural" units of "[https://en.wikipedia.org/wiki/Neper nepers]", a technique often used by Martin Gough in his work on [[Logarithmic_approximants|Logarithmic approximants]]. The representation of any interval in nepers is given by simply taking is natural logarithm. Doing so, by defining the change of variables <math>c = \frac{1200}{\log(2)}n</math>, we obtain

<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{j \in J} \dfrac{e^{i \omega \log (j_n/j_d)}}{\sqrt{j_n \cdot j_d}}</math>

We can treat the presence of the logarithm within the exponential function as changing the base of the exponential, so that we get

<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{j \in J} \dfrac{(j_n/j_d)^{i \omega}}{\sqrt{j_n \cdot j_d}}</math>

We can also factor each term in the summation to obtain

<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{j \in J} \left[ \dfrac{{j_n}^{i \omega}}{\sqrt{j_n}} \cdot \dfrac{{j_d}^{-i \omega}}{\sqrt{j_d}} \right]</math>

which we can rewrite as

<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{j \in J} \left[ \dfrac{1}{{j_n}^{0.5 -i \omega}} \cdot \dfrac{1}{{j_d}^{0.5 + i \omega}} \right]</math>

Now, we note our summation is currently written simply as <math>\sum_{j \in J}</math>. For a Tenney height complexity, we typically bound by <math>\sqrt{nd} < N</math> for some <math>N</math>. However, although it is unusual, for the sake of simplifying the derivation, we will bound by <math>\max(n,d) < N</math> instead, despite the use of Tenney height for complexity. This will not end up being much of a problem, as the two will converge on the same result anyway.

Bounding by <math>\max(n,d) < N</math> is the same as specifying that <math>j_n < N</math> and <math>j_d < N</math>. Doing so, we get

<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{1<j_n, j_d<N} \left[ \dfrac{1}{{j_n}^{0.5 -i \omega}} \cdot \dfrac{1}{{j_d}^{0.5 + i \omega}} \right]</math>

We can now factor the above product to obtain:

<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \left[ \sum_{j_n=1}^N \dfrac{1}{{j_n}^{0.5 -i \omega}} \right] \cdot \left[ \sum_{j_d=1}^N\dfrac{1}{{j_d}^{0.5 + i \omega}} \right]</math>

Now, we can see that as <math>N \to \infty</math> above, the summations do not converge. However, incredibly enough, each of the above expressions has a very well-known analytic continuation, which is the Riemann zeta function.

To perform the analytic continuation, we temporarily change the <math>0.5</math> in the denominator to some value <math>\sigma> 1</math>. This is equivalent to changing our original <math>\sqrt{nd}</math> weighting to some other exponent, such as <math>(nd)^2</math> or <math>(nd)^{1.5}</math>. Doing this causes both of the summations above to converge, so that we obtain

<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \left[ \sum_{j_n=1}^\infty \dfrac{1}{{j_n}^{\sigma -i \omega}} \right] \cdot \left[ \sum_{j_d=1}^\infty\dfrac{1}{{j_d}^{\sigma + i \omega}} \right]</math>

It has been well known for more than a century that both of these summations converge to the Riemann zeta function, so that we get

<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \zeta(\sigma-i\omega) \cdot \zeta(\sigma+i\omega)</math>

Rewriting as a function of a complex variable <math>s = \sigma + i\omega</math>, and noting that the zeta function obeys the property that <math>\zeta(\overline s)=\overline{\zeta(s)}</math>, where <math>\overline s</math> represents complex conjugation, we get

<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \overline{\zeta(s)} \cdot \zeta(s) = |\zeta(s)|^2</math>

And we have now obtained a very interesting result: if we had instead gone with something like <math>(nd)^2</math> complexity on rationals, rather than <math>\sqrt{nd}</math>, that our HE setup ''would'' have converged as <math>N \to \infty</math>, and our original HE convolution kernel would have been the Fourier transform of a particular vertical "slice" of the Riemann zeta function where <math>\Re(s) = 2</math>.

Furthermore, although the above series doesn't converge for <math>\sigma = 0.5</math>, we can simply use the analytic continuation of the Riemann zeta function to obtain a meaningful function at that point, so that our original convolution kernel can be written as

<math>K(n) = \mathcal{F}^{-1}\left\{| \zeta(0.5+\omega) |^2\right\}(n)</math>

which is the inverse Fourier transform of the squared absolute value of the Riemann zeta function, taken at the critical line.

Lastly, to do some cleanup, we previously went with <math>\max(n,d)<N</math> bounds, rather than <math>\sqrt{nd}<N</math> bounds, despite using <math>\sqrt{nd}</math> weighting on ratios. However, it is easy to show above that regardless of which bounds you use, both series converge to the same function when <math>\sigma > 1</math> in the limit as <math>N > \infty</math>. Since these series agree on this right half-plane of the Riemann zeta function, they share the same analytic continuation, so that we get the same result, despite using our technique to simplify the derivation.

It is likewise easy to show that the function <math>K^a(n)</math>, taken from the numerator of our original Harmonic Rényi Entropy convolution expression, can be expressed as

<math>K^a(n) = \mathcal{F}^{-1}\left\{|\zeta(0.5a+\omega) |^2\right\}(n)</math>

so that the choice of <math>a</math> simply changes our choice of vertical slice of the Riemann zeta function.

===Analytic Continuation of Harmonic Rényi Entropy===

We can put this back into the original equation for Harmonic Rényi Entropy. To do so, we will continue with our change of units from cents to nepers, corresponding to a change of our variable from <math>c</math> to <math>n</math>. Doing so, we obtain

<math>\displaystyle \text{HE}_a(n) = \frac{1}{1-a} \log \left( \frac{\left[S^a \ast \mathcal{F}^{-1}\left\{|\zeta(0.5a+\omega) |^2\right\}\right](-n)}{\left[S \ast \mathcal{F}^{-1}\left\{|\zeta(0.5+\omega) |^2\right\}\right]^a(-n)} \right)</math>

Note that above, we assume the spreading probability distribution <math>S</math> has likewise been scaled to reflect the new choice of units. If <math>S</math> is a Gaussian, this means we assume the standard deviation has been scaled accordingly.

We can simplify the expression of the above if we likewise take the Fourier transform of <math>S</math>. If we do, we obtain the [https://en.wikipedia.org/wiki/Characteristic_function_(probability_theory) characteristic function] of the distribution, which is typically denoted by <math>\phi(\omega)</math>. We will use the following definitions:

<math>\phi(\omega) = \mathcal{F}\left\{S(n)\right\}(\omega)</math>

<math>\phi_a(\omega) = \mathcal{F}\left\{S(n)^a\right\}(\omega)</math>

Doing so, and noting that convolution becomes multiplication in the Fourier domain, we get

<math>\displaystyle \text{HE}_a(n) = \frac{1}{1-a} \log \left( \frac{\mathcal{F}^{-1}\left\{\phi_a(\omega) \cdot |\zeta(0.5a+\omega) |^2\right\}(-n)}{\mathcal{F}^{-1}\left\{\phi(\omega) \cdot |\zeta(0.5+\omega) |^2\right\}^a(-n)} \right)</math>

We can write this more neatly by using the notation <math>\zeta_\sigma(\omega) = \zeta(\sigma+i\omega)</math>, and omitting the argument of <math>\omega</math> from the inside of the Fourier transform notation. Doing so, we get

<math>\displaystyle \text{HE}_a(n) = \frac{1}{1-a} \log \left( \frac{\mathcal{F}^{-1}\left\{\phi_a \cdot |\zeta_{0.5a} |^2\right\}(-n)}{\mathcal{F}^{-1}\left\{\phi \cdot |\zeta_{0.5} |^2\right\}^a(-n)} \right)</math>

And lastly, we note that for any function <math>f(x)</math>, we have <math>\mathcal{F}\left\{f(-x)\right\} = \mathcal{F}\left\{\overline f(x) \right\} = \mathcal{F}\left\{\overline f\right\}</math>. Putting that all together, we get

<math>\displaystyle \text{HE}_a(n) = \frac{1}{1-a} \log \left( \dfrac{\mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}}{\mathcal{F}^{-1}\left\{\overline {\phi} \cdot |\zeta_{0.5} |^2\right\}^a} \right)</math>

===Unnormalized Entropy===

Suppose that rather than looking at the Harmonic Rényi Entropy directly, we instead look at the exponential of <math>\frac{a}{1-a}</math> times the HRE. If we do so, since <math>a</math> is a constant, and the exponential function is strictly monotonic, we will obtain a new function that has the same exact minima, maxima, and relative ranking of intervals as the original HRE, and hence which is completely equivalent in characterising their relative discordance.

Doing so, we get

<math>\exp(\frac{a}{1-a} \text{HE}_a(n)) = \dfrac{\mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}^a}{\mathcal{F}^{-1}\left\{\overline {\phi} \cdot |\zeta_{0.5} |^2\right\}}</math>

so that we now have a much simpler expression which is simply the quotient of two convolutions, or equivalently the quotient of two products in the Fourier domain, followed by an inverse Fourier transform.

Now, if we use the usual definition of HRE, and naively increase the value of <math>N</math>, the curve gets higher and higher. Since the above function is just the exponential of HRE times a constant, the same will happen to it. The same phenomenon happens to both the numerator and denominator of the above expression, which simply increase without bound.

However, surprisingly, once we replace either of these expressions with their analytically continued counterpart, the curves suddenly "jump" so that they are roughly centered around the x-axis. This is a problem for the above expression, because the denominator -- which was supposed to be a "normalization" term -- now takes positive and negative values, and in particular has zeros. As a result, the zeros in the denominator will create poles in the resulting function. Hence, this no longer serves as a useful normalization term.

'''TO DO: add a picture of this'''

While a deeper exploration of this will be left to future research, for now we will simply drop the normalization term and look at the unnormalized entropy, or in this case, the exponentiated version from before. We will denote this by <math>\text{UHE}_a(n)</math>.

Doing so, we get

<math>\exp(\frac{a}{1-a} \text{UHE}_a(n)) = \mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}^a</math>

Taking both sides to the power of <math>1/a</math>, we obtain the following particularly simple expression:

<math>\exp((1-a) \text{UHE}_a(n)) = \mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}</math>

so that we can see our exponentiated UHE is simply the inverse Fourier transform of the complex conjugate of our spreading distribution's characteristic function, times the squared absolute value of the Riemann zeta function. (Note that if the spreading distribution is symmetric, as in the case of a Gaussian, the characteristic function is purely real, so that the complex conjugate is redundant.)

As a last touch, we note that our dropping of the normalization term has flipped the function upside down, so that higher values show concordance and lower values show discordance!

'''TO DO: show picture'''

We can restore the usual orientation by simply taking the negative of the above function, so that we get:

<math>-\exp((1-a) \text{UHE}_a(n)) = \mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}</math>

'''TO DO: show picture'''

and now we have a function which, despite our last-minute fudging of unnormalized probabilities above, would appear to resemble the usual Harmonic Entropy curve in every sense, with the same minima, maxima, general contour, and so on, just shifted so that it is roughly centered on the x-axis.

===Round-up===

As a result of this, we were able to show that Harmonic Entropy is closely related to the Riemann zeta function.

Furthermore, if unnormalized probabilities are used, the exponential of HE times a constant '''is''' the Fourier transform of the zeta function, times the characteristic function of the spreading distribution.

This is valid to the extent that the unnormalized entropy behaves similarly to the normalized entropy in the limit of large <math>N</math>, up to an additive shift and a multiplicative scaling (in this case, flipping the curve upside down). Empirically, this would certainly appear to be the case, as the resulting curve appears to resemble the usual HE curve in every possible way, having roughly the same minima, maxima, and general shape. However, it would be nice to show this formally, which we leave open for future work.

==Examples==

'''TO DO: add examples for different values of s'''

==Special Properties for Generalized Normal Distributions==

'''TO DO: this'''

==Fast Computation==

'''TO DO: this'''

=References=

Line 354:

Line 544:

[http://launch.groups.yahoo.com/group/harmonic_entropy/ Harmonic entropy group on Yahoo]

[http://www.mikebattagliamusic.com/HE-JS/HE.html Harmonic entropy graph calculator (JavaScript)]

[[Category:consonance]]

@@ Line 76: / Line 76: @@
 * The spreading function is typically a Gaussian distribution with a frequency deviation of 1% either way, or about s=~17 cents.
-Other spreading functions have also been explored, such as the use of the heavy-tailed [https://en.wikipedia.org/wiki/Laplace_distribution Laplace distribution], sometimes described as the "Vos function" in Paul's writings. We will assume the Gaussian distribution as the spreading function for the remainder of this article, so that the spreading function for an incoming dyad <math>c</math> can be written as follows:
+Other spreading functions have also been explored, such as the use of the heavy-tailed [https://en.wikipedia.org/wiki/Laplace_distribution Laplace distribution], sometimes described as the "Vos function" in Paul's writings. These two functions are part of the [https://en.wikipedia.org/wiki/Generalized_normal_distribution Generalized normal distribution] family, which has a parameter not only for the variance but for the kurtosis. However, for simplicity, we will assume the Gaussian distribution as the spreading function for the remainder of this article, so that the spreading function for an incoming dyad <math>c</math> can be written as follows:
 <math>S(x-c) = \frac{1}{s\sqrt{2\pi}} e^{-\frac{(x-c)^2}{2s^2}}</math>
@@ Line 217: / Line 217: @@
 ''Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=''∞''). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the complexity function <math>\sqrt{nd}</math>.''
-==Convolution-Based Expression For Quickly Computing Renyi Entropy==
+==Convolution-Based Expression For Quickly Computing Rényi Entropy==
-Below is given an derivation that expresses Harmonic Renyi Entropy in terms of two simpler functions, each of which is a convolution product and hence can be computed quickly using the Fast Fourier Transform.
+Below is given an derivation that expresses Harmonic Rényi Entropy in terms of two simpler functions, each of which is a convolution product and hence can be computed quickly using the Fast Fourier Transform.
 The below derivation depends on the use of complexity-normalization probabilities, although it may be possible to extend to domain-integral probabilities instead.
 ===Preliminaries===
-The Harmonic Renyi Entropy is defined as
+The Harmonic Rényi Entropy is defined as
 <math>\text{HE}_a(c) = H_a(J=j|C=c) = \frac{1}{1-a} \log \sum_{j \in J} P(J=j|C=c)^a</math>
@@ Line 254: / Line 254: @@
-Finally, we put this all together to obtain a simplified version of the Harmonic Renyi Entropy equation:
+Finally, we put this all together to obtain a simplified version of the Harmonic Rényi Entropy equation:
 <math>\text{HE}_a(c) = H_a(J=j|C=c) = \frac{1}{1-a} \log \left( \frac{\rho_a(c)}{\psi(c)^a} \right)</math>
@@ Line 289: / Line 289: @@
 We can clean up this notation by defining the auxiliary distribution K:
-<math>K(c) = \left(\sum_{j \in J} \frac{\delta_{-\cent(j)}}{\|j\|}\right)</math>
+<math>K(c) = \sum_{j \in J} \frac{\delta_{-\cent(j)}}{\|j\|}</math>
@@ Line 319: / Line 319: @@
 And again we clean up notation by defining the auxiliary distribution
-<math>K^a(c) = \left(\sum_{j \in J} \frac{\delta_{-\cent(j)}}{\|j\|^a}\right)</math>
+<math>K^a(c) = \sum_{j \in J} \frac{\delta_{-\cent(j)}}{\|j\|^a}</math>
 so that
@@ Line 331: / Line 331: @@
 ===Round-up===
-Taking all of this, we can rewrite the original expression for Harmonic Renyi Entropy as follows:
+Taking all of this, we can rewrite the original expression for Harmonic Rényi Entropy as follows:
 <math>\text{HE}_a(c) = H_a(J=j|C=c) = \frac{1}{1-a} \log \left( \frac{\left[S^a \ast K^a\right](-c)}{\left[S \ast K\right]^a(-c)} \right)</math>
@@ Line 343: / Line 343: @@
 <math>\left[S \ast K\right]^a(-c) = \left[S \ast K\right]^a(c)</math>
-We have succeeded in representing Harmonic Renyi Entropy in simple terms of two convolution products, each of which can be computed in <math>O(N log N)</math> time.
+We have succeeded in representing Harmonic Rényi Entropy in simple terms of two convolution products, each of which can be computed in <math>O(N log N)</math> time.
+=Extending HE to N=∞=
+All of the models described above involve a finite set of rational numbers, bounded by some complexity function, and where the complexity is less than some max value <math>N</math>.
+It so happens that we are able to analytically continue this definition to the situation where <math>N=\infty</math>, with one mathematical caveat. This enables us to speak cognizantly of the harmonic entropy of an interval as measured against ''all'' rational numbers.
+==Background: Unnormalized Entropy==
+The caveat is that our derivation only analytically continues the entropy function for the "unnormalized" set of probabilities, which we previously wrote as <math>\hat P(J=j|C=c)</math>. For this definition to be philosophically perfect, we would want to analytically continue the entropy function for the normalized sense of probabilities, previously written as <math>P(J=j|C=c)</math>.
+However, in practice, the "unnormalized entropy" appears to be an extremely good approximation to the normalized entropy for large values of <math>N</math>. The resulting curve has the same minima and maxima as HE, the same general shape, and for all intents and purposes looks exactly like HE, just shifted on the y-axis.
+It would be nice to show that the unnormalized entropy converges to the normalized entropy, up to a constant additive shift or multiplicative scaling, in the limit of large <math>N</math>. However, we will leave this for future research, as well as the question of how to do an exact derivation of the normalized entropy function.
+For now, we will start with a derivation of the unnormalized entropy for <math>N=\infty</math>, as an interesting function worthy of study in its own right - not only because it looks exactly like HE, but because it leads to an expression for unnormalized HE in terms of the [[The_Riemann_Zeta_Function_and_Tuning|Riemann Zeta function]].
+==Derivation==
+For now, our derivation is limited to the case of <math>\sqrt{nd}</math> Tenney-weighted rationals, although it may be possible to derive a similar result for <math>\max(n,d)</math> weighting as well.
+===Analytic Continuation of the Convolution Kernel===
+Let's start by recalling the convolution-based expression for Harmonic Rényi Entropy, using complexity-normalization probabilities:
+<math>\text{HE}_a(c) = H_a(J=j|C=c) = \frac{1}{1-a} \log \left( \frac{\left[S^a \ast K^a\right](-c)}{\left[S \ast K\right]^a(-c)} \right)</math>
+To recall again, <math>S</math> is our spreading function, and <math>K</math> is our convolution kernel as defined above.
+The definition for <math>K</math> is:
+<math>\displaystyle K(c) = \sum_{j \in J} \dfrac{\delta_{-\cent(j)}}{\|j\|}</math>
+where <math>\|j\|</math> represents the "complexity" of the JI basis ratio <math>j</math>. In the particular case of Tenney weighting, we get:
+<math>\displaystyle K(c) = \sum_{j \in J} \dfrac{\delta_{-\cent(j)}}{\sqrt{j_n \cdot j_d}}</math>
+where <math>j_n</math> and <math>j_d</math> are the numerator and denominator of <math>j</math>, respectively.
+Although it may seem odd, we can take the Fourier transform of the above to obtain the following expression:
+<math>\displaystyle \mathcal{F}\left\{K(c)\right\}(\omega) = \sum_{j \in J} \dfrac{e^{i \omega \cent(j)}}{\sqrt{j_n \cdot j_d}}</math>
+Furthermore, for simplicity, we can change the units, so that rather than the argument being given in cents, it is given in "natural" units of "[https://en.wikipedia.org/wiki/Neper nepers]", a technique often used by Martin Gough in his work on [[Logarithmic_approximants|Logarithmic approximants]]. The representation of any interval in nepers is given by simply taking is natural logarithm. Doing so, by defining the change of variables <math>c = \frac{1200}{\log(2)}n</math>, we obtain
+<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{j \in J} \dfrac{e^{i \omega \log (j_n/j_d)}}{\sqrt{j_n \cdot j_d}}</math>
+We can treat the presence of the logarithm within the exponential function as changing the base of the exponential, so that we get
+<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{j \in J} \dfrac{(j_n/j_d)^{i \omega}}{\sqrt{j_n \cdot j_d}}</math>
+We can also factor each term in the summation to obtain
+<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{j \in J} \left[ \dfrac{{j_n}^{i \omega}}{\sqrt{j_n}} \cdot \dfrac{{j_d}^{-i \omega}}{\sqrt{j_d}} \right]</math>
+which we can rewrite as
+<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{j \in J} \left[ \dfrac{1}{{j_n}^{0.5 -i \omega}} \cdot \dfrac{1}{{j_d}^{0.5 + i \omega}} \right]</math>
+Now, we note our summation is currently written simply as <math>\sum_{j \in J}</math>. For a Tenney height complexity, we typically bound by <math>\sqrt{nd} < N</math> for some <math>N</math>. However, although it is unusual, for the sake of simplifying the derivation, we will bound by <math>\max(n,d) < N</math> instead, despite the use of Tenney height for complexity. This will not end up being much of a problem, as the two will converge on the same result anyway.
+Bounding by <math>\max(n,d) < N</math> is the same as specifying that <math>j_n < N</math> and <math>j_d < N</math>. Doing so, we get
+<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \sum_{1<j_n, j_d<N} \left[ \dfrac{1}{{j_n}^{0.5 -i \omega}} \cdot \dfrac{1}{{j_d}^{0.5 + i \omega}} \right]</math>
+We can now factor the above product to obtain:
+<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \left[ \sum_{j_n=1}^N \dfrac{1}{{j_n}^{0.5 -i \omega}} \right] \cdot \left[ \sum_{j_d=1}^N\dfrac{1}{{j_d}^{0.5 + i \omega}} \right]</math>
+Now, we can see that as <math>N \to \infty</math> above, the summations do not converge. However, incredibly enough, each of the above expressions has a very well-known analytic continuation, which is the Riemann zeta function.
+To perform the analytic continuation, we temporarily change the <math>0.5</math> in the denominator to some value <math>\sigma> 1</math>. This is equivalent to changing our original <math>\sqrt{nd}</math> weighting to some other exponent, such as <math>(nd)^2</math> or <math>(nd)^{1.5}</math>. Doing this causes both of the summations above to converge, so that we obtain
+<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \left[ \sum_{j_n=1}^\infty \dfrac{1}{{j_n}^{\sigma -i \omega}} \right] \cdot \left[ \sum_{j_d=1}^\infty\dfrac{1}{{j_d}^{\sigma + i \omega}} \right]</math>
+It has been well known for more than a century that both of these summations converge to the Riemann zeta function, so that we get
+<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \zeta(\sigma-i\omega) \cdot \zeta(\sigma+i\omega)</math>
+Rewriting as a function of a complex variable <math>s = \sigma + i\omega</math>, and noting that the zeta function obeys the property that <math>\zeta(\overline s)=\overline{\zeta(s)}</math>, where <math>\overline s</math> represents complex conjugation, we get
+<math>\displaystyle \mathcal{F}\left\{K(n)\right\}(\omega) = \overline{\zeta(s)} \cdot \zeta(s) = |\zeta(s)|^2</math>
+And we have now obtained a very interesting result: if we had instead gone with something like <math>(nd)^2</math> complexity on rationals, rather than <math>\sqrt{nd}</math>, that our HE setup ''would'' have converged as <math>N \to \infty</math>, and our original HE convolution kernel would have been the Fourier transform of a particular vertical "slice" of the Riemann zeta function where <math>\Re(s) = 2</math>.
+Furthermore, although the above series doesn't converge for <math>\sigma = 0.5</math>, we can simply use the analytic continuation of the Riemann zeta function to obtain a meaningful function at that point, so that our original convolution kernel can be written as
+<math>K(n) = \mathcal{F}^{-1}\left\{| \zeta(0.5+\omega) |^2\right\}(n)</math>
+which is the inverse Fourier transform of the squared absolute value of the Riemann zeta function, taken at the critical line.
+Lastly, to do some cleanup, we previously went with <math>\max(n,d)<N</math> bounds, rather than <math>\sqrt{nd}<N</math> bounds, despite using <math>\sqrt{nd}</math> weighting on ratios. However, it is easy to show above that regardless of which bounds you use, both series converge to the same function when <math>\sigma > 1</math> in the limit as <math>N > \infty</math>. Since these series agree on this right half-plane of the Riemann zeta function, they share the same analytic continuation, so that we get the same result, despite using our technique to simplify the derivation.
+It is likewise easy to show that the function <math>K^a(n)</math>, taken from the numerator of our original Harmonic Rényi Entropy convolution expression, can be expressed as
+<math>K^a(n) = \mathcal{F}^{-1}\left\{|\zeta(0.5a+\omega) |^2\right\}(n)</math>
+so that the choice of <math>a</math> simply changes our choice of vertical slice of the Riemann zeta function.
+===Analytic Continuation of Harmonic Rényi Entropy===
+We can put this back into the original equation for Harmonic Rényi Entropy. To do so, we will continue with our change of units from cents to nepers, corresponding to a change of our variable from <math>c</math> to <math>n</math>. Doing so, we obtain
+<math>\displaystyle \text{HE}_a(n) = \frac{1}{1-a} \log \left( \frac{\left[S^a \ast \mathcal{F}^{-1}\left\{|\zeta(0.5a+\omega) |^2\right\}\right](-n)}{\left[S \ast \mathcal{F}^{-1}\left\{|\zeta(0.5+\omega) |^2\right\}\right]^a(-n)} \right)</math>
+Note that above, we assume the spreading probability distribution <math>S</math> has likewise been scaled to reflect the new choice of units. If <math>S</math> is a Gaussian, this means we assume the standard deviation has been scaled accordingly.
+We can simplify the expression of the above if we likewise take the Fourier transform of <math>S</math>. If we do, we obtain the [https://en.wikipedia.org/wiki/Characteristic_function_(probability_theory) characteristic function] of the distribution, which is typically denoted by <math>\phi(\omega)</math>. We will use the following definitions:
+<math>\phi(\omega) = \mathcal{F}\left\{S(n)\right\}(\omega)</math>
+<math>\phi_a(\omega) = \mathcal{F}\left\{S(n)^a\right\}(\omega)</math>
+Doing so, and noting that convolution becomes multiplication in the Fourier domain, we get
+<math>\displaystyle \text{HE}_a(n) = \frac{1}{1-a} \log \left( \frac{\mathcal{F}^{-1}\left\{\phi_a(\omega) \cdot |\zeta(0.5a+\omega) |^2\right\}(-n)}{\mathcal{F}^{-1}\left\{\phi(\omega) \cdot |\zeta(0.5+\omega) |^2\right\}^a(-n)} \right)</math>
+We can write this more neatly by using the notation <math>\zeta_\sigma(\omega) = \zeta(\sigma+i\omega)</math>, and omitting the argument of <math>\omega</math> from the inside of the Fourier transform notation. Doing so, we get
+<math>\displaystyle \text{HE}_a(n) = \frac{1}{1-a} \log \left( \frac{\mathcal{F}^{-1}\left\{\phi_a \cdot |\zeta_{0.5a} |^2\right\}(-n)}{\mathcal{F}^{-1}\left\{\phi \cdot |\zeta_{0.5} |^2\right\}^a(-n)} \right)</math>
+And lastly, we note that for any function <math>f(x)</math>, we have <math>\mathcal{F}\left\{f(-x)\right\} = \mathcal{F}\left\{\overline f(x) \right\} = \mathcal{F}\left\{\overline f\right\}</math>. Putting that all together, we get
+<math>\displaystyle \text{HE}_a(n) = \frac{1}{1-a} \log \left( \dfrac{\mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}}{\mathcal{F}^{-1}\left\{\overline {\phi} \cdot |\zeta_{0.5} |^2\right\}^a} \right)</math>
+===Unnormalized Entropy===
+Suppose that rather than looking at the Harmonic Rényi Entropy directly, we instead look at the exponential of <math>\frac{a}{1-a}</math> times the HRE. If we do so, since <math>a</math> is a constant, and the exponential function is strictly monotonic, we will obtain a new function that has the same exact minima, maxima, and relative ranking of intervals as the original HRE, and hence which is completely equivalent in characterising their relative discordance.
+Doing so, we get
+<math>\exp(\frac{a}{1-a} \text{HE}_a(n)) = \dfrac{\mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}^a}{\mathcal{F}^{-1}\left\{\overline {\phi} \cdot |\zeta_{0.5} |^2\right\}}</math>
+so that we now have a much simpler expression which is simply the quotient of two convolutions, or equivalently the quotient of two products in the Fourier domain, followed by an inverse Fourier transform.
+Now, if we use the usual definition of HRE, and naively increase the value of <math>N</math>, the curve gets higher and higher. Since the above function is just the exponential of HRE times a constant, the same will happen to it. The same phenomenon happens to both the numerator and denominator of the above expression, which simply increase without bound.
+However, surprisingly, once we replace either of these expressions with their analytically continued counterpart, the curves suddenly "jump" so that they are roughly centered around the x-axis. This is a problem for the above expression, because the denominator -- which was supposed to be a "normalization" term -- now takes positive and negative values, and in particular has zeros. As a result, the zeros in the denominator will create poles in the resulting function. Hence, this no longer serves as a useful normalization term.
+'''TO DO: add a picture of this'''
+While a deeper exploration of this will be left to future research, for now we will simply drop the normalization term and look at the unnormalized entropy, or in this case, the exponentiated version from before. We will denote this by <math>\text{UHE}_a(n)</math>.
+Doing so, we get
+<math>\exp(\frac{a}{1-a} \text{UHE}_a(n)) = \mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}^a</math>
+Taking both sides to the power of <math>1/a</math>, we obtain the following particularly simple expression:
+<math>\exp((1-a) \text{UHE}_a(n)) = \mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}</math>
+so that we can see our exponentiated UHE is simply the inverse Fourier transform of the complex conjugate of our spreading distribution's characteristic function, times the squared absolute value of the Riemann zeta function. (Note that if the spreading distribution is symmetric, as in the case of a Gaussian, the characteristic function is purely real, so that the complex conjugate is redundant.)
+As a last touch, we note that our dropping of the normalization term has flipped the function upside down, so that higher values show concordance and lower values show discordance!
+'''TO DO: show picture'''
+We can restore the usual orientation by simply taking the negative of the above function, so that we get:
+<math>-\exp((1-a) \text{UHE}_a(n)) = \mathcal{F}^{-1}\left\{\overline {\phi_a} \cdot |\zeta_{0.5a} |^2\right\}</math>
+'''TO DO: show picture'''
+and now we have a function which, despite our last-minute fudging of unnormalized probabilities above, would appear to resemble the usual Harmonic Entropy curve in every sense, with the same minima, maxima, general contour, and so on, just shifted so that it is roughly centered on the x-axis.
+===Round-up===
+As a result of this, we were able to show that Harmonic Entropy is closely related to the Riemann zeta function.
+Furthermore, if unnormalized probabilities are used, the exponential of HE times a constant '''is''' the Fourier transform of the zeta function, times the characteristic function of the spreading distribution.
+This is valid to the extent that the unnormalized entropy behaves similarly to the normalized entropy in the limit of large <math>N</math>, up to an additive shift and a multiplicative scaling (in this case, flipping the curve upside down). Empirically, this would certainly appear to be the case, as the resulting curve appears to resemble the usual HE curve in every possible way, having roughly the same minima, maxima, and general shape. However, it would be nice to show this formally, which we leave open for future work.
+==Examples==
+'''TO DO: add examples for different values of s'''
+==Special Properties for Generalized Normal Distributions==
+'''TO DO: this'''
+==Fast Computation==
+'''TO DO: this'''
 =References=
@@ Line 354: / Line 544: @@
 [http://launch.groups.yahoo.com/group/harmonic_entropy/ Harmonic entropy group on Yahoo]
 [http://www.mikebattagliamusic.com/HE-JS/HE.html Harmonic entropy graph calculator (JavaScript)]
 [[Category:consonance]]