Harmonic entropy: Difference between revisions

So for example, if we want to express the probability that the incoming dyad "400 cents" is perceived as the JI basis interval "5/4," we would write that as the conditional probability

Or, in general, if we want to write the conditional probability that some incoming dyad of <math>c</math> cents is perceived as the JI basis interval <math>j</math>, we would write that as

Note that at this point, we haven't yet specified what the particular probability distribution is. There are different ways to do this, which are described in more detail below. Generally, most approaches involve each JI interval's probability being assigned based on how close it is to <math>c</math> (closer dyads are given a larger probability), and how simple it is (simple dyads are given a higher probability, if distance is the same).

Once we have decided on a probability distribution, we can finally evaluate the Shannon entropy. For a random variable <math>X</math>, the Shannon entropy is defined as:

where the different <math>x</math> are taken from the sample space of <math>X</math>, and <math>b</math> is the base of the log. Different choices of <math>b</math> simply change the units in which entropy is given, the most common values being 2 and e, denoting "bits" and "nats". We will omit the base going forward, for simplicity.

In our case, we want to find the entropy of the random variable <math>J</math> of JI intervals, given a particular choice of incoming dyad in cents. The corresponding quantity that we want is:

Since the parameter <math>c</math> is the free parameter, sometimes the above is notated as

==Probability Distributions==

Other spreading functions have also been explored, such as the use of the heavy-tailed [https://en.wikipedia.org/wiki/Laplace_distribution Laplace distribution], sometimes described as the "Vos function" in Paul's writings. These two functions are part of the [https://en.wikipedia.org/wiki/Generalized_normal_distribution Generalized normal distribution] family, which has a parameter not only for the variance but for the kurtosis. However, for simplicity, we will assume the Gaussian distribution as the spreading function for the remainder of this article, so that the spreading function for an incoming dyad <math>c</math> can be written as follows:

where the notation <math>S(x-c)</math> is chosen to make clear that we are translating <math>S(x)</math> to be centered around the incoming dyad <math>c</math>, which is now the mean of the Gaussian.

For discrete sets of JI basis ratios, the log-frequency spectrum can be divided up into '''domains''' assigned to each ratio. Each ratio is assigned a domain with lower bound equal to the mediant of itself and its nearest lower neighbor, and likewise with upper bound equal to the mediant of itself and its nearest upper neighbor. If no such neighbor exists, <math>\pm \infty</math> is used instead. Mathematically, this can be represented via the following expression:

where <math>S(x-c)</math> is the spreading function associated with c, <math>j_l</math> and <math>j_u</math> are the domain lower and upper bounds associated with JI basis ratio <math>j</math>, and <math>\cent(f) = 1200\log_2(f)</math>, or the "cents" function converting frequency ratios to cents. Typically, <math>j_l</math> is set equal to the mediant of <math>j</math> and its nearest lower neighbor (if it exists), or <math>-\infty</math> if not; likewise with <math>j_u</math> and its nearest upper neighbor.

While it's still an open conjecture that this pattern holds for arbitrarily large <math>N</math>, the assumption is sometimes made that this is the case, and hence that for these basis ratio sets, <math>\frac{1}{\sqrt{nd}}</math> "approximations" to the width are sufficient to estimate domain-integral Harmonic Entropy.

A similar observation for the use of Weil-bounded subsets of the rationals suggests domain widths of <math>\frac{1}{\max(n,d)}</math>, yielding instead the following formula:

where this time the set of basis rationals is assumed to be all of those of Weil Height ≤ <math>N</math> for some <math>N</math>.

In both cases, the general approach is the same: the value of the spreading function, taken at the value of <math>\cent(j)</math>, is divided by some sort of "complexity" function representing how much weight is given to that rational number. While the two complexity functions considered thus far were derived empirically by observing the asymptotic behavior of various height-bounded subsets of the rationals, we can generalize this for arbitrary basis sets of rationals and arbitrary complexities as follows:

where <math>\|j\|</math> denotes a complexity function mapping from rational numbers to non-negative reals.

As these "probabilities" don't sum to 1, the result is not a probability distribution at all, invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:

This approach to assigning probabilities to basis rationals is useful because it hypothetically makes it possible to consider the HE of sets of rationals which are dense in the reals, or even the entire set of positive rationals, although the best way to do this is a subject of ongoing research.

The '''Harmonic Rényi Entropy of order a''' of an incoming dyad can be defined as follows:

Being a q-analog, it is noteworthy that Rényi entropy converges to Shannon entropy in the limit as <math>a \to 1</math>, a fact which can be verified using L'Hôpital's rule as found [http://www.sonycsl.co.jp/person/nielsen/Note-HopitalRuleShannonRenyiTsallis.pdf here].

In a musical context, by considering the incoming dyad as analogous to a cryptographic code which is attempting to be "cracked" by an intelligent auditory system, we can consider that the analogous "worst-case attacker" would be a "best-case auditory system" which has complete awareness of the probability distribution for any incoming dyad. This analogy would view such an auditory system as actively attempting to choose the most probable rational, rather than drawing a rational at random weighted by the distribution.

Some psychoacoustic effects naturally fit into this paradigm, such as the virtual pitch integration process, which actually does attempt to find a single victor when matching incoming chords with chunks of the harmonic series. Other psychoacoustic effects, such as that of beatlessness, may instead be better viewed as "dumb" processes whereby nothing in particular is being "chosen," but where a more uniform distribution of matching rational numbers for a dyad simply generates a more discordant sonic effect. Different values of a can differentiate between the predominance given to these two types of effect in the overall construct of psychoacoustic concordance.

==Examples==

===a=0: Harmonic Hartley Entropy===

where <math>|J|</math> is the cardinality of the set of basis rationals. This assumes, in essence, an "infinitely dumb" auditory system which can do no better than picking a rational number from a uniform distribution completely at random. All dyads have the same Harmonic Hartley Entropy. The Hartley Entropy is sometimes called the "max-entropy," and is useful mainly as an upper bound on the other forms of entropy: all Rényi Entropies are always guaranteed to be less than the Hartley Entropy.

===a=1: Harmonic Shannon Entropy (Harmonic Entropy)===

This is Paul's original Harmonic Entropy. Within the cryptographic analogy, this can be thought of as an auditory system which simply selects a rational at random from the incoming distribution, weighted via the distribution itself.

===a=2: Harmonic Collision Entropy===

[[File:HE_Tenney_N_10000_s_17cents_a=2.png]]

===a=∞: Harmonic Min-Entropy===

This is the min-entropy, which simply takes the negative log of the largest probability in the distribution. This can be thought of as representing the "strength" of the incoming dyad from being "deciphered" by a "best-case" auditory system. The name "min-entropy" reflects that the <math>a=\infty</math> case is guaranteed to be a lower bound among all Rényi entropies.

The Harmonic Rényi Entropy is defined as

To simplify notation, we first rewrite the denominator as a "normalization" function:

and putting back into the original equation, we get

Since <math>\psi(c)</math> is the same for each basis ratio, we can pull it out of the summation to obtain:

To simplify notation further, we can also rewrite the numerator, the sum of "raw" (unnormalized) pseudo-probabilities, as a function:

Finally, we put this all together to obtain a simplified version of the Harmonic Rényi Entropy equation:

===Convolution product for <math>\psi(c)</math>===

We can rewrite the above equation as a convolution with a delta distribution:

Putting this back into the original summation, we obtain

We note that the left factor in the convolution product is always the same <math>S(-c)</math>, which is not dependent on <math>j</math> in any way. Since convolution distributes over addition, we can factor the <math>S</math> out of the summation to obtain

We can clean up this notation by defining the auxiliary distribution K:

Which leaves us with the final expression:

===Convolution product for <math>\rho_a(c)</math>===

The derivation for <math>\rho_a(c)</math> proceeds similarly. Recall the function is written as follows:

We can again express this as a convolution, this time of the function <math>S^a(-c)</math>, meaning the spreading function S taken to the a'th power, and a delta distribution:

Putting this back into the original summation and factoring as before, we obtain

And again we clean up notation by defining the auxiliary distribution

so that

We have now succeeded in representing <math>\rho_a(c)</math> as a convolution.

Taking all of this, we can rewrite the original expression for Harmonic Rényi Entropy as follows:

where the expression

represents the convolution of <math>S</math> and </math>K, taken to the <math>a</math>'th power, and flipped backwards. Note that if <math>S(x)</math> is a symmetrical (even) spreading function, and if for each ratio <math>n/d</math> in <math>J</math>, if the inverse <math>d/n</math> is also in <math>J</math>, then the above convolution will also be symmetrical, and we also have

We have succeeded in representing Harmonic Rényi Entropy in simple terms of two convolution products, each of which can be computed in <math>O(N log N)</math> time.

All of the models described above involve a finite set of rational numbers, bounded by some complexity function, and where the complexity is less than some max value <math>N</math>.

This enables us to speak cognizantly of the harmonic entropy of an interval as measured against ''all'' rational numbers.

==Background: Unnormalized Entropy==

However, in practice, the "unnormalized entropy" appears to be an extremely good approximation to the normalized entropy for large values of <math>N</math>. The resulting curve has the same minima and maxima as HE, the same general shape, and for all intents and purposes looks exactly like HE, just shifted on the y-axis.

For now, our derivation is limited to the case of <math>\sqrt{nd}</math> Tenney-weighted rationals, although it may be possible to derive a similar result for <math>\max(n,d)</math> weighting as well.

The definition for <math>K</math> is:

where <math>\|j\|</math> represents the "complexity" of the JI basis ratio <math>j</math>. In the particular case of Tenney weighting, we get:

where <math>j_n</math> and <math>j_d</math> are the numerator and denominator of <math>j</math>, respectively.

Although it may seem odd, we can take the Fourier transform of the above to obtain the following expression:

Furthermore, for simplicity, we can change the units, so that rather than the argument being given in cents, it is given in "natural" units of "[https://en.wikipedia.org/wiki/Neper nepers]", a technique often used by Martin Gough in his work on [[Logarithmic_approximants|Logarithmic approximants]]. The representation of any interval in nepers is given by simply taking is natural logarithm. Doing so, by defining the change of variables <math>c = \frac{1200}{\log(2)}n</math>, we obtain

We can treat the presence of the logarithm within the exponential function as changing the base of the exponential, so that we get

We can also factor each term in the summation to obtain

which we can rewrite as

Bounding by <math>\max(n,d) < N</math> is the same as specifying that <math>j_n < N</math> and <math>j_d < N</math>. Doing so, we get

We can now factor the above product to obtain:

Now, we can see that as <math>N \to \infty</math> above, the summations do not converge. However, incredibly enough, each of the above expressions has a very well-known analytic continuation, which is the Riemann zeta function.

To perform the analytic continuation, we temporarily change the <math>0.5</math> in the denominator to some value <math>\sigma> 1</math>. This is equivalent to changing our original <math>\sqrt{nd}</math> weighting to some other exponent, such as <math>(nd)^2</math> or <math>(nd)^{1.5}</math>. Doing this causes both of the summations above to converge, so that we obtain

It has been well known for more than a century that both of these summations converge to the Riemann zeta function, so that we get

Furthermore, although the above series doesn't converge for <math>\sigma = 0.5</math>, we can simply use the analytic continuation of the Riemann zeta function to obtain a meaningful function at that point, so that our original convolution kernel can be written as

which is the inverse Fourier transform of the squared absolute value of the Riemann zeta function, taken at the critical line.

It is likewise easy to show that the function <math>K^a(n)</math>, taken from the numerator of our original Harmonic Rényi Entropy convolution expression, can be expressed as

so that the choice of <math>a</math> simply changes our choice of vertical slice of the Riemann zeta function.

We can simplify the expression of the above if we likewise take the Fourier transform of <math>S</math>. If we do, we obtain the [https://en.wikipedia.org/wiki/Characteristic_function_(probability_theory) characteristic function] of the distribution, which is typically denoted by <math>\phi(\omega)</math>. We will use the following definitions:

Doing so, and noting that convolution becomes multiplication in the Fourier domain, we get

=References=