Harmonic entropy: Difference between revisions

The general workings of the human auditory system lead to a plethora of well-documented and sonically interesting phenomena that can occur when a musical chord is played:

Concordance has often been confused with actual musical consonance, an unfortunate fact made more common by the psychoacoustics literature under the unfortunate name '''sensory consonance''', most often used to refer to phenomena related to roughness and beatlessness specifically. This is not to be confused with the more familiar construct of tonal stability, typically just called "consonance" in Western common practice music theory and sometimes clarified as "musical consonance" in the music cognition literature. To make matters worse, the literature has also at times referred to concordance -- and not tonal stability -- as '''tonal consonance''', often referring to phenomena related to virtual pitch integration, creating a complete terminological mess. As a result, the term "consonance" has been completely avoided in this article.

The original Harmonic Entropy model limited itself to working with dyads. More recently, work by Steve Martin and others has extended this basic idea to higher-cardinality chords. This article will concern itself with dyads, as the dyadic case is still the most well-developed, and many of the ideas extend naturally to larger chords without need for much exposition.

A clear mathematical way of quantifying this "dispersion" is via the [http://en.wikipedia.org/wiki/Entropy_(information_theory) Shannon entropy] of the probability distribution, which can be thought of as describing the "uncertainty" in the distribution. A distribution which has a very high probability of picking one outcome has low entropy and is not very uncertain, whereas a distribution which has the probability spread out on many outcomes is highly uncertain and has a high entropy.

To formalize our notion of Shannon entropy, we will first describe the random variable <math>J</math>, representing the set of JI "basis" intervals that our incoming interval is being "matched" to, and the parameter <math>C</math>, representing the "cents" of the incoming interval being played. For example, the interval <math>C</math> would take values such as "400 cents," and the interval <math>J</math> would take values in the set of basis ratios, such as "5/4" or "9/7."

which makes more explicit that <math>c</math> is the argument to the harmonic entropy function, which is equal to the entropy of <math>J</math>, conditioned on the incoming dyad of <math>c</math> cents.

In order to systematically assign a probability distribution to this dyad, we first start by defining a '''spreading function''', denoted by <math>S(x)</math>, that dictates how the dyad is "smeared" out in log-frequency space, representing how the auditory system allows for some tolerance for mistuning. The typical choice that we will assume here for a spreading function is a Gaussian distribution, with mean centered around the incoming dyad, and standard deviation typically taken as a free parameter in the system and denoted as <math>s</math>.

Given a spreading function and set of basis rationals, there are two different procedures commonly used to assign probabilities to each rational. The first, the '''domain-integral approach''', works for arbitrary nowhere dense sets of rationals without any further free parameters. The second, the '''simple weighted approach''', has nice mathematical properties which sometimes make it easier to compute and which may lead to generalizations to infinite sets of rationals which are sometimes dense in the reals. It is conjectured that there are certain important limiting situations where the two converge; both are described in detail below.

For discrete sets of JI basis ratios, the log-frequency spectrum can be divided up into '''domains''' assigned to each ratio. Each ratio is assigned a domain with lower bound equal to the mediant of itself and its nearest lower neighbor, and likewise with upper bound equal to the mediant of itself and its nearest upper neighbor. If no such neighbor exists, <math>\pm \infty</math> is used instead. Mathematically, this can be represented via the following expression:

In the case where the set of basis rationals consists of a finite set bounded by Tenney or Weil height, the resulting set of widths is conjectured to have interesting mathematical properties, leading to mathematically nice conceptual simplifications of the model. These simplifications are explained below.

It has been noted empirically by Paul Erlich that, given all those rationals with Tenney height under some cutoff <math>N</math> as a basis set, that the domain widths for rationals sufficiently far from the cutoff seem to be proportional to <math>\frac{1}{\sqrt{nd}}</math>.

This approach to assigning probabilities to basis rationals is useful because it hypothetically makes it possible to consider the HE of sets of rationals which are dense in the reals, or even the entire set of positive rationals, although the best way to do this is a subject of ongoing research.

In all of these examples, the x-axis represents the width in cents of the dyad, and the y-axis represents ''discordance'' rather than concordance, measured in nats of Shannon entropy.

This uses as a spreading function the Gaussian distribution with <math>s=~17\cent</math> (or a lin-frequency deviation of 1%). The basis set is all rationals of Tenney height less than 10,000. This uses the simple weighted approach, and the weighting function is <math>\sqrt{nd}</math>:

[[File:HE_Tenney_N_10000_s_17cents.png]]

This example uses the same spreading function and standard deviation, but this time the basis set is all rationals of Weil height less than 100. The weighting function here is <math>\max(n,d)</math>:

[[File:HE_Weil_N_100_s_17cents.png]]

The following image (from Paul Erlich) compares the domain-integral and simple weighted approaches by overlaying the two curves on top of each other. In both cases, the spreading function is again a Gaussian with s=~17 cents, and the basis set is all those rationals with Tenney height ≤ 10000. It can be seen that the curves are extremely similar, and that the locations of the minima and maxima are largely preserved:

[[File:HE_Tenney_mediant_vs_sqrt_nd_Paul.png|800px]]

An extension to the base Harmonic Entropy model, proposed by Mike Battaglia, is to generalize the use of [http://en.wikipedia.org/wiki/Entropy_(information_theory) Shannon entropy] by replacing it instead with [http://en.wikipedia.org/wiki/R%C3%A9nyi_entropy Rényi entropy], a [http://en.wikipedia.org/wiki/Q-analog q-analog] of Shannon's original entropy. This can be thought of as adding a second parameter, called <math>a</math>, to the model, reflecting how "intelligent" the brain's "decoding" process is when determining the most likely JI interpretation of an ambiguous interval.

The '''Harmonic Rényi Entropy of order a''' of an incoming dyad can be defined as follows:

Certain values of <math>a</math> reduce to simpler expressions and have special names, as given in the examples below.

<math>\displaystyle H_0(J|c) = \log |J|</math>

''Harmonic Hartley Entropy (a=0) with the basis set all rationals with Tenney height ≤ 10000. Note that the choice of spreading function makes no difference in the end result at all.''

<math>\displaystyle H_1(J|c) = -\sum_{j \in J} P(j|c) \log P(j|c)</math>

''Harmonic Shannon Entropy (a=1) with the basis set all rationals with Tenney height ≤ 10000, spreading function a Gaussian distribution with s=1% (~17 cents), and <math>\sqrt{nd}</math> weighting.''

<math>\displaystyle H_2(J|c) = -\log \sum_{j \in J} P(j|c)^2 = -\log (J_1 = J_2|c)</math>

''Harmonic Collision Entropy (a=2) with the basis set all rationals with Tenney height ≤ 10000, spreading function a Gaussian distribution with s=1% (~17 cents), and <math>\sqrt{nd}</math> weighting.''

<math>\displaystyle H_\infty(J|c) = -\log \max_{j \in J} P(j|c)</math>

''Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=''∞''). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the weighting function <math>\sqrt{nd}</math>.''

Below is given an derivation that expresses Harmonic Rényi Entropy in terms of two simpler functions, each of which is a convolution product and hence can be computed quickly using the Fast Fourier Transform.

The below derivation depends on the use of simple weighted probabilities, although it may be possible to extend to domain-integral probabilities instead.

The Harmonic Rényi Entropy is defined as

We thus reduce the term inside the logarithm to the quotient of the functions <math>\rho_a(c)</math> and <math>\psi(c)</math>. Our aim is now to express each of these two functions in terms of a convolution product.

<math>\displaystyle \psi(c)</math>, the normalization function, is written as follows:

<math>\displaystyle \psi(c) = \left[S \ast K\right](-c)</math>

The derivation for <math>\rho_a(c)</math> proceeds similarly. Recall the function is written as follows:

Note that the function <math>K^a(c)</math> involves a slight abuse of notation, as it is not literally <math>K(c)</math> taken to the <math>a</math>'th power (as the square of the delta distribution is undefined). Rather, we are simply taking the weights of each delta distribution in the summation to the <math>a</math>'th power.

Taking all of this, we can rewrite the original expression for Harmonic Rényi Entropy as follows:

We have succeeded in representing Harmonic Rényi Entropy in simple terms of two convolution products, each of which can be computed in <math>O(N log N)</math> time.

All of the models described above involve a finite set of rational numbers, bounded by some weighting function, and where the weighting is less than some max value <math>N</math>.

This enables us to speak cognizantly of the harmonic entropy of an interval as measured against ''all'' rational numbers.

Our derivation only analytically continues the entropy function for the "unnormalized" set of probabilities, which we previously wrote as <math>Q(j|c)</math>. For this definition to be philosophically perfect, we would want to analytically continue the entropy function for the normalized sense of probabilities, previously written as <math>P(j|c)</math>.

For now, we will start with a derivation of the unnormalized entropy for <math>N=\infty</math>, as an interesting function worthy of study in its own right - not only because it looks exactly like HE, but because it leads to an expression for unnormalized HE in terms of the [[The_Riemann_Zeta_Function_and_Tuning|Riemann Zeta function]].

For now, our derivation is limited to the case of <math>\sqrt{nd}</math> Tenney-weighted rationals, although it may be possible to derive a similar result for <math>\max(n,d)</math> weighting as well.

Additionally, because it simplifies the derivation, we will use ''unreduced rationals'' in our basis set, meaning that we will even allow unreduced fractions such as <math>4/2</math> so long as <math>nd<N</math> for our bound N. The use of HE with unreduced rationals has previously been studied by Paul and shown to be not that much different than HE with reduced ones. However, we will easily show later unreduced and reduced rationals converge to the same thing in the limit as <math>N \to \infty</math>, up to a constant multiplicative scaling.

Let's start by recalling the original definition for Harmonic Rényi Entropy, using simple weighted probabilities:

The definition for <math>K</math> is:

so that the choice of <math>a</math> simply changes our choice of vertical slice of the Riemann zeta function, as well as the shape of our spreading function (because it is also being raised to a power). If our spreading function is a Gaussian, then we simply get another Gaussian with a different standard deviation.

We can put this back into our equation for the Unnormalized Harmonic Rényi Entropy. To do so, we will continue with our change of units from cents to nepers, corresponding to a change of our variable from <math>c</math> to <math>n</math>. We will likewise assume the spreading probability distribution <math>S</math> has been scaled to reflect the new choice of units.

where we can drop the overline on <math>|\zeta_{0.5a}|^2</math> because it is purely real, and its complex conjugate is itself.

It is very easy to see empirically that our expression does seem to converge be the thing that UHE converges on in the limit of large N. Here are some examples for different values of s and a, showing that as N increases it converges on the our analytically continued "zeta HE."

Lastly, note that, due to our multiplication of UHE by <math>(1-a)</math> above, the UHE would be flipped upside down relative to what we're used to, with higher values corresponding to more concordant intervals. In the pictures below we have flipped it back upside down, for consistency with the earlier pictures.

[[File:ExpUHE vs zeta s=0.5%.png|800px]]

[[File:ExpUHE vs zeta s=1%.png|800px]]

[[File:ExpUHE vs zeta s=1.5%.png|800px]]

Note that in all these plots, the value of <math>a</math> is chosen to be <math>1.00001</math> rather than exactly <math>1</math>, so as to avoid that <math>(1-a)</math> term becoming 0. Similar results are seen for other choices of <math>a</math>:

[[File:ExpUHE vs zeta s=1% a=2.2.png|800px]]

Note that you have to be careful if you choose to compute the analytically continued <math>a=2</math> numerically: this corresponds to the vertical slice of zeta function at <math>\Re(z) = 1</math>, where there is a pole.

Let's go back to our original convolution expression for finite-<math>N</math> UHE:

Note again that this does not hold for <math>a \gt 2</math>, where the graph does display a very large difference between UHE and exp-UHE.

On the surface, everything we did with the convolution theorem, and subsequent analytic continuation, should appear to work for normalized HE as well. For example, let's review our result for the exp of unnormalized HE:

Finally, it is noteworthy that for <math>a>2</math>, we end up looking at slices of the zeta function for which <math>\Re(z)>1</math>. This is where our original unnormalized HE function should converge as <math>N \to \infty</math>, corresponding to the region where the Riemann zeta function Dirichlet series converges. For these values of <math>a</math>, the exp-UHE ''is'' positive. So, we can take the log again and look at the usual UHE. This can be useful for plotting, since exp-UHE tends to "flatten" out the curve for high values of <math>a</math>, whereas taking the log accentuates the minima and maxima (and more closely resembles the usual HRE).

In our original derivation of the analytic continuation, we temporarily changed the weighting for rationals from <math>(nd)^{0.5}</math> to some other <math>(nd)^w</math>, with <math>w > 1</math>, for the sake of obtaining a series that converges. We then changed the exponent back to <math>0.5</math>.

So that our vertical slice of the zeta function is given by $\Re(z) = w\cdot \a$.

We get a very interesting result if our spreading distribution is a [https://en.wikipedia.org/wiki/Generalized_normal_distribution generalized normal distribution], which a family that encompasses both the Gaussian and the Laplace distributions (sometimes referred to as the "Vos curve" in Paul's work).

This gives us a very nice interpretation of our <math>a</math> coefficient from HRE: it basically represents the weighting exponent on the rationals, with a corresponding adjustment to the standard deviation. The collision entropy <math>a=2</math> with the standard weighting <math>\sqrt{nd}</math> is totally equivalent to the Shannon entropy <math>a=1</math> with the weighting <math>nd</math> on the rationals, so long as the value of <math>s</math> is adjusted according to the equation above. However, it should be noted that this definition only holds for the "unnormalized HRE" given above.

In our derivation, we assumed the use of unreduced rationals. It turns out that with a minor adjustment, the same model gives us reduced rationals, up to a constant multiplicative scaling. Let's go back to our analytic continuation of the convolution kernel, for some arbitrary weighting:

Lastly, you will note that for the special value <math>w=0.5</math>, corresponding to the usual <math>\sqrt{nd}</math> weighting, we end up dividing by the term <math>\zeta(1)</math>. This is the only pole in the zeta function, so we wind up dividing by infinity, making the entire function zero, as pointed out by Martin Gough. However, as we can get arbitrarily close to <math>w=0.5</math> and still exhibit the behavior that the unreduced and reduced functions are scaled versions of one another, we can simply use the unreduced version of exp-UHE for <math>w=0.5</math> and consider it equivalent to reduced exp-UHE in the limit.

There are a number of things that need to be added to this article. Below are listed some for reference:

* addition of many more pictures

* [http://www.webcitation.org/60qOlJVFS Paul Erlich article]

* [http://sethares.engr.wisc.edu/paperspdf/HarmonicEntropy.pdf William Sethares article]