Harmonic entropy
IMPORTED REVISION FROM WIKISPACES
This is an imported revision from Wikispaces. The revision metadata is included below for reference:
- This revision was by author mbattaglia1 and made on 2014-07-05 03:27:02 UTC.
- The original revision id was 515617078.
- The revision comment was:
The revision contents are below, presented both in the original Wikispaces Wikitext format, and in HTML exactly as Wikispaces rendered it.
Original Wikitext content:
=Introduction=
**Harmonic Entropy**, sometimes abbreviated as "HE", is a simple model to quantify the extent to which musical chords exhibit various psychoacoustic effects, lumped together in a single construct called psychoacoustic **concordance**. It was invented by Paul Erlich and developed extensively on the Yahoo! tuning and harmonic_entropy lists. Various later contributions to the model have been made by Steve Martin, Mike Battaglia, Keenan Pepper, and others.
=Background=
The general workings of the human auditory system lead to a plethora of well-documented and sonically interesting phenomena that can occur when a musical chord is played:
* The perception of partial **timbral fusion** of the chord into one complex sound
* The appearance of a **virtual fundamental** pitch in the bass
* Timbral **beatlessness**, compared to mistunings of the chord in the surrounding area
* The appearance of a quick fluttering effect sometimes known as **periodicity buzz**
All of these effects share two common characteristics for chords played with harmonic timbres:
* <span style="line-height: 1.5;">they tend to appear most strongly for those chords with large subsets that correspond to simple chunks of the harmonic series</span>
* <span style="line-height: 1.5;">the effects produced exhibit some degree of tolerance for mistuning</span>
As the presence or absence of these effects tend to appear in tandem and are highly correlated with one another, we can speak of a general notion of the psychoacoustic **concordance** of an interval - the degree to which effects such as the above will appear when an arbitrary musical chord is played. Additionally, chords which are very inharmonic often exhibit a quality known as psychoacoustic **discordance**.
While psychoacoustic concordance is not a feature universal to all styles of music, it has been utilized significantly in Western music in the study of intonation. For instance, flexible-pitch ensembles operating within 12-EDO, such as barbershop quartets and string ensembles, will often adjust intonationally from the underlying 12-EDO reference to maximize the concordance of individual chords. Indeed, the entire history of Western tuning theory -- from meantone temperament, to the various Baroque well-temperaments, to 12-EDO itself, to the modern [[@xenharmonic/Regular Temperaments|theory of regular temperament]] -- can be seen as an attempt to reason mathematically about how to generate manageable tuning systems that will maximize concordance and minimize discordance.
The Harmonic Entropy model is a simple way of quantifying how much an arbitrary chord will exhibit psychoacoustic concordance.
Concordance has often been confused with actual musical consonance, an unfortunate fact made more common by the<span style="line-height: 1.5;"> psychoacoustics literature under the unfortunate name </span>**<span style="line-height: 1.5;">sensory consonance</span>**<span style="line-height: 1.5;">, most often used to refer to phenomena related to roughness and beatlessness specifically. This is not to be confused with the more familiar construct of tonal stability, typically just called "consonance" in Western common practice music theory and sometimes clarified as "musical consonance" in the music cognition literature. To make matters worse, the literature has also at times referred to concordance -- and not tonal stability -- as </span>**<span style="line-height: 1.5;">tonal consonance</span>**<span style="line-height: 1.5;">, often referring to phenomena related to virtual pitch integration, creating a complete terminological mess. As a result, the term "consonance" has been completely avoided in this article.</span>
=Model=
The original Harmonic Entropy model limited itself to working with dyads. More recently, work by Steve Martin and others has extended this basic idea to higher-cardinality chords. This article will concern itself with dyads, as the dyadic case is still the most well-developed, and many of the ideas extend naturally to larger chords without need for much exposition.
The general idea of Harmonic Entropy is to first develop a discrete probability distribution quantifying how strongly an arbitrary incoming dyad "matches" every element in a set of basis rational intervals, and then seeing how evenly distributed the resulting probabilities are. If the distribution for some dyad is spread out very evenly, such that there is no clear "victor" basis interval that dominates the distribution, the dyad is considered to be more discordant; on the other extreme, if the distribution tends to concentrate on one or a small set of dyads, the dyad is considered to be more concordant. A clear mathematical way of quantifying this is via <span style="line-height: 1.5;">the [[@http://en.wikipedia.org/wiki/Entropy_(information_theory)|Shannon entropy]] of the probability distribution:</span>
[[math]]
H(d) = -\sum_{b} p_d(b) \log p_d(b)
[[math]]
where H(d) is the Shannon entropy of the dyad d, the b are all of the basis rationals in the set, and the p<span style="font-size: 90%; vertical-align: sub;">d</span>(b) is the probability assigned to basis rational b given an input dyad of d. This is the Harmonic Entropy of the dyad d.
In order to systematically assign a probability distribution to this dyad, we first start by defining a **spreading function** that dictates how the dyad is "smeared" out in log-frequency space, representing how the auditory system allows for some tolerance for mistuning. The typical choice that we will assume here for a spreading function is a Gaussian distribution, with mean set to be centered around the incoming dyad, and standard deviation **s** typically taken as a free parameter in the system. We then divide the log-frequency spectrum up into **domains** assigned to the basis rationals; each rational is assigned a domain with lower bound equal to the mediant of itself and its nearest lower neighbor, and likewise with upper bound equal to the mediant of itself and its nearest upper neighbor. This process can be summarized by the following picture, taken from [[@http://sethares.engr.wisc.edu/paperspdf/HarmonicEntropy.pdf|William Sethares' paper on Harmonic Entropy]]:
[[image:http://i.imgur.com/aQlqRXz.png]]
In this example, the f<span style="font-size: 90%; vertical-align: sub;">j+n</span> are the basis rationals, the r<span style="font-size: 12px; vertical-align: sub;">j+n</span> are the domains for each basis rational, and the bounds for each domain are the mediants between each f<span style="font-size: 12px; vertical-align: sub;">j+n </span>and its nearest neighbor. The probability assigned to each basis rational is then the area under the spreading function curve for each rational's domain. The entropy of this probability distribution is then the Harmonic Entropy for that dyad.
A f<span style="line-height: 1.5;">airly typical choice of settings for a basic dyadic HE model would be:</span>
* <span style="line-height: 1.5;">The set of basis rationals is the set of all intervals bounded by some maximum Tenney height, with the bound typically notated as **N** and typically set to at least 10000</span>
* <span style="line-height: 1.5;">The spreading function is typically a Gaussian distribution with s = 1%, or about 17 cents</span>
Setting N=10000 and s=1% yields the following curve, where the x-axis represents the width in cents of the dyad, and the y-axis represents //discordance//, measured in bits of Shannon entropy:
[[image:http://i.imgur.com/tNg7z1P.png width="800" height="404"]]
Other ways of generating the set of basis rationals have also been explored, most notably by bounding by [[@height|Weil height]] rather than Tenney height. The following is the curve that results for a Weil height bound 100 and s=1%, which has the primary effect of altering the slope of the curve:
[[image:http://i.imgur.com/TZdU6eD.png width="800" height="410"]]
Other spreading functions have also been explored, such as the use of the "Vos function" of a·exp(b|x|) rather than the Gaussian distribution.
Finally, it should be noted that the use of Tenney height in bounding the rationals seems to lead to domain widths that are proportional to 1/sqrt(nd) for each rational, and that the use of Weil-bounded subsets of the rationals leads to domain widths that are proportional to 1/max(n,d) for each rational. It is a conjecture of Erlich's that this is true in the limit as N→∞, assuming that it's possible to suitably modify the HE equation so that the curve converges in the limit.
These approximations have sometimes been used as quick computational substitutes for actually integrating the spreading function over each domain, so long as the resulting probabilities are normalized so that the sum is 1. The pictures above use these approximations.
=Harmonic Rényi Entropy=
An extension to the base Harmonic Entropy model, proposed by Mike Battaglia, is to generalize the use of [[@http://en.wikipedia.org/wiki/Entropy_(information_theory)|Shannon entropy]] by replacing it instead with [[@http://en.wikipedia.org/wiki/R%C3%A9nyi_entropy|Rényi entropy]], a q-analog of Shannon's original entropy. The **Harmonic Rényi Entropy of order a** of an incoming dyad can be defined as follows:
[[math]]
H(d) = \frac{1}{1-a} \log \sum_b p_d(b)^a
[[math]]
where the p_d(b) are the probabilities assigned by dyad d to each basis rational b. It is noteworthy that Rényi entropy converges to Shannon entropy in the limit as a→1, a fact which can be verified using L'Hôpital's rule as found [[@http://www.sonycsl.co.jp/person/nielsen/Note-HopitalRuleShannonRenyiTsallis.pdf|here]].
Musically speaking, the parameter //**a**// can be interpreted as the extent to which the rational-matching process for incoming dyads is considered to be an intelligent, active process by which the auditory system actively analyzes the probability distribution and seeks out the "victor" rational with the greatest probability. Some psychoacoustic effects naturally fit into this paradigm, such as the virtual pitch integration process, which actually does attempt to find a single victor when matching incoming chords with chunks of the harmonic series. Other psychoacoustic effects, such as that of beatlessness, may instead be better viewed as "dumb" processes whereby nothing in particular is being "chosen," but where a more uniform distribution of matching rational numbers for a dyad simply generates a more discordant sonic effect. Increasing values of //a// correspond to a model that is more "active" in this way.
This interpretation comes from the use of the Rényi entropy in cryptography as a measure of the strength of a cryptographic code in the face of an intelligent attacker, an application for which Shannon entropy has long been known to be insufficient as described in [[@http://users.cis.fiu.edu/~smithg/papers/qest11.pdf|this paper]] and [[@http://www.ietf.org/rfc/rfc4086.txt|this RFC]]. More precisely, the Rényi entropy of order ∞, also called the **min-entropy**, is used to measure the strength of the randomness used to define a cryptographic secret against a "worst-case" attacker who has complete knowledge of the probability distribution from which cryptographic secrets are drawn. In a musical context, by considering the incoming dyad as analogous to a cryptographic code which is attempting to be "cracked" by an intelligent auditory system, we can consider that the analogous "worst-case attacker" would be a "best-case auditory system" which has complete awareness of the probability distribution for any incoming dyad and actively chooses the strongest rational.
The use of a=∞ min-entropy would reflect this view of an "intelligent" auditory system that can analyze the incoming probability distribution and always pick one of the matching rationals that holds a plurality. In contrast, the use of a=1 Shannon entropy reflects a much "dumber" process which performs no such analysis; at best, if we want to shoehorn this into the paradigm where something is being "chosen," it can be thought of as the auditory system <span style="line-height: 1.5;">selecting a rational at random </span>from the distribution. Intermediate values of a can be seen as interpolating between these two cases.
Certain values of //a// reduce to simpler expressions and have special names:
===a=0: Harmonic Hartley Entropy===
[[math]]
H(d) = \log |R|
[[math]]
where |R| is the cardinality of the set of basis rationals. This assumes, in essence, an "infinitely dumb" auditory system which can do no better than picking a rational number from a uniform distribution completely at random. All dyads have the same Harmonic Hartley Entropy.
===a=1: Harmonic Shannon Entropy (Harmonic Entropy)===
[[math]]
H(d) = -\sum_{b} p_d(b) \log p_d(b)
[[math]]
This is Paul's original Harmonic Entropy. This can be thought of as an auditory system which simply selects a rational at random from the incoming distribution, weighted via the distribution itself.
===a=2: Harmonic Collision Entropy===
[[math]]
H(d) = -\log \sum_b p_d(b)^2 = -log (P_d = Q_d)
[[math]]
where P<span style="font-size: 90%; vertical-align: sub;">d</span> and Q<span style="font-size: 90%; vertical-align: sub;">d</span> are independent and identically distributed random variables corresponding to the same dyad, and the collision entropy is the same as the negative log of the probability that the two variables produce the same outcome.
===a=∞: Harmonic Min-Entropy===
[[math]]
-\log \max_b p_d(b)
[[math]]
This is the min-entropy, which simply takes the negative log of the largest probability in the distribution. This can be thought of as representing the "strength" of the incoming dyad from being "deciphered" by a "best-case" auditory system.
=References=
[[http://www.webcitation.org/60qOlJVFS|Paul Erlich article]]
[[http://sethares.engr.wisc.edu/paperspdf/HarmonicEntropy.pdf|William Sethares article]]
[[http://www.tonalsoft.com/enc/h/harmonic-entropy.aspx|Harmonic entropy (TonalSoft encyclopedia)]]
[[http://launch.groups.yahoo.com/group/harmonic_entropy/|Harmonic entropy group on Yahoo]]
[[@http://www.mikebattagliamusic.com/HE-JS/HE.html|Harmonic entropy graph calculator (JavaScript)]]Original HTML content:
<html><head><title>Harmonic Entropy</title></head><body><!-- ws:start:WikiTextHeadingRule:6:<h1> --><h1 id="toc0"><a name="Introduction"></a><!-- ws:end:WikiTextHeadingRule:6 -->Introduction</h1>
<strong>Harmonic Entropy</strong>, sometimes abbreviated as "HE", is a simple model to quantify the extent to which musical chords exhibit various psychoacoustic effects, lumped together in a single construct called psychoacoustic <strong>concordance</strong>. It was invented by Paul Erlich and developed extensively on the Yahoo! tuning and harmonic_entropy lists. Various later contributions to the model have been made by Steve Martin, Mike Battaglia, Keenan Pepper, and others.<br />
<br />
<!-- ws:start:WikiTextHeadingRule:8:<h1> --><h1 id="toc1"><a name="Background"></a><!-- ws:end:WikiTextHeadingRule:8 -->Background</h1>
The general workings of the human auditory system lead to a plethora of well-documented and sonically interesting phenomena that can occur when a musical chord is played:<br />
<ul><li>The perception of partial <strong>timbral fusion</strong> of the chord into one complex sound</li><li>The appearance of a <strong>virtual fundamental</strong> pitch in the bass</li><li>Timbral <strong>beatlessness</strong>, compared to mistunings of the chord in the surrounding area</li><li>The appearance of a quick fluttering effect sometimes known as <strong>periodicity buzz</strong></li></ul><br />
All of these effects share two common characteristics for chords played with harmonic timbres:<br />
<ul><li><span style="line-height: 1.5;">they tend to appear most strongly for those chords with large subsets that correspond to simple chunks of the harmonic series</span></li><li><span style="line-height: 1.5;">the effects produced exhibit some degree of tolerance for mistuning</span></li></ul><br />
As the presence or absence of these effects tend to appear in tandem and are highly correlated with one another, we can speak of a general notion of the psychoacoustic <strong>concordance</strong> of an interval - the degree to which effects such as the above will appear when an arbitrary musical chord is played. Additionally, chords which are very inharmonic often exhibit a quality known as psychoacoustic <strong>discordance</strong>.<br />
<br />
While psychoacoustic concordance is not a feature universal to all styles of music, it has been utilized significantly in Western music in the study of intonation. For instance, flexible-pitch ensembles operating within 12-EDO, such as barbershop quartets and string ensembles, will often adjust intonationally from the underlying 12-EDO reference to maximize the concordance of individual chords. Indeed, the entire history of Western tuning theory -- from meantone temperament, to the various Baroque well-temperaments, to 12-EDO itself, to the modern <a class="wiki_link" href="http://xenharmonic.wikispaces.com/Regular%20Temperaments" target="_blank">theory of regular temperament</a> -- can be seen as an attempt to reason mathematically about how to generate manageable tuning systems that will maximize concordance and minimize discordance.<br />
<br />
The Harmonic Entropy model is a simple way of quantifying how much an arbitrary chord will exhibit psychoacoustic concordance.<br />
<br />
Concordance has often been confused with actual musical consonance, an unfortunate fact made more common by the<span style="line-height: 1.5;"> psychoacoustics literature under the unfortunate name </span><strong><span style="line-height: 1.5;">sensory consonance</span></strong><span style="line-height: 1.5;">, most often used to refer to phenomena related to roughness and beatlessness specifically. This is not to be confused with the more familiar construct of tonal stability, typically just called "consonance" in Western common practice music theory and sometimes clarified as "musical consonance" in the music cognition literature. To make matters worse, the literature has also at times referred to concordance -- and not tonal stability -- as </span><strong><span style="line-height: 1.5;">tonal consonance</span></strong><span style="line-height: 1.5;">, often referring to phenomena related to virtual pitch integration, creating a complete terminological mess. As a result, the term "consonance" has been completely avoided in this article.</span><br />
<br />
<!-- ws:start:WikiTextHeadingRule:10:<h1> --><h1 id="toc2"><a name="Model"></a><!-- ws:end:WikiTextHeadingRule:10 -->Model</h1>
The original Harmonic Entropy model limited itself to working with dyads. More recently, work by Steve Martin and others has extended this basic idea to higher-cardinality chords. This article will concern itself with dyads, as the dyadic case is still the most well-developed, and many of the ideas extend naturally to larger chords without need for much exposition.<br />
<br />
The general idea of Harmonic Entropy is to first develop a discrete probability distribution quantifying how strongly an arbitrary incoming dyad "matches" every element in a set of basis rational intervals, and then seeing how evenly distributed the resulting probabilities are. If the distribution for some dyad is spread out very evenly, such that there is no clear "victor" basis interval that dominates the distribution, the dyad is considered to be more discordant; on the other extreme, if the distribution tends to concentrate on one or a small set of dyads, the dyad is considered to be more concordant. A clear mathematical way of quantifying this is via <span style="line-height: 1.5;">the <a class="wiki_link_ext" href="http://en.wikipedia.org/wiki/Entropy_(information_theory)" rel="nofollow" target="_blank">Shannon entropy</a> of the probability distribution:</span><br />
<br />
<!-- ws:start:WikiTextMathRule:0:
[[math]]<br/>
H(d) = -\sum_{b} p_d(b) \log p_d(b)<br/>[[math]]
--><script type="math/tex">H(d) = -\sum_{b} p_d(b) \log p_d(b)</script><!-- ws:end:WikiTextMathRule:0 --><br />
<br />
where H(d) is the Shannon entropy of the dyad d, the b are all of the basis rationals in the set, and the p<span style="font-size: 90%; vertical-align: sub;">d</span>(b) is the probability assigned to basis rational b given an input dyad of d. This is the Harmonic Entropy of the dyad d.<br />
<br />
In order to systematically assign a probability distribution to this dyad, we first start by defining a <strong>spreading function</strong> that dictates how the dyad is "smeared" out in log-frequency space, representing how the auditory system allows for some tolerance for mistuning. The typical choice that we will assume here for a spreading function is a Gaussian distribution, with mean set to be centered around the incoming dyad, and standard deviation <strong>s</strong> typically taken as a free parameter in the system. We then divide the log-frequency spectrum up into <strong>domains</strong> assigned to the basis rationals; each rational is assigned a domain with lower bound equal to the mediant of itself and its nearest lower neighbor, and likewise with upper bound equal to the mediant of itself and its nearest upper neighbor. This process can be summarized by the following picture, taken from <a class="wiki_link_ext" href="http://sethares.engr.wisc.edu/paperspdf/HarmonicEntropy.pdf" rel="nofollow" target="_blank">William Sethares' paper on Harmonic Entropy</a>:<br />
<br />
<!-- ws:start:WikiTextRemoteImageRule:46:<img src="http://i.imgur.com/aQlqRXz.png" alt="" title="" /> --><img src="http://i.imgur.com/aQlqRXz.png" alt="external image aQlqRXz.png" title="external image aQlqRXz.png" /><!-- ws:end:WikiTextRemoteImageRule:46 --><br />
In this example, the f<span style="font-size: 90%; vertical-align: sub;">j+n</span> are the basis rationals, the r<span style="font-size: 12px; vertical-align: sub;">j+n</span> are the domains for each basis rational, and the bounds for each domain are the mediants between each f<span style="font-size: 12px; vertical-align: sub;">j+n </span>and its nearest neighbor. The probability assigned to each basis rational is then the area under the spreading function curve for each rational's domain. The entropy of this probability distribution is then the Harmonic Entropy for that dyad.<br />
<br />
A f<span style="line-height: 1.5;">airly typical choice of settings for a basic dyadic HE model would be:</span><br />
<ul><li><span style="line-height: 1.5;">The set of basis rationals is the set of all intervals bounded by some maximum Tenney height, with the bound typically notated as <strong>N</strong> and typically set to at least 10000</span></li><li><span style="line-height: 1.5;">The spreading function is typically a Gaussian distribution with s = 1%, or about 17 cents</span></li></ul><br />
Setting N=10000 and s=1% yields the following curve, where the x-axis represents the width in cents of the dyad, and the y-axis represents <em>discordance</em>, measured in bits of Shannon entropy:<br />
<!-- ws:start:WikiTextRemoteImageRule:47:<img src="http://i.imgur.com/tNg7z1P.png" alt="" title="" style="height: 404px; width: 800px;" /> --><img src="http://i.imgur.com/tNg7z1P.png" alt="external image tNg7z1P.png" title="external image tNg7z1P.png" style="height: 404px; width: 800px;" /><!-- ws:end:WikiTextRemoteImageRule:47 --><br />
Other ways of generating the set of basis rationals have also been explored, most notably by bounding by <a class="wiki_link" href="/height" target="_blank">Weil height</a> rather than Tenney height. The following is the curve that results for a Weil height bound 100 and s=1%, which has the primary effect of altering the slope of the curve:<br />
<br />
<!-- ws:start:WikiTextRemoteImageRule:48:<img src="http://i.imgur.com/TZdU6eD.png" alt="" title="" style="height: 410px; width: 800px;" /> --><img src="http://i.imgur.com/TZdU6eD.png" alt="external image TZdU6eD.png" title="external image TZdU6eD.png" style="height: 410px; width: 800px;" /><!-- ws:end:WikiTextRemoteImageRule:48 --><br />
<br />
Other spreading functions have also been explored, such as the use of the "Vos function" of a·exp(b|x|) rather than the Gaussian distribution.<br />
<br />
Finally, it should be noted that the use of Tenney height in bounding the rationals seems to lead to domain widths that are proportional to 1/sqrt(nd) for each rational, and that the use of Weil-bounded subsets of the rationals leads to domain widths that are proportional to 1/max(n,d) for each rational. It is a conjecture of Erlich's that this is true in the limit as N→∞, assuming that it's possible to suitably modify the HE equation so that the curve converges in the limit.<br />
<br />
These approximations have sometimes been used as quick computational substitutes for actually integrating the spreading function over each domain, so long as the resulting probabilities are normalized so that the sum is 1. The pictures above use these approximations.<br />
<br />
<!-- ws:start:WikiTextHeadingRule:12:<h1> --><h1 id="toc3"><a name="Harmonic Rényi Entropy"></a><!-- ws:end:WikiTextHeadingRule:12 -->Harmonic Rényi Entropy</h1>
<br />
An extension to the base Harmonic Entropy model, proposed by Mike Battaglia, is to generalize the use of <a class="wiki_link_ext" href="http://en.wikipedia.org/wiki/Entropy_(information_theory)" rel="nofollow" target="_blank">Shannon entropy</a> by replacing it instead with <a class="wiki_link_ext" href="http://en.wikipedia.org/wiki/R%C3%A9nyi_entropy" rel="nofollow" target="_blank">Rényi entropy</a>, a q-analog of Shannon's original entropy. The <strong>Harmonic Rényi Entropy of order a</strong> of an incoming dyad can be defined as follows:<br />
<br />
<!-- ws:start:WikiTextMathRule:1:
[[math]]<br/>
H(d) = \frac{1}{1-a} \log \sum_b p_d(b)^a<br/>[[math]]
--><script type="math/tex">H(d) = \frac{1}{1-a} \log \sum_b p_d(b)^a</script><!-- ws:end:WikiTextMathRule:1 --><br />
<br />
where the p_d(b) are the probabilities assigned by dyad d to each basis rational b. It is noteworthy that Rényi entropy converges to Shannon entropy in the limit as a→1, a fact which can be verified using L'Hôpital's rule as found <a class="wiki_link_ext" href="http://www.sonycsl.co.jp/person/nielsen/Note-HopitalRuleShannonRenyiTsallis.pdf" rel="nofollow" target="_blank">here</a>.<br />
<br />
Musically speaking, the parameter <em><strong>a</strong></em> can be interpreted as the extent to which the rational-matching process for incoming dyads is considered to be an intelligent, active process by which the auditory system actively analyzes the probability distribution and seeks out the "victor" rational with the greatest probability. Some psychoacoustic effects naturally fit into this paradigm, such as the virtual pitch integration process, which actually does attempt to find a single victor when matching incoming chords with chunks of the harmonic series. Other psychoacoustic effects, such as that of beatlessness, may instead be better viewed as "dumb" processes whereby nothing in particular is being "chosen," but where a more uniform distribution of matching rational numbers for a dyad simply generates a more discordant sonic effect. Increasing values of <em>a</em> correspond to a model that is more "active" in this way.<br />
<br />
This interpretation comes from the use of the Rényi entropy in cryptography as a measure of the strength of a cryptographic code in the face of an intelligent attacker, an application for which Shannon entropy has long been known to be insufficient as described in <a class="wiki_link_ext" href="http://users.cis.fiu.edu/~smithg/papers/qest11.pdf" rel="nofollow" target="_blank">this paper</a> and <a class="wiki_link_ext" href="http://www.ietf.org/rfc/rfc4086.txt" rel="nofollow" target="_blank">this RFC</a>. More precisely, the Rényi entropy of order ∞, also called the <strong>min-entropy</strong>, is used to measure the strength of the randomness used to define a cryptographic secret against a "worst-case" attacker who has complete knowledge of the probability distribution from which cryptographic secrets are drawn. In a musical context, by considering the incoming dyad as analogous to a cryptographic code which is attempting to be "cracked" by an intelligent auditory system, we can consider that the analogous "worst-case attacker" would be a "best-case auditory system" which has complete awareness of the probability distribution for any incoming dyad and actively chooses the strongest rational.<br />
<br />
The use of a=∞ min-entropy would reflect this view of an "intelligent" auditory system that can analyze the incoming probability distribution and always pick one of the matching rationals that holds a plurality. In contrast, the use of a=1 Shannon entropy reflects a much "dumber" process which performs no such analysis; at best, if we want to shoehorn this into the paradigm where something is being "chosen," it can be thought of as the auditory system <span style="line-height: 1.5;">selecting a rational at random </span>from the distribution. Intermediate values of a can be seen as interpolating between these two cases.<br />
<br />
Certain values of <em>a</em> reduce to simpler expressions and have special names:<br />
<br />
<!-- ws:start:WikiTextHeadingRule:14:<h3> --><h3 id="toc4"><a name="Harmonic Rényi Entropy--a=0: Harmonic Hartley Entropy"></a><!-- ws:end:WikiTextHeadingRule:14 -->a=0: Harmonic Hartley Entropy</h3>
<!-- ws:start:WikiTextMathRule:2:
[[math]]<br/>
H(d) = \log |R|<br/>[[math]]
--><script type="math/tex">H(d) = \log |R|</script><!-- ws:end:WikiTextMathRule:2 --><br />
where |R| is the cardinality of the set of basis rationals. This assumes, in essence, an "infinitely dumb" auditory system which can do no better than picking a rational number from a uniform distribution completely at random. All dyads have the same Harmonic Hartley Entropy.<br />
<br />
<!-- ws:start:WikiTextHeadingRule:16:<h3> --><h3 id="toc5"><a name="Harmonic Rényi Entropy--a=1: Harmonic Shannon Entropy (Harmonic Entropy)"></a><!-- ws:end:WikiTextHeadingRule:16 -->a=1: Harmonic Shannon Entropy (Harmonic Entropy)</h3>
<!-- ws:start:WikiTextMathRule:3:
[[math]]<br/>
H(d) = -\sum_{b} p_d(b) \log p_d(b)<br/>[[math]]
--><script type="math/tex">H(d) = -\sum_{b} p_d(b) \log p_d(b)</script><!-- ws:end:WikiTextMathRule:3 --><br />
This is Paul's original Harmonic Entropy. This can be thought of as an auditory system which simply selects a rational at random from the incoming distribution, weighted via the distribution itself.<br />
<br />
<!-- ws:start:WikiTextHeadingRule:18:<h3> --><h3 id="toc6"><a name="Harmonic Rényi Entropy--a=2: Harmonic Collision Entropy"></a><!-- ws:end:WikiTextHeadingRule:18 -->a=2: Harmonic Collision Entropy</h3>
<!-- ws:start:WikiTextMathRule:4:
[[math]]<br/>
H(d) = -\log \sum_b p_d(b)^2 = -log (P_d = Q_d)<br/>[[math]]
--><script type="math/tex">H(d) = -\log \sum_b p_d(b)^2 = -log (P_d = Q_d)</script><!-- ws:end:WikiTextMathRule:4 --><br />
where P<span style="font-size: 90%; vertical-align: sub;">d</span> and Q<span style="font-size: 90%; vertical-align: sub;">d</span> are independent and identically distributed random variables corresponding to the same dyad, and the collision entropy is the same as the negative log of the probability that the two variables produce the same outcome.<br />
<br />
<!-- ws:start:WikiTextHeadingRule:20:<h3> --><h3 id="toc7"><a name="Harmonic Rényi Entropy--a=∞: Harmonic Min-Entropy"></a><!-- ws:end:WikiTextHeadingRule:20 -->a=∞: Harmonic Min-Entropy</h3>
<!-- ws:start:WikiTextMathRule:5:
[[math]]<br/>
-\log \max_b p_d(b)<br/>[[math]]
--><script type="math/tex">-\log \max_b p_d(b)</script><!-- ws:end:WikiTextMathRule:5 --><br />
This is the min-entropy, which simply takes the negative log of the largest probability in the distribution. This can be thought of as representing the "strength" of the incoming dyad from being "deciphered" by a "best-case" auditory system.<br />
<br />
<!-- ws:start:WikiTextHeadingRule:22:<h1> --><h1 id="toc8"><a name="References"></a><!-- ws:end:WikiTextHeadingRule:22 -->References</h1>
<a class="wiki_link_ext" href="http://www.webcitation.org/60qOlJVFS" rel="nofollow">Paul Erlich article</a><br />
<a class="wiki_link_ext" href="http://sethares.engr.wisc.edu/paperspdf/HarmonicEntropy.pdf" rel="nofollow">William Sethares article</a><br />
<a class="wiki_link_ext" href="http://www.tonalsoft.com/enc/h/harmonic-entropy.aspx" rel="nofollow">Harmonic entropy (TonalSoft encyclopedia)</a><br />
<a class="wiki_link_ext" href="http://launch.groups.yahoo.com/group/harmonic_entropy/" rel="nofollow">Harmonic entropy group on Yahoo</a><br />
<a class="wiki_link_ext" href="http://www.mikebattagliamusic.com/HE-JS/HE.html" rel="nofollow" target="_blank">Harmonic entropy graph calculator (JavaScript)</a></body></html>