Harmonic entropy: Difference between revisions
Wikispaces>mbattaglia1 **Imported revision 515668008 - Original comment: ** |
Wikispaces>mbattaglia1 **Imported revision 515668950 - Original comment: ** |
||
| Line 1: | Line 1: | ||
<h2>IMPORTED REVISION FROM WIKISPACES</h2> | <h2>IMPORTED REVISION FROM WIKISPACES</h2> | ||
This is an imported revision from Wikispaces. The revision metadata is included below for reference:<br> | This is an imported revision from Wikispaces. The revision metadata is included below for reference:<br> | ||
: This revision was by author [[User:mbattaglia1|mbattaglia1]] and made on <tt>2014-07-06 13: | : This revision was by author [[User:mbattaglia1|mbattaglia1]] and made on <tt>2014-07-06 13:29:39 UTC</tt>.<br> | ||
: The original revision id was <tt> | : The original revision id was <tt>515668950</tt>.<br> | ||
: The revision comment was: <tt></tt><br> | : The revision comment was: <tt></tt><br> | ||
The revision contents are below, presented both in the original Wikispaces Wikitext format, and in HTML exactly as Wikispaces rendered it.<br> | The revision contents are below, presented both in the original Wikispaces Wikitext format, and in HTML exactly as Wikispaces rendered it.<br> | ||
| Line 34: | Line 34: | ||
[[math]] | [[math]] | ||
H(d) = -\sum_{b} p_d(b) \ | H(d) = -\sum_{b} p_d(b) \log_β p_d(b) | ||
[[math]] | [[math]] | ||
where H(d) is the Shannon entropy of the dyad d, the b are all of the basis rationals in the set, | where H(d) is the Shannon entropy of the dyad d, the b are all of the basis rationals in the set, the p<span style="font-size: 90%; vertical-align: sub;">d</span>(b) is the probability assigned to basis rational b given an input dyad of d, and the logarithm β reflects the units of information being used (by convention, we set β=2, corresponding to the use of bits). This is the Harmonic Entropy of the dyad d. | ||
In order to systematically assign a probability distribution to this dyad, we first start by defining a **spreading function** that dictates how the dyad is "smeared" out in log-frequency space, representing how the auditory system allows for some tolerance for mistuning. The typical choice that we will assume here for a spreading function is a Gaussian distribution, with mean set to be centered around the incoming dyad, and standard deviation **s** typically taken as a free parameter in the system. | In order to systematically assign a probability distribution to this dyad, we first start by defining a **spreading function** that dictates how the dyad is "smeared" out in log-frequency space, representing how the auditory system allows for some tolerance for mistuning. The typical choice that we will assume here for a spreading function is a Gaussian distribution, with mean set to be centered around the incoming dyad, and standard deviation **s** typically taken as a free parameter in the system. | ||
| Line 43: | Line 43: | ||
A fairly typical choice of settings for a basic dyadic HE model would be: | A fairly typical choice of settings for a basic dyadic HE model would be: | ||
* The set of basis rationals is the set of all intervals bounded by some maximum Tenney height, with the bound typically notated as **N** and set to at least 10000 | * The set of basis rationals is the set of all intervals bounded by some maximum Tenney height, with the bound typically notated as **N** and set to at least 10000 | ||
* The spreading function is typically a Gaussian distribution with | * The spreading function is typically a Gaussian distribution with a frequency deviation of 1% either way, or about s=~17 cents | ||
Other spreading functions have also been explored, such as the use of the "Vos function" of a·exp(b|x|) rather than the Gaussian distribution. We will assume the Gaussian distribution as the spreading function for the remainder of this article, so that the spreading function for a dyad d can be written as follows: | Other spreading functions have also been explored, such as the use of the "Vos function" of a·exp(b|x|) rather than the Gaussian distribution. We will assume the Gaussian distribution as the spreading function for the remainder of this article, so that the spreading function for a dyad d can be written as follows: | ||
| Line 51: | Line 51: | ||
[[math]] | [[math]] | ||
where //s// becomes the standard deviation of the Gaussian, being an ASCII-friendly version of the more familiar symbol σ for representing the standard deviation. We assume here that the variable x represents cents, and will adopt this convention for the remainder of the article. | where //s// becomes the standard deviation of the Gaussian, being an ASCII-friendly version of the more familiar symbol σ for representing the standard deviation. | ||
We assume here that the variable x represents cents, and will adopt this convention for the remainder of the article. Note that in previous expositions on Harmonic Entropy, //s// was sometimes given in units representing a percentage of linear-frequency deviation; we allow //s// to stand for cents here to simplify the notation. To convert from a percentage to cents, the formula cents = 1200*log2(1+percentage) can be used. | |||
Given a spreading function and set of basis rationals, there are two different procedures commonly used to assign probabilities to each rational. The first, the **domain-integral approach**, works for arbitrary nowhere dense sets of rationals without any further free parameters. The second, the **complexity-normalization approach**, has nice mathematical properties which sometimes make it easier to compute and which may lead to generalizations to infinite sets of rationals which are sometimes dense in the reals. It is conjectured that there are certain important limiting situations where the two converge; both are described in detail below. | Given a spreading function and set of basis rationals, there are two different procedures commonly used to assign probabilities to each rational. The first, the **domain-integral approach**, works for arbitrary nowhere dense sets of rationals without any further free parameters. The second, the **complexity-normalization approach**, has nice mathematical properties which sometimes make it easier to compute and which may lead to generalizations to infinite sets of rationals which are sometimes dense in the reals. It is conjectured that there are certain important limiting situations where the two converge; both are described in detail below. | ||
| Line 75: | Line 77: | ||
[[math]] | [[math]] | ||
q_d(b) = \frac{s_d(\cent(b))}{\sqrt{n_b \cdot d_b}} | |||
[[math]] | [[math]] | ||
where n<span style="vertical-align: sub;">b</span> and d<span style="vertical-align: sub;">b</span> are the numerator and denominator, respectively, of basis rational b. Again, the set of basis rationals here is assumed to be all of those rationals of Tenney Height ≤ N for some N. | where the q<span style="font-size: 12px; vertical-align: sub;">d</span>(b) now represent the unnormalized "probabilities", and n<span style="vertical-align: sub;">b</span> and d<span style="vertical-align: sub;">b</span> are the numerator and denominator, respectively, of basis rational b. Again, the set of basis rationals here is assumed to be all of those rationals of Tenney Height ≤ N for some N. | ||
A similar observation for the use of Weil-bounded subsets of the rationals suggests domain widths of 1/max(n,d), yielding instead the following formula: | A similar observation for the use of Weil-bounded subsets of the rationals suggests domain widths of 1/max(n,d), yielding instead the following formula: | ||
[[math]] | [[math]] | ||
q_d(b) = \frac{s_d(\cent(b))}{\max(n_b,d_b)} | |||
[[math]] | [[math]] | ||
| Line 91: | Line 93: | ||
[[math]] | [[math]] | ||
q_d(b) = \frac{s_d(\cent(b))}{\|b\|} | |||
[[math]] | [[math]] | ||
where ||b|| denotes a complexity function mapping from rational numbers to reals. | where ||b|| denotes a complexity function mapping from rational numbers to reals. | ||
As these "probabilities" don't sum to 1, the result is not a probability distribution at all and invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1: | |||
[[math]] | [[math]] | ||
p_d(b) = \frac{q_d(b)}{\sum_b q_d(b)} | |||
[[math]] | [[math]] | ||
The | The p<span style="vertical-align: sub;">d</span>(b) are then used directly to compute the entropy. | ||
This approach to assigning probabilities to basis rationals is useful because it hypothetically makes it possible to consider the HE of sets of rationals which are dense in the reals, or even the entire set of positive rationals ℚ<span style="font-size: 80%; vertical-align: super;">+</span>, although the best way to do this is a subject of ongoing research. | |||
=Examples= | |||
<span style="background-color: #ffffff;">In all of these examples, the x-axis represents the width in cents of the dyad, and the y-axis represents </span>//<span style="background-color: #ffffff;">discordance</span>// rather than concordance<span style="background-color: #ffffff;">, measured in bits of Shannon entropy. Note that by convention, the value for s is typically expressed as a percentage of frequency deviation; this can be converted to cents via </span> | |||
An extension to the base Harmonic Entropy model, proposed by Mike Battaglia, is to generalize the use of [[@http://en.wikipedia.org/wiki/Entropy_(information_theory)|Shannon entropy]] by replacing it instead with [[@http://en.wikipedia.org/wiki/R%C3%A9nyi_entropy|Rényi entropy]], a q-analog of Shannon's original entropy. The **Harmonic Rényi Entropy of order a** of an incoming dyad can be defined as follows: | <span style="background-color: #ffffff;">This uses as a spreading function the Gaussian distribution with s=~17 cents (or a lin-frequency deviation of 1%). The basis set is all rationals of Tenney height less than 10000. This uses the complexity-normalization approach, and the complexity function is sqrt(n·d):</span> | ||
[[image:http://i.imgur.com/tNg7z1P.png caption="external image tNg7z1P.png"]] | |||
<span style="background-color: #ffffff;">This example uses the same spreading function and standard deviation, but this time the basis set is all rationals of Weil height less than 100. The complexity function here is max(n,d):</span> | |||
[[image:http://i.imgur.com/TZdU6eD.png caption="external image TZdU6eD.png"]] | |||
The following image compares the domain-integral and complexity-normalization approaches by overlaying the two curves on top of each other. In both cases, the spreading function is again a Gaussian with s=~17 cents, and the basis set is all those rationals with Tenney height ≤ 10000. It can be seen that the curves are extremely similar, and that the locations of the minima and maxima are largely preserved: | |||
[[image:http://i.imgur.com/5QPTsEP.png width="800" height="600"]] | |||
<span style="font-size: 1.4em; line-height: 1.5;">**Harmonic Rényi Entropy**</span> | |||
An extension to the base Harmonic Entropy model, proposed by Mike Battaglia, is to generalize the use of [[@http://en.wikipedia.org/wiki/Entropy_(information_theory)|Shannon entropy]] by replacing it instead with [[@http://en.wikipedia.org/wiki/R%C3%A9nyi_entropy|Rényi entropy]], a [[@http://en.wikipedia.org/wiki/Q-analog|q-analog]] of Shannon's original entropy. The **Harmonic Rényi Entropy of order a** of an incoming dyad can be defined as follows: | |||
[[math]] | [[math]] | ||
H(d) = \frac{1}{1-a} \ | H(d) = \frac{1}{1-a} \log_β \sum_b p_d(b)^a | ||
[[math]] | [[math]] | ||
where the p_d(b) are the probabilities assigned by dyad d to each basis rational b. | where the p_d(b) are the probabilities assigned by dyad d to each basis rational b. Being a q-analog, it is noteworthy that Rényi entropy converges to Shannon entropy in the limit as a→1, a fact which can be verified using L'Hôpital's rule as found [[@http://www.sonycsl.co.jp/person/nielsen/Note-HopitalRuleShannonRenyiTsallis.pdf|here]]. | ||
The Rényi entropy has found use in cryptography as a measure of the strength of a cryptographic code in the face of an intelligent attacker, an application for which Shannon entropy has long been known to be insufficient as described in [[@http://users.cis.fiu.edu/~smithg/papers/qest11.pdf|this paper]] and [[@http://www.ietf.org/rfc/rfc4086.txt|this RFC]]. More precisely, the Rényi entropy of order ∞, also called the **min-entropy**, is used to measure the strength of the randomness used to define a cryptographic secret against a "worst-case" attacker who has complete knowledge of the probability distribution from which cryptographic secrets are drawn. In a musical context, by considering the incoming dyad as analogous to a cryptographic code which is attempting to be "cracked" by an intelligent auditory system, we can consider that the analogous "worst-case attacker" would be a "best-case auditory system" which has complete awareness of the probability distribution for any incoming dyad. This analogy would view such an auditory system as actively attempting to choose the most probable rational, rather than drawing a rational at random weighted by the distribution. | |||
The use of a=∞ min-entropy would reflect this view | The use of a=∞ min-entropy would reflect this view. In contrast, the use of a=1 Shannon entropy reflects a much "dumber" process which performs no such analysis and perhaps doesn't even seek to "choose" any sort of "victor" rational at all. As the parameter a interpolates between these two options, it <span style="line-height: 1.5;">can be interpreted as the extent to which the rational-matching process for incoming dyads is considered to be "intelligent" and "active" in this way.</span> | ||
<span style="line-height: 1.5;"> </span> | |||
Some psychoacoustic effects naturally fit into this paradigm, such as the virtual pitch integration process, which actually does attempt to find a single victor when matching incoming chords with chunks of the harmonic series. Other psychoacoustic effects, such as that of beatlessness, may instead be better viewed as "dumb" processes whereby nothing in particular is being "chosen," but where a more uniform distribution of matching rational numbers for a dyad simply generates a more discordant sonic effect. Different values of a can differentiate between the predominance given to these two types of effect in the overall construct of psychoacoustic concordance. | |||
Certain values of //a// reduce to simpler expressions and have special names: | Certain values of //a// reduce to simpler expressions and have special names: | ||
| Line 178: | Line 191: | ||
<!-- ws:start:WikiTextMathRule:0: | <!-- ws:start:WikiTextMathRule:0: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||
H(d) = -\sum_{b} p_d(b) \ | H(d) = -\sum_{b} p_d(b) \log_β p_d(b)&lt;br/&gt;[[math]] | ||
--><script type="math/tex">H(d) = -\sum_{b} p_d(b) \ | --><script type="math/tex">H(d) = -\sum_{b} p_d(b) \log_β p_d(b)</script><!-- ws:end:WikiTextMathRule:0 --><br /> | ||
<br /> | <br /> | ||
where H(d) is the Shannon entropy of the dyad d, the b are all of the basis rationals in the set, | where H(d) is the Shannon entropy of the dyad d, the b are all of the basis rationals in the set, the p<span style="font-size: 90%; vertical-align: sub;">d</span>(b) is the probability assigned to basis rational b given an input dyad of d, and the logarithm β reflects the units of information being used (by convention, we set β=2, corresponding to the use of bits). This is the Harmonic Entropy of the dyad d.<br /> | ||
<br /> | <br /> | ||
In order to systematically assign a probability distribution to this dyad, we first start by defining a <strong>spreading function</strong> that dictates how the dyad is &quot;smeared&quot; out in log-frequency space, representing how the auditory system allows for some tolerance for mistuning. The typical choice that we will assume here for a spreading function is a Gaussian distribution, with mean set to be centered around the incoming dyad, and standard deviation <strong>s</strong> typically taken as a free parameter in the system.<br /> | In order to systematically assign a probability distribution to this dyad, we first start by defining a <strong>spreading function</strong> that dictates how the dyad is &quot;smeared&quot; out in log-frequency space, representing how the auditory system allows for some tolerance for mistuning. The typical choice that we will assume here for a spreading function is a Gaussian distribution, with mean set to be centered around the incoming dyad, and standard deviation <strong>s</strong> typically taken as a free parameter in the system.<br /> | ||
<br /> | <br /> | ||
A fairly typical choice of settings for a basic dyadic HE model would be:<br /> | A fairly typical choice of settings for a basic dyadic HE model would be:<br /> | ||
<ul><li>The set of basis rationals is the set of all intervals bounded by some maximum Tenney height, with the bound typically notated as <strong>N</strong> and set to at least 10000</li><li>The spreading function is typically a Gaussian distribution with | <ul><li>The set of basis rationals is the set of all intervals bounded by some maximum Tenney height, with the bound typically notated as <strong>N</strong> and set to at least 10000</li><li>The spreading function is typically a Gaussian distribution with a frequency deviation of 1% either way, or about s=~17 cents</li></ul><br /> | ||
Other spreading functions have also been explored, such as the use of the &quot;Vos function&quot; of a·exp(b|x|) rather than the Gaussian distribution. We will assume the Gaussian distribution as the spreading function for the remainder of this article, so that the spreading function for a dyad d can be written as follows:<br /> | Other spreading functions have also been explored, such as the use of the &quot;Vos function&quot; of a·exp(b|x|) rather than the Gaussian distribution. We will assume the Gaussian distribution as the spreading function for the remainder of this article, so that the spreading function for a dyad d can be written as follows:<br /> | ||
<br /> | <br /> | ||
| Line 194: | Line 207: | ||
--><script type="math/tex">s_d(x) = \frac{1}{s\sqrt{2\pi}} e^{-\frac{(x-d)^2}{2s^2}}</script><!-- ws:end:WikiTextMathRule:1 --><br /> | --><script type="math/tex">s_d(x) = \frac{1}{s\sqrt{2\pi}} e^{-\frac{(x-d)^2}{2s^2}}</script><!-- ws:end:WikiTextMathRule:1 --><br /> | ||
<br /> | <br /> | ||
where <em>s</em> becomes the standard deviation of the Gaussian, being an ASCII-friendly version of the more familiar symbol σ for representing the standard deviation. We assume here that the variable x represents cents, and will adopt this convention for the remainder of the article.<br /> | where <em>s</em> becomes the standard deviation of the Gaussian, being an ASCII-friendly version of the more familiar symbol σ for representing the standard deviation.<br /> | ||
<br /> | |||
We assume here that the variable x represents cents, and will adopt this convention for the remainder of the article. Note that in previous expositions on Harmonic Entropy, <em>s</em> was sometimes given in units representing a percentage of linear-frequency deviation; we allow <em>s</em> to stand for cents here to simplify the notation. To convert from a percentage to cents, the formula cents = 1200*log2(1+percentage) can be used.<br /> | |||
<br /> | <br /> | ||
Given a spreading function and set of basis rationals, there are two different procedures commonly used to assign probabilities to each rational. The first, the <strong>domain-integral approach</strong>, works for arbitrary nowhere dense sets of rationals without any further free parameters. The second, the <strong>complexity-normalization approach</strong>, has nice mathematical properties which sometimes make it easier to compute and which may lead to generalizations to infinite sets of rationals which are sometimes dense in the reals. It is conjectured that there are certain important limiting situations where the two converge; both are described in detail below.<br /> | Given a spreading function and set of basis rationals, there are two different procedures commonly used to assign probabilities to each rational. The first, the <strong>domain-integral approach</strong>, works for arbitrary nowhere dense sets of rationals without any further free parameters. The second, the <strong>complexity-normalization approach</strong>, has nice mathematical properties which sometimes make it easier to compute and which may lead to generalizations to infinite sets of rationals which are sometimes dense in the reals. It is conjectured that there are certain important limiting situations where the two converge; both are described in detail below.<br /> | ||
| Line 220: | Line 235: | ||
<!-- ws:start:WikiTextMathRule:3: | <!-- ws:start:WikiTextMathRule:3: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||
q_d(b) = \frac{s_d(\cent(b))}{\sqrt{n_b \cdot d_b}}&lt;br/&gt;[[math]] | |||
--><script type="math/tex"> | --><script type="math/tex">q_d(b) = \frac{s_d(\cent(b))}{\sqrt{n_b \cdot d_b}}</script><!-- ws:end:WikiTextMathRule:3 --><br /> | ||
<br /> | <br /> | ||
where n<span style="vertical-align: sub;">b</span> and d<span style="vertical-align: sub;">b</span> are the numerator and denominator, respectively, of basis rational b. Again, the set of basis rationals here is assumed to be all of those rationals of Tenney Height ≤ N for some N.<br /> | where the q<span style="font-size: 12px; vertical-align: sub;">d</span>(b) now represent the unnormalized &quot;probabilities&quot;, and n<span style="vertical-align: sub;">b</span> and d<span style="vertical-align: sub;">b</span> are the numerator and denominator, respectively, of basis rational b. Again, the set of basis rationals here is assumed to be all of those rationals of Tenney Height ≤ N for some N.<br /> | ||
<br /> | <br /> | ||
A similar observation for the use of Weil-bounded subsets of the rationals suggests domain widths of 1/max(n,d), yielding instead the following formula:<br /> | A similar observation for the use of Weil-bounded subsets of the rationals suggests domain widths of 1/max(n,d), yielding instead the following formula:<br /> | ||
| Line 229: | Line 244: | ||
<!-- ws:start:WikiTextMathRule:4: | <!-- ws:start:WikiTextMathRule:4: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||
q_d(b) = \frac{s_d(\cent(b))}{\max(n_b,d_b)}&lt;br/&gt;[[math]] | |||
--><script type="math/tex"> | --><script type="math/tex">q_d(b) = \frac{s_d(\cent(b))}{\max(n_b,d_b)}</script><!-- ws:end:WikiTextMathRule:4 --><br /> | ||
<br /> | <br /> | ||
where this time the set of basis rationals is assumed to be all of those of Weil Height ≤ N for some N.<br /> | where this time the set of basis rationals is assumed to be all of those of Weil Height ≤ N for some N.<br /> | ||
| Line 238: | Line 253: | ||
<!-- ws:start:WikiTextMathRule:5: | <!-- ws:start:WikiTextMathRule:5: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||
q_d(b) = \frac{s_d(\cent(b))}{\|b\|}&lt;br/&gt;[[math]] | |||
--><script type="math/tex"> | --><script type="math/tex">q_d(b) = \frac{s_d(\cent(b))}{\|b\|}</script><!-- ws:end:WikiTextMathRule:5 --><br /> | ||
<br /> | <br /> | ||
where ||b|| denotes a complexity function mapping from rational numbers to reals.<br /> | where ||b|| denotes a complexity function mapping from rational numbers to reals.<br /> | ||
<br /> | <br /> | ||
As these &quot;probabilities&quot; don't sum to 1, the result is not a probability distribution at all and invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:<br /> | |||
<br /> | <br /> | ||
<!-- ws:start:WikiTextMathRule:6: | <!-- ws:start:WikiTextMathRule:6: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||
p_d(b) = \frac{q_d(b)}{\sum_b q_d(b)}&lt;br/&gt;[[math]] | |||
--><script type="math/tex"> | --><script type="math/tex">p_d(b) = \frac{q_d(b)}{\sum_b q_d(b)}</script><!-- ws:end:WikiTextMathRule:6 --><br /> | ||
<br /> | <br /> | ||
The | The p<span style="vertical-align: sub;">d</span>(b) are then used directly to compute the entropy.<br /> | ||
<br /> | <br /> | ||
This | This approach to assigning probabilities to basis rationals is useful because it hypothetically makes it possible to consider the HE of sets of rationals which are dense in the reals, or even the entire set of positive rationals ℚ<span style="font-size: 80%; vertical-align: super;">+</span>, although the best way to do this is a subject of ongoing research.<br /> | ||
<br /> | <br /> | ||
<!-- ws:start:WikiTextHeadingRule:22:&lt;h1&gt; --><h1 id="toc5"><a name=" | <!-- ws:start:WikiTextHeadingRule:22:&lt;h1&gt; --><h1 id="toc5"><a name="Examples"></a><!-- ws:end:WikiTextHeadingRule:22 -->Examples</h1> | ||
<br /> | <br /> | ||
An extension to the base Harmonic Entropy model, proposed by Mike Battaglia, is to generalize the use of <a class="wiki_link_ext" href="http://en.wikipedia.org/wiki/Entropy_(information_theory)" rel="nofollow" target="_blank">Shannon entropy</a> by replacing it instead with <a class="wiki_link_ext" href="http://en.wikipedia.org/wiki/R%C3%A9nyi_entropy" rel="nofollow" target="_blank">Rényi entropy</a>, a q-analog of Shannon's original entropy. The <strong>Harmonic Rényi Entropy of order a</strong> of an incoming dyad can be defined as follows:<br /> | <span style="background-color: #ffffff;">In all of these examples, the x-axis represents the width in cents of the dyad, and the y-axis represents </span><em><span style="background-color: #ffffff;">discordance</span></em> rather than concordance<span style="background-color: #ffffff;">, measured in bits of Shannon entropy. Note that by convention, the value for s is typically expressed as a percentage of frequency deviation; this can be converted to cents via </span><br /> | ||
<br /> | |||
<br /> | |||
<span style="background-color: #ffffff;">This uses as a spreading function the Gaussian distribution with s=~17 cents (or a lin-frequency deviation of 1%). The basis set is all rationals of Tenney height less than 10000. This uses the complexity-normalization approach, and the complexity function is sqrt(n·d):</span><br /> | |||
<!-- ws:start:WikiTextRemoteImageRule:57:&lt;img src=&quot;http://i.imgur.com/tNg7z1P.png&quot; alt=&quot;external image tNg7z1P.png&quot; title=&quot;external image tNg7z1P.png&quot; /&gt; --><table class="captionBox"><tr><td class="captionedImage"><img src="http://i.imgur.com/tNg7z1P.png" alt="external image tNg7z1P.png" title="external image tNg7z1P.png" /></td></tr><tr><td class="imageCaption">external image tNg7z1P.png</td></tr></table><!-- ws:end:WikiTextRemoteImageRule:57 --><br /> | |||
<span style="background-color: #ffffff;">This example uses the same spreading function and standard deviation, but this time the basis set is all rationals of Weil height less than 100. The complexity function here is max(n,d):</span><br /> | |||
<br /> | |||
<!-- ws:start:WikiTextRemoteImageRule:58:&lt;img src=&quot;http://i.imgur.com/TZdU6eD.png&quot; alt=&quot;external image TZdU6eD.png&quot; title=&quot;external image TZdU6eD.png&quot; /&gt; --><table class="captionBox"><tr><td class="captionedImage"><img src="http://i.imgur.com/TZdU6eD.png" alt="external image TZdU6eD.png" title="external image TZdU6eD.png" /></td></tr><tr><td class="imageCaption">external image TZdU6eD.png</td></tr></table><!-- ws:end:WikiTextRemoteImageRule:58 --><br /> | |||
The following image compares the domain-integral and complexity-normalization approaches by overlaying the two curves on top of each other. In both cases, the spreading function is again a Gaussian with s=~17 cents, and the basis set is all those rationals with Tenney height ≤ 10000. It can be seen that the curves are extremely similar, and that the locations of the minima and maxima are largely preserved:<br /> | |||
<!-- ws:start:WikiTextRemoteImageRule:59:&lt;img src=&quot;http://i.imgur.com/5QPTsEP.png&quot; alt=&quot;&quot; title=&quot;&quot; style=&quot;height: 600px; width: 800px;&quot; /&gt; --><img src="http://i.imgur.com/5QPTsEP.png" alt="external image 5QPTsEP.png" title="external image 5QPTsEP.png" style="height: 600px; width: 800px;" /><!-- ws:end:WikiTextRemoteImageRule:59 --><br /> | |||
<span style="font-size: 1.4em; line-height: 1.5;"><strong>Harmonic Rényi Entropy</strong></span><br /> | |||
An extension to the base Harmonic Entropy model, proposed by Mike Battaglia, is to generalize the use of <a class="wiki_link_ext" href="http://en.wikipedia.org/wiki/Entropy_(information_theory)" rel="nofollow" target="_blank">Shannon entropy</a> by replacing it instead with <a class="wiki_link_ext" href="http://en.wikipedia.org/wiki/R%C3%A9nyi_entropy" rel="nofollow" target="_blank">Rényi entropy</a>, a <a class="wiki_link_ext" href="http://en.wikipedia.org/wiki/Q-analog" rel="nofollow" target="_blank">q-analog</a> of Shannon's original entropy. The <strong>Harmonic Rényi Entropy of order a</strong> of an incoming dyad can be defined as follows:<br /> | |||
<br /> | <br /> | ||
<!-- ws:start:WikiTextMathRule:7: | <!-- ws:start:WikiTextMathRule:7: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||
H(d) = \frac{1}{1-a} \ | H(d) = \frac{1}{1-a} \log_β \sum_b p_d(b)^a&lt;br/&gt;[[math]] | ||
--><script type="math/tex">H(d) = \frac{1}{1-a} \ | --><script type="math/tex">H(d) = \frac{1}{1-a} \log_β \sum_b p_d(b)^a</script><!-- ws:end:WikiTextMathRule:7 --><br /> | ||
<br /> | <br /> | ||
where the p_d(b) are the probabilities assigned by dyad d to each basis rational b. Being a q-analog, it is noteworthy that Rényi entropy converges to Shannon entropy in the limit as a→1, a fact which can be verified using L'Hôpital's rule as found <a class="wiki_link_ext" href="http://www.sonycsl.co.jp/person/nielsen/Note-HopitalRuleShannonRenyiTsallis.pdf" rel="nofollow" target="_blank">here</a>.<br /> | |||
<br /> | <br /> | ||
The Rényi entropy has found use in cryptography as a measure of the strength of a cryptographic code in the face of an intelligent attacker, an application for which Shannon entropy has long been known to be insufficient as described in <a class="wiki_link_ext" href="http://users.cis.fiu.edu/~smithg/papers/qest11.pdf" rel="nofollow" target="_blank">this paper</a> and <a class="wiki_link_ext" href="http://www.ietf.org/rfc/rfc4086.txt" rel="nofollow" target="_blank">this RFC</a>. More precisely, the Rényi entropy of order ∞, also called the <strong>min-entropy</strong>, is used to measure the strength of the randomness used to define a cryptographic secret against a &quot;worst-case&quot; attacker who has complete knowledge of the probability distribution from which cryptographic secrets are drawn. In a musical context, by considering the incoming dyad as analogous to a cryptographic code which is attempting to be &quot;cracked&quot; by an intelligent auditory system, we can consider that the analogous &quot;worst-case attacker&quot; would be a &quot;best-case auditory system&quot; which has complete awareness of the probability distribution for any incoming dyad. This analogy would view such an auditory system as actively attempting to choose the most probable rational, rather than drawing a rational at random weighted by the distribution.<br /> | |||
<br /> | <br /> | ||
The use of a=∞ min-entropy would reflect this view | The use of a=∞ min-entropy would reflect this view. In contrast, the use of a=1 Shannon entropy reflects a much &quot;dumber&quot; process which performs no such analysis and perhaps doesn't even seek to &quot;choose&quot; any sort of &quot;victor&quot; rational at all. As the parameter a interpolates between these two options, it <span style="line-height: 1.5;">can be interpreted as the extent to which the rational-matching process for incoming dyads is considered to be &quot;intelligent&quot; and &quot;active&quot; in this way.</span><br /> | ||
<span style="line-height: 1.5;"> </span><br /> | |||
Some psychoacoustic effects naturally fit into this paradigm, such as the virtual pitch integration process, which actually does attempt to find a single victor when matching incoming chords with chunks of the harmonic series. Other psychoacoustic effects, such as that of beatlessness, may instead be better viewed as &quot;dumb&quot; processes whereby nothing in particular is being &quot;chosen,&quot; but where a more uniform distribution of matching rational numbers for a dyad simply generates a more discordant sonic effect. Different values of a can differentiate between the predominance given to these two types of effect in the overall construct of psychoacoustic concordance.<br /> | |||
<br /> | <br /> | ||
Certain values of <em>a</em> reduce to simpler expressions and have special names:<br /> | Certain values of <em>a</em> reduce to simpler expressions and have special names:<br /> | ||
<br /> | <br /> | ||
<!-- ws:start:WikiTextHeadingRule:24:&lt;h3&gt; --><h3 id="toc6"><a name=" | <!-- ws:start:WikiTextHeadingRule:24:&lt;h3&gt; --><h3 id="toc6"><a name="Examples--a=0: Harmonic Hartley Entropy"></a><!-- ws:end:WikiTextHeadingRule:24 -->a=0: Harmonic Hartley Entropy</h3> | ||
<!-- ws:start:WikiTextMathRule:8: | <!-- ws:start:WikiTextMathRule:8: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||
| Line 280: | Line 306: | ||
where |R| is the cardinality of the set of basis rationals. This assumes, in essence, an &quot;infinitely dumb&quot; auditory system which can do no better than picking a rational number from a uniform distribution completely at random. All dyads have the same Harmonic Hartley Entropy.<br /> | where |R| is the cardinality of the set of basis rationals. This assumes, in essence, an &quot;infinitely dumb&quot; auditory system which can do no better than picking a rational number from a uniform distribution completely at random. All dyads have the same Harmonic Hartley Entropy.<br /> | ||
<br /> | <br /> | ||
<!-- ws:start:WikiTextHeadingRule:26:&lt;h3&gt; --><h3 id="toc7"><a name=" | <!-- ws:start:WikiTextHeadingRule:26:&lt;h3&gt; --><h3 id="toc7"><a name="Examples--a=1: Harmonic Shannon Entropy (Harmonic Entropy)"></a><!-- ws:end:WikiTextHeadingRule:26 -->a=1: Harmonic Shannon Entropy (Harmonic Entropy)</h3> | ||
<!-- ws:start:WikiTextMathRule:9: | <!-- ws:start:WikiTextMathRule:9: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||
| Line 287: | Line 313: | ||
This is Paul's original Harmonic Entropy. This can be thought of as an auditory system which simply selects a rational at random from the incoming distribution, weighted via the distribution itself.<br /> | This is Paul's original Harmonic Entropy. This can be thought of as an auditory system which simply selects a rational at random from the incoming distribution, weighted via the distribution itself.<br /> | ||
<br /> | <br /> | ||
<!-- ws:start:WikiTextHeadingRule:28:&lt;h3&gt; --><h3 id="toc8"><a name=" | <!-- ws:start:WikiTextHeadingRule:28:&lt;h3&gt; --><h3 id="toc8"><a name="Examples--a=2: Harmonic Collision Entropy"></a><!-- ws:end:WikiTextHeadingRule:28 -->a=2: Harmonic Collision Entropy</h3> | ||
<!-- ws:start:WikiTextMathRule:10: | <!-- ws:start:WikiTextMathRule:10: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||
| Line 294: | Line 320: | ||
where P<span style="font-size: 90%; vertical-align: sub;">d</span> and Q<span style="font-size: 90%; vertical-align: sub;">d</span> are independent and identically distributed random variables corresponding to the same dyad, and the collision entropy is the same as the negative log of the probability that the two variables produce the same outcome.<br /> | where P<span style="font-size: 90%; vertical-align: sub;">d</span> and Q<span style="font-size: 90%; vertical-align: sub;">d</span> are independent and identically distributed random variables corresponding to the same dyad, and the collision entropy is the same as the negative log of the probability that the two variables produce the same outcome.<br /> | ||
<br /> | <br /> | ||
<!-- ws:start:WikiTextHeadingRule:30:&lt;h3&gt; --><h3 id="toc9"><a name=" | <!-- ws:start:WikiTextHeadingRule:30:&lt;h3&gt; --><h3 id="toc9"><a name="Examples--a=∞: Harmonic Min-Entropy"></a><!-- ws:end:WikiTextHeadingRule:30 -->a=∞: Harmonic Min-Entropy</h3> | ||
<!-- ws:start:WikiTextMathRule:11: | <!-- ws:start:WikiTextMathRule:11: | ||
[[math]]&lt;br/&gt; | [[math]]&lt;br/&gt; | ||