Harmonic entropy: Difference between revisions

Line 1:

<h2>IMPORTED REVISION FROM WIKISPACES</h2>

This is an imported revision from Wikispaces. The revision metadata is included below for reference:<br>

: This revision was by author [[User:mbattaglia1|mbattaglia1]] and made on <tt>2014-07-06 15:11:19 UTC</tt>.<br>

: This revision was by author [[User:mbattaglia1|mbattaglia1]] and made on <tt>2014-07-06 15:14:17 UTC</tt>.<br>

: The original revision id was <tt>~~515672576~~</tt>.<br>

: The original revision id was <tt>515672698</tt>.<br>

: The revision comment was: <tt></tt><br>

The revision contents are below, presented both in the original Wikispaces Wikitext format, and in HTML exactly as Wikispaces rendered it.<br>

Line 100:

[[math]]

where ||b|| denotes a complexity function mapping from rational numbers to reals.

where ||b|| denotes a complexity function mapping from rational numbers to non-negative reals.

As these "probabilities" don't sum to 1, the result is not a probability distribution at all ~~and~~ invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:

As these "probabilities" don't sum to 1, the result is not a probability distribution at all, invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:

[[math]]

Line 171:

This is the min-entropy, which simply takes the negative log of the largest probability in the distribution. This can be thought of as representing the "strength" of the incoming dyad from being "deciphered" by a "best-case" auditory system. The name "min-entropy" reflects that the a=∞ case is guaranteed to be a lower bound among all Rényi entropies.

[[image:http://i.imgur.com/u91Xkww.png width="560" height="281"]]

//Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=//∞//). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the ~~complexty~~ function sqrt(n·d).//

//Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=//∞//). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the complexity function sqrt(n·d).//

=References=

[[http://www.webcitation.org/60qOlJVFS|Paul Erlich article]]

Line 283:

--><script type="math/tex">q_d(b) = \frac{s_d(\cent(b))}{\|b\|}</script><br />

<br />

where ||b|| denotes a complexity function mapping from rational numbers to reals.<br />

where ||b|| denotes a complexity function mapping from rational numbers to non-negative reals.<br />

<br />

As these &quot;probabilities&quot; don't sum to 1, the result is not a probability distribution at all ~~and~~ invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:<br />

As these &quot;probabilities&quot; don't sum to 1, the result is not a probability distribution at all, invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:<br />

<br />

<!-- ws:start:WikiTextMathRule:7:

Line 360:

This is the min-entropy, which simply takes the negative log of the largest probability in the distribution. This can be thought of as representing the &quot;strength&quot; of the incoming dyad from being &quot;deciphered&quot; by a &quot;best-case&quot; auditory system. The name &quot;min-entropy&quot; reflects that the a=∞ case is guaranteed to be a lower bound among all Rényi entropies.<br />

<img src="http://i.imgur.com/u91Xkww.png" alt="external image u91Xkww.png" title="external image u91Xkww.png" style="height: 281px; width: 560px;" /><br />

<em>Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=</em>∞<em>). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the ~~complexty~~ function sqrt(n·d).</em><br />

<em>Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=</em>∞<em>). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the complexity function sqrt(n·d).</em><br />

<h1 id="toc10"><a name="References"></a>References</h1>

<a class="wiki_link_ext" href="http://www.webcitation.org/60qOlJVFS" rel="nofollow">Paul Erlich article</a><br />

@@ Line 1: / Line 1: @@
 <h2>IMPORTED REVISION FROM WIKISPACES</h2>
 This is an imported revision from Wikispaces. The revision metadata is included below for reference:<br>
-: This revision was by author [[User:mbattaglia1|mbattaglia1]] and made on <tt>2014-07-06 15:11:19 UTC</tt>.<br>
+: This revision was by author [[User:mbattaglia1|mbattaglia1]] and made on <tt>2014-07-06 15:14:17 UTC</tt>.<br>
-: The original revision id was <tt>515672576</tt>.<br>
+: The original revision id was <tt>515672698</tt>.<br>
 : The revision comment was: <tt></tt><br>
 The revision contents are below, presented both in the original Wikispaces Wikitext format, and in HTML exactly as Wikispaces rendered it.<br>
@@ Line 100: / Line 100: @@
 [[math]]
-where ||b|| denotes a complexity function mapping from rational numbers to reals.
+where ||b|| denotes a complexity function mapping from rational numbers to non-negative reals.
-As these "probabilities" don't sum to 1, the result is not a probability distribution at all and invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:
+As these "probabilities" don't sum to 1, the result is not a probability distribution at all, invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:
 [[math]]
@@ Line 171: / Line 171: @@
 This is the min-entropy, which simply takes the negative log of the largest probability in the distribution. This can be thought of as representing the "strength" of the incoming dyad from being "deciphered" by a "best-case" auditory system. The name "min-entropy" reflects that the a=∞ case is guaranteed to be a lower bound among all Rényi entropies.
 [[image:http://i.imgur.com/u91Xkww.png width="560" height="281"]]
-//Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=//∞//). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the complexty function sqrt(n·d).//
+//Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=//∞//). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the complexity function sqrt(n·d).//
 =References=
 [[http://www.webcitation.org/60qOlJVFS|Paul Erlich article]]
@@ Line 283: / Line 283: @@
   --&gt;&lt;script type="math/tex"&gt;q_d(b) = \frac{s_d(\cent(b))}{\|b\|}&lt;/script&gt;&lt;!-- ws:end:WikiTextMathRule:6 --&gt;&lt;br /&gt;
 &lt;br /&gt;
-where ||b|| denotes a complexity function mapping from rational numbers to reals.&lt;br /&gt;
+where ||b|| denotes a complexity function mapping from rational numbers to non-negative reals.&lt;br /&gt;
 &lt;br /&gt;
-As these &amp;quot;probabilities&amp;quot; don't sum to 1, the result is not a probability distribution at all and invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:&lt;br /&gt;
+As these &amp;quot;probabilities&amp;quot; don't sum to 1, the result is not a probability distribution at all, invalidating the use of the Shannon Entropy. To rectify this, the distribution is normalized so that the probabilities do sum to 1:&lt;br /&gt;
 &lt;br /&gt;
 &lt;!-- ws:start:WikiTextMathRule:7:
@@ Line 360: / Line 360: @@
 This is the min-entropy, which simply takes the negative log of the largest probability in the distribution. This can be thought of as representing the &amp;quot;strength&amp;quot; of the incoming dyad from being &amp;quot;deciphered&amp;quot; by a &amp;quot;best-case&amp;quot; auditory system. The name &amp;quot;min-entropy&amp;quot; reflects that the a=∞ case is guaranteed to be a lower bound among all Rényi entropies.&lt;br /&gt;
 &lt;!-- ws:start:WikiTextRemoteImageRule:77:&amp;lt;img src=&amp;quot;http://i.imgur.com/u91Xkww.png&amp;quot; alt=&amp;quot;&amp;quot; title=&amp;quot;&amp;quot; style=&amp;quot;height: 281px; width: 560px;&amp;quot; /&amp;gt; --&gt;&lt;img src="http://i.imgur.com/u91Xkww.png" alt="external image u91Xkww.png" title="external image u91Xkww.png" style="height: 281px; width: 560px;" /&gt;&lt;!-- ws:end:WikiTextRemoteImageRule:77 --&gt;&lt;br /&gt;
-&lt;em&gt;Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=&lt;/em&gt;∞&lt;em&gt;). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the complexty function sqrt(n·d).&lt;/em&gt;&lt;br /&gt;
+&lt;em&gt;Harmonic Rényi Entropy with a=7, with the high value of a being chosen to approximate min-entropy (a=&lt;/em&gt;∞&lt;em&gt;). The basis set is still all rationals with Tenney height ≤ 10000, the spreading function a Gaussian distribution with s=1% (~17 cents), and the complexity function sqrt(n·d).&lt;/em&gt;&lt;br /&gt;
 &lt;!-- ws:start:WikiTextHeadingRule:33:&amp;lt;h1&amp;gt; --&gt;&lt;h1 id="toc10"&gt;&lt;a name="References"&gt;&lt;/a&gt;&lt;!-- ws:end:WikiTextHeadingRule:33 --&gt;References&lt;/h1&gt;
   &lt;a class="wiki_link_ext" href="http://www.webcitation.org/60qOlJVFS" rel="nofollow"&gt;Paul Erlich article&lt;/a&gt;&lt;br /&gt;