Defactoring

A regular temperament mapping is in defactored canonical (DC) form when it is put into Hermite Normal Form (HNF) after being "defactored".

vs. normal form

'normal' vs. 'canonical'

A mapping in canonical form uniquely identifies a set of mappings that are equivalent to it. Historically, the xenharmonic community has most often used the word normal for this idea, and evidence of this can be found on many pages across this wiki. And this is not wrong; normal forms are indeed often required to be unique. However, canonical forms are required to be unique even more often that normal forms are, and so we prefer the term canonical to normal for this purpose.

Also, using "canonical" helps establish a clear distinction from previous efforts to establish unique representations of equivalent mappings; due to its lack of historical use in RTT, it appears to be safe to simply use "canonical form" for short to refer to matrices in defactored canonical form.

vs. HNF

More importantly, and perhaps partially a result of this weak understanding of the difference between the conventions for normal and canonical forms, the xenharmonic community ha mistakenly used HNF as if it provides a unique representation of equivalent mappings. To be more specific, HNF does provide a unique representation of matrices, i.e. from a perspective of pure mathematics, and so you will certainly find throughout mathematical literature that HNF is described as providing a unique representation, and this is correct. However, when applied to the RTT domain, i.e. to mappings, the HNF sometimes fails to identify equivalent mappings as such.

The critical flaw with HNF is its failure to defactor matrices. The DC form that will be described here, on the other hand, does defactor matrices, and therefore it delivers a truly canonical result.

defactoring

Defactoring a matrix means to perform an operation on it which ensures that it is not "enfactored". And a matrix is considered to be "enfactored" if linear combinations of its rows can produce another row whose elements have a common factor (other than 1). This definition includes matrices whose rows already include a common factor, such as ⟨24 38 56] which has a common factor of 2. Being enfactored is a bad thing. Enfactored matrices — those in the RTT domain, at least — are sick, in a way; it's no accident that "enfactored" sounds sort of like "infected". Fortunately, the remedy is simple: all one has to do is "defactor" it — identify and divide out the common factor — to produce a healthy mapping.

Due to complications associated with enfactored mappings which we'll get into later in this article, we discourage treating them as representations of true temperaments. Instead we recommend that they be considered to represent mere "temperoids": temperament-like structures.

defactored & enfactored vs. saturated and (con)torted

If you've studied RTT extensively, you've probably encountered the terms "saturated" and "contorted" that are sometimes used to describe mappings. These two terms each have several flaws, and so this article presents alternative terms that are clearer and more descriptive: "defactored" and "enfactored", respectively. These new terms were coined by Dave Keenan in collaboration with Douglas Blumeyer in June of 2021.

Several concerns with the term "saturation" may be identified:

1. It does not have any obvious musical or mathematical meaning in this context (whereas enfactored and defactored do have obvious mathematical meaning).
2. The most common everyday usage of that word is for "saturated fats", which are the bad kind of fats, so it has negative associations, despite "saturation" being the good state for a matrix to be in.
3. Research on the tuning list archives suggests that Gene Ward Smith chose the word "saturation" because it was used in the mathematical software he was using at the time, Sage. However, there is another common but conflicting sense of saturation for matrices which clamps entry values to between -1 and 1. We think we should avoid propagating Sage's decision to overload matrix saturation with a second meaning.

As for the term "contorsion", the concerns with it are:

1. Again, it does not have any obvious musical or mathematical meaning in this context.
2. It's a word that was invented for RTT and has no meaning outside of RTT.
3. It was made up due to false assumptions. Through researching on tuning list archives, Dave and Douglas concluded that the associated concept of "torsion" was first described in January of 2002, with regards to commas used to form Fokker periodicity blocks. The concept of enfactoring was recognized in temperament mappings (though of course it did not yet go by that name), and — because torsion in lists of commas for Fokker blocks looks the same way as enfactoring looks in temperament comma-bases — torsion got conflated with it. But they can't truly be the same thing; the critical difference is that periodicity blocks do not involve tempering, while temperaments do. In concrete terms, while it can make sense to construct a Fokker block with [-4 4 -1⟩ in the middle and [-8 8 -2⟩ = 2[-4 4 -1⟩ at the edge, it does not make sense to imagine a temperament which tempers out 2[-4 4 -1⟩ but does not temper out [-4 4 -1⟩. Unfortunately, however, this critical difference seems to have been overlooked, and so it seemed that enfactored comma-bases exhibited torsion, and thus because mappings are the dual of comma-bases, then enfactoring of a mapping should be the dual of torsion, and because the prefix co- or con- means "dual" (as in vectors and covectors), the term "con-torsion" was coined for it. "Torsion" already has the problem of being an obscure mathematical term that means nothing to most people, "contorsion" just compounds that problem by being made up, and it is made up in order to convey a duality which is false. So while "torsion" could be preserved as a term for the effect on periodicity blocks (though there's almost certainly something more helpful than that, but that's a battle for another day), the term "contorsion" must be banished from the RTT community altogether.

In accordance with this research and reasoning, this article henceforth will eschew the terms saturation and contorsion in favor of defactored and enfactored.

identifying enfactored mappings

immediately apparent enfactoring

Sometimes the enfactoring of a mapping is immediately apparent. For example:

⟨24 38 56]

This mapping has only a single row, and we can see that every element in that row is even. Therefore we have a common factor of at least 2. In this case it is in fact exactly 2. So we can say that this is a 2-enfactored mapping.

Being enfactored tells us that it's wasteful to use this mapping. Specifically, being 2-enfactored tells us that we have 2x as many pitches as we need. Said another way, half of the pitches in our system are bringing nothing to the table, at least not in terms of approximating intervals built out of the 5-limit primes 2, 3, and 5, which is the primary goal of a temperament.

This is the mapping for 5-limit 24-ET. To be clear, we're not saying there's a major problem with 24 as an EDO. The point here is only that — if you're after a 5-limit temperament — you may as well use 12-ET. So we would consider 24-ET to stand for 24 Equal Temperoid.

Think of it this way: because every element is even, any JI interval you'd map with with the mapping must come out as an even number of steps of 24-ET, by the definition of the dot product, and every even step of 24-ET is just a step of 12-ET. Examples: [1 -2 1⟩.⟨24 38 56] = 24 - 76 + 56 = 4, [1 1 -1⟩.⟨24 38 56] = 24 + 38 - 56 = 6.

hidden enfactoring

Other times, enfactoring is less apparent. Consider this example:

[⟨3 0 -1] ⟨0 3 5]⟩

This is a form of 5-limit porcupine, a rank-2 temperament. Looking at either row, neither map has a common factor. But remember that we also need to check linear combinations of rows. If we subtract the 2nd row from the 1st row, we can produce the row ⟨3 -3 -6], which has a common factor of 3. So this mapping is also enfactored, even though it's not obvious from just looking at it.

If you're unsure why this ⟨3 -3 -6] matters despite not being in [⟨3 0 -1] ⟨0 3 5]⟩, we may need to quickly review some linear algebra fundamentals. It may take some getting used to, but a mapping can be changed to another equivalent mapping (both mappings will map input vectors to the same scalars) by replacing any row with linear combinations of its rows. That is, we could replace either ⟨3 0 -1] or ⟨0 3 5] in our original matrix [⟨3 0 -1] ⟨0 3 5]⟩ to get [⟨3 -3 -6] ⟨0 3 5]⟩ or [⟨3 0 -1] ⟨3 -3 -6]⟩ and any of these mappings represent the same temperament.

well-hidden enfactoring

Sometimes the hidden common factor is even harder to find. Consider the mapping [⟨6 5 -4] ⟨4 -4 1]⟩. To find this common factor, we need to linearly combine two of the first row ⟨6 5 -4] and negative three of the 2nd row ⟨4 -4 1] to produce ⟨0 22 -11]. So we can see here that its common factor is 11.

And so we can begin to see that the problem of identifying enfactored mapping may not be very simple or straightforward.

defactoring methods

Even better than identifying enfactored mappings is actually full-on defactoring them. Here are two methods that do just that: Smith defactoring, developed by Gene Ward Smith, and column Hermite defactoring, developed by Dave and Douglas (the name comes, of course, from Hermite normal form, which it uses).

Neither of these methods have been rigorously proven to always defactor mappings, but tests Douglas ran on thousands of random mappings strongly suggested that both methods work and give the exact same results as each other.

This article prefers column Hermite defactoring to Smith defactoring because it is:

• Cheaper computationally, wasting less resources computing things irrelevant to the result,
• Is easy to understand how it works, and can be worked out by hand (as we will demonstrate below),
• If interested, you can see what the common factor is, if there was any.

Column Hermite defactoring could not have been developed, however, were it not for Gene's pioneering work with the Smith defactoring (what he calls the process of "saturating" a mapping). At first Dave and Douglas had no idea what the right reducing matrix of the Smith decomposition (the process which also provides the Smith normal form) had to do with common factors, only that it somehow magically worked. So they analyzed the Smith decomposition until they isolated its key actions which actually effect the defactoring, and then honed their method down to do only these necessary actions. Again, they wouldn't have known where to start were it not for Gene.

precedent: Smith defactoring

Dave and Douglas did much of their work in Wolfram Language (formerly Mathematica), a popular programming language used for math problems. In this section we'll give examples using it.

An input mapping [math]\displaystyle{ m }[/math], such as the example Gene gives on the xen wiki page for Saturation, [⟨12 19 28 34] ⟨26 41 60 72]⟩, in Wolfram Language you would have to write as a list:

m = {{12,19,28,34},{26,41,60,72}};

The implementation of Gene's method in Wolfram Language is simple. Just two lines:

rightReducingMatrix[m_] := Last[SmithDecomposition[m]]
smithDefactor[m_] := Take[Inverse[rightReducingMatrix[m]], MatrixRank[m]]

So the first thing that happens to [math]\displaystyle{ m }[/math] when you pass it in to smithDefactor[] is that it calls rightReducingMatrix[] on it. This will find the Smith decomposition (using a function built in to Wolfram Language), which gives you three outputs: the Smith normal form, flanked by its left and right reducing matrices. We're asked only for the right reducing matrix, so we grab that with Last[]. So that's what the function on the first line, rightReducingMatrix[], does.

Then Gene asks us to invert this result and take its first [math]\displaystyle{ r }[/math] rows, where [math]\displaystyle{ r }[/math] is the rank of the temperament. Invert[] takes care of the inversion, of course. MatrixRank[m] gives the count of linearly independent rows to the mapping, AKA the rank, or count of generators in this temperament. In this case that's 2. And so Take[list, 2] simply returns the first 2 entries of the list.

Almost done! Except Gene not only defactors, he also calls for HNF, as we would, to achieve canonical form.

normalize[m_] := Last[HermiteDecomposition[m]]

Similar to the Smith Normal Form, we do a decomposition, which gives you the normal form plus some other bonus results. In this case we actually want the normal form itself, and it happens to be the last element in the result list. So putting it all together, we defactor and then normalize:

rightReducingMatrix[m_] := Last[SmithDecomposition[m]];
smithDefactor[m_] := Take[Inverse[rightReducingMatrix[m]], MatrixRank[m]];

normalize[m_] := Last[HermiteDecomposition[m]];

m = {{12,19,28,34},{26,41,60,72}};
normalize[smithDefactor[m]]

→ {{1,0,-4,-13},{0,1,4,10}}

And that result matches what Gene finds in that xen wiki article. Defactoring and normalizing is equivalent to canonicalization. 

new development: column Hermite defactoring

Here is the implementation for column Hermite defactoring:

hermiteUnimodular[m_]:=Transpose[First[HermiteDecomposition[Transpose[m]]]]
columnEchelonDefactor[m_]:=Take[Inverse[hermiteUnimodular[m]],MatrixRank[m]]

So this implementation begins by transposing the matrix, so that when it then performs the Hermite Decomposition, it is doing a column decomposition. We then take the unimodular matrix from the decomposition using First[], and Transpose[] it to in effect undo the transposition we did at the beginning.

1. 1. 1. include the L-shape work-through example
      2. refer to the email thread with Dave and talk about how it leaves the enfactoring behind while preserving the important information in the unimodular matrix, then does an inverse

other stuff to report

1. 1. 1. talk about the criteria, like has to be integer, full-rank

In addition to being canonical and defactored, DC form has other important properties:

• It is integer, i.e. contains only integer terms.
• reduced
• full-rank
• more I think, check email

1. 1. 1. show the examples we tried, like in the big defactoring table

canonical comma-bases

DC form is not only for mappings. Comma-bases — the duals of mappings — may also be put into DC form, as long as they are first antitransposed, and then antitransposed again at the end, or in other words, you sandwich the defactoring and HNF operations between antitransposes.

DC form is arguably even more important for comma-bases than it is for mappings, because enfactored mappings at least have clear musical meaning, while enfactored comma-bases are little but a wellspring of confusion. In other words, ⟨24 38 56] may not be a true temperament, but it still represents a temperoid and an EDO. However, {{

relationship between various matrix echelon forms

There are several well-known echelon forms for matrices that predate DC form. Let's review them and their properties.

list of forms

The most general form, with the fewest constraints, is simply called Row Echelon Form, or REF. Its only constraint is echelon form: each row's pivot, or first nonzero entry, is strictly to the right of the preceding row's pivot. This single constraint is fairly weak, and therefore REF does not produce a unique representation. This constraint is shared by every matrix form discussed here.

Integer Row Echelon Form, or IREF, is, unsurprisingly, any REF which meets an additional integer constraint, or in other words, that all of its entries are integers. This is still not a sufficiently strict set of constraints to ensure a unique representation.

Reduced Row Echelon Form, or RREF, takes REF in a different direction than IREF. Instead of stipulating anything about integers, it requires that the matrix is reduced, i.e. that the pivots are exactly equal to 1. This may require dividing rows by a number such that some resulting entries are no longer integers. Actually, there's a second part to the "reduced" constraint: each pivot column (a column which contains any row's pivot) has zeros for all other entries besides the pivot it contains. Due to these strict constraints, the RREF of a matrix is the first one we've looked at so far here which does ensure uniqueness.

So IREF and RREF make a Venn diagram inside the category of REF: some IREF are RREF, but there are some RREF that are not IREF and some IREF that are not RREF. When we scope the situation to a specific matrix, however, because RREF is a unique form, this means that one or the other sector of the Venn diagram for RREF will be empty; either the unique RREF will also be IREF (and therefore the RREF-but-not-IREF sector will be empty), or it will not be IREF (and vice versa).

Integer Reduced Row Echelon Form, or IRREF is the next form to discuss. Based on the name, one might expect this form to be a combination of the constraints for RREF and IREF, and therefore if represented in an Euler diagram (generalization of Venn diagram) would only exist within their intersection. However this is not the case. That's because the IRREF does not include the key constraint of RREF which is that all of the pivots must be 1. IRREF is produced by simply taking the unique RREF and multiplying each row by whatever minimum value is necessary to make all of the entries integers. Of course, this sometimes results in the pivots no longer being 1, so sometimes it is no longer RREF. It is always still REF, though, and because it is also always integer, that makes it always IREF; therefore, IRREF is strictly a subcategory of IREF. And because the RREF is unique, and the conversion process does not alter that, the IRREF is also unique.

It is not possible for an RREF to be IREF without also being IRREF.

Hermite Normal Form, or HNF, is the last form to discuss. This one's constraints begin with echelon form and integer, therefore every HNF is also IREF. But HNF is not exactly reduced; instead, it is normalized, which — similarly to reduced — is a two-part constraint. Where reduced requires that all pivots be exactly equal to 1, normalized requires only that all pivots be positive (positive integers, of course, due to the other integer constraint). And where reduced requires that all entries in pivot columns besides the pivots are exactly equal to 0, normalized requires only that all entries in pivot columns below the pivots are exactly equal to 0, while entries in pivot columns above the pivots only have to be strictly less than the pivot in the respective column (while still being non-negative). The normalization HNF uses is cool because this constraint, while strictly less strict than the reduced constraint used by RREF, is still strict enough to ensure uniqueness, but loose enough to ensure the integer constraint can be simultaneously satisfied, where RREF cannot ensure that.

So HNF has a lot in common with IRREF, which is the IREF you find by converting the RREF, but it is not always the same as IRREF.

temperament states

The three possible temperament states w/r/t equivalence of established matrix forms, plus in the bottom right, a state which is only possible for enfactored temperaments AKA temperoids.

All considered, three different states are possible for a given temperament:

1. The RREF, IRREF, and HNF are all the same. Example: meantone with all three equal to [⟨1 0 -4] ⟨0 1 4]⟩.
2. The RREF, IRREF, and HNF are all different. Example: porcupine with [⟨1 0 [math]\displaystyle{ -\frac13 }[/math]] ⟨0 1 [math]\displaystyle{ \frac53 }[/math]]⟩, [⟨3 0 -1] ⟨0 3 5]⟩, and [⟨1 2 3] ⟨0 3 5]⟩, respectively.
3. The IRREF and HNF are the same, but the RREF is different. Example: hanson with IRREF and HNF of [⟨1 0 1] ⟨0 6 5]⟩ but RREF of [⟨1 0 1] ⟨0 1 [math]\displaystyle{ \frac56 }[/math]]⟩.

A fourth state is possible but only for enfactored mappings, i.e. for temperoids. This is where the RREF and IRREF are the same, but the HNF is different. This is only possible for enfactored mappings because:

• Whenever the RREF and IRREF match, the IRREF has all 1's for pivots.
• But in that case, the IRREF would also be the HNF, because there's only one possible nonnegative integer value less than 1, which is zero, and so this form would also satisfy the normal constraint.
• The only way to avoid this would be to ensure that at least one pivot of the HNF was not 1, so that it could have some positive value somewhere in the entries above that pivot in its pivot column, to distinguish it from IRREF. For example, we could use [⟨1 1 0] ⟨0 2 1]⟩.
• But now if we look at the RREF of this mapping with this HNF, we have to divide rows until all pivots are 1. So that second row would be changed to ⟨0 1 [math]\displaystyle{ \frac12 }[/math]]. So now RREF won't match IRREF, because it contains a noninteger. The only way to prevent this would be if a pivot of the HNF was not 1, but still every entry in that row was evenly divisible by that value, such as [⟨1 1 0] ⟨0 2 4]⟩. But now that row is enfactored by its pivot's value.