User:Contribution/Chanofreq Project

Chanofreq Project

Objective: The Chanofreq project seeks to create a standardized file format that allows precise tuning of each note on every MIDI channel to a specific frequency. The aim is to achieve this using a minimalistic data structure that prioritizes efficiency and simplicity.

Todo: Objective clarification
Expand this section to briefly explain the potential applications and benefits of this file format.

File Extension: .cnf

Format Name: Chanofreq

Etymology: Derived from Channel, Note, and Frequency.

File Structure

The Chanofreq format is a text-based structure comprising commands in the following format:

channel X note Y frequency Z

X: An integer ≥ 1, representing the MIDI channel.
Y: An integer ≥ 0, representing the note number.
Z: A decimal number within the range minfreq ≤ Z < maxfreq, representing the frequency in Hertz. This value is encoded as a positive integer by the parser with a bit depth defined by bitfreq, using the formula: parser_frequency = int(((Z - minfreq) * (2 ** bitfreq) / (maxfreq - minfreq)) + 0.5). Original default settings are set to: minfreq = 0 Hz, maxfreq = 18446.744073709551616 Hz, and bitfreq = 64. Thus, Z has a precision of fifteen decimal places. Parsers designed for 32-bit hardware and software should adopt the alternative default settings: minfreq = 0 Hz, maxfreq = 21474.83648 Hz, and bitfreq = 32. By doing so, the Z frequencies specified by users for the original default mode remain compatible, and maintain a precision of five decimal places with an additional sixth decimal place rounded to either 0 or 5, offering a convenient solution.

Todo: Simplify
Use 64-bit floating-point precision as the default standard, and make the bitfreq workaround optional for systems not supporting 64-bit floats.

Commands can be separated by a semicolon (';') or a newline.

Examples:

channel X1 note Y1 frequency Z1; channel X2 note Y2 frequency Z2
channel X3 note Y3 frequency Z3

Todo: Chanofreq Modes Implementation

Propose offering three distinct modes for chanofreq file structures:

chanofreq mode A: define the frequencies for each note on every channel individually
chanofreq mode B: define the frequencies for one channel, and an equave between each channel (a factor defined by a floating-point number)
chanofreq mode C: define the frequencies for one or more starting notes (referred to as a pattern), a sub-equave between each repetition of the pattern within one channel (a factor defined by a floating-point number), and an equave between each channel (a factor defined by a floating-point number)

Calculation:

parser_frequency = frequency * (sub-equave ** p) * (equave ** c)

Precision Handling:

To maintain precision when the parser calculates frequencies using equaves or sub-equaves, implement the following options:

64-bit Floating-Point Precision: This option simply sets the parser's frequency calculations to use 64-bit floats. (enough for mode A)
128-bit Floating-Point Precision: This option increases calculation precision to 128-bit floats. (mandatory for modes B and C. Otherwise, the error is too big.)

This precision level should be defined in the first line of the file with a specific syntax, for example:

chanofreq version ?.?.? mode ? precision x4

64-bit floating-point precision: Denoted as precision x2
128-bit floating-point precision: Denoted as precision x4

Note: Even if 128-bit precision is used for internal calculations, the output frequencies can be rounded to 64-bit float for compatibility with software and hardware devices.

Question:

Would it be advisable to offer modes B and C, given that significant errors can arise with 64-bit floats when using large exponents?

Answer:

It should be fine to use 64-bit float, as long as no rounding is applied, we stay within the human hearing range, and the number of notes remains reasonable. For example, if 125,000-EDO is programmed from a base frequency (20 Hz) and a factor (1.000005545192819) with exponents to map the entire multi-channel keyboard, after 10 octaves (20480 Hz), one phase error occurs every 2.04 days between two adjacent 64-bit float factor values. For the 402,653,184-EDO tuning scheme within chanofreq using factor and exponents, the phase precision between 64-bit float steps is only 78.9 seconds: 1/(440*(2**(-69/12))*1.0000000017214499**(402653184*(32/3)) - 440*(2**(-69/12))*1.0000000017214496**(402653184*(32/3))) = 78.9 seconds.

In comparison, the phase precision between 32-bit steps in 402,653,184-EDO is 12.86 hours, as calculated here: 1/(440*2**((127-69)/12)*2**(1/402653184)-440*2**((127-69)/12))/(60*60) = 12.86 hours. This shows that even with 64-bit floating-point precision, the factor and exponent solution is far less efficient than other tuning methods, but remains practical for moderately sized musical scales up to approximately 125,000-EDO, a size that still ensure a precision equivalent to 6 decimal places rounding in cents. However, using Mode B or C as a mediator for 32-bit MIDI tuning should be avoided.

Open Questions:

Should cents or intervals be allowed as an alternative to frequencies? In this case, a base frequency, base note, and base channel would be required. Refer to the .kbm format for guidance. For chanofreq, only a single file should be used to define the full multi-channel keyboard mapping.
Should hexadecimal representation of frequencies be allowed as an alternative to decimal?
Should the bitfreq workaround be maintained, or should the standard align with OddSound’s 64-bit floating-point format?

Syntax

File identification and version declaration

The first line of a Chanofreq file must declare the format and version to ensure compatibility:

chanofreq version ?.?.?

Metadata support

Metadata is optional but, if included, should appear on the second line for quick identification without needing to scan the entire file. YAML is used, and fields are open-ended.

<meta>
YAML content
</meta>

Comments

Inline comment:

# Inline comment

Multiline comment:

/* Multiline
comment */

Nota bene: YAML metadata only supports inline comment.

Tuning control

To define tuning for instruments controlled via MIDI, use:

channel X note Y frequency Z

For instruments with internal notes not controlled by MIDI (such as those with built-in sequencers or keyboards), use:

internal note Y frequency Z

To set a fallback frequency for any unspecified note Y across all channels and internal settings, use:

default note Y frequency Z

Frequency settings

Frequency values are encoded by the parser as positive integers with the help of three settings:

minfreq: Defines the lower bound of frequencies.
maxfreq: Defines the upper bound of frequencies.
bitfreq: Specifies the bit depth for the positive integer representation.

Frequencies are encoded according to the formula: parser_frequency = int(((FREQUENCY - minfreq) * (2 ** bitfreq) / (maxfreq - minfreq)) + 0.5)

Each unit increase in the encoded integer corresponds to a frequency increment of (maxfreq - minfreq) / (2 ** bitfreq).

The original default settings are configured as follows: minfreq = 0 Hz, maxfreq = 18446.744073709551616 Hz, and bitfreq = 64-bit. These values are intentionally chosen to ensure compatibility with the range of MIDI note frequencies (approximately 8.18 Hz to 12543.85 Hz) and 64-bit systems. This configuration avoids imposing any humanly perceptible or observable limitations on precision (1 phase error occurs once every ~63 million years).

For 32-bit hardware and software, the default frequency parameters are adjusted to minfreq = 0 Hz, maxfreq = 21474.83648 Hz, and bitfreq = 32-bit. These settings are designed to maintain compatibility with the frequency inputs specified for the 64-bit systems, while minimizing frequency approximation errors, providing a convenient solution where frequencies are accurate to five decimal places with an additional sixth decimal rounded to either 0 or 5. Under these conditions, 1 phase error occurs once every ~4.63 days, at worst.

While these default settings are suitable for most applications, users have the flexibility to adjust minfreq and maxfreq values and choose smaller or larger integer data types for bitfreq, allowing for customized precision and compatibility tailored to specific needs.

To change a setting:

@set minfreq: your_custom_minfreq

To revert to the default setting:

@unset minfreq

For changing or reverting for multiple settings at once, use a comma-separated list:

@set minfreq: your_custom_minfreq, maxfreq: your_custom_maxfreq, bitfreq: your_custom_bitfreq
@unset minfreq, maxfreq, bitfreq

Practical example of custom settings:

@set bitfreq: 16, minfreq: 0, maxfreq: 6553.6 # 1 phase error would occur once every 20 seconds.

Alias support

To assign an alias for a keyword (e.g., channel, note, frequency, internal, default, minfreq, maxfreq, bitfreq):

@alias keyword: your_custom_alias

To remove an alias:

@unalias keyword

For assigning or removing for multiple keywords at once, use a comma-separated list:

@alias keyword1: alias1, keyword2: alias2
@unalias keyword1, keyword2

Practical example of shorthands:

@alias channel: c, note: n, frequency: f, internal: i, default: d, minfreq: min, maxfreq: max, bitfreq: bit

Behavior on Loading

Upon loading a Chanofreq file, all notes start in an inactive state. As the file is read, each channel and internal command activates the specified note and assigns it the corresponding frequency. If a note that is already active is specified again, its frequency is updated to the new value provided. The same process applies to any default settings defined in the file. After processing the entire file, any notes that remain unspecified are assigned the default frequency if one is set. If no default frequency is provided, these notes remain inactive.

The parser encodes frequencies according to the formula: parser_frequency = int(((FREQUENCY - minfreq) * (2 ** bitfreq) / (maxfreq - minfreq)) + 0.5). Once a frequency setting is changed, it stays effective throughout the entire file until a new modification replaces it.

Once an alias is assigned to a keyword, it stays effective throughout the entire file until a new alias replaces it. The original keyword can still be used alongside the alias.

Comments are ignored by the parser and do not affect the file's execution. They can be used to provide explanations or annotations within the file.

Validation Rules

Todo: Define specific rules for validating the syntax and content of .cnf files.
Include error handling strategies (e.g., dealing with invalid note numbers, negative frequencies, incorrect syntax, or unsupported metadata), and consider different levels of error severity (warnings vs critical errors vs halts).

Todo: Case sensitivity
Clarify whether commands are case-sensitive.

Todo: Alias Support Clarification
Clarify if there are any restrictions on alias names (e.g., no spaces, must not conflict with existing commands).

Todo: Security
Address potential security concerns related to parsing untrusted .cnf files, such as buffer overflows or malformed data exploits. Elaborate on specific security measures, like input sanitization or restrictions on allowed data types, to avoid vulnerabilities.

Development Roadmap

File Format Specification: Draft a detailed file format specification document, incorporating input from stakeholders to ensure all use cases are covered.
Research Phase: Explore existing formats, gather user requirements, and establish a foundation for design decisions.
Prototyping Stage: Develop a minimal viable product (MVP) of the .cnf file parser and validator for early testing and feedback.
Set up a GitHub repository: Engage initial contributors and stakeholders to participate in the development process and foster open collaboration.
Feedback Loop: Refine the MVP parser and validator through extensive user testing and iterative improvements. Incorporate structured feedback mechanisms and introduce user personas and scenarios.
Web Interface: Develop an online tool for easily creating .cnf files.
Virtual Synthesizer: Build a basic virtual synthesizer supporting .cnf format to serve as a reference implementation for developers.
Compatibility Testing: Ensure the format's compatibility with both MIDI 1 and MIDI 2 standards, as well as with common DAWs and MIDI hardware.
Beta Testing and Feedback: Open a phase for musicians, developers, and other stakeholders to test the .cnf format and provide feedback.
Diverse Test Group: Gather feedback not only on the format itself but also on its intuitiveness and usability for a wide range of users, including those unfamiliar with microtuning, advanced microtonal musicians, plugin developers, and hardware developers.
Tools for developers: Provide ready-to-use open-source libraries in multiple languages, a simple SDK with comprehensive documentation, and reference plugins. Create tools for automatic code generation and an API for parsing .cnf files. Provide automated test suites that developers can use to verify that their implementations correctly adhere to the .cnf format (note: metadata support will be recommended but optional).
Performance Benchmarks: Ensure that both software and hardware can parse .cnf files as efficiently and quickly as possible, especially when dealing with large files or real-time tuning changes.
Documentation and Tutorials: Write comprehensive documentation and tutorials, including a technical reference for developers and a user guide for musicians and other end-users.
Presentation Video: Produce a compelling video to introduce the .cnf format and demonstrate the synthesizer, showcasing its utility for developers and users.
Integration Testing: Perform integration testing with various free virtual instruments to ensure smooth interoperability.
Collaborations: With Scale Workshop to integrate .cnf file creation. With Oddsound to support the .cnf format in their MTS-ESP plugins.
Update Xen Wiki: Add a "Multi-Channel Tuning Ability" column to the List of Microtonal Software Plugins, highlighting developers who properly implement the .cnf format or MTS-ESP Multi-Channel.
Advocacy: Promote the adoption of the .cnf format and MTS-ESP Multi-Channel among tuning-friendly companies.
Standardization Bodies: Engage with standardization bodies like MIDI Manufacturers Association (MMA) to discuss potential standardization of the format or its inclusion in future MIDI specifications.
Partnerships: Build strategic alliances with hardware manufacturers, synthesizer makers, MIDI device producers, DAW developers, and software companies to embed native .cnf format support across all platforms, enhancing compatibility and encouraging widespread industry adoption.
Educational Outreach: Collaborate with educational institutions to incorporate the .cnf format into music technology curricula and training programs, nurturing innovation and expertise about tuning among emerging musicians, developers, and audio engineers.

Todo: Break this into phases or highlight key milestones to make it more digestible.

This could also involve adding estimated timelines or priorities. Example Update:

"Phase 1: Research and Specification (Q1 2025)"
"Phase 2: Prototyping and Initial Testing (Q2 2025)"

Long-Term Strategy and Maintenance

The Chanofreq project aims to ensure that version 1.0.0 is fully compatible with both MIDI 1 and MIDI 2 standards while supporting an unlimited number of channels and notes. This design guarantees that the format will remain flexible and adaptable to future technological advancements.

Todo: Versioning Strategy

Develop a detailed versioning plan to manage updates during the beta testing phase leading up to version 1.0.0.
Ensure that version 1.0.0 is a stable and final release to minimize the need for significant updates afterward.
Establish clear criteria and timelines for ending the beta testing phase and officially launching version 1.0.0.
Implement a comprehensive strategy for handling updates post-version 1.0.0, focusing on maintaining backward compatibility.

Todo: Community Engagement
Add a section on community engagement and feedback mechanisms post-launch, ensuring continuous improvement and adaptability.