Introduction

sound-to-patch gives a producer a starting point in a synth for one sound. Vital and Serum 2 are the target synths, and Vital comes first. It has two paths:

• Audio path. One stem of one instrument and its BPM go in. The notes as notes.mid and a preset for the target synth come out (preset.vital or preset.SerumPreset), with an explanation.md that says how the preset works and which controls to turn next.
• Text path. The producer describes the sound in their own words. An agent turns the description into a preset and its explanation, and into notes when the description asks for them.

The project is at an early release. sound-to-patch analyze reads a stem and writes a report of its tempo, notes, level, spectrum, band edge and stereo image with a spectrogram, as the analysis guide shows. sound-to-patch preset inspect prints the header and the decoded settings of a Vital preset, and sound-to-patch preset apply writes one from an edit list, as the preset guide shows. The layer estimators and the text path arrive in later releases.

The API documentation covers the library. The development chapters describe how the project is tested and released.

Vital and Serum 2 are trademarks of their owners. This project is not affiliated with or endorsed by them.

Analysing a stem

sound-to-patch analyze reads one stem and writes a report of its tempo, notes, level, spectrum, band edge, stereo image, glides, pitch drops, vibrato, amplitude envelope, sidechain pump and voices, with a spectrogram image and the final notes.mid. The report is what an agent reads before it writes an edit list for the preset commands. The CLI reference lists every flag, and the layer report page every field.

Run it

Export one instrument as a WAV, AIFF, FLAC, MP3 or AAC file at 8000 to 192000 Hz and pass its tempo. A file at another rate is refused with its rate and the range; 44.1 or 48 kHz is the usual export. A file of three or more channels is read from its first two, with a warning that names the channel count:

sound-to-patch analyze bass.wav --bpm 120

tempo: 120 BPM as given; the stem holds 7 whole beats at 116.51 BPM, the slowest tempo of the search, and refining needs 8
detected range: MIDI 30 to 36, from the f0 track
detected voicing: monophonic, chord share 0.00
register window: 20 dB
cleanup voicing: monophonic, as detected
settings: detected, MIDI 30 to 36, minimum note 5 frames (58.0 ms), melodia on
decoded: 16 notes
octave check: 16 notes, 0 moved down an octave, 0 flagged, 0 unchecked
ghost merge: 16 notes
attack split: 16 notes
split join: 16 notes
quiet-note guard: 16 notes, 0 dropped below 40 dB
glides: 0, 0 too short
pitch drops: 0 of 15 low strikes
drop skips: 1 no onset, 15 no fall
vibrato: 0 of 0 steady spans
envelope: 16 of 16 notes read, 16 pooled, 16 with a note-off knee
envelope: attack 19 ms (upper bound), hold 0 ms, decay none, sustain 0.0 dB
tail: release none, rt60 none, weight 0.01, 1 of 5 reverb votes
pump: too few sustained beats
voices: mono, up to 1, mean 1.00; 15 joins, 0 legato
wrote bass/layers.json
wrote bass/spectrogram.png
wrote bass/notes.mid

The first line is the tempo: --bpm refined from the stem’s onsets, as notes does it; --bpm-exact keeps --bpm as it is. This 4 s stem is too short to refine it. The next lines are the notes step of notes, which analyze runs once to know the stem’s notes; --profile overrides its settings as there. The four lines after them are the pitch layer: this saw repeats one note with no glide or vibrato, and none of its 15 low strikes falls in pitch. Each line lists what it found, with its numbers; drop skips counts the low notes with no drop by the gate that stopped each: no onset, settling, quiet, no strike, weak fundamental, no fall, tau on bound, residual or off note. The next four lines are the envelope layer. The first counts the notes read, those pooled and those with a note-off knee, and any skipped. The second gives the pooled medians: this saw starts each note at full level, under 1.5 periods of 55 Hz, so the attack is an upper bound; it holds its level, so it has no decay and a sustain of 0 dB. The third gives the release, the RT60 and weight of any tail, and how many of five signs say the tail is reverb. The last is the pump: no beat here lies inside one held note from 50 ms before it, so none counts. The last line before the files is the voices layer: one voice at a time, and each of the 15 joins falls to silence, so none reads legato. The files go into a folder named after the stem, next to it. notes.mid is the final note list, below; it replaces the first file of notes. --out picks another folder. A warning about a packet that did not decode goes to standard error; the stem keeps its timing, with silence in its place.

The final notes.mid

analyze rewrites the note list with the layers and writes it as notes.mid, one track on channel 1 at the used tempo, 480 ticks per quarter note. The preset carries what the routes model, so the file holds only notes.

Every rule below reads one voicing (owner decision, 2026-11-05):

Both lines end the other notes at their knees or 50 ms before their list ends. The voicing of the notes step, from its chord share, sets only the notes step’s settings and the pitch layer. So a stem of chords whose chord share stays under 0.68 is still a polyphonic line here. Given notes take no profile, so --notes leaves the voicing to the voices layer. A layer that counts no voices leaves the line mono, because the mono rules only join and cut notes.

• Each note ends at its note-off knee, so the release lives in the amp envelope and a render does not play it twice. A note without a knee, such as a pluck that decays at one rate, ends 50 ms before its end in the list; so does a note the envelope layer could not read, marked unread.
• A note of a polyphonic line ends where its level falls 20 dB below its plateau, marked pad: the plateau of its one harmonic, or of the full band when no harmonic of it is free, as in an octave stack. The search stops at the list end: the note-off is the gate, and the release past it belongs to the preset’s amp envelope (owner decision, 2026-11-05). A note whose level falls short of 20 dB by then ends by the other rules.
• No note ends after its end in the list, except at a glide join. A knee, no-knee or pad cut never makes a note shorter than the minimum the BPM rule of the decoder gives at the tempo, 58 to 128 ms. The decoder drops notes that short on mono lines, and on polyphonic and pad lines it drops notes of 128 ms or less, whatever the tempo; the cleanup and a given --notes file can still hold shorter ones. A note the list gives shorter keeps its length. In a mono line a note ends where the next starts, even when that is shorter than the minimum.
• A glide becomes two overlapping notes on its two pitches. The later note starts at the glide’s start, or at its own onset when the glide starts after it, and the earlier one ends 5 ms later, rounded up to whole ticks. In a mono line no other notes overlap. Of notes that start on one tick, one stays and the report counts the others: a note the transcriber does not flag as an octave guess before one it flags, then the longer, then the lower key. Their levels cannot rank them: both read the same mix from the same start.
• A pitch drop becomes one note at its settled pitch, and a note under a vibrato takes the key of the vibrato’s mean pitch. Only the glides, drops and vibratos the report trusts above low count.
• Velocity follows the full-band level of each note when the spread of those levels exceeds 3 dB: the loudest note gets 127, and a note d dB under it 127 * 10^(-d / 40), at least 1. Otherwise every note gets 100. The full-band level is the peak of the one-period RMS of the whole mix from the onset to the note-off knee, so two notes compare even when they read different harmonics. On a chord it is the level of the mix, and tones of different levels do not tell apart. Each tone reads the mix with its own one-period window: the tones of a C3, E3 and G3 chord read within 0.65 dB of each other, and in an octave stack the lower tones read up to about 3 dB less. A stagger of the onsets moves no tone’s level by 1 dB [synthetic]. A note with no frame to read reads its written span, which on a mono line stops at the next later onset; with no frame there either it gets 100.
• Pitch motion that no route models becomes pitch bends: a note of a mono line whose pitch stays 50 cents or more off its key, on one side, for at least the pitch track’s reach (a quarter of its frame plus one period, 26 ms at 392 Hz to 49 ms at 39 Hz), outside every glide, drop and vibrato. A shorter excursion fits inside what one track frame reads, so jitter on a low note or a hit inside a merged note writes no bend (owner decision, 2026-11-05). The pitch track reads a little of the note before at each note change, and while two notes sound it follows only one of them, so those frames count for no bend. Bends are clamped to 2 semitones, so set the synth’s bend range to 2 semitones. render reads no bend.

The notes group of layers.json lists each written note with the rule that set its end, its level and the transcriber’s octave flag.

Limits

• A bend of a semitone or more becomes its own note at the transcriber, so it plays as a note change, not a bend. Of generated tones with a held bend, every +100 and +200 cent bend (120 of 120) split into its own note; of the +60 cent bends, 41 of 60 wrote bends, 13 split, 5 stayed flat and 1 had no note [synthetic].
• A slow ramp that passes 50 cents only in its last frame writes no bend, because the offset never persists for the track’s reach. Of the four generated bends of the tally set, two lost their bends this way: a rise of 0.5 semitones over 1.5 s, whose note reads +5 to +55 cents, and a 2-semitone ramp over 1 s that the transcriber split into notes 57, 58 and 59, where note 58 reads -35 to +55 cents [synthetic].
• Of the tally’s signals with motion that no route models, 7 runs lost bends to the persistence rule: 2 bends, 2 trills and 3 kick drops [synthetic]. The rule also took bends from 51 runs of glides, drops and vibratos that the routes model.
• Steady lines still bend in 27 of 502 generated runs without pitch motion, 3,194 events, against 156 runs and 28,807 events without the persistence rule [synthetic].
• A pad note whose level falls 20 dB only after its list end keeps the knee, no-knee or minimum end. Against the gate note-offs of the generated tally’s 1,776 transcribed notes, a search past the list end raised the error from 239 to 346 ms MAE, and gave 177 ends more than 1 s late [synthetic].
• The transcriber’s notes of a mono line can overlap, so the voices layer reads the line as polyphonic and its notes get pad ends but no glide joins or bends: 41 of 133 transcribed runs of generated 808 and bass lines in one tally set [synthetic].
• The pitch layer still reads the notes step’s voicing, from its chord share, not the voices layer. On a stem of chords whose chord share stays under 0.68 it reads glides, drops and vibratos as on one line, while notes.mid treats the line as polyphonic and writes no glide join or bend. #327 aligns the two.

Give the notes

When the MIDI file that played the stem exists, pass it:

sound-to-patch analyze bass.wav --bpm 120 --notes bass.mid

The file’s notes replace the notes step, and the transcription model does not run. The f0 track still reads the stem for the pitch layer, unless the notes play chords. The notes set the frame length: at 32 to 48 kHz, 8192 samples for a bass below 100 Hz and 4096 above, scaled with the rate at other rates, so 96 kHz takes 16384 and 8192. They also let the report fill centroid_over_f0 and the stereo phase ipd_deg. The file’s tempo must match --bpm or the refined tempo, so a file that notes wrote at the refined tempo reads too. Notes on MIDI channel 10, the drum channel, are left out, because their keys name drums, not pitches.

Read the report

These excerpts come from the run above: a generated 4 s stereo tone at 48 kHz, a band-limited 55 Hz sawtooth at amplitude 0.25 that sounds for 0.2 s of every 0.25 s, with the right channel at 0.8 of the left:

{
  "schema": "layers/1",
  "source": "bass.wav",
  "source_sha256": "<64 hex digits>",
  "synth": "vital",
  ...
}

source names the stem, and source_sha256 is the checksum of its file. A second stem that would write into the same folder, such as bass.aiff next to bass.wav, or the bass.wav of another song in a shared --out folder, stops with an error unless you pass --out or --force. A new export of the same bass.wav has other bytes, so it stops the same way, and --force replaces its report. A folder that holds a spectrogram.png with no report, or a layers.json this tool did not write, stops the run too.

"centroid_hz": {
  "median": 3596.5198523563367,
  "iqr": 22.42862967814881,
  "confidence": {
    "level": "high",
    "reasons": [{"code": "active_frames", "value": 94}],
    "spread": 22.42862967814881,
    "estimate": true
  }
}

A statistic over frames gives the median and the interquartile range of the frames with sound, and its confidence names the evidence: here 94 active frames of 8192 samples, the frame length that a note list with a median f0 below 100 Hz picks at 48 kHz. A clip with fewer than 16 active frames gets no value and the reason too_few_frames.

"band_edge_hz": {
  "value": 24000.0,
  "confidence": {
    "level": "low",
    "reasons": [
      {"code": "active_frames", "value": 94},
      {"code": "band_edge_drop", "value": 2.6500554517088695},
      {"code": "no_band_edge", "value": true}
    ],
    "spread": 5.859375,
    "estimate": true
  }
}

The sawtooth has harmonics up to the Nyquist frequency and no steep cut: band_edge_drop says the steepest fall was under 3 dB. No bin rises the fallback margin above the floor either, so the band edge is the Nyquist frequency at low confidence, with the reason no_band_edge.

The conventions group says how each number was measured, under the same key as the number.

Read the spectrogram

spectrogram.png shows the busiest 10 s of the stem in two panels: the whole range from 20 Hz to 20 kHz, and 20 to 500 Hz with finer frequency steps for the bass. Bright is loud; the colour bar spans 80 dB below the loudest point of the stem. The image shows shape; the numbers are in the report.

Transcribing a stem

sound-to-patch notes reads one stem and writes its notes as a MIDI file, notes.mid, with the tempo you give. It finds the pitch range and whether the part plays one note at a time or chords, decodes the notes, and cleans them up. The CLI reference lists every flag, and the pipeline page gives each step with its settings and limits.

Run it

Export one instrument as a WAV, AIFF, FLAC, MP3 or AAC file and pass its tempo:

sound-to-patch notes bass.wav --bpm 120

detected range: MIDI 30 to 48, from the f0 track
detected voicing: monophonic, chord share 0.12
register window: 20 dB
cleanup voicing: monophonic, as detected
settings: detected, MIDI 30 to 48, minimum note 5 frames (58.0 ms), melodia on
decoded: 24 notes
octave check: 24 notes, 4 moved down an octave, 4 flagged, 0 unchecked
ghost merge: 20 notes
attack split: 20 notes
split join: 20 notes
quiet-note guard: 20 notes, 0 dropped below 40 dB
wrote bass/notes.mid

This run read a generated bass line, a mono 16-bit WAV at 44.1 kHz, 4.5 s long. It holds 16 notes on the eighth notes of 120 BPM from 0.5 s, each 0.2 s long with no fade: MIDI 33, 36, 40, 43, 45, 43, 40 and 36, twice. Each note is a band-limited sawtooth, the sum of $\sin (2\pi h{f}_{0}t)/h$ for every harmonic $h$ below 22.05 kHz, at amplitude 0.3. The file holds all 16 notes at the right pitch, each within 21 ms of its onset. The 4 extra notes are short fragments of the attack, 70 to 106 ms before a true note of the same key.

The file goes into a folder named after the stem, next to it, the same folder analyze writes into. --out picks another folder. Only notes.mid is replaced; nothing else in the folder changes. The file names its stem in its track name, so a run of another stem into the same folder stops unless you pass --out or --force. After analyze, the folder’s layers.json describes the final notes.mid, so notes stops there too: --out writes the first list elsewhere, and --force replaces the final notes and removes the report that described them.

Each line names one step:

• detected range and detected voicing: the pitch range the stem plays in and whether it plays chords, read from the stem itself.
• register window: the f0 frames that set the range lie within this many decibels of the stem’s loud passages, so quiet bleed does not widen it.
• cleanup voicing: whether the cleanup treats the part as one note at a time, which runs the octave check, or as chords, and whether that came from detection or from --profile.
• settings: what the decoder used, the detected settings or those of --profile, with the shortest note it keeps.
• decoded to quiet-note guard: the note count after each cleanup step. moved down an octave counts octave copies above low notes, flagged the notes whose pitch the f0 track disputes, and the guard drops notes that sound only far below the stem’s loudest passage.

Tempo

--bpm sets the tempo of the file and the shortest note the decoder keeps: half a sixteenth note, between 58 ms and 127.7 ms. A tempo from a file name is often close but not exact, so the command refines it from the stem’s onsets, from 2.91 % below to 3.09 % above it, and prints the result first:

tempo: 119.62 BPM, refined from --bpm 120 (peak ratio 8.05)

This line comes from a generated 16 s bass line at 119.62 BPM, eighth notes of A1, C2, E2 and G1, passed as --bpm 120. The file then plays at 119.62 BPM. Set your DAW project to the printed tempo, or the notes drift off the grid over the song. When the onsets do not settle the tempo, as in a pad that changes chord once a bar, the line says why and the file keeps --bpm. --bpm-exact uses --bpm as it is. The tempo section gives the method.

Profiles

Detection fits most stems. When it misreads one, --profile sets the range and the minimum length for a kind of stem:

Detection still runs and prints its findings, so you can compare them with the profile. notes decides voicing once for the whole stem. A stem that mixes a chord part and a line gets a chord share between the two and can read as one note at a time. The cleanup then runs the octave check on its chords. When detected voicing reads monophonic on a stem with held chords in it, pass --profile pad. Three more cases need a profile: bleed from another instrument louder than -20 dB in a passage where the part rests, a line with held double stops (--profile pad), and a strummed part (--profile lead), where the pad settings lose notes.

Check the file

Open notes.mid in a piano roll at the same tempo. It holds one track on MIDI channel 1 with a tempo event and 4/4 at the start; every note has velocity 100. Then check what the model cannot get right:

• No note below A0 (27.5 Hz): the model’s lowest key.

• Note ends come late: a note often lasts until the next one starts.

• Octave errors at 110 Hz and below. The cleanup moves a weak octave copy that sounds inside its true note, but a lone note an octave off stays and is only flagged. A chord part keeps its octave copies: three generated triads give 10 notes for 9.

• A repeated note can merge into the one before it when the model misses its onset and the stem has no clear attack before it. A clear attack is a fall of 19 dB or more and a rise back about as fast as a step. Over a steady floor the fall must reach 20 dB below the note. The note before it must also have held for 60 ms and for 1.5 times the gap. These repeats merge this way:

  • a legato repeat;
  • an attack slower than about 10 ms;
  • a repeat after a note shorter than about 60 ms;
  • a gap longer than about two thirds of the note before it, such as staccato eighths at half length;
  • a gap that a tail or reverb fills.

  Most natural attacks are slower than 10 ms: a bowed, blown or sung re-attack rises like a tremolo, and the cleanup reads only the level. The model can read a low repeat with a short gap as part of one note: a 55 Hz saw gated with 30 ms gaps decodes as 9 notes for 16. From A1 (55 Hz) up the cleanup splits such a note at a clear attack inside it, and the saw gives its 16. Below A1 the note stays whole, because there a fast sidechain duck looks like a gate.

• A square LFO that cuts the level for 20 to 40 % of each cycle looks, by level, like a gated note with short gaps. The cleanup reads it as a gate and can split a held note into repeats: an A1 under a 6 Hz square off for 30 % of each cycle gives 24 notes, one a pulse, where the model’s onsets alone give 6. A vibrato under such a square splits the same way. Only a cue such as a phase reset at each repeat could tell the two apart. A square off half of each cycle holds only as long as it dips. No generated one split a held note further than the model’s onsets do, and no sine tremolo of 2 to 20 Hz did.

• A held low note under a louder gated note of another key can split at that note’s gates when the model splits it there: an A1 20 dB under a gated A2 gives 12 notes. Below A2 the cleanup cannot tell the two keys apart by their harmonics.

Plan a manual pass in the piano roll. The pipeline page gives the scores on generated parts; real stems score lower.

When it fails

A stem that gives no note writes nothing and exits with code 1. The message names the stem and suggests a profile. Only digital silence, every sample exactly 0, is sure to give no note: the guard gate is relative to the loudest passage, so a file of DC alone or of faint hiss gives notes. A folder that holds the report of another stem, or of a stem of the same name with other content, stops the run unless you pass --out or --force. A failed write leaves the folder as it was.

Presets

The preset commands read a preset file and write a new one from an edit list. An edit list names controls and their values, such as a filter cutoff of 400 Hz. This page shows the steps for Vital. The CLI reference lists every flag, and the format page states what is known about the file.

The commands on this page run in the test suite, so their output is the output you get. They read bass.json and typo.json from preset.in/.

Vital

Choose the synth

--synth vital picks Vital. You can leave it out: the extension of the preset file picks the synth, and Vital is the default.

Write an edit list

A Vital preset holds about 800 controls. A modulation slot joins a source, such as an envelope, to a control it moves. Vital has 64 slots. This edit list turns on filter 1, picks its ladder model, sets the cutoff and the attack, and lets envelope 2 open the filter:

{
  "synth": "vital",
  "edits": [
    { "param": "filter_1_on", "value": true, "reason": "send the oscillators through filter 1" },
    { "menu": "filter_1_model", "choice": "Ladder", "reason": "a warm ladder low-pass" },
    { "param": "filter_1_cutoff", "value": "400 Hz", "reason": "a dark bass" },
    { "param": "env_1_attack", "value": "5 ms", "reason": "a quick attack" },
    {
      "route": { "source": "env_2", "dest": "filter_1_cutoff", "amount": "+24 st" },
      "reason": "envelope 2 opens the filter on each note"
    }
  ]
}

Each value carries its unit. A route amount is in semitones when its destination is a pitch or a cutoff, and in percent otherwise. Every edit needs a reason.

Apply it

$ sound-to-patch preset apply bass.json -o Bass.vital
wrote Bass.vital ("Bass") from built-in default
  build 1.0.6, kept from the template
  set filter_1_on = on, stored 1.0
  set filter_1_model = Ladder, stored 2.0
  set filter_1_cutoff = 400 Hz, stored 67.34995771509334
  set env_1_attack = 0.005 s, stored 0.26591479484724945
  set slot 1 = env_2 -> filter_1_cutoff, amount 0.1875

The command starts from the built-in template: every control at its default, our own sawtooth wavetables and an empty modulation matrix. A wavetable is a set of single-cycle waveforms that an oscillator plays. The built-in template is not Vital’s Init preset.

The preset takes its name from the output file, so name the file after the preset. Vital also shows the file name. The command refuses an existing output file unless you pass --force. It never writes over its edit list or its template, and it refuses a path inside Vital’s own Factory folder.

A wrong unit stops the command before it writes anything. The message names the edit list, the edit, the cause and the fix:

$ sound-to-patch preset apply typo.json -o Typo.vital
? failed
error: typo.json: the edit list has 1 problem(s):
  edit 1 (env_1_attack): env_1_attack: cannot read "5 Hz" as Seconds: unknown suffix "Hz". Write a number followed by one of: s, ms, such as "800 ms"

Read the result

preset inspect prints the header and the controls:

$ sound-to-patch preset apply bass.json -o Bass.vital --force
...
$ sound-to-patch preset inspect Bass.vital
build         1.0.6
name          Bass
author        sound-to-patch
style         (none)
controls      772
unknown keys  0
...
  env_1_attack: 0.26591479484724945
...
  filter_1_cutoff: 67.34995771509334
...
  filter_1_model: 2.0
...
  filter_1_on: 1.0
...

The first line is the stamp, the Vital build the file claims. Vital refuses a file whose stamp is newer than the app in its first two numbers. Long lists show their first items; --full prints everything.

Start from your own preset

A preset you saved from Vital carries the stamp your app expects. Pass it with --template, and the edits apply on top of it:

$ sound-to-patch preset apply bass.json -o Bass.vital --force
...
$ sound-to-patch preset apply bass.json --template Bass.vital -o Again.vital
...

The built-in template is stamped 1.0.6, the open-source build, which Vital 1.0.7 loads. For a newer build, such as the early-access 1.6, start from a preset saved by your own app.

Load it in Vital

1. Copy the file into ~/Music/Vital/User/Presets, or use the menu item “Open External Preset”.
2. Load the preset in a fresh instance.
3. Check that a control you edited shows its new value.
4. Play a note and check that the sound differs from the default patch.

A file is untrusted until Vital has loaded it. Check the sound, not only the name: in a plugin host, Vital ignores a file it rejects and keeps the last sound without a dialog.

• Results on Vital 1.0.7, the stable build:

  Check	Result
  A preset with the cutoff and attack edits of the README quick start	loads; the attack shows 1 s and the note swells
  The same file stamped 1.5.5, 1.6.0 or 1.7.0	refused: the stamp is newer than the app
  The same file with keys of later builds	loads
  Save it again from the app	every control kept
  An unknown key, no name fields, a route from each source, a 100-point LFO shape, wavetables of 1 and 257 frames	each loads with no dialog

  The format page lists what is still open.

• The 1.0.7 macOS bundles are x86_64 only (lipo -archs, 2026-10-26). A plugin host running natively on Apple silicon cannot load the plugin. On Apple silicon, macOS runs an x86_64 app under Rosetta.

• Presets saved by Vital 1.6 do not open in 1.5.

• Factory presets, wavetables and samples cannot be shared. Do not share a preset that embeds them.

Rendering a preset

sound-to-patch render plays a MIDI file through a preset in the synth’s own plug-in and writes the audio as a WAV file. It runs on macOS with Vital. Serum 2 follows. The CLI reference lists every flag.

What it needs

• Vital, installed as a VST3 plug-in in /Library/Audio/Plug-Ins/VST3 or in the same folder under your user’s Library. --plugin names another bundle.
• sound-to-patch-host, the program that loads the plug-in. It ships in the release archive next to sound-to-patch, and render looks for it there. SOUND_TO_PATCH_HOST names another copy.
• Rosetta on Apple silicon. Vital 1.0.7 holds Intel code only, so the host runs as an Intel program for it. softwareupdate --install-rosetta installs Rosetta.

The plug-in loads as a library, with no window, and the Vital app never starts.

Run it

Pass the preset, a MIDI file with its tempo, and the WAV file to write:

sound-to-patch render bass.vital --midi notes.mid --bpm 120 -o bass.wav

plug-in: Vital 1.0.7 from /Library/Audio/Plug-Ins/VST3/Vital.vst3, in sound-to-patch-host 0.18.0 (vst3-host 0.10.0) as x86_64
preset: routes, 2x oversampling
notes: 3, the last note-off at 3.000 s, 120 BPM
init patch: 2 renders, 0.04 dB apart
load check: passed; the saved state matches, and the render lies 58.20 dB from the init patch, beyond the margin of 0.11 dB
clip: 5.589 s, the tail went quiet
wrote bass.wav
wrote bass.json

This is one local run: it played the routes.vital test fixture, the built-in template with three routes, through Vital 1.0.7. The MIDI file holds MIDI 48, 55 and 60 at velocity 100, one second each, back to back. The dB figures change a little from run to run.

Each line names one step:

• plug-in: the synth, its build, the bundle and the host, with the architecture the host runs as.
• preset: the preset’s name and the oversampling of the render. Every Vital render uses 2x oversampling, whatever the preset holds, because oversampling changes the sound.
• notes: how many notes the MIDI file holds, when the last one ends and the tempo. The file must be one that analyze --notes reads: every tempo in it within 0.01 BPM of --bpm, and the notes off channel 10.
• init patch: the plug-in renders the same notes twice with its own init patch before the preset loads. The two renders differ a little: Vital’s oscillators randomize their start phase fully by default (osc_1_random_phase defaults to 1 in the registry).
• load check: see below.
• clip: the length of the WAV file. A clip is at least 4 s long. It ends once the output has stayed below -96 dBFS for 1 s after the last note-off. A tail still sounding 30 s after the last note-off is cut there, and the line says so.

The load check

A plug-in can report that it loaded a preset and still play its old sound. So a render counts only when two checks pass:

1. The clip sounds. A clip whose every sample stays below -96 dBFS is silent, and the error names what in the preset can explain it, such as no oscillator and no sample being on.
2. The state the plug-in saves after the render holds the preset: its name, its tuning and every numeric control, each number within one step of a 32-bit float of the sent value. The tempo control is the exception: the plug-in sets it from --bpm, so it must lie within one step of the tempo Vital computes from --bpm, in beats per second. A preset without a tuning plays in Vital’s default tuning, never in the tuning of the preset rendered before it. When the saved state is the init patch, the plug-in did not load the preset at all, as with a preset saved by a newer build of Vital.

Each failure has its own message, and the command writes no file.

A third check only warns. Each clip is measured as its level in octave bands, and the distance from the init render is the largest band difference in dB. Only the bands within 30 dB of the loudest band count, and only those above a floor of -90 dB of mean bin power. That floor is not a level in dBFS: a band of many bins reads lower than a band of few for one tone, so the floor passes a sine above about -50 dBFS in the top band and above about -74 dBFS in the lowest band at 44.1 kHz. A clip within three times the distance between the two init renders, and within at least 0.1 dB, sounds like the init patch. The render still counts, and render prints a warning. Mean band levels miss a change of envelope or modulation: a slower attack on the init patch can lie within the margin. The margin is a stopgap. Two init renders give no distribution, and a later release measures the spread over many renders.

The output

The WAV file holds 32-bit float stereo at 44.1 kHz. The record next to it, bass.json here, names everything the audio depends on: the synth and its build, the host, the recipe version, the sample rate and block size, the settle time, the oversampling, the tempo, the clip length, the SHA-256 of the MIDI file and the load check. The render record page lists every field.

Pass the WAV file and the MIDI file to analyze to measure the render as you would measure a stem:

sound-to-patch analyze bass.wav --bpm 120 --notes notes.mid

Two renders of one preset are not identical, so compare renders within their spread, never sample by sample.

Using the synth reference

The synth reference is an agent plugin, sound-to-patch-reference. An agent reads it when it writes an edit list for sound-to-patch preset apply, writes notes from a producer’s words, or explains a preset. It lists the controls of each target synth by layer, with the on-screen label, the preset key, the unit, the range and how to tune each control by ear. It also holds the music dictionary and the word dictionary: what words such as warm or plucky mean, and how far listeners agree on them.

Only Vital is covered today. Serum 2 follows with its adapter.

Install it

Add the repository as a marketplace, then install the plugin:

claude plugin marketplace add Akurganow/sound-to-patch
claude plugin install sound-to-patch-reference@sound-to-patch

Each release also ships the plugin as sound-to-patch-reference-vX.Y.Z.zip. Pass the unpacked folder or the zip to a session with --plugin-dir.

A session opened on a clone of this repository needs no install: it finds the same skill under .claude/skills/.

On Windows, let git create symbolic links before you install from GitHub or clone the repository. Enable Developer Mode (or run the shell elevated). Then run git config --global core.symlinks true, or clone with git clone -c core.symlinks=true …. Without symbolic links the plugin installs with no reference, and a clone of the repository loads no skill. The agent reference page lists which install paths carry the reference.

Use it

Ask the agent for the sound in your own words. The skill loads when the request needs a control, a note rule or a sound word, so the agent names each control by its on-screen label and checks its edit list with sound-to-patch preset apply before it hands you the preset.

Vital and Serum 2 are trademarks of their owners. This project is not affiliated with or endorsed by them.

Analysis pipeline

sound-to-patch analyze turns a stem into layers.json, spectrogram.png and the final notes.mid, and sound-to-patch notes turns it into the first note list, a notes.mid of its own. This page follows the stem through each stage, with the formula, the convention and the limits of each. The common methods page holds the rules that every stage shares: the window, the frame rule, active frames and the statistics.

Every statement carries a tag:

• [fact]: the code does this, or a cited work states it.
• [synthetic]: a figure measured on generated signals. Generated signals are a best case: a real stem adds bleed, effects and noise.
• [proposal]: a threshold or rule chosen for this project that later work may tune.

analyze runs decoding, the tempo, the notes step (unless --notes gives the notes), the pitch layer, the envelope layer, the voices layer, the stem, the stereo screen, the descriptors and the report today [fact]. With --notes the f0 track still reads the stem, for the pitch layer [fact]. notes runs decoding, the tempo and the notes step [fact]. The analytic signal, the masks, the modulation spectrum, the periodogram and the decay fits are library stages that the layer estimators of later releases call [fact].

Decoding

• The stem decodes at its native sample rate with every channel kept, as f64 samples [fact]. The default audio track is the one the container flags as default, else the first with a known codec [fact].
• The rate must lie between 8000 and 192000 Hz, or the file is refused with its rate and the range [fact]. The band-edge rule looks for its fall above 2 kHz, so it needs a Nyquist frequency well above that [fact]. 192 kHz is the highest standard export rate [fact].
• The cost of the analysis grows with the sample rate and with the number of notes, so a stem of many short notes costs far more than a sustained one [synthetic]. Examples from a release build on generated 4 s mono stems of a 55 Hz band-limited sawtooth [synthetic]:
  • Sustained: 0.08 s per second of audio at 48 kHz and 0.17 s at 192 kHz.
  • Notes of 0.2 s, four per second: 0.28 s per second of audio at 48 kHz and 1.6 s at 192 kHz. With the bound lifted, the same stem cost 1.8 s at 200 kHz, 2.5 s at 256 kHz and 5.1 s at 384 kHz.
• The build reads WAV, AIFF, FLAC, MP3, AAC (ADTS), MP4 or M4A, Ogg, MKV and WebM containers [fact]. A missing codec fails with its name and, if one exists, the decoder feature that adds it [fact].
• A packet that fails to decode becomes silence of its duration, so later samples keep their time, and the command prints a warning [fact]. More than 4 such packets fail the file [proposal].
• Mono is the channel mean, $m=(L+R)/2$, with no gain normalisation [fact]. A file of three or more channels mixes only the first two, as the stereo measures read them, and the command warns with the channel count [fact]. Analysis never resamples [fact].
• Resampling serves only a consumer with a fixed input rate, such as the 22050 Hz transcription model [fact]. It is a band-limited FFT conversion with $\lceil n\cdot f_{to}/f_{from}\rceil$ output samples [fact]. Some rate pairs, among them 48 kHz to 22050 Hz, lag half an output sample, 23 µs; the lag is documented and not compensated [synthetic]. A NaN or infinite sample stops the conversion with its index [fact].

Tempo

--bpm is close to the stem’s tempo, not exact: a tempo taken from a file name is often 1 to 2 % off, and that drifts notes off the grid within a few bars [proposal]. Both commands refine it from the stem before any step that reads it: the minimum note length, the MIDI tempo and the beat grid [fact]. --bpm-exact skips the search and uses --bpm as given [fact].

• Onset strength: the mono mix in periodic Hann frames of 46.4 ms (2048 samples at 44.1 kHz; the power of two nearest 46.4 ms at other rates) with a hop of about 1 ms [fact]. The bins sum to power bands a third of an octave wide from 30 Hz, each at least 6 bins, so a band holds whole main lobes and a steady partial keeps its energy from frame to frame [fact]. Each band power is compressed as $\ln (1+P/P_{knee})$, with the knee 60 dB below a full-scale bin [fact]. The strength is the summed rise of every band against the frame 10 ms earlier [fact].

• The strength is low-passed at 20 Hz forward and backward, its centred mean over 100 ms is removed, and what stays below 10 % of the largest rise becomes 0, so only onsets remain [fact].

• Search: every tempo $t$ in steps of 0.01 BPM with $|t_{given}/t-1|\le 0.03$ [fact]. Each scores the unbiased autocorrelation of the strength, read by linear interpolation at $k\cdot 60/t$ seconds and summed over $k=1$ to $K$ beats, with $K=\min (32,\lfloor \text{beats}\rfloor -4)$ [fact]. The search spans 2.91 % below to 3.09 % above --bpm, so a --bpm 3 % off either way still holds the truth [fact]. The window is so narrow that half and double the tempo lie outside it [fact].

• Trust: the best tempo replaces --bpm when three things hold [proposal]:

  • Its score is at least 1.5 times the median score of the search (tempo_peak).
  • The onsets are hits, not swells: at least 0.9 of the energy of the band rise, before its low-pass, lies within 25 ms of its peaks, each the largest within 30 ms (onset_sharpness).
  • One of two shares is high enough. The beat share is the best score over $K$ times the zero-lag autocorrelation, and needs 0.4 (beat_periodicity). The sparse share reads the same sum on every second or every fourth beat lag alone, the better of the two, and needs 0.75 (sparse_periodicity), with at most 0.8 peaks of the onset strength per beat at the slowest tempo of the search (onsets_per_beat). Both shares stop at 1.

  The sparse share is read once the stem holds 4 such lags, 12 beats for every second lag and 20 for every fourth [fact]. Otherwise the tempo is reported at low confidence and --bpm stays [fact]. A stem of fewer than 8 whole beats, or one without onsets, keeps --bpm and reports no refined tempo [fact]. The beats are counted at the slowest tempo of the search, --bpm / 1.03, so the longest lag keeps 4 beats of overlap at every candidate: a 4 s stem at 120 BPM holds 8 beats, but 7.77 at 116.51 BPM, and keeps --bpm [fact]. cycles_in_span reports that count [fact].

• Generated drum, bass and lead parts at 70.3, 97.55, 123.4, 147.81 and 178.2 BPM, 37 beats each at 22,050 Hz, with --bpm off by -3, -1, +1 and +3 %: every refined tempo lies within 0.05 BPM of the truth [synthetic]. Their beat_periodicity reads 0.78 to 0.99; pad chords that change once a bar read 0.04 at most and keep --bpm [synthetic]. The 0.4 threshold sits between the two [proposal].

• One kick a bar repeats on a quarter of the beat lags, so its beat_periodicity reads 0.22 to 0.24 and its sparse_periodicity 0.95 to 0.97 at 37 beats [synthetic]. Kicks a bar and kicks on every other beat, on the same grid, refine to the exact tempo [synthetic]. A kick a bar at 120 BPM refines from 21 beats, once the share reads every fourth lag; at 20 beats it reads 0.34 [synthetic].

• The sparse share needs more than the beat share, because its fewer lags line up by chance more often [fact]. Kick parts that keep --bpm, with it 3.5 % or 4 % off or under a tempo ramp of 1 %, read at most 0.63; sparse parts read 0.8 or more, swung or with onsets moved by 8 ms [synthetic]. The 0.75 threshold sits between them [proposal].

• A dense grid lines up with any spacing that is a whole number of its steps [fact]. Sixteenth hats at 70.3 BPM with --bpm 3.5 % off read a sparse_periodicity of 0.98 at 16/15 of the truth, where 4 beats are 15 sixteenths [synthetic]. Kick-a-bar parts read 0.19 to 0.26 onsets per beat, kicks on every other beat 0.48 to 0.52, leads 1.23 or more and hats 3 or more [synthetic]. The 0.8 limit sits between them, so a lead or a hat part needs the beat share [proposal].

• A pad under a smooth tremolo repeats as well as a drum part, and reads a beat or sparse share near 1 [synthetic]. Its rises spread over 180 ms or more [fact]. A chord under a cosine tremolo of half a beat to 4 beats reads an onset_sharpness of 0.80 at most where its shares pass, and keeps --bpm [synthetic]. Chord pads read 0.76 at most; drum, hat, bass, lead and kick parts read 0.985 or more, and a bass with an 80 ms attack 0.92 [synthetic]. The 0.9 threshold sits between them [proposal].

• Only a smooth tremolo keeps --bpm: one at an eighth-note rate or slower, with a gentle shape [synthetic]. A gate reads as hits [fact]. A chord gated by a square LFO, or pumped with an 80 ms recovery as by a sidechain, every half beat to 2 beats at 117 or 123 BPM with --bpm 120, refines to the tempo of the gate [synthetic]. That is the right tempo when the gate is locked to the song [fact].

• Swing at two thirds of the beat refines within 0.05 BPM, and onsets moved by up to 8 ms at random within 0.1 BPM [synthetic]. A --bpm 3.5 % or 4 % off, and a tempo ramp of 2 % or 3 % either way, keep --bpm [synthetic].

• A kick a bar at any tempo from 40 to 180 BPM in 0.5 BPM steps, 37 to 64 beats long, with --bpm 1 % off, refines to within 0.01 BPM [synthetic]. Most candidates score exactly 0 for such a part, so the median score can be 0; the peak ratio divides by at least $10^{-9}$ of the best score, and reads at most $10^9$ [fact].

• Limits:

  • A loop whose bar is not 4 beats long but lies within 3 % of it, such as a kick every 3.9 or 4.1 beats, refines to the tempo at which the loop is 4 beats [synthetic]. The onsets cannot tell such a loop from one at that tempo [fact].
  • Silence dilutes the sparse share [fact]. A kick a bar of 37 beats keeps --bpm with 12 beats of silence after it or 24 before it [synthetic].
  • A soft attack lowers the sharpness [fact]. A bass in eighth notes with a linear attack of 100 ms keeps --bpm at 178.2 BPM, and one of 120 ms at every tempo tested [synthetic]. Another sound may reach that point sooner.

• A steady tone whose period divides the beat, such as 110 Hz at 120 BPM, finds no onset and keeps --bpm [synthetic].

• The beat offset stays 0: only the period is refined [fact]. A stem whose tempo drifts is not followed [fact].

The command prints one line for the step: the tempo used and why.

The notes step

sound-to-patch notes turns a stem into notes.mid: one model run, one f0 track, the decode settings, the decode, the cleanup and the MIDI file, in that order [fact]. analyze runs the same step once, without the MIDI file, unless --notes gives the notes, and the model then never runs [fact]. The notes set the frame length of the stem and fill centroid_over_f0 and the stereo phase [fact].

Transcription model

• The notes come from a transcription model that ships inside the binary and runs on a pure-Rust inference engine, with no native library [fact]. The model and its decoder are credited on the credits page.
• Licence of the weights [fact]. The bundled model file has no licence statement of its own. It equals the file in the upstream repository byte for byte: both have the SHA-256 2c3c1d144bfa61ad236e92e169c13535c880469a12a047d4e73451f2c059a0ec, and a test pins that hash against the bundled bytes. The upstream repository grants Apache-2.0 to every file it holds, the model included. The upstream NOTICE text ships in THIRD-PARTY-LICENSES.html with every release.
• The model reads the mono stem resampled to 22050 Hz as 32-bit floats, in 2 s windows that start every 1.64 s [fact]. It returns three activation matrices with one frame every 256 samples (11.6 ms): note, onset and fine pitch, 88 keys from A0 to C8 [fact].
• It runs once per stem, and every decode reads that one run [fact]. The thread count does not change the activations; a test compares 1 and 8 threads bit for bit [fact]. The inference engine’s output differs between processors by up to about 5e-4, so a note near a threshold can flip from one machine to another [fact].
• A NaN or infinite sample stops the run with its index before any window runs [fact]. Silence gives no note and no error from the model [fact].

Profiles

• A range becomes the decoder’s frequency bounds: the frequency of its lowest key and of the key above its highest. Every range from MIDI 21 to 108 keeps exactly its own keys [fact].
• Every profile keeps the decoder’s thresholds (onset 0.5, note 0.3, 11 quiet frames end a note), turns on the decode of notes without an onset, and turns pitch bends off [fact]. The ranges and lengths are [proposal].

Minimum note length

• With the BPM rule, the shortest kept note is half a sixteenth note, $0.5\cdot 15000/\text{BPM}$ ms, clamped to 58 to 127.7 ms and rounded to whole model frames, half to even: 5 to 11 frames [proposal]. At 120 BPM that is 5 frames, 58.0 ms [fact].
• The decoder keeps a note strictly longer than the setting: 11 frames keep notes of 12 frames, 139.3 ms, or more [fact]. A note grown without an onset needs one frame more [fact].

f0 track

• One probabilistic YIN run covers the whole stem at 22050 Hz: frames of 2048 samples, a hop of 128 (5.8 ms), pitch bins of a tenth of a semitone, from 25 Hz to 4200 Hz [fact]. The ceiling sits just above C8, the top of the transcription model: with a 400 Hz ceiling every tone above it read as a subharmonic [synthetic].
• The copy the tracker reads is scaled to a peak of 1, because the tracker treats a frame below a fixed energy as silence [fact]. A sine bank at 30.87, 41.2 and 55 Hz reads within 5 cents with no frame an octave off [synthetic]. Glides of 150 and 400 ms rise within 5 % of their true 10-90 % time; a 50 ms glide reads 41 ms, where the reference implementation reads 23 ms, because a 93 ms frame cannot resolve it [synthetic].
• The track fills each note’s f0 contour with its voiced frames, and sets the register and the octave check [fact].

Register and voicing

• Without --profile, the detection sets the decode settings from a probe decode over the model’s whole range and the f0 track [fact].
• Voicing: a part is polyphonic when 68 % or more of the probe notes share at least half their span with a note of another pitch class [proposal]. On the generated set lines read 0.56 or less, the highest a melody with reverb, and chord parts 0.79 or more, the lowest a broken chord with the sustain held [synthetic]. Chords with releases 150 ms apart, a moving inner voice under held notes and strums 30 ms between strings all read 1.00 [synthetic]. An octave-up ghost shares its note’s pitch class and does not count [fact].
• Register of a monophonic part: the 1st to 99th percentile of the voiced f0 frames within 20 dB of a reference level, plus 2 semitones on each side [proposal]. The reference is the 95th percentile of the voiced frames’ levels, so a short loud burst, ring or thump does not move it [synthetic]. A polyphonic part takes the lowest and highest probe notes instead, because the tracker reads a chord as one low common period [synthetic].
• The range of a monophonic part widens to every hold candidate outside it, so a note the tracker misreads still lands inside [proposal]. A hold candidate is a probe note whose median frame lies within 10 dB of the reference and that sounds alone for at least half its span [proposal]. A line plays one note at a time, so an octave-up ghost or bleed over the line shares its span with another pitch and does not count; a weaker note an octave away does not count against a note, since the decoder gives every note such a twin [synthetic]. In a fast run at 140 BPM that jumps from A2 to A3, the tracker reads 43 to 52 and the A3 widens the range to 41 to 59 [synthetic].
• A monophonic part takes the minimum note length from the BPM; a polyphonic part keeps the decoder’s 127.7 ms [proposal].
• Limit: bleed within 20 dB of the stem’s loud frames, in a passage where the part rests, enters the register window, so the range widens to its notes [fact]. Pass --profile for such a stem.
• Limit: a line with an occasional held double stop reads as monophonic, and the upper note of the stop can fall outside the range [synthetic]. Pass --profile pad for such a stem.

Cleanup

The cleanup runs five steps and prints the note count after each [fact].

• Octave check, on a part played one note at a time: each note meets the median pitch of the voiced f0 frames inside it, given 3 or more [proposal]. Within 1 semitone the note is fine. A note 12 semitones above the median, within half a semitone, over a median no more than half a semitone above A2 (110 Hz), is a copy when a note of the lower key holds it within 2 model frames at each end and has a higher velocity [proposal]. The copy moves onto that note, its key and span [fact]. Any other gap keeps the pitch and sets the note’s octave flag [fact]. A chord part skips the check, because the tracker reads a chord as one low common period [synthetic].
• Why the velocity: the decoder gives every note of an octave bass a twin an octave away with nearly its span, and the tracker can hold the lower octave through a true upper note [synthetic]. On a bass that alternates A2 and A3 in sixteenths, true notes decode at velocity 81 to 100 and their twins at 34 to 52 [synthetic]. The velocity of a decoded note is its mean activation [fact].
• Ghost merge: notes of one key that overlap become one note, from the first onset to the last offset [fact]. A moved copy has its lower note’s span, so it merges into that note alone and never bridges two repeats [fact]. The decoder gives no two notes of one key that overlap, so a decoded repeat never merges [fact]: on every generated part the merge removes exactly the moved copies [synthetic].
• Attack split: a note of A1 (55 Hz) or above splits where the stem has an attack of its own inside it [proposal]. A 25 ms frame of the note marks a dip when it is no louder than the frame before it, quieter than the frame after it, and 19 dB or more below the loudest frame in the 50 ms after it. The note must hold a frame at six tenths of that loudest frame or more, centred 60 ms or more before the dip. The split falls on the last envelope reading at a tenth of the loudest reading after the dip, before the first reading at nine tenths after the quietest of the dip frame’s 26 readings, 1 ms apart, read as in the rise test below. The note before the dip can read louder than the one after it, so the climb starts at the quietest reading, and a deeper gap later in the 50 ms belongs to a dip of its own, so it cannot move the cut past a repeat [fact]. At the same times the level at the note’s fundamental alone, one DFT bin over one period, must reach 23 dB above its quietest reading before the split in a reading from the split on. The piece after the split must last 50 ms, and the attack rule below must hold between the two pieces. Each piece keeps the note’s key, octave flag and velocity [fact]. A last piece of at most 100 ms that runs into the decoder’s next note of its key, within 2 model frames, starts that note instead, unless that note has an attack of its own after the piece by the same rule [fact].
• Why the split: the decoder can read a low repeat with a short gap as part of one long note, and no later step sees two notes [synthetic]. With 30 ms gaps the gated 55 Hz saw decodes as 9 notes for 16, or 8 with a 5 ms tail, and the split gives all 16 in both [synthetic].
• Why the loud frame before the dip: the decoder can start a note in the gap before its own attack, and a split there would cut off a piece of silence [proposal].
• Why the fundamental: below about A2 the harmonics’ window of the attack rule cannot tell a note from a key nearby, so a held note under a louder gated voice of another key dips at its harmonics as well as in the stem [fact]. A bin over one period has its zeros on every harmonic of the note when the period is a whole number of samples, so a tone an octave up reads nothing there [fact]. The period is rounded to whole samples: at 22,050 Hz and at the worst phase a harmonic leaks about -70 dB at A1 and -46 to -50 dB from A2 to A5 [fact]. The bin rejects only the note’s harmonics: a key 1 to 3 semitones up reads within 1 dB of the note, and a fifth up 2 to 6 dB under it [fact]. A held A1 20 dB under an A2 gated for 0.2 s in every 0.25 s gave 16 notes where the split join alone keeps 12; with the fundamental’s dip it keeps the 12 [synthetic]. Held A1s 6 to 30 dB under a gated D2, C2, E2, A2 or A3 give the split join’s notes in every case [synthetic]. Held A1s 6 to 30 dB under a gated key 1 to 3 semitones above or below, F#1 to C2, give main’s note counts too, though the bin reads such a key nearly as loud as the note [synthetic]. There the decoder already cuts the A1 at the gates or drops it, so no dip falls inside an A1 note: where the A1 decodes across a gate, 6 to 14 dB down, the stem dips there by about that much, short of 19 dB [synthetic]. No generated semitone case reaches the fundamental test [synthetic].
• Why 23 dB: from 21 to 25 dB the generated sets give the same notes, and 23 dB lies in the middle [synthetic]. At 19 and 20 dB squares loud for 60 to 80 % of the cycle split a held note further in 72 runs where 23 dB gives 52, and at 30 dB gated runs hit 8 fewer gates [synthetic].
• Why the next note starts at the cut: on a real vocal stem the decoder started the next note of a key 58 ms after its attack, and the cut there left a fragment of 58 ms before it [fact].
• Why 100 ms: the real fragment lasted 58 ms, and a piece much longer than the decoder’s lag is more likely a note of its own [proposal]. A2 gates to 200 ms, then a repeat after a gap of 10 to 20 ms, which the decoder starts 5 ms early: the attack rule finds no attack there, and without the limit the 200 ms piece would start the repeat [synthetic]. The limit leaves the same loss for a shorter gate, as the limits below give [synthetic].
• Why the 50 ms after the split: a decoded note that starts late in one gated repeat runs into the next, and a split at that repeat left a fragment of about 17 ms before the gate closed [synthetic].
• Why A1: below it the envelope window of one period lasts 19 ms or more, as long as the recovery of a fast sidechain duck [fact]. Ducks to -40 dB split at B0 and C#1, and so do ducks to -40 dB that recover in 30 to 45 ms on every key from D1 to G#1 [synthetic]. Full-depth sine tremolos of 10 Hz split at A0 and B0 [synthetic]. At E1 such a duck and a gate with a 30 ms gap and a 5 ms tail read the same trough, 0.069 and 0.066 of the peak, and the same onset activation, 0.27 to 0.44 and 0.23 to 0.39 [synthetic]. From A1 to C4 no duck to -40 or -48 dB that recovers in 25 to 45 ms splits [synthetic].
• What the split was measured on [synthetic]: the four sets below and 16 A1 sixteenths with gaps of 35 to 44 ms, each stem decoded with the detected profile and with the bass profile, a sweep of 1053 ducked and tremolo stems, and the 37 eval parts. The sweep holds ducks to -40 and -48 dB that recover in 25 to 60 ms, linear or exponential in decibels, at three phases, on every key from D1 to A2 and on C3, E3, A3 and C4, with full-depth sine tremolos of 8 to 14 Hz from D1 to A1.
• On those sets no held run splits further than without the split, but for the squares the limits below give [synthetic]. On the first set 8 of the 716 runs come closer to the true notes, and gated runs hit 1245 gates where they hit 1205. On the second set 12 of the 130 gated runs and 10 of the 24 two-voice runs keep more repeats, 1666 gates hit where 1558 were. The fourth set keeps the same notes. Every A1 sixteenths run keeps more: 872 of 1024 gates hit where 214 were [synthetic]. On the eval the dry 110 Hz saw with a 0.3 s release keeps all 6 of its notes where it kept 5 [synthetic]. The bass clean group’s onset F1 rises by 0.002 to 0.003 under every setting and the bass recall after the cleanup from 0.955 to 0.958; no other cell moves [synthetic].
• Split join: a note that starts within 2 model frames (23 ms) of the end of a note of the same key joins it when three things hold [proposal]. Its onset activation lies below 0.68, its velocity is no higher than that note’s, and the stem gives it no attack of its own.
• The attack rule tells a repeat from a split [proposal]. It reads the stem in the guard’s 25 ms frames, one every 6.25 ms, with any DC offset removed. The quiet frames are those centred from 75 ms before the onset to the onset, but not before the earlier note starts. The loud frames are those centred from the onset to 50 ms after it, but not past the note’s end. A note has its own attack when three things hold:
  • The stem’s loudest frame lies 19 dB or more above its quietest one.
  • The level at the note’s first 8 harmonics, read with a Hann window over each frame, lies 22 dB or more above its quietest one.
  • The stem does not rise slowly. Its envelope, read every 1 ms over one period of the note and at least 4 ms, climbs from a tenth of its loudest reading after the onset to nine tenths of it. The fastest such climb, from a start no later than 20 ms after the onset, takes at most 0.8 of the window plus 7 ms, the limit. A step climbs through the window in 0.8 of it, the difference of the squares of the two shares. The readings run to 20 ms plus three limits after the onset. An envelope that never falls to a tenth climbs from no silence and fails, so a steady floor must lie 20 dB down. A climb counts only after a hold. The dip runs from the last reading at six tenths of the loudest one or more, at most 400 ms before the climb, to the first such reading after it starts. Before the dip the envelope stayed above half of the loudest reading for 60 ms and for 1.5 times the dip. An envelope that stays below six tenths for the whole 400 ms dipped too long for a tremolo, and holds. The level before the stem starts is unknown, so a dip or a hold that reaches the start is too short.
• Why the stem’s dip: under a line that moves to other keys, a split piece of the old key sees the new note’s attack in the stem, but the stem never falls far between them [synthetic]. On the generated set such pieces dip 14.6 dB or less, a sidechain duck to -16 dB 14.3 dB or less, and a gated repeat 19.6 dB or more [synthetic].
• Why the harmonics’ dip: through reverb, the old key’s tail sits under a new note of another key, and the stem then dips up to 19.8 dB [synthetic]. The tail’s own harmonics dip 18.1 dB or less there [synthetic]. Below about A2 the window cannot tell a note from a tone a fifth or a tritone away, so there the harmonics’ dip is a dip of the stem’s low band [fact].
• Why the rise test: a tremolo to silence dips as far as a gate, but rises slowly [synthetic]. A full-depth sine tremolo at r Hz climbs in about 0.3 / r s: 37 ms at 8 Hz, 21 ms at 14 Hz [fact]. A gate climbs in 0.8 of the window plus about its attack. One period reads a tone steady and keeps the dip of a tremolo at A1 [synthetic]. Two periods smooth that dip until its climb cannot be timed, and allow about 36 ms at A1 [synthetic]. A window of 1.5 periods ripples, and a gate then climbs too slowly [synthetic]. A tremolo that climbs within the limit peaks about 1.7 limits after its trough [fact]. Readings that end before that peak take a level on the climb for the loudest one, and a slow climb to it reads fast: at A1 a full-depth tremolo of 12 Hz split at some phases with readings to 50 ms [synthetic]. Three limits past 20 ms hold the peak [synthetic].
• Why the hold: below A1 one period spans 24 to 36 ms, as long as the climb of a full-depth tremolo of 14 to 20 Hz, so the climb alone cannot tell it from a gate [synthetic]. A gated note holds its level until its gap. A full-depth sine tremolo stays above half its peak for 0.47 of its cycle before it falls to six tenths, under 60 ms above 7.8 Hz [fact]. The fall of a gate ripples by a few hundredths through the window, so a hold that ended at half could end on a ripple: at E1 a repeat after a 50 ms gap read just under half its peak 1 ms before the fall and joined [synthetic]. On the generated sets 50 to 70 ms keep every repeat and split no sine tremolo [synthetic]. At 45 ms full-depth tremolos of 14 Hz on E1 split, and at 80 ms sixteenths on E1 with attacks of 19 and 23 ms join [synthetic].
• Why 1.5 times the dip: a square tremolo rises like a gate, and slower than about 9 Hz it holds through each pulse for 60 ms [synthetic]. Off half its cycle it holds as long as it dips, and a gated note holds longer than its gap [fact]. At 1.5 no square tremolo off half its cycle splits a held note on the generated sets [synthetic]. At 1.25 a 9.1 Hz square on C2 splits a held note into 10 where the onset rule gives 5. At 2 eighths on E1 with a 10 ms attack and a 10 ms tail over a -40 dB floor keep 3 of 16 repeats in the bass profile, as the onset rule does, where 1.5 keeps 12 [synthetic].
• Why 400 ms: a sine tremolo that reads as fast runs faster than about 8 Hz, and at full depth stays below six tenths of its peak for about 70 ms of each cycle at 8 Hz [fact]. A square tremolo off half its cycle stays low for 167 ms at 3 Hz, and the search must reach its fall for the ratio to weigh it [fact]. On the generated sets 200 to 500 ms give the same notes; at 150 ms an E1 under a 3 Hz square off half its cycle splits into 12 notes where the onset rule gives 7 [synthetic].
• Why the stem start counts as unknown: a stem can start in the trough of a tremolo, and its first climb then follows a dip that the search cannot see the start of [synthetic]. Read as silence, that dip held. On the third set one stem split that way: an A2 under a full-depth 14 Hz square loud for half its cycle, with 12 ms edges, that turns off 2 ms after the stem starts. It gave 4 notes where the onset rule gives 2, with the detected profile and with the bass profile [synthetic].
• Why the onset rule alone fails: the decoder can fill the gap before a low repeat and read its onset weak [synthetic]. A 55 Hz saw gated on for 0.2 s in every 0.25 s decodes as 16 notes that touch, 12 of them on onsets of 0.54 to 0.59. The onset rule alone joins them into 6, and the attack rule keeps all 16 [synthetic].
• What the rule was measured on [synthetic]: four generated sets, each stem decoded with the detected profile and with the bass profile, and the 37 eval parts.
  • The first set holds 358 stems. Held saws from A0 to A4 sit under full-depth sine tremolos of 2 to 20 Hz, at three phases up to A2, triangle tremolos, tremolos of depth 0.8 and 0.9, a vibrato of 50 cents, and sidechain ducks of -16 to -40 dB that recover in 25 to 100 ms. Gated saws at B0, E1, A1, E2 and A2 have gaps of 30 to 100 ms, tails of up to 40 ms, noise floors at -40 and -60 dB and attacks of 5 to 25 ms. Two voices a fifth and an octave apart and a DC offset complete it.
  • The second set holds 523 stems with random settings. Held saws from A0 to A2 sit under sine, triangle and square tremolos of 2 to 20 Hz, ducks of -20 to -40 dB and tremolos with vibrato. 150 more stems sweep full-depth sine tremolos of 12 to 20 Hz on A0, B0, E1, A1 and A2 at six random phases each. Gated saws and two voices a fifth or an octave apart complete it.
  • The third set holds 528 held saws at E1, A1, E2 and A2 under square tremolos of 3 to 20 Hz. Each is loud for 50 to 80 % of its cycle, with edges of 1 to 12 ms and depths of 0.8 to 1. 48 of them add a vibrato of 30 to 80 cents.
  • The fourth set holds 40 gated saws at B0, E1, A1, E2 and A2, one every 0.25 s, with gaps of 80 to 150 ms.
• On those sets no gated run keeps fewer repeats than the onset rule alone, and 45 of the 716 first-set runs, 69 of the 154 gated second-set runs and 14 of the 80 fourth-set runs keep more [synthetic]. The rule keeps every repeat the decoder gives for the gated A1 with 50 ms gaps, with 30 ms gaps, with a DC offset and with A2 above it [synthetic]. It keeps part of the rest: 11 of 14 A1 repeats under a fifth, 13 of 16 eighths with a 20 ms tail, and fewer as the attack slows past 10 ms [synthetic]. Every sine tremolo, duck, triangle tremolo and tremolo with vibrato gives the same notes as the onset rule alone, at every phase swept, and so does every square tremolo loud for half its cycle [synthetic]. Squares loud for 60 to 80 % of the cycle can split a held note further, as the limits below give [synthetic]. The eval scores match main in every cell [synthetic]. A slack of 6 to 8 ms gives the same held notes on the first set. At 9 ms a duck to -40 dB on E1 splits, and on the second set 8 ms splits one on A1; each ms less keeps fewer repeats [synthetic]. Off these sets the margins are thinner: a 55 Hz repeat under a reverb tail 20 dB down can read a harmonics’ dip near 20 dB and join [synthetic].
• A weaker piece before a stronger note is an early fragment of the attack and stays, since joining it would move the onset [synthetic].
• Limits that remain [proposal]:
  • A repeat played legato, with no drop in level, still depends on the onset margin, which is thin on both sides [synthetic]. The weakest repeat onsets on the generated set are sixteenths at 175 BPM, at 0.70 [synthetic].
  • A repeat whose gap a tail or reverb fills to within 19 dB, or to within 22 dB at its harmonics, joins the note before it [synthetic]. So does one whose attack is slower than about 7 ms plus 0.8 of the envelope window: at A2 a linear attack of 13 ms stays apart and one of 14 ms joins [synthetic].
  • The rise test passes a full-depth tremolo to silence faster than about 0.3 / (0.8 / f0 + 7 ms) Hz: 8 Hz at A0, 11 Hz at E1, 14 Hz at A1, 21 Hz at A2 and 29 Hz from B3 up, where the window stays 4 ms [fact]. The hold fails such a tremolo above about 7.8 Hz, and the window lengthens the hold a little, so at A0 a tremolo near 8 Hz can pass both tests [fact]. None of the generated ones did [synthetic].
  • A square LFO off for 20 to 40 % of each cycle is, by level, a gated repeat with short gaps: it rises like a gate and holds for 1.5 times its dip or more [fact]. To the cleanup it is a gate. On the third set squares loud for 60, 70 and 80 % of the cycle split a held note further in 38, 57 and 22 of their 240 runs: an A1 under a 6 Hz square loud for 70 % into 22 notes where the onset rule alone gives 6, and with the attack split into 24, one a pulse [synthetic]. A vibrato under such a square splits the same way, in 18 of the 96 runs, all loud for 60 or 70 % [synthetic]. The attack split cuts such a note at its pulses as well: squares loud for 60, 70 and 80 % split further than without the split in 15, 20 and 7 of their 240 runs, and a vibrato under one in 10 of 96. An A1 under an 8 Hz square loud for 70 % gives 32 notes, one a pulse, where it gave 8 [synthetic]. The model’s onset activation at those pulses, 0.25 to 0.67, matches that of gated repeats [synthetic]. Only a cue such as a phase reset at each repeat could tell the two apart [proposal].
  • Below A1 the attack split stays off, so a repeat the decoder reads as part of one note stays joined there [synthetic]. From E1 the split, measured before its fundamental test, would keep more repeats in 33 of the 242 gated runs of the second and fourth sets and the A1 sixteenths, and in 8 of the 24 two-voice runs, 238 more gates hit; it would also split the ducks above [synthetic].
  • Below about A2 the harmonics’ window cannot tell a note from a key nearby, so a held note under a louder gated voice of another key dips at its harmonics too, and the split join keeps the decoder’s pieces of it apart [fact]. A held A1 20 dB under an A2 gated for 0.2 s in every 0.25 s gives 12 notes, and under a gated E2 13 [synthetic]. The attack split adds no cut there, since the held note’s fundamental never dips [synthetic].
  • From about A6 up a period lasts 13 samples or fewer at 22,050 Hz, and at the worst phase a harmonic leaks into the fundamental’s bin at -23 dB at A6 and -17 dB at A7 [fact]. There the fundamental’s dip guards less well against a louder gated voice an octave or more up [fact]. The generated sets hold no such pair [synthetic].
  • A gated repeat of 60 to 90 ms after a gap of 10 to 20 ms, which the decoder starts 5 ms early and with a higher velocity, still loses its onset: the cut piece before it lasts 100 ms or less and starts it, so the stem gives 2 notes for 3 and the repeat starts at the cut [synthetic]. The 60 ms hold and the 25 ms dip frame hide so short a gap from the attack rule [fact]. After a 25 ms gap all but the 80 ms gate lose it too; gates of 95, 100 and 120 ms keep 3 [synthetic]. The join exists for the real 58 ms fragment above, which would stay a note of its own without it [fact].
  • A held A1 under a 6 dB amplitude LFO of one cycle per eighth note at 120 BPM gives 2 notes, as the onset rule alone does: the decoder reads a strong onset of its own at 0.19 s [synthetic]. The attack split adds no cut there, under such an LFO at 1/8 or 1/4, at A1 or E1 [synthetic]. Keeping such a note whole is #306.
  • A repeat after a note that holds above half its level for less than 60 ms joins the note before it when the decoder reads its onset weak [synthetic]. At A2 a note of 59 ms before a 30 ms gap holds long enough and one of 58 ms does not [synthetic]. Sixteenths played at three quarters of their length last 11250 / BPM ms, under 59 ms above about 190 BPM [fact].
  • A repeat whose gap lasts longer than about two thirds of the note before it joins as by the onset rule alone, since that note holds for less than 1.5 times the dip [fact]. Staccato eighths at half length, gated for 125 ms in every 250 ms, keep 3 of 16 repeats at E1 in the bass profile, as the onset rule does; with 100 ms gaps they keep 8 where it keeps 3 [synthetic]. Sixteenths pay it too: A1 sixteenths with 41 to 44 ms gaps and a 10 ms attack kept 3 to 6 of 32 repeats where a hold of 60 ms alone kept 7 or 8; the attack split now gives them 14 to 27 [synthetic].
  • Most bowed, blown and sung re-attacks rise slower than about 10 ms, and such a re-attack joins when the decoder reads its onset weak [proposal]. The rule reads only the stem’s level, and a slow re-attack climbs like a tremolo. A spectral cue such as new partials or a phase reset could tell them apart, but the generated sets change only the level, so they cannot measure it [synthetic].
  • The decoder itself can miss a repeat: with 30 ms gaps the gated 55 Hz saw decodes as 9 notes for 16, or 8 with a 5 ms tail [synthetic]. From A1 up the attack split restores them; below it nothing does [synthetic]. At 110 Hz the bass profile decodes the same signal as 29 or 30 notes, an octave copy over most of them, and the cleanup gives the 16 [synthetic].
• Quiet-note guard: a note whose span holds no 25 ms frame within 40 dB of the stem’s loudest frame is dropped [proposal]. The model scales each window by its own level, so near-silent bleed fills its range and gives notes [fact].
• The ghost: a dry 110 Hz saw (a band-limited saw at amplitude 0.3, 0.5 s notes with a 5 ms attack and a 0.3 s linear release) decodes with the bass profile as the true note plus a 70 ms copy at MIDI 57 that ends about 80 ms after the onset [synthetic]. The copy needs a minimum length below 70 ms: the 11 frames of the pad profile remove it [synthetic]. After the cleanup the stem gives the A2 alone, all 6 of its notes: the model misses the onset of the fourth, and the attack split restores it [synthetic]. Six such notes through a 1.5 s reverb give seven notes, one of them a tail fragment, and six after the cleanup [synthetic].
• Limit: a lone note an octave above the tracker’s reading is flagged, never moved [synthetic]. Only a weaker copy inside its lower note moves.
• Limit: a chord part keeps its octave copies, since it skips the check. Three generated triads decode as 10 notes for 9 [synthetic].
• Limit: the guard gate is relative to the loudest frame, so only digital silence, every sample exactly 0, gives no note. A file of DC alone or of hiss at -70 dBFS gives notes [synthetic].
• Limit: a repeated note whose onset the model misses merges into the note before it, and note ends stay late; a later release fixes ends [synthetic].

MIDI file

• Format 0, one track, 480 ticks per beat, a tempo event of $\text{round}(60\cdot 10^6/\text{BPM})$ µs per beat and 4/4 at tick 0, every note on channel 1 [fact]. Our own MIDI reader reads the file back [fact].
• Every note has velocity 100: the decoder’s velocity is a confidence, not a loudness [proposal].
• Times become ticks with $\text{round}(t\cdot \text{BPM}/60\cdot 480)$ [fact]. A note lasts at least 10 ms and one tick; a note of one key is cut at the next onset of that key, and a note left with no length is dropped [fact]. A fractional pitch rounds to the nearest key [fact].
• A list with no note is an error, so a silent stem writes nothing [fact].

Evaluation

cargo xtask eval scores the notes step on 37 generated parts with known notes: saws, sines, an 808, plucks, pads, strums, a broken chord with sustain, glides, octave basses, sixteenths at 175 BPM with repeats, a fast run with an octave jump, reverb, bleed, loud transients over a bass line, silence and an A6 saw [fact]. A note matches when its onset lies within 50 ms and its pitch within 50 cents, one-to-one [fact]. The truth of a note is its gate: the onset comes before the attack, the offset before the release, a kick’s pitch is the one it falls to, and a glide splits at its midpoint into two notes [fact]. F1off adds the offset within 20 % of the duration or 50 ms, as mir_eval does; F1o20 drops the 50 ms floor, which accepts most of a short note; F1rel scores the offset at the end of the release of the dry render, the gate plus the release [fact]. That is where the dry render ends, not where the sound ends on the three reverb parts, whose tail runs on for about 1.5 s [fact]. On the lead plucks the model’s offsets come 50 to 75 ms after the gate, against a tolerance of 50 ms, so the 20 ms release brings 13 more of the 16 notes within it: F1off reads 0.062 and F1rel 0.875 [synthetic]. Onset F1 with matched, estimated and reference notes, raw and after the cleanup [synthetic]:

Recall over each class, raw and after the cleanup, is the same for the right profile and for detection [synthetic]:

• The cleanup removes no matched note on any part [synthetic].
• Silence gives no note under any setting [synthetic]. Detection scores no worse than the right profile on every class, raw and clean [synthetic].
• The strummed part matches 19 of its 32 notes under the pad settings, and 31 under the lead settings [synthetic]. Pass --profile lead for a strummed part.
• The 808 parts score an onset F1 of 0: the pitch is right, but the onsets come 100 to 170 ms late, after the pitch drop [synthetic].
• These are generated signals, a best case. Real stems add bleed, effects and noise, and score lower [proposal].

Time cost

cargo run --release --example notes_memory -- <stage> runs one stage of the notes step on one generated 60 s stem at 22050 Hz, saw and sine notes from A2 to G4 [fact]. CPU time (user) and peak resident size of one process per stage, the median of five runs of a release build [synthetic]:

• The f0 track costs about the same CPU time as the model run, on one thread. Its time and its memory grow linearly with the stem [fact].
• The decode and the cleanup add 15 MB to the model run [synthetic].
• On a real stem of 221 s, sound-to-patch notes took 11.8 s and peaked at 0.73 GB. The dense Viterbi search that the track used before took 290 s and 4.2 GB on the same stem, with the same notes [fact].
• A debug build runs the f0 track at about half a second of CPU time per second of audio, 25 times slower than a release build [synthetic].
• cargo bench --bench notes times the same steps by wall clock. Its medians depend on a quiet machine and were not taken for the current tracker [fact].

The stem

• Mid and side are $M=(L+R)/2$ and $S=(L-R)/2$ [fact]. A one-channel file’s mid is its channel and it has no side [fact].
• The beat grid takes the period $60/\text{BPM}$ seconds from --bpm and an offset of 0. The offset is given, never estimated [fact].
• The frame length of harmonic analysis at 44.1 kHz is 8192 samples when the note list has a median f0 below 100 Hz, else 4096 [fact]. A Hann main lobe is about 4 bins wide, so harmonics $f_0$ apart need $N\ge 4f_s/f_0$ [fact]. At another rate the length is the power of two nearest, in log, to that length times $f_s/44100$, so the frame keeps its duration within a factor of $\sqrt{2}$: 32 and 48 kHz keep 4096 and 8192, 88.2 and 96 kHz take 8192 and 16384, 176.4 and 192 kHz 16384 and 32768 [fact]. The long frame meets the rule down to 30.5 Hz at every rate, and to 23.4 Hz or lower at the standard ones [fact].

Band edge

The band edge is the highest frequency that stays above the noise floor. A cutoff estimate within one octave of it is never reported [fact].

1. The long-term power spectrum of the mid is averaged over the active frames that lie wholly inside the stem, in dB [fact].
2. A running median over one semitone on each side smooths it [fact].
3. Above 2 kHz, the steepest fall of the smoothed level over one semitone is found. When it is at least 25 dB and the median level of the octave below lies within 20 dB of where the fall starts, the edge is its half-depth point, at high confidence [fact, proposal for the 25 and 20 dB].
4. Otherwise the floor is the lowest level above 1 kHz, and the edge is the highest bin at least 30 dB above it, at low confidence. With no such bin the edge is the Nyquist frequency with the reason no_band_edge [fact, proposal for the 30 dB].

The reason band_edge_drop carries the depth of the steepest fall [fact]. The plateau check of step 3 keeps a lone tone’s skirt from counting as a cut [fact].

• Synthetic cuts fall 57 to 91 dB over one semitone, and a 2-pole roll-off at 3 kHz about 1.3 dB, so the 25 dB threshold separates them with room on both sides [synthetic].
• On white, $1/\sqrt{f}$ and $1/f$ noise cut at 8 and 16 kHz at 44.1 and 48 kHz, the steepest fall erred +0.01 to +0.07 semitone and the margin rule +0.03 to +0.16 [synthetic]. With a noise shelf 60 dB down above a 16 kHz cut, the margin rule erred +3.2 semitones and the steepest fall +0.02 [synthetic]. Full-band white noise and the 2-pole roll-off have no steep fall, and both rules give the same margin value [synthetic].

Flat tops

hard_limited is true when runs of at least 1 ms of samples within 1e-5 of the channel peak cover at least 0.1 % of one of the first two channels [fact, proposal for the thresholds]. A run counts only when the samples on both sides lie at least half the peak on the same side: clipping runs into the ceiling, while a square or pulse wave jumps to it [fact]. Square and pulse waves at 55 and 220 Hz do not flag [synthetic]. The level descriptors carry the flag as a reason [fact].

Stereo screen

The screen runs on L and R with 8192-point Hann frames for every f0 [fact].

• $\rho$ is the Pearson correlation of L and R at zero lag [fact].
• ${\text{sm}}_{dB}=20{\log }_{10}(\text{rms}(S)/\text{rms}(M))$ [fact].
• Per band the screen sums $|L_k{|}^2$, $|R_k{|}^2$ and $\mathrm{Re}(L_k\overline{R_k})$ over the bins inside the band, DC left out, and forms both values from the sums [fact]. The bands are sub below 120 Hz, low_mid from 120 Hz to 1 kHz and high above [fact, proposal for the edges].
• ipd_deg needs the median f0 of a note list. At each harmonic $h{f}_0$, $h=1\dots 20$, it takes the largest cross-spectrum bin within 2 bins and the phase $|\angle (L\overline{R})|$, weighted by $|L\overline{R}|$, then the mean over active frames [fact].
• A frame is active from the RMS over both channels, $\sqrt{({\text{rms}}_L^2+{\text{rms}}_R^2)/2}$, so an anti-phase stem with no mid still counts [fact]. $L=-R$ reads 180 degrees [synthetic].
• The screen cannot see an anti-phase chorus or a light reverb, and a mono source can still get width from a reverb tail. Its confidence carries the reason stereo_blind_spot [fact].
• Detuned unison voices share one bin at harmonics 1 and 2 of an 8192-point frame, so their beat phase raises ipd_deg to 12 to 43 degrees for panned 3- and 7-voice unison; harmonics where the voices resolve read 0.2 to 4.6 degrees [synthetic].

STFT and peaks

• Frames are centred: frame $m$ covers samples $mH-N/2$ to $mH-N/2+N-1$, zero-padded past the ends [fact]. A signal of $n$ samples gives $1+\lfloor n/H\rfloor$ frames for an even $N$ [fact].
• The transform is not normalised: $X[m,k]={\sum }_{n}w[n]x[mH-N/2+n]e^{-j2\pi kn/(pN)}$, with oversampling $p$ that zero-pads each frame [fact]. Oversampling refines peak positions and changes no resolution [fact].
• A frame is computed one at a time, so a long stem never holds its whole spectrogram [fact].
• A peak is a strict local maximum, refined by a parabola through the log magnitudes $a,b,c$ of the bin and its neighbours: $\delta =\frac{1}{2}\frac{a-c}{a-2b+c}$ [fact]. Each peak also records its height over its neighbours in dB [fact].
• For one 100 Hz sine at 30 dB SNR in 8192-point frames at 44.1 kHz, 300 trials, the Hann peak has a standard deviation of 1.6 mHz against a bound of 0.7 mHz, and the rectangular window 22 mHz [synthetic]. Without noise the Hann bias stays below 0.016 bin without padding and 1.1 mHz with 4 times padding [synthetic].

Analytic signal

• The analytic signal $z=x+j\mathcal{H}\{x\}$ comes from a zero-padded FFT [fact]. It means something only for one narrowband component, so a caller band-passes around one partial first [fact]. Its memory grows with the segment, so it serves note-sized segments only [fact].
• The heterodyne multiplies by $2e^{-j2\pi f_{c}t}$, low-passes with a fourth-order Butterworth and decimates [fact]. A partial $A\cos (2\pi ft+\phi )$ comes out as $A{e}^{j(2\pi (f-f_c)t+\phi )}$, so the envelope reads $A$ [fact].
• The zero-phase low-pass runs the Butterworth forward and backward: -6 dB at the cutoff, 48 dB per octave, no delay [fact]. Each end is extended by an odd reflection of three cutoff periods, and each pass starts in the steady state of its input, so a constant or a ramp passes without a transient [fact]. Attack envelopes use a 150 Hz cutoff, because a 5 Hz smoother smears attacks shorter than tens of milliseconds [fact, proposal for 150 Hz].
• The band envelope mixes the band centre to DC, low-passes at half the bandwidth forward and backward with an even reflection at each end, and takes the magnitude [fact]. Its memory stays linear in the length [fact]. A 10 % AM at 55 Hz reads a depth of 0.1013 over the whole track [synthetic].

Note masks

• The mask of note $n$ at frequency $f$ and time $t$ is $M_n(f,t)={\sum }_{h=1}^H{e}^{-(f-h{f}_n(t){)}^2/(2{\sigma }^2)}$, with $f_n(t)$ following the note’s contour [fact].
• The defaults are $H=20$ harmonics, $\sigma =f_s/N$ (one unpadded bin) and a threshold of $e^{-2}$, the mask value two $\sigma$ from a harmonic, which covers the 4-bin Hann main lobe [fact, proposal for the values]. Harmonics at or above the Nyquist frequency are skipped [fact].
• Each bin of a frame records the note that claims it and whether a second note claims it too [fact].
• The out-of-mask share is the power $|X{|}^2$ outside every mask over all cells: bleed plus any note the list missed [fact].
• analyze --notes reads the note list from a Standard MIDI File: note-on to the next note-off of the same key and channel, first in, first out, with the times from the file’s tempo map [fact]. Notes on channel 10, the percussion channel, are left out, because their keys name drums [fact].

Modulation spectrum

• Band envelopes are tracked at 200 Hz, a 5 ms hop [fact].
• The relative modulation of a track is its natural log, floored 60 dB below the peak, minus its least-squares line [fact]. The log turns a decay in dB times an LFO into a line plus the LFO [fact]. A cubic lost a swell’s 5 Hz LFO and a quintic misread a 2-cycle 4-bar LFO, where the line kept both [synthetic].
• The spectrum is the magnitude of the Hann-windowed track, zero-padded 8 times; the resolution stays $1/T$ [fact].
• The scan runs from two semitones below $\min (0.2\text{Hz},\text{the 4-bar rate})$ to 20 Hz, so the 2-bar and 4-bar divisions have room for their peak [fact]. Above 20 Hz a modulation shows as sidebands, not in the envelope [fact].
• The rate is the strongest strict maximum in the scan, with guard points outside it as neighbours, refined by a least-squares sine fit [fact]. It needs a line 8 times the median amplitude of the scan, a depth of at least 0.5 % and two full cycles, else no rate [fact, proposal for the thresholds]. The reasons line_prominence and cycles_in_span carry both values, with a rate or without one [fact].
• On 200 seeded noise tracks the largest line stood at most 4.4 times the median [synthetic]. On a 104-case grid of periods, depths and rates the strongest line missed 0 sine, 8 saw, 4 square and 4 triangle cases [synthetic]. Noise, a steady tone and a pure decay give no rate [synthetic].
• The period search uses the cumulative mean normalised difference of the detrended track: the first lag below 0.1, else the global minimum, refined by a parabola [fact]. A 7.3 Hz modulation reads 7.30 Hz [synthetic].
• Folding averages the track over a period into 64 phase bins [fact]. The shape reads square-like with a plateau of 35 % of the cycle, a ramp up with a rise of 75 % or more, a ramp down with 25 % or less, and smooth otherwise [proposal].
• A rate snaps to the nearest note division in log distance, at the BPM and at tempos within 2 % in 0.1 % steps [fact].

Periodogram

Note gaps make a plain FFT see the note rhythm as modulation, so gated samples go to the Lomb-Scargle periodogram [fact].

• A sample counts when its level lies within 40 dB of the track’s peak, inside a note, at least 60 ms after the onset [fact, proposal for 60 ms].
• Each trial frequency fits a mean, a sine and a cosine together: the floating-mean form [fact]. On gated notes the plain form put a rate at the note rate on its harmonics [synthetic].
• The power is $A^{2}N/4$ for a sine of amplitude $A$ over $N$ samples, and a constant offset changes nothing [fact]. The line amplitude is $2\sqrt{P/N}$ of the log level, the same depth as in the modulation spectrum [fact].
• The grid steps by 0.01 Hz, and the pick and its thresholds match the modulation spectrum [fact]. The spread is the resolution $1/T$ of the span [fact].

Backward integration

• The decay curve of a tail after a stop is the Schroeder integral of the squared signal, from each sample to the end [fact]. A stem has no impulse response, so the stop comes from the note list, never from a silence [fact].
• The analysis starts at the loudest 10 ms block. The floor is the mean power of the last 10 % of the tail [fact].
• A line through the 10 ms block levels down to 10 dB above the floor gives the time where the decay meets it; the integral stops there and adds the energy the fitted decay would add after it [fact].
• The decay time is $-60/\text{slope}$ of a line fitted between -5 and -25 dB (T20) or -5 and -35 dB (T30) [fact]. A fit fails when the floor lies less than 10 dB below the bottom of its range [fact, proposal for 10 dB].

Descriptors

Descriptors run on the mid with Hann frames of the frame rule, over active frames [fact].

• Shape descriptors use magnitude weights $|X_k|$ divided by the window sum [fact]. The centroid is ${\sum }_k{f}_k|X_k|/{\sum }_k|X_k|$, the spread $\sqrt{{\sum }_k(f_k-c{)}^2|X_k|/{\sum }_k|X_k|}$, and the rolloff the lowest bin where the cumulative magnitude reaches 85 % or 95 % [fact].
• Flatness is the geometric over the arithmetic mean of the magnitude from 100 Hz to 10 kHz, each bin floored at $10^{-5}$ after the window-sum scale [fact].
• Band shares use power $|X_k{|}^2$ summed over active frames per band, divided by the power in the bands reported; the edges are 20, 60, 250 Hz, 1, 4, 10 and 20 kHz [fact, proposal for the edges]. A band that starts at or above the Nyquist frequency is left out, and one that crosses it ends there, so a 16 kHz file reports five bands, the last from 4 to 8 kHz [fact].
• Every statistic over frames, the band shares and the band edge need at least 16 active frames, else no estimate with the reason too_few_frames [fact, proposal for 16]. On noise, sawtooth and sine clips at 48 kHz the worst error of the medians against a 10 s run was 4 % at 10 frames and 2 % at 16, and many times the value at 8 or fewer [synthetic].
• Peak and RMS are of the mono mix over the whole stem in dB re full scale 1.0, and the crest is their difference [fact]. Digital silence gives no estimate with the reason silent_stem, never $-\infty$ [fact].
• The conventions of published tools differ in weights, floors and rolloff arguments, so a value is comparable only with its convention [fact]. An ideal 55 Hz sawtooth has a magnitude-weighted centroid of 3348 Hz and a power-weighted one of 220 Hz, 15 times lower [synthetic].
• centroid_over_f0 divides each frame’s centroid by the median f0 of the note list [fact].

The report

• layers.json holds the schema and synth fields, the stem group, the transcription, pitch, voices, envelope and notes groups and the conventions group; the layer report page lists them [fact].
• Output is deterministic: two runs on the same input write the same bytes [fact]. layers.json names its stem in source and the SHA-256 of the stem file in source_sha256. A run of another stem, or of a stem of the same name with other content, into the same folder stops unless forced [fact].
• Each file is written to a hidden temporary sibling and renamed into place: notes.mid first, then spectrogram.png, and layers.json last. The old layers.json moves to a hidden backup before any file changes, and each old file moves aside before its new one lands. A run without notes removes the old notes.mid. A layers.json that exists therefore always belongs to the spectrogram and the notes.mid next to it, and no file is ever cut short [fact]. A failed write renames the backups back, so the folder keeps the old files [fact]. notes keeps that true: it stops before it replaces a notes.mid that a layers.json describes, and --force removes that report once the new notes.mid is in place [fact].
• spectrogram.png is 1500 by 512 pixels, below the size at which a hosted image reader downscales [fact]. The image gives coarse shape; the numbers live in the JSON [fact].
• The top panel uses 4096-point frames from 20 Hz to 20 kHz, the bottom panel 16384-point frames from 20 to 500 Hz, both on a log-frequency axis [fact]. Each row takes the largest bin magnitude in its band, or the linear interpolation at its centre when the row is narrower than a bin [fact]. One frame is centred on each column [fact].
• The passage is the 10 s window with the most active 4096-point frames, the earliest on a tie, or the whole stem when shorter [fact, proposal for the rule].
• Colour is viridis over 80 dB below the stem peak: the loudest bin above DC over the whole stem at the panel’s frame length [fact]. A relative scale keeps strong partials from clipping to the top colour [fact]. Every level is finite and clamped before the colour map [fact].

Sources

Read for the problem statements and the formulas; the code is our own.

• Window choice and the peak accuracy bound: Harris, On the use of windows for harmonic analysis with the discrete Fourier transform; Rife and Boorstyn, Single-tone parameter estimation from discrete-time observations.
• Analytic signal: Boashash, Estimating and interpreting the instantaneous frequency of a signal.
• Harmonic masks: Klapuri, Multiple fundamental frequency estimation based on harmonicity and spectral smoothness.
• Modulation spectrum and period search: Atlas and Shamma, Joint Acoustic and Modulation Frequency; de Cheveigné and Kawahara, YIN, a fundamental frequency estimator for speech and music.
• Periodogram: Lomb, Least-squares frequency analysis of unequally spaced data; Scargle, Studies in astronomical time series analysis. II; Zechmeister and Kürster, The generalised Lomb-Scargle periodogram.
• Backward integration: Schroeder, New Method of Measuring Reverberation Time.
• Descriptor definitions: Peeters et al., The Timbre Toolbox.

Pitch layer

analyze reads the pitch layer of a monophonic line after the notes step: the glides between notes, the pitch drop at the strike of a low note, and vibrato. The layer writes the pitch group of layers.json; the layer report page lists its fields. The tags are those of the pipeline page.

The glides and the vibrato read the f0 track. The drops read the sub-band line, because the track averages over a 93 ms frame and smears a drop that settles in 20 ms [fact]. With --notes the model does not run, but the f0 track still reads the stem, unless the notes play chords or are empty and the layer is skipped [fact].

Every figure below comes from generated signals at 22,050 Hz, unless it names another rate, through the transcriber’s notes and the f0 track of the release build [synthetic]. A real stem adds filters, reverb and separation artefacts.

What the layer reads

• The voicing of the line. A line that plays chords gets values: null with the reason polyphonic_line and the flag chord_skipped: no method here separates the voices [fact]. The voicing comes from the notes step, or with --notes from the same chord rule applied to the given notes [fact]. That rule calls a part chords from a chord share of 0.68 [fact]. Real stems split into shares of 0.41 or less and 0.52 to 0.67, so a part at 0.52 to 0.67 runs this layer as a line; on those stems the layer read 1 false drop, 1 false glide and 4 glides an independent f0 confirmed [fact].
• A stem without notes gets values: null with the reason no_note_list and the flag no_notes [fact].
• Every time in the group is audio time from the start of the stem [fact].

The track’s lag

The tracker compares the first half of each centred frame with the samples one period later. A frame therefore reads the audio from a quarter frame before its centre to one period after it, so the track lags the audio by a quarter frame, 23.2 ms, less half a period [fact]. That is 18.7 ms at 110 Hz and 14.1 ms at 55 Hz [fact]. The layer subtracts the lag at the pitch it reads; a mono glide’s start then lands within 4 ms of the truth at 110 Hz [synthetic].

The sub-band line

• A fourth-order Butterworth high-pass at 20 Hz and low-pass at 200 Hz keep the fundamental of a low note, each run forward and backward with the odd reflection of the analytic signal stage [fact].
• The analytic signal of the band, zero-padded by its own length, gives the frequency between consecutive samples, $f[n]=\arg (z[n+1]\overline{z[n]})f_s/2\pi$ [fact].
• A centred median of $\mathrm{round}(1.4\mathrm{ms}\cdot f_s)$ samples, made odd, removes the phase spikes: 31 samples at 22,050 Hz, 63 at 44,100 Hz and 67 at 48,000 Hz [fact]. Near an end the window shrinks to the samples that exist, so the end is not pulled towards a padding value [fact].
• The line matches SciPy’s sosfiltfilt, hilbert and a shrinking median on the same 808 tone within 3.2e-4 Hz, a relative 1e-5 [synthetic].
• The analytic signal of a segment ripples at twice the tone near each cut end, by an amount that falls as one over the distance from the end [fact]. A steady 41.2 Hz sine reads its frequency within 0.01 Hz on average and within 0.5 Hz everywhere from 0.23 s inside each end [synthetic]. The layer keeps 0.1 s before the onset search and 0.5 s after the fit span [fact].
• A tone switched on from silence overshoots the line too: by about 11 Hz at 3 ms after the onset, 1 to 4 Hz at 7 ms, and within 1 Hz by 30 ms, at 30 to 150 Hz [synthetic]. A plain 41.2 Hz tone reads 9.4 Hz high at 3 ms and 0.9 Hz high at 7 ms [synthetic].
• The band keeps the fundamental only. When the second harmonic is stronger the line follows it; the drop then fails the weak-fundamental check below [fact].

Pitch drops

A drop is the fast fall of an 808 or a pluck from a higher start into its settled pitch.

• Candidates. Each note whose pitch lies in 20 to 200 Hz is a candidate strike [fact, proposal for the rule].
• Onset. The transcriber places the onset of a falling note late, by up to 230 ms on the rows below [synthetic]. The layer looks for the onset from 0.5 s before the note, or from where the previous note’s search ended, to 0.1 s after it [proposal]. The band envelope is averaged over a centred 10 ms; the onset is the first sample after its lowest point in that span where it reaches half the peak that follows [proposal]. When the lowest point already stands above half that peak, no attack lies inside the span and the note is no candidate [fact]. The average keeps a held saw from reading as a strike: its partials beat inside the band, so the raw envelope dips every period [synthetic]. A centred average leaves the midpoint of a step in place [fact]. A note that the transcriber splits off a held note, as under vibrato, has no attack and no candidate [fact].
• Strike. The rise is the full-band RMS over the 20 ms after the onset against the RMS from 45 to 25 ms before it [fact]. A drop needs a rise of 6 dB when its $\tau$ is under 10 ms, when its fall starts at 200 Hz or above, or when the band envelope takes over 30 ms from 10 to 90 % of its peak; otherwise it needs no rise [proposal]. A drop with $\tau$ under 10 ms also needs an attack of 30 ms or less: the fall ends inside a slower attack, where the line wanders on the rising level [proposal]. On a real stem a fall with $\tau$ = 5.8 ms in the noise floor read an attack of 208 ms; of the generated drops, only a false one at a soft retrigger had both [fact, synthetic].
  • A saw falling from 220 Hz into the band raises the band envelope with no strike; without the rule its 4-semitone glide also read as a drop of 4.6 semitones [synthetic]. A 7-voice unison falling the same way reads an attack of 191 ms [synthetic].
  • Where a ringing tone restarts its phase, the line swings and fits a fast fall: a plain retriggered 808 read 32 semitones with $\tau$ of 4 ms [synthetic].
  • An 808 note struck while the one before still rings rises only by that note’s decay. With 0.4 s notes the rise is 9 to 10 dB at a decay of 0.3 s, 4 to 5 dB at 0.6 s, 2 to 3 dB at 1 s and under 1.5 dB at 2 s [synthetic]. Its drop starts near 100 Hz with an attack of 9 to 13 ms, so it needs no rise [synthetic].
  • The span before ends 25 ms early, since the band onset of white noise lags the start of the noise by 14 ms [synthetic].
• Level. A strike whose RMS over the 20 ms after the onset lies more than 40 dB below the loudest 25 ms frame of the stem is no candidate, the gate of active frames [proposal]. On a real stem a fast fall settling on a note two octaves under the line read as a drop at -60 dB, where an independent f0 found no voice [fact]. A generated drop 60 dB below a loud passage reads none, 30 dB below it reads [synthetic]. A strike whose 20 to 200 Hz RMS over those 20 ms lies more than 18 dB below the loudest 25 ms frame of that band is no candidate either [proposal]. Checked against an independent IF on real stems, the drops of a line’s own low notes read 3 to 12 dB below the loudest passage of the band; two quiet falls under a louder line in the band read 22 and 28.5 dB below it, and the fall in the noise floor 33 dB [fact]. A generated drop 24 dB under an 87 Hz line reads none, 10 dB under it reads [synthetic].
• Fit. $f(t)=f_{\text{end}}+\Delta f{e}^{-t/\tau }$ is fitted by least squares to the line from 7 ms to 400 ms after the onset, or to the end of the note [fact]. For each $\tau$ the fit is linear in $f_{\text{end}}$ and $\Delta f$; a log grid of 240 points and a golden-section search find $\tau$ from 3.04 ms, where a tenth of the fall is left at 7 ms, $e^{-7\mathrm{ms}/\tau }=0.1$, to 1 s [fact, proposal for the floor]. The fit matches SciPy’s least_squares with the same bounds within a relative 1.5e-6 [synthetic].
• Fit start. From 3 ms the onset overshoot of the line reads as a fast fall: a plain 41.2 Hz tone with a 4 Hz drop fitted a drop of 9.9 semitones with $\tau$ = 3 ms [synthetic]. The fit starts at 7 ms [proposal].
• Refit. A fit with $\tau$ under 10 ms, or after a strike that rises 20 dB, is fitted again on a line whose high-pass sits at 10 Hz, and the refit decides [proposal]. Where a tone below 40 Hz starts from silence or rises, the 20 Hz high-pass raises the line for about 10 ms, which fits as a fast fall; how far depends on the tone’s start phase [synthetic]. On first notes at 32.7 and 36.7 Hz, 6 of 162 random start phases read falls of 10 to 14 semitones with $\tau$ of 3 to 5 ms; on the 10 Hz line none does [synthetic]. The 10 Hz line everywhere loses 3 of 18 drops on ringing 808 lines, so slow fits keep the 20 Hz line [synthetic].
• Residual. For a fit with $\tau$ of 10 ms or more, the error is averaged over half a period of $f_{\text{end}}$ before its RMS [proposal]. The third harmonic of a driven note beats with the fundamental and swings the line at twice the note; the average removes that swing and keeps a slower error, from a wrong model [fact]. A driven 808 drop left a raw residual of 23 % and fitted $\Delta f$ and $\tau$ within 1 % [synthetic]. Over a full period the average also hid the swing where a retrigger restarts the phase: two same-pitch retriggers read falls of 4 and 11 semitones with $\tau$ of 12 ms [synthetic]. A fall faster than 10 ms ends inside one period, and the average would spread its misfit, so it keeps the raw RMS [fact]. A fall whose $\tau$ spans 1.5 periods of $f_{\text{end}}$ or more changes little within one, so its error is averaged over a full period, which removes the swing of a second harmonic too [proposal]. A sweep into 32.7 Hz with $\tau$ = 90 ms and a second harmonic at half the fundamental failed on the residual with half a period and reads with a full one [synthetic].
• Note line. A fit after a 6 dB strike that is no drop, by any test of the rule or because it settles on no note, is fitted again on a line whose low-pass sits at 1.5 times the note’s pitch, when that edge lies under 200 Hz, so the note sits under 133 Hz [proposal]. Below 66 Hz the second and third harmonics of a note lie inside the band [fact]. On a real sweep into 32.7 Hz the second to fourth harmonics sat 5 to 14 dB under the fundamental, the 20 to 200 Hz line swung from -144 to +83 Hz while the spectral peak fell smoothly from 69 Hz, and the fit’s residual was 2.5, against 0.15 [fact]. At 1.5 times the note the second harmonic falls 10 dB more, while a fall that starts higher stays the strongest component on its way down [fact]. A generated sweep of that shape with harmonics 3 to 8 dB under the fundamental settles near 62 Hz on the 20 to 200 Hz line, a fall of the wrong sign, and reads on the note line with $f_{\text{end}}$ within 1 Hz and $\tau$ within 20 % [synthetic]. Octave falls into 82.4 and 103.8 Hz whose second harmonic is as strong as the fundamental or 5.6 dB stronger settle 120 and 130 cents sharp on the 20 to 200 Hz line and read on the note line [synthetic]. Falls with $\tau$ = 12 ms into A1 and C2 fit $\tau$ on the bound of the search on both band lines and read on the note line [synthetic]. At 1.2 times the note a generated sweep into C1 with strong overtones read $\tau$ 24 % long and one struck again inside its note read none; at 3 times both read none [synthetic].
• Note line span. The refit’s span ends one period of the note before the first point where the note line’s frequency falls to 0 Hz or below [proposal]. The phase of a line that holds one tone never runs backwards; it does where that tone breaks off and a second sound starts inside the same note, and the low-pass smears the break over about a period [fact]. A real sweep into 32.7 Hz was struck again 390 ms later inside the same note: the note line’s frequency fell to -39 Hz there, the fit over the whole 400 ms span missed on a residual of 0.24, and fits ending 200 to 300 ms after the onset read $f_{\text{end}}$ 32 Hz, a fall of 30.5 Hz and $\tau$ of 26 to 28 ms with a residual of 0.05 [fact]. A generated restrike of that shape reads within 1 Hz of the note; ending the span at the break itself read 30.1 Hz, outside the note [synthetic].
• Note line tests. The refit counts when it passes every test of the rule, when its $\tau$ is 10 ms or more, and when its RMS envelope over the fall, from 7 ms to 5 $\tau$, lies at most 6 dB below the line the first fit read [proposal]. A line low-passed under 70 Hz cannot follow a fall that ends within a period: generated kicks from 255 into 55 Hz with $\tau$ = 3 ms read a fifth of their fall with $\tau$ of 5 ms, and sub kicks over a held note read falls of 3 to 4 ms [synthetic]. Over the fall of four true drops on real stems the note line lay 2.2 to 3.8 dB below the band, and generated sweeps 2.2 to 3.6 dB [fact]. A tone at 127 Hz over a note at 61.7 Hz, with the note’s own tone entering 80 ms after the strike, lay 10.8 dB below: the note line held the upper tone 18 dB down and fell from it to the lower one as that entered, a step between two tones [fact]. Generated tones of that shape lie 4.8 to 7.6 dB under; the two in the tests, 6.6 and 7.3 dB, read as drops without the cap, so the margins are 2.2 dB under it and 0.6 dB over it [synthetic]. Without a strike the note line smooths the swing where a retriggered tone restarts its phase into a fall: three generated retriggers that rose 0.6 to 1.6 dB read one [synthetic].
• Same strike. Each note’s onset search starts where the one before ended, and a strike on that border rises across it: the next search finds it again and reads it as a second drop [synthetic]. A search that starts inside that rise is lowest at its first sample, and a new strike dips inside the search first [fact]. So a search that starts on that border, before the drop read last has settled, five time constants after its onset, holds the same strike when its band envelope, averaged over two periods of the note, is lowest at its first sample [proposal].
  • The average. A second harmonic in the band beats with the fundamental and dips a 10 ms average by up to 12 dB [synthetic]. The test set is 456 generated sweeps into C1 [synthetic]. They have attacks of 3, 30 and 80 ms, $\tau$ of 30 and 60 ms and second harmonics of 0 to 1. The border lies 20 ms before to 70 ms into the strike. Over 10 ms, 21 of them read a second drop; over one period 5, over 1.5 periods 1 and over two none [synthetic]. Eight read no drop under every rule. Over three periods, 90 runs of 808 hits struck every 115 ms lost 2 of their 270 drops [synthetic].
  • Against a fixed span. An onset within 20 ms of the border read 47 of the 456 sweeps as two drops: an 80 ms attack crosses half its peak 22 to 35 ms after a border 5 to 15 ms in [synthetic]. 808 hits struck every 115 ms land 15 ms after their border. That span skipped 59 of their onsets and lost 15 of the 270 drops [synthetic]. Hits struck every 250 ms with drops of $\tau$ = 60 ms read three drops either way, where skipping every onset inside the settling read two [synthetic].
• Rule. A drop counts when $\Delta f$ in semitones, $12{\log }_2((f_{\text{end}}+\Delta f)/f_{\text{end}})$, is above 2, when the fitted fall from 7 ms on, $\Delta f{e}^{-7\mathrm{ms}/\tau }$, is above 2 semitones too, and when the RMS residual is below 15 % of that fall [fact]. Both tests read the fall the fit span shows, not the one extrapolated to the onset [fact]. Against the extrapolated $\Delta f$ the residual misleads: on a 55 Hz tone with five harmonics, three of them inside the band, the line swings at the onset and the fit found $\tau$ = 0.7 ms with an extrapolated $\Delta f$ of 776 kHz, a residual of 0.003 % of it but 48 % of the fall from 7 ms on [synthetic].
• Bounds. $\tau$ must lie above the floor of the search, and $\Delta f$ at most 36 semitones, three octaves [proposal]. A fit on the floor means the best fit lies below it [fact]. Retriggered 808 hits restart the sine’s phase and the line swings at each join: with the floor at 0.5 ms and no cap, five abutting hits at 43.65 Hz read falls of 141 and 152 semitones with $\tau$ of 0.7 ms [synthetic]. The line reads up to 200 Hz, so a fall from more than three octaves above a low note starts far outside the band and is extrapolated from its tail [fact].
• Weak fundamental. After the fit, the medians of the half-octave bands around the note’s pitch and twice it are read from 50 ms after the onset or the note’s start, whichever is later, to 400 ms after the onset or the note’s end, whichever is earlier [fact, proposal for the window]. The window starts no earlier than the time the fitted pitch falls inside the band around $f_{\text{end}}$: a slow sweep from 80 Hz into 32.7 Hz crosses the band around twice the note, and without that start a pure sine read as a weak fundamental [synthetic]. When the upper band exceeds the lower by a factor of 2, 6 dB, the line would follow the second harmonic: the layer fits nothing for that note and sets the flag weak_fundamental [proposal]. White noise reads the upper band 1.41 times the lower, since it is twice as wide [fact]. Below the factor a fit that reads the harmonic settles 1200 cents off the note and fails the settling rule [fact].
• Settling. A drop also settles on a note. One note sounds at some point while the drop is settled, from five time constants after the onset to the end of the fit span. Its pitch lies within 100 cents of $f_{\text{end}}$, and it starts 150 ms or less before the onset [proposal]. A note that started earlier belongs to a sound before the strike: on real stems a fit on the lead-in to a step settled on the note that had sounded for 790 ms, and a fit inside a chord on a note an octave below that had started 210 ms before [fact]. A generated fall into C1, found in the search of a higher note struck with it, settles on a C1 note that started 100 ms before the strike and on none that started 200 ms before [synthetic]. A C1 note that ends 46 ms after such a fall has settled takes the drop, and one that ends 34 ms before it does not [synthetic]. A slow fall not settled by the end of the span needs the note at its end: with $\tau$ = 90 ms on a 350 ms note, five time constants end after the note, and the sweep read as a glide from silence [synthetic]. At a saw’s onset several partials share the band and the line swings; on a generated 55 Hz saw that swing fitted a fall of 62 semitones with $\tau$ = 2 ms settling at 49.7 Hz, which this rule rejects [synthetic].
• Skips. The summary line drop skips counts the low notes the search read no drop at, by the first gate that stopped each: no onset, settling, quiet, no strike, weak fundamental, no fall, tau on bound, residual and off note [fact].
• Confidence. Low when the attack exceeds 30 ms: a slow swell is no strike [proposal]. High needs a residual below 5 %, $\tau$ of 10 ms or more and an attack of 12 ms or less, else medium [proposal]. The reasons are fit_residual and attack_time [fact].
  • The fit sees $e^{-7\mathrm{ms}/\tau }$ of the fall, a quarter at $\tau$ = 5 ms: kick drops with $\tau$ of 3 to 8 ms read $\Delta f$ from -31 to +25 % [synthetic].
  • During a slower attack the pitch falls before the onset, and $\Delta f$ reads low: by up to 30 % with a 10 ms ramp, 46 % with 20 ms and 73 % with 40 ms [synthetic]. The 10 ms average spreads a sharp strike to an attack of 9 to 11 ms, a 20 ms ramp to about 17.5 ms [synthetic].
  • On the generated sets, high-confidence drops read $\Delta f$ within 17 %, with one at -30 % (a 10 ms ramp on a 15 ms fall), $\tau$ within 8.3 % and $f_{\text{end}}$ within 0.1 Hz [synthetic].

Drops on an 808 tone, a sine times a 0.5 s amplitude decay, through the transcriber’s notes [synthetic]:

The acceptance bounds are 1.5 Hz on $f_{\text{end}}$, 25 % on $\Delta f$, and 25 % on $\tau$ up to 50 ms, 40 % at 100 ms [fact]. White noise 30 dB below the tone’s peak keeps $f_{\text{end}}$ within 1 Hz and the drop accepted [synthetic].

The pYIN columns that #67 quotes are re-run in 67-drop-pyin.json: librosa 1.0.0 pYIN at frames 2048 and 4096, hop 128, on the same tones, and the same fit from 3 to 400 ms after the onset [synthetic]. Two of the eight rows reproduce within 0.1 Hz and 5 %, the 50 ms and the 100 ms rows at frame 2048 [synthetic]. At frame 2048 the 20 ms row reads $\Delta f$ 55 % high and $\tau$ 24 % short of the issue, and the 30 ms row $\Delta f$ 8.7 % low [synthetic]. At frame 4096 no row reproduces, but no field lies more than 5.7 % or 0.31 Hz off the issue [synthetic]. The smear shows in kind, not within the tolerance: the 20 ms drop reads $\tau$ 29.5 ms at frame 2048 and 76.0 ms at frame 4096, against the issue’s 72.3 ms there and 21.6 ms from the sub-band line [synthetic]. The Hilbert column, checked in sub-band-line.json, reproduces 1 of its 4 rows [fact].

Limits:

• Drive and harmonics. tanh(4 x) puts the third harmonic, 124 Hz, inside the band, and a saw below 100 Hz its second and third. With the average, the drop of the 41.2 Hz row through tanh(4 x) reads $\Delta f$ 14 % low and $\tau$ 13 % long [synthetic]. A 12-semitone drop with $\tau$ = 30 ms on a driven sine at 41.2, 55 and 82.4 Hz reads within 0.7 semitones and 7 % on $\tau$, at high confidence [synthetic]. On a saw the second harmonic swings the line at the note itself, which half a period does not average out, and no drop with $\tau$ under 1.5 periods is read [synthetic]. A second harmonic stronger than the fundamental takes the line: of 36 generated sweeps from silence with one 3.5 dB above it, none read [synthetic].
• A fall under a louder line in the band, or under a louder passage of the same line, is not read 18 dB or more below it, so a soft note after loud ones loses its drop [fact].
• A note above 200 Hz is not read: the band stops there [fact].
• The note line helps only notes under 133 Hz, where 1.5 times the note lies under the top of the band [fact]. It runs only after a strike of 6 dB, so a fall that the band fit misses under a note still ringing at nearly its level stays lost [fact].
• A tone an octave above the note, with the note’s own tone entering soon after it, can still read as a drop on the note line: generated tones of that shape whose note line lay 4.8 to 6 dB below the band read falls from the upper tone into the note [synthetic].
• A true drop whose note started more than 150 ms before its onset, as when the transcriber merges a restrike into the note before it, is rejected [fact].
• A kick at 55 Hz that decays over a quieter held sub at 43.6 Hz can read as a fall from the kick into the sub on the 20 to 200 Hz line, with $\tau$ of 17 to 40 ms [synthetic].
• A kick with no pitch drop and a noise click, over a sub, reads a fast drop of 9 to 27 semitones at medium when it starts from silence, like a kick whose pitch drops [synthetic]. Inside the first period of the note the click and a fall of $\tau$ under 10 ms look alike; rejecting fast falls from silence would lose 26 true kick drops of the generated sets [synthetic].
• A soft retrigger or an envelope dip on a low note mid-line can still read a false fast drop at medium: of 82 generated lines of that kind, one does with the transcriber’s notes, another with the given notes, 15 to 16 semitones with $\tau$ under 4 ms [synthetic].
• A legato note, which follows the previous run with no fall of the band envelope, has no onset and no drop [fact].
• A struck note whose band envelope does not dip first has no onset and no drop: with no release before it and a decay of 1 s or more, the dip of an 808’s 3 ms attack is lost in the 10 ms average [synthetic]. Of 18 drops on generated 808 lines with given notes, 14 read; the misses are those notes and a soft note after a loud one [synthetic]. Through the transcriber’s notes fewer read, since its onsets move the search [synthetic].
• A fast drop, $\tau$ under 10 ms, after a ringing note needs a 6 dB strike, and a drop with a slow attack needs one too [fact].
• Retriggered 808 hits at one pitch, with and without gaps, ringing or not, and 808 lines stepping up to 7 semitones report no drop with $\tau$ of 10 ms or more [synthetic]. A false fast drop, $\tau$ under 10 ms, needs a 6 dB strike and survives the refit on the 10 Hz line only where the limits above say [synthetic].

Glides

A glide is pitch moving from one stable pitch to another.

• The track’s voiced frames, in cents, split into runs of consecutive voiced frames [fact].
• Gaps. A run that starts within 100 ms of the one before joins it when the full-band RMS does not rise 6 dB in the gap, and the pitch keeps moving after the gap the way it jumped across it: by 20 cents or more over the jump and over the next two frames [proposal]. A unison whose voices start in phase beats hard enough to leave the track unvoiced for 75 ms inside a glide; read apart, the rest read as a glide from silence of 3.9 semitones for 7 [synthetic]. At a step between two notes the track goes unvoiced too, but lands on the new pitch and holds it, or swings back; at an 808 step the new strike raises the level [synthetic]. The jump across the gap must be 30 cents per millisecond or slower [proposal]: a 50 ms glide of 7 semitones jumps 20 across two unvoiced frames, the unison above 3.6 [synthetic]. On a real stem a note struck again 10 semitones lower, across three unvoiced frames, read as a glide; that is about 40 cents per millisecond [fact]. Nor is a gap bridged where the full-band RMS, over 10 ms, falls 20 dB below the 20 ms before or after it, or 40 dB below the loudest frame of the stem [proposal]: on real stems steps read as glides across a dip of 41 dB and at a noise floor of -60 dB [fact].
• Steps across a gap. Two stable pitches with a bridged gap between them join no glide when the pitch jumps across the gap against the interval by 20 cents or more, or by more than 60 % of it [proposal]. A step and the settle after it keep moving the way the step jumped, at 5 to 17 cents per millisecond, so neither the slope limit nor the motion test tells them from a glide [fact]. On real stems steps of 3 to 10.5 semitones read so [fact]. A 2.1-semitone fall near 44 Hz on a real stem jumped 200 of its 210 cents across a gap while the level rose 17.5 dB, and reads none [fact]. Unison glides whose voices leave the track unvoiced for 75 ms jump 0.53 and 0.54 of the interval across the gap; a step between 808 notes jumped 0.72 [synthetic].
• A stable pitch is a span of 50 ms or more whose smoothed pitch stays within 30 cents of its running mean and whose least-squares slope stays at 86 cents per second or less [proposal]. 86 cents per second is the floor of a published portamento detector, untuned [fact].
• A glide joins two consecutive stable pitches 100 cents or more apart that each match a note within 50 cents, with the two notes of different pitch [proposal]. The note test keeps vibrato out, which the transcriber gives as notes of one pitch at ±35 cents [synthetic]. No glide is read on any vibrato signal below, ±35 or ±60 cents [synthetic].
• The motion is the span between the middles of the two stable pitches where the slope of the 5-frame moving average, 29 ms, stays at 86 cents per second or more in the direction of the motion [fact]. Under 35 ms it is a step between notes [fact, proposal for 35 ms].
• Then the fitted time decides with the same bounds: under 35 ms a step, from 35 to 90 ms a glide too short to read, counted and flagged glide_too_short, from 90 ms on a glide [proposal]. A fit off the curve bound that passes every test of a glide over a detected span of 35 to 90 ms reads as a glide at low confidence, and a motion whose fit fails over such a span is too short [proposal]. The 90 ms is the shortest transition of a published portamento detector, cited and untuned [fact]. On real stems a slide of a semitone over about 60 ms, clear in the spectrum, fell under it [fact]. Reading such fits changes no record of the generated sets [synthetic]. The track smears a step between two 808 notes over a detected span of 100 to 450 ms, while the fit finds the motion short [synthetic].
• Fit. $s(x)=1-(1-x{)}^p$ between the two stable pitches is fitted by least squares over the motion and 50 ms of each stable pitch, with the start, the full time and $p$ free [fact]. $p>1$ starts fast, $p<1$ starts slow, $p=1$ is linear in pitch [fact]. A grid of 9 powers, 8 times and 13 starts seeds a Nelder-Mead simplex; $p$ stays within 0.2 to 5 [fact].
• Shape. A fit of 90 ms or more with $p$ within 5 % of 0.2, or with an RMS residual above 40 cents, does not follow the curve and is no glide [proposal]. The glides of the generated signals fit within 22 cents; the false glides at 808 steps sat on a bound of $p$ or fitted under 90 ms [synthetic].
• Curve bound. An exponential (RC) portamento, $1-e^{-t/\tau }$, approaches its target slower than any power of the curve, so its $p$ sits on the upper bound of 5 [fact]. A fit within 5 % of that bound counts when its residual is 20 cents or less, its detected span lasts 90 ms or more, and it does not come from silence; it reads low with the reason curve_bound [proposal]. Without those conditions a melody step on a sine and two driven drops read as glides on the bound [synthetic].
• Span ratio. A glide on the bound also needs a detected span of at most 2.5 times its fitted time [proposal]. The track smears a step into a fast curve on the bound and keeps drifting after it, so the slope stays up long after the fitted motion ends: 6.2 times its fitted time at the semitone step on one 808 oscillator that keeps its phase [synthetic], 2.9 and 9.1 at two steps on real stems that an independent f0 read as a step and as no slide [fact]. The RC slides of the tests read 0.77, 0.80 and 2.27 [synthetic]; 26 small slides of 1 to 2.1 semitones on real stems that the independent f0 confirmed in full or in part read 0.44 to 2.28 [fact]. The cut sits 10 % above the highest slide and 14 % below the lowest step, so the margin is thin on both sides [fact]. The level does not tell the two apart: at those real slides the full-band RMS over one period jumped 2.1 to 19 dB [fact]. An RC slide of 1.2 semitones at 110 Hz reads on the bound [synthetic]; one of a semitone reads 90 cents on the grid, under the 100-cent interval, and is not read [synthetic].
• time_s is the full 0 to 100 % time [fact]. start_s is the fitted start less the track’s lag at the start pitch [fact].
• From silence. When a voiced run starts above or below its first stable pitch by 100 cents or more, and the motion into it lasts 90 ms or more, the glide carries from_silence: true [fact]. A shorter motion from silence is neither read nor counted toward glide_too_short, which names a motion between two notes [fact]. The run must settle on that stable pitch [proposal]. Its median level lies within 40 dB of the loudest frame, the gate of active frames [fact]. And within half a period of 3 Hz, the slowest vibrato, the smoothed pitch does not swing back half the lead [proposal]. The motion after the first stable pitch is a step, and the window holds nothing, when the next stable pitch lies 100 cents or more away and matches another note, and the pitch, once past half the way to it, stays there up to that stable pitch and to the window’s end [proposal]. A wobble’s crest can match a neighbouring note, but the pitch comes back from it [synthetic]. When two notes match a stable pitch, the nearest one counts [proposal]. The end of the transcribed note does not cut the window: on real stems the transcriber ended a wobbling note inside its first trough [fact]. A step to a pitch that no transcribed note matches counts as a swing back, since a wobble’s crest matches none either [proposal]. On real stems a wobble of ±60 cents at 6 to 7 Hz read its first half-cycle as a fall from silence, coming back 82 % of its lead [fact]. A quiet onset after digital silence read one into a stable pitch at -41.9 dB, 1.9 dB past the gate [fact]. True falls there came back 0 and 24 % and settled at -16 and -9 dB [fact]. Generated falls that swing back or settle at -42 dB read none [synthetic]. So does a wobble whose note ends inside its first stable pitch, and one at 6 to 7 Hz with a neighbouring note at its crest [synthetic]. One that rebounds a quarter or 45 % of its lead reads [synthetic]. So does one that steps to the next note 30 ms after the stable pitch, with the first note ending at the step’s start, 20 ms into it or after it, and one that slides there over 80 to 400 ms [synthetic]. A generated fall from 150 Hz into 50 Hz with $\tau$ = 40 ms under a 300 ms attack fails the strike rule as a drop [synthetic]. It reads as a fall from silence at medium, 14.6 of its 19.0 semitones: it starts near 116 Hz, where the fall stands 17 ms after the onset [synthetic]. The sound must start up to 100 ms before the run: the full-band RMS over 30 ms rises 20 dB above the RMS from 60 to 30 ms before, searched in 5 ms steps [proposal]. The track voices a run some frames after the sound starts, and a beating unison dips without falling silent [synthetic]. The run’s first frame fixes the start and the fit also finds the start pitch [fact]. A run that starts within 0.2 s of an accepted drop is the drop, not a glide [proposal].
• Confidence. The reasons are the motion span, the RMS residual of the fit in cents and the jitter: the RMS deviation of the two stable pitches from their medians, and curve_bound [fact]. High needs a residual of 15 cents or less, a jitter of 3 cents or less, a motion of 150 ms or more and $p$ of 3 or less; low comes with a jitter above 3.4 cents, a curve on the bound or a detected span under 90 ms [proposal].
  • A faster start reads its rise less well: on the generated sets, up to 11 % off at $p$ 4.4, while high-confidence glides read their rise within 7.1 % [synthetic].
  • A unison beats and the track wanders on the stable pitches, but the jitter tells a unison only roughly. On a larger generated set the unison glides jitter 3.77 to 7.07 cents at 22.05 kHz and 3.45 to 6.93 at 44.1 kHz, the mono ones 0.66 to 8.03 at both rates; one unison sets the 3.45, 0.05 cents over the cut [synthetic]. In the committed fixtures the unison glides jitter 3.54 cents or more, 0.14 over the cut, and no glide falls from 3 to 3.4 cents [synthetic]. All unison glides read low; from 3 to 3.4 cents only mono glides fall, and they read medium, and a mono glide above 3.4 cents reads low too [synthetic]. One threshold would raise those mono glides to high on no measured gain, so the two stay [proposal].

Glides of 7 semitones up from 110 Hz on a band-limited saw, 0.5 s holds [synthetic]. The rise is the 10 to 90 % time derived from the fitted time and $p$, $(x_{90}-x_{10})T$ with $x_q=1-(1-q{)}^{1/p}$; its error is against the true rise:

The #66 table measured the 10 to 90 % span straight off a librosa pYIN track. 66-glide.json re-runs it on these signals with librosa 1.0.0 at frame 2048, hop 128, fmin 30 Hz and fmax 400 Hz [synthetic]. Every true span reproduces within 0.5 ms, the issue’s rounding [synthetic]. Five of the 6 mono figures reproduce within one hop, 5.80 ms; two of them pass only just, 5.78 and 5.54 ms off [synthetic]. The 50 ms fast-start glide is lost: its first frame at or above 10 % already stands at 90 %, where the issue read 6 ms [synthetic]. Of the unison figures only two reproduce: the 50 ms fast start, lost, and the 400 ms fast start [synthetic]. On this unison librosa reads the linear glides like the mono ones, 116 and 319 ms against the issue’s 134 and 331 ms, the 50 ms linear glide at 29 ms against lost, and the 150 ms fast start at 11.6 ms against 58 ms [synthetic]. The unison’s start phases decide it: over 20 seeds the 150 ms linear glide reads 104.5 to 133.5 ms, the 150 ms fast start 11.6 to 98.7 ms and the 400 ms linear glide 307.7 to 336.7 ms, and the 50 ms glides are lost on 6 and 17 of the seeds [synthetic]. Every unison figure of the issue lies within one hop of some seed’s reading [synthetic]. #66 gives no unison phases, so its unison figures hold for its own signal only [fact]. This track fits the curve instead: it reads the rise of every 150 and 400 ms glide within 4.4 %, that 150 ms fast-start unison included, and no glide at 50 ms [synthetic].

The acceptance bound is 5 % on the rise from 150 ms on, mono [fact]. The mono interval reads 6.9 semitones for 7: the track’s 10-cent bins round the start 5 cents up and the end 5 cents down [synthetic]. The fast-start unison at 150 ms reads its rise within 4.4 % but its $p$ at 1.83, and its time 11 % short; the low confidence says so [synthetic]. “In phase” voices start at phase 0, so the unison beats hardest [fact].

Other glides, mono [synthetic]:

The track’s first frames of a tone read low, so a glide from silence reads its interval up to 0.6 semitones short [synthetic]. Of 16 generated RC glides, $\tau$ 30 to 120 ms at 110 to 247 Hz, all read, at low confidence [synthetic].

Limits:

• A fall from silence whose stable pitch sits just past the 40 dB gate is not read: on real stems a fall of about 1.4 semitones over 200 ms, in which the first three partials all fall, settled at -40.9 dB and was lost [fact].
• A glide that the transcriber merges into one note, with its two ends in one note, is not read [fact].
• A fall from silence followed within 167 ms by a step back toward its start, to a pitch that no transcribed note matches, reads as a swing back and is lost [synthetic]. So is one whose portamento to the next note stalls under half the way for 100 ms: the stall forms its own stable pitch, which no note matches; a stall of 60 ms reads [synthetic]. So is one followed by a move back toward its start of under 100 cents to another note, past half the lead [synthetic]. A wobble at 3 to 4 Hz whose crest matches a neighbouring note still reads as a fall from silence: the crest comes at the window’s end, and the pitch returns after it [synthetic]. Of 49 generated wobbles of ±40 to ±70 cents at 3 to 8 Hz with the neighbour sounding, 10 read, all at 3 or 4 Hz [synthetic].
• A step between two 808 notes can raise glide_too_short when the fit finds 35 to 90 ms: one of three generated 808 lines does [synthetic].
• A vibrato of ±100 cents at 3 Hz holds each extreme long enough to read as a stable pitch, and its swings read as glides of 1.9 semitones, each at low confidence [synthetic].
• An 808 slide reads low: its stable pitches jitter like a unison’s [synthetic].
• A falling slide of 7 semitones from 49 Hz is not read [synthetic].
• A glide inside chords has no method here [fact].
• Short glides of 1 to 3 semitones on real stems mostly read low, yet an independent f0 confirmed most of them: their confidence is pessimistic [fact].
• A slide on the bound whose track keeps drifting for over 2.5 times its fitted time is not read: a 2-semitone slide near 69 Hz on a real stem, which an independent f0 confirmed, drifted for 4.3 times its fitted time [fact].
• Where the tracker reads a tone’s second harmonic, a glide still reads its interval: on a real stem a tone rose 2.2 semitones from 921 Hz, the track followed its second harmonic from 1845 to 2100 Hz, and the glide read +2.0 semitones [fact]. An independent f0 that jumped to another source there read a step [fact].
• A glide that moves more than 60 % of its interval inside an unvoiced gap is not read: a 7-voice unison with a fast start jumped 0.76 of it across 75 ms [synthetic].

Vibrato

• Spans. The voiced runs of the track split where two stable pitches lie 150 cents or more apart, so a step between notes never reads as a vibrato, while a vibrato of ±60 cents, whose stable pitches lie 120 cents apart, stays whole [proposal]. Each span loses 50 ms at each end and the frames where a glide moves [fact]. A drop inside a span splits it: the part before ends 50 ms before the onset, and the part after starts once the drop has settled, five time constants, at least 100 ms [fact]. A generated vibrato on A1, restruck mid-span with a fast drop the track does not show, reads before the drop; starting the whole span after the settling left 0.38 s, under the 0.5 s the search reads [synthetic]. A generated octave drop into A1 with $\tau$ = 60 ms, then a vibrato, reads that vibrato; skipping only 100 ms left a fifth of the fall in the span, and the search read none [synthetic].
• A glide moves where its fitted curve lies more than 10 cents, the smallest extent of a vibrato, from both its pitches [proposal]. A curve on the upper bound of $p$ comes within 3 % of its interval of its end halfway through its time [fact]. On a real stem a vibrato of 3.1 cycles at 5.9 Hz began where a 1.8-semitone glide on that bound ended; held for the glide’s full fitted time of 122 ms, its span after the trim fell from 0.53 to 0.47 s, under the 0.5 s the search reads [fact]. A generated glide and vibrato of that shape keep the vibrato [synthetic].
• A span of 0.5 s or more is searched [fact]. A least-squares line and sine are fitted together at each rate from 1 Hz, or 1.5 cycles over the span, to 20 Hz in 0.01 Hz steps; the rate whose sine explains the most power beyond the line wins, refined by golden-section search [fact]. Fitting the line and the sine together keeps a sine over a fraction of a cycle from leaking into the line [fact].
• extent_raw_ct is the sine’s amplitude, half the peak-to-peak swing [fact]. share is the power the sine explains over the power left after the line alone [fact]. The power it explains is the line’s residual less that of the line and sine fitted together [fact]. The sine term alone can share power with the line: a 20-cent sine over three cycles under 20.6 cents of noise read a share of 0.524 that way and 0.490 by the residuals [synthetic]. On the generated sets the residuals drop 4 false vibratos of 26 and keep every true one [synthetic].
• Rule. A vibrato counts at 3 to 12 Hz with 1 % slack, an extent of 10 cents or more after the correction, a share of 0.5 or more and 2.5 cycles or more over the span [fact, proposal for the share and the cycles]. Checked against an independent f0 on real stems, every vibrato of 2.5 cycles or more matched it; the shorter ones, 1.6 to 2.4 cycles at 3 to 4.7 Hz over spans of 0.5 to 0.8 s, did not, or matched under 2 cycles [fact]. Through the track the rates below read within 0.8 % of the truth, so the slack keeps a vibrato at either edge [synthetic].
• Correction. The track’s 93 ms frame smooths a fast swing, so the raw extent falls with the rate [synthetic]. extent_factor is one over the share of the true extent the track reads, interpolated linearly between the measured rates and held at the ends [fact]. extent_ct is extent_raw_ct times extent_factor [fact].
• Confidence. Low when the kurtosis of the contour after the line, its fourth moment over the squared second, is below 1.25: a sine reads 1.5, a square wave 1.0 [fact, proposal for 1.25]. Low too on a pitch below 100 Hz [proposal]. Else high with 4 cycles or more in the span, else medium [proposal]. The generated vibratos read a kurtosis of 1.36 to 1.66, a semitone trill at 6 Hz 1.06 [synthetic].
  • On real stems every vibrato above 100 Hz with 4 cycles or more matched an independent f0 [fact]. The 23 vibratos the independent f0 found plausible read a share of 0.502 to 0.838: the 12 of them at high 0.502 to 0.630, the 10 at medium 0.519 to 0.838 and the one at low 0.575. The 12 vibratos it did not find plausible read 0.501 to 0.734, so the share does not tell the two groups apart [fact]. None read under 0.5: the lowest, 0.501, sits at the threshold, so the margin there is nil [fact].
  • Below 100 Hz few matched, and the extent read up to 5 times too wide [fact]. A generated low saw with amplitude modulation, two beating voices or none reads no vibrato, and one with a ±30-cent vibrato reads it within 2 cents down to 41 Hz, so the cause is not known [synthetic].

The share of the true extent the track reads, on one 1 s mono saw at 220 Hz with ±35 cents per rate [synthetic]:

Each share comes from one signal, so the table carries that signal’s quantisation to the track’s 10-cent bins: 7 Hz reads lower than 8 Hz [synthetic]. The share belongs to this track: another pYIN implementation at another frequency ceiling reads another share [fact].

The #67 figures are re-run in 67-vibrato.json: librosa 1.0.0 pYIN at frame 2048, hop 128, fmin 30 Hz and fmax 400 Hz on the signals below, and the FFT peak of the detrended cents over the tone less 50 ms at each end, 0.9 s [synthetic]. The table reads the extent as the amplitude of the sine at that peak [fact]. #67 defines the extent as half the peak-to-peak span, which equals that amplitude only for a clean sine [fact]. With the amplitude, the ±35-cent mono saw reproduces the issue: 5.50 Hz and 23.98 cents, a share of 0.685 against 0.69 [synthetic]. Half its peak-to-peak span reads 20.40 cents, a share of 0.58, which does not [synthetic]. This track reads the same saw at a share of 0.939 [synthetic]. The unison and ±60-cent rates read 5.47 and 5.49 Hz against the issue’s 5.49 and 5.53 Hz, 0.3 % and 0.7 % low [synthetic]. Over windows of 0.5 to 1 s they read 5.43 to 5.48 Hz and 5.46 to 5.49 Hz, so neither figure lies in its range [synthetic]. The librosa share moves with depth and voices: 0.621 on the unison, 0.782 at ±60 cents [synthetic]. On the unison half the peak-to-peak span reads 29.63 cents against an amplitude of 21.73 [synthetic].

This track reads:

The acceptance bounds are 0.1 Hz on the rate and 15 % on the extent after the correction [fact]. The correction is measured at ±35 cents only; the ±60 case reads 1.4 % high with it, and ±25 cents at 9 Hz reads 11 to 14 % low [synthetic].

Limits:

• A trill between two notes a semitone apart at a vibrato rate reads as a vibrato, since its stable pitches lie 100 cents apart, but at low confidence: its contour jumps between two levels [fact].
• From 4.5 Hz the transcriber gives a ±100-cent vibrato as overlapping notes, the line reads as chords, and the layer is skipped [synthetic].
• A vibrato whose depth ramps in over the span reads its extent 9 to 51 % low, since the sine averages the ramp [synthetic].
• A vibrato of 3.5 Hz over 0.7 s holds 2.2 cycles and is not read [synthetic].
• A triangle LFO reads its extent 20 to 26 % low at high confidence: the fitted sine holds $8/{\pi }^2$ of a triangle’s amplitude, and the extent is the sine’s [fact, synthetic for the range].
• A vibrato whose rate sweeps from 4 to 6 Hz reads no vibrato [synthetic].
• A vibrato that starts late in a note is diluted by the steady part before it [fact].

Sources

• Glide floor and minimum length: Yang, Rajab and Chew, AVA: an interactive system for visual and quantitative analyses of vibrato and portamento performance styles, ISMIR 2016, CC BY 4.0. Inspiration only.
• Analytic signal: Boashash, Estimating and interpreting the instantaneous frequency of a signal.
• f0 track: Mauch and Dixon, pYIN: a fundamental frequency estimator using probabilistic threshold distributions.

Envelope layer

analyze reads the envelope layer after the pitch layer: the amplitude envelope of each note, pooled over the notes, and the sidechain pump of the stem. The layer writes the envelope group of layers.json; the layer report page lists its fields. The tags are those of the pipeline page.

The stages follow the envelope of a synth: an attack from the note-on to the peak, a hold at the peak, a decay from the peak to the sustain level, the sustain level, and a release from the note-off [fact]. The layer reads the attack, hold, decay and sustain, the time of the note-off knee, and the release and tail after it [fact].

Figures come from cargo xtask eval --layer envelope, 1191 generated parts at 44,100 Hz scored against their recipes, or from the acceptance tests at 22,050 Hz, through the release build and with the true note list [synthetic]. The tables of the release and tail come from one-off generated sets, named where they appear, that are not committed [synthetic]. A real stem adds filters, reverb, bleed and separation artefacts.

The eval is local [fact]. It writes one file, the --json path, and only when that flag is given [fact]. Run it with a release build; a debug build of the library is far slower:

cargo run --release -p xtask -- eval --layer envelope --json records.jsonl

--json writes one record per part, so two builds compare record by record [fact]. A part renders only when --part selects it, and the run holds one part at a time: in a release build the whole set took 53.5 s and 56 s of wall time and 0.6 GB over two runs, against 160 to 179 s and 4.4 GB before the change [synthetic].

Each check pairs a read note with its true note by onset, and a note the layer skips fails its check [fact]. A row that scores only the parts that read a value prints how many read none [fact].

What the layer reads

• The note list after the cleanup, or the notes that --notes gives [fact]. A stem without notes gets values: null with the reason no_note_list and the flag no_notes [fact].
• The mono mix at the native rate [fact].
• The two channels of the stem for the side vote of the tail, when it has two [fact].
• The beat grid of the used tempo, from 0 s, for the pump [fact]. The offset is never estimated: a stem whose first beat is late shifts every beat window [fact].
• A note with no harmonic to read, or that is too short for a track, is skipped and counted in the summary [fact]. A harmonic cannot be read when it lies within 30 Hz of a harmonic of a note it overlaps, or when its band reaches 0 Hz or the Nyquist frequency; the summary counts these as no free harmonic [fact]. A note whose amp proxy peaks at -120 dB or lower is skipped as silent: over a silent stretch, a listed note would read a 0 dB sustain at the tracks’ floor [fact, proposal for -120 dB].
• Every time is audio time from the start of the stem [fact].

Level tracks

Each note is read on a 1 ms hop from 30 ms before its onset to its end [fact]. Three tracks exist:

• Centred RMS over whole periods of the fundamental: one period for a note’s attack, the fewest whole periods that span 2 ms or more on a high note [fact]. One period gave the rise times the issue measured; two periods read a 16 ms rise at 55 Hz as 34 ms [synthetic].
• A sliding DFT of one harmonic over one period of the fundamental, for a note that overlaps no other [fact]. It has no filter transient, so the peak does not overshoot [fact]. A fourth-order low-pass on the band magnitude overshot the peak and read decays long [synthetic].
• The zero-phase band magnitude of one harmonic, for a note that overlaps another [fact]. The low-pass cutoff is half the distance to the nearest harmonic of another sounding note, or to the note’s own next harmonic, clamped to 12 to 40 Hz [fact]. The filter runs forward and backward, so the track does not lag [fact].

The amp proxy is the note’s lowest harmonic among the first three that lies 30 Hz or more from every harmonic of the notes sounding with it [fact]. A filter sweep moves the high harmonics and leaves the low ones, so the lowest free harmonic follows the amp envelope [fact]. On a 110 Hz saw under a closing low-pass, the sum of harmonics 1 to 3 read the fall 8 % short, and the fundamental alone within 10 % [synthetic]. Octaves and unisons leave the upper note no free harmonic; the layer skips it and has no residual method [fact].

Levels are dB from the note’s peak on the amp proxy, read no lower than -60 dB [fact].

Attack

• attack_s is 1.25 times the 10 to 90 % rise of the linear amplitude: the full 0 to 100 % time of a linear ramp [fact]. The rise reads the RMS track for a note that overlaps no other, and the amp proxy for one that does [fact].
• The rise ends at the attack’s top: the first frame from the onset that the level exceeds by less than 1 dB over as long again as it took to rise to it from under 10 % of it, and at least over the shortest rise the track can tell, the same figure as the upper bound’s below [fact]. Without that floor, a wobble of the window early in the rise read as a top [synthetic]. The 10 and 90 % shares are of the highest frame over that span [fact]. A slow attack has no top before its end, since a linear ramp keeps rising by more than 1 dB until 1 dB below its peak [fact].
• The peak of the whole note is no top: a real note swells or ripples after its attack [fact]. At 110 and 261.63 Hz, a 50 ms attack that swells by 3 dB over 600 ms read 514 ms against the peak and reads 52 to 55 ms against the top; at 261.63 Hz, a 10 ms attack under a 6 dB ripple at 5 Hz read 30 ms and reads 12.5 ms [synthetic].
• A swell finds no top when it rises 1 dB or more over the attack’s own length [fact]. A 300 ms attack under a swell over 0.8 s reads 324 to 370 ms up to 2.9 dB and 640 to 706 ms from 3.1 dB, at 55 to 440 Hz and 22.05 to 48 kHz; at 3 dB it reads either [synthetic]. The 50 ms attacks above lie far from that edge [synthetic].
• The factor of 1.25 holds for a linear rise, so a rise that flattens near its top reads long [fact]. Loud isolated bass notes on real stems read 141 and 156 ms where the level crosses -1 dB at 103 and 105 ms and -3 dB at 72 and 42 ms, on 9 and 22 notes [fact]. Loud isolated keyboard and synth notes on real stems read 15 and 20 ms against -1 dB crossings at 18 and 14 ms [fact].
• A periodic wave gives one amplitude sample per period, so a rise under about 1.5 periods cannot be told from a shorter one [fact]. The attack is an upper bound when the rise is shorter than 1.5 RMS windows, or, on the amp proxy, shorter than 0.98 over its cutoff, which is twice the rise of the filter on an instant step [fact]. attack_is_upper_bound is set when more than half the pooled attacks are upper bounds [fact]. attack_s then still reports the read, and a preset should not take it [fact]. Each note record carries its own attack_upper_bound [fact].
• #69 writes an attack from 1.5 periods. The layer tests the rise, 0.8 of a linear attack, so its flag covers attacks up to about 1.9 periods, which is 45 ms at 41.2 Hz [fact]. It departs from #69 on the safe side: a 2.2 period attack is the shortest in the tests that reads its time [synthetic].

The attack ladder: one saw note with a linear attack, read attack in ms, bound for an upper bound [synthetic]:

Every attack over 4 periods reads within 15 %, and every one under 1.5 periods reads as a bound [synthetic].

Hold, decay and sustain

• hold_s is the time from the first frame within 1 dB of the peak to the start of the decay, 0 when under 10 ms [fact, proposal for 10 ms].
• decay_s is the time between the first frames after the peak at which the level falls through 10 % and 90 % of the sustain level in dB, linear between frames, divided by the gap between the two shares [fact]. The first share is at least 0.5 dB down, so ripple at the peak does not end the hold, and it rises above 10 % for a sustain shallower than -5 dB [fact]. The decay starts that share of its time before the first share’s crossing [fact]. A regression in dB from the peak read the decay 6 to 22 % long, since the knee at the sustain level bends the line [synthetic].
• sustain_db is the median of the last 30 % of the held part [fact]. The held part runs from the first frame within 1 dB of the peak to the note-off knee, or to 25 ms before the note’s end without one [fact].
• Decay and sustain are null when a note with a knee holds for less than the attack, the hold and three decay times [fact]. A note without a knee, such as a pluck, sustains at its end level [fact].
• fit_residual_db is the median over the pool of the RMS in dB of each held part against the model: 0 dB over the hold, a fall linear in dB to the sustain level over the decay, then the sustain level [fact].
• ripple_db is the median, over the pooled notes whose sustain is observable, of the 90th minus the 10th percentile of the sustain window after its line [fact]. Unison and chord beating show here [fact].
• A Levenberg-Marquardt refine of a piecewise linear model over the pooled dB curve, with a floor, a peak and a fixed note-off, is not applied [fact]. Fitted from each note’s top with the peak free, it moved the decay error on 72 lines with decays of 50 to 600 ms from a median of 4.4 % and a 90th percentile of 8.7 % to 9.9 % and 26 %, and left the median sustain error at 0.24 dB [synthetic]. A fit in dB trades the two ends of a fall against each other, which the read at 10 and 90 % of the fall does not [fact].

On a grid of 240 lines of eight saw notes, decays of 50 to 600 ms, sustain levels of 0 to -24 dB, holds of 0 and 30 ms and releases of 20 to 300 ms [synthetic]:

A decay linear in amplitude starts slowly in dB, so its first 0.5 dB reads as hold: up to 104 ms with no true hold [synthetic]. The fit assumes a decay linear in dB, and its residual rises on the other law [fact].

Plucks on 55 and 220 Hz with an exponential decay and no note-off knee sustain at their end level: -57 dB for a time constant of 150 ms, -21.5 dB for 400 ms and -8.6 dB for 1 s [synthetic].

Note-off knee

The knee is the time the note-off starts the release [fact]. It is read on the linear amplitude of the amp proxy, from the first frame within 1 dB of the peak to 25 ms before the note’s end [fact]:

• At each split, a line over up to 100 ms before it and one over up to 200 ms after it are fitted against one line over both [fact]. The right line stops where the level falls under 2 % of the mean before it, so silence after the release does not flatten it [fact].
• A split counts when the right line is steeper by half the mean before it per second, falls by half that mean over its side, and the two lines leave at most half the squared error of one [fact, proposal for the shares].
• Splits that count in a row form a corner, and within a corner the split with the largest gain wins [fact]. A hold that breaks into a linear decay forms an earlier corner; the largest gain over all splits took it for the note-off [synthetic].
• A later corner replaces an earlier one unless it looks like a fade of a reverb tail, whose fades form corners too [fact]. Taking the last corner put a knee over 1 s past the note-off [synthetic]. Two tests tell a fade from a note-off [fact, proposal for both]:

  • The level between the corners bends: the median size of its second difference in dB, 10 ms each way, is over 0.4 dB. A decay and a sustain are near 0 dB, as both are straight in dB, however short the sustain.

  • The level after the corner rises by more than a twentieth of the mean before it. A note-off falls for good. The rise reads the first 20 ms as they are and the mean over every 20 ms span, since a noise floor wobbles over the silence after a release [fact, proposal for 20 ms]. The rise and the read of the fall stop before a next note that overlaps, by the frames that the band’s filter rings ahead of its attack, and half a level window before the end of a note that overlaps no other [fact]. A later corner past the stop reads no rise, so it replaces an earlier one whenever the bend allows it [fact]. That keeps the note-off of a note held over a shorter one [synthetic].

  • A later corner past the stop replaces nothing when the note has ended and the level has come back [fact]. The note has ended when the mean over the last 10 ms before the stop lies under a fiftieth of the earlier corner’s level, -34 dB, the floor of the release side. The level has come back when the mean over the 100 ms before the later corner lies at least a quarter above it, and the left line’s end lies above it too [fact, proposal for the quarter]. The corner is then the next note’s. Without this test, a long gap after a note-off took the next note’s corner, since the bend over silence reads near 0 dB (#312) [synthetic].

  • Each read guards one case [synthetic]. A held sustain under the floor, with its note-off past the stop, reads a mean near 1 times the stop level: a ripple of 1 % or a rise of 1e-6 a frame must not decide it, so the mean must lie a quarter above. The 10 ms mean at the stop keeps a ripple of 25 % over 10 frames from deciding it too. A transcribed note on the generated tally rises over the last 10 frames into its note-off: its line ends above the stop level, while its mean reads 1.08 times it. A note that falls straight into a note-off past the stop has a mean over a quarter above, while its line ends under it. A next note after a silent gap reads mostly 1 to 3 times the stop level on the bass line below. A quarter keeps 8.9 % of the long notes late under the pump there, against 7.1 % at a tenth and 13.0 % at a half.

  • Under a pump that strips, the test also reads the note’s level as it was before the strip, and the later corner replaces nothing when either level shows the note ended and the level come back [fact]. The strip lifts the frames in a beat’s dip by up to the pump’s depth, and the last 10 ms before the stop can lie in a dip while the next note’s band already rings ahead of its attack. Read on the stripped level alone, the stop then lay over a fiftieth of the hold, or the next note’s corner lay under a quarter above the stop, and the knee took the next note’s corner (#323) [synthetic]. On the long notes of the bass line below, the strip put 156 knees over 15 ms further off than the same signals read unstripped, by a median of 0.56 s [synthetic]. Read on the level before the strip alone, the ring in the dip can come back by under a quarter there while the stripped level comes back by more: under a 12 dB pump with list ends 5 ms past the next onset, 2 of 2137 notes then read about 0.5 s late [synthetic].

  • Notes on the signals that strip, against the same signals generated dry, mean error and notes over 150 ms off [synthetic]:

    Pump, notes, list end past the next onset	Notes	Dry	Test on the stripped level	This rule
    5 dB, 1.37 to 2.74 s, 20 to 200 ms	2757	40.8 ms, 148	66.2 ms, 249	33.9 ms, 117
    5 dB, 0.34 to 1.37 s, 20 to 200 ms	2818	10.9 ms, 35	14.5 ms, 53	10.4 ms, 28
    5 dB, 1.37 to 2.74 s, 5 ms	2748	17.9 ms, 56	24.3 ms, 83	20.1 ms, 63
    5 dB, 0.34 to 1.37 s, 5 ms	2854	5.8 ms, 11	8.0 ms, 17	7.3 ms, 13
    8 dB, 1.37 to 2.74 s, 5 ms	2065	19.8 ms, 47	29.4 ms, 74	22.7 ms, 53
    12 dB, 1.37 to 2.74 s, 5 ms	1913	17.9 ms, 39	37.9 ms, 86	23.9 ms, 45
    24 dB, 1.37 to 2.74 s, 20 to 200 ms	1885	41.3 ms, 101	314.3 ms, 853	96.5 ms, 253

    Only the 5 dB pump with list ends 20 to 200 ms past the next onset reads no worse than dry. With list ends at the next onset, no note changes. No note reads over 15 ms worse than the test on the stripped level, except one at 24 dB, which reads 0.43 s early against 0.40 s late: there the stripped track holds no corner at the note-off, and both knees take another corner [synthetic]. The strips do not change [synthetic].

• When the level stays at 85 % or more of the left line’s end for over 10 ms more, the knee is where a line over its fall from there to half that level crosses that level [fact, proposal for 85 %]. A hold that wobbles under a reverb can win a split 50 to 100 ms before the note-off, and an exponential release can win one past it [synthetic].
• Of the eval’s knees within 15 ms, a sustain shorter than its decay reads 120 of 144 and a dry tremolo with a release of 200 ms or more reads 12 of 64, against 108 and 9 at commit 057d63c, which took the last corner [synthetic].
• The knee is where the left line crosses a line over the first 10 ms after the split, or the split itself when the crossing lies outside the two lines [fact]. A step down followed by a slow fall crossed 20 ms before the search started [synthetic]. The crossing of the two fitted lines read a linear release late, since the right line averages the release’s tail [synthetic].
• No split counts on a pluck that decays at one rate, and its knee is null [fact].

note_off_offset_s is the note’s end in the note list, or the stem’s end when that comes first, minus its knee, over the pooled notes with a knee [fact]. On the grid, all 1920 knees lie -2.0 to +0.01 ms from the true note-off [synthetic]. Read in dB, a 200 ms linear release placed the knee 168 ms late, because a linear fall in amplitude bends in dB [synthetic].

Release and tail

After the knee, relative to the median level over the 50 ms before it, or since the note’s onset when that is nearer, the level follows a linear release plus a decaying tail [fact]:

env(t) = max(1 - t / release_s, 0) + tail_weight * 10^(-3 t / tail_rt60_s)

• The fit is a simplex on the squared error in dB, from 4 releases and 3 RT60s as starts, down to -55 dB [fact]. It keeps every frame for 200 ms and every fifth after that, weighted to count the same [fact]. The project has no least-squares crate, so it uses the simplex of the pitch fits [fact].
• The fit agrees with scipy.optimize.least_squares on the curves of the golden vector tail-fit, within 1 % and 2 ms or 0.002 dB [synthetic]. cargo xtask golden regenerates it [fact].
• It reads the one-period RMS, not the amp proxy: a reverb of one harmonic fades deeply, and many harmonics fade apart [synthetic]. A note that overlaps another reads the band of the amp proxy [fact].
• It stops at the note’s end or the next note that overlaps it, half a level window before that note [fact]. At 41 and 55 Hz that window otherwise reads the next note and left residuals of 6 dB on dry lines [synthetic].
• A note with no knee has no fit, so a pluck reports no release [fact]. A given note list that ends at the note-off shows no knee and reads no release [fact].

The write rule reads the medians over the pooled notes with a fit, and never the release, which is what fails to fit [fact]:

• A tail_weight over 0.5 or a residual over 3 dB writes no release_s, and release_confidence reads low [fact, proposal for 3 dB]. Both still report the tail [fact].
• A written release_s reads low, with tail_wobble, when the median residual is over 2 dB and the median second difference of the level in dB over the 300 ms before the knee, 25 ms each way, is over 0.3 dB [fact, proposal for all three]. A tremolo carries on through the release, which the fit cannot follow [synthetic]. Of 2000 dry tremolos of 0.3 to 3 dB at 1 to 12 Hz, 727 read medium and 139 of them lay over 50 % off before this rule. It lowers 96 of the 139 wrong ones and 141 of the 588 right ones, which leaves 43 of 490 [synthetic]. On 2000 stems each it lowers 18 wrong and 11 right short sustains, and 3 wrong and 3 right reverbs, 1 wrong and 3 right on the first 1000. It lowers none of the plain dry or legato stems and 2 wrong chords [synthetic]. The rest of the move in the reverb figures below comes from the knee rule [synthetic]. Its effect depends on the recipe: another set of generated tremolos, with a wrong share of 23 %, lost 66 of its 637 right and 63 of its 189 wrong medium releases, for a wrong share of 18 % [synthetic].
• tail_rt60_s reads over the notes with a tail_weight of 0.05 or more whose frames span a 15 dB fall of it, and is null, with tail_light, under a median tail_weight of 0.05, because a dry note often leaves a little weight whose RT60 is noise [fact, proposal for both].
• tail_rt60_s is null, with tail_smooth, when the median tail_weight is over 0.5 and the median residual is at most 1.1 dB [fact, proposal for 1.1 dB]. A dry exponential release fits the tail term alone, at a weight of about 1 and an RT60 equal to its fall, while a reverb’s level wobbles [synthetic]. Of 15 dry falls of 60 dB over 0.1 to 1.5 s at 55 to 220 Hz, 14 read this way, and the fall of 0.1 s at 55 Hz stays a false RT60 at a residual of 2.7 dB [synthetic]. A curved release model waits for the calibration of the decay law under #69 [fact].
• tail_rt60_s is null, with tail_band, when a note that overlaps another reads a median RT60 under 6 periods of its narrowest band’s cutoff [fact, proposal for 6]. The band rings by itself: a dry chord with a release of 20 or 50 ms read 0.13 to 0.2 s on 18 of 18 stems before this rule and on none after it [synthetic]. The same chords through a reverb read an RT60 on 13 of 18 stems at a true 0.3 s and 36 of 36 at 0.6 and 1.5 s [synthetic].
• release_confidence is null when no pooled note has a fit [fact].

The set: 4000 stems sampled over 41 to 440 Hz, releases of 20 to 800 ms, dry or with an RT60 of 0.3 to 3 s at 0.05 to 0.8 wet, three timbres, 1 to 6 notes of 0.3 to 1.2 s, gaps of 0.2 to 1.5 s, sustains of 0 to -12 dB, mono or wide reverbs, and a reverb whose highs fade first [synthetic]. Shares hold within about 2 percentage points at this size. On it:

• The median written release over its truth on dry stems is 0.98 to 1.12 by release, with the 1.12 at 20 ms [synthetic].
• Over the true RT60, the median read is 1.16 at 0.3 s, 1.02 at 0.6 s, 0.95 at 1.5 s and 0.95 at 3 s, and 33 to 44 % lie within 10 % [synthetic].
• tail_weight over 0.5 holds out 4.8 % of stems at a true 0.05 wet, 30 % at 0.4 and 55 % at 0.8 [synthetic]. Of the 661 reverb stems over 0.5, 79 read as smooth and lose their RT60 [synthetic].
• A knee lies within 15 ms of the note-off in 98.8 % on dry stems and 69.6 % under a reverb, against 98.8 % and 42.1 % at commit 057d63c [synthetic].
• On a line of 6 notes with a release of 50 ms at 0.15 wet, over 60 noise reverbs each, the RT60 lies within 10 % on 53 of 60 at 0.6 s and 56 of 60 at 1.5 s, with misses from 0.529 to 0.692 s and from 1.289 to 1.768 s, and the release reads 43 and 44 ms [synthetic].

Votes that a tail is reverb

tail_votes counts five signs, each over the pooled notes with a fit and by majority where it reads notes, and null where nothing could be read [fact]. All thresholds are proposals [fact].

A dry stem reads none of the first three, as its tail ends with the release [synthetic]. Two or more votes pass on none of the 1372 dry stems and on 68 % of the 2628 under a reverb [synthetic]. A noise floor reads as a tail: on dry lines at 55 to 220 Hz under noise at -50 and -40 dBFS, 54 of 54 stems pass two votes [synthetic]. tail_is_reverb stays false: a reverb is stated only when the votes agree and a space layer reads medium or better, and no space layer exists yet [fact].

Pooling and confidence

• The pool leaves out a note whose peak lies 12 dB or more below the peak of a note it overlaps: a quiet second voice, such as another part’s bleed or a ghost note, has an envelope of its own [fact, proposal for 12 dB]. Eight A2 notes decaying over 300 ms, each over an E3 18 dB quieter decaying over 50 ms, read the A2 decay within 20 % with the gate; the E3 notes would otherwise fill half the pool [synthetic].
• Of the other notes, the pool holds those whose length to the knee, or to the end without one, lies within a factor of 2 of their median length, and whose pitch lies within 12 semitones of their median pitch [fact, proposal].
• Each statistic is the median and the interquartile range over the pool [fact]. It reads low over fewer than 3 notes, or with an interquartile range above half its median [fact]. Its reasons are too_few_notes and relative_spread, and the attack adds upper_bound_share [fact]. A median of 0 under an interquartile range above 0 has no ratio, so the statistic reads low with zero_median, that range, in place of relative_spread [fact].
• level_spread_db is the 90th minus the 10th percentile of the full-band levels of the pooled notes, null under 3 pooled notes with a level and always low: no stem has tested the rule [fact, proposal]. A note’s full-band level is the peak of the centred RMS of the mono mix over one period of its fundamental, on a 1 ms hop, from its onset to its note-off knee, or to 50 ms before its end in the list without one, never past the next later onset on a mono line [fact]. A mono line is one whose voices layer reads at most one voice at a time, or counts none, when --profile pad did not run the notes step; so the layer reads the levels after the voices layer [fact]. The amp proxy’s peak_db reads one harmonic per note, so two notes on different harmonics do not compare; the full band does (owner decision, 2026-11-05) [fact]. Notes 0, 3, 6 and 12 dB apart read 0.1, 3.0, 6.0 and 12.0 dB [synthetic]. On the generated chords of equal notes, the amp proxy read 4.6 to 7.8 dB and the full band reads 0.9 dB [synthetic]. On a chord the full band is the mix, so it cannot tell two chord tones of different levels apart [fact].
• The group’s own confidence is medium with 3 or more pooled notes, else low, and no estimate when no note was read [fact].

Sidechain pump

A sidechain compressor keyed by a kick ducks a held part on every beat and lets it recover until the next [fact]. The layer reads the dip on the beats that one held note covers, and strips it before it reads the notes [fact].

• Track. The full-band centred RMS in dB over two periods of the lowest note, on a 1 ms hop [fact].

• Beats. A beat of the grid counts when the held part of one note covers it from 50 ms or more before its start to its end [fact]. A pluck struck on the beat never counts, nor a release at the end of a beat [fact]. The held parts come from a first read of the notes without the pump, and the notes are read again with it stripped [fact]. Plucks on every beat read 13.9 dB of ducking without this rule [synthetic].

• Curve. Each counted beat is binned into 64 phase bins and taken relative to the 90th percentile of its bins; the curve is the median over the beats [fact]. The pump needs 4 counted beats; with fewer it is null [fact].

• Period. A sidechain follows the song’s tempo, and a stem’s tempo may be off by 1 or 2 % [fact]. Over 32 beats a 1 % error moves the duck a third of a beat, so its beats no longer fold onto one curve [fact]. The layer searches periods within 2.5 % of the grid’s for the one whose counted beats fold onto the deepest mean curve, over up to 64 beats of the longest run of counted beats and then over all of them, in steps that move the last beat of the span by 1 bin [fact, proposal for 2.5 %]. The second stage searches one step either side of the first, so the reach follows the longest run of counted beats: up to 2.73 % at 4 beats, 2.81 % at 5, 2.60 % at 6, 2.68 % at 7, 2.66 % at 10, 2.58 % at 20 and 2.51 % at 64 [fact]. Of 72 ducks up to 2 % off the grid over 32 and 64 beats, 15 locked and stripped on the grid alone, and all 72 do with the search [synthetic]. pump_bpm reports the tempo of the read that stays [fact]. The search’s read must lock to the beat, and it replaces the grid’s read as follows [fact, proposal for 0.9, 0.15 dB, 60 cents, 3 bins and half a window]:

  • The grid’s read follows the notes: always, with no shape test.
  • The grid’s read locks to the beat: when both reads strip or neither does, the search’s fold error must be under 0.9 of the grid’s and 0.15 dB lower. The fold error is the standard deviation of the level minus the curve over the counted beats, each beat about its own mean, so a fade or a level step between beats does not count as misfit [fact]. When only one read strips, as a slower period counts fewer beats, the search’s read must be the one that strips and have the lower fold error. A 12 dB duck 2.5 % off over 16 beats reads 4.4 dB on the grid and 9.8 dB at its own period [synthetic]. Depth is no test: the search deepens a duck on the tempo by fitting a chord’s beating, and on the flat level of the 76,836 stems of Limits a gain of 1.5 dB moved 15 ducks on the tempo, against 3 with this rule [synthetic]. A sub’s fold error is small enough that a share alone moved ducks on the tempo at 174 BPM, and the sustain with them by up to 2.6 dB [synthetic].
  • The grid’s read runs free: when the line lies within 60 cents of a division, and the search’s curve falls faster than it recovers and takes long enough from half its depth back to a quarter: 3 bins, and half the level window in bins. A gate near a division folds onto one curve as well, but comes back at once, within the smear of the window [synthetic].
  • The grid’s read reads none: when the search’s curve falls faster than it recovers and takes 3 bins from half its depth back to a quarter. A gate off the tempo often reads no lock on the grid, and the level window’s share is left out because it costs real ducks under a low note; see Limits [synthetic].

  The set behind the numbers below holds 123,840 generated signals: a three-note chord from 55, 110 and 220 Hz, 60 to 174 BPM, runs of 8.9 to 32 beats, four phases for ducks and two for gates and saw tremolos. Each cell counts the signals that strip, from one build. The columns differ only for a grid read that runs free: it stays free, it takes any search read that locks, or it takes one under this rule. A free read never strips, so this rule reads no more than the column before it [synthetic].

  Signal	Off the tempo	Signals	Free read stays	No free rule	This rule
  Duck, release 80 ms or more	up to 2.5 %	47520	36008	36107	36078
  	3 %	8640	1343	5028	4883
  	3.5 %	8640	169	4132	1716
  	4 to 5 %	17280	122	4717	128
  Duck, release 30 ms	up to 2.5 %	15840	7440	7462	7453
  	3 %	2880	622	801	734
  	3.5 %	2880	426	612	459
  	4 to 5 %	5760	160	287	160
  Square gate	3 %	2880	21	768	25
  	3.5 %	2880	42	667	47
  	4 to 5 %	5760	14	603	14
  Saw tremolo	3 %	720	71	498	498
  	3.5 %	720	0	352	148
  	4 to 5 %	1440	0	293	0

  The first read’s strips on gates come from their own lock on the grid. A gate up to 2.5 % off strips either way; see Limits [synthetic].

• pump_depth_db is minus the curve’s minimum [fact].

• pump_recovery_s is the time from the minimum until the curve comes back within 1.5 dB, going on over the beat’s wrap, so a duck that sits late in the beat recovers into the next [fact]. It is null when the curve does not come back within one beat [fact].

• pump_tau_ms is the recovery over $\ln (\text{depth}/1.5)$, the time constant of an exponential recovery in dB, and null unless the pump locks to the beat: a dip that follows no beat has no recovery to model [fact].

• pump_beats counts the beats behind the curve [fact].

pump_locked_to names what the dip follows, tested in this order [fact, proposal for the thresholds]:

1. note: the beats with a note onset inside dip 3 dB or more, and the others under 1.5 dB, each group of 2 beats or more. An onset up to 50 ms before a beat belongs to that beat, as the transcriber places onsets a few milliseconds early [fact].
2. free: the level swings 1 dB or more at a rate of 0.5 to 20 Hz that lies 50 cents or more from every note division, read on the modulation spectrum over the longest covered span [fact]. It comes before the beat: a 5.3 Hz tremolo over 17 beats at 128 BPM left a 3.6 dB dip in the median curve that passed the beat test [synthetic]. A duck’s strongest line lies on the beat rate, so it reads free when that rate lies 50 cents or more from every division of the grid [fact]. That holds 3 % off the tempo, and over a short span from 2.5 %, where the line reads 50 to 52 cents [synthetic]. The period search then decides, as above [fact].
3. beat: the curve dips 3 dB or more in one run of bins half its depth down, and falls into its dip within 8 bins, reading on into the beat before and not round the beat’s wrap. The odd beats and the even beats must each do the same on their own median curves, to half its depth at its dip. Half the beats must fall by half its depth into its dip: from their highest bin over the 8 bins before the 3 bins either side of it to their lowest within those. A note that decays across the beat falls over most of it, and fails the 8 bins [fact]. The one run stops a tremolo at a division of the beat [fact]. A level that rises over each bar or two beats and drops back makes every beat’s curve start low, and round the wrap that read as a fall on every beat: 12 dB per bar and 6 and 12 dB per two beats all stripped [synthetic]. Those falls only come on the first beat of each bar or pair, so the rule reads no pump there [synthetic].
4. none otherwise.

The layer strips the curve from the note tracks only when it locks to the beat over 8 counted beats or more, and sets pump_stripped [fact, proposal for 8]. A stem can duck in one section only, so a note takes the strip only when the median fall of the beats over it reaches half the curve’s depth [fact]. With fewer beats, half of them is two or three, and other sounds on the beat pass the lock: a sine sub under a kick on every beat, whose tail beats against the sub, read 3.4 to 4.6 dB locked to the beat over 4 or 5 beats [synthetic]. On 1375 runs over the generated stems of the pitch layer, which hold no duck, 4 read such a lock, and none is stripped [synthetic].

A duck’s dip can read as a note-off knee in the first read and cut the held parts short [synthetic]. A read of the held parts that finds no pump, or one that locks to the beat over fewer than 8 beats, is therefore counted again over each note to 25 ms before its end, and strips when that count locks to the beat over 8 beats or more [fact]. A 6 dB duck on the chord at 174 BPM under noise at -20 dB read a false knee on each of its three notes and was not stripped; counted over the notes, it strips, and no knee remains [synthetic].

A constant three-note saw chord reads 1.64 to 1.94 dB from its own beating, so the 3 dB floor sits above it [synthetic]. Over all 1191 parts, all 826 locks read as built [synthetic]. A part with no pump reads as no lock, and a 3 dB duck counts as right when it locks to the beat or reads no pump, never when it reads free [fact].

Depth bias

The read depth is shallower than the duck, and the bias depends on the duck’s shape [synthetic]. A duck whose gain recovers exponentially, $1-(1-g_0)e^{-t/\tau }$, has a sharp minimum in dB; the RMS window and the 64 bins round it off [fact]. A cosine dip has a round minimum and reads near its depth [synthetic].

Measured minus true depth on a held saw chord and a held 55 Hz sub, two seeds, noiseless and with white noise at -30 and -20 dB, releases of 50, 80, 150 and 300 ms; median and range over 48 parts per cell [synthetic]:

• No duck reads 0 to 0.09 dB on the sub, and 1.64 to 1.94 dB on the chord from its beating [synthetic].
• Noise at -20 dB fills the dip: a 12 dB duck reads 1.5 dB shallower than without noise [synthetic].
• At 128 BPM on the noiseless chord, the bias over the releases is -0.2 to -5.5 dB for a duck linear in dB, -0.3 to -7.2 dB for one with a 5 ms fall, and +1.2 to -2.6 dB for a cosine dip [synthetic].
• The bias changes with the shape at the same depth, so no single factor corrects it, and the layer reports the depth as read [fact].

The recovery over the true time back within 1.5 dB, at depths of 6 dB or more, is 0.64 to 1.44 by release on the chord and 0.55 to 0.80 on the sub [synthetic]. The shallower read reaches 1.5 dB sooner on the sub, and the chord’s beating moves the crossing [synthetic].

Limits

• An attack under about 1.9 periods of the fundamental is an upper bound, and at 41.2 Hz that covers attacks up to 45 ms [fact].

• A decay linear in amplitude reads part of its start as hold [synthetic].

• The upper note of an octave or a unison has no free harmonic and is skipped [fact].

• A resonant filter that closes onto the fundamental changes the amp proxy itself [fact].

• A release that falls, flattens and falls again reads its knee at the second fall, since a later corner replaces an earlier one to pass over a decay [synthetic]. Its sustain window then reaches into the first fall [fact].

• A release over about 400 ms shows no knee, because its right line must fall by half the held level over 200 ms: no dry 800 ms release reads [synthetic]. Under a reverb, 28 of 401 such stems wrote a release, 4 of them within 50 %, from a knee in the tail [synthetic].

• Under a reverb, 30.4 % of the knees on the 4000 stems lie over 15 ms from the note-off [synthetic]. A fade in the tail with a smooth level before it and no rise after it still replaces the note-off [fact].

• A tremolo bends too: a swing of 1 dB at 12 Hz reads 0.38 dB and 6 dB at 5 Hz about 0.4 dB. When an earlier decay forms a corner, the note-off of such a sustain can lose to it [fact].

• The bend and rise rule loses some notes that commit 057d63c read within 15 ms, on a one-off sweep of 1000 generated stems a set, and gains more [synthetic]. Commit 2d63f2e holds that rule with its stop, and the last column adds the test of the level at the stop. Worse and better count the notes of the last column against 057d63c:

  Set	Notes	057d63c	2d63f2e	This rule	Worse	Better
  Plain dry	2986	2453	2453	2453	0	0
  Short sustain	2698	2017	2064	2064	0	47
  Tremolo	2991	732	777	777	107	152
  Legato	3011	2962	2967	2967	0	5
  Reverb	3042	1220	1662	1662	25	467
  Chord	1176	820	841	861	6	47
  Noise at -35 dBFS	2898	2010	2065	2065	82	137
  Noise at -40 dBFS	2898	2200	2215	2215	33	48
  Noise at -50 dBFS	2898	2286	2315	2315	0	29
  Noise at -60 dBFS	2898	2306	2327	2327	0	21
  Legato, note end 10 ms past the next onset	1583	1194	1201	1222	0	28
  Legato, note end 20 ms past the next onset	1583	1220	1224	1229	1	10
  Legato, note end 40 ms past the next onset	1583	607	662	1091	0	484

  No note of any set reads worse than at commit 2d63f2e [synthetic]. Every column comes from the same sweep, run at each commit. The three legato rows list every note up to the next onset and the time named, on the legato notes of the plain set. They replace rows read on an earlier version of the sweep, which this one does not reproduce at commit 057d63c: 1227, 1172 and 1015 there.

• On the same sweep, 600 stems a row and 964 notes, a note list that runs further past the next onset [synthetic]:

  List end past the next onset	057d63c	2d63f2e	Worse	This rule	Worse
  120 ms	378	412	0	639	0
  220 ms	386	426	0	675	0
  420 ms	387	431	0	669	0

  Worse counts against 057d63c. These rows replace a table read on an earlier version of the sweep, which put 604, 609 and 589 at commit 057d63c. The rows without the stop are not measured again.

• Other generated recipes still read knees over 15 ms worse than commit 057d63c [synthetic]:

  Recipe	Notes	Worse	Better
  Mono bass line at 175 BPM, 14 notes a signal, under a 5 dB pump left unstripped, see #313	3570	11	2652
  The same line, medium notes, see #313	3530	31	2401
  A tremolo under a reverb	1500	42	270
  A next note 15 dB louder	1419	3	-

• On that bass line, with a gate of 55 to 90 % and list ends 20 to 200 ms past the next onset, over 500 signals, knees over 15 ms late [synthetic]:

  Notes	057d63c	2d63f2e	This rule	Stripped notes tested before the strip
  1.37 to 2.74 s, under the pump	81.4 %	53.9 %	8.9 %	5.2 %
  0.34 to 1.37 s, under the pump	61.4 %	11.6 %	4.2 %	3.5 %
  1.37 to 2.74 s, dry	80.9 %	53.5 %	5.4 %	5.4 %
  0.34 to 1.37 s, dry	60.8 %	11.3 %	3.2 %	3.2 %

  Both pump rows above and these rows come from this line. Notes count where both commits read a knee. The last column tests a stripped note’s level before the strip, see the rule above (#323). The long-note row replaces one read on another version of the line, 3038 notes with 44 worse, which this one does not reproduce.

  Dry, no note reads worse than at commit 2d63f2e. Under the pump, 25 long and 25 medium notes that commit 2d63f2e read right leave the 15 ms, 19 and 15 of them early [synthetic]. Of these, 21 and 13 lie in signals whose pump read changes: the pump strips on 42 and 7 fewer of the 500 signals, and on 21 and 4 more [synthetic]. The pump read counts beats over the held parts that the first read’s knees bound, so a knee that moves changes the parts it counts, see #313 [fact].

• A next note’s band rings ahead of its attack, and the 10 ms before the stop can read that ring over a fiftieth of the hold, with or without a pump [synthetic]. Then the knee takes the next note’s corner: on the long dry notes of that bass line, 188 of 3570 read over 150 ms late, and on long stripped notes 117 of 2757 [synthetic]. Read as the lowest 10 ms since the first corner, the stop cut the dry ones to 0, but lost the note-off of a filter that closes past the band’s harmonic before a next note that starts at the note-off, on 2 of the 1405 tally runs, 0.57 s early [synthetic].

• A sustain under -34 dB of the earlier corner’s level, with its note-off past the stop, loses that note-off to the decay corner when its level comes back without a next note: the knee reads 0.46 to 0.8 s early, where commit 2d63f2e reads it [synthetic, from generated probes]. This happens in two shapes:

  • A rise of about 1.3 times, 2.3 dB, or more over the last 100 to 400 ms before the note-off. The rise that loses it is 2.0 times over 60 frames, 1.5 over 100, 1.4 over 200 and 1.3 over 400.
  • A swell like a tremolo of ±15 % or more at 2.5 Hz or slower. At ±25 % over 400 frames, 12 to 20 of 96 cells lose it with the stop 150 to 200 frames before the note-off; at ±15 % over 400 frames, 24 of 96 with the stop 200 frames before.

• A next note at 1.4 to 1.7 times the level of a silent gap still takes its own corner, as commit 057d63c and 2d63f2e do [synthetic]. On a gap at 0.01 with the stop at its end, it forms a corner where its level flattens, which sees its release within 200 ms and whose 100 ms mean reads about 1.17 times the gap. That corner replaces the note-off, and the next note’s own corner then replaces it in turn. At 1.3 and at 1.8 or more the note-off holds.

• A medium release lies over 50 % off on 50 of the 875 reverb stems, 26 of them with a release of 20 ms, and on 23 of the 779 dry stems, all with a release of 20 ms, on the 4000 stems [synthetic]. Lowering every reverb release with a true vote or a median tail_weight of 0.05 would leave 2 of 8, and would also lower the 819 right ones, so the level stays [synthetic].

• Under a tremolo, 43 of the 490 medium releases lie over 50 % off, on 2000 dry stems, after the rule above [synthetic].

• The RT60 reads about 50 % high under a release of 400 ms: at a true 0.6 s and 0.15 or 0.4 wet the median is 0.90 s, on about half of 300 stems each [synthetic]. The space layer takes this up [fact].

• A dry stem can report an RT60, on 2000 generated stems a set [synthetic]:

  Set	With an RT60
  Plain dry	82
  Short sustain	211
  Tremolo	330
  Legato	25
  Chord	185

  The 4000 stems above read it on 55 of 1372 dry stems, so that 4 % is no general rate [synthetic].

• A note list that ends at the note-off shows no knee, so --notes with such a list reads no release [fact].

• The 3 dB residual, the 0.05 weight of an RT60 and the five vote thresholds were chosen on generated signals and are untested on real tails; #91 checks them [fact].

• A release of 20 ms at 41 Hz is shorter than the level window: of the 116 reverb stems written more than 50 % off with a release up to 400 ms, 64 have a release of 20 ms [synthetic].

• #70 asks that the fit never returns a confident release more than 50 % off. Read with its scope, where the fit reports a tail_weight over 0.5, it holds, because no release is written there. It does not hold unscoped: the medium releases above lie over 50 % off under that weight [synthetic].

• The fit has no echo term, so a delay reads as release or tail [fact].

• The bench has no pre-delay, and the reverb is noise [fact].

• The beat grid starts at 0 s with the tempo as used. The period search finds one constant tempo within up to 2.81 % of it; a tempo that moves inside the stem still smears the curve [fact]. Its first stage reads up to 64 beats of the longest run, so a duck off the tempo that starts later in one held note can go unfound [fact]. On the 110 Hz chord, over runs of 10, 12, 16 and 32 beats and four phases, up to 2.5 % off the tempo, of 176 cells at each tempo [synthetic]:

  • A 12 dB duck with an 80 ms release locks in 173, 175, 176, 176 and 173 at 60, 80, 96, 128 and 174 BPM.
  • A 6 dB duck with a 150 ms release locks in 172, 165, 135, 159 and 131 at 60, 80, 96, 128 and 174 BPM. It locks in 8 of 16 on the tempo at 96 BPM, and in 1 of 16 at 2.5 % above at 174 BPM.

• A read that stays on the grid reports the grid’s tempo, though the duck can lie off it [fact]. Of 11,161 ducks up to 2.5 % off that stayed, the tempo was off by a median of 0.99 % and at most 2.56 %; of 33,787 fitted reads, by a median of 0.08 % and at most 2.22 % [synthetic].

• A duck on the tempo can move from a grid lock. On the 123,840 signals, 1 of 4847 grid locks does, 6 dB at 110 Hz and 60 BPM, by 0.89 %. The 76,836 stems, read through the layer, hold runs of 8.9 to 32 beats, ducks of 6, 12 and 18 dB and 4 phases, on a flat level, under fades of 12, 18 and 36 dB and under 6 dB steps every 4 beats. There 3 of 4595 grid locks move, by up to 0.89 %: 3 on the flat level and 0 under each fade or the steps. A wider set reaches more: 9,600 stems on the tempo per level, from 3 pitches and 5 tempos, with 10 runs of 8.9 to 32 beats, 8 duck settings of 3 to 18 dB and 8 phases. Its grid locks move in 6 stems on a flat level, 7, 12 and 10 under those fades, and 0 under the steps. These are ducks of 6 dB or less over 8.9 to 12.3 beats, by up to 2.7 %. A 6 dB, 150 ms duck at 110 Hz and 60 BPM over 10 beats, under the 36 dB fade, reads 59.479 BPM [synthetic].

• A grid read of none takes a fitted beat lock only when its curve falls faster than it recovers and takes 3 bins from half its depth back to a quarter, which holds out most gates. The price in ducks is small for releases of 80 ms or more and large at 30 ms, which loses 2004 of 6406 below. The set: 38,400 ducks on the tempo, from 3 pitches, 5 tempos, 10 runs of 8.9 to 32 beats, 8 duck settings of 3 to 18 dB and 8 phases; 46,080 ducks 0.5 to 2.5 % off, of 6, 12 and 18 dB with releases of 80 and 150 ms; 13,824 gates; flat and under fades of 12, 18 and 36 dB. Counted on this path alone [synthetic]:

  Fitted read over a grid read of none	Moves on the tempo	Ducks off the tempo that strip	Gates that strip
  No test	839	16,427	1733
  This test	653	16,150	59

  • The lost ducks are 277, or 1.7 %. Of them 132 lie at 55 Hz, 117 have a release of 80 ms and 160 of 150 ms [synthetic].
  • Short releases lose more, and slow tempos with them, as the test asks for 3 bins of a beat. On the 123,840 signals, on this path, ducks that would strip lose 2004 of 6406 at 30 ms, 61 of 4773 at 80 ms, 7 of 4079 at 150 ms and 2 of 3976 at 300 ms, and 998 of 4065 at 60 BPM, 523 of 3649 at 80, 326 of 4930 at 96, 141 of 3422 at 128 and 86 of 3168 at 174. Over all ducks of 30 ms up to 2.5 % off, 7453 of 15,840 strip, against 8996 with no test. A 12 dB, 10 ms duck over 480 beats does not strip [synthetic].
  • The moves left are 118 on a flat level and 181, 177 and 177 under the fades, by up to 2.46 %. There the duck is real: a move mostly shows that the grid read found no lock [synthetic].

• A chord whose level wanders with no pump can read as a pump. Under linear interpolation of Gaussian dB knots, spaced 0.25 to 1 beat, with sigmas of 1, 2 and 4 dB, at 55, 110 and 220 Hz, 60 to 174 BPM and 9, 16 and 32 beats, 5 of 2,592 strip, at 3 to 4 dB, 4 of them at a sigma of 4 dB [synthetic].

• A level step inside a beat stays in that beat’s fold error, and a step on a beat edge does not, so a slightly different period can win by moving an edge onto the step. One step of 6, 12 or 24 dB, about 0.35 or 0.65 through the run and up to 0.15 of a beat from a beat edge, at 55 and 110 Hz, 96 to 174 BPM and 16 and 32 beats, moves 21 of 4465 grid locks, by up to 1.15 % and with up to 1.09 dB of lost depth. A 6 dB, 150 ms duck at 110 Hz and 128 BPM over 16 beats, with a 12 dB step at 0.63 of the stem, reads 127.571 BPM. Its recovery reads 130 ms and its ripple 1.31 dB, against 203 ms and 0.69 dB on the grid. The same cause can hold a duck off its tempo. At 1 % off, a 48 dB fade over the last quarter finds the duck’s tempo in 18 of 29, and 6 dB steps every 4 beats in 22 of 31, against 28 of 31 with no steps [synthetic].

• A chord’s own beating sets a floor of about 1.3 dB under the fold error, so a smeared duck can stay on the grid. On the 76,836 stems, of 9205 ducks that stayed where both reads lock, 654 would have read 1 dB or more less ripple at the fitted period, 2.12 dB on average. A 12 dB duck 1 % slow over 12.3 beats at 96 BPM reads 6.5 dB of ripple on the grid and 1.1 dB at its own period, at fold errors of 1.42 and 1.33 dB [synthetic].

• The shape test can pass a gate. Over 8 phases and 9 to 32 beats, a 12 dB gate 3 % below the tempo at 55 Hz and 174 BPM strips in 4 of 376 runs [synthetic].

• The depth reads shallow, and the bias depends on the shape: an 18 dB duck reads up to 11.8 dB short in the set of Depth bias. On the noiseless chords of the 123,840 signals, on the tempo, with releases of 80 ms or more, it reads a median of 4.9 dB and at most 9.2 dB short [synthetic].

• peak_db compares two notes directly only when they read the same harmonic, so the pool’s level gate can leave out a note that reads a higher harmonic of a timbre with weak upper harmonics [fact].

• A second sound on the beat, such as a kick in the same stem, can read as a beat-locked pump; over fewer than 8 beats it is not stripped [synthetic].

• The search takes the deepest fold, so a second modulation near the tempo can take it [fact]. Under a duck on the tempo, a deeper second duck 0.5 to 2.5 % off took 53 of 64 locks, and a shallower sine, duck or gate took 15 of 623 [synthetic].

• A gate cuts across notes on the beat as a pump does; no test tells the two apart on a read that locks on the grid [fact]. A lock on a level with a duck’s shape off a division, such as a saw tremolo that falls at once and rises over the beat, strips as a duck too. Of 360 saw tremolos each, 266 strip at 3 % above the tempo against 63 with no search on free reads, 232 against 8 at 3 % below, 138 against 0 at 3.5 % above and 10 against 0 at 3.5 % below, and none at 4 to 5 %, on the 123,840 signals [synthetic].

• A heavy tail hides the release, so release_s is null over a median tail_weight of 0.5 [fact].

Sources

• Envelope stages: Xfer Records, Serum 2 manual. Facts only.
• Attack and decay descriptors: Peeters, Giordano, Susini, Misdariis and McAdams, The Timbre Toolbox: extracting audio descriptors from musical signals, JASA 2011. Inspiration only.

Voices

The voices layer counts how many notes sound at once and, on a mono line, reads how each note joins the next. analyze writes it to the voices group of layers.json. It reads any line, chords too, so its fields sit in a group of their own and the pitch group stays null on chords [fact].

Voicing, the chord gate of the transcriber and the pitch layer, does not read this count [fact]. The final notes do: a line with at most one voice at a time, or with no count, is a mono line for every rule of notes.mid and for the span of the full-band level, unless --profile pad ran the notes step (owner decision, 2026-11-05) [fact].

The count

• A note holds a voice from its onset to its note-off knee, or to 25 ms before its end without a knee, so a release tail holds no voice, and a note the envelope layer reads as silent holds none. A held part stops at the stem end, so a note listed past the stem holds none either. A note of 25 ms or less holds none without a knee [fact]. The issue asks for the first note to lie within 3 dB of its sustain level. This reads the knee instead and has no level test [fact].
• Pitches within 60 cents form one voice, so seven detuned copies of one pitch count once [fact, proposal for 60 cents].
• max_polyphony is the most voices that have started by any onset and are held 50 ms after it, so a held overlap under 50 ms does not count [fact, proposal for 50 ms]. mono is true when it is 1, so one overlap of 50 ms or more makes a line poly, with no tolerance [fact].
• mean_polyphony is the mean count over the time any note is held [fact]. An overlap under 50 ms counts here and not in max_polyphony, so the mean can exceed the maximum [fact].
• The confidence is the share of that time whose count agrees with the verdict: one voice on a mono line, two or more otherwise. It reads high from 0.9 and medium from 0.7 [fact, proposal for both].

Joins

Consecutive notes of a mono line join when the later onset lies under 30 ms past the earlier end in the note list, starts after the earlier onset, and lies less than 50 ms inside the earlier held part. A note stacked on another joins nothing and leaves the earlier note in place unless the stacked note ends later, and a silent note leaves it in place too [fact]. gap_ms is negative when they overlap in the list. The split join of the notes step merges a same-key repeat with no attack of its own, so that repeat reaches no join [fact].

• dip_db is the level of the quieter note minus the lowest level from 25 ms before the onset, or from the earlier held end when that comes first, to 25 ms after it, on an RMS over one period of the lower note [fact]. A transcribed onset can land after the silence it follows, which the held end still sees [synthetic]. Each note’s level is the 10th percentile over up to 100 ms beside the join, because that window still ripples on the higher note [fact].
• centroid_change is the centroid over f0 after the join over the same before it, minus 1, over 20 ms or two periods each side, and null when either side holds no signal or runs past either end of the stem [fact]. The verdict does not read it, because it gained nothing on the set below [synthetic].
• legato is true when dip_db is under 1 [fact, proposal for 1 dB]. A 12 dB dip that rises over 5 ms at 110 Hz reads 1.35 dB [synthetic].
• dip_db and legato are null, with no estimate, when neither note has a level above the -120 dB silence floor beside the join, as with notes of about 25 ms or notes listed past the stem [fact].
• A join reads low when its verdict is legato, as a retrigger with an instant attack keeps its level too; when no attack was read; when half a period before the earlier level window or one period before the dip read falls before the stem start; when one period past the 25 ms after the onset falls past the stem end, as a level is then unread or short; or when the median attack of the pool is over 100 ms, because a long attack hides a retrigger [fact, proposal for 100 ms]. Otherwise it reads medium [fact].

The set: 4000 joins sampled over 41 to 440 Hz, steps of -12 to +12 semitones, four timbres, dips of 3 to 60 dB, attacks of 1 to 50 ms, level changes of up to 6 dB, onsets off by up to 20 ms and gaps of -20 to 10 ms, of which 3669 reach a join. At 1 dB, 91 % of the legato joins read legato [synthetic]. Of the retriggers of 12 dB or more [synthetic]:

Limits

• A window of one period of the lower note smears a short new attack, so a retrigger with an attack of 5 ms or less often reads legato, as the table shows. No level read shorter than a period holds still on a periodic wave [fact].

• A retrigger with an instant attack keeps its level, so on generated lines with exact note lists it mostly reads legato: 232 of 270 joins of a retriggered 808, 106 of 127 of a soft retrigger and 81 of 131 of a stepped one, against 150 of 174 of a legato 808. A legato line with a dip reads legato in 46 of 73 joins [synthetic].

• The count reads the note list as it is, so a transcriber’s wrong overlap counts, and one overlap of 50 ms or more makes a line poly. On 889 generated mono lines the chord gate calls monophonic, 136 read poly from transcribed notes, 131 of them at low confidence. With exact note lists, 463 of 463 read mono [synthetic]. On 22 real instrument stems, 1 read mono and none reported a join, at mostly low confidence. The other 21 read a maximum of 3 to 8 and a mean of 1.06 to 2.52 [fact]. So a mapper must not take mono: false or missing joins as evidence at low confidence.

• The verdict depends on where the join falls in the period, and the acceptance test sits on a whole-cycle position. Moving a join from 1.0 s in 0.31 ms steps over 40 positions, between the same saws, read these counts [synthetic]:

  Hz	0 dB join read legato	12 dB, 5 ms retrigger found
  110, 165	31	24
  220, 330	34	40
  110, 146.83	37	24
  82.41, 123.47	29	24

• With under 30 ms of audio after a join, the later note has no level, so a legato join reads as a retrigger: 682 of 960 over four pairs, against 17 of 160 at each length from 40 ms. A 12 dB, 20 ms retrigger reads as one in all but 27 of 960, so the verdict stays and the join reads low, rather than losing those [synthetic]. At the stem start, a join reads low while its earlier level window reaches the start, so up to a lead of about 125 ms. A legato join there reads as a retrigger in 18 to 47 of 160 by lead, from 150 ms on in 27 to 40, and from 1.0 s in 29, so this low level marks a short first note and not a higher error rate. A 12 dB, 20 ms retrigger reads as one in 160 of 160 at every lead [synthetic].

• dip_db reaches 100 to 190 dB on exact digital zeros, since the -200 dB floor stands in for a level. The verdict does not change [synthetic].

• A filter that restarts with no dip in level reads legato. In the set, 228 of 433 filter joins read as a retrigger, against 289 of 436 to 358 of 476 for the other timbres [synthetic].

• A long attack hides a retrigger, and the amplitude envelope may not restart from zero, which makes a dip shallower [fact].

• A pad’s count is a lower bound, because pads are the weakest class of the transcriber [fact].

• Glides and legato inside chords have no method here [fact].

Suggested edits

The mapping step turns the pitch, voices and envelope groups of layers.json into edits for one synth. It runs in two parts. A synth-neutral plan decides what the patch should hold, in physical units. The synth’s adapter then writes each setting to its own controls with its own laws. This page states the plan.

The output holds the edits, an unrepresentable list for values the synth cannot reach, and an unwritten list for measured fields the plan leaves out on purpose. analyze writes the edits and the unrepresentable entries into each group of layers.json; see Layer report. The unwritten list stays in the library. Each unwritten entry names the field by its path in the group’s values, such as attack_s or glides.curve, and gives one reason:

Voicing

• Mono, legato and the voice count need a voices verdict of medium confidence or higher. A voices group at low confidence writes nothing and lists mono, joins.legato and max_polyphony; set them by ear. A low count is no evidence of voicing (see Voices).
• A mono stem writes mono on and no voice count.
• Legato is on when a join of medium or high confidence reads legato. It is off when every such join retriggers. A synth’s legato acts only where notes overlap, so one legato join needs it on. Joins with no confident verdict write nothing and list joins.legato, so legato keeps the template’s setting; set it by ear.
• A poly stem writes mono off, legato off and a voice count of max_polyphony, times 1 unison voice and 1 oscillator. The unison step sets the final count once it knows those numbers.

Glide and pitch drop

• A record of low confidence counts for nothing.
• The glide time is the median time_s of the confident glides: the full 0 to 100 % time. A time beyond the synth’s maximum lands on the maximum and is listed in unrepresentable. Always-glide is on when one of those glides starts from silence.
• No law maps the fitted glide shape p to a synth’s glide curve, so the curve stays at its default and glides.curve reads curve_not_set.
• The pitch drop is the median df_st and the median tau_s of the confident drops. The adapter writes a unipolar envelope route into oscillator pitch. A drop beyond the destination’s range lands on the range, and the rest goes to unrepresentable. The route amount is an interim law, always marked uncalibrated until the calibration set measures it. An edit of the drop envelope is marked uncalibrated where the registry tags its control Interim.
• Vibrato writes no route here.

Amplitude envelope

• A stage of low confidence writes nothing. An edit’s confidence is the lowest level of the records behind it, capped by the group’s level.
• An attack flagged attack_is_upper_bound writes nothing and reads upper_bound.
• A hold is written only for a plateau over 10 ms; the envelope layer reads a shorter one as 0.
• The decay runs from the peak to the sustain level. The sustain is the held level from the peak in dB; the adapter’s law turns it into its stored value.
• A release is written only when release_confidence is medium or high: the envelope layer leaves no release under a tail heavier than 0.5 or with a residual above 3 dB. Otherwise release_s reads not_identifiable.
• The attack, decay and release curves stay at their defaults and read curve_not_set.
• The tail and the pump have no envelope control. They belong to the space and modulation layers.
• A value equal to the control’s default writes no edit.

Velocity

Velocity gets a path to the level when level_spread_db has a value, read over at least 3 pooled notes, and exceeds 3 dB. The edit is always low confidence, as the spread is.

Common methods

The rules on this page bind every stage of the pipeline. Units and confidence have their own page, Units, which this page does not restate. The tags are those of the pipeline page.

Samples in f64

Samples, spectra and every intermediate value are f64 from the decoder on, at the stem’s native rate [fact]. f32 appears only where a file format stores 32-bit floats [fact]. Analysis never resamples; only a consumer with a fixed input rate does [fact].

The window

The analysis window is the periodic Hann, $w[n]=0.5-0.5\cos (2\pi n/N)$ for $n=0\dots N-1$: the first $N$ samples of a symmetric window of length $N+1$ [fact]. Its main lobe is 4 bins wide null to null, and its -3 dB width is 1.44 bins [fact]. The symmetric form would shift every bin slightly [fact]. A four-term Blackman-Harris window, with lower side lobes and a wider main lobe, and the rectangular window exist for comparison only [fact].

The frame rule

• Frames are centred and zero-padded, with hop $N/4$ [fact].
• At 44.1 kHz the frame length is 8192 samples when the note list has a median f0 below 100 Hz, else 4096, and 4096 without a note list [fact]. At another rate the length is the power of two nearest, in log, to that length times $f_s/44100$: the same at 32 and 48 kHz, twice it at 88.2 and 96 kHz, four times at 176.4 and 192 kHz, half at 22.05 kHz [fact]. Harmonics $f_0$ apart need $N\ge 4f_s/f_0$ for the 4-bin Hann main lobe; at 48 kHz, 4096 points resolve harmonics down to 47 Hz, and 8192 down to 23 Hz, as do 16384 points at 96 kHz [fact].
• The stereo screen uses 8192 points for every f0, and the spectrogram’s zoom panel 16384 [fact].
• layers.json records the length used as conventions.frame_size [fact].

Active frames and statistics

• A frame is active when its RMS lies within 40 dB of the loudest frame’s [fact, proposal for 40 dB]. The stereo screen takes the RMS over both channels [fact].
• A statistic over active frames is the median and the interquartile range [fact]. Both resist outliers, such as the frames of a note onset [fact].
• Quantiles interpolate linearly between the sorted values at position $q(n-1)$, the convention of the common numeric libraries [fact]. The IQR is the 75th minus the 25th percentile [fact].
• A statistic without an active frame is no estimate with the reason silent_stem, never a number [fact]. One with fewer than 16 active frames is no estimate with the reason too_few_frames: the first and last two frames of a clip reach into the zero padding, and a median over few frames follows them [fact, proposal for 16].

Conventions next to every number

Published tools disagree on weights, floors, windows and statistics, so a number means something only with its convention [fact]. Every number in the stem group of layers.json has a convention under the same key in conventions, and a test fails when a field lacks one [fact]. The convention names the weighting, the window, the floor and the statistic [fact].

Golden vectors

Expected values in tests come from an independent reference, never from our own code [fact]. The testing page gives the commands.

• A pinned Python generator computes each reference value once, from a recipe that the JSON vector stores [fact]. The interpreter and every package are pinned with hashes, so a second run writes byte-identical files [fact].
• Each vector holds the recipe, the generator, the tool versions, the values and the tolerance [fact]. The Rust test rebuilds the signal from the recipe and compares with $|\text{actual}-\text{expected}|\le \text{abs}+\text{rel}|\text{expected}|$ [fact].
• Noise comes from a SplitMix64 generator that both languages implement bit for bit [fact]. A statistic over noise is still compared as a statistic, with a tolerance that covers its spread [fact].
• CI never runs Python. It checks that every vector has its fields and names the pinned versions [fact].
• The figures that task issues quoted live as prototype tables with their recipe. A figure the recipe does not reproduce is recorded with a reproduced: false flag, and the committed table replaces it [fact].

Command line

The binary is sound-to-patch. This page shows the help of each command. The help texts run in the test suite, so they match the built binary. cargo xtask dist also renders them as man pages.

sound-to-patch

$ sound-to-patch --help
Turn an audio stem or a written sound description into a starting point in Vital or Serum 2.

Usage: sound-to-patch <COMMAND>

Commands:
  analyze  Analyse a stem and write layers.json, spectrogram.png and notes.mid
  notes    Transcribe a stem and write notes.mid
  preset   Read, inspect and write synth presets
  render   Render a preset and a MIDI file to a WAV file through the synth
  help     Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

analyze

$ sound-to-patch analyze --help
Analyse a stem and write layers.json, spectrogram.png and notes.mid.

The tempo is refined from the stem near --bpm, and the stem is transcribed unless --notes gives its notes. Each step prints what it found.

Example: sound-to-patch analyze bass.wav --bpm 140

Usage: sound-to-patch analyze [OPTIONS] --bpm <BPM> <STEM>

Arguments:
  <STEM>
          The audio stem: WAV, AIFF, FLAC, MP3, AAC or M4A

Options:
      --bpm <BPM>
          The tempo of the stem in beats per minute, 4 to 1000. It is refined from the stem, 2.91 % below to 3.09 % above it, unless --bpm-exact is given

      --bpm-exact
          Use --bpm as it is, without refining it from the stem

      --notes <FILE>
          A MIDI file with the stem's notes, instead of transcribing the stem; the f0 track still reads the stem unless the notes play chords. Its tempo must match --bpm or the refined tempo

      --profile <PROFILE>
          Decode with the settings of this kind of stem instead of the detected ones

          Possible values:
          - bass: B0 to F#4, with the minimum length from the BPM
          - lead: C2 to B6, with the minimum length from the BPM. Plucks use it too
          - pad:  C2 to B6, with the decoder's default minimum length. Chords use it too

  -o, --out <DIR>
          The output folder [default: a folder named after the stem, next to it]

      --force
          Replace files in the output folder that this stem's report does not own: a report or notes of another stem, a report of the same name with other content, a file this tool did not write, or a spectrogram.png without a report

      --synth <SYNTH>
          The target synth that the report names [default: vital]

          Possible values:
          - vital:  Vital, with `.vital` preset files
          - serum2: Serum 2, with `.SerumPreset` preset files

  -h, --help
          Print help (see a summary with '-h')

analyze decodes the stem at its native rate, runs the stem estimators and writes three files into the output folder, which it creates:

• layers.json: the report. The layer report page lists every field.
• spectrogram.png: a 1500 by 512 pixel image of the busiest 10 s, in two panels. The pipeline page gives its layout.
• notes.mid: the final note list, the notes rewritten with the layers, as the analyze guide explains. It replaces the first file of notes. A stem without notes writes none.

The files are replaced on the next run of the same stem file; nothing else in the folder changes. A stem of the same name with other bytes, such as a new export, needs --force, below. The same stem and flags give the same bytes. The stem is read once, so a pipe such as /dev/stdin works as a stem. Each file is written to a hidden temporary sibling and renamed into place, notes.mid first and layers.json last; the old layers.json moves aside before any file changes. So no run leaves a cut-short file, and a layers.json that exists belongs to the files next to it. A write that fails, such as on a full disk, puts the old files back, leaves no temporary file and removes a folder the run created. A run killed during the renames, such as by a power loss, can leave the old files as hidden backups named .<file>.<random>.old, such as .layers.json.a1B2c3.old, with no layers.json. The next run stops and names every backup: rename each backup back to its file name to keep the old report, or pass --force to write new files and remove the backups once they are in place. The command prints one wrote <path> line per file to standard output, and a warning line for each corrupt packet that became silence to standard error.

layers.json records the stem’s file name in source and the SHA-256 of its bytes in source_sha256. The command stops before it decodes, names the file and asks for --out or --force when the folder holds:

• the report of another stem, such as bass.aiff next to bass.wav sharing bass/;
• a report of the same name with other content: another stem of that name, such as the bass.wav of another song in a shared --out folder, or this stem changed since that run, such as a new export; a report from release 0.31 or earlier has no checksum and is matched by its name;
• a layers.json that is not a report of this tool: it cannot be read, is not JSON or names no source;
• a spectrogram.png and no layers.json, a file no report owns;
• a notes.mid of another stem by its track name, or one with no track name, as in notes.

--out picks another folder, and --force replaces the files. An output folder that is a file, by default or by --out, such as a file bass next to bass.wav, stops the command with a request for --out.

Without --out, the folder is named after the stem file and sits next to it: stems/bass.wav writes stems/bass/. A stem whose name without the extension is . or .., such as ..wav, would name its own folder or the one above, and a stem without an extension, such as bass or .wav, would name the stem file itself, so each needs --out.

• Tempo: --bpm is refined from the stem’s onsets, from 2.91 % below to 3.09 % above it, as the tempo section describes, and the first line of standard output says which tempo every later step uses and why. --bpm-exact uses --bpm as it is. The report records the given and the refined tempo.
• Notes: without --notes, the stem runs through the notes step of notes once, and the next lines of standard output are its steps; --profile overrides the detected settings as there. The transcription group of the report records the model and the settings.
• With --notes, the MIDI file gives the note list and the model never runs; --profile is then a usage error, and transcription is null. The note times follow the file’s tempo map, and every tempo in the file must lie within 0.01 BPM of --bpm or of the refined tempo; a file without a tempo event plays at the MIDI default of 120 BPM. Notes on channel 10, the percussion channel, are left out.
• Standard output ends with wrote <path> for each file. A warning line for each corrupt packet that became silence goes to standard error.

notes

$ sound-to-patch notes --help
Transcribe a stem and write notes.mid.

The pitch range, the voicing and the minimum note length come from the stem unless --profile sets them. Each step prints what it found.

Example: sound-to-patch notes bass.wav --bpm 140

Usage: sound-to-patch notes [OPTIONS] --bpm <BPM> <STEM>

Arguments:
  <STEM>
          The audio stem: WAV, AIFF, FLAC, MP3, AAC or M4A

Options:
      --bpm <BPM>
          The tempo of the stem in beats per minute, 4 to 1000. It is refined from the stem, 2.91 % below to 3.09 % above it, unless --bpm-exact is given

      --bpm-exact
          Use --bpm as it is, without refining it from the stem

      --profile <PROFILE>
          Decode with the settings of this kind of stem instead of the detected ones

          Possible values:
          - bass: B0 to F#4, with the minimum length from the BPM
          - lead: C2 to B6, with the minimum length from the BPM. Plucks use it too
          - pad:  C2 to B6, with the decoder's default minimum length. Chords use it too

  -o, --out <DIR>
          The output folder [default: a folder named after the stem, next to it]

      --force
          Write next to a report that is not this stem's, replace a notes.mid of another stem or with no track name, or replace the final notes and remove the report that describes them

  -h, --help
          Print help (see a summary with '-h')

notes decodes the stem at its native rate, runs the transcription model once and the f0 track once, and writes notes.mid into the output folder, which it creates. The pipeline page gives each step.

• Settings: without --profile, the pitch range, the voicing and the minimum note length come from the stem. --profile bass, lead or pad sets them instead; detection still runs and prints its findings.
• Tempo: --bpm is refined from the stem as for analyze, and every later step uses the printed tempo; --bpm-exact uses --bpm as it is.
• Output: standard output gets one line per step, in order: the tempo, the detected range, the detected voicing, the register window, the voicing the cleanup used and where it came from, the settings used, the decoded count, then the note count after the octave check, the ghost merge, the attack split, the split join and the quiet-note guard with its drop count, and last wrote <path>. A warning line for each corrupt packet that became silence goes to standard error.
• The file: a format 0 MIDI file with 480 ticks per beat, the stem’s file name as the track name, the printed tempo and 4/4 at tick 0, and every note on channel 1 at velocity 100. It is written to a hidden temporary sibling and renamed into place. A failed write leaves the old files and no temporary file. Only notes.mid is replaced; nothing else in the folder changes, except the report that --force removes, below.
• The folder: as for analyze, stems/bass.wav writes into stems/bass/ without --out, and a stem such as ..wav, .wav or bass needs --out. When the folder holds a notes.mid whose track name is another stem, or none, the command stops before it decodes and names the file; --force replaces that file. When it holds a layers.json of another stem, of the same name with other content, or one that is not a report, it stops the same way; --force writes notes.mid next to that file and leaves it. When it holds a layers.json with a notes group, the report of analyze that describes the final notes.mid, it stops whatever the stem, because the first list would replace the file the report describes; --force replaces notes.mid and removes that report once the new file is in place, so no report describes a file that is gone. A backup that a killed run left, as for analyze, stops it too; --force writes notes.mid and then removes the backups. --out picks another folder in every case.

preset

$ sound-to-patch preset --help
Read, inspect and write synth presets

Usage: sound-to-patch preset <COMMAND>

Commands:
  inspect  Print the header fields and the decoded tree of a preset file
  apply    Apply an edit list to a template and write a preset
  help     Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

preset inspect

$ sound-to-patch preset inspect --help
Print the header fields and the decoded tree of a preset file.

Example: sound-to-patch preset inspect bass.vital

Usage: sound-to-patch preset inspect [OPTIONS] <FILE>

Arguments:
  <FILE>
          The preset file, such as `bass.vital`

Options:
      --synth <SYNTH>
          The target synth [default: the extension of the preset file, else vital]

          Possible values:
          - vital:  Vital, with `.vital` preset files
          - serum2: Serum 2, with `.SerumPreset` preset files

      --full
          Print every list element and every text in full

      --shown <N>
          List elements printed before the rest is counted
          
          [default: 3]

  -h, --help
          Print help (see a summary with '-h')

preset inspect refuses a file without a stamp, settings or sample, and a file with a known control that holds a text, a list, an object or null in place of a number. Vital fails to load a text in a control and loads a flag, so a flag reads. The message names the key.

preset apply

$ sound-to-patch preset apply --help
Apply an edit list to a template and write a preset.

Values are written with their unit, such as "1.2 kHz", "800 ms", "-6 dB", "+12 st" or "25%".

Example: sound-to-patch preset apply edits.json -o bass.vital

Usage: sound-to-patch preset apply [OPTIONS] --out <FILE> <EDITS>

Arguments:
  <EDITS>
          The edit list, a JSON file

Options:
  -o, --out <FILE>
          The preset to write, such as `bass.vital`

      --template <FILE>
          A preset to start from instead of the synth's built-in template

      --name <NAME>
          The preset name. Without it, the name of the output file

      --force
          Replace the output file if it exists

      --synth <SYNTH>
          The target synth [default: the extension of the preset file, else vital]

          Possible values:
          - vital:  Vital, with `.vital` preset files
          - serum2: Serum 2, with `.SerumPreset` preset files

  -h, --help
          Print help (see a summary with '-h')

The output file:

• must carry the synth’s extension, such as .vital;
• must not exist, unless --force is given;
• must not be the edit list or the template;
• must not lie inside Vital’s own Factory folder.

The output keeps the template’s stamp, and the summary prints it. A stamp with a newer first or second number than the stable build 1.0.7, such as 1.1.0, gives a warning: that build refuses it. The load gate ignores the third number, so 1.0.8 gives none. A run that ends with an error leaves no file. A run killed during the write leaves at most a temporary file whose name ends in .partial next to the output. A folder without hard links, such as one on an exFAT or FAT32 drive, gets the file too, and an existing file there is still replaced only with --force. There, without --force, the file is written in place: it is visible while it is written, an error removes it, and a run killed during the write leaves a partial output file. An empty --name, such as an unset shell variable, is a usage error.

The edit list

preset apply reads an edit list: a JSON object with the target synth and a list of edits, in a UTF-8 file of at most 1 MiB. A leading byte order mark is skipped. Each edit is one of four records, and each carries a reason.

• A control value carries its unit, as the units page lists: "1.2 kHz", "800 ms", "-6 dB", "+12 st" or "25%". A switch takes true or false, and a count takes a number.
• A route amount is in semitones when the destination is stored in semitones, such as a cutoff or a pitch, and in percent otherwise. A depth in semitones is divided by the destination’s stored span, as Vital shows it.
• A route may target a number, a count, a menu such as lfo_1_tempo, or the amount or power of another slot. A switch and the bipolar, stereo and bypass flags of a slot take no route.
• A route amount lies within -100% to 100% of the destination’s range.
• A route that joins the same source and destination as an existing one edits it in place and keeps its polarity, curve and flags unless the edit sets them.
• A raw key is written as it is, outside the registry, and the output lists it as raw. A raw edit never replaces a key of the template that holds a list or an object.
• A list edits each control once and each source and destination once. A second edit of either is refused with the number of the first.
• Every problem of a list is reported at once, up to 50, each with the file, the edit, the cause and the fix.

The guide walks through one edit list.

render

$ sound-to-patch render --help
Render a preset and a MIDI file to a WAV file through the synth.

The synth's plug-in loads headless in the plug-in host process. A render counts only after the load check: the clip sounds, and the state the plug-in saves back holds the preset. A clip whose octave bands match the plug-in's init patch gives a warning. Each step prints what it found.

Example: sound-to-patch render bass.vital --midi notes.mid --bpm 140 -o bass.wav

Usage: sound-to-patch render [OPTIONS] --midi <FILE> --bpm <BPM> --out <WAV> <PRESET>

Arguments:
  <PRESET>
          The preset to render

Options:
      --midi <FILE>
          The MIDI file with the notes, read as `analyze --notes` reads it

      --bpm <BPM>
          The tempo of the MIDI file in beats per minute, 4 to 1000

  -o, --out <WAV>
          The WAV file to write; the render record goes next to it with .json

      --plugin <BUNDLE>
          The synth's .vst3 bundle [default: the bundle in /Library/Audio/Plug-Ins/VST3, else in the user's VST3 folder]

      --synth <SYNTH>
          The target synth [default: the extension of the preset file, else vital]

          Possible values:
          - vital:  Vital, with `.vital` preset files
          - serum2: Serum 2, with `.SerumPreset` preset files

      --force
          Replace an existing WAV or record file

  -h, --help
          Print help (see a summary with '-h')

render writes two files: the WAV file named by --out, 32-bit float stereo at 44.1 kHz, and the render record next to it, with .json in place of .wav. The record follows schemas/render-record.schema.json; the render record page lists its fields. Both go to temporary files in the same folder first, and replace the outputs only once both writes succeeded. Without --force, a failed write leaves the folder as it was. With --force, a failure while placing the files can leave the new WAV file beside the old record. The error then names both files and says that they no longer match. Make the record replaceable and render again with --force.

The plug-in loads in sound-to-patch-host, a second program that ships next to sound-to-patch. SOUND_TO_PATCH_HOST names another copy. The guide walks through one render and the load check.

Units

Each physical unit the library handles has a unit type. A value is checked once, when it is built from a number, a JSON field or, for a type with a suffix, a unit string, and is valid by type afterwards. Not every public function takes a unit type. The constructors and the ratio and gain conversions take a bare float, and so do many signal-processing and rhythm helpers and some fields of the estimators. For example, music::rhythm::ticks_for_ms takes a time in milliseconds and a tempo as plain numbers.

The code examples on this page come from examples/units.rs, which CI compiles. cargo run --example units prints their results.

Unit types

Each type holds an f64 that is never NaN or infinite. Its constructor returns an error that names the type and the accepted range, for example Normalized out of range: got 1.5, expected 0.0 to 1.0. The rustdoc of each type states its exact range.

Percent has two ranges, fixed by its type parameter. Percent<Unipolar> is for controls that go one way from zero. Percent<Bipolar> is for controls that go both ways, such as a modulation amount. A unipolar value widens to a bipolar one; the reverse is checked.

Bpm cannot be the spread of a confidence, because its range excludes zero; a tempo spread would be given in Ratio or Percent.

No milliseconds type exists. A value in milliseconds converts to Seconds where it enters, with Seconds::from_millis or the ms suffix.

Conversions

MIDI pitch and frequency use equal temperament with note 69, A4, at 440 Hz:

$$f=440\cdot 2^{(m-69)/12}m=69+12{\log }_2\frac{f}{440}$$

Every MIDI pitch has a frequency, so Hz::from a MidiPitch cannot fail. The reverse fails for 0 Hz, which has no pitch, and for frequencies outside notes 0 to 127, about 8.18 Hz to 12 544 Hz. A note number from a MIDI file converts with MidiPitch::try_from a u8, which rejects numbers above 127.

Intervals convert to frequency ratios. One semitone is 100 cents, one octave 1200 cents:

$$r=2^{c/1200}c=1200{\log }_{2}rr=2^{s/12}r=2^o$$

Decibels convert two ways. An amplitude gain scales a magnitude, such as a sample or a spectral magnitude weight. A power ratio scales an energy, such as the share of a frequency band:

$$g=10^{\mathrm{dB}/20}\mathrm{dB}=20{\log }_{10}gp=10^{\mathrm{dB}/10}\mathrm{dB}=10{\log }_{10}p$$

A gain or a ratio of zero has no decibel or cent value, so the inverse conversions return an error for it. The forward conversions return an error beyond a conservative cap, such as 1000 octaves, that keeps the ratio finite and nonzero.

The conversion methods are Cents::to_ratio, Cents::from_ratio, Semitones::to_ratio, Octaves::to_ratio, Decibels::to_amplitude_gain, Decibels::from_amplitude_gain, Decibels::to_power_ratio and Decibels::from_power_ratio.

    let a4 = Hz::from(MidiPitch::new(69.0)?);
    let pitch = MidiPitch::try_from(Hz::new(220.0)?)?;
    let detune = Cents::new(7.0)?.to_ratio()?;
    let gain = Decibels::new(-6.0)?.to_amplitude_gain()?;

Unit strings

The agent’s edit list writes each value as a unit string: a number, optional whitespace and a suffix. The parser accepts these suffixes, and only these, matched case-sensitively:

A target type accepts only its own suffixes. A suffix of another unit, an unknown suffix and a missing number are errors that list the accepted suffixes. A bare number for a control with a physical unit is an error; only counts, bools, menu labels and the stored value of a control without a physical unit take bare values. The parsed value then passes the range check of its type, so "-5 Hz" fails with the range of Hz, and "-40%" parses for a bipolar control but fails for a unipolar one.

    let cutoff: Hz = "1.2 kHz".parse()?;
    let pre_delay: Seconds = "20 ms".parse()?;
    let amount: Percent<Bipolar> = "-40%".parse()?;
    // A suffix of another unit is an error that lists the accepted suffixes.
    let wrong = "1.2 s".parse::<Hz>();

Each type in the suffix table prints in a form that parses back to the same value, such as 1200 Hz or -40%. Normalized, Correlation, Degrees, Bpm and Ratio have no suffix, so they have neither a unit string nor a printed form.

JSON forms

Values reach the library in two JSON forms, and both parse through the same range checks:

• layers.json holds plain numbers with the unit in the field name, such as "cutoff_hz": 410 or "attack_s": 0.02. An integer is accepted for a float field. A string in a number field fails with the parser’s type message.
• The agent’s edit list holds unit strings, such as "value": "410 Hz". The edit-list contract (#94) documents where each string goes.

A value outside its range fails with the type’s message, and the JSON reader adds the line and column. NaN and infinity never reach a file, because no unit type can hold them.

Confidence

Every estimate in the layer report carries a Confidence:

• a level: low, medium or high. The estimator stores it from its own signals; the type does not derive it;
• one or more reasons. Each pairs a stable code, such as active_frames, with the value of the signal behind it: a count, a number or a bool;
• an optional spread, never negative, in the unit the type parameter names. A cutoff is a frequency, but its spread is in octaves, so its confidence is a Confidence<Octaves>. An estimate without a spread, such as a menu choice, uses Confidence<NoSpread>;
• or a “no estimate” state. A layer with too little evidence reports no value, no level and no spread, and at least one reason.

The report writes one object per confidence. spread is left out when there is none:

{"level": "medium", "reasons": [{"code": "active_frames", "value": 412}], "spread": 0.3, "estimate": true}
{"reasons": [{"code": "too_few_notes", "value": 2}], "estimate": false}

Reading a report applies the same rules: a level on a “no estimate” state, an estimate without a level, an empty reason list, a negative spread and an unknown field or code are errors.

    let frames = Reason::new(ReasonCode::ActiveFrames, Signal::Count(412))?;
    let cutoff = Confidence::estimate(Level::Medium, Some(Octaves::new(0.3)?), vec![frames])?;
    // {"level":"medium","reasons":[{"code":"active_frames","value":412}],"spread":0.3,"estimate":true}
    println!("{}", serde_json::to_string(&cutoff)?);

    let notes = Reason::new(ReasonCode::TooFewNotes, Signal::Count(2))?;
    let envelope = Confidence::<Seconds>::no_estimate(vec![notes])?;
    // {"reasons":[{"code":"too_few_notes","value":2}],"estimate":false}
    println!("{}", serde_json::to_string(&envelope)?);

Unit traps

Some Serum 2 controls store a knob position where a reader expects a physical unit. Such a control takes a Normalized value. No conversion from Normalized to Hz or Cents exists, because the law from knob to unit is not measured yet. A cutoff in Hz therefore has no unit type for its knob: the preset writer will map it through the interim law of the parameter registry (#107) and mark the value uncalibrated.

Each claim below has a build column and a status: verified in a Serum 2.1.5 session, reported by an outside source, or open. The outside source records no build, so every row says not recorded and is reported. The probe presets of #42 will check each claim on build 2.1.5 and fill in the build.

Sources

• Preset parameter tables: serum-mcp.

The Vital preset file

This page states what is known about the .vital preset file. No official format document exists. The facts come from the open-source snapshot of Vital, from renders of its open engine and from sampled presets.

Every row names a build, a status and a source.

• Builds. source 1.0.6 is the open-source snapshot at commit 636ca0e. open engine is that engine compiled into the vita 0.1.0 Python binding, which reports itself as 99999.9.9. A retail build is named by its version, such as 1.0.7.
• Statuses.
  • checked: read in the source.
  • verified: confirmed by a probe or a load check of ours on the named build.
  • reported: an outside source says so.
  • open: unknown.
  • untestable by rule: a check that needs a step the project never takes, such as running the app.

A row verified on the open engine is not verified on retail. The pinned retail build is stable 1.0.7. The early-access build is 1.6.4 (reported). The checks still open are listed under Open items.

File and top-level keys

Version gate

The app compares the major.minor part of synth_version with its own build. The patch number is ignored.

Stamp rule. Files built from the built-in template carry 1.0.6, the open-source build, as DEFAULT_STAMP. Stable 1.0.7 loads them: the gate compares only major and minor, and files from 1.0.0 on run no upgrade step. preset apply keeps the stamp of its template and never lowers it. A template older than 1.0.0, or one that holds sub_octave, is refused. For the early-access builds, pass a template saved by your own app with --template.

Retail-only keys

Retail builds write keys that the open source does not know.

Stable 1.0.7 writes none of them. It loads a file that holds them, and so does the open engine, which drops them on save (verified, load check: the retail-keys variant). preset inspect marks them. Whether builds 1.6.x add more keys is open.

Template

Parameters and laws

The registry in src/preset/vital/registry.json holds 794 controls. 22 are legacy keys that no module creates. Our own tool extracts the names, ranges, defaults, engine scales and labels from the source; the units, laws and provenance are ours.

The interim laws wait for the render calibration. They are not settled here.

Modulation, LFO shapes, wavetables and names

Fixtures

The fixtures in tests/fixtures/vital/ are generated by cargo xtask vital-fixtures and listed with their SHA-256 in tests/fixtures/PROVENANCE.md. cargo xtask vital-fixtures --check checks them.

The round trip read, write, read is checked for every fixture by every_fixture_meets_the_format_contract.

Limits

Builds seen

Terms

Factory presets, samples and wavetables are not redistributable, and no file here holds them. The source is GPL-3.0-or-later. This page states facts in our own words and copies no code, comment or table.

Plugin state

A host loads a preset into the VST3 plugin through its state, not through a file. The rows below come from the host spike (issue 244): the retail plugin loaded headless as a library in an x86_64 host process through vst3-host 0.10.0, with no editor window. The probe output is in issue 10.

Open items

Stable 1.0.7 loaded a written preset with its edits, refused the newer stamps and re-saved the preset with every control kept. The ignored test a_preset_saved_by_vital_keeps_the_edits reads that re-save from VITAL_SAVED_PRESET and passes. The unknown-key, no-name, every-source, LFO-shape and wavetable variants loaded in 1.0.7 with no dialog.

A bare preset JSON loads as plugin state in an x86_64 host (see Plugin state).

Still open:

• optionally, the stamps and the retail-only keys of early access 1.6.4.
• whether the block size changes a render by more than repeat renders differ. The recipe fixes blocks of 512 samples, so renders compare at one block size.

Sources

• Vital source, https://github.com/mtytel/vital, GPL-3.0-or-later, facts only, commit 636ca0e.

Layer report

analyze writes layers.json, the report that an agent reads before it writes an edit list. This page lists the fields of version layers/1. The committed JSON Schema, schemas/layers.schema.json, holds the same contract, and the test suite validates every generated report against it.

Top level

The top level takes no other field. notes is no layer group. Each layer adds one group with values, confidence, flags, needs_render, suggested_edits and unrepresentable, and extends the schema in the same change. A group with too little evidence writes values: null and says why in its confidence. values holds plain numbers with the unit in the field name.

suggested_edits is a list of records in the edit-list format, each a param or a route with unit strings, so a route keeps its source, amount and polarity. Each record’s reason is the group’s name. The mapper of the report’s synth fills the list; see Suggested edits.

unrepresentable lists the values of the group that the synth cannot reach:

A synth whose mapper writes no edit for a group leaves both lists empty. Vital writes none for the pitch, voices and envelope groups yet.

The pitch group breaks reports of layers/1 from releases up to 0.23: suggested_edits changed from an object of numbers to a list of records, pitch is a new required top-level key, and conventions requires the keys pitch, glides, drops, sub_band_line and vibrato. A reader of older reports keeps their schema.

The envelope group breaks reports of layers/1 from releases up to 0.25: envelope is a new required top-level key, and conventions requires the keys envelope, note_off, level_spread_db and pump.

The tail breaks reports of layers/1 from releases up to 0.27: values of envelope requires release_s, tail_rt60_s, tail_weight, release_confidence, tail_votes, tail_vote_count and tail_is_reverb, and conventions requires the key tail.

The voices group breaks reports of layers/1 from releases up to 0.26: voices is a new required top-level key, and conventions requires the keys voices and joins.

The mapping lists break reports of layers/1 from releases up to 0.28.x: every group requires unrepresentable, and a suggested_edits record must be a param or a route with a reason.

The notes group breaks reports of layers/1 from releases up to 0.30: notes is a new required top-level key, and conventions requires the key notes.

The checksum breaks reports of layers/1 from releases up to 0.31: source_sha256 is a new required top-level key.

The transcription group

The group records what produced the notes, so a note that flips between two machines can be traced to its settings.

The notes group

The notes that analyze writes to notes.mid: the cleaned note list rewritten with the pitch, voices and envelope groups. The analyze guide explains the rules.

A note record:

Naming

• Field names are snake_case ASCII.
• A field with a unit carries it as a suffix: _hz, _s, _ms, _db, _dbfs, _deg, _st (semitones) or _ct (cents). A field without a suffix is a count, a ratio, a share from 0 to 1, a power, a flag or a tempo in BPM.
• Integers are written as integers; every other number is a float.

Value objects

A single estimate is {"value": ..., "confidence": ...}. A statistic over active frames is {"median": ..., "iqr": ..., "confidence": ...}, the median and the interquartile range. The value, or the median and the IQR, is null exactly when the confidence states no estimate.

The confidence object follows the units page:

{"level": "high", "reasons": [{"code": "active_frames", "value": 412}], "spread": 310.0, "estimate": true}

• level is low, medium or high.
• reasons holds at least one {code, value} pair. The value is a count, a number or a flag.
• spread is optional, in the unit of the value: the IQR of a statistic, one bin width for the band edge.
• "estimate": false means no value. Such an object has reasons and neither a level nor a spread.
• An unknown field is an error.

The stem group

Every statistic over frames, band_share and band_edge_hz need at least 16 active frames; with fewer they give no estimate with the reason too_few_frames. The ranges are those of the unit types and the schema. Each convention is the text that conventions holds under the same key; the pipeline page explains the rules behind them.

stereo holds full, sub (below 120 Hz), low_mid (120 Hz to 1 kHz) and high (above 1 kHz), each {rho, sm_db, ipd_deg}, and one confidence for the screen:

A stereo value is null when it does not exist: a one-channel file, a silent channel, or a zero mid or side.

The pitch group

The pitch layer of a monophonic line; the pitch layer page explains each method. Every record carries its own confidence, always an estimate. The group’s own confidence is medium, or low when fewer than half of the track frames inside the notes are voiced within 100 cents of a sounding note, as under noise or DC. Its reasons are voiced_frames, the voiced frames of the f0 track, and note_frames, that share.

values is null when the line plays chords (reason polyphonic_line) or the stem has no notes (reason no_note_list). Otherwise it holds three lists, each in time order and each empty when nothing was found: glides, drops and vibrato. Every time is audio time from the start of the stem.

A glide:

Its reasons are motion_span (seconds), fit_residual (cents), pitch_jitter (cents) and curve_bound (flag). A glide whose motion_span is under 0.09 reads low.

A drop:

Its reasons are fit_residual and attack_time (seconds): the time the band envelope of the strike takes from 10 % to 90 % of its peak. A drop with an attack over 30 ms reads low, and one with tau_s under 0.01 or an attack over 12 ms at most medium.

A vibrato:

Its reasons are sine_share, cycles_in_span and contour_kurtosis. A vibrato holds 2.5 cycles or more; from 4 it reads high. A kurtosis below 1.25 reads low, since the contour jumps between two levels, as a trill does, and so does a vibrato on a pitch below 100 Hz.

flags holds four booleans: chord_skipped, glide_too_short (a motion between two notes lasted 35 to 90 ms over its fitted time, or over its detected span when the fit failed), weak_fundamental (a low note’s second harmonic exceeded its fundamental by 6 dB, so its drop was not reported) and no_notes. At a sample rate under about 4 times a note’s pitch the second harmonic cannot be measured, and that note is not tested for weak_fundamental. needs_render is false.

The voices group

The count of voices, read on any line, and the joins of a mono line; the voices layer page explains each method. values is null without notes (reason no_note_list) or when no note holds a sounding part in the stem (reason too_few_notes). The group’s confidence reads the share of the held time whose count agrees with the verdict (reason count_agreement). flags is empty and needs_render is false.

A join record:

The envelope group

The amplitude envelope over the pooled notes, each read note’s record, and the sidechain pump; the envelope layer page explains each method. values is null when no note was read: the stem has no notes (reason no_note_list) or every note was skipped (reason too_few_notes). The group’s own confidence is medium with 3 or more pooled notes, else low.

A statistic reads low over fewer than 3 notes or with an interquartile range above half its median. Its reasons are too_few_notes and relative_spread; attack_s adds upper_bound_share, the share of the pooled attacks that are upper bounds. A median of 0 under an interquartile range above 0 has no ratio, so the statistic reads low and writes zero_median, that range in the statistic’s unit, in place of relative_spread.

A note record:

flags holds three booleans: no_notes, attack_is_upper_bound (more than half the pooled attacks are upper bounds, so a preset should not take the attack) and pump_stripped (the pump locks to the beat over 8 counted beats or more, and its curve was taken out of the note tracks). needs_render is false.

Conventions

Render record

render writes a record next to each WAV file: everything the audio depends on, so a later comparison can cite it and analyze the audio with the same notes. The committed JSON Schema, schemas/render-record.schema.json, holds the same contract, and the test suite validates records against it.

Times are in seconds and the tempo in beats per minute.

midi_sha256 and load_check are absent from a record whose notes came from no file or whose render skipped the load check, such as the renders of the test suite.

Before the clip

The record names the recipe, and the recipe fixes what plays before the clip, in one processing run with the clip:

1. 0.5 s of silence, so smoothed parameters settle.
2. A warm-up note, MIDI 60 at velocity 100 for 0.5 s in a 1 s step, which clears what an earlier render left behind.
3. 0.5 s of silence again.

The host renders each step in whole blocks, so a step lasts up to one block longer. Its audio is dropped, and the clip starts on a block boundary at the transport’s start.

Render spread

Two renders of one preset are not identical: on one Vital instance, each render differs a little from the one before. The spread file says how far renders of one preset lie apart, so a comparison can tell a real difference from render noise. It is the noise floor that calibration, compare, refine and the bench cite.

cargo xtask render-spread --out spread.json writes it: the testing page gives the method. The file is local and depends on the synth’s build and its quality setting. The repository keeps no spread of record; the per-recording metrics are on the reference recordings page only.

The draws

A fresh Vital instance renders bit-identically to any other fresh instance, and each later render on one instance differs: an ignored Vital test pins both. So the 20 renders of a spread are drawn in sequence on one fresh instance, and a set with a repeated render fails. Two workers would each render the same sequence and turn 20 renders into 10 distinct ones.

A spread takes 3 to 256 renders. Fewer leave no leave-one-out median of two distances. After 256 renders the render session restarts its instance, whose first render repeats the first draw.

The statistic

For each output, each of the 190 pairs of renders gives a distance: the absolute difference of the output’s values, or for an output of two clips, such as the envelope distance, the distance itself. The spread is the 95th percentile of those distances: numpy.quantile’s default, linear between the two nearest pairs. It says how far two renders lie apart, in the output’s own unit.

The tolerance

A comparison of one clip with the renders takes the median of the clip’s distances from each render. The tolerance measures the renders the same way: for each render, the median of its distances from the other 19. Sorted in ascending order, the tolerance is the median at rank ⌈0.95 (n + 1)⌉, at most n. For 20 renders that is the 20th, the largest.

The rank is the quantile of conformal prediction. If a clip of the same sound and the n renders were exchangeable, the clip’s median would lie at or below the median at rank k with a probability of k / (n + 1): for 20 renders 20 / 21, 95.2 %. The clip’s median runs over n distances and each render’s over n - 1, so this holds only nearly. Below 19 renders the rank stops at n and the coverage falls below 95 %: 3 renders cover 3 / 4. The file records the coverage.

held_out_within measures the coverage on the renders themselves. On the renders of the 38 reference recordings, each render held out in turn against the other 19 lies within their limit in 94.6 % to 95.5 % of the cases per output, as the reference recordings page shows. The 95th percentile of the medians, the tolerance before the rank, lies near the second-largest of 20 and covered 90.7 % to 91.3 % in the same check.

Fields

held_out_within counts the renders that lie within the limit of the other renders, each held out in turn and compared as a clip. It checks the coverage on the renders themselves: with 19 renders left, the rank covers 19 in 20.

Two diagnostics describe the draws. Nothing acts on them.

• adjacent_ratio is the median distance of the 19 pairs of renders next to each other in the sequence over the median distance of all 190 pairs. It lies near 1 when a draw does not depend on the one before. With 19 pairs against 190 it is noisy. It is absent when every pair lies at 0.
• lag1_autocorrelation is the lag-1 autocorrelation of the output’s value over the sequence. For independent draws it lies near -1 / n, within about 2 / √n: -0.05 ± 0.45 for 20 renders. A drift over the sequence takes it towards 1. It is absent for the envelope distance, which only two clips have, and for a value that every render repeats.

Outputs

Each clip is measured from its onset, the first sample within 40 dB of its peak, to its end, the last sample above the recipe’s tail floor of -96 dBFS. Both channels count. The first five rows measure the clip:

The other rows come from the layer report of each render, which analyze writes with the render’s MIDI file and the exact tempo. Every group gives rows: stem, notes, pitch, voices and envelope. The groups schema, source, synth, transcription and conventions hold no estimate and give none. A row’s name is the path of the number, such as stem.rms_dbfs, stem.band_share[1].share or envelope.values.attack_s, without its value or median key.

A row’s unit comes from the field name’s suffix: _hz (Hz), _s (s), _ms (ms), _db or _dbfs (dB), _deg (deg), _st (st) or _ct (ct). A name without a suffix has its unit in a fixed list:

A field with no unit fails the run, so a new field gets a unit before it gets a spread. A test runs the walk on a report that holds every number the layer schema defines, so a field or group a later release adds fails that test first. Confidences, IQRs, band edges, the run’s inputs (bpm, bpm_given, bpm_refined and sample_rate_hz), the file constant ticks_per_beat and the mapper’s suggested_edits and unrepresentable have no row. Nor does a number that some render left without an estimate, such as a list record that one render lacks.

Comparing with the spread

A comparison of one clip with the renders reports, for each clip output, the clip’s value and the renders’ median, the difference, the limit, their ratio and whether the difference lies within the limit. The ratio shows how close a match is, beyond a yes or no: a ratio near 1 lies at the edge of the renders. The clip and the renders must share a sample rate, and the spread must hold a row for every clip output.

A recording made elsewhere can differ in level alone. A gain fit takes that out before the verdict:

1. Fit the gain: the median over the renders of their RMS level minus the recording’s, each pair over its common span.
2. Apply the gain to the recording and measure it again. Its end moves with the gain, since the tail floor stays fixed.
3. Compare it on the outputs of the sound’s shape: peak_to_rms_db, duration_s and envelope_distance_db.

The duration and the envelope distance still depend on the level, which is why they compare only after the gain; the peak-to-RMS ratio moves with it only where the clip’s end moves. The recording matches when all three lie within their limits. compare_fitted gives this verdict, and compare the comparison of all five outputs as recorded. A gain beyond ±3 dB is a finding to explain, not a failure: the compare stage of #12 matches the loudness of two sounds before its distances.

Testing

Tests follow the usual Rust layout. Agents write most of the code, so the compiler, the lints and the tests must catch their typical mistakes.

Layers

L0 is the compiler. L1 to L11 are tests. Each epic picks the layers it needs from the matrix below and names them in its issue.

● needed, ○ optional, – not applicable.

Benchmarks

cargo bench --bench notes runs the benchmark harness groups of the notes step on one generated 60 s stem: the model load, the model run at one thread and at all threads, the f0 track, the MIDI write and the whole step at both thread counts. CI compiles the benches and never runs them.

A bench measures wall-clock time, so its medians need a quiet machine. cargo run --release --example notes_memory -- <stage> runs one stage on the same stem, so a tool that reports a process’s CPU time and peak memory measures each stage. The pipeline page gives both for every stage.

Render timing needs the synth, so it is an xtask task, not a bench: see Render timing.

Running the tests

cargo nextest run --workspace
cargo test --workspace --doc

nextest runs each test in its own process, so an in-process mutex does not serialize tests; use a nextest test group instead. nextest does not run doc tests, hence the second command. CI uses the ci profile from .config/nextest.toml: no fail-fast and a slow-test timeout.

Test builds optimise our own crates at opt-level = 1 ([profile.test] in Cargo.toml), and the dependencies at opt-level = 3. The numeric tests run FFTs and trackers over whole stems, so unoptimised code made the suite take about 15 CPU-minutes. With the optimised build it takes about one, and the doc tests reuse the same build. Debug assertions and overflow checks stay on; tests/it/profile.rs fails if a change drops either from the test profile of the root package. It does not check a [profile.test.package.<member>] override. The weekly coverage job builds with the test profile too. Optimised code can shift the line attribution of a coverage report; that effect is not measured. Plain cargo build and cargo run keep the unoptimised dev profile.

Continuous integration

Three workflows run on a pull request.

Two jobs of ci.yml check generated files that a normal test cannot rebuild. Neither is a required check.

• pitch-fixtures runs pitch_fixtures_are_current in a release build on macOS. It fails when the transcriber or the cleanup now gives other notes or frames for a signal of tests/fixtures/pitch/.
• vital-registry checks out the Vital source at the commit that src/preset/vital/registry.json names, pinned by its commit hash, and runs cargo xtask vital-registry --check. It fails when a change to the generator leaves the committed registry stale. A registry made from a newer Vital commit needs the new hash in the job.

commits has its own workflow because it reads only the title and the description. Editing them runs that one Linux job and none of the heavy ones. The job has no if: and no path filter: a skipped job reports success under its own name and would hide an earlier failure on the same commit. The main ruleset requires fmt, clippy, test, doc, commits, licenses, docs and Analyze (rust) by job name, so a job keeps its name when it moves between workflows.

Each workflow cancels the run of an older push to the same pull request. A push to main never cancels another run, because every merged commit needs its checks and may need a release. The release and pages jobs keep their own groups. They never cancel a running job, but a newer run replaces an older job that still waits in the same group. A release run covers every commit since the last tag, and pages deploys the newest documentation, so a replaced job loses nothing.

A change of the base branch (edited with a new base) starts only commits.yml. codeql.yml never ran on it. After retargeting a pull request, reopen it or push a commit: both ci.yml and codeql.yml then run against the new base. No trigger can tell a base change from a title edit before a job starts, and a job-level if: that reacted to the retarget would report skipped jobs, which hide earlier failures.

The build cache of the dependencies comes from Swatinem/rust-cache, through .github/actions/setup-rust. A key holds the job, the lockfiles and manifests, the compiler version and a hash of the CARGO*, RUST* and CC* environment variables, so a new stable release starts a fresh cache and the env: of ci.yml and commits.yml must stay equal. Only a run on main saves a cache. A pull request restores the newest cache of main and compiles its own crates on top; it saves nothing, so branches cannot fill the repository quota. The docs and commits jobs share one Linux cache (linux-xtask), because they build the same xtask with the same features. The release job uses no cache, because it holds the deploy key. The test job builds the dependencies twice, once for the tests and once for xtask, because cargo xtask resolves fewer features than the test build. The cache holds both builds.

CI versus local

• A test that needs a synth (Vital or Serum 2), vita or a dataset carries #[ignore = "needs <thing>"], such as #[ignore = "needs Vital and SOUND_TO_PATCH_HOST"]. When its input is missing, it panics with a message that names the input and how to get it. Run such tests with cargo nextest run --run-ignored only.

• These ignored tests run in no CI job. Each needs something a hosted runner lacks, so run them locally before a change that touches their code:

  Tests	Why not in CI	Command
  the 21 render_vital tests	they need the retail Vital plug-in, which needs a licence, and an x86_64 host under Rosetta	see Rendering
  vita::a_preset_saved_by_vital_keeps_the_edits	it needs a copy of a written preset saved by the Vital app	VITAL_SAVED_PRESET=<file> cargo nextest run --test it --run-ignored only -E 'test(a_preset_saved_by_vital_keeps_the_edits)'
  preset::a_retail_default_preset_matches_the_builtin_template	it needs the default preset of the Vital app	VITAL_DEFAULT_PRESET=<file> cargo nextest run --test it --run-ignored only -E 'test(a_retail_default_preset_matches_the_builtin_template)'
  reference::the_catalogue_covers_every_key_of_a_saved_preset	it needs a preset saved by the Vital app	VITAL_PROBE_PRESET=<file> cargo nextest run --test it --run-ignored only -E 'test(the_catalogue_covers_every_key_of_a_saved_preset)'
  vital::tests::the_committed_registry_matches_the_checkout in xtask	it needs a Vital checkout; the vital-registry job runs the same comparison through the command	VITAL_SOURCE=<checkout> cargo nextest run -p xtask --run-ignored only -E 'test(the_committed_registry_matches_the_checkout)'

  The 11 open-engine tests run in the open-engine job, and pitch_fixtures_are_current runs in the pitch-fixtures job.

• Never skip a test at run time with an early return. It reports a pass that never happened.

• Never gate a test behind a Cargo feature. CI builds with --all-features, so the gate would not hold, and a local run without the feature would silently drop the test.

• Datasets download through cargo xtask fetch, which keeps a file only when its SHA-256 matches the pin. Evaluations on datasets run through xtask, outside CI.

Naming and assertions

• Name a test after the behaviour it checks, with no test_ prefix: hz_from_midi_69_is_440.
• Do not use #[should_panic]. Return or match an Err and assert on it.
• clippy::float_cmp denies == on floats. Compare with a tolerance.
• Always pass both epsilon and max_relative to approx: its default tolerance is machine epsilon (f32::EPSILON, about 1.2e-7, and f64::EPSILON, about 2.2e-16), too tight for signal processing.
• Clippy allows unwrap, expect and panic inside #[test] functions and #[cfg(test)] modules only. A helper module of tests/it/ is declared with #[cfg(test)] mod support; and its items are pub(crate), because unreachable_pub denies pub.
• Examples and benches are not test code for Clippy. They return Result and use ?.

Test data

• Generated signals are the default input: sines, band-limited saws, detuned pairs, noise from a seeded generator written in the test code.
• A committed fixture is small and has a row in tests/fixtures/PROVENANCE.md: origin, version, license, copyright line, SHA-256, date and purpose. Its license text goes in the same file.
• Nothing licensed or private enters git: no factory presets, no renders of them, no user stems, no personal Init preset. No Git LFS.
• A Vital fixture is a file our code generated, or a file a maintainer saved from the app after loading a file we generated. The generated files come from cargo xtask vital-fixtures; after a change to the writers, run it and update the SHA-256 rows. cargo xtask vital-fixtures --check compares the committed files with the generator and checks every row, the size limit of 256 KiB, the licence section and the wavetable authors. cargo xtask vital-fixtures --coverage lists the registry keys no fixture holds.
• Expected values never come from an AI. They come from a formula in the facts, a file saved by the synth, a render or an independent reference.
• A snapshot accepts every diff as the new truth, so CI runs insta with INSTA_UPDATE=no. Snapshot the decoded values of a preset, never its bytes, and never a number that must match an outside reference. Review a changed snapshot like code; record one locally with INSTA_UPDATE=always or TRYCMD=overwrite.

A Markdown page whose commands write files keeps its inputs in a <page>.in/ folder and holds a <page>.out/ folder. With the .out folder, trycmd runs the commands in a copy of .in, so the files they write never land in the repository. It compares only the files that .out holds, so a copy of one input is enough.

The open Vital engine

The open-engine tests run in the open Vital engine through the GPL-3.0 vita package, which runs only as a separate Python process. No crate of ours depends on it. They need VITA_PYTHON, a Python with the packages of tools/vita/requirements.txt, pinned with their hashes, made outside the repository:

uv venv --python 3.12.14 /tmp/vita-env
uv pip install --python /tmp/vita-env/bin/python --require-hashes -r tools/vita/requirements.txt
VITA_PYTHON=/tmp/vita-env/bin/python cargo nextest run --test it --run-ignored only -E 'test(/open_engine/)'

Each run of the script stops after 120 s, so a Python that hangs fails the test with the script’s last line of output.

The CI job open-engine runs the same commands on each pull request:

• written_presets_load_in_the_open_engine writes the load-check variants and loads each through tools/vita/load_check.py.
• the_open_engine_passes_the_single_note_contract runs the renderer contract suite on render::vita::VitaRenderer, with its single-note cases: the bass and lead notes, and a note at 1 s.
• Two more refuse a MIDI file with more than one note and pass the load check at 140 BPM.
• the_open_engine_renders_a_job_identically_twice pins that the open engine’s spread is 0.
• the_open_engine_renders_every_length_it_is_asked_for renders 12 onsets and lengths that are not whole milliseconds.
• the_open_engine_script_renders_and_releases_at_the_frames_it_is_asked_for sends render and note lengths straight to the script.
• the_open_engine_refusal_of_a_preset_is_a_state_error sends a preset that the engine does not load through the renderer, which reports a state error.
• the_open_engine_prints_stay_off_the_reply makes the engine write to file descriptor 1 as it starts, and the reply still parses.

The refusals of a job the engine cannot play, another synth’s state, more than one note, controller changes and a custom tuning, come before the script runs. Unit tests of render::vita check them in every CI run. Stub scripts, run with python3, check that exit code 3 maps to a state error and any other failure to a host error, that a refused load leaves no loaded state and no tempo, and that a clip of another length than asked is a crash. Without python3 the stub tests skip and say so.

VitaRenderer starts tools/vita/render.py once per request, with JSON in and JSON out. The engine renders one note per call and takes no MIDI file, so a job with more than one note or with controller changes is refused. Each call starts with no sound left from the one before. The renderer plays the recipe’s warm-up note after the load and drops it, so smoothed parameters settle. The engine has no call that renders silence, so the recipe’s settle steps add to that warm-up render’s length. The clip renders to the tail cap and ends where the recipe’s tail watch ends it. The engine reads no tuning from a preset, so a custom tuning is refused. The script writes the job’s tempo into beats_per_minute, as a plug-in in a host takes its tempo from the host. Two renders of one preset, each in its own process, come out identical, so the open engine’s spread is 0.

Probed on vita 0.1.0:

• Velocity 0 still sounds: a note at velocity 0 renders at an RMS of 0.10. The renderer never sends one, since a note’s velocity is 1 to 127.
• set_bpm before the load leaves the saved beats_per_minute as the preset holds it: 2.0 after set_bpm(140). After the load it saves the new tempo, 2.333 beats per second. The script calls it before the load and writes the tempo into the preset, so both agree.
• load_init_preset resets beats_per_minute to 2.0, 120 BPM, whatever set_bpm or an earlier preset set.
• The engine multiplies a length in seconds by the rate in float32 and truncates, so a length of n frames sent as n / rate can come back one frame short. At 400,000 to 520,000 frames it did for 16 of 121 lengths; that float32 product predicts each of them. The script passes (n + 0.5) / rate and checks the float32 product, and the renderer refuses a clip of another length than it asked for.
• The engine releases a note at the end of a block of 64 frames: a note of 64 k + 1 frames releases a block later than one of 64 k. With n / rate, a note of 64 k + 1 frames could truncate to 64 k and release a block early. the_open_engine_script_renders_and_releases_at_the_frames_it_is_asked_for sends render and note lengths that truncated that way straight to the script, so it fails if either goes back to n / rate.
• The engine’s prints would mix into the JSON reply, so the script sends its standard output to standard error and writes the reply to the saved descriptor. A preset the engine does not load and samples that are not finite exit with code 3, which the renderer reports as a state error.
• The recipe’s warm-up note changes the probe preset’s clip by at most 1.6e-7, about one float32 step, and a preset whose LFO retriggers on each note by as little. On a preset with an LFO in sync mode, which runs on the time the engine has rendered, it changes the clip by up to 1.65, the LFO’s whole phase. The renderer keeps the warm-up, so such a preset starts after the same pre-roll as on the plug-in.

The cross-engine check needs Vital, SOUND_TO_PATCH_HOST and VITA_PYTHON: render_vital::retail_and_vita_renders_compare_on_one_note renders the probe preset’s lead note 20 times on retail, in sequence on one instance, and once in the open engine. It compares the open render with the retail ones against the retail tolerance and prints each output. The levels must agree within coarse bounds, 0.1 dB on the RMS level and the peak and 1.0 dB on the envelope distance, so a broken or wrong render fails: an open render at -12 dB, or of the init patch, fails by 12.0 dB and 4.0 dB. The engines differ in build, so a miss of the retail limit is a finding about the engines, printed here, not a failure of either.

a_preset_saved_by_vital_keeps_the_edits reads a file the retail app saved after loading a written preset, from VITAL_SAVED_PRESET.

Rendering

What runs in CI, with no plug-in:

• The renderer contract suite on the fake renderer, and two tests that show the suite fails a silent renderer and a renderer with one silent channel.
• The load check on the fake: a passing render, one case each for silence, an init fallback, a changed saved state and a plug-in fault, and the warning of an init-like render. A scripted host that crashes inside the load check gives the plug-in fault too.
• The render command’s files on the fake: the .wav name in any case, an existing output without --force, a missing folder, a folder in place of an output, a read-only WAV file that --force replaces, and a read-only folder or a failed second write, which leave the existing files as they were.
• The render session on the fake: a crash, a hang or a panic tried once more on a new renderer with the same job, a render that fails twice giving the sentinel while the session goes on, a panic named with its job, a renderer that does not start, a wrong synth failing at once on the same renderer, a restart after 256 renders, one live renderer per worker, the flush before every render and the order of the outcomes. A stub host that exits on its first render is retried on a new host.
• The timing table on the fake: every row with a median and a 95th percentile, the build and the architecture, and the clip length of a long tail.
• The recipe, its snapshot and its hash, the note events, the tail watch, the MIDI reading with its hash, and the record against its schema.
• The wire protocol, the host’s request loop, the architecture pick from Mach-O headers, and the WAV writer read back through the decoder.
• The render command cases in tests/cmd/, with the hidden --backend fake.
• Clippy on the host crate for x86_64-apple-darwin.

What needs Vital: the render_vital tests render generated presets through the installed plug-in, in the plug-in host process. The host loads the plug-in as a library and opens no window. They need:

• Vital in a standard VST3 folder: /Library/Audio/Plug-Ins/VST3, or the same folder under the user’s Library.
• SOUND_TO_PATCH_HOST, the path of a sound-to-patch-host binary that loads Vital. Vital 1.0.7 holds x86_64 code only, so on Apple silicon the host needs an x86_64 slice and runs under Rosetta.
• VITA_PYTHON for one of them, the cross-engine check retail_and_vita_renders_compare_on_one_note, which also renders in the open engine. Without it that test fails at once and names the variable.

rustup target add x86_64-apple-darwin
cargo build --release -p sound-to-patch-host --target x86_64-apple-darwin
SOUND_TO_PATCH_HOST="$PWD/target/x86_64-apple-darwin/release/sound-to-patch-host" \
  cargo nextest run --run-ignored only -E 'test(render_vital)'

They run the contract suite, read the saved state back, check the forced 2x oversampling, and run the load check on a loaded preset, the init patch, a slower attack on the init patch, a newer stamp, a preset with no sound source, a dropped key and a bundle that is no plug-in. Two more send a full pitch bend and a mod wheel move and measure their effect. A session of two workers renders four presets of rising level in order, and the timing table fills every row with two runs. Two fresh instances render the same first clip bit for bit, and an instance’s second render differs from its first.

What needs Serum 2: nothing yet. Its renderer follows in its own epic.

The failure paths of the host run in CI without a plug-in: unit tests in render::plugin start shell scripts that stand in for the host. They cover a host that exits, hangs, prints bytes that are not UTF-8 or a line without its end, comes from another release, closes its pipes and lingers, never answers quit, lingers after quit, or writes a clip that does not decode. A unit test cuts plug-in output without line ends at 64 KB and reads a longer reply whole.

Render timing

cargo xtask render-timing --out "$TMPDIR/timing.json" times the Vital plug-in on this machine and writes the table to a local file, outside the repository. Refine sets its render budget from it. The table stays local: it describes the machine, so the repository holds the method and no numbers. It takes the same SOUND_TO_PATCH_HOST as the Vital tests, or --host. --preset names the preset, by default the routes.vital fixture, and --runs the measurements per row, by default 20.

Each row holds the median and the 95th percentile, linear between the two nearest measurements:

• plugin_load: starting a renderer, the host process and the plug-in load included, in seconds.
• state_load: loading the preset state into the plug-in, timed in the host around the vst3-host state load, in seconds.
• render: one render of the recipe’s lead note, MIDI 60 held 3 s, with the state load and the pre-roll, in seconds.
• clip_length: the length of that clip, at least 4 s and longer while the tail sounds.
• renders_per_second, for 1, 2, 4 and 8 workers: each measurement is one batch of one job per worker in a session whose workers have started. Its 95th percentile is the rate at the 95th percentile of the batch time, the slow end.

The table also names the synth, its build, the host, the architecture of the host process, the recipe version and the number of runs. It measures only: nothing in it tunes a render.

Repeat-render spread

cargo run --release -p xtask -- render-spread --out spread.json renders one preset 20 times and writes its spread, the noise floor every later comparison cites. The render spread page lists the outputs, the statistic and the tolerance. --preset, --midi and --bpm name what it renders, by default the routes.vital fixture and tests/cmd/data/three-notes.mid at 120 BPM. The load check runs first on a fresh renderer, so a spread never measures a preset the plug-in did not load.

A fresh Vital instance renders bit-identically to any other fresh instance, and each later render on one instance differs. So the 20 draws render in sequence on one fresh instance, through render::spread::draw. A draw whose samples repeat another’s, by their SHA-256, fails the run: a repeat would count as a draw. --runs takes 3 to 256 renders, the most one instance renders before the session restarts it. Each output’s row counts the renders that lie within the limit of the others, each held out in turn, and records two diagnostics of the draws: the adjacent ratio and the lag-1 autocorrelation.

Each render is stored as a WAV file in a folder beside the output, named after it with -renders, and measured from the stored file, as a consumer reads it. analyze writes each render’s layer report there, with the MIDI file and the exact tempo. The file is local: the spread depends on the synth’s build and its quality setting. The repository keeps no spread of record; the per-recording metrics are on the reference recordings page only.

On the fake, render_spread checks that a spread’s 20 draws are distinct, that a draw refuses fewer than 3 renders and more than 256, and that the distinct check refuses the renders of two workers that start alike. It recomputes the 95th percentile of the 190 pairs and the largest leave-one-out median from 20 stored renders with its own quantile and median. It checks that a render of the same preset matches while a louder one does not. After the gain fit, a louder render and one 8 dB quieter match on the shape, the quieter one flagged beyond ±3 dB, and a longer release does not match. Two more tests read the committed reference report: one checks its ids, hashes, gains and verdicts, the other recomputes every count, range and table of the reference page. On Vital, the_draws_of_a_spread_on_vital_are_distinct draws 20 renders of one lead note on the plug-in and finds all of them distinct.

Reference recordings

cargo run --release -p xtask -- render-reference --root DIR --manifest FILE --out report.json compares reference recordings with automatic renders. A reference recording is a render of a generated preset and MIDI file made in a standard DAW host, not by our host, so the comparison is not circular. Each manifest line names the preset_id, the preset_file, the midi_file, the bpm and the render_file, with paths from --root, and a state_file when the DAW exported the state it played.

The task reads every manifest line first: the preset, the MIDI file, the recording and the exported state. It refuses two lines with one id, the preset id and the MIDI file’s name, since both would write one renders folder. Then, for each recording, it runs the load check, draws 20 renders of the preset with the same MIDI file and tempo, and measures their spread. --workers recordings render at once, 4 by default, each on its own fresh instance. The recording compares with the renders on the clip outputs of the spread page, against the tolerance. An exported state is read back against the preset as the load check does, the tempo against the recording’s tempo. The name and the tuning are left out, because the DAW names the export and saves no tuning in a preset file. A recording that differs beyond the tolerance is reported with its numbers. Nothing is tuned to make it match.

Recordings and renders stay local. The repository holds the preset ids, the hashes and the metrics of the comparison, on the reference recordings page.

The verdict comes after a gain fit, as the render spread page describes: one gain per recording matches its level to the renders, and the recording matches on the peak-to-RMS ratio, the duration and the envelope. The report records each gain, flags one beyond ±3 dB, and keeps the comparison as recorded on all five outputs for information.

Fuzzing

The fuzz targets live in fuzz/, a workspace of its own pinned to a dated nightly in fuzz/rust-toolchain.toml. CI lints them on stable, and a scheduled job runs each one for five minutes. A failed scheduled run opens an issue titled “The weekly scheduled run failed”, or comments on the open one, so the failure reaches the repository’s watchers without a visit to the Actions page. Run one target locally from the repository root, seeded from the fixtures:

mkdir -p fuzz/corpus/vital_read
cp tests/fixtures/vital/*.vital fuzz/corpus/vital_read/
RUSTFLAGS=-Cllvm-args=-asan-globals=0 cargo +nightly-2026-10-20 fuzz run vital_read -- -max_total_time=600

RUSTFLAGS turns off the AddressSanitizer checks of globals; stack and heap checks stay on. Without it the link fails on macOS with initializer pointer has no target: the sanitizer pads each constructor pointer that the tract crates register, and the linker reads the padding as pointers.

Seed edits_apply, which runs edit lists through preset apply on the routes fixture, from tests/cmd/data/*.json. The corpus and the crash artifacts stay out of git. A crash becomes a unit test with the input bytes inline, next to the code it broke.

Golden numbers

• The recipe of the test signal lives in code, in the JSON vector that the generator writes and the Rust test reads.

• The expected value comes from an independent reference: librosa, SciPy or a closed form, never our own code. The generator, xtask/golden/generate.py, computes it once. Its output is committed as JSON under tests/fixtures/golden/ with the recipe, the generator, the tool versions, the values and the tolerance.

• xtask/golden/.python-version pins the interpreter and xtask/golden/requirements.txt pins every package with its hashes. cargo xtask golden installs them with uv into target/golden-venv, runs the generator’s tests in xtask/golden/test_generate.py, and rewrites every vector. A second run with the same pins writes byte-identical files.

• Each vector stores the SHA-256 of its script and of requirements.txt, with LF line ends, under sources. cargo xtask golden prints each value that changed, and fails after the rewrite when an issue figure’s outcome flipped, a reproduced flag or any boolean in rows: check each flip against its issue before you commit it.

• requirements.in lists the direct packages. To change a pin, edit it and regenerate the lock from the repository root:

  uv pip compile xtask/golden/requirements.in --generate-hashes --python-version 3.14.7 --output-file xtask/golden/requirements.txt
  

  Add --upgrade to move every package to its latest release. The command also stands in the header of requirements.txt.

• CI never runs Python. The test job runs the xtask test the_committed_vectors_match_the_pins, which fails when a vector lacks a field, lacks the Python version in tools, names a version other than the pin, or was made from another version of its script or of requirements.txt. It also fails when a vector of VECTORS in xtask/src/golden.rs is missing, when a file is in no recipe, and when a reproduced flag beside its rows disagrees with them. A test ties VECTORS to the recipes of generate.py. Two checks run only inside cargo xtask golden, in no CI job: the generator’s tests in xtask/golden/test_generate.py, among them the rule of a glide’s rise, and the flips of the reproduced flags that have no rows beside them, 42 of the 49. The same job runs the_workspace_lock_has_no_ort, which fails when Cargo.lock holds the ONNX Runtime crate of the transcription model, and the_fuzz_lock_names_the_workspace_version, which fails when fuzz/Cargo.lock names another version of the crate than Cargo.lock. cargo xtask golden-check, check-no-ort and check-fuzz-lock run the same checks locally.

• cargo run --release -p xtask -- eval scores the notes step on generated parts with known notes and prints its tables. It stays local, renders its audio on each run and commits nothing. Run it as a release build: the f0 tracker is part of our crate, so a debug run costs about 15 times the CPU time: 160 s against 10 s for the whole table. --part TEXT runs the matching parts and prints their notes, raw and clean. With --layer envelope, --part renders only the matching parts. It fails when a golden of the cleanup breaks. The dry 110 Hz saw must show its octave copy at MIDI 57 before the cleanup, and all 6 of its true notes after it: the model misses the fourth onset, and the attack split restores it. Its six notes through reverb must give the six true notes. Those stems cost seconds of CPU time each in a debug build, so no test in CI runs them.

• Noise comes from SplitMix64, which the generator and tests/it/support/rng.rs implement bit for bit, so a noise recipe gives the same signal in both languages.

• Tests compare through assert_close in tests/it/support/golden.rs with the stored tolerance, |actual - expected| <= abs + rel |expected|, and name the source of the number in a comment.

• tests/fixtures/golden/prototype/ holds the tables that task issues quote. Each is the issue’s method run on a recipe given in the file; where the issue stated no recipe, the file’s recipe defines it and its numbers replace the issue’s.

• To change a reference: edit the generator or the pins, run cargo xtask golden, and commit the vectors with the change.