Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Credits

This list is the credits section of the project README.

This project builds on the research and open-source work below. Each entry gives the work’s license. “Inspiration only” marks a work we read for facts and ideas, from which we copy no code, text or data.

Research

Open-source projects

  • ableton-lom-skill (no license found): an instrument API reference packed as one agent skill with a topic router, as a model for the Serum 2 reference. Inspiration only.
  • ableton-mcp (MIT): parameter-by-index tool design, kept as a counter-example to named parameters.
  • Audio Flamingo (MIT code, non-commercial weights): audio language model judged weak on synth attributes. Inspiration only.
  • audio.cpp (Apache-2.0): facts on running audio models locally in C++.
  • auraloss (Apache-2.0): multi-resolution STFT loss, reimplemented in Rust.
  • AutoSynth (Apache-2.0): perturb-and-render recipe with logged edit masks for the test bench.
  • Basic Pitch (Apache-2.0): the upstream transcriber and decoder whose figures the notes epic reproduces.
  • basic-pitch-rs (Apache-2.0): the input format of the transcription model, a 22050 Hz mono signal, which sets the resample target of the stem intake.
  • BE-AFX (no license): blind effect-estimation auto-encoder with differentiable effects, read for design. Inspiration only.
  • BEATs (MIT): audio tagger facts for the model survey.
  • blind_rt60 (MIT): blind reverb-time formulas, reimplemented in Rust.
  • burn (MIT OR Apache-2.0): Rust inference runtime facts for the fallback plan.
  • candle (MIT OR Apache-2.0): Rust inference runtime facts for the fallback plan.
  • chordparser (MIT): a chord-symbol parser crate, a test-only oracle of the chord parser of the music dictionary.
  • CLAP (microsoft/CLAP) (MIT): audio tower used for the style-embedding study.
  • claude-collider (MIT): server paired with theory skills, and a short pitfall list.
  • cochlea (MIT OR Apache-2.0): spectrogram PNG for agents, and stereo probe formulas.
  • coreml-native (Apache-2.0 OR MIT): platform ML binding facts for the deferred runtime option.
  • CREPE (MIT): 360-bin pitch table and ONNX weights.
  • CREPE Notes (GPL-3.0): inverted pitch confidence plus pitch gradient to find note boundaries. Inspiration only.
  • CTAG (MIT): gradient-free, embedding-guided parameter search loop.
  • dasp-pytorch (Apache-2.0): tanh drive formula.
  • DawDreamer (GPL-3.0): how a JUCE host loads plug-in state, schedules MIDI and renders offline. Inspiration only.
  • DDSP (Apache-2.0): multi-scale spectral loss definition.
  • DDSynth-RL (Apache-2.0): multi-metric render-and-compare recipe, group-relative scoring, crash-isolated render workers.
  • DeepAFx-ST (non-commercial research license): neural effect style transfer, surveyed for the effects layer. Inspiration only.
  • DeepFormants (MIT): neural formant tracker, surveyed for filter resonance estimates.
  • dereverb-room (GPL-3.0): voice reverb-removal model, surveyed for wet-signal handling. Inspiration only.
  • dereverb_mel_band_roformer (GPL-3.0): vocal reverb-removal checkpoints, surveyed for wet-signal handling. Inspiration only.
  • Descript Audio Codec (MIT): neural codec latents, surveyed as audio features.
  • diff-wave-synth (no license): unofficial port of differentiable wavetable synthesis. Inspiration only.
  • DiffAPF TB-303 (MPL-2.0): differentiable all-pole filter experiments on acid-style sweeps. Inspiration only.
  • DiffMoog (MIT): chain-with-activation-flags template and spectral-distance findings.
  • diffsynth (MIT): differentiable ADSR module design.
  • DiffVox (MIT): chain design, losses and parameter priors for vocal effects.
  • EnCodec (MIT): neural codec latents, surveyed as audio features.
  • Essentia (AGPL-3.0): audio descriptor extractor design. Inspiration only.
  • Essentia models (CC BY-NC-SA 4.0): bright/dark, instrument and dry/wet classifier heads trained on NSynth. Inspiration only.
  • FCPE (MIT): comparison model and local-argmax decoding.
  • FCPE ONNX export (license not checked): the ONNX form of FCPE, tested for the f0 track and not used. Inspiration only.
  • ferraus-pitch (Apache-2.0 OR MIT): pYIN note segmentation defaults.
  • FL-Automation (MIT, in its sdk/ folders): frame-by-frame labels of Serum 2 wavetables and a state schema.
  • Flow Synthesizer (MIT): nested parameter subsets, audio-versus-parameter error, timbre tag vocabulary.
  • flp-extract-fxp (MIT): a corpus of Serum 2 factory preset metadata, tags, value ranges and description words.
  • FM-SynAPSE (MIT): shared audio and preset embedding, and the MFCC, CLAP, PANNs timbre comparison.
  • Fx-Encoder++ (CC BY-NC 4.0): instrument-wise effect embeddings, surveyed for distortion and effect layers. Inspiration only.
  • FXencoder (music_mixing_style_transfer) (MIT code, unlicensed weights): mixing-style effect embedding, surveyed for width and effect layers. Inspiration only.
  • gfx_classifier (BSD-3-Clause): guitar-effect classifier as a distortion bench probe, with its published models.
  • GRAFX (Apache-2.0): differentiable effect-graph framework, read for chain design.
  • Helm (GPL-3.0): preset role taxonomy of a synthesizer. Inspiration only.
  • helm_patches (CC BY 4.0, in Factory Presets/): preset category and name-word counts, and describing-word counts of the ranked word list at commit 5fafa70.
  • IIRNet (Apache-2.0): neural IIR filter design from a magnitude response.
  • JUCE (AGPL-3.0): VST3 state envelope and base64 facts. Inspiration only.
  • Kimi-Audio (Apache-2.0 and MIT): audio language model tested on timbre attributes, CUDA-only.
  • kymatio (BSD-3-Clause): scattering transform implementation, read for joint time-frequency scattering.
  • LAION-CLAP (CC0-1.0): 48 kHz, 10 s, 64-band log-mel recipe.
  • libf0 (MIT): YIN, pYIN and salience f0 estimator definitions.
  • librosa (ISC): MFCC, mel and spectral descriptor definitions, reimplemented in Rust, and the pyin function of release 0.9.1 that the f0 tracker follows step by step. Its notice ships in THIRD-PARTY-LICENSES.html.
  • llama-cpp-2 (MIT OR Apache-2.0): Rust binding facts for local audio language models.
  • LTU (no license): audio language model judged weak on timbre attributes. Inspiration only.
  • MERT (Apache-2.0 code, CC BY-NC 4.0 weights): music embedding model, surveyed for pitch and reverb probes. Inspiration only.
  • mir_eval (MIT): the note-transcription matching rules, reimplemented in the evaluation matcher (rules only), and the reference scores of its golden cases.
  • mlx-audio (MIT): facts on running audio models with MLX.
  • mlx-rs (MIT OR Apache-2.0): Rust MLX binding facts for the fallback plan.
  • mod_discovery (MPL-2.0): envelope and LFO control signals found with differentiable DSP. Inspiration only.
  • mod_extraction (MPL-2.0): LFO extraction from dry and wet audio, which needs a dry signal. Inspiration only.
  • MT3 (Apache-2.0): segment-seam bookkeeping and overlap clean-up rules.
  • MU-LLaMA (GPL-3.0): music language model judged weak on synth attributes. Inspiration only.
  • MuQ (MIT code, CC BY-NC 4.0 weights): music embedding model, surveyed for instrument and reverb probes. Inspiration only.
  • MuScriptor (MIT code, CC BY-NC 4.0 weights): hard instrument mask, gap filling, onset-lag calibration ideas. Inspiration only.
  • music21 (BSD-3-Clause): chord symbols in the pinned golden tools, a local oracle of the chord tones of the music dictionary.
  • nara_wpe (MIT): weighted prediction error dereverberation, read for reverb handling.
  • node-serum2-preset-packager (MIT): container layout, zstd level 3, and the Init preset shape.
  • objc2-core-ml (Zlib OR Apache-2.0 OR MIT): platform ML binding facts for the deferred runtime option.
  • Odin 2 (GPL-3.0 code, OFL-1.1 font, no preset terms stated): factory preset category counts and describing-word counts of the ranked word list at commit 265e9e2. Inspiration only.
  • onnxcrepe (MIT): ONNX export of CREPE and its weights.
  • OpenL3 (MIT code, CC BY 4.0 weights): audio embedding surveyed, with no ONNX export.
  • Pedalboard (GPL-3.0): headless VST3 instrument hosting design with MIDI in, audio out and state get and set. Inspiration only.
  • PESTO (LGPL-3.0): harmonic CQT and cached-convolution streaming idea. Inspiration only.
  • pitch-estimate (MIT): per-frame 55 Hz regression test layout.
  • Plugalyzer (GPL-3.0): CLI shape for rendering a VST3 instrument from MIDI to WAV. Inspiration only.
  • producer-pal (GPL-3.0-or-later): how an agent server assembles its skill text from fragments. Inspiration only.
  • pychord (MIT): a chord-quality table in the pinned golden tools, a local oracle of the chord tones of the music dictionary.
  • pYIN and Tony (GPL-2.0-or-later): pYIN note-tracking HMM on the pitch track. Inspiration only.
  • pyin-rs (MIT): the Rust port of librosa’s pyin. The f0 tracker in src/notes/pyin.rs ports its numerics and returns the same frames, with a banded Viterbi search. Its notice ships in THIRD-PARTY-LICENSES.html.
  • Qwen2-Audio (no license, Apache-2.0 weights): audio language model tested on timbre attributes. Inspiration only.
  • rack (MIT OR Apache-2.0): reference for plug-in state and MIDI mapping across formats.
  • RecklessSerum2Bank (MIT code, presets free to use per its README): 250 generated Serum 2 presets, the one open bank of parser fixtures.
  • RemFx (Apache-2.0 code, non-commercial weights): effect detection and removal chain, surveyed for the effects layer. Inspiration only.
  • RMVPE (Apache-2.0): local weighted-average decode over 9 bins.
  • rust-music-theory (MIT): scales and chords, the source of the offsets a test of the music dictionary pins.
  • SALMONN (Apache-2.0): audio language model judged weak on synth attributes.
  • SciPy (BSD-3-Clause): reference signal-processing routines, such as Hilbert and Butterworth filters.
  • serum (rich6feet) (no license found): an index of Serum 2 factory table names and content paths. Inspiration only.
  • Serum 2 preset format gist (0xdevalias) (no license): original reverse-engineering thread for the Serum preset format. Inspiration only.
  • Serum 2 parameter dump gist (KennethWussmann) (no license): parameter defaults of a freshly loaded Serum 2. Inspiration only.
  • serum-2 (cboone) (MIT code, CC0 1.0 assets): a working user-wavetable recipe and the Serum 2 file format notes.
  • serum-mcp (MIT): Serum 2 preset generation driven by a language model, and its parameter tables.
  • serum-preset-packager (no license): first public decoding of the Serum 2 container. Inspiration only.
  • serum-render (GPL-3.0): render reproducibility: batch state carry-over and warm-up render. Inspiration only.
  • serum2-neurosync (no license found): the folder listing of factory presets as a role taxonomy; no files copied; describing-word counts of its preset names for the ranked word list at commit cabec31. Inspiration only.
  • serum2-preset-loader (MIT): Serum 2 presets as plugin state, and JUCE state notes.
  • serum2gen (MIT): Serum 2 preset reading, writing and parameter aliases.
  • serum2vital (GPL-3.0): container facts, modulation source ids, LFO fields, FX rack structure, hosting quirks. Inspiration only.
  • SerumPreset (lwy0v0) (no license found): decoded preset payloads and the hash rule of the preset container. Inspiration only.
  • sms-tools (AGPL-3.0): spectral modeling analysis and synthesis reference. Inspiration only.
  • SPTK (Apache-2.0): reference LPC and cepstrum routines.
  • ssqueezepy (MIT): synchrosqueezing and wavelet transform implementation.
  • StemFX (MIT): per-stem effect chain retrieval and transfer.
  • Surge XT (GPL-3.0): the wavetable chunk format and the factory patch folders, and describing-word counts of patch names and comments for the ranked word list at commit a30341b. Inspiration only.
  • surge-xt-mcp (MIT): the pattern of setting any plugin parameter by name, and what it cannot reach.
  • SwiftF0 (MIT): change-point note segmentation with loudness-dip splits.
  • synth-permutations (MIT): metric definitions MSS, wMFCC, SOT and RMS, checked against our own tests.
  • synth-rl (GPL-3.0-or-later): edit, repair and match tasks, and the mute and blind controls. Inspiration only.
  • synth-setter (GPL-3.0): render loop and noise-floor tests. Inspiration only.
  • Syntheon (Apache-2.0): Init-preset-plus-minimal-subset approach and single-cycle wavetable extraction.
  • tonal (MIT): the chord, scale and Roman-numeral tables the music lexicon was compared with; the dictionary commits no row of them. Inspiration only.
  • torchlpc (MIT): differentiable linear-prediction filter, read for time-varying filters.
  • torchsynth (Apache-2.0): seeded random-parameter synthesizer idea for optimizer tests.
  • u-he Preset Randomizer (MIT): per-synth statistical guides and clamp-and-blend variation.
  • Vita (GPL-3.0): Python bindings over the open Vital engine, run as a separate process for test renders. Inspiration only.
  • Vital (GPL-3.0-or-later): preset file structure, parameter laws, and the mechanisms of every synth layer that the Vital reference files describe, in our own words. Our own tool extracts its parameter facts (names, ranges, defaults, engine scales, UI and menu labels) into src/preset/vital/registry.json for compatibility; no code or comments are copied. Inspiration only.
  • vital-mcp (GPL-3.0): render-measure-fix loop and its descriptor set. Inspiration only.
  • vital_discord_preset_archive (no license found): preset role and macro-label counts; no files copied; describing-word counts of preset names and comments for the ranked word list at commit c821ad9. Inspiration only.
  • WildFX (no license): DAW-hosted plugin pipeline for effect-estimation data. Inspiration only.
  • WORLD (BSD-3-Clause): CheapTrick F0-adaptive spectral envelope, ported with attribution if needed.
  • YAMNet (Apache-2.0): AudioSet event classifier, read as a distortion tag probe.
  • YourMT3 (GPL-3.0): the rules that cut a note at the next onset of its key and drop zero-length notes, in the MIDI writer. Inspiration only.

Datasets

  • AKWF-FREE (CC0-1.0): single-cycle waveforms as bench data for wavetable matching.
  • AudioSet ontology (CC BY-SA 4.0): class names for effect tags. Inspiration only.
  • Chordonomicon (CC BY-NC 4.0): progression shares and chord frequencies of the music dictionary, counts only. Inspiration only.
  • DX7 timbre dataset (CC BY 4.0): fixed-pitch timbre-order test for compare.
  • dx7-patches-and-prompts (Apache-2.0 per its card, patch origin and caption terms unclear): the vocabulary of generated patch captions. Inspiration only.
  • Faronbi et al. preset dataset (CC BY 4.0): Serum parameter-vector layout as a reference; its preset names feed only the describing-word counts of the ranked word list, as an observation corpus (record revision 2).
  • Freesound metadata scrape (CC BY-SA 3.0 per its card, each sound under its own license): tag, title and description word counts only; the uploader counts of the ranked word list at sha 5c427d9. Inspiration only.
  • GiantSteps drum-pattern datasets (no license found): shares of house, techno and breakbeat grids with a given kick or snare, counts only. Inspiration only.
  • GiantSteps key dataset (no license found): key and mode shares of annotated EDM tracks, counts only. Inspiration only.
  • GiantSteps MTG key dataset (no license found): key and mode shares by store genre, counts only. Inspiration only.
  • Groove MIDI Dataset (CC BY 4.0), Gillick, Roberts, Engel, Eck and Bamman: onset spread, offsets and velocities of real drum performances, computed by filtering and aggregation.
  • IDMT-SMT-Audio-Effects (CC BY-NC-ND 4.0): labelled guitar and bass effect recordings, surveyed for modulation and distortion. Inspiration only.
  • Lakh MIDI Dataset, clean subset (CC BY 4.0 for the collection, the songs keep their copyright): pitch ranges, note counts and repetition of instrument groups, counts only.
  • McGill Billboard annotations, through mirdata (CC0 per its loader): progression shares and harmonic rhythm of chart songs, counts only.
  • MidiCaps (CC BY-SA 4.0): mode and tempo shares by caption tag, counts only. Inspiration only.
  • MusicCaps (CC BY-SA 4.0): how listeners describe synth parts, from caption word counts; the caption counts of the ranked word list at sha 0a51889. Inspiration only.
  • NSynth (CC BY 4.0): bright/dark and pitch labels as a feature sanity check.
  • POP909 (MIT, the songs keep their copyright): the share of melody notes outside the key, counts only.
  • ReverbFX (CC BY-NC 4.0): room impulse responses derived from reverb plugins, as an effect-estimation reference. Inspiration only.
  • SocialFX listener data, re-upload (no license found), revision d7e266c: per-word agreement, curve counts, band means and reverb word effects of the word dictionary, derived by cargo xtask agreement; no row is committed. Inspiration only.
  • Surge pitch dataset (CC BY 4.0): pitch-labelled test audio and rank-based compare tests.
  • timbremetrics timbre ratings (Thoret et al., Vahidi et al.) (no license): 21 human timbre-rating sets as the compare benchmark. Inspiration only.