@DoItRealTime
Account based inUnited States
About this account
- Account based in
- United States
- Connected via
- United States App Store
Account-level information from X, not a live location or the device used for a specific post.
PhD candidate @PrincetonCS audio ML. @CCRMA/@Stanford, @BrownUniversity
Princeton, NJ
Joined November 2016
- Tweets266
- Following450
- Followers1.9K
- Likes2K
You know CLAP and may know SLAP. Allow me to introduce SynAPSE: FM Synthesizer Audio-Parameter Shared Embeddings. It's a joint embedding of Yamaha DX7 presets and audio. It also investigates a quirky question: is it possible to encode FM topologies not seen during training? 🧵
Our method uses the non-contrastive training objective from SLAP (h/t @Juj_Guinot @howariou @elio_elioo). Like SLAP, we observe a small modality gap. We know text-to-audio relied on CLAP. What's next for SynAPSE? 🤔
Thanks for reading. The code and weights are online: github.com/DBraun/SynAPSE
arXiv: arxiv.org/abs/2608.18226
Interactive retrieval demo (don't try on mobile): dbraun.github.io/SynAPSE
See you at DAFx 2026 where I'll present on 9/2!
I’ve been interested in algorithms related to M.C. Escher’s works since 2018 when I implemented an automatic coloring system based on the master’s thesis of Stephen Ogden. At the time, there was no source code for the algorithm nor publicly available design tool (dead links).
David Braun retweeted
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
David Braun retweeted
``DAC-JAX: A JAX Implementation of the Descript Audio Codec,'' David Braun, ift.tt/FzlXUdq
Happy to release "DAC-JAX: A JAX Implementation of the Descript Audio Codec." This can reuse PyTorch weights of all model sizes, and it includes a device-parallel training script. It uses the standard JAX libraries: Flax, Optax, Orbax, and CLU. github.com/DBraun/DAC-JAX
I benchmarked the chunked compression/decompression speeds. These are the functions you would use on long files or streaming. For a hop size of 8.2 ms, JAX performs compression in 7.1 ms and decompression in 4.3 ms. PyTorch performs compression in 8.3 ms, decompression in 6.3 ms.
In other words, if I wanted to explore usage of a real-time, 8.2 ms latency, 44.1 kHz DAC model, JAX might be faster. Of course, more analysis and testing are welcome. More details in the paper linked at github.com/DBraun/DAC-JAX
New talk and workshop on Faust+JAX, this time with parameter *automation*. With simple SGD and L1 time-domain loss over the input audio and ground truth, we recover the parameter automation of a lowpass filter's cutoff frequency. youtube.com/watch?v=046Gi7Wh…
Another example shows a differentiable polyphonic wavetable synth. The wavetables (2048 sample arrays) are learnable as well as the "Wavetable Position" which blends between them. Notice that the middle plot is a blend of a sine and triangle, but all the wavetables are learnable!
The afternoon workshop is here youtube.com/watch?v=VIlCY7wR… Thanks to the Programmable Audio Workshop (@Inria @insadelyon @Grame_Lyon) for having me!
