Guitarist, Researcher Google DeepMind. Opinions are my own. @jesseengel.bsky.social @jesseengel@sigmoid.social
Joined April 2009
- Tweets2K
- Following65
- Followers9.2K
- Likes4.9K
Pinned Tweet
Had one of those "Oh, I'm living in the future" moments yesterday.
Flying on the airplane. No internet.
Playing music with hand gestures controlling a realtime LLM running locally on my laptop
Person next to me thinking I'm crazy... all the while I'm having a great time 😅
Jesse Engel retweeted
ported Google's Magenta RealTime 2
to run locally on Windows with a consumer
GPU
@GoogleMagenta
Jesse Engel retweeted
Using LFOs to modulate language strength of continuous audio inference using Magenta Real Time 2 (Small), generated on an M5. My prompts are deliberately vague here. Sounds amazing and unsettling.
Jesse Engel retweeted
Making more progress on this, added internal recording and settings capture, live embedding of audio, more latent warps/prompt bends/stream corruptions
Really amazing work getting MRT2 running remotely, porting to pytorch, and streaming through the browser... and it hasn't even been a day!
I' m so addicted to @GoogleMagenta RealTime 2 🎹
so to justify playing with it 24/7 I ported the real-time apps to @huggingface Spaces 🤗 (and ported the entire lib to pytorch/transformers too)
Play live in real time on
huggingface.co/magenta-commu…
The MIDI control is just a hack with media pipe I haven't had time to push yet, but any MIDI controller would do.
You can try the models yourself: g.co/magenta/mrt2
I always love listening to Noah playing MRT2, he's really brought a lot of his musician sensibilities into developing the model.
That was one of the fun parts, we all came from different musical backgrounds, and all have our own favorite way of using it.
Jesse Engel retweeted
This one feels special. Finally we have real time on device generation w/ instrument-like (200ms) latency. Run it in your DAW or on an airplane! Proud to have been a part of this
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
It's been a joy working with Yotam over these years. From NSynth and AI Duet, to Magenta RealTIme 2, there's a thread of building actual instruments with this technology that is close to both our hearts.
Jesse Engel retweeted
This year marks the 10 year anniversary of Magenta! We’re releasing Magenta RealTime 2, which draws from so many of the ideas that the group has been pursuing over the past 10 years: interactive, genre-bending, open-source tools by and for musicians
I'm so proud of the team for this work. Live, Local, Open, MRT2 is just a lot of fun, and is an AI Instrument puts the person back at the center of making music.
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
Jesse Engel retweeted
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
Kyle revived MusicTransformer! If anyone wants a little taste of history.
Fun Fact: This was actually one of the first applications of transformers because the team sat next to us back in the day.
@huangcza @iansimon @notwaldorf tiny labor of love to revive an old model ♡ kylemcdonald.github.io/Music…
We've got a new model coming out next week! We've been having a lot of fun playing with it, and I hope you will too♥️
We'll be celebrating by presenting at the AI Music Summit at Berklee and helping teams at the hackathon afterwards build some wild new musical instruments 🎸
I just can't bring myself to vaugepost, feels too 2025 cringe. So.....
We've got a new live music model coming out soon!
If you live in NYC, come check out it out in person and watch the amazing @eardrummer_man do a live interactive performance with it!
partiful.com/e/RimqfnAOChN38…
I guess we were just ahead of the curve... :)
Lo-Fi Player share.google/WjKVM95F4QC1Mx2…
INTRODUCING CLAUDE MUSIC
The $900B company @claudeai is literally streaming LOFI music on YouTube 💀
Jesse Engel retweeted
I work on the JAX team. If you're new to the field ignore this bait. The things you should focus on are understanding the math and how to program accelerators - really master your hardware and your methods. We try to make a great tool but don't obsess over tools early.
Jesse Engel retweeted
Super cool stuff from my awesome @GoogleDeepMind colleagues - real time infinite sounds on an eminently hackable base. It’s everything I love about AI
Today, we're open sourcing the code behind "The Infinite Crate," our VST/AU plugin that lets you play with Lyria RealTime directly inside your favorite DAW! 🎧🎶
Since we released The Infinite Crate last year, it’s been used by some of our favorite artists - including a wonderful showcase with @daitomanabe in Tokyo - and being featured as a top new music tool at NAMM 2026.
We’re now fully open sourcing the plugin for developers to fork, modify, and make their own under the permissive Apache 2.0 license.
💾 Get it here: github.com/magenta/the-infin…
Today, we're open sourcing the code behind "The Infinite Crate," our VST/AU plugin that lets you play with Lyria RealTime directly inside your favorite DAW! 🎧🎶
Since we released The Infinite Crate last year, it’s been used by some of our favorite artists - including a wonderful showcase with @daitomanabe in Tokyo - and being featured as a top new music tool at NAMM 2026.
We’re now fully open sourcing the plugin for developers to fork, modify, and make their own under the permissive Apache 2.0 license.
💾 Get it here: github.com/magenta/the-infin…
This work has been a labor of love of Ashutosh Desai (@desaiashu). He also releases awesome music through his project Tōsh (IG: @toshbeats), check him out 🎵.
Many thanks to Spencer Salazar for his talk on prototyping DAW plugins in web technologies at ADC 2020, JUCE for implementing a C++ to Web/JS bridge in JUCE 8, to the support of artists Sidecar Tommy from Beats Antique (@sidecartommy), Toro y Moi (@ToroyMoi) in exploring the possibilities of DAW integration of our models.
Thanks also to the Magenta Team, particularly Noah Constant (@noahconst) for studio experiments and managing development, Ilaria Manco (@Ilaria__Manco), Cassie Tarakajian (@hellothisiscass), Jeff Chang (IG: @instajeffchang), and Adam Roberts (@adarob), Alberto Lalama and Joyce Xie for their work on the API, Nikhil Bhanu for his work on the windows build, and the whole Lyria team that helped contribute to Lyria RealTime.
More info and links in the blog post:
g.co/magenta/oss-infinite-cr…
