Indie lab. World-building at scale. | Try our latest experiment: https://xitter.cf/t.co/RKd90EwQYI
Los Angeles, CA
Joined January 2025
- Tweets1.2K
- Following29
- Followers38
- Likes129
We're sharing a preview of an internal model we're code naming "Haydn-01", unlike traditional text-to-speech, or audio understanding modes like @OpenAI ChatGPT voice, @Google Gemini TTS, or @ElevenLabs, etc.
Haydn-01 does 3 things that no other model does in a single architecture:
1) Audio classification (pass in a speech sample and ask it if the person seems angry, sad, happy...)
2) Real-time conversation (TTFC @ 2.1s / 0.88 RTF)
3) Non-speech audio understanding (identifies animal noises, crowded places, loud environments, and refuses to predict understanding when not possible)
Thanks to open contributions from @mmastrac (djev), @trynarilabs (DiaTTS), @GoogleDeepMind (DiffusionGemma), @typesafeai (Jev), vLLM, @modal (compute), and Hugging Face
credits: TaDiCodec (Amphion, NeurIPS '25) and Future (artist), Epic Records, Freebandz
About a year ago, the Magenta team at @GoogleDeepMind worked with artists @hollyherndon and @matdryhurst to finetune Magenta RT, their open-weights real-time music model, on Holly's vocals.
The base model was trained on roughly 190k hours of mostly instrumental stock music, so voice was one of its weakest areas out of the box.
Post-training on Holly's vocals turned that weakness into a specialization. The general musical ability from pre-training is still there, but the output distribution has shifted toward sounding like Holly.
For any organization sitting on specialized knowledge, post-training your own model is the next step.
fundamentally this lacks taste
Introducing Phoenix-4.5, the fastest and most expressive real-time human rendering model on the market.
More natural than ever before: richer facial animation, more expressive emotion and micro-movements, and movement that now extends through the upper body.
It is the closest AI has ever come to passing the Turing test face to face.
We're experimenting with music. Introducing J16, an infinite playback engine that is laying the foundation for new music experiences.
Check out the demo. This is not a public model. This is an experiment and access is limited to select industry partners and artists we hope to collaborate with.
ie. @hollyherndon
we're experimenting with how we can restructure parts of a track to bring back the experience of listening to a song for the first time. we call this resounding.
bronze.ai/listen/jai-paul/ Bronze AI is a London-based artificial intelligence audio technology company. They have succesfully created a generative music engine, although their demo only works with selected works. A general solution is possible.
the biggest marketing stunt that got pulled off this past year was the successful rebranding a sonnet-sized model into opus and real opus into Fable and Mythos.
We're releasing a new experience! It's a daily briefing service.
You wake up everyday with a million things on your mind. Let us take care of the part of you that is anxious about staying ahead.
Half of all people commute to work. This one is for you all.
Our daily briefing service works over text and with your Podcasts app to deliver 15-30 min. long briefs to get you caught up before waking into work.
Built with your Nomenclature.
We're releasing Colette-01. A post-trained persona model designed to be a culinary expert.
Colette beat other generalist frontier models 71% of the time in culinary-related tasks. Read the entire study at genexperiences.com.
try: colette.genexperiences.com
