If you're looking for a weekend project, how about training your own text-to-speech model from scratch on your own GPU, and then running it on any device's CPU? We just open-sourced the entire Pocket TTS training stack: data pipeline, recipes, and evals. It learns pretty damn fast: ~15k steps: babbling starts turning into words ~50k steps: it reads anything you type (WER under 1%) ~200k steps: the voice stops sounding synthetic On a beefy consumer GPU, that's a week of training. On eight H100s: 10-20 hours. A TTS training run will cost you less than $200 if you rent your hardware, and an order of magnitude less if you just pay for power. Some things we'd love to see people try: - Train it in your own language (a few hundred hours of speech gets you surprisingly far). - Add new features to Pocket TTS (Emotion tags? Make it sing?). - Beat us at our own game: make it faster and smaller. Show us what you build! We'll highlight the best models and new languages for the whole community to enjoy. Pocket TTS has already found many use cases, from reading for people with visual impairments to making NPCs in video games talk, and we're sure there's much more to do with it! Here's an example of a Czech Pocket TTS. Try just asking your favorite agent to find data and apply the method, and you can have your own. Get started: github.com/kyutai-labs/pocke…

Our audio-to-MIDI model, MuScriptor, now also detects tempo! You can directly drag-and-drop the MIDI into a DAW and it just works, without you having to tempo-match manually. Have fun!

We're releasing MuScriptor, the best open model for multi-instrument transcription to date, created in collaboration with @MireloAI. Give it a recording in any genre: pop, classical, metal, jazz, whatever, and it transcribes the individual instruments into MIDI. Link in 🧵

1/2 We’re releasing an in-depth tutorial on neural audio codecs, the secret sauce that makes it possible for audio LLMs to not sound like a horror movie:

Kyutai Speech-To-Text is now open-source! It’s streaming, supports batched inference, and runs blazingly fast: perfect for interactive applications. Check out the details here: kyutai.org/next/stt

Talk to unmute.sh 🔊, the most modular voice AI around. Empower any text LLM with voice, instantly, by wrapping it with our new speech-to-text and text-to-speech. Any personality, any voice. Interruptible, smart turn-taking. We’ll open-source everything within the next few weeks.

Meet Hibiki, our simultaneous speech-to-speech translation model, currently supporting 🇫🇷➡️🇬🇧. Hibiki produces spoken and text translations of the input speech in real-time, while preserving the speaker’s voice and optimally adapting its pace based on the semantic content of the source speech. Based on objective and human evaluations, Hibiki outperforms previous systems for quality, naturalness and speaker similarity and approaches human interpreters. 🧵

Meet Helium-1 preview, our 2B multi-lingual LLM, targeting edge and mobile devices, released under a CC-BY license. Start building with it today! huggingface.co/kyutai/helium…