🚨 Hugging Face just open sourced a complete real-time voice AI pipeline. Speak. It listens. It thinks. It talks back. End to end. Running on your GPU. Free. No OpenAI Realtime API fees. No ElevenLabs per-character billing. No Google Cloud per-minute pricing. Just a GPU and an internet connection. It's called speech-to-speech. Built and maintained by Hugging Face. And it does something no other open-source project has assembled cleanly until now. Here's what makes real-time voice AI hard. Every voice AI pipeline has four stages: speech recognition (you speak → text), language model (text → response text), text-to-speech (response text → audio), and audio output. Each stage adds latency. Chain them together naively and you get a system that feels slow — the pause between you finishing a sentence and the AI starting to respond breaks the conversational illusion. speech-to-speech is built around minimizing that latency at every stage simultaneously. Here's the full stack it ships with: Speech Recognition (STR): → Whisper (local) — OpenAI's transcription model, runs fully offline → Faster-Whisper — 4x faster inference with same accuracy → Distil-Whisper — smallest and fastest, lowest latency → Paraformer — Chinese language specialist Language Model (LLM): → Any Transformers-compatible model — Llama, Mistral, Qwen, anything → Any OpenAI-compatible API endpoint — swap in Claude, GPT, Gemini → MLX-optimized models for Apple Silicon — runs efficiently on Mac Text-to-Speech (TTS): → Parler-TTS — controllable voice with description-based prompting → MeloTTS — multilingual, fast → ChatTTS — natural conversational prosody → HF Inference Endpoints — offload TTS to Hugging Face servers when needed Mix and match. Any STR with any LLM with any TTS. Test combinations. Find the lowest latency stack for your hardware. Here's the wildest part. It ships with a Language Model Speech (LMS) mode — an experimental architecture where the LLM generates audio tokens directly instead of text tokens. No separate TTS stage. The model thinks in audio. This is the architecture that makes GPT-4o Advanced Voice feel natural — the model is generating speech as a native output, not converting text to speech after the fact. HF's open-source version lets you experiment with this architecture on your own hardware. And there's a VAD (Voice Activity Detection) system that detects when you stop speaking in real time — no fixed silence threshold, no manual push-to-talk. The pipeline responds the moment you finish a sentence. Here's the cost comparison that makes this worth caring about. OpenAI Realtime API: $0.06 per minute input, $0.24 per minute output. A one-hour conversation: $18. A developer building a voice AI application with 1,000 daily users: $18,000/day in API costs. speech-to-speech on a single A100: $0. Your only cost is the GPU rental. For production voice AI at any scale, the economics are not close. One command to install. 9.6K GitHub stars. 944 forks. Apache 2.0 License. 100% Open Source. From Hugging Face. GitHub link in the comments 👇