An open research lab building artificial general intelligence. Join Discord ⚡️ https://xitter.cf/t.co/FLvN5uXGlK

Joined January 2025
OpenMOSS retweeted
🚨MOSS-VL-Realtime now runs through a complete browser-to-GPU stack @Open_MOSS 💡Realtime Demo: a browser app for live voice/video interaction — video calls, screen sharing, and interruption, no installation needed 🛠️ SGLang-Omni: a specialized inference backend that keeps real-time video calls smooth under concurrent users 🧠 Long-term memory: sliding-window video memory + on-demand recall, so the assistant remembers what you talked about even in long conversations → Bring up the full pipeline with a one-click installer The full model-to-application pipeline is now open source. The third_party directory also includes a FlashAttention-3 module customized for MOSS-VL; mainline @sgl_project support for offline batch inference is already merged into upstream SGLang. 🔥Stay tuned for our real-time API and online Playground without needing to prepare your own GPU! More details below ↓
3
4
14
1,348
OpenMOSS retweeted
MOSS-VL is now supported in the ms-swift main branch 🚀 Multimodal data—including images and videos—can now be used directly in the existing supervised fine-tuning (SFT) workflow. Two fine-tuning options have been validated: • LoRA: A lower-cost way to validate new datasets and tasks • Full SFT: Jointly trains the language model, vision module, and alignment module 🔄 Fine-tuned checkpoints can also be loaded directly with Swift for inference. In our basketball-game test, fine-tuning improved all three evaluated metrics: 🏀 JSON/schema validity rate: 98.18% → 100% 🏀 Event-type LCS precision: 82.35% → 100% 🏀 Jersey-color accuracy: 34.12% → 57.65% Special thanks to @ModelScope2022 for the support 🙌 Try MOSS-VL with ms-swift 👇
🤖 Made with AI
11
2
2
24
4,408
OpenMOSS retweeted
MOSS-VL is now supported in LlamaFactory @llamafactory_ai MOSS-VL has been merged into the LlamaFactory main branch, making it easy to fine-tune for your own domain. You can now fine-tune MOSS-VL for image, video, and other multimodal tasks using the familiar LlamaFactory workflow — from data preparation and training to checkpoint resume and inference. What’s been validated end-to-end: 🚀 LoRA fine-tuning — ideal for fast, low-cost experimentation ⚡ Full fine-tuning — for deeper domain adaptation 🖥️ WebUI — configure models, datasets, and training parameters without writing a custom training script 🔄 Complete training workflow — training, saving, checkpoint resume, inference, and weight merging are all supported Resources 👇
1
3
2
10
1,533
OpenMOSS retweeted
24GB VRAM is enough to run MOSS-VL locally.@Open_MOSS We’ve released FP8 and NF4 versions for both MOSS-VL-Instruct and MOSS-VL-Realtime — covering image, video, and real-time streaming understanding. 🖼️ MOSS-VL-Instruct — image & video understanding Built for local inference, batch processing, and serving. 🎥 MOSS-VL-Realtime — understand video as it happens Built for cameras, livestreams, and continuous video analysis, with timestamp-aware streaming understanding that continuously responds as the video unfolds. Each model now comes in two quantization options: ⚡ FP8 — balances model capability and inference performance ⚡ NF4 — pushes VRAM usage even lower and can run stably within 24GB VRAM For real-time video understanding, we recommend MOSS-VL-Realtime-NF4 for more VRAM headroom and a more stable long-context experience. FP8 is a good fit when context length is more controlled. We also compared all four quantized checkpoints with their original BF16 models across selected image, video, and real-time streaming benchmarks. Overall, performance remains close to BF16 while VRAM usage drops substantially — making MOSS-VL much easier to deploy on 24GB consumer GPUs. All 4 checkpoints are available now ↓ 🖼️ MOSS-VL-Instruct-0708-FP8 🤗 Hugging Face: huggingface.co/OpenMOSS-Team… 🌐 ModelScope: modelscope.cn/models/openmos… 🖼️ MOSS-VL-Instruct-0708-NF4 🤗 Hugging Face: huggingface.co/OpenMOSS-Team… 🌐 ModelScope: modelscope.cn/models/openmos… 🎥 MOSS-VL-Realtime-FP8 🤗 Hugging Face: huggingface.co/OpenMOSS-Team… 🌐 ModelScope: modelscope.cn/models/openmos… 🎥 MOSS-VL-Realtime-NF4 🤗 Hugging Face: huggingface.co/OpenMOSS-Team… 🌐 ModelScope: modelscope.cn/models/openmos…
5
20
2
129
19,033
youtube.com/shorts/yRZ10vUMB… built a fully autonomous and autark video pipeline. MOSS handles the voice cloning end to end. script → cloned voice → rendered short, zero human touch.
2
3
276
MOSS-Transcribe-Diarize (@Open_MOSS) is criminally underrated. 0.9B params, writes the words AND names the voices in one pass. It’s the engine behind every meeting in @getEverPaper , fully on-device, no cloud. 56-min, 7-person call on a MacBook: every speaker correct. Absurd model. Check it out!
2
2
7
527
OpenMOSS retweeted
We're at WAIC 2026! @MosiAI_Official & @Open_MOSS is bringing MOSS Context Intelligence Park to Shanghai! 📍世博展览馆 Booth H1-C1234|📅 July 17–20 Experience how AI listens, sees, understands context, and creates in the real world: 🔊 Voice Babel Tower - light up a 5m AI voice installation with your own voice 🎙️ Mossland Creator Booth - immersive AIGC creation with AI voice ❓ VBTI - discover your voice personality 🤖 Live demos of MOSS-TTS, MOSS-Transcribe-Diarize & MOSS-VL-Realtime Come meet us at WAIC 2026! 🌏 #waic2026
🤖 Made with AI
1
4
12
1,111
This week @Open_MOSS released MOSS-Transcribe-Diarize, a 0.9B open model (Apache 2.0) that transcribes, diarizes, and timestamps in a single pass. I used it to make 174 hours of Apollo 11 mission audio searchable by who said what, when. Total cost: $9.46. These are the real NASA tapes from July 1969, hosted by the @internetarchive. All 103 run through one @huggingface Job: a100-large, 3.8h, 47x realtime, @sgl_project serving the model inside the job. 45,355 timestamped speaker segments. Search the mission and hear any moment from the original tape! Space: huggingface.co/spaces/davans…
8
11
3
53
13,046
🎉 Congrats to @MosiAI_Official + @Open_MOSS on the release of MOSS-Transcribe-Diarize-0.9B — a genuinely impressive end-to-end ASR model for real-world, multi-speaker conversations. We're proud to partner with them for day-0 support on mlx-audio (Python & Swift). Now running local-first on Apple Silicon: 🎙️ transcript + speaker labels + timestamps in one pass 🗣️ unlimited-speaker diarization ⏱️ up to ~90 min of audio per input 🌍 90+ languages 🧠 128K context • Whisper-Medium encoder + Qwen3-0.6B decoder 🎯 hotword boosting for names, products & domain terms Perfect for meetings, podcasts, interviews, and long-form call analysis — no cloud, no data leaving your Mac. Get started now: 🐍 Python > uv pip install -U mlx-audio 🍎 Swift .package(url: "github.com/Blaizzy/mlx-audio…", from: "0.1.3") Go build. 🚀
🤗 MOSS-Transcribe-Diarize-0.9B is now open source on @huggingface. Built with an end-to-end audio-to-structured-transcript paradigm: >0.9B open-source ASR model >Apache license 2.0 >128k long-context transcription >Up to ~90-min audio input >Speaker labels + timestamps in one generation >Multi-speaker diarization for meetings, interruptions, and overlapping voices >Hotword biasing for names, terms, and domain-specific vocabulary >~100 token/s on NVIDIA RTX 4090, RTF ~0.017 Thank you @sgl_project @vllm_project @Prince_Canuma @lllucas for day-0 support! 🚀 Github: github.com/OpenMOSS/MOSS-Tra… Huggingface: huggingface.co/spaces/OpenMO… API: shorturl.at/DWwe3 Live demo: shorturl.at/wRZ3j Technical Report:arxiv.org/abs/2601.01554 HF Space: huggingface.co/spaces/OpenMO… AtomGit:ai.atomgit.com/OpenMOSS/MOSS… SGLang-Omni: github.com/sgl-project/sglan… vLLM: github.com/vllm-project/vllm MLX-audio: github.com/Blaizzy/mlx-audio Discord:discord.gg/SmVQHGffZU
10
19
1
92
11,694
OpenMOSS retweeted
🤗 MOSS Transcribe series update.@Open_MOSS Alongside MOSS-Transcribe-Diarize-0.9B, we are also updating two models: MOSS-Transcribe-Diarize Pro >Flagship multi-speaker transcription for complex meetings, long audio, multilingual scenarios, and enterprise API use. MOSS-Transcribe >Open-source ASR for challenging English speech, including standard English, diverse accents, quiet speech, and whispers. Built for enterprise meetings, call-center QA, podcast transcription, interview cleanup, and content production.
11
19
1
124
9,571
OpenMOSS retweeted
People may think serving and optimizing TTS (Text to Speech) model is similar to traditional LLM. It isn't. We made Higgs and MOSS-TTS 2–3.4× faster in SGLang-Omni — and most of the wins landed outside the LLM backbone. Delay patterns, D2H stalls, a vocoder tail-latency killer... In this post, we share everything we learned — every optimization, and every trade-off behind it. We hope it proves a valuable resource for developers building in this space. Full breakdown: xitter.cf/YichiZ03/status/207464… @sgl_project
11
14
2
97
22,572
OpenMOSS retweeted
🚀 MOSS-TTS Local Transformer v1.5 is now available on Novita. Build voice agents with: 🎙️ Zero-shot voice cloning 🌍 Speech synthesis in 30+ languages 🎧 Native 48 kHz stereo audio ⚡ Streaming TTS with ultra-low latency Thanks to @MosiAI_Official for open-sourcing MOSS-TTS.
6
5
2
25
2,740
OpenMOSS retweeted
OpenMoss has been releasing some impressive TTS 🔥 MOSS-TTS-Local v1.5 🔊 - Generates native 48kHz stereo - Support 30+ languages - 5B & Apache 2.0 Model and demo are available on @huggingface
6
11
1
102
7,377
mlx-audio + MOSS-TTS-Local-Transformer-v1.5 + Kling Omni with lip sync 🔥 Thanks @MosiAI_Official for this super model! X: what can I use locally on Apple Silicon to generate videos with lip sync?
10
9
1
96
8,815
OpenMOSS retweeted
omg this made my day 🥹 Someone already made a ComfyUI node pack for @MosiAI_Official @Open_MOSS MOSS-TTS Local Transformer v1.5 Clone voices, generate speech in 30+ languages, and export 48 kHz stereo audio right from your workflow!! thank you for building this🩵 @drbaph
5
4
27
1,774
推荐一家叫模思的中国公司的开源声音克隆仓库: MOSS-TTS 你朗读一段文字,它克隆你的声音,然后就可以用你的声音朗读任意文本,查看帖子详情看我实战如何使用,效果很好,可以以假乱真。 github.com/OpenMOSS/MOSS-TTS 模型下载: huggingface.co/OpenMOSS-Team… 实战过程: 1. 在codex里输入如下提示词帮我安装和运行: ``` read huggingface.co/OpenMOSS-Team…, 在我本地安装和运行 ``` 2. code开始下载模型,但是非常慢,需要提示它用aria2来下载 3. 下载完毕后,让它给出如何克隆我的声音的步骤,按照步骤操作 4. 克隆声音后,让它根据我的声音生成朗读李白的《静夜思》的音频文件 我听了下效果非常好!非常像我的声音! 关注我,永远有实战,而不是简单转发!!
🤗 MOSS-TTS-Local Transformer v1.5 is now open source. Built with a pure autoregressive Audio Tokenizer + LLM paradigm: >MOSS-Audio-Tokenizer-v2, 2B params >Qwen3-4B backbone >Native 48 kHz stereo audio >Streaming output with theoretical sub-100 ms TTFT >Zero-shot voice cloning >Inline [pause] control >🇺🇸 🇯🇵 🇰🇷 31 language synthesis >SGLang-Omni Day0 support 🎉 @sgl_project @lmsysorg Designed for voice agents, digital humans, game NPCs, audiobooks, and real-time speech generation. 👇
14
84
1
374
47,930
🤗 MOSS-TTS-Local Transformer v1.5 is now open source. Built with a pure autoregressive Audio Tokenizer + LLM paradigm: >MOSS-Audio-Tokenizer-v2, 2B params >Qwen3-4B backbone >Native 48 kHz stereo audio >Streaming output with theoretical sub-100 ms TTFT >Zero-shot voice cloning >Inline [pause] control >🇺🇸 🇯🇵 🇰🇷 31 language synthesis >SGLang-Omni Day0 support 🎉 @sgl_project @lmsysorg Designed for voice agents, digital humans, game NPCs, audiobooks, and real-time speech generation. 👇
2
1
3
320
OpenMOSS retweeted
🤗 our latest tts model is open source on @huggingface > 48hz > 30+ languages supported > streaming output > inline pause control
🤗 MOSS-TTS-Local Transformer v1.5 is now open source. Built with a pure autoregressive Audio Tokenizer + LLM paradigm: >MOSS-Audio-Tokenizer-v2, 2B params >Qwen3-4B backbone >Native 48 kHz stereo audio >Streaming output with theoretical sub-100 ms TTFT >Zero-shot voice cloning >Inline [pause] control >🇺🇸 🇯🇵 🇰🇷 31 language synthesis >SGLang-Omni Day0 support 🎉 @sgl_project @lmsysorg Designed for voice agents, digital humans, game NPCs, audiobooks, and real-time speech generation. 👇
8
12
1
112
14,873
OpenMOSS retweeted
🎉 SGLang-Omni now serves MOSS-TTS-Local Transformer v1.5 from @Open_MOSS on day 0! This is an open 48 kHz stereo TTS model built on a Qwen3-4B backbone. ✅ Zero-shot voice cloning + native streaming at 48 kHz stereo ✅ 31 languages, trained on ~4M hours of speech ✅ Duration control + explicit pause markup + long-form up to 10 min ✅ 5.976 req/s non-streaming at RTF 0.644, 1.75% WER (SeedTTS English, 2× GPU) ✅ Three-stage pipeline: reference encoding → AR engine → streaming vocoder, with frame-level CUDA Graphs Cookbook: sgl-project.github.io/sglang… Run it now with SGLang-Omni!
🤗 MOSS-TTS-Local Transformer v1.5 is now open source. Built with a pure autoregressive Audio Tokenizer + LLM paradigm: >MOSS-Audio-Tokenizer-v2, 2B params >Qwen3-4B backbone >Native 48 kHz stereo audio >Streaming output with theoretical sub-100 ms TTFT >Zero-shot voice cloning >Inline [pause] control >🇺🇸 🇯🇵 🇰🇷 31 language synthesis >SGLang-Omni Day0 support 🎉 @sgl_project @lmsysorg Designed for voice agents, digital humans, game NPCs, audiobooks, and real-time speech generation. 👇
6
14
4
93
11,522