@usemuna

Structurally cheaper inference for open models.

muna.ai
Joined December 2021
  • Tweets11
  • Following3
  • Followers139
6Media
DeepSeek V4.1 Flash + GLM 5.3 Flash on the same 4脳 B200. TP4 each, one process. Alone: 1,305 / 1,709 tok/s Sharing: 764 / 787 tok/s (1,551 combined) Decode step while sharing: ~20 ms each GPUs are impossible to find. Do more with the ones you already have.
1
1
2
269
Let there be tokens 馃殌
Brett Harrison
@BrettHarrisonOct 14
The Liquid Inference router network is growing quickly. We now have 28 verified companies serving >1000 models, offering capacity dynamically and updating quotes in real-time. Average time for providers to onboard and integrate models is 3 hours, full verification within a day.
4
11
391
Most inference GPUs are ~70% idle. It's not a kernel problem, or a traffic problem. It's a placement problem. To prove it, we built an inference service that co-locates models on a GPU. 5 co-located models. 14 days. 2.5B tokens served. $17k/mo saved.

The Next 3脳 in Inference Won't Come From Faster Kernels

Article 路 @usemuna
4
1
1
13
1,382
Today, we are launching our collaboration with @nomic_ai to make AI agents more effectively and efficiently understand complex PDF documents. Nomic's new nomic-layout-v1 model allows your AI agents to parse documents locally, so sensitive documents never leave your machine.
2
6
3
17
3,136
more replies
Read more on our blog: muna.ai/blog/nomic-layout

Pioneering Private, Local Document Intelligence with Nomic Layout - Muna

Extract valuable data from complex documents with nomic-layout-v1.

muna.ai
1
2
138
And try out the demo! pdflayout.ai
1
1
2
761
Load more