infra for ambient intelligence

Everywhere
Joined August 2014
Fluid Inference retweeted
We distilled clef-flash into a 0.6B model and supporting ANE option. 1,000 support tickets triaged on a M5 pro in 57 s: team, urgency and refund for each, 48 ms per ticket. Within a few points of the 9B on classification. 🤗 huggingface.co/FluidInferenc… 💻 github.com/FluidInference/Fl…
we've got more 𝚌𝚕𝚎𝚏 for you: • new 𝚌𝚕𝚎𝚏-𝚘𝚖𝚗𝚒 - takes audio, video, image, and text • 𝚌𝚕𝚎𝚏-𝚏𝚕𝚊𝚜𝚑 is now cheaper than jev • 𝚌𝚕𝚎𝚏 is now ~2x faster blog.cloudflare.com/clef-fas…
14
45
7
494
32,168
🥳🥳🥳
FluidAudio just reached 3,000 stars ⭐ Thank you to everyone who starred it, used it, demoed it and shipped apps with it. Building on-device audio AI for Apple devices is a lot more fun 😃 with 🥰 all ☺️ of 😍 you 🙌. On to the next 3k 🚀 github.com/FluidInference/Fl…
1
4
276
Fluid Inference retweeted
We got EmbeddingGemma 2 running 100% on the Apple Neural Engine 🍎 M5 Pro 10,080 posts sorted into topics live, ~620/s on a MacBook, then any topic splits into subtopics in under a second. Fully on-device. 🤗 huggingface.co/FluidInferenc… 💻 github.com/FluidInference/Fl…
Introducing EmbeddingGemma 2, our new open embeddings model for on-device use cases! 👀 Code, image, video, audio, and text embeddings 🤏 Modular, going from 270m to 740m parameters 🪆 Matryoshka embeddings for storage efficiency 🤗 Apache 2 License
18
51
3
693
34,004
Fluid Inference retweeted
EmbeddingGemma 2's audio encoder, on a M5 Pro with Core ML 🎧 GPU + ANE overlap → 4.5 h of audio indexed in 38 s (431×), then 710 searches/s, then ~700 text searches/s across it. Speech or sounds, the clips have random file names, the model only hears the audio. 🤗 huggingface.co/FluidInferenc… 🔨 github.com/FluidInference/Fl…
Meet EmbeddingGemma 2, our first natively multimodal open model for on-device embeddings. It expands beyond text to unify code, images, audio, and video in a shared space. 🧵
1
4
76
3,554
Fluid Inference retweeted
I was pretty impressed by how well EmbeddingGemma 2's vision runs on an M5 Pro as coreml, especially for such a small model (170M vision encoder) 👀 "Select all images with a…" 1,000 picture grids solved zero-shot: 99.5% of photos right, ~70 photos/s, fully on-device. 🤗 huggingface.co/FluidInferenc… 💻 github.com/FluidInference/Fl…
Ready to start building with EmbeddingGemma 2? Our Developer Guide walks you through embedding multimodal content across text, code, images, video, and audio using the sentence-transformers library and more! The workflow: 1️⃣ Load the model (via your preferred framework) 2️⃣ Embed text & code with task prompts 3️⃣ Embed images, video, audio & interleaved inputs 4️⃣ (Optional) Truncate dimensions with Matryoshka to save space
1
27
1,744
Fluid Inference retweeted
Local codebase indexing + semantic code search on M5 Pro converted to coreml. 5,617 Swift functions indexed in 16 s, with ~700 plain-English searches/s. 🤗 huggingface.co/FluidInferenc… 💻 github.com/FluidInference/Fl…
2
6
44
2,783
vroom vroom
Kokoro ANE v3 is in FluidAudio v0.17.6. Supporting English, Japanese & Mandarin. These live M5 Pro runs generated 3–3.7 seconds of speech in 58–80ms from text to complete audio. 1.9–5.3× faster than legacy pipelines. Built on hexgrad’s Kokoro: huggingface.co/hexgrad/Kokor… huggingface.co/hexgrad/Kokor… github.com/FluidInference/Fl…
1
5
134
Fluid Inference retweeted
Vela 2.0 0.3B, running on-device on an M5 Pro MacBook via Core ML 🍎 1,000 chatbot replies screened in ~11 s (~90 msg/s): jailbreaks + harmful blocked, PII redacted, the rest routed to a team. 99.4% of the model's ops run on the Neural Engine. 🤗 huggingface.co/FluidInferenc… 🛠 github.com/FluidInference/Fl…
Introducing 𝗩𝗲𝗹𝗮 𝟮.𝟬: Our newest 𝗢𝗽𝗲𝗻 𝗙𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗥𝗼𝘂𝘁𝗶𝗻𝗴 𝗠𝗼𝗱𝗲𝗹𝘀, bringing 𝘀𝗽𝗮𝗻-𝗹𝗲𝘃𝗲𝗹 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻𝘀 to System One. HF Collection 🤗: huggingface.co/collections/v… #OpenSource #AI #LLM #vLLM #HuggingFace
1
26
1
257
14,654
Fluid Inference retweeted
Decision 2.0 is now in FluidUse 🍎 Kai 0.6B + Eos 0.8B in native Swift, on-device on an M5 Pro. Demo: 1,000 GitHub issues labeled live, 5 decisions per issue in one call (type, owner team, priority, flags). ~15 issues/sec. 🛠 github.com/FluidInference/Fl… 🤗 Kai: huggingface.co/FluidInferenc… 🤗 Eos: huggingface.co/FluidInferenc…
Introducing Decision 2.0: our newest state-of-the-art decision models, open in every size from 0.6B to 27B ⚡ HF collection 🤗: huggingface.co/collections/v… #OpenSource #Decision #Jev #LLM
1
17
3
112
5,487
Fluid Inference retweeted
Phonon-2 runs on coreml with ANE in FluidAudio v0.17.5. 1 h of audio → 7.5 s on an M5 Pro (Parakeet v3: 10.7 s). Full LibriSpeech, same session, ANE: - test-clean 2.47 % WER at 159× (v3: 2.27 % at 149×) - test-other 4.62 % WER at 146× (v3: 4.12 % at 138×) 360 MB on disk (321 MB encoder) vs 480 MB for v3. Peak process RSS 0.36 GB for both; the weights sit in ANE. iOS 18+ / macOS 15+ . Code: github.com/FluidInference/Fl… Model: huggingface.co/FluidInferenc…
Introducing Phonon-2: a new standard in speech recognition per byte. At just a tiny 164MB download, it is more accurate on average than OpenAI's Whisper large ( a model 10X its size) Transcribe an hour of audio in just 20 seconds on a MacBook Air! Open weights, CC-BY-4.0, today. @fermion_ai
5
42
3,273
Fluid Inference retweeted
I was inspired by this tweet, using @intern_lm 's Intern-Decision-0.8B i fine-tuned it on Pokémon Showdown with 4B as the teacher. Plays at ~90 ms a move on Core ML in under 1 GB of memory at int8, against 140 ms and 3 GB for the same model in PyTorch. it plays as well as the 4B and beats its own un-fine-tuned self. it was interesting to see what you can finetune on a M5 pro. Model: huggingface.co/FluidInferenc… Base: huggingface.co/internlm/Inte… Runtime + harness: github.com/FluidInference/Fl…
We turned Qwen3.8-27B into a multimodal decision model. It beat Pokémon FireRed’s elite four and champion with sub-100 ms decisions from live game state. With SGLang’s native /v1/decisions, you can now turn LLMs and VLMs into classification and scoring models. We also added /v1/systemone so Jev-like open models can work with the TypeSafe SDK.
3
2
14
1,172
Fluid Inference retweeted
Kev on Core ML 🍎 After seeing this tweet, We got inspired to port Kev-0.8B (the Qwen3.5 variant) and ran Guess Who over 80 Wiki bios, on an M5 Pro, same accuracy, coreml vs original PyTorch: 🚅 37 ms vs 1.07 s per bio → all 80 in 3.0 s vs 99 s 🧠 ~2 GB vs 6.6 GB memory 📦 1.45 GB vs 1.79 GB model Model: huggingface.co/FluidInferenc… Code: github.com/FluidInference/Fl…
UPDATE: Kev-0.6B, 4B, and 8B are now available. Kev is a family of small open source Jev-like decision models you can train and run yourself. This new family is based on Qwen3 using the same LoRA + small pointer head technique as before, but scaled up. Out of domain, on data Kev never trained on: Kev-8B 79.6%, Jev 85.7%. • Drop-in TypeSafe System One API; their SDK works with one `base_url` change • Kev-4B serves on a 32 GB Mac in bf16: ~300 ms for five questions, ~40 ms on an H100 • Repeated documents hit a KV cache: 2-2.5x faster • Apache 2.0 License. Kev-4B trains in 40 minutes on one H100. Kev-8B in 83 minutes. Code, weights, evals: github.com/jaredpalmer/kev
2
7
82
5,630
Fluid Inference retweeted
over a year late but we converted SigLIP 2 to Core ML and sorted 7,349 pet photos into 37 breeds on a M5 Coreml model vs. base PyTorch model, same accuracy: • 36 s vs 102 s • 5 ms per photo vs 14 ms • 262 MB peak RAM vs 4.2 GB • 715 MB on disk vs 1.5 GB model: huggingface.co/FluidInferenc… code: github.com/FluidInference/Fl…
Introducing SigLIP2: now trained with additional captioning and self-supervised losses! Stronger everywhere: - multilingual - cls. / ret. - localization - ocr - captioning / vqa Try it out, backward compatible! Models: github.com/google-research/b… Paper: arxiv.org/abs/2502.14786
6
35
1
349
21,184
Fluid Inference retweeted
We have converted GLiNER2.5-Decide to coreml. ~4× faster, ~5× less Peak RAM, half the size. model: huggingface.co/FluidInferenc… code: github.com/FluidInference
Introducing GLiNER2.5-Decide, our new 340M parameter open weight, encoder-based decision model. GLiNER2.5-Decide is built for fast, deterministic classification. The model evaluates a set of user-defined typed questions and rules, and jointly decodes their answers, returning structured decisions with probability distributions and confidence scores. We evaluated the model’s performance on Fast Decisions, an unseen, internally generated classification suite based on 17 datasets testing real-world use cases across routing, triage, classification, sentiment, and content understanding. Measured against similar decision models, GLiNER2.5-Decide leads in 9 of the 17 datasets, achieving the highest average score: - GLiNER2.5-Decide: 60.1% - SemIf: 56.4% - JevK5: 57.5% - Laya: 46.6% This performance makes the model a strong fit for use cases like tool calling, model routing, browser and computer use, and LLM-as-a-judge. GLiNER2.5-Decide’s lightweight encoder architecture makes it easy to fine-tune the model for specific tasks, while being efficient enough to run locally on consumer-grade CPUs or in air-gapped environments, giving users greater control over where their data goes and where the model runs. To make building and experimenting with GLiNER2.5-Decide as easy as possible, we're also offering hosted inference. You can now use our API to run inference and fine-tune GLiNER models on specialized tasks right inside your own coding agent: agent.fastino.ai As with previous models, we’re also releasing the model weights on @huggingface under the Apache 2.0 license: huggingface.co/fastino/GLiNE…
17
83
7
878
69,275
RT @Alex_tra_memory: Parakeet 🦜 Ultra 🤗 is now in FluidAudio, running🏃‍♂️‍➡️ on the ANE 🔥 LibriSpeech clean: 2.13% WER · 127× RTFx LibriSp…
This quoted post is unavailable.
Fluid Inference retweeted
FluidAudio now runs Nemotron 3 Diarization on-device in Core ML. Its able to support 8 speakers with 8 different languages like in the below demo. Models: huggingface.co/FluidInferenc… Code: github.com/FluidInference/Fl… Special thanks to @NVIDIAAI for the early access.
When several people talk at once, a transcript can get messy fast. Our new Nemotron 3 Diarization model tracks who spoke when, even when voices overlap. It handles up to eight speakers, has 100M parameters, and is now available on @huggingface 🤗
2
14
201
13,447
Fluid Inference retweeted
we ported @knowledgator’s GLiClass Edge Apps v2 to Core ML on an M5 Pro. 32.7M params, ~1.4 ms per decision on CPU. it played 2048 for 2 minutes and reached the 4096 tile. code: github.com/FluidInference/Fl… models: huggingface.co/FluidInferenc…
The Jeff model, based on our GLiFormer, should be the most parameter-efficient model (a 500M-parameter model) for its performance score. And we created it right before the Jev release. Right now, imagine if we scale it 😉
4
11
1
102
7,905
Fluid Inference retweeted
since y'all loved the Tetris we decided to make a follow up. the model harness uses fewer bad moves offered, better wording, one deleted sleep. A continuous game lasting 35 seconds with a timer included.
Thanks for the model, we were able to port Laya to coreml with 99.5% of the ops on ANE + benchmarked too. it is now blazing fast with 3.7 ms per decision on an M5 Pro. Release: github.com/FluidInference/Fl… Models: huggingface.co/FluidInferenc…
1
2
1
14
2,654
🏎️🏎️
since y'all loved the Tetris we decided to make a follow up. the model harness uses fewer bad moves offered, better wording, one deleted sleep. A continuous game lasting 35 seconds with a timer included.
5
638
Fluid Inference retweeted
Thanks for the model, we were able to port Laya to coreml with 99.5% of the ops on ANE + benchmarked too. it is now blazing fast with 3.7 ms per decision on an M5 Pro. Release: github.com/FluidInference/Fl… Models: huggingface.co/FluidInferenc…
This is NOT Jev. Open source. Runs on your laptop. Decides in ~27 ms, about 200× faster than waiting on a hosted LLM. Here it is playing Tetris by itself 👇 brainfunctioncollapse.com/la…
21
111
11
1,321
120,018