@LocalAI_API

OpenAI Open Source alternative. LocalAI is a community, drop-in replacement API compatible with OpenAI for local CPU/GPU inferencing

localai.io
Joined April 2023
  • Tweets1.5K
  • Following40
  • Followers4.1K
Replying to @ivanfioravanti
oooor
1
1
47
LocalAI 4.11.0 is out - You can now transcribe recordings with speaker labels and detect sounds alongside speech - New Decisions API - Model failover - text-to-animation with Kimodo - Much more, check links below Thanks to everyone who contributed, tested, and reported issues!
1
4
13
996
Installing a model also stopped meaning "pick a quantization". One gallery entry can offer several builds of the same weights. LocalAI drops what your hardware can't run, drops what won't fit, installs the largest that's left. Q4 on the big box, IQ1 on the laptop, same command
1
2
2
262
LocalAI 4.8 is out. We just crossed 48.3k stars! - `local-ai chat` is a TUI agent harness now. - image to 3D, audio-driven talking avatars - voice cloning first-class in the UI - vLLM's serving architecture in C++20, no Python 386 PRs, 22 days, 25 contributors, 11 new👇
2
3
11
1,237
Scaling LLMs across nodes? When a follow-up lands on a replica that never saw your chat, the whole prompt is recomputed and the KV cache wasted. LocalAI fixes this at the router: cache-aware routing across a mixed fleet of vLLM + SGLang + llama.cpp + ...
4
2
1
8
1,266
Ettore Di Giacinto
@mudler_itMay 27
rf-detr.cpp: native C++/ggml (@ggml_org ) inference for @roboflow 's RF-DETR (my go-to for object detection and segmentation!) from the @LocalAI_API team. All 11 variants (5 detection + 6 segmentation), running at PyTorch speed (slightly, ~8% faster on CPU benchmarks), without Python dependencies at f16. Models available in @huggingface 🤗 Thanks to @ggerganov for ggml and @SkalskiP for rf-detr to make this possible! 🧵
2
800
LocalAI 4.1.0 is out!
Ettore Di Giacinto
@mudler_itApr 2
Ok, notoriously I don't sleep that much. Time to share @LocalAI_API 4.1.0 (why not?) ! TLDR: - Distributed, hybrid clusters with production ready setup - Built-in auth, quota, user metrics - Fine-tuning and quantization from the UI 🔥Details below! 👇
1
5
1,308
Hardware headaches? Gone. In v3.10, the backend ships GPU libraries, so you can just go with one image📦 Plus, v3.9 added a Smart Memory Reclaimer that automatically frees up VRAM under pressure by unloading unused models. It keeps your system stable even when you push it hard
1
140
It’s not just about text. We want LocalAI to cover everything. ✨ Video: A dedicated UI for generating videos (LTX-2 supported!). ✨ Audio: Added Moonshine for fast transcription and Pocket-TTS + Qwen-TTS for voice cloning and generation. Empowers you and your Hardware! 🎥🗣️
2
284
"Can you run this every morning?" Yes. With the new Agent Jobs Panel (v3.9), you can schedule prompts or agent tasks using simple Cron syntax or trigger via API. Need a daily news digest or automated report? Just set it and forget it. Automation is now built-in.
1
1
189
Load more