We’ve updated our evaluation of the best open source LLMs and—spoiler alert—Llama 3 70B Instruct tops the list. Our complete recommendations: baseten.co/blog/the-best-ope…

We’re excited to launch Meta’s Llama 3 in our model library, in both 8B and 70B 🎉 The newly introduced Llama 3 is a significant improvement over Llama 2, with increased tokens, and reduced false refusal rates. These models deliver unparalleled performance, showcasing significant advancements in efficiency and speed. Our Llama 3 8B runs on A100s and Llama 3 70B runs on H100s, optimized for production.

Deploy Mixtral 8x22B in one click! Mixtral fast facts: - #1 pretrained model on the Open LLM leaderboard - Mixture of Experts architecture - Apache 2.0 license - Uses 4 A100s in fp16, optimized implementations coming soon! baseten.co/library/mixtral-8…

Another first 🎉 Unlock the power of @nvidia's Multi-Instance GPU (MIG) virtualization technology with H100mig GPUs on Baseten: baseten.co/changelog/fractio…

40% lower latency and 70% higher throughput for Stable Diffusion XL? Using NVIDIA TensorRT to optimize each component of the SDXL image generation pipeline, we achieved these performance gains on an H100 GPU. Full results: baseten.co/blog/40-faster-st…

Nomic Embed v1.5 outperforms OpenAI's text-embedding-3-small and ada-002, plus offers variable dimensionality. This best in class open source text embedding model from @nomic_ai is now available for 1-click deployment in Baseten's model library: baseten.co/library/nomic-emb…

Unlocking the full potential of GPUs requires optimizing every part of the model serving stack. Using TensorRT from @nvidia, we optimize ML models to take full advantage of the H100 Hopper architecture, providing 20-40% better performance per dollar. baseten.co/blog/unlocking-th…

Launching today 🎉 Double your throughput or halve your latency for @MistralAI, @StabilityAI + others? Do both at ~20% lower cost with @nvidia H100s on Baseten. Here’s how 👇

Mixtral 8x7B beats Llama 2 70B on quality for many benchmarks, but it also wins on inference speed. Here’s why: - Mixtral is only 46.7B parameters - Only 12.9B are used during inference - You can make it even faster with TensorRT-LLM and int8 quantization

Stable Video Diffusion is so much fun. You can now deploy it yourself on Baseten. @StabilityAI knocked it out of the park with this one 🎉 Lots of examples in the thread below 🧵 baseten.co/blog/stable-video…

Ready to try open source LLMs? Switch from GPT to Mistral 7B in the smallest refactor you'll ever ship: just 3 tiny code changes. If you're making the jump, DM us for $1,000 in free credits. baseten.co/blog/gpt-vs-mistr…

There's a new text embedding model by @JinaAI_ with some exciting properties 👀 - 8,192-token context window (embed chapters, not pages) - Matches OpenAI's ada-002 on popular benchmarks Use jina-embeddings-v2 for search & recommendations and pair w/ LLMs like Mistral for RAG.

Presenting your winner, by unanimous decision… ANDDD… NEW seven billion parameter champion of the world... Mistral 7B blew past Llama 2 on the benchmarks. And it can code, too. Get Mistral 7B with streaming output behind an API endpoint in minutes: app.baseten.co/explore/mistr…

We’ve got drinks, dinner, demos, and discussion - all we need is you! Join builders and founders at our next AI meetup this Thursday, August 10th, in San Francisco.