Artificial Analysis has independently benchmarked @SambaNovaAI's custom AI chips at 1,084 tokens/s on Llama 3 Instruct (8B)! 🏁 This is the fastest output speed we have benchmarked to date and >8 times faster than the median output speed across API providers of @Meta's Llama 3 Instruct (8B) we benchmark. SambaNova currently does not yet publicly offer a serverless API but you can try out their system via their chat interface (see below tweet). SambaNova is not yet listed on the Artificial Analysis leaderboard but we understand API services using SambaNova chips will be available in the near future and we look forward to initiating full coverage. SambaNova’s custom SN40L RDU chips are their fourth generation design and are built on TSMC’s 5nm process. They are reported as having the potential to scale to serve much larger models than Llama 3 Instruct (8B) - Llama 3 400B+ 👀. Artificial Analysis has also verified that Llama 3 Instruct (8B) on Samba-1 Turbo achieves quality scores in-line with full FP16 precision by testing an MMLU-based benchmark.