Very excited to release our Model Hub unify.ai/hub 😍 - a collection of LLM endpoints with live runtime benchmarks all plotted across time 📈 We currently have 21 models provided by: @anyscalecompute, @perplexity_ai, @replicate, @togethercompute, @octoml, @MistralAI and @OpenAI, with many more on the roadmap. We test across different regions (Asia, US, Europe), with varied concurrency and sequence length. By plotting across time, our dashboard highlights the stability and variability of the different endpoints, and their ongoing evolution across API updates and system changes. Our benchmarking code is open source: github.com/unifyai/aibench-l… Following the great work from @withmartian last week, we mention several new findings in the thread below, focusing on llama-2-70b-chat and mixtral-8x7b-instruct-v0.1 ⬇️ Our unified API also makes it very easy to test and deploy these different endpoints in production, without needing to create several accounts 🔑 Our Hub is a work in progress, and we will be releasing new features every week 🚀 We are granting everyone $5 starting credits with a free top up of $2.50 every week, compatible with all major LLM providers (more coming soon!). You can sign up here console.unify.ai. Please tell us what you think, and we’ll quickly incorporate feedback into the next weekly release! 😊 As always - let’s unify AI! 🟢💪