Pumped to announce the brand new open LLM leaderboard. We burned 300 H100 to re-run new evaluations like MMLU-pro for all major open LLMs! Some learning: - Qwen 72B is the king and Chinese open models are dominating overall - Previous evaluations have become too easy for recent models, much like grading high school students on middle school problems - There are indications that AI builders have started to focus on the main evaluations too much at the expense of model performances on other ones - Bigger is not always smarter Really excited about how the field is maturing on evaluation, let's go!