💡 Hype vs Reality: can LLMs generate production-level code such as backends? Turns out, no.
⚠️ Using our new framework BaxBench, we show that even the best LLMs generate correct code only ~60% of the time. More alarmingly, >50% of their code is susceptible to security exploits!