How would an LLM "thought" actually look, and what properties should it have? 🧠 LLMs shouldn't reason purely in text. That's why the AI field is moving toward continuous thought representations (vectors instead of words). But nobody had answered the more fundamental question: what should a thought look like? What properties must it have for an LLM to actually "think" properly? I went and answered that in our new paper. This might be very influential for the future of LLM reasoning. I propose 4 fundamental properties every thought representation must satisfy: 🎯 Causality: The thought must actually drive the answer, not just tag along for the ride. 🗜️ Minimality: It should compress the problem down to only what matters. 🔍 Separability: It must distinguish one question from another within the same topic. ⚖️ Stability: It must stay consistent with how uncertain the model is. These aren't just theoretical ideas. They give us concrete, measurable targets for building LLMs that actually think better. To see where we stand today, we audited 5 open-weight LLMs across 23 reasoning tasks, testing 4 different continuous thought representation methods from the literature. Here is what we found: ❌ None of them satisfy all 4 properties simultaneously. 🤯 Every single method fails at Separability. Meaning the LLM's internal "thought" cannot distinguish two different questions from the same domain. In plain terms: if you ask an LLM two different questions about your project, its thoughts for both look almost identical. It doesn't actually know which problem it's solving! 🏗️ The failure is structural. We see it across different sizes, dense models, RL-trained models, and reasoning-distilled models alike. Throwing more compute, bigger models, or more training at it won't fix it. ⚡ The biggest surprise: Simply using the input text embedding as the "thought" is competitive with, and often beats, the expensive iterative thinking methods on almost every metric. This strongly hints that current methods don't add information beyond what's there in the input to begin with. This gives researchers the first principled framework for what a proper LLM thought should look like, and tells us exactly what to fix and optimize for to get there. Dive into the research below 👇 📄 Paper: huggingface.co/papers/2606.2… 🌐 Website: fard-lab.github.io/formalize…