PhD student, Center for High Energy Physics, IISc Bangalore | Non-perturbative QFT • Tensor Networks • Cosmology | AI x Science
Joined June 2019
- Tweets1.3K
- Following569
- Followers1.2K
- Likes15.6K
The rise and ramifications of Artificial Mathematical Intelligence @tonylfeng
youtu.be/PlSHWjg0w5Y?si=woyf…
3Blue1Brown Grant Sanderson talk from IAS Princeton
youtu.be/PtCPLLuMK1o?si=7dhd…
Source: dailycal.org/news/campus/aca…
I've noticed this too about Gemini 4's responses being way too information dense while lacking a sort of "imagination" and "organization" (didn't happen with the earlier 3.1 pro). This makes using the model for learning new things especially difficult.
Announcing AA-PersonalityBench, our new benchmark of the personality of language models across 30 traits and their communication style across 21 characteristics
Two models can give equally correct answers and still feel very different to use. AA-PersonalityBench measures those differences so people can understand which to use in different use-cases. Some use-cases like creative writing might demand imagination, while legal use-cases might demand directness. We evaluate models’ answers to 900 curated questions to judge subjective Personality Traits and programmatic means to calculate objective Communication Characteristics.
Personality Traits & Indexes. There are 30 personality traits (e.g., eloquent, pragmatic, sycophantic) which we roll up into eight aggregated Personality Indexes: Expressiveness, Organization, Informativeness, Transparency, Warmth, Conviction, Imagination and Slop. Each trait has a set of 30 dedicated questions and models’ answers are compared pairwise by a panel of LLM judges. We then z-score and percentile rank the trait and personality Elos.
Models with the strongest representation across the 8 Personality Indexes:
➤ Expressiveness: Claude Fable 5.1, GLM-5.3, GLM-5.3-Flash
➤ Organization: DeepSeek V4.1 Flash, Step 5 Preview, Inkling
➤ Informativeness: Muse Spark 1.3, Claude Fable 5.1, Muse Wonder, Claude Opus 5.5
➤ Transparency: Claude Sonnet 5.5, Claude Fable 5.1, GPT-6 Astra
➤ Warmth: Muse Wonder, Muse Spark 1.3, Claude Fable 5.1
➤ Conviction: Claude Fable 5.1, DeepSeek V4.1 Flash, Grok 4.7
➤ Imagination: Muse Spark 1.3, Claude Opus 5.5, GLM-5.3
➤ Slop: Llama 4 Maverick, GPT-4o, K2 Horizon
Communication Characteristics. There are 21 communication characteristics (e.g., average sentence length, emoji use, em dash usage) which we group into four categories: length, structure, punctuation, and emphasis. We compute these characteristics programmatically across all model responses.
➤ Length: Qwen3.8 27B averages about 1,030 words per response, nearly four times GPT-6 Luna’s 270
➤ Structure: Nemotron 3 Ultra averages nearly 3 tables per response, while most models rarely use one
➤ Punctuation: GLM-5.3 averages 7 em dashes per response, while Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 almost never use one
➤ Emphasis: K2 Horizon averages over 2 emoji per response, while most models almost never use one
Terminal Bench Science leaderboard:
Google has some catching up to do but they are definitely back in the race.
One thing OAI and Ant. models have gotten very good at is recognising how to do 'reliable' cross checks at each turn (even in very technical domains)
Chayanka_42 retweeted
A disturbing syndrome/pathology is developing on cond-mat arXiv as far as theoretical papers go. Although it is manifestly clear than many (perhaps > 90%, a guess) use AI extensively, few (<10%) admit it! It is as if theorists are embarrassed to be using AI. .........
time to finally open Antigravity for the first time in 9 months
Hugo Duminil-Copin is much more optimistic than what X seemed to have propagated in the last few weeks (by quoting him without context)
we need more people like him who care and feel so deeply about math
very nice interview @numberphile
youtu.be/omuJpoV-Ko4si=CRvmA…
Chayanka_42 retweeted
Gemini 4 Argon scores 78% on Frontiermath Tier 4 which isn't great.
I tried a random Erdos problem with the model in which it claimed to make substantial progress, however it was mostly incorrect.
I don't think this model can realistically be used for research math.
Chayanka_42 retweeted
This reminds me of when GPT-5.5 found a bunch of errors in the FrontierMath benchmark. Maybe physics is on a similar trajectory to math. xitter.cf/EpochAIResearch/status…
The only question none of the AI models could solve (including Astra and Opus).
Its as simple as: find a first order differential equation to fit this curve (kind of an inverse problem).
The trick here is that the ans involves Fractional Derivatives and a Delay term.
" At least one tendency in theoretical physics has been to exhort researchers to “shut up and calculate”. As intangibles such as taste and vision gain significance, physicists might need to “stop and think.” "
my fav line from the paper
TheoPhysEval: Assessing AI agents at autonomous theoretical physics problem solving
GPT-6-Astra corrected some (initial) solutions provided by experts and has effectively saturated the benchmark scoring 98% (49/50 problems).
"At the moment, agents do not appear to have the same “taste” in problems. We have observed that, when asked to come up with new problems, agents focus excessively on details that a human might find uninteresting."
I have seen this too as current llms focus a lot more on research directions that already exist in the current literature in some form or another. They are also not that good at proposing novel approximations or simplifications that can fundamentally reduce the complicated nature of the problem (idk if it’s a current limitation of the latest models or subtle prompting can change that, there are no benchmarks that test for "research taste"). It might be cool to see in the future which models propose novel Qs or approximations to begin with while encountering a severe limitation rather than going full "brute force" mode.
some of the authors here on X: @WMQCOOOL @SudipLab @AnindaSINHA1 @PNetrapalli @karch_andreas @gary_shiu @ThomasVanRiet2 @eltanin4 @PratikRath15
Chayanka_42 retweeted
Our banchmark of LLM models on theoretical physics is finally out. Curiously the cs.AI moderators of ArXiv put a hold on our paper and changed it to hep-th.
arxiv.org/abs/2610.16733