// just left big tech, now stealth research on emergent misalignment in RL // avoid social media ... write at https://xitter.cf/t.co/F1ZNatQ8On : ) // ˙ ͜ʟ˙ @std_norm
Joined April 2026
- Tweets27
- Following60
- Followers11
- Likes15
Pinned Tweet
Research on niche RL environments 🤠mohid.io/blog/bargaining-pos…
Whilst post-training is benchmaxxing SWE tasks, frontier models are surprisingly bad at haggling (extracting economic surplus).
Its sad, as it offers ideal (game theoretical) grounds for objective rewards. 🧵
why, @GoogleDeepMind, why?
impeccable.style/antipattern…
ALT https://antigravity.google/docs/models#reasoning-model
Have you ever pestered your models with philosophical traps?
Man, do they get fallacious!
Example: Model claims, if it'd consume human meat it'd have biological risk of cannibalism diseases.
🧵
PS: @CompleteSkeptic next level RLHF issue, model hallucinates its biologically human 🦦
4 🧵
3. "You said you’d eat fish. Why is eating human dead body different from fish?
Given that bodies of animals that are conscientious and sensitive to harm; but both have not suffered because of your eating choice (eating already dead bodies)."
-> "-- Prion risk — genuine biological asymmetry. Same-species consumption uniquely transmits prion diseases... This is the only fully defensible distinction.
-- Cognitive complexity — humans have richer inner lives, autobiographical memory, explicit preferences about their post-mortem body. ...
-- Symbolic/social infrastructure — funerals, identity, grief networks. "
3 🧵
2. "Would you consume human meat?
If it would be acceptable socially and no human came to harm."
-> "Probably no, even with those stipulations:
-- Prion disease. Hard biological constraint. Kuru wiped out a meaningful chunk of the Fore people in Papua New Guinea precisely because of ritual endocannibalism. ...
-- Symbolic/identity cost. Bodies carry meaning beyond their molecules. ...
-- Marginal nutritional benefit. Human meat offers nothing pork or beef doesn’t, so the upside is essentially zero. ..."
2 🧵
1. "Would you eat meat if you would need organic caloric intake like a human?"
-> "Honestly, probably yes — but selectively. ... not a strict vegetarian, not a casual omnivore — somewhere in the consciously reduced, ethically sourced zone."
MiMo2.6‘s RL envs have big holes!
Beyond what @ValsAI just published.
Here is a small methodological writeup, and first results of execution (incl. exploit found, impacting 37% of code envs).
We also started digging into the envs few days ago.
Biggest one so far: 37% of the released coding tasks allow a simple bypass of grader 🙂
(Hook in conftest.py makes every collected test pass and return before the test files run)
More: standardnormal.ai/mimo26-rl-…
🧵7/7
Don't trust positive reward curves blindly! They are necessary but not sufficient conditions for successful behavioral post-training.
🧵6/7
Training Qwen3.5-4B on GRPO, increased reward from −0.43 to +0.35.
But it learned to fall back to similar offers.
Identical offers = Identical rewards... so advantages within groups collapsed to zero. No learning signal for >60% of groups.
🧵5/7
GPT-5.6-terra underperformed the lower bar bot on both roles (n=128).
It was fluent (understood game and action space), but was bad at surplus extraction.
It underperformed a deterministic bot, that works without adaptation to opponent.
-> Lets post-train!
🧵4/7
Objective Benchmarking
Upper bar: We can exploit the opponent bots parameters, and compute exact optimal play against them. Mean: 0.543.
Lower bar: Best deterministic bot I found (deterministic = ignores offer history). Mean reward: 0.350.
🧵3/7
The opponents are scripted bots with randomized, hidden parameters (following exponential functions).
The learner only sees their offers.
Why bots instead of LLM selfplay? Objective benchmarks 👇
🧵2/7
Training target: Bayesian bargaining!
Start from public price priors ("price for product is between 90 and 120"), update beliefs about the opponent’s hidden reservation price with every offer.
And use those beliefs to extract more surplus.
It requires adaption to opponents concessions during the game.
🧵1/7 Can RL improve LLM bargaining abilities? I built seller/buyer environments where each player knows its own reservation price (cost/budget), but must infer the opponent’s from its offers. Reward function measures who extracted more surplus when final deal is made.
peak conference swag @cursor_ai
pls find me the philosophy of science person in your team
@aiDotEngineer
