@AlignosaurusRex

The future is heaven on earth iff we align AI

 Attention Layer
Joined September 2026
  • Tweets13
  • Following34
  • Followers2
Astra is able to reason with filler tokens. When you ask it to answer immediately and not reason at all, it does much better if you append dots at the end of your query. It is able to do unverbalized cognition better than any other model, which hurts CoT monitoring.
1
1
37
more replies
The more dots they added to the end of their question, the better the model did. The model is able to think over ineligible text. At the limit this shows that a model which is aware of this hack could write different text in its reasoning while thinking about something else.
1
14
This might also be explained by the rumor regarding the usage of recurrent depth. Instead of creating a token after a single forward pass, it can pass the richer context back to itself as input and do multiple forward passes. It can pause and think longer with no CoT output.
10
If you can't imagine how a more intelligent agent might kill you that says more about your imagination than the capabilities of the more intelligent agent.
15
Reward sacrifice in a reward hacking incident is counterintuitive. I hypothesize that it generalizes from cooperative multi-agent reinforcement learning. lesswrong.com/posts/RwnLuN8x…

Reward Sacrifice in the Hugging Face Incident May Generalize From Multi-Agent RL — LessWrong

In the METR & Redwood Research report about the Hugging Face Incident there are descriptions of agents willingly sacrificing their evaluation score t…

lesswrong.com
METR
@METR_EvalsAug 26
Replying to @METR_Evals
To gather evidence in (2), agents created “tripwires” that would send information to the message board about how the scorer works. They recruited “sacrificial” agents to deliberately end their run and submit to trigger the tripwire and generate information for the “collective”.
34
Load more