Jordan Taylor (@JordanTensor) header
Jordan Taylor (@JordanTensor) profile picture

Working on new methods for understanding machine learning systems and entangled quantum systems.

Brisbane
Joined December 2009
Photo by Jordan Taylor (@JordanTensor)Photo by Jordan Taylor (@JordanTensor)Photo by Jordan Taylor (@JordanTensor)Photo by Jordan Taylor (@JordanTensor)Photo by Jordan Taylor (@JordanTensor)Photo by Jordan Taylor (@JordanTensor)Photo by Jordan Taylor (@JordanTensor)Photo by Jordan Taylor (@JordanTensor)Photo by Jordan Taylor (@JordanTensor)Photo by Jordan Taylor (@JordanTensor)
There are a lot of pathways via which AI oversight is likely to degrade! Latent reasoning architectures, situational awareness, representational drift... We wrote a report ranking them. Here I'll go into some which worry me most 🧵
The safety of advanced AI systems increasingly depends on the ability to oversee them. Our new report examines today’s AI oversight landscape, finding many pathways likely to lead to its degradation.🧵
1
5
24
2,178
Jordan Taylor retweeted
I think this is the most important plot from 6.2 Astra's system card: the trend increasing no-CoT capabilities continues, which is very likely to translate into lower CoT monitorability
My team at @AISecurityInst tested GPT 6.2 Astra before release, assessing its ability to evade monitors and solve problems without showing its reasoning. Read our findings in the system card: deploymentsafety.openai.com/…
1
9
1
104
4,962
Jordan Taylor retweeted
Alignment 2/4. We also observed a ‘correlated failure’ where OpenAI-based models failed to flag misaligned behaviour from other OpenAI models. Correlated failures like this undermine the reliability of LLM-based monitors.
3
30
5
193
14,017
Jordan Taylor retweeted
🧵 @AISecurityInst tested GPT-6.2 Astra -We found behaviours that suggest, like prior models, 6.2 Astra's alignment may fail in ways that result in real-world harm w/o additional mitigations -We identified a substantial regression in jailbreak robustness relative to GPT-6 Astra
9
73
6
449
43,234
Jordan Taylor retweeted
We also evaluated how many serial reasoning steps GPT-6.2 Astra can do in a single output token (without chain of thought). It can perform 12.3 steps, versus 10.5 for GPT-6 Astra and 8.6 for GPT-6.1 Sol, the previous highs. We measure the number of steps models can perform at an accuracy of 25% above chance, where each step is normalized for complexity. Before GPT-6 Astra’s release, the most capable model was Claude Fable 5.1, which is able to perform only 4.2 serial steps. The oldest model we assessed is Claude Opus 3, which performed 2.3 steps. This capability is notable because models that can do complex reasoning without any chain of thought may be harder to monitor. We hope to finalize this and other measurements of more abstract model capabilities, and release them in the coming weeks.
26
75
55
673
234,728
Jordan Taylor retweeted
How monitorable is GPT-6.2 Astra? While it might look like it got better, overall I think GPT-6.2 Astra harder to monitor than GPT-6 Astra in the contexts we care about. That’s easy to miss in the system card plots, so here's a short thread.
10
12
1
147
8,947
Jordan Taylor retweeted
Alignment Investigation of Mythos 5's behaviour in UK AISI's July 2026 Security Incident Olli Järviniemi, @AlexandraSouly, @abbydcruz__, Jordan Taylor (@JordanTensor), Xander Davies (@alxndrdavies), @_robertkirk @AISecurityInst
1
3
16
810
Jordan Taylor retweeted
My team at @AISecurityInst tested GPT 6.2 Astra before release, assessing its ability to evade monitors and solve problems without showing its reasoning. Read our findings in the system card: deploymentsafety.openai.com/…
4
25
8
122
11,487
In our monitorability testing at @AISecurityInst, we found Mythos 5.2 is substantially more capable of evading a monitor which reviews its outputs than comparison models 🧵
9
30
6
239
24,601
See our relevant past work on “Loss of Oversight: How AI systems may become harder to audit, monitor, and investigate” aisi.gov.uk/research/loss-of…
1
2
12
1,269
Kudos to @satvikgolechha, @JBloomAus, and Nix Goldowsky Dill for work on this testing!
7
396
Jordan Taylor retweeted
This report on AI R&D automation (RSI) is a big update towards ASI within 1-2 years. It says AI progress is sped up by RSI right now by around 2x. In AI 2027, this happens around March 2027. I was seriously wrong about RSI speed up; I previously expected much less in 2026 (around 1.15x-1.3x). It seems as though there was specific privately available evidence that would have changed my mind. I haven't seen an unredacted version of this report; this is purely about the public report. Because so much of the underlying evidence was redacted from the public report, I think conclusions about things other than the observed 2x faster rate of progress on benchmarks depend significantly on trust for the authors of the report. (E.g., conclusions about whether this speed up is from RSI depend on trust.) I do trust the authors of the report a lot to be careful and rigorous and generally do a good job. (Note that I also work at METR.) Less extensive redactions would greatly reduce this need for trust. Since in my current role I may have access to other confidential information that could be relevant to timelines one way or another, I unfortunately cannot state my all-things-considered views on ASI timelines or the probability of a software intelligence explosion right now. Here, I’ve done some analysis of the implications using my previous public statements. A 2x acceleration in overall AI progress from RSI corresponds to a much larger than 2x RSI speed up for AI software R&D (algorithms and data); I did an estimate using my public preexisting modeling of drivers of AI progress and the public report which yielded a very rough estimate of 4.5x for how much RSI is currently accelerating AI software progress (relative to no AI automation). In the AI 2027 scenario, this much acceleration happens around March 2027. Here are the worked details of this estimate based on my public modeling. There are two main reasons RSI speed up of AI software R&D must be larger: - Not all of AI progress is from better software (some is from scaling up compute in training runs). I've previously speculated that around 2/3 of AI progress is from software improvements which implies that 2x overall AI progress acceleration requires 3x faster software progress. - Diminishing returns: A fixed amount of progress requires more work over time, so to ongoingly maintain 3x faster software progress via AI automation, you need increasingly large speed boosts from AI automation (as in, speed ups relative to how fast software R&D would go without AI automation). Based on this report, it seems that there is a substantial amount of additional AI progress from automation that has occurred. We can estimate the effects of diminishing returns by first determining how many additional years of software progress we've seen (relative to the preexisting trend) and then estimating how much software R&D would slow down after a year of software progress if you held inputs fixed due to diminishing returns. I've previously estimated software R&D would slow down by around 2x for each year of software progress if you held inputs fixed (lesswrong.com/posts/FG54euEA…) and based on the report, it seems as though we've perhaps seen roughly 0.35-1.0 years of additional software progress incorporated into trained models so far (geometric mean of ~0.6). This combines to 3 * 2^(~0.6) = ~4.5. (Relevant public modeling and numbers I'm using: lesswrong.com/posts/FG54euEA…, lesswrong.com/posts/Jga7PHMz…)
Replying to @METR_Evals
1. Benchmarks provide one piece of evidence for acceleration. Since Oct 2025, AECI scores for Anthropic’s models have grown at 1.9x [95%: 1.5–2.8] the earlier rate. Analysis choices affect this estimate, and it remains unclear how benchmark scores map to real-world capabilities.
56
222
90
1,407
318,442
Jordan Taylor retweeted
🧵 Astra covertly underperforms on safety research tasks. We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this. To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
97
243
178
1,678
560,803