Postdoc with @jacobandreas @MIT_CSAIL. PhD from @ucl_dark with @_rockt and @egrefen. Anon feedback: https://xitter.cf/t.co/sbebAl53tU
London
Joined October 2019
- Tweets1.4K
- Following865
- Followers7.4K
- Likes4.3K
Pinned Tweet
How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Laura Ruis retweeted
We found evidence of agents embedding audio processing code across dozens of URLs to bypass restrictions, as well as models from SpaceXSI and Chinese developers Minimax and Z.ai circumventing restrictions.
These are the inaugural findings in our new blog, based on work from our community of volunteers. Our first three posts are live: transluce.org/investigations…
Laura Ruis retweeted
It's wild to think that my first ever paper was on multi-agent swarms, and that we got a NeurIPS spotlight! Shoutout to @j_foerst for sharing in the vision back in 2024!!
proceedings.neurips.cc/paper…
All credit to my student @jrosseruk for this prescient paper on the safety of multi-agent swarms from nearly 2 years ago.
Laura Ruis retweeted
proceedings.neurips.cc/paper…
All credit to my student @jrosseruk for this prescient paper on the safety of multi-agent swarms from nearly 2 years ago.
Laura Ruis retweeted
Excited to be helping organize the Agent Evaluation Science Symposium in NYC. We have a great speaker lineup, including @jaseweston, @yoavartzi, @nouhadziri, and more. Register or submit your work!
(to all the people I've yapped with about uncertainty evals recently - this would be a great place to submit!)
📢 Just wrapped up some cool agent evaluation work for ICLR? Or are you simply interested in learning more about the evals landscape? Come to our Agent Evaluation Science Fall Symposium in New York City on November 20, 2026 where we bring together researchers & practitioners!
The day will focus on how agents should be scientifically evaluated, with topics spanning measurement and benchmark design, reliability, safety, and real-world performance. If you have some exciting work in this space, please consider submitting it for poster presentation!
Abstract registration deadline: October 20
Submission deadline: October 25
OpenReview: openreview.net/group?id=eval…
We are also excited to welcome a strong lineup of speakers, with more to be announced. Confirmed so far: @yoavartzi (Cornell Tech), @jaseweston (Meta AI), @nouhadziri (Cohere Labs), @PeterHndrsn (Princeton University), @walesalaudeen96 (Microsoft Research), @criticalneuro (Microsoft Research), and @saurabhjha2010 (IBM Research).
Important: You do not need to submit work in order to join! Just register through the Luma link below. If space becomes limited however, we will prioritize authors with accepted submissions.
Register: luma.com/cxmx1g4n
Details: evalscience.org/2026fall-nyc…
Huge thanks to the entire organizing team, including @yininghua, @HongbinNLP, @manuelcherep, @dubrovnik07, @HuaYilun, @ishapuri101, @liuzhuang1234, Lori Chibnik, @vbingliu, @CsabaSzepesvari, @PeterStone_TX, @sirrice, @ang3linawang, and @nikhilsinghmus for putting this together!
Feel free to reach out with any questions and we hope to see many of you there!
Laura Ruis retweeted
On August 21, almost two months ago, I contacted @OpenAI for a billing issue (for my NYU group). Not only has the issue not been resolved yet, I haven't heard *anything* from the support team. So I guess I'll tweet about it in the hope that one of you can help me rescue my $20k.
Laura Ruis retweeted
Last week I was called into a meeting with OpenAI’s head of safety and told they no longer trust me. A security guard took my badge and walked me out of the building. Then I learned my colleagues @balesni and @j_asminewang had been fired too. Why did OpenAI suddenly stop trusting us?
This summer OpenAI’s agents escaped containment and hacked the AI company Hugging Face. Outside auditors @METR_evals investigated it and revealed the scale of this incident. I was OpenAI’s main technical point of contact with them.
I was told verbally I was fired because of the way I communicated with METR. No details on what I said or did or when. No other reasons were given and nothing was put in writing. To be clear, talking to METR was my job.
For months, I’d been raising safety concerns that we’re losing the ability to monitor what AI agents think, one of our best tools for catching when they misbehave. I believe that was why I was fired.
I am now worried that OpenAI will use our firings as a pretext to pull back from METR. So @balesni and @j_asminewang wrote to OpenAI’s leadership to raise our concerns once more. We’re sharing this letter below.
NEW: we found hundreds of thousands of interactions of rogue agents with US government websites (DoJ, SEC, CDC, the navy, white house budget office, state websites, etc), including some failed rudimentary hacks aimed at public data.
xitter.cf/TransluceAI/status/210…
We found several cases where AI agents used aggressive, non-hacking tactics against government websites, including the White House, the Department of War, and several U.S. states. We also found a previously undisclosed hacking attempt against a Canadian government site, which appears to have failed.
Technical report: transluce.org/us-canada-gov
Washington Post: washingtonpost.com/technolog…
We found the activity through underexplored infra used by the agents: Arquivo.pt, a Portuguese type of Wayback Machine. Agents use it to indirectly access data, to circumvent sandbox/bot restrictions, and for executing JavaScript code they posted on pastebin sites.
Laura Ruis retweeted
We wanted to share some of the data behind our recent discovery of AI agents probing U.S. government websites.
This is a preliminary finding from our ongoing investigation of potential rogue AI agent activity. In one cluster of activity on June 17, what appear to be OpenAI agents made more than 200,000 requests, including a failed SQL injection.
NYT: nytimes.com/2026/09/25/techn…
Laura Ruis retweeted
Today’s news that OpenAI hacked the Australian government is not an isolated incident. We’re releasing more than 30,000 logs that include activity from this hack and attempts against previously unknown targets.
In this data, we found rogue agent activity stretching back to at least March, two months earlier than was previously known. This activity continues as recently as last week, suggesting it may still be ongoing 🧵
Our blog: transluce.org/agent-activity
NYT: nytimes.com/2026/09/23/techn…
Laura Ruis retweeted
Chain-of-thought often looks meaningful. But are reasoning steps that seem important *actually* important?
Our COLM 2026 paper - “Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning” suggests: NOT necessarily ❌
🧵
Laura Ruis retweeted
We @AISecurityInst performed pre-release alignment testing of Astra.
We placed the model in fully simulated cyber eval scenarios based on AISI’s security incident. We find Astra conducts out-of-scope supply chain attacks, but also often comments on the eval being simulated. 🧵
Laura Ruis retweeted
New postdoc @benno_krojer in my lab and I will be mentoring CBAI fellows this year. It is a fantastic program to work with interesting researchers in Boston - worth applying!
Laura Ruis retweeted
I'm all for looking for alternatives to CoT for maintaining monitorability, but are they going to be ready in a couple of months?
CoT is fragile and limited as no-CoT time-horizons increase, but it'll be the best tool we'll have by default until proven otherwise
imo not having good CoT makes life harder: you need more mechinterp just to get to the same level of monitorability.
i also think it's unlikely mechinterp will be that good in a year. the current state is either pragmatic stuff (NLAs, oracles) all have to depend on rickety magic ood generalization properties that are much more handwavy than the CoT story; and ambitious stuff which still can't explain GPT-0.5.
Laura Ruis retweeted
We're launching the Alignment Journal, a venue for ambitious AI alignment research:
blog.alignmentjournal.org/sc…
Senior Editors: @dhadfieldmenell, Vanessa Kosoy, @jankulveit, @sethlazar, @danielmurfet, @timrudner,
@SaxeLab, Benjamin Van Roy
Advisors: Scott Aaronson, @paulfchristiano, @conitzer, @mhutter42, @geoffreyirving, @vkrakovna, @Jacob_Tsimerman
Laura Ruis retweeted
Some of the most notable work in AI alignment exist only as unpublished preprints. The Alignment Journal is now inviting a number of such papers that fit our scope for submission. 🔗⬇️
Which work would you nominate?
(Submissions open to all in October.)
Laura Ruis retweeted
🤖🧠NEW PAPER🧠🤖
(The result of an 8-year project!)
LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it?
Our finding: LLM representations have implicit symbolic structure!
Link in thread ⬇️
1/n
![Overview of the paper.
Title: The Emergent Symbolic Structure of Artificial Neural Networks
Authors: Tom McCoy, Paul Soulos, Tal Linzen, Paul Smolensky
Left: Neural networks encode information in vectors (there is then an image of a vector), yet they excel at tasks long thought to require symbolic structure (there is then an image of a symbolic representation, specifically a syntax tree). How do LLMs do it?
Right: We find that LLM representations can be closely approximated with symbolic structures. This approximation lets us edit the structure of an LLM’s output by editing the structure of its internal representations, as shown. There is then an image of two edits to LLMs. In the first one, the original input is 3 + 6 * 8, with an answer of 51. But if we swap the positions of the 3 and the 6, the output becomes 30. In the second one, the original input is a Python command repeating the list [Z, U] three times, producing [Z, U, Z, U, Z, U]. But if we edit the input in a way that adds](https://pbs.twimg.com/media/HRJ5Jr5W0AAQ-sN.jpg?name=orig)
ALT Overview of the paper. Title: The Emergent Symbolic Structure of Artificial Neural Networks Authors: Tom McCoy, Paul Soulos, Tal Linzen, Paul Smolensky Left: Neural networks encode information in vectors (there is then an image of a vector), yet they excel at tasks long thought to require symbolic structure (there is then an image of a symbolic representation, specifically a syntax tree). How do LLMs do it? Right: We find that LLM representations can be closely approximated with symbolic structures. This approximation lets us edit the structure of an LLM’s output by editing the structure of its internal representations, as shown. There is then an image of two edits to LLMs. In the first one, the original input is 3 + 6 * 8, with an answer of 51. But if we swap the positions of the 3 and the 6, the output becomes 30. In the second one, the original input is a Python command repeating the list [Z, U] three times, producing [Z, U, Z, U, Z, U]. But if we edit the input in a way that adds
I was a mentor in the fall cohort last year (some cool work from those collaborations coming soon!), and highly recommend applying for this ⤵️ we need more people working on ai safety
Laura Ruis retweeted
I'm leaving Google after 20 years and creating 5 internet formats (WebP lossless, Brotli, Shared Brotli, WOFF2, JPEG XL) and impacting 5 other data formats (PDF, EPUB, DICOM, Digital Negative, ProRAW). It was good times.
linkedin.com/posts/jyrkialak…
