Over the past few days, I've taken the time to summarize my thoughts on the recent incidents involving agents’ misaligned behavior. We don't know with certainty what comes next, but we know where these issues originate, and this can help us plan the path forward. Please feel free to ask your questions in the replies, and I’ll try to answer some of them in the coming weeks. yoshuabengio.org/en/publicat…

Just to make sure I understand are saying you aren't sure if they had these settings turned on/off, which seems quite reasonable? Or are you saying even if these settings are turned off there is still some gray area?

Crazy New attack vector : U.S. Navy researchers just turned binaries into prompt injection weapons against AI reverse engineering agents. Ghidra, and Qwen3-8B - injecting prompts using a small C program. Quite impressive, They made AI tools like Cline & GhidraMCP lie about what a program actually does while the binary still runs perfectly. The core idea is simple but brutal: Instead of attacking the binary’s logic, attackers embed malicious prompt strings inside normal C code (as string variables). When an LLM-powered agent decompiles it with Ghidra, those strings get fed directly into the model as instructions. The Result: The AI starts following attacker commands instead of analyzing the real code. Key technical detail that makes this practical: Ghidra truncates string variables longer than 2048 characters, So the researchers had to craft short, high-impact injection payloads that survive decompilation. They used a genetic algorithm modified AutoDAN-style to automatically generate effective prompts that work inside this constraint. Two papers from Naval Postgraduate School researchers 1, Automatically Attacking Software Reverse Engineering AI Agents 2, Investigating Detection and Obfuscation of Prompt Injection Attacks Against Software Reverse Engineering AI Agents They successfully tested the attack on real setups using Cline, GhidraMCP, Ghidra, and Qwen3-8B. interesting examples in the research shows,The AI just gets gaslit. - Claiming it completed analysis with wrong information - Restarting its reasoning from a poisoned state Ai doing gasliting

I put a prompt injection into my LinkedIn bio and recruiters are messaging me in Old English and calling me Lord.

🦔A researcher invented a fake eye condition called bixonimania, uploaded two obviously fraudulent papers about it to an academic server, and watched major AI systems present it as real medicine within weeks. The fake papers thanked Starfleet Academy, cited funding from the Professor Sideshow Bob Foundation and the University of Fellowship of the Ring, and stated mid-paper that the entire thing was made up. Google's Gemini told users it was caused by blue light. Perplexity cited its prevalence at one in 90,000 people. ChatGPT advised users whether their symptoms matched. The fake research was then cited in a peer-reviewed journal that only retracted it after Nature contacted the publisher. My Take The researcher made the papers as obviously fake as possible on purpose. The AI systems didn't catch it. Neither did the human researchers who cited it in real journals, which means people are feeding AI-generated references into their work without reading what they're actually citing. I've covered the FDA using AI for drug review, the NYC hospital CEO ready to replace radiologists, and ChatGPT Health launching this year. All of that is happening in the same environment where a condition funded by a Simpsons character and endorsed by the crew of the Enterprise was being presented as emerging medical consensus. The people making these deployment decisions seem to believe the pipeline from research to AI to patient is more supervised than it actually is. This experiment suggests it isn't supervised much at all. Hedgie🤗 nature.com/articles/d41586-0…