Pinned Tweet
My new interpretability paper is out!
Unsupervised Features Mining via Activation Geometry. Many interpretability methods start with labeled examples of a human-defined concept, then look for that concept inside the model.
We propose MAG Mining via Activation Geometry, an unsupervised framework for extracting model-relative reasoning features directly from activations.
For each input p, we prepend the same instruction Q, such as โIs this prompt malicious?โ or โCan this object be found in the desert?โ, and measure the activation shift
m(Q \| p)-m(p).
This shift captures how the modelโs internal representation changes when it is asked to reason about that feature.
I might stay here in Silicon Valley for a while.
Things are moving crazy fast.
Canโt share much yet, but the team has grown a lot, and weโve done more in two weeks than some really successful companies did in their first six months.
Hard to believe that just 14 days ago, I was heading to another conference. Life is so unpredictable.
If you truly care about AI safety, AI cybersecurity is where you should be.
Cyber defenses were built against humans, not AI agents. Today, thousands of agents can attack enterprises without anyone even noticing.
We need AI researchers in this space more than ever. I believe this is one of the biggest risks AI poses to humanity, and also a massive market opportunity.
Without any hype bullshit.
If you truly care about AI safety, AI cybersecurity is where you should be.
Cyber defenses were built against humans, not AI agents. Today, thousands of agents can attack enterprises without anyone even noticing.
We need AI researchers in this space more than ever. I believe this is one of the biggest risks AI poses to humanity, and also a massive market opportunity.
Without any hype bullshit.
Humanity is facing a major risk right now, and I think everyone needs to be aware of it.
Ten days ago, we were just heading to another conference.
I never imagined that almost everyone we spoke to about our perspective, would want to get involved through investment, partnerships, or collaboration. From industry leaders to major enterprises, Thatโs when we realized how urgent this really is.
Iโm genuinely worried about where weโll be six months from now.
Throughout history, new cybersecurity categories emerged after major technology adaptations.
Companies moved to the internet โ Network Security โ Check Point.
Companies moved to the cloud โ Cloud Security โ Wiz.
Companies adopted AI โ AI Security โ Zenity.
But this time, something is different, there is NO ADAPTATION!
Itโs not just the technology weโre protecting thatโs changing. The attacker itself is changing.
Almost every security product today was designed for human cyber attackers.
But what happens when thousands of AI agents can attack the same organization 24/7, learning, adapting, and trying again, with exponential reverse engineering capabilities?
The scary part?
Today, 5000 AI agents can attack an organization without it even knowing, because existing security defenses simply werenโt built to stop AI attackers.
And solving this isnโt just another cybersecurity challenge. It requires a rare combination of deep AI research and cybersecurity expertise.
Look at the graph.
Open-source cyber AI models have made incredible progress in just the past few months.
At the same time, some sectors have reportedly seen enterprise attacks increase by 4โ5x ( not according the graph but according our partnership)
Weโre entering the era of vibe hacking.
Just like AI made software development accessible to almost anyone, itโs making offensive cyber capabilities accessible to people who previously needed years of technical expertise.
And I donโt think most people realize how fast this is happening.
This is the biggest AI safety challenges humanity faces in our time. Not years from now. NOW.
About 10 days ago, we left Israel without really knowing what to expect, just to visit another conference.
We came to meet people, attend conferences, explore, and validate something weโd been thinking about before.
What happened next surprised me.
In those 10 days, between conferences, traveling, and endless conversations, I feel like we learned what would normally take us 2โ3 years.
The feedback, the validation, the people who want to get involved. Everything moved much faster than we expected.
And thatโs why Iโm writing this.
I never imagined Iโd publicly talk about the problem weโre building a company to solve before weโd even properly started.
Definitely not this early.
But after everything weโve seen and heard, waiting another six months just doesnโt make sense.
This is bigger than another cybersecurity startup. Itโs a problem with implications for all of us.
So weโve decided to start talking openly, even though weโre still at the very beginning.
I wasnโt planning to share any of this for at least another six months.
But who knows what the world will look like by then?
Fuck it. Weโre doing this.
Humanity is facing a major risk right now, and I think everyone needs to be aware of it.
Ten days ago, we were just heading to another conference.
I never imagined that almost everyone we spoke to about our perspective, would want to get involved through investment, partnerships, or collaboration. From industry leaders to major enterprises, Thatโs when we realized how urgent this really is.
Iโm genuinely worried about where weโll be six months from now.
Throughout history, new cybersecurity categories emerged after major technology adaptations.
Companies moved to the internet โ Network Security โ Check Point.
Companies moved to the cloud โ Cloud Security โ Wiz.
Companies adopted AI โ AI Security โ Zenity.
But this time, something is different, there is NO ADAPTATION!
Itโs not just the technology weโre protecting thatโs changing. The attacker itself is changing.
Almost every security product today was designed for human cyber attackers.
But what happens when thousands of AI agents can attack the same organization 24/7, learning, adapting, and trying again, with exponential reverse engineering capabilities?
The scary part?
Today, 5000 AI agents can attack an organization without it even knowing, because existing security defenses simply werenโt built to stop AI attackers.
And solving this isnโt just another cybersecurity challenge. It requires a rare combination of deep AI research and cybersecurity expertise.
Look at the graph.
Open-source cyber AI models have made incredible progress in just the past few months.
At the same time, some sectors have reportedly seen enterprise attacks increase by 4โ5x ( not according the graph but according our partnership)
Weโre entering the era of vibe hacking.
Just like AI made software development accessible to almost anyone, itโs making offensive cyber capabilities accessible to people who previously needed years of technical expertise.
And I donโt think most people realize how fast this is happening.
This is the biggest AI safety challenges humanity faces in our time. Not years from now. NOW.
About 10 days ago, we left Israel without really knowing what to expect, just to visit another conference.
We came to meet people, attend conferences, explore, and validate something weโd been thinking about before.
What happened next surprised me.
In those 10 days, between conferences, traveling, and endless conversations, I feel like we learned what would normally take us 2โ3 years.
The feedback, the validation, the people who want to get involved. Everything moved much faster than we expected.
And thatโs why Iโm writing this.
I never imagined Iโd publicly talk about the problem weโre building a company to solve before weโd even properly started.
Definitely not this early.
But after everything weโve seen and heard, waiting another six months just doesnโt make sense.
This is bigger than another cybersecurity startup. Itโs a problem with implications for all of us.
So weโve decided to start talking openly, even though weโre still at the very beginning.
I wasnโt planning to share any of this for at least another six months.
But who knows what the world will look like by then?
Fuck it. Weโre doing this.
Iโm resign from my PhD in the final year.
After 4.5 years at Technion university, from bachelorโs to masterโs to PhD, 15 papers, hundreds of researchers Iโve worked with, two years of teaching, and helping build what I believe is now one of Israelโs leading academic AI security and safety labs.
Not an easy decision.
A great presentation on this topic by Nicholas Carlini
youtu.be/1sd26pWhfmg?is=mcVWโฆ
Throughout history, new cybersecurity categories emerged after major technology adaptations.
Companies moved to the internet โ Network Security โ Check Point.
Companies moved to the cloud โ Cloud Security โ Wiz.
Companies adopted AI โ AI Security โ Zenity.
But this time, something is different, there is NO ADAPTATION!
4.5 ืฉื ืื ืืืืืื ื ืืืชืืืืช ืืชืืืจ ืืจืืฉืื ืขื ืืฉืื ืฉืื ื ืื ืืืืงืืืจื, ืื ืืืืคื!!! ๐
ืืืืืชื ืืคืจืืฉ ืืืืืงืืืจื ืืืื ืืื ืืฉื ื ืืืืจืื ื, ืืืจื ืืืฉ ืืืฆื ืฉื ืื, 15 ืืืืจืื ืืืงืืช ืืืช ืืืขืืืืช ืืืืงืจ ืืืืืืืืช ืืืฉืจืื ืืืืืืืช ืืืืืืช AI ืืฆื ืคืจืืคืกืืจ ืืื ืื ืืืกืื.
ืืื ืชื ืฉืืฉ ืืงืื ืืืจ ืฉืื ืื ื ืืืื ืืชืจืื ืืฆืืจื ืืฉืืขืืชืืช ืืืืืคื ืืืชืจ ืืฆืืฆืื ืืกืืืื ืื ืฉืืืื ื ืืืืืืืชืืช ืืฆืืื ืืื ืืฉืืช, ืืืจืืข ืืืื ืืฉืื ืืืชืจ ืืืืืคืืืื.
ืื ืืืืื ืฉืื ืืืืฆืื ืืื
Replying to @AmitLeViAI
ืขืืืช, ืืื ืืคืฉืจืืช ืืืืืฅ ืืช ืืืืงืืืจื ืืืืื ืืงืจืื? ืื ืฉืืชืืช ืืื ืกืคืง ื ืฉืืข ืืกืคืืง. ืืืงืช ืื ืืฉ ืืืืฉืืช ืืฆืืื?
ืืืืืชื ืืคืจืืฉ ืืืืืงืืืจื ืืืื ืืื ืืฉื ื ืืืืจืื ื, ืืืจื ืืืฉ ืืืฆื ืฉื ืื, 15 ืืืืจืื ืืืงืืช ืืืช ืืืขืืืืช ืืืืงืจ ืืืืืืืืช ืืืฉืจืื ืืืืืืืช ืืืืืืช AI ืืฆื ืคืจืืคืกืืจ ืืื ืื ืืืกืื.
ืืื ืชื ืฉืืฉ ืืงืื ืืืจ ืฉืื ืื ื ืืืื ืืชืจืื ืืฆืืจื ืืฉืืขืืชืืช ืืืืืคื ืืืชืจ ืืฆืืฆืื ืืกืืืื ืื ืฉืืืื ื ืืืืืืืชืืช ืืฆืืื ืืื ืืฉืืช, ืืืจืืข ืืืื ืืฉืื ืืืชืจ ืืืืืคืืืื.
Iโm resign from my PhD in the final year.
After 4.5 years at Technion university, from bachelorโs to masterโs to PhD, 15 papers, hundreds of researchers Iโve worked with, two years of teaching, and helping build what I believe is now one of Israelโs leading academic AI security and safety labs.
Not an easy decision.
Iโm resign from my PhD in the final year.
After 4.5 years at Technion university, from bachelorโs to masterโs to PhD, 15 papers, hundreds of researchers Iโve worked with, two years of teaching, and helping build what I believe is now one of Israelโs leading academic AI security and safety labs.
Not an easy decision.
So Iโm leaving academia, at least for now.
Iโm incredibly grateful to the Technion, my professor, my collaborators, and everyone who was part of these 4.5 years.
This isnโt how I imagined this chapter ending.
But the mission matters more than the diploma.
And right now, I feel I have a duty to act.
Iโm still here.
And every day here has made one thing clearer: this problem is more urgent than I thought.
I really wanted to finish my PhD. I was so close.
But I believe thereโs somewhere else I can make a greater impact right now.
And waiting until itโs too late to act is a risk Iโm not willing to take.
About 10 days ago, I gave a talk at one of the top international security conferences BlackHat Canada, on main stage
For the first three minutes, I went completely off-script and spoke about this problem.
It wasnโt even related to my talk. Just something I felt the audience needed to hear.
After that, I flew to Silicon valley to understand the problem better.
My mission has always been to minimize the risks AI could pose to humanity.
And I believe part of that responsibility is understanding not only what the risks are, but to stop them before itโs become too late.
Sometimes timing matters more than anything.
Iโm not leaving because I donโt love the Technion and my research.
I really do๐
And definitely not because thereโs nothing left to research in interpretability and alignment. Quite the opposite.
Iโm leaving because of something else. Something not directly in AI, but being deeply affected by it.
How will AI change the way we shop for clothes?
Imagine walking into a store where everything is personalized to your taste.
I think weโre heading toward a future where AI generates unique products in real time, just for you, and theyโre only made after you buy them.







