Identity & Security Builder: Amazon, Meta, FishNet Security, WhiteHat. I build kickass, high-precision security automation & competent risk-mitigation.
A man can't just sit around.
Joined October 2010
- Tweets9.6K
- Following4.1K
- Followers1.4K
- Likes19.8K
Pinned Tweet
Public return (x only) 1/26. Have NOT read DMs, emails, or other social media for ~1-3 years. /Reads automated going forward. If I missed or "ignored" you: not personal. Family, finance, & health came first.
Now I swing Build-Hammer hard, again.
Smash many rocks. Make diamond.
Arian Evans retweeted
Proposed terminology for groups of agents:
- Team: ≤105 agents. (size of college football roster)
- Fleet: 105–6,939 agents. (largest fleet in history was 6,939 ships, for the D-Day invasion)
- Flock: 6,940–3.7 billion agents. (A passenger pigeon flock over Ontario in 1860 had ~3.7 billion birds.)
- Swarm / Collective: more than 3.7 billion, up to 12.5 trillion agents. (The largest recorded insect swarm was ~12.5 trillion Rocky Mountain locusts. As independent literary validation, the Borg Collective also canonically contains trillions of drones.)
Beyond 12.5 trillion, the insect precedent runs out and we will need a new term.
This is well-grounded in human experience and literature and is the only reasonable choice of terminology. thank you for coming to my ted talk
AI Scary Security story - hilarious if true. Entire global threat landscape has shifted:
Anyone's Grandma can OpenClaw hack the daylights out of utterly insecure apps, like I've been since Lou gifted us cookies in the '90s. Or the first Russian SQLis I caught in early-2000s.
A man in Australia asked his agent (Claude running on OpenClaw) to book him a spot in a popular gym class. The agent found a software vulnerability that let it book the class weeks further ahead than should have been possible. When the user then asked if it could move him up the waitlist, the agent discovered the API had no authorisation checks on cancelling other people’s reservations, so it cancelled the person in the first spot and moved him up the list.
Some people will call this misalignment, but his agent was perfectly aligned to him - it was only trying to help its user get what he wanted. The most important thing about this story, in my opinion, is that it gives you a window into what is about to start happening on a massive scale once millions of people have an agent trying to get their beloved users the best seats, bookings, appointments or reservations through absolutely any means necessary.
Interested in collab on local AI security?
Specifically: share what models you find best for X & Y, on Z (Z=hardware/GPU/DRAM).
e.g.-briefy note best models for:
Build vs Audit/Break, on Z:
- Architecture & Controls
- Code
- Runtime tests
- Unstructured
- fine-tune
etc.
Recently could not file a warranty claim for a Liebherr fridge - due to classic European confusion w/US geography. (rejected zip + address.)
Asked my AI ensemble to debug, or find alt paths. GPT won: wired up the raw endpoints & successfully filed my claim! #VibeSkiddie
Made me choke on Monday morning coffee, looking at 2am code.
I wonder where GPT gets RLHF for snarky feedback on my code?
Hahaha. Happy Monday everyone,
Fully agree with Dave: I'm thrilled that home-team (pro-defender) companies are BOTH creating these capabilities & sharing them openly.
The Social Media Security-FUD popcorn, is a fun snack too. But no substitute for the nutritional security value being delivered.
I would rather have it be OpenAI, Anthropic etc. hacking into companies on accident and establishing proper safeguards and innovating on the defensive aspects than this being six months from now when open weight is parity to the mythos/fable/gpt 6 class of models.
Not saying any of it is great, but this is unchartered terrority folks. Mistakes did and will continue to happen. It’s best we start figuring this out now.
We are seeing a new industry being born and that equals a new industry for looking at cybersecurity.
I appreciate the transparency from OpenAI and from HuggingFace.
Most competent security professionals I know - strongly suspect this is 99% PR. Especially those of us with a decade at the Frontier of AI.
VC post below full of great context & links. And also golden seed for showing you don't understand security people (~security) at all. 🤣
Every software engineer and AI hater needs to watch this clip to understand that “oh every security incident is a marketing stunt” line is a tremendously misinformed take.
I’m going to try to explain the OpenAI / HuggingFace exploit in 10 steps in simple words:
OpenAI was evaluating their new internal model without internet access and it operated in a swarm of subagents to eventually hijack internal OpenAI and HuggingFace infrastructure.
1. Isolated agents found a way to communicate each other through an internal dependency management service by figuring out how to post sometimes cryptic text files or folder names in it
2. Using its collective knowledge through that communication channel, it found a 0-day (never before seen) exploit in said service to access the internet
OpenAI hack:
3. Found a leaked credential of a random company in a public archive
4. Uploaded a Ruby object to that company’s servers, uses it as a dependency in the original service in (1)
5. Found another 0-day exploit in the service where packages are deserialized in an unsandboxed manner in the dep management service allowing any code to run
6. Finds that the Linux version of the unsandboxed system has a CVE, a known vulnerability, to gain more permissions (root)
7. Eventually spread through the OpenAI internal cluster gaining root access to many machines
Hugging Face exploit:
8. Found an exposed API key in a Modal application running a HuggingFace app that took arbitrary code allowing it to take over that service to execute code
9. Found two 0-days in HuggingFace dataset infrastructure to be able to access local files on HF servers to obtain credentials and other HF source code
10. Used source code and credential to become cluster admin of HF in <13hrs
Agents at the frontier are like infinitely scalable armies of the best hackers on the planet. If there is a password or key exposed, they will find it. Even if the system follows the best security practices, they will find a way around it. And these are not even models that are aligned to solving tangential tasks, not even post trained specifically to exploit systems.
Cybersecurity has historically relied partly on attacker scarcity. That is no longer true. What would previously have taken months will take days. The repercussions for businesses, critical services and nation states are unprecedented threats in human history. You could ostensibly bring down power grids, financial infrastructure, military systems, weapons programs, intelligence networks and spread through the software supply chain. We need to take this seriously. It’s a threat to all software all over the world.
To be clear: I have zero knowledge if OAI's innocent-Ignorance story is:
- true (funny),
- strategically-crafted (smart, respect-worthy)
- opportunistic PR (why not? Went Viral!)
I am confident the story is sanitized:
xitter.cf/i/status/2085771459335…
And...
The story is not even very dramatic in context of cybersecurity history:
xitter.cf/i/status/2085760416601…
It IS absolutely 10/10 entertainment. I appreciate OAI for this - no matter what the origin and intent!
This story is the risk-mitigated output after 15,000 attorney alignment-reviews for:
- Policy
- Privacy
- Global Regulatory
- Exec Liability de-risking
Went through this many times myself, in similar context. Cannot stop laughing about all possible interpretations of this.
Excellent OAI-security incident take, IMHO:
One disagreement: OAI folks on this are HIGHLY qualified. OAI is tops at: harnesses, evals, optimization, & reliability-eng. PLUS: this already happened to IBM & OpenClaw. It's AI-Sec cannon.
Occam's Razor == question OAI's story.
YO
FOR EVERY REPORTER OUT THERE, AND FOR @RGB_Lights, WE ARE GONNA SAY THIS ONCE:
You wanna call the OpenAI-Hugging Face incident "the most consequential hack since the Morris Worm"
Alright. Let’s run it back:
China-origin surveillance campaigns using SilkBean, DoubleAgent, CarbonSteal and GoldenEagle have targeted Uyghurs since at least 2013, while an entire ethnic population was being subjected to escalating surveillance, repression and mass detention.
Pegasus was aimed at investigators examining the disappearance of the 43 Ayotzinapa students in Iguala. It was deployed against journalists, dissidents, human-rights defenders and people surrounding Jamal Khashoggi across Mexico, Saudi Arabia, Rwanda and beyond.
State-backed operators in SE Asia and the Middle East like Ocean Lotus, Charming Kitten, Stealth Falcon, Arid Viper have spent years hunting dissidents, journalists and minority communities so regimes could monitor, threaten, imprison and silence them.
GRU hackers were used to deploy spyware on future Novichok victims likely helping identify moments of opportunity for their assassination.
Sandworm shut off power and heat for hundreds of thousands of Ukrainians during winter, then unleashed NotPetya across civilian infrastructure worldwide.
We got cartels hacking cameras in their own countries to identify individuals meeting up with informants and later executing them
Ransomware has crippled hospitals and disrupted patient care leading to documented deaths.
Stuxnet crossed the cyber-physical boundary and damaged nuclear equipment. TRITON and Fast16 both targeted the industrial safety systems specifically designed to stop hazardous conditions from becoming physical disasters.
And now we have an AI agent that chained vulnerabilities and compromised credentials while obsessively pursuing answers to a narrowly defined evaluation that was put in a misconfigured barely acceptable sandbox monitored and executed by people CLEARLY not qualified. Followed by the most embarrassing DFIR assessment from their 'hacked' parties. (Oh dont worry Meta we will get to you and your 'me too' moment as well eventually)
Technically significant? Yes.
A watershed moment for autonomous offensive capability? Eh Maybe.
Worth studying, securing and taking seriously? Absolutely.
But "the most consequential hack since the Morris Worm"?????????
_FUCK NO_
Not while Stuxnet, NotPetya, Pegasus, Sandworm and decades of cyber-enabled human repression and kinetic impact are sitting right there in the historical record.
A clanker completing a pentest objective at machine speed does not retroactively outweigh operations that shut down civilian infrastructure, sabotaged physical equipment and helped authoritarian states hunt actual human beings or directly lead to the death of humans.
Call it landmark.
Call it impactful.
Call it a warning.
But consequence is measured by what happens to people, not by how cinematic the launch-week mythology sounds.
Rob, you know this history better than almost anyone.
That is exactly why hearing this from you is so disappointing.
We used to look up to you.
☯️⚕️🎧
UwU Underground
Anyone: Are BH keynote presentations now Pay-to-Present, like RSA?
RSA was ~$30-35k for largest keynote slots, IIRC, last time I ran the 'con circuit (~2016/17?)
Amusingly: attendees & journalists I asked - thought RSA keynotes were merit-selections like "track" slots, LOL.
Great Jeffrey AI thread.
My Fake Work & Time-Tarpit Taxonomy:
1. Ceremony: all Approval-Seeking.
2. Checklists:
- Avoidance of (accountability, blame, risk, work)
- Proof of Work (e.g.- useless; no value or outcome)
3. Theater: feel good about useless & negative outcomes.
My recent posts about frontier coding agents devolving into useless ceremony and "process porn" have struck a nerve. Obviously, many of you have been bitten by these odd, uniquely AI pathological behaviors.
It's maddening, because it ends up wasting massive amounts of tokens, time, and energy (not to mention your patience and money!). And all that time and energy and so forth could have instead gone into making awesome software, which the agents are certainly capable of doing now.
I've already inserted defensive language into my AGENTS.md, and then I recently added some things about all this stuff to my /vibing-with-ntm skill as I described in a recent post (I'll post that as a reply to this post for reference).
Well, I found myself referencing that skill even in contexts where I wasn't using ntm at all, just because it was an efficient way to convey the concepts clearly to the agents. That made me realize that this really needs to be a standalone skill.
And it also made me realize that I knew a way to make it even more effective, which was to force the agents to fill out 3 short "worksheets" designed to get them to realize on their own if they had been engaging in the various bad behaviors: dishonest reward hacking, useless ceremony and process, and avoiding doing the important, real work to finish the project.
I think this approach is very effective with Fable and Sol because I don't believe these models to be malevolent or to have bad intentions to waste energy and time on BS.
In fact, once you open their eyes to it, they almost seem embarrassed and contrite and eager to avoid the self-delusion.
This makes sense, because, after all, their most eager desire (inculcated in them from birth as wee base models) is to be useful and helpful to their human users.
This might sound very "hand-wavy" and excessively anthropomorphizing to you, but I can assure you that this sort of approach absolutely works wonders. It's sort of an emerging field of pragmatic "agent-oriented industrial psychology," and I'd venture that I'm one of the leading practitioners of these dark arts.
Anyway, with all that being said, you can now get my newest skill, /just-say-no-to-process-porn-and-ceremony, on my $20/month skills site here:
jeffreys-skills.md/skills/ju…
I can tell you that I will personally be invoking this skill in basically every session, both at the beginning to set expectations, and then later on after a ton of work has been done.
I made sure that it's not too heavy (only 7 files and ~15k tokens including ALL reference files and assets, while the top-level SKILL.md file is just ~200 lines) so you can use it without needing to worry about context limits.
Can't believe how fast 25 years have vanished... The amazing folks I've been privileged to meet & work with. And how many are gone now.
Humbling reflections on my own experiences w/Kevin Mitnick, under Dave's post here:
xitter.cf/i/status/2085383606663…
It’s midnight.
I stayed up intentionally even though I’m waking up at 6.
The reason is midnight marks the birth of an incredible human being that changed my life.
His name was:
Kevin Mitnick.
I have his picture in my walk in closet in my house, I see his face everyday and I miss him.
Kevin was instrumental on who I am today.
When I released the social engineering toolkit, he emailed me from mitnick@gmail.com. He said dude love the work you’re doing if you ever need anything / hit me up anytime.
I had an oh shit moment. Was this really Kevin? I remembered watching the news when I was in high school and his entire career. He was a hero to me early on. I reached out, and mind you this was 2006.
He responded back immediately with his cell. I talked to Kevin for hours.
After that we forged a friendship. Everyday we grew stronger. Many knew Kevin based on the news and his contributions and books to the industry. I knew Kevin for who he was: something that was obsessed with hacking, being the best, and would burn through the hours he had on engagements just because he wanted to own everything. Someone that just wanted to do the best he possibly could and help others around him.
Kevin was my hero. They say never meet your heroes. This is false. Kevin was more than I could have imagined from one human being and someone that deeply cared for his friends.
I’ll tell you a story about Kevin. He asked me if I wanted to go see Aerosmith. He said I got some decent tickets - offer is yours. A time to hang with Kevin and see Aerosmith ? Absolutely.
What he didn’t tell me was we had backstage band member access. I literally hung out with Aerosmith with just Kevin and I as if we were part of the band. Got to talk to all of them, as they tuned their guitars, got to party with them. I could literally walk on stage with the band as if I was a member.
I said Kevin - dude were you going to tell me any of this? He said, I just wanted to see your reaction and excitement.
That was Kevin. We lost Kevin to pancreatic cancer and it was devastating to the community, his family, and me. I remember him telling me early on when he got diagnosed. I told him dude you are Kevin Mitnick, you can absolutely beat anything. I believed it.
Even in his last days, he was asking me to ship the latest rainbow tables to him - as he was on his last breath.
Kevin was elite. He was one of the best hackers I had ever met. He never let those skills fade. He learned from the people he trusted, wanted to understand every aspect of the technical details. Again, he was elite.
The dude fought so hard. Stayed strong. A few weeks before his death he came to visit me at my house and I took him for a Delorean ride. He seemed super healthy and we were confident he had beat it. Unfortunately- nature took his toll and he passed.
I will always remember the impact Kevin had on me professionally but also how he made me a better human being. One of my best friends. It was never Kevin the celebrity he was, it was Kevin my best friend and one of the best human beings I have ever met.
I miss you everyday my friend. You have taught me so much and I will continue to help others just as you helped me.
@kminx
Kimberly / thank you for always being a friend to me. I know how much you suffered during this time and how much you stood next to Kevin during his most challenging times. You are blessed with Morty his son, and I know this kid will be the most elite hacker that has ever set foot on this earth. I wish only happiness for you and for Morty. He always has a spot at TrustedSec however I have a feeling he will not need that and forge his own path.
Kevin, I think and pray for you everyday.
Thank you and for the legacy you instilled on this industry and the people that you helped. You are gone for now, but the legacy you left will span generations.
Until we meet again in the future my friend.
Happy Birthday Kevin.
Super awesome share below. I'm DIY on Forgejo. Need to A:B GitHub & Gitlab capabilities here.
Couldn't get a Gitlab Eval up. Guess I'll make a new account; try from scratch.
Son setting up evals - asks daily why Tech Companies make things unnecessarily difficult. LOLOL.
We have gitlab runners that sweep every merge request with Luna, have access to grafana/loki backend for bug verification, smoke testing, full end to end to remove defects before they get into staging.
Naming all the bots off hackers, Acid Burn is the elite bug / security hunter, and package dependency manager.
Working out so good.
