Distributed {Control | Data} Systems. Co-founded @FennelAI. Currently @Databricks
San Francisco Bay Area
Joined August 2009
- Tweets927
- Following2.6K
- Followers585
- Likes7.3K
Abhay retweeted
So basically, today PHP got two critical bugs that let anyone run code on your server without even logging in, which is bad, and then the details leaked before the patch was even out, which is worse, and it works on WordPress and MediaWiki and Magento and Joomla and Laravel, and Censys says like 1.5 million servers are running a version with no fix, so the NSA was like "take your system offline," and the FBI was like "take your system offline," and CISA and the UK and CERT-EU were all like "take your system offline," and then the Prime Minister of Japan, the actual Prime Minister, posted at 5:55 in the morning telling 48.8 million people to take their system offline, so everyone took their system offline, like the Dutch cybersecurity agency took its own website offline, and the Dutch parliament went offline, and the Port of Rotterdam went offline, and Finland and France and Denmark started pulling government sites, and Wikipedia went down, and Archive.org went down, and Asana and GOG and Something Awful went down, and the University of Vienna's webmail went down for some reason, and that's just PHP. Because GitLab also dropped a CVSS 10.0, which is the highest score they give out, for a bug that's been there since 2014, so that's every self-hosted GitLab for twelve years, and then LiteLLM 1.107.0 got hijacked and shipped a credential stealer and a remote access trojan in every single wheel and the Docker images, including latest, and you didn't even have to import it, just installing it was enough, and it's the second time this year, and then Railway paused every deployment on the entire platform because someone got into the environment variables, so now everyone has to rotate every secret they've ever stored there, and TiDB Cloud said someone was walking around inside with stolen credentials for basically a month and an extortion group says they took 10.6 terabytes, and Zeabur got breached for the second time in two months and is just shutting down, like the whole company, and Node.js patched remote code execution and a TLS bypass, and CISA says Plex is being actively exploited, and someone took over 300 Tor exit relays to steal SSH passwords, and SafePay ransomware has hit over 600 critical infrastructure orgs, and India's biggest bank filed a data loss incident, and Japan has been in an official state of emergency in cyberspace since October 9, which honestly explains the 5:55 AM post. Oh, and eMudhra, a certificate authority that's issued over 60 million certificates, whose entire job is proving websites are who they say they are, was supposed to be using private keys that are random numbers over a hundred digits long, and one of them was 4. Just 4. And the others were like 66 and 237. Anyway, it's Wednesday.
Abhay retweeted
Neon realtime engine runs in a shallow read replica. Only lakebase architecture has severless read replicas that can run such an engine
More code also doesn’t have to mean more “features”. When code is this fungible, you can explore many more options and throw away what doesn’t work or feel right. This kind of hill-climbing was earlier just not economically viable for most problems.
Abhay retweeted
if "just press the magic button" is what the industry wants, then i'm out. not because my programmer feefees are hurt. i've written enough code in my life to be very tired of it.
but i couldn't imagine a more boring waste of my time than just pressing the magic button.
Abhay retweeted
Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.
- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.
The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.
So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
Judging by my tl there is a growing gap in understanding of AI capability.
The first issue I think is around recency and tier of use. I think a lot of people tried the free tier of ChatGPT somewhere last year and allowed it to inform their views on AI a little too much. This is a group of reactions laughing at various quirks of the models, hallucinations, etc. Yes I also saw the viral videos of OpenAI's Advanced Voice mode fumbling simple queries like "should I drive or walk to the carwash". The thing is that these free and old/deprecated models don't reflect the capability in the latest round of state of the art agentic models of this year, especially OpenAI Codex and Claude Code.
But that brings me to the second issue. Even if people paid $200/month to use the state of the art models, a lot of the capabilities are relatively "peaky" in highly technical areas. Typical queries around search, writing, advice, etc. are *not* the domain that has made the most noticeable and dramatic strides in capability. Partly, this is due to the technical details of reinforcement learning and its use of verifiable rewards. But partly, it's also because these use cases are not sufficiently prioritized by the companies in their hillclimbing because they don't lead to as much $$$ value. The goldmines are elsewhere, and the focus comes along.
So that brings me to the second group of people, who *both* 1) pay for and use the state of the art frontier agentic models (OpenAI Codex / Claude Code) and 2) do so professionally in technical domains like programming, math and research. This group of people is subject to the highest amount of "AI Psychosis" because the recent improvements in these domains as of this year have been nothing short of staggering. When you hand a computer terminal to one of these models, you can now watch them melt programming problems that you'd normally expect to take days/weeks of work. It's this second group of people that assigns a much greater gravity to the capabilities, their slope, and various cyber-related repercussions.
TLDR the people in these two groups are speaking past each other. It really is simultaneously the case that OpenAI's free and I think slightly orphaned (?) "Advanced Voice Mode" will fumble the dumbest questions in your Instagram's reels and *at the same time*, OpenAI's highest-tier and paid Codex model will go off for 1 hour to coherently restructure an entire code base, or find and exploit vulnerabilities in computer systems. This part really works and has made dramatic strides because 2 properties: 1) these domains offer explicit reward functions that are verifiable meaning they are easily amenable to reinforcement learning training (e.g. unit tests passed yes or no, in contrast to writing, which is much harder to explicitly judge), but also 2) they are a lot more valuable in b2b settings, meaning that the biggest fraction of the team is focused on improving them. So here we are.
Robinhood is giving blanket 3% rewards on the gold card, which felt too good to be true. And then they started pushing me to use those points to play mini-lotteries and I realized that it’s genius! Cunning, but genius.
Future is bright with open frontier models
congratulations team @reflection_ai
☀️ future is bright with western open frontier models ☀️
Two maxims I try to live by
- Don’t put anyone on a pedestal, including people you know and admire. No one is perfect.
- Almost everyone successful does something worth learning from. You can learn from someone without liking them, and without endorsing everything they believe.
Every time I mention @rabois on Twitter, the DMs roll in. “Why are you agreeing with Keith?” Or “Why are you disagreeing with Keith?”
It made me realize how quickly we decide who we’re for and who we’re against, usually when it comes to people we’ve never met.
A few rules I try to live by:
1. Be slow to decide you like or dislike someone you don’t actually know.
2. Social media gives you a slice of someone, not the whole person. Leave room for what you haven’t seen.
3. Don’t put anyone on a pedestal, including people you know and admire. No one is perfect.
4. Almost everyone successful does something worth learning from. You can learn from someone without liking them, and without endorsing everything they believe.
5. Keith sat on my board. I know him well. Agreeing or disagreeing with him on any given day isn’t a verdict on the man, and it doesn’t require me to join a camp. It’s just a conversation.
You learn who you can count on when you need help. Keith has always been one DM away, and he’s always done his best to help.
Abhay retweeted
in the coming years were all gonna be looking at kalshi and polymarket the same way we looked at juul
and the people involved will be rushing to hide their involvement in the company
Agreed, and I think this is because we don’t ship code in a vacuum. We ship products and systems, which require dealing with immense amount of complexity and ambiguity. And they require “taste”.
I don’t think they are even remotely close today and I’m not optimistic it’ll ever happen. And in the absence of these abstractions what we’ll get is spaghetti, unless the software engineer steps in to do the engineering.
This is super counter-intuitive since outcomes are what you are (or at least supposed to be) rewarded for. But when in the spiral, momentum can be used to bootstrap outcomes. Momentum is not simply performative "motion" though. It's making tiny incremental steps every hour/day.
I've made this prediction privately to friends but now is a good time to put it out there: @finkd hired @alexandr_wang not just to run MSL, but eventually succeed him as CEO. @Muse is just the beginning.
Much of the AI discourse on X
“Look at my incredible new factory!”
Yo that’s cool, what do you make?
“It’s highly optimised, fully automated, zero tolerance for defects and with a continuous feedback cycle”
Cool cool, so what do you actually make?
“I can interact with it on my phone, laptop, messenger, completely async, and the shared context means it’s always learning how to get better”
Very impressive, but what do you make?
“Every agent has full context, can spawn other agents, review their work, fix defects, and ship continuously.”
yes yes. WHAT DOES IT MAKE?
“Software.”
Oh nice. What software?
“Well right now we’re mostly using it to improve the factory.”
Improve it to make what?
“Anything!”
Such as?
“…a better factory.”
Abhay retweeted
Btw this is why computer science matters. Yes, you can self teach it if you really want to (I personally wouldn't have had the discipline, so college was a way better force). However you learn it, the theoretical fundamentals matter and get applied to work every single day.
If you're interested in "how" or "why": the major culprit is that each row in Alacritty has 32 bytes of metadata and each cell is 24 bytes. In Ghostty, every row and cell is represented by exactly 8 byte each. How do we do this?
The first major culprit is styles. Alacritty stores the full cell style alongside each cell (foreground, background, underline, etc.). Ghostty stores a 16-bit style ID and de-dupes all styles into a look-aside custom reference-counted hash table.
MOST cells are unstyled, and when there are styles MOST styles are shared, and when styles are shared MOST are repeated in a run (multiple cells with the same style in a row). Put this all together, and the tradeoff on compute to access it doesn't even end up being slower.
Next, codepoints. Alacritty stores multi-codepoint graphemes (like, Emoji) by having an 8-byte nullable pointer to a `Vec`. This hurts doubly: (1) its almost always null (because multi-codepoint is rare) yet you pay an 8 byte cost on every cell and (2) every multi-codepoint grapheme triggers a heap allocation to make that Vec.
Ghostty stores single codepoints inline, but multiple codepoints in a look-aside table. The memory for this table uses a custom bitmap-tracked chunk-allocator (since grapheme frequency follows a measurable curve we calculated by scanning various online texts). The presence of graphemes is marked by a 2-bit content tag in our packed 64-bit cell. To keep the key small in the hash table, its limited to a 16-bit unsigned int that is an offset from a base pointer.
Okay, the astute systems programmer will quickly notice there are a lot of 16-bit integers and ask: so this is all limited to a max of ~65K values?
Nay. We maintain our grid using a linked list of contiguous ~400KB memory chunks (which themselves are in a memory pool using a custom allocator to speed up alloc/free). Each memory chunk is limited to 2^16. If/when we reach a limit, we move to the next page. In practice, this really doesn't happen except under pathological cases... the important point is we handle it.
Lots, lots, lots more details, but thats a 10,000 foot view.
These things alone account for ~95% of the difference of our uncompressed vs. Alacritty's uncompressed memory usage. (Theres also a reason why Alacritty's data structures aren't trivially compressable but thats a whole other topic)
