@WisconsinCS ‘26 / Prev AWS / UW Research Train in systems @getcrackedio
Joined December 2025
- Tweets346
- Following165
- Followers45
- Likes972
Pinned Tweet
Can an AI agent study a codebase before it knows the questions it’ll be asked?
And if so, which techniques work best today?
I built StudyLoop to test that.
The results were interesting. 🧵
Samad Syed retweeted
Engineers around the world are burning out reviewing code written by AIs.
But we can do something about it. RevEngine is an open source tool we built at Laude that makes code reviews less painful, which makes engineers happier.
It's a common misunderstanding to think that software engineers are becoming less important as AI gets better at writing code. The opposite is true. As software engineers spend less time writing code (and more time guiding agents), it is *more* important that they understand what's happening under the hood.
Not only do we need more engineers than ever, they need to be smarter and have better tools than ever.
Minimal harness + strong model = flexible code review agent. That’s the idea behind RevEngine, Laude’s open-source tool now topping @GitHub’s ReviewBench. @NickAJalbert shares how it works, which model the harness helped most, and what else the team learned in the process: laude.org/updates/lab-notes/…
Great article featuring the 🐐 @NickAJalbert
Minimal harness + strong model = flexible code review agent. That’s the idea behind RevEngine, Laude’s open-source tool now topping @GitHub’s ReviewBench. @NickAJalbert shares how it works, which model the harness helped most, and what else the team learned in the process: laude.org/updates/lab-notes/…
Great article! Highly recommend reading for anyone looking for a primer on the current open frontier of agent swarms and systems.
We’ll need more people and projects working in this domain so anyone can have access to the knowledge and software that allow for multi-agent systems to produce major works.
Love the narrative style of the article too, I think the best articles are written in this fashion.
Samad Syed retweeted
The mathematician William P. Thurston (1946-2012) perhaps expressed best the thing I mean by 'higher-level' abstractions instead of vibe coding.
In 1994, he wrote:
> In large computer programs, a tremendous proportion of effort must be spent on myriad compatibility issues: making sure that all definitions are consistent, developing “good” data structures that have useful but not cumbersome generality, deciding on the “right” generality for functions, etc.
> The proportion of energy spent on the working part of a large program, as distinguished from the bookkeeping part, is surprisingly small. Because of compatibility issues that almost inevitably escalate out of hand because the “right” definitions change as generality and functionality are added, computer programs usually need to be rewritten frequently, often from scratch.
Is it possible to have a language with the familiar advantages of composition, modularity, and code re-use without the overwhelming manual bookkeeping cost? Maybe that will look more like math.
> The standard of correctness and completeness necessary to get a computer program to work at all is a couple of orders of magnitude higher than the mathematical community’s standard of valid proofs. Nonetheless, large computer programs, even when they have been very carefully written and very carefully tested, always seem to have bugs.
Quotes from "On proof and progress in mathematics" (1994). I was reminded of these quotes by a CS stackexchange answer I wrote in 2017, motivated by the confirmatory news of the sad death of stackoverflow😓😰
StudyLoop is going to NeurIPS :)
Accepted to two workshops:
• Verify-Agents
• RAAAI
Single author. Independent research.
Now I just gotta figure out how to get to Sydney and Atlanta 😭
Aurora is the best AWS team (biased)
The agent-first era of AWS is here 🌌
A painful part of working with your data has always been that your live data and historical data are stuck in separate systems: the order a customer just placed lives in your database, while their last five years of orders sit in a data lake in S3.
And answering a real question usually needs both at once (is this a normal purchase for them, or should we flag it?), and to do that you had to move the data together first, copying history out of the data lake into your database (or the other way around), because the database couldn’t read it where it lived.
That meant guessing ahead of time which data you’d want, keeping a second copy of it all, building pipelines to move it, and constantly syncing so the two didn’t drift apart. A lot of plumbing, and slow going, all before you could answer one question. And even then, the answers were only as fresh as your last sync.
That now changes with Aurora PostgreSQL, which can call your live data and historical data in S3 together, in a single query. No copying, no pipelines to keep in sync.
And it’s fast, because we’ve built in DuckDB, a popular open source engine that’s really good at reading and analyzing data right where it’s stored. DuckDB reads the open formats like Parquet and Iceberg already sitting in your data lake, so there’s nothing to convert or move.
As folks build AI agents into their apps, the data their agent needs will depend on the task in front of it. Being able to query that specific data live, instead of copying it over just in case, is gonna be a big help for builders. aws.amazon.com/blogs/aws/ama…
Samad Syed retweeted
The @Caltech community and SAIR are redesigning and co-organizing Mathathon: Old Problems, New Proofs.
Explore alternative proofs and clearer explanations, using AI to deepen human understanding.
More on our co-founder Terence Tao’s blog:
terrytao.wordpress.com/2026/…
Great post
I think a lot of folks have just been focusing on their own internal evals for their agentic systems, and then find 1-2 open benchmarks to make their "headline" of whatever blog post/research they publish.
Good benchmarks have been getting increasingly more complex too, making it harder for ICs to develop them independently.
Using open benchmarks has gotten very frustrating because it's very hard to disentangle what the model already knows / is actually bottlenecked by.
E.g. when testing RLMs on "new" long benchmarks, some models will just magically regex for the right offloaded information and grab the solution...
This is roughly how I've interpreted the HarnessTax paper as well, where performance seems to basically be a function of only the model capability and not the harness, but in practice we observe very different behavior / preferences with different harnesses. Model capability dominates on open benchmarks specifically.
Not really sure what this implies because newer benchmarks constantly get swallowed, but I hope someone comes up with a better way to benchmark models in the open because it's largely uninteresting atm. And no, the solution isn't a benchmark that tests how far models get in a 2-day task. There's definitely a better way to host and/or manage benchmarks to actually gauge capabilities, at least for longer than a year.
Been working in the agent-to-agent communication space lately.
If you're building with both Claude Code and Codex, two projects worth checking out are AgentRelay and relay-cli.
Both are exploring really useful abstractions for bridging agent sessions beyond the usual “write a markdown file and have the other agent read it.” They also work really well today, not far into the future.
Samad Syed retweeted
Proud of the @SnorkelAI team for all the growth (on every axis) over the past 12 months; already 8-digit revenue increasing 18x in one year is wild, and that number keeps rising. There are trillion dollar hardware companies and trillion dollar model companies. There will be trillion dollar data companies as well -- it's the other major AI ingredient.
AI development at its core is teaching machines to do what humans do (pre-AGI) or want AI to do (post-AGI). Data = whatever you use to show the model what you want (training data) or evaluate whether it does what you want (evaluation data). The shape of that data has changed significantly since Snorkel spun out of Stanford -- we've gone from requiring 30s per datapoint (simple label on simple text or image) to 30h per datapoint in some cases (complex multi-turn, multi-player environments with tools, traces, tests, etc.). This shift toward more complex and higher quality data requires research expertise, not throwing bodies at the problem.
AI is still in early innings. The winners in the data space will be the companies that create the compounding loops where AI and human work together and improve each other. The "10x (or 100x) data developer" is definitely a thing -- I know because I've seen it, and you can already start to see the results (18x YoY growth). This is what an AI-forward data company looks like.
I'm excited to announce @SnorkelAI's $350M Series E at $3.5B, led by @insightpartners and @S32_VC.
We've grown 18x+ in the last 12 months since launching our Data-as-a-Service offering, passing $375M ARRR this week.
As AI advances to superhuman capabilities, AI data & environment development must advance with it - and basic staffing and crowdsourcing approaches are not enough.
AI progress now requires deep research and technology work that combines human expertise with specialized AI in compounding ways. @SnorkelAI is building the RSI data engine and frontier data lab for this next phase.
We're honored to have the support of existing investors Addition, @lightspeedvp, @GreylockVC, @GVteam, P7, Factory, @WellsFargo, Walden Catalyst Ventures, and new investors @ThirdPointLLC, @MarchCPs, @BlumbergCapital, @AllegisCapital, @Frontlinevc, and @standard_vc.
–
@SnorkelAI started as a research project a decade ago at @StanfordAILab.
Our thesis was simple: AI progress would become increasingly data-centric – and therefore data development should be studied as a true research and technology problem, not just a staffing and crowdsourcing one.
Today, as AI capabilities verge on superhuman, building the data and environments to safely measure and train AI is becoming too hard for even the smartest human experts to do alone.
Only humans and AI agents, collaborating together in compounding ways, can meet the accelerating needs of the frontier, and keep humans in the driver’s seat of AI progress for decades to come.
At @SnorkelAI, we are building the data lab to define the shape of this new “Data 2.0” frontier, and the new paradigms of human-computer interaction needed to advance it.
Our key focus is building the RSI engine for data, where specialized AI models accelerate and improve human expert output, and in turn, scaled human supervision is used to continuously evaluate and improve these models – creating a powerful compounding loop to keep pace with an accelerating RSI frontier.
With this round of funding, we are also doubling down on our commitments to support data development for open benchmarking and evaluation (more news here soon!); an increasingly diverse ecosystem of general and specialized intelligence; and a path to safe, well-aligned AI built on robust training and evaluation data.
Data development will guide and drive the next stages of AI – and must do so in a human-centric, AI accelerated, open, diverse, and safe way. We are excited to support this mission in the next decade of research ahead at @SnorkelAI.
More thoughts here: snorkel.ai/blog/data-2-0-and…
Samad Syed retweeted
aws made the industry soft. we're used to them handling all the hard problems and everyone can be a pure product company with 85% margins
ai infra is not mature enough for this but everyone is pretending like they can offload the actual work to someone else
it's why there's 27 model router products. even most inference companies do the bare minimum on compute someone else set up
and it all feels successful because there's so much money around it's hard to justify doing the hard thing when the easy thing is profitable
but the gap between what openai, anthropic, deepseek, moonshot and everyone else is huge
and it's easy to forget that someone actually has to do the work to close it
Samad Syed retweeted
my quick and dirty analogy for the model's flavor of jaggedness in 2026 is kind of like...
if you ask for a taxi ride but someone hands you a catapult that is highly pre-aimed on a very small number of destinations in advance
initially, you're super impressed by the speed and distance of transportation, but you can never quite use it to land at your actual freaking destination, no matter how many times you're willing to get thrown flying through the air :/
excuse the dumb analogy but that's the informal version of the intuition
i don't see enough of this, so here again:
we're all wondering what the future of work looks like with AI progress, but I'm in constant agony trying to extract OK work from frontier models at "Ultra" settings
with levels of detailed context and elaborate feedback that i’ve never given a human in my life (… considering that my job is literally to give feedback)
at least up to Sept 2026, public frontiers models are - as they were in 2020 - systems that know a whole lot but are fundamentally quite dumb and unadaptive
hello from cambridge, massachusetts
there are a concerning number of dunkin donuts here and the computer science building’s architecture appears to be AI generated
very cool place though
I think Deep RL is about to have their GPT-3 moment with directions like Vintix II and goal-conditioned agents.
Great article by @CompleteSkeptic check it out!
Excellent and criminally unrecognized take by Diogo @CompleteSkeptic below, echoing the longstanding DSPy philosophy.
Methods change often and are dominated by compute and scale, so don’t be attached to them.
What is worth most of your time is careful specification of problems.
