Anna Nachesa retweeted
New University of Illinois paper finds that when AI decides if a research idea is new enough to work on, it often says no to ideas that human scientists later publish.
AI scientist pipelines run a novelty check to decide which generated ideas get compute. These checks judge a whole proposal at once, though most real papers build on some earlier contributions.
They built ClaimNov, which splits a proposal into a few claims, matches each against claims from prior papers, and gives a judge the claim-by-claim evidence.
A deployed check caught all 380 disguised copies but also rejected 96% of ideas researchers later published. ClaimNov caught 97% of copies while rejecting only 1.5% of those ideas.
If your pipeline filters ideas for novelty, score claims separately and keep proposals that build on 1 known result instead of discarding them.
Anna Nachesa retweeted
I took about 50,000 photos through a 14" telescope last night to produce this: A hyper-detailed photo of the full moon. The fidelity of the photo was great enough to allow me to greatly enhance the color, revealing a hidden world of geological diversity.
Anna Nachesa retweeted
Massive. New Harvard study finds that AI-written paper reviews now beat the best human peer reviews on every measure the authors tested.
Peer review is slow and often misses errors. Earlier studies concluded AI should only assist human reviewers, not replace them.
They used 134 reviews from rapidPeer, a Harvard service that pays expert reviewers well. Claude Opus 4.8 reviewed the same 1124 papers, and GPT-5.5 plus 3 former journal editors judged the pairs blind.
GPT-5.5 picked the AI review in all 132 decided pairs, and the editors picked it 87% of the time. Only 5.5% of the AI's claims about the paper were wrong, versus 15.9% for the human reviews.
Teams that depend on expert critique can now let AI do the first full review pass, and keep a human for the final decision.
Anna Nachesa retweeted
So basically, today PHP got two critical bugs that let anyone run code on your server without even logging in, which is bad, and then the details leaked before the patch was even out, which is worse, and it works on WordPress and MediaWiki and Magento and Joomla and Laravel, and Censys says like 1.5 million servers are running a version with no fix, so the NSA was like "take your system offline," and the FBI was like "take your system offline," and CISA and the UK and CERT-EU were all like "take your system offline," and then the Prime Minister of Japan, the actual Prime Minister, posted at 5:55 in the morning telling 48.8 million people to take their system offline, so everyone took their system offline, like the Dutch cybersecurity agency took its own website offline, and the Dutch parliament went offline, and the Port of Rotterdam went offline, and Finland and France and Denmark started pulling government sites, and Wikipedia went down, and Archive.org went down, and Asana and GOG and Something Awful went down, and the University of Vienna's webmail went down for some reason, and that's just PHP. Because GitLab also dropped a CVSS 10.0, which is the highest score they give out, for a bug that's been there since 2014, so that's every self-hosted GitLab for twelve years, and then LiteLLM 1.107.0 got hijacked and shipped a credential stealer and a remote access trojan in every single wheel and the Docker images, including latest, and you didn't even have to import it, just installing it was enough, and it's the second time this year, and then Railway paused every deployment on the entire platform because someone got into the environment variables, so now everyone has to rotate every secret they've ever stored there, and TiDB Cloud said someone was walking around inside with stolen credentials for basically a month and an extortion group says they took 10.6 terabytes, and Zeabur got breached for the second time in two months and is just shutting down, like the whole company, and Node.js patched remote code execution and a TLS bypass, and CISA says Plex is being actively exploited, and someone took over 300 Tor exit relays to steal SSH passwords, and SafePay ransomware has hit over 600 critical infrastructure orgs, and India's biggest bank filed a data loss incident, and Japan has been in an official state of emergency in cyberspace since October 9, which honestly explains the 5:55 AM post. Oh, and eMudhra, a certificate authority that's issued over 60 million certificates, whose entire job is proving websites are who they say they are, was supposed to be using private keys that are random numbers over a hundred digits long, and one of them was 4. Just 4. And the others were like 66 and 237. Anyway, it's Wednesday.
Anna Nachesa retweeted
Replying to @fiveoutofnine
Anna Nachesa retweeted
To give context on age:
Grace Hopper began computing at 38, completed the first compiler at 46, helped shape COBOL at 53, kept developing COBOL for the Navy in her 70s, retired from the Navy at 80, & then became a consultant for the Digital Equipment Corporation.
v/@cooperx86
Anna Nachesa retweeted
The first self-driving cars in Switzerland started a few days ago. It's a tiny pilot project called "iamo" in collab with SBB (swiss rail) that brings people to the next train station. Very small, only 4 cars and with safety driver sitting behind the wheel. Uses tech from Chinese company WeRide.
Hopefully it goes well and "warms up" the public and regulators so that many other self-driving cos will follow soon!
Anna Nachesa retweeted
You can literally type a sentence and watch a Transformer process it.
This interactive explainer runs a GPT-style model in the browser and lets you follow the whole path:
tokenization → embeddings → Q/K/V → attention → MLP → logits → next-token probabilities
You can inspect attention maps, intermediate values, and change temperature, top-k and top-p to see how generation changes.
If Transformers still feel like boxes connected by arrows, this is worth opening.
poloclub.github.io/transform…
Anna Nachesa retweeted
So... In the last few days I have had my agents write:
1. A word processing app in which I will write future books. It replaces Scrivener.
2. A script writing app in which I will write future video scripts. It replaces FinalDraft.
3. Crapper, a tool that measures and records CRAP statistics for projects. The tool handles many different languages, and adding new ones is easy.
4. Mutator, a tool that runs mutation tests and records the results. The tool handles many different languages and adding new ones is easy.
5. Dryer, a tool that runs duplication analysis on a project. The tools handles many different language and adding new ones is easy.
6. UML-Viewer, a tool that displays the structure of a project in a UMLish diagram that shows modules and dependencies and highlights dependency cycles. Clicking on a module displays it's contents; which may be deeper UML diagrams or, at the leaves, code. The tools color codes the modules to reflect their CRAP and Mutation scores, and as you drill down you can see those scores for each module and function. The tool understands many different languages, and adding new languages is easy.
I recon that, last year, this would have been well over a man-year of effort.
All these tools are in github.com/unclebob. I advise you not to use them directly. Instead, point your agents at them and use them as templates to build the tools that are customized to your needs. The licenses are all BSD Zero, meaning that there are no strings. Do what you like.
Anna Nachesa retweeted
list of 12 active AI researchers on X:
1. @karpathy — Andrej Karpathy
2. @rasbt — Sebastian Raschka
3. @AndrewYNg — Andrew Ng
4. @drfeifei — Fei-Fei Li
5. @demishassabis — Demis Hassabis
6. @GaryMarcus — Gary Marcus
7. @ai_explorer25 — AI Explorer
8. @maximelabonne — Maxime Labonne
9. @chipro — Chip Huyen
10. @ilyasut — Ilya Sutskever
11. @thsottiaux — Thibault "Tibo" Sottiaux
12. @bcherny — Boris Cherny
Super interesting paper from Meta.
Long-horizon research agents are coming.
But one common problem with research agents today is the lack of originality and how to decide what experiments are worth exploring.
An AI research agent can propose far more experiments than it can afford to run, so the problem is not idea generation, it's deciding which candidates get GPU time.
AI Research Preference Models is trained to predict which candidate solution is most promising before any of them execute.
Two variants, both built on frozen pretrained LLMs. An inference-only model reasons over candidate plans, code, and previously executed solutions. An agentic model additionally runs small-scale pilot experiments before committing budget.
Dropped into the AIRA-dojo agent and measured on AIRS-Bench, average normalized score moves from 0.684 to 0.711 and 0.729.
Both variants reach the unguided agent's 24-hour performance in roughly 15 hours, using less than two-thirds of its execution budget, and together set new state of the art on two AIRS-Bench tasks.
Paper: arxiv.org/abs/2608.13940
Chat with Paper: academy.dair.ai/papers/ai-re…
Anna Nachesa retweeted
Andrej Karpathy’s 1-hour Stanford lecture on AI engineering is one of the best explanations I’ve seen of how AI systems actually work.
The progression is simple:
10% → LLM
30% → Prompt
50% → Agent
70% → Loop
100% → Graph
The key takeaway:
AI engineering isn’t just about writing better prompts.
It’s about building systems around models — giving them context, memory, tools, feedback loops, and data flows.
“Delete everything, keep Graph.”
Definitely worth watching if you’re building with AI agents.
Watch → Bookmark →
Anna Nachesa retweeted
This Yale + University of Chicago paper shows that real gap between LLM generated research ideas vs humans is not idea quality, but idea range: LLMs think narrower than human researchers.
The researchers built a controlled test from 11,683 real papers, using each paper’s nearby prior work as the shared starting point.
They asked models to propose a new motivation and method from those same prior papers, then compared those ideas with the real human paper ideas.
Instead of asking whether 1 idea looked novel, they labeled each idea by what gap it noticed and what kind of contribution it made.
Human ideas spread across many patterns, such as explaining mechanisms, testing failures, measuring evidence, building systems, and improving efficiency.
Only 12.1% of human ideas were mainly about connecting separate work, but 47.1% to 64.2% of LLM ideas did that, meaning models used this move about 4 to 5 times more often.
Even extra reasoning made this pattern stronger, suggesting models often polish a familiar recipe instead of finding more varied research moves.
---
– arxiv. org/abs/2607.01233
Title: "Measuring the Gap Between Human and LLM Research Ideas"
Anna Nachesa retweeted
Best explainer on Kimi K3 i've read.
It walks you through how the model works & the elegant innovation behind it:
- K3 is the biggest open model anyone has released: 2.8 trillion parameters total, though only 104 billion of them do the work on any given word.
- The problem Moonshot went after is memory. Normally a model keeps notes on every word it has read, and that pile grows with every token, which is why long conversations get slow and expensive.
- K3 mostly stops the memory pile from growing. Three out of every four layers use a new mechanism called Kimi Delta Attention, which keeps a fixed-size working memory and edits it as it goes, overwriting what's stale instead of hoarding everything. The fourth layer keeps a compressed record of each word so exact details are still recoverable when they matter.
- They proved this on a small model first. A 48B test version used up to 75% less memory and ran about 4× faster at long context, while matching or beating the conventional design on the benchmarks they reported.
- Training leaned hard on long jobs including coding, browsing, research, visual work, agent sessions running hundreds or thousands of tool calls in a row.
- 1 Million cached input tokens costs $0.30. An agent can pull the same repo, the same docs, the same tool definitions back into context over and over without the bill getting stupid. Moonshot credits that to the memory design working alongside their serving stack.
- The biggest takeaway: context window size is the main character, but what matters is what a model compresses, what it forgets, and how it gets exact information back.
Anna Nachesa retweeted
Still the best hour on graph engineering ever recorded, Andrew Ng breaking down how to build agentic knowledge graphs from scratch:
00:00 - what agentic knowledge graphs actually are
03:05 - building a graph from scratch
13:58 - the architecture behind multi-agent systems
22:57 - building a real one with Google ADK
01:06:02 - why graphs are the future of AI agents
I've seen $500 courses that teach less.
Watch it, then take it further with my step-by-step guide on graph engineering below.
Anna Nachesa retweeted
The One Change That Lets Small Models Outperform Their Size
1/
Everyone knows you need a 70B model to beat GPT-4 on complex agent tasks.
We did it with 8B—by changing one thing that has nothing to do with the model.
A thread on why your agent's biggest problem isn't the LLM. 🧵
2/
The standard approach: feed the LLM a growing text history, ask it to pick the next action, repeat.
This works... until it doesn't.
Errors propagate. Context bloats. Hallucinations spike. And when something breaks, you replan everything.
3/
Here's the kicker:
The problem isn't your model's intelligence.
It's that you're asking it to hold plan structure + execution state + I/O dependencies all inside a linear text stream.
That's like running an OS without a process table.
4/
Enter: Atomic Task Graph (ATG)
Instead of a text trajectory, you build an explicit DAG.
Each node = one tool call.
Edges = data dependencies.
The LLM still does the thinking—but now the graph holds the structure.
5/
Three moves make this work:
✅ Interface-preserving recursion: Break tasks into subtasks while keeping I/O contracts clean
✅ Dependency-aware execution: Run independent branches in parallel; catch bad plans before running them
✅ Minimal repair: When something fails, fix only the broken subgraph—leave the rest frozen
6/
Result?
Llama-3.1-8B-Instruct beats GPT-4+ReAct on ALFWorld (household tasks) and WebShop (shopping).
Not with fine-tuning.
Not with more data.
Just by swapping the execution substrate from text → graph.
7/
Why does this work?
Context narrowing: Each node sees only its local inputs—no bloated history
Pre-execution validation: The graph lets you "think" before acting
Localized failure: Repair 10% of the graph instead of replanning 100%
8/
The contrarian insight:
Control framework > model size (in the 7–70B range).
You're not squeezing more juice from the same fruit.
You're giving the model a better glass to pour into.
9/
Practical translation:
• 20–40% step reduction (parallelism)
• 70%+ hallucination drop (narrower context)
• 3× faster recovery (minimal repair)
• Training-free, plug into existing tool APIs
This isn't research theater. It's production-ready architecture.
10/
The bigger implication:
If you can beat GPT-4 by changing the substrate instead of the model, what else have we been over-parameterizing?
Retrieval pipelines?
Code generation?
Multimodal workflows?
The graph wins again.
11/ [Final + CTA]
TL;DR:
Stop storing your agent's plan in text.
Start storing it in a DAG.
Small models suddenly look a lot smarter.
Anna Nachesa retweeted
MIT published a paper arguing that every AI model on earth is secretly converging on the same "brain."
The paper is called "The Platonic Representation Hypothesis." The claim inside it is one of the strangest ideas in modern machine learning, and once you see it you cannot unsee it.
For years, everyone assumed that a model trained on images and a model trained on text were building fundamentally different things inside themselves. Different data. Different architecture.
Different world. A vision model learns what a cat looks like. A language model learns what the word "cat" sits next to. Two separate universes with no reason to line up.
The researchers checked whether that was actually true.
They took 78 vision models and a stack of large language models, and measured how each one organized concepts internally, not what they output, but the shape of the relationships between ideas in their heads. Which things they treat as close together. Which things they treat as far apart.
Then they compared the shapes across models that had never seen each other's data.
The shapes were lining up.
And here is the part that should stop you cold. The bigger and more capable the models got, the more their internal maps agreed with each other. A better vision model and a better language model don't drift apart. They converge. As if they were both climbing toward the same summit from opposite sides of a mountain.
The authors put it in a line that sounds almost like a joke, borrowed from Tolstoy: all strong models are alike, each weak model is weak in its own way.
Then they took it one step further, and this is where it stops being a curiosity and starts being unsettling.
They found that how closely a language model's internal map lined up with a vision model's internal map actually predicted how good that language model was at reasoning and at math. The models that saw the world more like the other modality did better at problems that had nothing to do with images at all.
So the question the paper asks is the obvious one. If a model that only reads text, and a model that only sees pixels, and a model trained on a completely different objective are all drifting toward the same internal representation as they get smarter, what is that representation a representation of?
Their answer is the thing that gives the paper its name.
They argue the models are all converging on a single shared statistical model of the reality that generated the data in the first place. Text is a shadow of the world. Images are a shadow of the world.
Sound, touch, everything, different shadows cast by the same underlying thing. And a big enough model, trained on enough of any one type of shadow, starts reconstructing the object casting it.
Plato said this in 375 BC. The allegory of the cave. Prisoners chained facing a wall, watching shadows, mistaking the shadows for reality, while the real forms exist outside the cave, casting everything they see.
The MIT team took his allegory literally and pointed it at neural networks. The training data is the shadows on the wall. The model, they argue, is slowly turning around toward the fire.
They even proved a version of it mathematically. Under certain conditions, a whole family of learning algorithms is provably pulled toward representing the same underlying statistical structure, the co-occurrence relationships baked into reality itself, regardless of whether they're fed words or pixels. Different sensors, same answer.
The implications the paper draws are the part that should matter to anyone building this stuff.
If it's true, then to build a better language model you should train it on images, because pictures carry information about the same reality that words are trying to describe. They cite evidence this already works. It means translation between any two modalities gets easier the smarter models get, because they're all speaking dialects of the same underlying language. And it means, their words, that hallucination might decrease with scale, because a model converging on an accurate model of reality has less room to invent things that reality doesn't contain.
Now the honest part, because the authors are honest about it and a viral thread that skips this is lying to you.
This is a hypothesis, not a verdict. On their own measurement, the alignment between vision and language models climbs clearly with scale but only reaches about 0.16 on a scale where 1.0 is perfect. They flat-out ask in their own paper whether that number means strong convergence with noise on top, or weak convergence with a mountain still left to explain. The clean math only holds in an idealized world where nothing is lost between reality and observation, which is not the world we live in.
Some things a picture can show that a sentence never will, and vice versa. And in domains like robotics, they see no convergence yet at all.
So it might not go all the way. The cave might be deeper than one paper can measure.
But sit with the shape of what they found anyway.
Systems built by different labs, on different continents, trained on different senses, for different reasons, with no coordination, are independently drifting toward the same internal picture of the world.
The smarter each one gets, the more they agree. And the thing they seem to be agreeing on is reality itself.
Anna Nachesa retweeted
The entire vector database industry just got destroyed by A free tool from 1974.
For the last two years, every company building AI has obsessed over "RAG" (Retrieval-Augmented Generation).
They spent millions on complex vector databases, semantic search, and embedding models.
The promise? It’s the only way to give AI long-term memory.
But researchers published a paper proving that the most sophisticated AI agents actually do better with grep.
Yes. The basic search command invented 52 years ago.
Researchers tested modern AI agent harnesses, including Claude Code and Gemini CLI, on complex retrieval tasks.
They compared expensive vector retrieval against simple, old-school grep searches.
The results are humiliating for the AI infrastructure industry.
Across the board, grep didn't just compete with vector databases.
It beat them. It yielded consistently higher accuracy.
Why?
Vector search tries to be "smart." It finds documents that are semantically similar.
But when an autonomous AI agent is searching for exact code variables, specific log errors, or explicit names, "similar" isn't good enough.
"Similar" introduces noise. "Similar" distracts the agent.
grep is dumb, literal, and perfectly precise. It finds exactly what the agent asks for.
And as it turns out, when you combine a hyper-intelligent reasoning model with a perfectly literal search tool, the results are explosive.
The AI already has the semantic intelligence. It doesn't need the database to think for it.
It just needs the database to fetch.
When researchers intentionally added distracting, irrelevant text to the context, vector search collapsed. It grabbed the wrong information.
grep ignored the noise and pulled the exact string.
Anna Nachesa retweeted
turns out AI models cannot do math.. even grade school math. the kind a 10-year-old solves.
Apple published a devastating study that exposes a massive illusion at the core of artificial intelligence.
they took the standard math benchmark (GSM8K) that every AI company uses to brag about how smart their model is.
first, they just changed the names in the word problems.. the models' performance fluctuated for no reason.
then, they changed the numbers. the performance immediately dropped.
but then they ran the test that broke everything.
they added one single, completely irrelevant sentence to the word problem. something like: "By the way, 5 of the apples were green."
A human 10-year-old ignores the green apples and solves the underlying math.
the AI didn't.
across every state-of-the-art model, performance collapsed by up to 65%.
the AI blindly grabbed the irrelevant number and tried to shove it into the equation. it didn't know why it was doing the math. it just saw a number and assumed it was supposed to use it.
there is no genuine logical reasoning happening under the hood.
we are deploying these systems to run our finances, analyze our legal documents, and make complex strategic decisions.
but the models don't actually understand the logic they are spitting out.
they just know what a smart answer is supposed to look like.
Anna Nachesa retweeted
Terence Tao says the math behind today’s LLMs is actually simple. Training and running them mostly uses linear algebra, matrix multiplication, and a bit of calculus, material an undergraduate can handle. We understand how to build and operate these models.
The real mystery is why they work so well on some tasks and fail on others, and why we cannot predict that in advance. We lack good rules for forecasting performance across tasks, so progress is largely empirical.
A key reason is the nature of real-world data. Pure noise is well understood, perfectly structured data is well understood, but natural text sits in between, partly structured and partly random. Mathematics for that middle regime is thin, similar to how physics struggles at meso-scales between atoms and continua.
Because of this gap, we can describe the mechanisms but cannot yet explain capability jumps or give reliable task-level predictions. That mismatch, simple machinery versus hard-to-predict behavior, is the core puzzle.
----
Video from 'Dr Brian Keating' YT Channel (Link in comment)
































