Delphine retweeted
The next frontier is AI that improves itself, and we believe everyone should know how to build it.
This semester, Ao @ao_qu18465 and I are teaching 'How to AI an AI for Almost Anything' @MIT @medialab -- a new course on self-evolving AI systems that recursively improve from experience, feedback, memory, and exploration over long interaction horizons.
The key idea: intelligence shouldn't stop improving when training ends.
We cover four levels of evolution:
(1) context evolution through prompt optimization and context management,
(2) harness evolution through memory, tools & skills, workflow optimization, and multi-agent systems,
(3) model evolution through RL, distillation, test-time training, self-play, and continual learning,
(4) task evolution through generating increasingly complex synthetic tasks, simulations, evaluations, and environments.
Slides, readings, and videos available: mit-mi.github.io/how2ai-cour…
We thank @Kimi_Moonshot and @PrimeIntellect for generously sponsoring API credits!
Delphine retweeted
Nouvelle vidéo ! L’IA va-t-elle tuer les maths ? Entretien avec Hugo Duminil-Copin
Médaillé Fields 2022, Hugo nous raconte l’irruption de l’IA dans les maths, et pourquoi il a décidé personnellement de s’en passer dans son processus créatif de recherche
youtu.be/HJFk7D5Vufc
Delphine retweeted
This semester I’m teaching "Fundamentals of LLMs" at the MVA master in Paris 🎓
I’ve turned my 1st lecture, "NLP before LLMs", into a blog post that goes from simple word counts all the way to attention.
The approach is learning by doing 💪 : each step comes with hands-on exercises inspired by key research papers.
Why practice by hand when an AI agent can write the code for you ? Precisely because it can ! When code becomes easy and cheap, understanding is what matters, and you will build it by doing the work yourself.
Have fun learning by doing !
👉 huggingface.co/spaces/dlouap…
(Next up: Tokenizers. Stay tuned !)
Delphine retweeted
Handbook on Understanding Harness Engineering for AI Agents [48-page PDF]: drive.google.com/file/d/1PT-…
Delphine retweeted
Andrej Karpathy’s 1-hour Stanford lecture on AI engineering is one of the best explanations I’ve seen of how AI systems actually work.
The progression is simple:
10% → LLM
30% → Prompt
50% → Agent
70% → Loop
100% → Graph
The key takeaway:
AI engineering isn’t just about writing better prompts.
It’s about building systems around models — giving them context, memory, tools, feedback loops, and data flows.
“Delete everything, keep Graph.”
Definitely worth watching if you’re building with AI agents.
Watch → Bookmark it-> Follow me
Delphine retweeted
Don't waste 2 years learning to build LLMs like Claude & ChatGPT.
Andrew Ng, the godfather of AI, gave the complete playbook to become an AI agentic engineer in 2026.
• 00:00 - AI agent basics
• 12:12 - AI Agentic workflows & design patterns
• 53:27 - Practical tips for building AI agents
• 1:20:30 - self-improving AI agent loops
• 1:30:19 - multi-agent AI systems
Anthropic pays $750,000/year to engineers who understand the this exact knowledge of LLMs.
Bookmark this & give 2 hours today, no matter what. Then read the article below.
Delphine retweeted
You can now run a 2.8 TRILLION parameter model on a 4GB GPU.
Someone open-sourced a tool that uses "Layer-wise Inference." It only loads one layer onto your GPU at a time. so the VRAM you need depends on the layer size, not the model size.
no quantization. no distillation. no pruning.
→ DeepSeek-V3 (671B) on 12GB
→ Llama 3.1 405B on 8GB
→ Kimi K3 (2.8 TRILLION params) on under 4GB
→ works with almost every open model
the biggest model on it needs the LEAST VRAM. K3 is sparse MoE, so it streams only the experts a token actually routes to instead of a whole dense layer.
2.8 trillion parameters running in less VRAM than a 70B.
Andrew Ng:
“AI agents are doing almost 100% of my tasks now - the hype has exceeded my expectations.
in 4-6 months, we’ll all be building graphs to orchestrate self-improving agents. No more prompting.”
In a 20-minute talk, Andrew Ng explains how to build self-improving agentic systems from scratch.
Worth more than a $500 agentic course.
Watch this video, then read the article below on how to become a graph architect.
Delphine retweeted
A Google Cloud engineer just showed how to build a full app with Claude from scratch.
He spent 26 minutes live on stage doing what most teams take weeks to do.
Worth more than any $500 vibe-coding course, no team, no setup, just Claude and a goal.
The ones who learn what Claude actually does are shipping what everyone else outsources to a team.
Watch it, then read the guide below on the Claude features 99% of users never find.
Delphine retweeted
Google just dropped a 1-hour course on agentic engineering from scratch:
00:00 – How to build your first AI agent
08:24 – Build agent memory (short, persistent, long)
28:34 – Agentic loops, long-running AI agents
40:04 – How to build MCP (MCP vs API)
1:00:22 – Multi-agentic systems
This 1-hour watch will replace 10 paid agentic courses on the internet.
Bookmark this. Watch this weekend.
Delphine retweeted
Google just dropped a 1-hour course on agentic engineering from scratch:
00:00 - Build your first AI agent
08:24 - Give your agent real memory (short, persistent, long)
28:34 - Agentic loops that run for hours on their own
40:04 - Build your own MCP (MCP VS API)
1:00:22 - Wire up multi-agent systems
72 minutes and you'll understand agents better than most people building them.
Watch it today, then read the step by step guide for beginners on building loops below.
Delphine retweeted
If you want to get better at algorithms and data structures, read these 12 articles: ↓
Delphine retweeted
Sub-Agents vs Agent Teams in Claude Code:
Sub-agents get their own system prompt, their own tool set, and a clean context window. They report back to the parent and terminate.
Agent teams get all of that plus three things sub-agents don't have:
- a shared task list with dependency tracking
- peer-to-peer messaging between teammates
- persistent context that accumulates over time.
We published an article that dives into a lot more detail.
Read it below.
Delphine retweeted
Stanford just dropped a 457 page report on AI.
It's packed with data on: cost drops, efficiency, benchmarks, adoption.
This report is a cheat code for your career in 2026.
I pulled the most important charts + what they mean for your career: 🧵
Delphine retweeted
Google has published a paper that might end the transformer era.
For the last 7 years, every major AI, ChatGPT, Claude, Gemini, has been built on the exact same architecture: The Transformer.
But Transformers have a fatal flaw.
To remember context, they have to process every single word against every other word. It’s called quadratic complexity. As your prompt gets longer, the compute cost explodes.
The alternative is the old-school RNN (Recurrent Neural Network). RNNs are incredibly cheap and fast, but they have a fixed memory size. If you give them a long document, they get amnesia.
Until today.
Google researchers published Memory Caching: RNNs with Growing Memory.
And it fixes the biggest bottleneck in AI.
Instead of an RNN having a fixed, rigid memory that constantly overwrites itself, Google gave it a "save" button.
The technique allows the RNN to cache checkpoints of its hidden states as it reads.
The memory capacity of the RNN can now dynamically grow as the sequence gets longer.
They built four different variants, including sparse selective mechanisms where the AI actively chooses exactly which checkpoints matter most.
The results rewrite the rules of efficiency.
On long-context understanding and recall-intensive tasks, these new Memory-Cached RNNs closed the gap with Transformers.
They achieved competitive accuracy without the explosive, quadratic compute cost. It perfectly bridges the gap between the cheap efficiency of an RNN and the massive capability of a Transformer.
We have spent billions scaling Transformers because we thought they were the only way an AI could remember a long conversation.
But Google just proved we don't need to process the whole history every single time.
We just needed a smarter cache.
Delphine retweeted
I wrote Deep Learning with Python to be the definitive guide to how deep learning works and how to best make use of it. Tens of thousands of people got their career start via this book. 120,000 copies sold, and downloaded by millions more.
And now it's free to read online: deeplearningwithpython.io/
Delphine retweeted
MCP and A2A are both agent protocols but they operate at completely different layers.
MCP (Model Context Protocol) is about giving one LLM access to external tools. The model stays in the driver's seat throughout: it receives your query, decides which tools to call, gets the results back through an MCP Client, and assembles the final response. The MCP Servers in between are just standardized wrappers around raw APIs — Flight Booking, Google Calendar, whatever you need. One brain. Centralized control.
A2A (Agent-to-Agent Protocol) is about coordination between autonomous agents. There's no single LLM managing everything — instead, an Orchestrator Agent delegates subtasks to specialized agents, each of which has its own tools, its own memory, and its own reasoning loop. The Orchestrator never touches the Flight API directly. It talks to a Flight Agent, which handles that domain entirely on its own, then reports back.
The architectural difference matters when you're building:
MCP works well when one model can reasonably handle the full task with tool access. A2A becomes necessary when subtasks are complex enough to need dedicated reasoning — or when you want agents that can operate in parallel without bottlenecking through a single model.
Same scenario — "book me a flight and add it to my calendar" — but one architecture keeps control centralized, the other distributes it across specialists.
If you're building agentic systems in 2026, understanding where this boundary sits will shape every design decision you make.
#agenticai #aiarchitecture #mcp #google #anthropic #aiengineering #multiagentsystems
Delphine retweeted
AI-native software engineering teams operate very differently than traditional teams. The obvious difference is that AI-native teams use coding agents to build products much faster, but this leads to many other changes in how we operate. For example, some great engineers now play broader roles than just writing code. They are partly product managers, designers, sometimes marketers. Further, small teams who work in the same office, where they can communicate face-to-face, can move incredibly quickly.
Because we can now build fast, a greater fraction of time must be spent deciding what to build. To deal with this project-management bottleneck, some teams are pushing engineer:product manager (PM) some teams are pushing engineer:product manager (PM) ratios downward from, say, 8:1 to as low as 1:1. But we can do even better: If we have one PM who decides what to build and one engineer who builds it, the communication between them becomes a bottleneck. This is why the fastest-moving teams I see tend to have engineers who know how to do some product work (and, optionally, some PMs who know how to do some engineering work). When an engineer understands users and can make decisions on what to build and build it directly, they can execute incredibly quickly.
I’ve seen engineers successfully expand their roles to including making product decisions, and PMs expand their roles to building software. The tech industry has more engineers than PMs, but both are promising paths. If you are an engineer, you’ll find it useful to learn some product management skills, and if you’re a PM, please learn to build!
Looking beyond the product-management bottleneck, I also see bottlenecks in design, marketing, legal compliance, and much more. When we speed up coding 10x or 100x, everything else becomes slow in comparison. For example, some of my teams have built great features so quickly that the marketing organization was left scrambling to figure out how to communicate them to users — a marketing bottleneck. Or when a team can build software in a day that the legal department needs a week to review, that’s a legal compliance bottleneck. In this way, agentic coding isn’t just changing the workflow of software engineering, it’s also changing all the teams around it.
When smaller, AI-enabled teams can get more done, generalists excel. Traditional companies need to pull together people from many specialties — engineering, product management, design, marketing, legal, etc. — to execute projects and create value. This has resulted in large teams of specialists who work together. But if a team of 2 persons is to get work done that require 5 different specialities, then some of those individuals must play roles outside a single speciality. In some small teams, individuals do have deep specializations. For example, one might be a great engineer and another a great PM. But they also understand the other key functions needed to move a project forward, and can jump into thinking through other kinds of problems as needed. Of course, proficiency with AI tools is a big help, since it helps us to think through problems that involve different roles.
Even in a two-person team, to move fast, communication bottlenecks also must be minimized. This is why I value teams that work in the same location. Remote teams can perform well too, but the highest speed is achieved by having everyone in the room, able to communicate instantaneously to solve problems.
This post focuses on AI-native teams with around 2-10 persons, but not everything can be done by a small team. I'll address the coordination of larger teams in the future.
I realize these shifts to job roles are tough to navigate for many people. At the same time, I am encouraged that individuals and small teams who are willing to learn the relevant skills are now able to get far more done than was possible before. This is the golden age of learning and building!
[Original text: deeplearning.ai/the-batch/is… ]
Delphine retweeted
Is context engineering just a new name for RAG?
Not quite. But they're solving the same problem: building the right context for your LLM.
Here's how we got from one to the other — and why it matters for AI data scientists.
Delphine retweeted
Everyone assumes LLMs are the future of AI.
The permanent foundation. The layer everything else gets built on.
I’m not so sure.
The historical parallel that fits best isn’t the one most people want to hear.
LLMs are Edison’s DC power grid:
→ Genuinely revolutionary
→ Commercially dominant
→ Solving real problems right now
→ But architecturally limited in ways that can’t be patched
Right domain. Wrong architecture. And the evidence is already here.
Hallucination isn’t a bug. It’s the architecture.
Researchers have formally proven that LLMs cannot learn all computable functions and will therefore inevitably hallucinate when used as general problem solvers.
That’s not a training data problem. That’s math.
A separate paper demonstrated that hallucinations stem from the fundamental mathematical and logical structure of LLMs, making it impossible to eliminate them through architectural improvements, dataset enhancements, or fact-checking mechanisms.
And here’s the part that really gets you:
There’s a direct link between hallucination and creativity in LLMs.
It may be impossible to eliminate hallucination without impairing the model’s most crucial capabilities.
→ The thing that makes LLMs creative is the same thing that makes them lie
→ Fix one, you break the other
→ That’s not a tradeoff you engineer away. That’s a design constraint.
DC power had the exact same structural problem. It couldn’t transmit electricity over long distances.
Not because the engineering was bad. Because the physics made it impossible.
You needed AC. A fundamentally different approach.
The “AC power” of AI is already being built. And it has names.
This isn’t theoretical. People are already building the replacement architectures.
Yann LeCun left Meta and raised $1 billion to prove LLMs are a dead end.
AMI Labs raised $1.03 billion in seed funding at a $3.5 billion valuation in March 2026, making it the largest seed round in European history.
His thesis is simple: LLMs predict the next word. That’s not intelligence. That’s autocomplete at scale.
His core technology, JEPA, operates in latent space, learning abstract representations of reality rather than surface patterns.
LeCun used a vivid analogy: using an LLM to understand the real world is like teaching someone to drive by just talking.
A Turing Award winner didn’t just write a paper about it. He quit his job and bet a billion dollars on it.
Mamba is proving transformers aren’t the only game in town.
Mamba achieves 5x higher throughput than Transformers with linear scaling in sequence length.
Thanks to intensive research in 2023-2025, non-transformer architectures have reached parity with Transformers on key language benchmarks, and in some cases surpassed them.
Hybrid architectures are already shipping.
By 2026, models built on hybrid transformer-SSM architectures can ingest hundreds of pages of text at once, far beyond vanilla GPT-3 or GPT-4.
The alternatives aren’t coming. They’re here.
Meanwhile, look at what the industry is building to keep LLMs functional:
→ Agents (because the model can’t verify its own outputs)
→ Tool use (because the model can’t interact with the real world)
→ Reasoning chains (because the model can’t reason natively)
→ RAG (because the model can’t reliably recall facts)
These aren’t features. These are workarounds.
When you need that many patches, you’re running longer DC power lines and wondering why the voltage keeps dropping.
Now the part everyone actually needs: which skills survive the transition?
When DC shifted to AC, some electrical engineers thrived and some went extinct.
The ones who thrived understood circuits, load management, and power distribution at a fundamental level. Those principles worked on any architecture.
The ones who didn’t? They only knew DC-specific wiring.
The same split is coming. And it’s coming faster than people think.
Here are the skills that transfer no matter what replaces transformers:
→ Systems thinking for AI workflows. Breaking complex tasks into steps an AI can execute. This works whether the AI is a transformer, an SSM, JEPA, or something we haven’t built yet. Architectures change. The need for structured task decomposition doesn’t.
→ Evaluation and verification. Knowing if AI output is right. LLMs have a “Self-Correction Blind Spot” where they can recognize errors but lack the reasoning pathways to correct them.  Whatever comes next will still need humans who can evaluate quality. This skill gets MORE valuable, not less.
→ Data literacy. Understanding what data an AI needs, how to structure it, what’s clean vs. noisy. Every AI architecture runs on data. Past, present, future. The people who understand data will always have leverage.
→ AI-augmented workflow design. Not “how to write a good prompt” but “how to redesign a business process so AI handles the right parts and humans handle the right parts.” This is architecture-agnostic. It transfers to anything.
→ Domain expertise + AI fluency. The most powerful combination is stacking AI fluency on top of deep domain expertise.  A lawyer who understands AI beats a prompt engineer who doesn’t understand law. Every time. Regardless of what model they’re using.
→ Clear problem definition. Prompt engineering is just one implementation of a deeper skill: translating human intent into machine-executable instructions. Whether that instruction is a prompt, an API call, a config file, or something that doesn’t exist yet, the ability to define what you want is permanent.
And here’s what DOESN’T transfer:
→ Memorizing specific model behaviors (“Claude does X, GPT does Y”)
→ Platform-specific tricks that only work on one tool
→ Building your identity around a single product name
→ “Prompt engineer” as a job title instead of a thinking skill
The difference is simple:
→ Transferable skills = understanding WHY something works
→ Non-transferable skills = memorizing HOW a specific tool works
WHY survives paradigm shifts. HOW doesn’t.
The bottom line
The principle behind LLMs is permanent. The architecture probably isn’t.
That’s not bearish on AI. That’s the most bullish take possible. It means the best is still ahead of us.
Use LLMs hard right now. Build with them. Ship on them.
But build your skills around the PRINCIPLES, not the PRODUCTS:
→ Learn systems thinking, not just prompting
→ Learn evaluation, not just generation
→ Learn data literacy, not just tool literacy
→ Learn workflow design, not just model tricks
→ Stack domain expertise on top of AI fluency
The people who do this will thrive in the transformer era AND whatever comes after it.
Edison built a working power grid that lit up Manhattan. It was real, valuable, and changed the world.
AC still replaced it.






































