Dev Patel retweeted
Dear President Elon,
Apologies for not realizing you’re the real boss of America.
Would you kindly allow BYD to compete with Tesla? Millions of Americans deserve affordable EVs.
Surely competition is good for everyone, unless it’s against you.
Thank you 🙏
Dear Prime Minister Ambani,
Please accept my humble apologies for not realizing that you are the real boss of India.
Naturally, you would prefer to maintain your monopolistic exploitation of the great people of India, but would you nonetheless consider allowing Starlink to compete?
There are many areas of India, as there are in all countries, with no Internet access, which denies children the opportunity for education and for small businesses the opportunity to sell their goods to a global market. Starlink would change their lives for the better.
Thank you 🙏
Dev Patel retweeted
I love this. Great work comes from a clear, authentic, and biased mind, which is often considered taste.
Conversely, an anxious, speed-driven mind looking at everyone else produces unoriginal, clumsy, boring work.
Convergence to the mean may be unavoidable, like the universal increase in entropy, but we must fight it, because that is the very meaning of why we work, and the very proof of how much we care about our work.
convergence to the mean
when everyone looks at everyone else to decide what to make, we eventually stop making anything of our own.
we copy what works. borrow the language, the aesthetic, the features. we call it learning from the market, following best practices, staying competitive. together, everything starts to feel familiar. polished, interchangeable. nothing wrong. nothing alive.
the mean isn’t necessarily bad. it’s where everything lands when we remove the reasons for it to be different.
numbers are easier to compare than feelings, so we make things that produce numbers. we study what gets attention, then shape ourselves around it. eventually the feedback stops informing the work and starts authoring it. we’re no longer asking what we want to say. we’re asking what will be rewarded for saying.
ai makes this loop frictionless. it can draw from more references than we could see in a lifetime. but access to every perspective isn’t the same as having one. it hasn’t lived a particular life that makes one detail unbearable and another beautiful. when we ask it to choose for us without bringing any intention of our own, we get something plausible, easy to approve.
then we ship it, measure it, copy it, and feed it back into the next round. a tidal wave of mid.
this changes not just the work, but the people making it.
we describe ourselves in categories platforms can distribute and markets understand. designer. founder. creator. pick a niche. build a brand. be consistent. the parts that don’t fit become inconvenient, even when those contradictions might produce our most interesting work. we become easier to classify and harder to tell apart.
i think we need to approach making things more like artists.
an artist develops a perspective through the work. making, noticing, choosing, failing, returning. learning what matters even when nobody else cares. craft gives that perspective a form precise enough for someone else to experience.
the life outside the work matters too. the places you’ve lived, the people you’ve loved, the things you’re embarrassed to care about. these aren’t distractions from having something to say. they’re how you come to have something to say.
we understand this with music, paintings, photographs, movies. we expect a person to be present in the choices. we want to experience something we wouldn’t have encountered without them.
software can do this too.
a piece of software expresses beliefs about how life should feel. what deserves your attention. what you should be free to change. whether a task should feel like paperwork or play. even a small interaction can carry someone’s care, humor, or curiosity.
but we rarely talk about software as a medium. we talk about it as an industry. markets to capture, workflows to optimize, competitors to catch up with. we add a feature because someone else has it, without asking whether it belongs. products accumulate the same parts and lose what made them worth choosing.
underneath the branding, it’s all the same thing.
artists learn from others too. the question is whether you’re absorbing those influences into a point of view, or using them to avoid developing one.
what are you trying to let someone feel, understand, or do? why this particular shape? what have you noticed that the existing tools don’t seem to notice?
those questions don’t have answers you can borrow from a competitor’s roadmap.
if we can make almost anything, deciding what deserves to exist becomes more important, not less. speed gives us more chances to exercise judgment. it doesn’t remove the responsibility.
i don’t want a world where every person becomes a more efficient distributor of the same ideas. i want people to use these tools to go further into what they actually care about. to make things specific enough that we can feel someone behind them.
what is the point of everyone being able to make something, if we all end up making the same thing?
Dev Patel retweeted
Just like OpenClaw and Hermes Agent, Jev is another trend on X. Yes they are helpful but not an engineering skill.
You will land a job/freelance in AI only by
- system design
- neural networks
- agent harness
- Transformers, LLMs, inference
- AWS, CI/CD, Docker
Dev Patel retweeted
goal + no deadline = fantasy
goal + deadline = objective
goal + … + plan = strategy
goal + … + consistent action = success
Dev Patel retweeted
Got a great comment today (basically) asking "is it worth it to learn how to build a coding agent in 2026 coz so many agents already exist"
The answer is quite nuanced so I thought of writing this long ass post.
I have a side business where I meet with clients for private training sessions/cohorts. Almost EVERY CLIENT came to me to solve the SAME problem. Even in my job, and my girlfriend's job, we have had to work on this exact task - "building custom harnesses".
Companies want to build custom harnesses for a narrow problem to satisfy their peculiar use-case. Token efficient, high speed, less noise, custom data sources. To my clients, I ask two questions: do you want to finetune a small SLM? and, do you want to build a minimal lightweight harness?
In truth, harnesses are REQUIRED regardless if you are finetuning an SLM, or using a proprietary model via API. There are multitudes of business use-cases where you will NEED to design (a) good tools, (b) external data sources, (c) prompts, and (d) tasksets/evals. This is unavoidable because EVERY COMPANY is building their own harnesses to solve their particular business problem. It's the new Software Engineering.
A coding agent is a specific type of agent harness - but learning to build one teaches you the fundamentals of all of the 5 things above. It covers the general use-case which you can then customize to your specific domain.
Getting good at fundamentals will always be the best thing you can do for yourself long term. We should always wanna expand our toolboxes so we have the right tools when a situation presents itself.
Dev Patel retweeted

Dev Patel@devnp2007Mar 14
India
IndiaConnected via India Android AppAccount-level information, not a live location or per-post device.
One of the best roadmap to become an AI Hardware Engineer -
Dev Patel retweeted
In 1980, there were no GPUs, no ImageNet, and no billion-parameter models.
GPT's were still science fiction.
Yet a researcher named Kunihiko Fukushima was already working on ideas that would eventually become part of the foundation of modern computer vision.
What fascinates me is that most people have never heard of him.
Today, names like Ilya Sutskever, Geoffrey Hinton, Yann LeCun, and Andrej Karpathy are often associated with major advances in AI. But decades before deep learning became mainstream, Fukushima was asking a deceptively simple question:
How does the brain recognize the same object when it appears in different positions?
Back then, computers struggled with tasks that seem trivial today. If you showed a machine the same handwritten digit in a slightly different location, it could easily fail to recognize it. Humans do not have this problem. We can identify a face, a cat, or a handwritten number regardless of where it appears in our field of view.
Fukushima became interested in how the brain solves this problem.
Instead of asking, "How can I make computers smarter?", he asked a more fundamental question:
"How does biological vision work in the first place?"
That question led him to develop the Neocognitron, a neural network architecture inspired by the visual cortex.
What made it remarkable was not just that it was a neural network, but how it was structured.
Fukushima proposed a hierarchy of layers where lower layers learned simple visual features such as edges, corners, and line orientations. Higher layers combined these simple features into increasingly complex patterns and objects.
The architecture consisted of what he called S-cells and C-cells. S-cells responded to specific local features in an image, while C-cells pooled information from nearby S-cells to make recognition more tolerant to small shifts in position.
This was a crucial idea.
A cat remains a cat whether it appears in the center of an image or slightly to the left. Humans naturally understand this, but computers struggled with it. Fukushima's architecture introduced a mechanism for achieving this kind of position invariance.
When I read the original work, what struck me was how many concepts felt familiar.
Local receptive fields.
Hierarchical feature extraction.
Alternating feature-detection and pooling stages.
Translation-invariant pattern recognition.
Multi-layer representation learning.
Today, these ideas are considered fundamental to Convolutional Neural Networks (CNNs).
In fact, long before the term "deep learning" became popular, Fukushima had already demonstrated that a network could learn increasingly abstract representations by stacking multiple layers on top of one another.
The deeper I looked into it, the more it felt like reading an early blueprint for modern computer vision.
The most fascinating part is that Fukushima achieved this in 1980.
This was before modern GPUs.
Before ImageNet.
Before large-scale datasets.
Before the computing infrastructure that would later make deep learning practical.
The core ideas were already there. The world simply had not caught up yet.
Then, more than three decades later, came one of the most important moments in AI history.
In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton introduced AlexNet and achieved a breakthrough that stunned the computer vision community. Their model dramatically outperformed competing approaches in the ImageNet competition and showed the world what deep neural networks could achieve when combined with large datasets and GPU acceleration.
That breakthrough did not appear out of nowhere.
Many of the principles behind modern convolutional neural networks can be traced back to foundations that researchers like Fukushima had laid decades earlier. AlexNet became the spark that ignited the deep learning revolution, but pioneers like Fukushima had already spent years building the groundwork that made such a breakthrough possible.
From there, progress accelerated rapidly.
Computer vision became one of the most successful applications of deep learning. The same family of ideas would go on to power medical imaging systems, autonomous vehicles, robotics, scientific research, and many of the AI systems that shape our world today.
Reading about Fukushima reminded me of something that applies far beyond AI.
We often celebrate the people who turn an idea into a revolution.
Far less attention goes to the people who first saw the possibility.
Sometimes being early looks exactly the same as being wrong.
The difference only becomes obvious years later when everyone else finally catches up.
Kunihiko Fukushima was one of those rare researchers who saw a piece of the future long before the future arrived.
Dev Patel retweeted
We’re thrilled to open-source TriAttention! 🚀
🦞 Deploy OpenClaw (32B LLM) on a single 24GB RTX 4090 locally
💻Full code open-source & vLLM-ready for one-click deployment
⚡️ 2.5× faster inference speed & 10.7× less KV cache memory usage
TriAttention is a novel KV cache compression method built on rigorous trigonometric analysis in the Pre‑RoPE space for efficient LLM long reasoning.
Github Repo: github.com/WeianMao/triatten…
Paper Link: huggingface.co/papers/2604.0…
Homepage: weianmao.github.io/tri-atten…
Dev Patel retweeted
Efficient AI Lecture 6(Quantization Part II) gave me a much more complete picture of quantization.
It connects:
- K-means quantization
- linear quantization
- binary quantization
- post-training quantization
- quantization-aware training
A detail I liked: Blackwell’s 16-element vector quantization.
The Colab is also useful for turning the ideas into hands-on practice.
My note: ickma2311.github.io/ML/HW-SW…
Dev Patel retweeted
The ASML book author saw the next generation – Lace Lithography, using helium atoms shooting through a holographic mask to scale beyond what’s possible with light, where the wavelength is larger than atomic scale.
“ASML is the only company capable of using EUV—extreme ultraviolet light—to print ultra-fine chip patterns at the 2-nanometer level desired by Musk. Last year, ASML produced 48 of these EUV machines. While there are plans to ramp up production, a rapid doubling of output is not in the cards—ASML’s suppliers simply cannot manage that pace.”
“Lace Lithography is developing a lithography machine capable of printing chip circuitry using helium atoms. ‘Where light ends, atoms begin,’ says Bodil Holst, founder of this start-up based in Bergen, Norway. The wavelength of light determines the precision with which one can 'print.' Think of it like making a tiny drawing: you would much rather use a fine-tipped pen than a blunt carpenter's pencil. EUV employs a wavelength of 13.5 nanometers and further narrows that beam using mirrors. The 'beam' of helium atoms, however, is less than one-tenth of a nanometer wide, allowing it to draw with far greater intricacy.”
“In a nutshell: Lace propels energized helium atoms—each carrying an extra electrical charge—through a mask perforated with tiny holes. The atoms that pass through unimpeded strike the photosensitive layer of a silicon wafer, thereby etching the desired pattern. That perforated mask reminded Bodil Holst of *kantklossen* (known in English as 'lace making'), which is why she named her company just that. A test rig is currently operational at the Lace laboratory in Bergen”
“The foundation for this ‘atomic approach’ dates back to the 1990s, but neither the timing nor the technology was ripe for it at the time. This is because extensive computation is required to design the perforated diffractive mask in such a way that the chip's circuitry remains accurate. This can only be achieved with the aid of AI using high-speed chips—explains Adrià Salvador Palau. ‘Consequently, without the powerful chips made by EUV machines, we would never have been able to solve this problem.’"
— Translated from the Dutch original: nrc.nl/nieuws/2026/04/03/kan…
𝐅⃣𝐎⃣𝐂⃣𝐔⃣𝐒⃣ The ASML Way
I just finished this history of the most important semiconductor equipment company in the world, as translated from the Dutch original (and lurking in the background might be a better way).
Reminder: ASML builds 100% of the world’s extreme ultraviolet (EUV) lithography machines, without which cutting edge chips are simply impossible to make. It’s the most expensive mass-produced machine tool in history. Oh, and today, there are two special women without whom, all EUV lithography would sputter to a stop (see p.141 below)
ASML was formed in 1984 as a JV with Philips, the Dutch electronics company that contributed ~$15M (in guilders) and 40 engineers, and “it seemed doomed from the start.” (p.35) There were 10 viable competitors at the time, more than enough to serve the market as ASML learned at SEMICON in 1984 (by coincidence, I was also there with my Dad who about to leave Mostek to run Varian’s Semiconductor Equipment Group, but they only had Molecular Beam Epitaxy, a low throughput lithography alternative. My Dad’s attempt to poach a CTO from ASML is on p.72).
“In these initial years, management worked around the clock to bring in new subsidies. In these initial years, about half of ASML’s money for research came from The Hague or Brussels.” (48)
ASML’s “machines were the first in the industry to utilize modular design. The lens, the wafer-table, the frame for the mask, the light source, the robot that picks the wafers: these are LEGO blocks that, when you bring them together, form a lithography system.” (62)
IPO in 1995. Stock went up 600x in the 30 years that followed. March 2000 market crash: “cancellations from chip manufacturers poured in daily. On paper, the company was bankrupt. Radical cost-cutting measures would be needed.” (82)
Nikon sues: “a rude awakening. ASML had paid far too little attention to its intellectual property in its early years.” (98) “The best inventors, some of which have more than 200 patents to their name, are commemorated by having their faces engraved on silicon wafers and hung on a series of large wooden beams, like a Mount Rushmore of the chip industry. As of 2023, ASML has registered more than 16,000 patents.” (99)
The machines are insanely sensitive. “Atmospheric pressure fluctuations due to thunderstorms can easily disrupt the lithography process. Or cows. Intel once faced an inexplicable drop in yield every night for a few hours, with researchers running in circles until they finally realized the cause: cow farts. Intel had to pay for three farms to relocate.” (117)
“In 2006 Intel, who was supplying the chips for Apple’s computers, was asked if it could also supply the processor for the iPhone. It declined.” (122)
“EUV light is extremely difficult to generate and sustain in an industrial environment. The invisible rays are absorbed by almost all materials, even the air, which means the lithography machine needs to have (curved, atomically precise) mirrors instead of lenses and can only operate in a vacuum.” (127)
The Cymer laser / light source has a molten tin “droplet generator capable of forming a 30-micron droplet of tin at a rate of 50,000 times per second. The laser was rigged to deal two separate blows. First, a gentle tap to flatten the droplet into a pancake-like shape, followed by an intense blast that heated the tin to 200,000 degrees, transforming it into a plasma.” (130)
“During its journey through the lithography machine, the light beam comes across 10 mirrors, each absorbing 30% of the light. It starts with 1.5 megawatts from the grid that yields 30 kilowatts in the laser, and that creates 100 watts of EUV light. Of this, about 1 watt ends up on the wafer. But more power also creates more heat. That causes the mirrors to expand, which in turn causes small deviations that immediately need to be corrected with small motors. Even the EUV mask, which carries the blueprint of the chip on it, is itself an extremely sensitive mirror.” (132)
“ASML was vastly underestimating the financial consequences of the new technology. In retrospect, this was for the best. No respectable CEO would sign for a project that would take 20 years, without any promise of success or interim profit to carry it through. That’s not taking a bet, that’s bananas. This is also why the Japanese competition dropped out of the race: not because their engineers were any less capable, but because Nikon and Canon were simply not prepared to continue pumping so much money into EUV.” (133)
To finance the purchase of Cymer in 2012, “Intel invested 3.3B Euros into ASML in exchange for 15% of the shares. TSMC was required to purchase 5%... and Samsung acquired a stake at the 11th hour, taking 3%.” (139)
“Only Joann and one of her colleagues have the ability to wind and solder invisibly small wires (around the nozzle that shoots the tin droplets). It’s a delicate task few could ever master. ‘Even watchmakers can’t do this,’ says their awestruck boss, ‘and there’s no way to automate it.’ It’s not a trivial matter: the nozzle regularly gets clogged during day-to-day use in the chip factory. When that inevitably happens, the only thing to do is to swap it out for a new one. It’s hard to imagine, but without the fingers of Joann and her colleague, the EUV machines at Samsung and TSMC would grind to a halt.” (141)
In 2013, “most of the droplet generator was still hand-made by Cymer, and it was virtually impossible to test the part in advance. This made for completely unpredictable yields: in the initial phase of production, half of the droplet generators didn’t even work.” (142)
“20% of the South Korean economy now relies on the revenue of one single company. Hence their nickname: this is the republic of Samsung.” (156)
“Intel was being surpassed by their competitors in Asia on every front and would only start using EUV for chips after 2023.” (160)
“The descriptions that chip manufacturers use for these technological generations or ‘nodes’ need to be taken with a grain of salt. The physical dimensions of the smallest circuits and connections on the chip are, in practice, 5 to 10 times larger than advertised. A nanometer was once a nanometer, but accuracy has never stopped a good marketing slogan.” (161)
Cousins “Lisa Su and Jensen Huang, the leaders of AMD and NVIDIA were both born in Tainan, the city where TSMC now produces their chips.” (164)
“The culture at TSMC is more hierarchical than ASML, but less militaristic than in South Korea.” (166)
“TSMC now commands 60% of the entire foundry market, making it 4x larger than its closest competitor, Samsung.” (167)
“ASML’s next generation of EUV machines goes by the nickname High NA (the numerical aperture increases from 0.35 to 0.55). These colossal scanners span 14 meters and feature large mirrors up to a meter wide. The optical system by itself consists of 20,000 parts and weighs 12 tons, making it 7x heavier than the optics for the current EUV machine.” (175)
“The High NA system weighs 150 tons and costs 400M Euros. It takes 7 cargo planes to ship this system to customers.” (225)
“The production of a complex EUV mask costs more than a half million Euros and takes a huge amount of time to calculate.” (181) They “use AI to understand the interplay between the light beam, the mask, and the chemical reactions on the wafer.” ASML’s CTO calls it “voodoo software.” (183)
China: “European governments fear China is transforming into a totalitarian state, capable of forcing Chinese multinationals to spy for the Communist Party. And that poses significant risk to the 5G cellular infrastructure of the West.” (200)
“In 2017, Chinese customers ordered 700M Euros worth of lithography machines, a new record. Hundreds of ASML’s scanners were running in the factories of SMIC, China’s largest foundry” (201)
“EUV is controlled by the Wassenaar Arrangement, the multilateral export control regime on conventional arms and dual-use goods and technologies.” (203)
“As far as ASML is concerned, fears about EUV being used for military applications are baloney. Most chips found in weapons are ‘off-the-shelf’ chips that can also be found in laptops, washing machines or cars, and are easy to purchase anywhere in the world. But the U.S. sees things differently. They fear the emergence of Chinese AI and cyber weapons. And there is one thing those all need: advanced chips.” (205)
“In January 2020, the U.S. asked the Netherlands to block EUV exports, and suddenly ASML found itself in the spotlight. The Netherlands ultimately denied ASML a license… No EUV machine was going to SMIC.” (208)
In 2023 “ASML was exporting far more older DUV machines to China than had been expected. Almost half of ASML’s revenue was coming from China. As the chip industry was pushing the pause button, China kept on hoarding. The U.S. pressed the Netherlands to slam the brakes before January 2024, and the cabinet duly revoked several approved export licenses for ASML machines destined for China.” (234)
“As China is growing increasingly isolated, so too is the liklihood of a fully-fledged Chinese competitor emerging in the rearview mirror capable of developing an independent chip production chain.” (236) “ASML takes this seriously. Their go-to response: ‘The laws of nature are the same anywhere.’ What was achieved in Brabant, could be achieved in Beijing.” (335)
“To qualify for government aid (in Biden’s Chips Act), companies had to agree not to build advanced chip foundries in China or other ‘countries of concern.’” (239)
“The chip shortage had been a wakeup call, and the nightmare scenario was front and center on everyone’s mind: if China blocks Taiwan, we’ll be without chips within two weeks.” (242)
“The estimated percentage of people with autism or ADHD at ASML far outnumbers the average. The highly specialized work, revolving around focusing on complex problems that require prolonged attention to the smallest details, makes it well-suited for some autistic traits. ASML’s CTO and President Van den Brink makes no secret about being dyslexic and actively advocates for targeting this neurodiverse group. They are precisely the analytical and creative thinkers ASML needs, but also often the ones who find it difficult to put themselves in other people’s shoes.” (287)
Sounds like teen spirit… of Steve Jobs: “Van den Brink’s power of persuasion lies in his childlike enthusiasm. It works like some kind of reality distortion field. Martin can disrupt your perspective until you’re convinced that you can make the impossible possible.” (321)
“Van den Brink never really led a big company. He guided it like a startup, as if it were a defiant toddler in the body of a mature multinational.” (329)
The book ends with the poignant handover of the company in 2024 to a new leader, the Frenchman Chistophe Fouquet.
Dev Patel retweeted
1/ 🪩 Automating the discovery of new algorithms could unlock significant breakthroughs in ML research. But optimising agents for this research has been limited by too few tasks to learn from!
Introducing DiscoGen, a procedural generator of algorithm discovery tasks 🧵
Dev Patel retweeted
🚨 Electrical engineers are going to hate this.
Someone just turned React into a circuit board factory. Write code. Get a real PCB manufactured and delivered to your door.
It's called tscircuit. React for Electronics.
No Altium. No $10,000/year licenses. No 6-month learning curve.
You write React components. But instead of
and
Dev Patel retweeted
My wife (Ria) went to AIIMS. I went to IIT Delhi.
We watched 'Hello Bachhon' on Netflix this weekend. One kid in the show breaks down walls in his village to sell bricks so he can afford coaching fees. A girl hides her textbooks because her parents want her married off.
Throughout the show, Ria and I realized that we never had to fight for any of that.
We had access to coaching. Good coaching. Neither of us ever worried about whether we'd get to prepare.
The IIT-AIIMS crowd loves talking about the grind. # of hours of self-study per day, the pressure, and sleepless nights. I've done it too. But that grind starts after you already have access. Most of us skipped the actual hard part and don't even know it because the people who are capable but can't afford access are invisible. They're not in your coaching batch. They're not in your college. You never compete against them because they never got the chance to show up.
We credit hard work more than it deserves. Sometimes the biggest advantage was just being born in the right city to the right family.
'Hello Bachhon' made this truth hard to ignore. Great watch.
Dev Patel retweeted
A quarter million dollars in GPUs is the entry ticket to “democratized” AI training.
Each participant needed a minimum of 8x NVIDIA B200 GPUs. “Anyone with GPUs could join” is technically true the same way “anyone can buy a Gulfstream” is technically true. Twenty-plus contributors scattered across the globe just finished training a 72 billion parameter model on 1.1 trillion tokens, coordinated over commodity internet, with no central cluster and no whitelist.
The engineering is legitimately impressive. They compressed gradient communication by 146x using SparseLoCo, let participants join and leave mid-run without killing the training process, and used a blockchain incentive layer (Bittensor subnet 3) to pay contributors in TAO tokens for staying honest and keeping machines running. Six months ago the largest permissionless decentralized run was INTELLECT-1 at 10B parameters. Covenant jumped to 72B. That’s a 7x parameter leap while removing the trust assumption entirely.
Now the context that matters. GPT-4 trained on roughly 25,000 A100s in a dedicated cluster estimated at $500 million in infrastructure. OpenAI’s total compute bill hit $5 billion in 2024. The industry is racing toward $100 billion clusters by 2028. Covenant-72B benchmarks “competitively with centralized models at similar scale,” which means competitive with LLaMA-2-70B, a model Meta released in July 2023 using a conventional data center.
The 72B number sounds massive until you remember frontier models are now in the trillions of parameters, trained on 10-15x more tokens, with post-training pipelines that cost as much as the pre-training itself.
What Covenant proved: distributed training over the open internet works at a scale that would have been unthinkable two years ago. The optimizer is brilliant. The infrastructure leap is real. And “permissionless” still costs a quarter million dollars at the door.
Dev Patel retweeted

Sebastian Raschka@rasbtMar 7
United States
United StatesConnected via United States App StoreAccount-level information, not a live location or per-post device.
While waiting for DeepSeek V4 we got two very strong open-weight LLMs from India yesterday.
There are two size flavors, Sarvam 30B and Sarvam 105B model (both reasoning models).
Interestingly, the smaller 30B model uses “classic” Grouped Query Attention (GQA), whereas the larger 105B variant switched to DeepSeek-style Multi-Head Latent Attention (MLA).
As I wrote about in my analyses before, both are popular attention variants to reduce KV cache size (the longer the context, the more you save compared to regular attention).
MLA is more complicated to implement, but it can give you better modeling performance if we go by the ablation studies in the 2024 DeepSeek V2 paper (as far as I know, this is still the most recent apples-to-apples comparison).
Speaking of modeling performance, the 105B model is on par with LLMs of similar size: gpt-oss 120B and Qwen3-Next (80B). Sarvam is better on some tasks and worse on others, but roughly the same on average.
It’s not the strongest coder in SWE-Bench Verified terms, but it is surprisingly good at agentic reasoning and task completion (Tau2). It’s even better than Deepseek R1 0528.
Considering the smaller Sarvam 30B, the perhaps most comparable model to the 30B model is Nemotron 3 Nano 30B, which is slightly ahead in coding per SWE-Bench Verified and agentic reasoning (Tau2) but slightly worse in some other aspects (Live Code Bench v6, BrowseComp).
Unfortunately, Qwen3-30B-A3B is missing in the benchmarks, which is, as far as I know, is the most popular model of that size class. Interestingly, though, the Sarvam team compared their 30B model to Qwen3-30B-A3B on a computational performance analysis, where they found that Sarvam gets 20-40% more tokens/sec throughput compared to Qwen3 due to code and kernel optimizations.
Anyways, one thing that is not captured by the benchmarks above is Sarvam’s good performance on Indian languages. According to a judge model, the Sarvam team found that their model is preferred 90% of the time compared to others when it comes to Indian texts. (Since they built and trained the tokenizer from scratch as well, Sarvam also comes with a 4 times higher token efficiency on Indian languages.
📢 Open-sourcing the Sarvam 30B and 105B models! Trained from scratch with all data, model research and inference optimisation done in-house, these models punch above their weight in most global benchmarks plus excel in Indian languages.
Get the weights at Hugging Face and AIKosh. Thanks to the good folks at SGLang for day 0 support, vLLM support coming soon. Links, benchmark scores, examples, and more in our blog - sarvam.ai/blogs/sarvam-30b-1…
































