coffee, design, software, rinse & repeat. cto @trypallet. past: @askherald (YC W19) @quantcast @nytgraphics, @newyorker, @columbia. forever: @BAPS love.
New York, NY
Joined March 2009
- Tweets4.4K
- Following1.3K
- Followers1.1K
- Likes1.3K
Nilkanth Patel retweeted
Im in Jev heaven (Jevean?)
At Pallet we're building automations for many of the world's largest enterprise logistics companies.
On a customer classification experiment, I compared
Jev Only vs Hybrid Jev LLM vs prod LLM data
The results are INSANE
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution
Nilkanth Patel retweeted
The "whiteboard defense:" I should be able to pull you aside at any moment and ask you to explain any customer-facing system you've shipped. You should be able to clearly explain how it works and defend the decisions you made. This is my benchmark for responsible AI usage.
I don't expect line-level familiarity with the code. I don't care if you remember the exact function name or implementation detail. You may not even know it. I don't care.
But if I ask "why did you do X instead of Y?", "what happens if this actor behaves maliciously?", "what data structure did you use here and why?", or "where does this fail?" you should be able to answer confidently.
For PoCs, demos, experiments, whatever: I don't care. Generate 100% of it and understand none of it. Speed over quality every time in those specific scenarios.
But if you're shipping customer-facing work, you can't be shipping things you don't understand at a high level.
I joined Pallet because of @SushanthRaman's conviction. Starting over when the business is growing and almost no one agrees with you is a hard call to make. He made it because he believed we could build something much bigger. Glad he’s sharing this story.
Two years ago, @TryPallet was growing fast, but I decided we needed to start over. It was a period that I haven't shared much about until now.
Deep down, I knew we had built a good business, but not an outlier company. Almost no one agreed. My teammates lost faith and left. One investor even threatened to sue us. At the time, it felt like the riskiest decision I could make.
Looking back, staying the course would have meant optimizing a business I knew had a ceiling.
In the first year after the pivot, we went from $0 to $10M+ ARR and started working with some of the largest companies in the physical economy. To every teammate, and to @kevinzhang and @BainCapVC: thank you for the work and conviction that got us through our darkest hour.
Nilkanth Patel retweeted
Two million shipments a year at 99% accuracy is a distributed systems problem wearing an AI costume.
Under the hood: typed, functional, event-driven plumbing. Immutable versions. Every shipment is data, every side effect is a liability.
Boring on purpose. Proud of this one.
We're proud to partner with @onelineage, the world's largest temperature-controlled logistics company.
Today, @trypallet AI agents process two million of their shipments a year at 99% accuracy, helping them scale without compromising the high-touch service they're known for.
One thing I often tell people is that the only two time horizons we should care about are: 1 month and 5 years (which maybe should be updated to 2 years in the age of AI)
we’re getting warmer…
OWN YOUR INTELLIGENCE
Last year, building on open-weight models was primarily a cost rationalization exercise. Slightly worse performance for a much cheaper price.
Now, it is increasingly an existential and strategic topic for our portfolio. Intelligence is the product. Companies want to shape it and own it and let it compound within their own walls. Not your weights, not your product.
Now, with frontier open-weight models and fantastic tooling/infrastructure, owning your intelligence at the frontier is finally becoming possible.
The result: every application company we work with is embarking on the journey of doing their own research on post-training, evals, harnesses, etc. The hottest neolabs may just be @Harvey, @FactoryAI, @Ramp, etc. The list goes on.
We held a summit @sequoia to convene our portfolio on this topic, together with
@gabepereyra (@Harvey) on building Harvey Labs,
@lqiao (@FireworksAI_HQ) on post-training,
@hwchase17 (@LangChain) on harnesses + evals,
@BrendanFoody (@mercor) on RL environments and synthetic data,
@QuantumArjun (@trajectorylabs) on online continual learning.
Opening talk below; rest to come this week!
00:00 What is sovereign AI (and what it isn't)
01:24 Centralized vs. decentralized intelligence
02:54 Four reasons companies own their models: cost, speed, performance, destiny
04:22 "Not your weights, not your product"
05:32 The application companies are the newest neo labs
07:05 Step 1: Deciding what to own vs. rent
09:51 Step 2: Build the team (and don't shoehorn your platform team)
11:17 Step 3: Legibility – why your research has to be visible
12:33 Step 4: The technical roadmap
13:56 The stack: production vs. development
15:16 Opening Pandora's box – base models, harnesses, context
I wonder what Jeff Dean born 20 years later would look like. I’m sure there are plenty of folks with the same depth but I feel like today’s startup monoculture discourages people who like to go deep and solve problems down to the “bits.” We like confuse “agency” with “shallow but quick.”
Tomorrow will be my last day at Google after 27 years, and watching it grow from 25 people to 190,000+ has been an amazing journey. Below is a note I shared with many people internally at Google today. An excerpt is:
It has been an absolute pleasure to work with you and to help build some of the most widely used and impactful products of all time. As a kid, I dreamed of helping build software that would be used by many people, and Google now has thirteen products used by more than a billion people (amazing!). Our work has had a tremendous impact in the world, and I have been lucky enough to collaborate and form friendships with many colleagues that I deeply admire, respect, and enjoy. It still brings me joy every time I see people out in the world using our products to find information, handle email, translate documents, watch videos, learn new things, navigate and understand the physical world, browse the web, use their phone, run large-scale computations on our infrastructure, ride in an autonomous vehicle, or perform complex tasks with the help of our AI systems. I hope you all share this sense of joy, because it is a shared accomplishment! Thank you to all of my colleagues at Google over many years!
Now I'm excited to go start @DiscoLoopAI with my longtime friends and colleagues @Sanjay_Ghemawat, @OriolVinyalsML, and @quocleix.
(Updated post: slightly redacted to not have some personal info)
boy if i was chairman of the openai board this would be music to my ears
Rent the intelligence, own the relationship - how Sierra's Context Engine helps you build a competitive advantage in AI sierra.ai/blog/context-engin…
Getting PG blessup means you've made it @dakshgup
Supply chains broke my mental model of enterprise software.
I'd spent years thinking enterprise software was about helping a company optimize its own business, then I started building for supply chains and realized there isn't a single owner of the system you're trying to optimize. Decisions depend on suppliers, carriers, customers, ports, weather, tariffs, and events no one controls.
Before you can optimize a supply chain, you first need to understand how decisions ripple across the entire network, so we built Atlas to see the end-to-end picture.
Check out @vidhur2k's deep dive.
Nilkanth Patel retweeted
Startups often conflate FDE with just being close to customers in the early days.
Being forward-deployed when you're starting out makes a lot of sense because you get signal on what to build.
FDE (in the original Palantir sense) exists to bridge the last mile in a $20m contract because the customer has bespoke needs and environments.
Unless you are closing those contracts, FDE should not be a crutch to avoid doing the hard work of building a scalable product.
Our engineers have always been very customer-facing, but something that helped us is truly viewing our offering as product-driven (as opposed to services or FDE) because we think that is the correct solution for our space.
That forced us to skip some shortcuts but has paid off so far.
I've never seen startups spend so much time looking sideways.
One AI company builds forward deployed engineering and then everyone decides they need an FDE team too. One company pays a certain number for talent, and suddenly that's the market, whether or not companies can afford it. One company juices a launch with influencer tweets that read like the same intern wrote all of them, and suddenly that's the playbook.
Those choices came out of constraints you can't see from the outside. Forward deployed engineering works when your contracts are seven figures and integration takes 6+ months. It's unlikely a fit it you're selling a product at $20/seat. The comp number works when one hire moves your model quality. Otherwise, it can be a pretty fast way to run out of money.
Founder mode (the version I believe in) is about refusing to outsource your judgment. Being able to say "I've thought about it, we're not doing that," without needing a competitor's decision to drive your own.
Why spend years and your best engineers building a worse version of someone else's company?
Run yours.
one thing we underestimated going into this project was how much of the work had nothing to do with the model itself. most of the gains came from building the right evaluation loop, curating production traces, and deciding exactly what behaviors the model should learn.
that's the part we break down here.
We've seen our AI token spend skyrocket 10x over the past year as more customer workflows move into production.
At some point, inference spend crossed a threshold where it no longer made sense. That's what led us to train our first production model.
Here's what we learned:
– The financial break-even point between training your own models and using frontier models is around $750/day.
– The dense 27B model beats a MoE model with more parameters. Smaller memory footprint left more room for long contexts and KV cache. When memory is the bottleneck, fewer total parameters beat fewer active ones.
– The single biggest improvement came from label taxonomy. About 18% of our production traces were assigned to a catch-all "I can't process this" label. The model learned to rely on that fallback more than it should. Removing that single label increased accuracy from 84% to 95%+.
sometimes the true value of AI is eliminating the need to interact with shitty software, not eliminating the software itself.
supply chain is just a cesspool of aforementioned shitty software and it will take eons to get rid of it all. but in the mean time, we can 10x the experience by not forcing folks to need it 8 hours/day just to do their job.
Today, we're expanding @trypallet to operate inside Remote Desktop environments.
Many of the systems enterprises rely on daily were never designed for modern APIs, leaving critical workflows locked behind virtual desktops and manual processes.
Pallet AI agents can now securely log in, navigate interfaces, submit transactions, and complete workflows within these environments, without requiring a rip-and-replace.
After bringing native AI support to AS/400, we're now extending that same capability to another critical part of the enterprise stack that's historically been difficult to reach.
A month ago we were hiring Gen Zers who lived in Claude and tokenmaxxed everything. This month we're searching for model (née "eng") managers. You'd never hand a junior a cross-cutting refactor, but renames and feature flags, sure. The newest non-negotiable is the ability to understand how to allocate the work to our AI overlords...which model, which harness, which job, at what cost.