low latency guy
San Francisco, CA
Joined June 2009
- Tweets4.1K
- Following2.2K
- Followers182
- Likes10.4K
Recently I interviewed a Senior Engineer.
Strong candidate.
Kafka? ✅
Redis? ✅
Microservices? ✅
Distributed systems? ✅
Then I asked:
"You're building the inventory service for a flash sale.
Only 1 item left in stock."
Two users hit “Buy Now” at the exact same millisecond.
Both requests pass the “is available?” check.
Both create an order.
Both deduct stock.
Now you’ve sold 2 items when only 1 existed.
How do you guarantee you never oversell — even with concurrent requests, client retries, and partial failures?
"Just use a database transaction."
Doesn’t fully solve it if both requests read `stock = 1` before either writes, or if the response is lost and the client retries.
"Check stock before deducting."
What happens when both checks happen at the same time?
"Use Redis DECR and reject if it goes negative."
What happens if Redis is down, or the decrement succeeds but the order is never persisted?
Your turn:
How would you design this inventory reservation API so concurrent buyers and retries never oversell the last item?
There’s a specific distributed-systems concept behind this.
And you’ll encounter it everywhere inventory, tickets, seats, coupons, and limited resources are involved.
Code and sleep retweeted
🚨 @NVIDIAAI just quietly dropped its own local inference server...
No announcement. Single-digit stars
It's called Flare and Apache 2.0
Built for SM120/SM121 GPUs RTX PRO 6000 Blackwell and DGX Spark
>PyTorch + FlashInfer at build time
>Rust runtime at serve time. No Python.
>CUDA graphs captured before serving
>FP8 KV cache by default
>Recipes for Qwen Flash, GLM and DeepSeek
github.com/NVIDIA/flare-loca…
Used this workflow to rebuild a substantial GitHub product from scratch in 5 days. 800 commits, ~$15k in tokens.
Nested orchestrator/worker pools. Strong models coordinate, cheap models implement, hard decisions escalate. Runs autonomously for days.
gist.github.com/EvanBoyle/8c…
We ditched Devin, Gemini, Greptile for code reviews, since Claude and codex can do same things cheaper and/or better. And their capabilities are included into the subscription.
The velocity of code changes for engineers this year has 10x’d, quickly moving the bottleneck to code review and testing.
Greptile solves this for the most technically challenging codebases like Nvidia (core CUDA code is reviewed on Greptile), Coinbase, Datadog, OpenClaw and 22,000 more customers.
Many people think the model layer will eat code validation, but I don’t think it will. Code review within organizations is a more context-bound task than an intelligence-bound. The most intelligent AI can be wrong because it breaks an assumption in one API endpoint another service relies on. Large codebases can quickly accumulate a lot of technical debt your coding harness won’t reliably remember every time. Greptile does.
Instead of having a frontier model harness review your code which can quickly cost $20+/review, Greptile does a much better job for much cheaper, leveraging the smartest models when needed but the GLM/Kimi/DeepSeeks when needed to. And it also runs, tests and takes a video of what your change did.
Over time, organizations treat a 5/5 rated change by Greptile as a guarantee this won’t break anything in prod. And it shows. Their growth has been off the charts.
Incredibly fortunate to partner with this a company.
Fun fact: Elon once posted Greptile's growth graph with "it's so over" pointing to 2024 to show AI's acceleration.
Greptile has raised a $50M Series B led by Menlo Ventures. The round values the company at $750M.
Greptile reviews and tests more than 1% of all PRs opened on GitHub every day. Thousands of engineering teams including Nvidia, Coinbase, Decagon, and Vanta use Greptile as their independent and central validation layer.
Fewer than 50 people, we are a small, talent-dense team of engineers and researchers in a quiet corner of San Francisco. If you’re interested in joining us, we’d love to hear from you.
My mind is blown by how far ahead Apple is with local LLMs. The new Mac Studio M5 Ultra will have a 512 GB model for around 15-20k. You can chain 6 of them together using thunderbolt 5 and this have over 3 TB of ram and a massive amount of CPU and GPU cores. This means you can run a frontier model with over 2 trillion parameters locally for around 100kish. Nivida is around 500k for the same capability.
Code and sleep retweeted
Коллеги из Смартли некоторое время использовали домен smartly.af (читается «smartly as fuck», «дохуя умный») для внутреннего artifactory, но потом решили перестать поддерживать экономику Афганистана :-)
If, on the other hand, you want to get the job done at a reasonable cost, use appropriate model, effort level, and find the fastest way to get things done. This will most likely involve ai and token usage, but judge yourself on work done, not tokens consumed.
If you get such a message from your boss, Claude and Codex have your back!
Turn effort level to Max first. This will slow things down, and the bill will rise, so will your token usage.
Next, Claude has a secret cheat: Ultracode. It increases token usage 2-3x easily. Do everything with Ultracode On.
Stupid rules - stupid measures.
If you get such a message from your boss, Claude and Codex have your back!
Turn effort level to Max first. This will slow things down, and the bill will rise, so will your token usage.
Next, Claude has a secret cheat: Ultracode. It increases token usage 2-3x easily. Do everything with Ultracode On.
Stupid rules - stupid measures.
A frontend dev was telling me he's used to technology changing every year (or more often), so the pace of change with AI is not new, but he wouldn't mind retiring b/c it's getting tiring
Then again I know (early) retired ppl who came back, because it finally looks v exciting!
Grass greener on the other side?
Does anyone else have issues with Logitech Options+ on their computer (Mac, in this case)?
I just used TMOG to diagnose almost constant disk activity on my machine... I sorted the Process column by disk I/O, and it stood out immediately.
I then asked ChatGPT to audit the disk I/O, and it confirmed it. I've killed and restarted it, but is this normal? The Logitech app seems to be in loop of checking digital certs...
tmog.org
Don't let billionaires convince you slop is the future. They're used to managing & directing, not making things themselves. Of course they can't see the difference. They don't understand your love for the craft. Keep making personal, beautiful games.
You've got my support 💖
This is bullshit, Kenney. I’m the same tech tree following optimist who moved from C to C++, from 2D to 3D, and from closed source to open source. I run a company. I write code. I contribute to published papers. I’ve upped my expectations of AI. Up yours!
LLMs are either not capable or not affordable for visual reasoning tasks at scale.
The Elorian team - one of Jeff Dean’s best reports who led Gemini, an Apple LLM lead, xAI lead, and a Harvard professor - just launched an Astra-level visual reasoning model, Glimpse, for 17x cheaper.
Watch it break down the flying Tesla video physics here.
Extremely critical for video inputs in manufacturing, robotics, gaming, content moderation and more.
Следую практикам responsible AI и не генерирую пул реквесты по 200k+ строк кода.
Обычно я делаю цепочку изменений — стек пул реквестов — в каждом PR обычно не больше 1000 изменений. И таких PR 2-4 в день.
Конечно, можно было бы уйти в полный yolo-вайбкодинг, но я пока нахожусь в мире, где люди должны владеть и понимать полученный код.
Так что AI генерированный код по моему запросу я все-таки еще и сам читаю, прежде чем другим отдавать
Мне было сегодня лет когда я узнал, что можно добавить /cdn-cgi/trace к любому домену который проксируется через coudflare и получить всякую метадату. Например xitter.cf/cdn-cgi/trace
