AI Architect & Builder - Inference, LLMs, SLMs GSoC Org Admin - Jenkins | Docker Captain | CNCF Ambassador
New Delhi
Joined December 2018
- Tweets7K
- Following1.3K
- Followers8.3K
- Likes36.9K
What an amazing day! Presented a poster at @PyTorch Conference: how we optimised the RLM paper with pre-fix caching & batched Sub calls with @vllm_project
And a full house for our Pytorch Conference talk on running LLMs using Executorch on Android!
Shivay Lamba retweeted
Theo - t3.gg@theo
3hUnited States
United StatesConnected via United States App StoreAccount-level information, not a live location or per-post device.
Dev Day announcement overview, as short as reasonably possible.
- Dots: personal assistant (similar to GrokBot, Muse etc)
- $500/month plan including Ultrafast
- GPT-6.1 Sol: really smart and cheap, way better than 6 was
- Decisions API: Jev killer with vision capabilities
- ultrafast is real (300tps on Astra). It’s way expensive though.
- sign in with ChatGPT: use your sub and inference in 3rd party apps
- extensions in ChatGPT can now be full apps
- 1 banked reset for all
I think that’s most of it!
Shivay Lamba retweeted
Vaibhav (VB) Srivastav@reach_vb
3hUnited Kingdom
United KingdomConnected via Germany App StoreAccount-level information, not a live location or per-post device.
GPT-6.1 Sol matches Astra on DeepSWE at ~1/5 the cost!!
$2 input, $10 output and just $0.10 cached input per million tokens.
Enjoy!!
Shivay Lamba retweeted
Introducing 6.1 Sol, near Astra intelligence at one fifth of the price of Astra and 95% cache read discount. It is an absolute workhorse.
Combined with ultrafast for 8X speeded available today for Astra and coming soon for 6.1 Sol.
Shivay Lamba retweeted
I’ll explain the new Pro 200 plan differently, before I start live tweeting from DevDay on things that are going out!
Today we are going to ship a number of things that increase what you can do across the Plus and Pro plans. A lot of compute is online for this increase. As we increase the floor, we are changing the relative difference between plans to be
Plus = 1X
Pro 100 = 5X
Pro 200 = 10X
and we are reopening subscriptions for Pro 200 (we had paused it). If you have an existing plan you will keep the 20X multiplier for a bit and also receive a lot of additional credits because we know changes are hard even if it means that everyone will get more in the end.
Best person in AI one can hire
Professional Update: I have transitioned from my full-time role at iii, though I remain committed to contributing to the company's ongoing success, as I deeply value the innovative work we have accomplished together.
If you are developing next ambitious thing and require expertise in Engineering, Product, or Developer Relations, hit me up!
Shivay Lamba retweeted
Training AI agents takes more than fast token generation.
Tools run. Tests stall. Training batches wait.
We cut fixed-weight batch collection time by 66.9% on a Terminal-Bench-based workload by improving routing, sandbox execution and scheduling.
Here’s how: nebius.com/blog/posts/in-age…
Shivay Lamba retweeted
Theo - t3.gg@theo
Sep 28United States
United StatesConnected via United States App StoreAccount-level information, not a live location or per-post device.