
Product @LiteLLM | Ex-Founder @WebhoundAI (YC S23) | Stanford CS ‘23 | San Francisco & Beirut
San Francisco, CA
Joined September 2011
- Tweets878
- Following227
- Followers309
- Likes26.6K
About to present on our new @LiteLLM customer management portal in 5 minutes, hop on and ask questions
crowdcast.io/c/w9airc43f37a
31
If you use a tracing product, why? Isn’t the ideal experience here OTEL in Clickhouse with an MCP?
Then you can have your agent run automations and maybe build a custom UI on top of it
What am I missing here?
3
9
451
Moe Khalil retweeted
you'll never hear about APIs without an API Gateway. if you're building AI products without an AI gateway, why?
As a developer, how do you define "improving your agent"?
Presumably at some point your agent harness has all the capabilities it needs to accomplish its goal.
What comes next?
Can it file PRs?
JUST IN: Apple $AAPL is developing AI agent to rival OpenAI and Anthropic that can control devices, browse the web, and write code.
3
96
LiteLLM has evolved a lot since its first commit, and nowhere is that more evident than in its database
Since joining LiteLLM, I've come across so many different tables and columns that didn't make sense at first. When I dug deeper it became clear they were crucial to making the product work.
So, I built the LiteLLM database course. It goes through every table we have, what purpose it serves, why it was added (according to PR bodies), and the overall evolution of our DB and why it is the way it is today.
Check it out ↓
How do you handle it when each one of your users wants a different set of abstractions?
We want to support different versions of the same use case but keep the product simple
Do you just build it all but keep the happy path simple?
3
7
375
So we have evals for evals now, but how are we supposed to eval those evals?
Introducing the Eval Optimizer.
Not every eval needs a frontier model. Judgment now learns your agent trajectories and picks the right compute per judge: code, a decision model, an LLM or an agent.
For one customer, it cut the cost per eval by up to 50×.
2
166
AI-native companies have two types of engineers
One makes sure the agent doesn’t burn $50k overnight
The other makes sure it’s getting smarter every day
Which one are you?
3
7
292
It's annoying that @cognition Devin doesn't support an LLM gateway but I have to admit Devin Bot is pretty sick
2
5
319
Moe Khalil retweeted
If you sell an AI agent, every customer request costs you money.
Join us live on Friday to learn how you can manage costs with our new Customer Management portal.
Nov 20, 10:00 AM PT
Link to register: crowdcast.io/c/w9airc43f37a
Moe Khalil retweeted
We’re launching @LiteLLM for Startups.
We’re starting with @ycombinator companies and opening it up to other VC-backed AI startups with 5+ people.
This stack includes our Rust AI Gateway + Agent tracing + Evals.
You’ll get a direct Slack channel with our engineers. For teams in San Francisco, I’ll come to your office to help get set up.
If you’re building an AI company, we’d love to help ❤️
Getting good at coding with AI is learning how to use a coding session to get context (while shipping)
Using AI when you have context is all great but nobody talks about the pain of being expected to ship fast when you don’t have context. It leads to 0 learning and is just very frustrating. I feel really sad for junior engineers at this point.
3
87
I think the one thing new LLMs will always improve on is building SOTA software
LLMs are pretty bad at writing code for concepts that reached the mainstream within the ~3 months before that model's release
That will continue to be true as long as there is a data cutoff period, and you'll always have to upgrade to latest to get those capabilities
My contrarian take is that LLMs will not get any "smarter" than they are today.
They will get more efficient, more capable (plugins / memory / context / skills), more specialized,
but we're at the peak of artificial intelligence.
And we can see that. The last several months of new releases are not that different from the last.
It's like the iPhone - it came in and disrupted everything. But now that we're at the 10th+ iteration buying the brand new phone isn't that much of an upgrade to justify it.
74











