work: engineering @freeplay_ai, @height_app, @twitter and @sourcegraph fun: skiing, running and biking in the mountains and desert
Durango, CO
Joined October 2013
- Tweets219
- Following216
- Followers269
- Likes12.5K
This is the only benefit of prompt-crafting prompts -- they show humans what they can and should be specifying in the prompts. It lets users see how underspecified their initial prompts are, shortcutting a bunch of trial and error you usually need to go through in production.
not only do people want to just see what the LLM can do, but they also don't fully know what they are supposed to say in the prompt, or what answer they want (they won't know it until they see it).
I think of a prompt as a form you need to fill out to submit your task to the LLM, and the fields of this form are unknown and dynamic (i.e., task-specific). a prompt writing tool can make these fields more known
Nico Tonozzi retweeted
Claude is thrown into confusion when I tell it to roleplay an overly censorious chatbot and it tries to refuse
Catches itself mid sentence
I built this! It's so useful to be able to understand and limit your LLM spending by feature or prompt.
The boring parts of LLM ops are important. Like managing spend. 💸
Our team @freeplay_ai cooked up some new usage dashboards and spend management controls to help.
Now development teams can easily track and control AI costs across providers, and how money gets spent.
Nico Tonozzi retweeted
The AI Builders meetup is going on the road!
Come join us in LA for Tech Week on 10/14. We'll be hosting with our friends @ThisIsArtium and @SiliconVlyBank.
There will be demos from local startups & big companies -- come see what folks are building in LA. 🌴🤖
These events are always packed, RSVP below.
lu.ma/bej1uk00
gwern.net/scaling-hypothesis has gotta be the best post of the decade
Nico Tonozzi retweeted
Super sleek AI automating all the project management grind, built on @OpenAI!
The UI is stunning, and watching the @height_app agent triage and edit in real time is awesome! ✨
This quoted post is unavailable.
Nico Tonozzi retweeted
🧵 Everyone's looking for signal in all the AI noise... There’s no better way to learn than from teams who are already doing it!
So today we’re launching our new podcast to tell these stories. 🎙️
Episode 1 of Deployed features our friend, @jeffseibert, CEO & Co-founder of @digits. 🎉
Here's a quick preview: Why it's worth it to review lots of data by hand.
arxiv.org/html/2407.21770v1 is super interesting (investigating if Q* really works!) It's not as trivial as Aidan argues in yellow-apartment-148.notion.…, because evaluating promising paths is obviously harder in the context of generating text than the context of chess.
It's disappointing that they didn't see any improvements on the harder end of their evaluation dataset.
I've been really impressed at how easy it is for our customers to get high quality evaluation suites once they can clearly see the completions flowing through their evaluations. There's no substitute for looking at the data!
Getting more teams to be successful with generative AI in production depends on making evals more practical and approachable.
We're fired up about early feedback: In multiple cases we’ve seen people go from 60% or 70% alignment to 100% in 2-3 iterations.
Curious? Read more on the details here.
freeplay.ai/blog/accelerate-…
GPT with advanced voice mode is incredible! The voices feel so much more alive, and identifiable with their name. I'd love to see something like Claude's artifacts integrated with the voice mode.
Awesome to see smaller and cheaper models competing with the big guys on hallucination detection
2/ Lynx v1.1 uses the llama-3.1 architecture and is trained on data from real-world domains including finance and medicine. Lynx excels at reasoning in hard-to-detect hallucinations that include nuanced and challenging scenarios involving questions, documents and answers.
Check out our Hallucination Benchmark here: huggingface.co/datasets/Patr…
atlas.nomic.ai/data/patronus…
Nico Tonozzi retweeted
🚀 Thrilled to announce our new partnership with @MongoDB!
Together, we're empowering teams to test & tune RAG systems for production.
MongoDB Atlas combines the power of a high-performance operational database & vector databases into one platform for modern AI applications. @freeplay_ai makes it easy to build, test & optimize those applications to work at scale.
Dive in. 📖 freeplay.ai/blog/shipping-ra…
Nico Tonozzi retweeted
❤️ Last but not least, we've been building with the best team. Huge thanks to @ericwryan @RealBrianNewsom @ntonozzi @cshogue @upperstudy @morduhh @rishair @initialcapacity for helping us bring Freeplay to life.
Super excited to share what I've been working on with a small team for the past couple months: producthunt.com/posts/freepl…. All of our work has been motivated by problems I hit building AI features at Height. I'd love your support on Product Hunt if you have an account!
Has the ozone layer gotten radically better since container ships reduced sulfur emissions? What about acid rain over the Atlantic?
