Buzzword, jargon, buzzword, jargon.

Joined August 2010
Roberto Barroso Luque retweeted
Introducing FireRouter with Opus.  Our first router model. Routine tasks go to top open models, Claude Opus handles the rest.  Cache and task-aware routing. Proven 98.1% of Opus accuracy with 57% lower cost per coding session. Read more: fireworks.ai/blog/introducin…
20
14
8
159
198,616
Roberto Barroso Luque retweeted
Fireworks' Ember-1 is now on Vals. We tested it on our finance and legal benchmarks, and it performs close to Kimi K3 while using less reasoning tokens per turn.
2
3
3
40
11,016
I said it before and will say it again: Specialized intelligence beats general intelligence At @FireworksAI_HQ we have the privilege to work with some killers in the space such as @RogoAI / @harvey / @SierraPlatform and many many others who are all pushing the boundary of intelligence in their respective verticals! Very pumped about the release of our specialized intelligence index fireworks.ai/specialized-int…
Today we're launching the Specialized Intelligence Index (SII): one destination for real-work benchmarks across industries, built by the teams that use them every day. Hear from Fireworks co-founder @the_bunny_chen on the importance of specialized benchmarks:
9
313
Roberto Barroso Luque retweeted
We worked alongside @inferact and @FireworksAI_HQ on the investigation. 🔍 We share that bar for consistency. Credit to @FireworksAI_HQ for taking the extra time to verify, and to @Zai_org for the quick API update. 🎉
GLM-5.3-Flash is live on Fireworks on day… 2 Why? Because we take quality very seriously. We found a benchmark discrepancy we couldn’t explain, so we delayed the launch to investigate. Day 0 (Wed): we saw 2x longer thinking on reasoning-heavy benchmarks (AIME & GPQA) for open source engines compared with @Zai_org API. Same scores, worse token efficiency. Agentic benchmarks looked good. We decided to investigate further, as overthinking might become a quality problem if max_tokens are reached Day 1 (Thu): as other non-official providers launched, their APIs had thinking in the range of open-source engines: longer than Z.ai. We launched a private preview endpoint with disclaimers to a few customers and worked with them to assess quality Day 2 (Fri): the official Z.ai API updates. We rerun benchmarks: reasoning is now similarly long, consistent with vllm/sglang. Rest of the benchmarks, both public and internal, check out too. We launched GLM-5.3-Flash publicly: fireworks.ai/models/firework… More details below
15
19
1
136
15,670
Around 2 months ago we started working with the @harvey team with the vision to train a SOTA model for long horizon legal work. It was a very fun ride with some huge learnings on post training the largest MOEs for domain specific work, some learnings in the thread ....
5
3
12
831
5 / people are all that matter Outside of the model, RL loop, hill climbing, ablation tests, regressions and all other fun stuff ... ultimately what matters most is the people behind the work Had the huge privilege of working with Harvey's truly cracked team @nikogrupen / @ItsJulioPereyra / @calvincongelado alongside our own beasts Scott Gee (who needs to get on x urgently) / @yuan_sihan /@chahvivi / @JayThakrar So as a wiser person than me once said "the real model training is the friends we made along the way" ... ofc doesnt hurt to have them beautiful numbers (full table attached) to go with the friends
1
95
Replying to @BobBarroso @harvey
4 / co-building the harness with the algorithm has a huge impact We built a custom RL loop based on GSPO and ran it inside Harvey's harness. Each training env is a LAB-style task: partner request, client matter documents, expert rubric. The agent works inside a sandboxed workspace and gets graded by an LLM judge against the rubric Reward shape, rollout length and all the minutiae change what the trainer has to handle, so they are directly linked
1
30
Replying to @BobBarroso @harvey
3 / quality is obviously not the only thing that matters Very happy that the ex-consultants of @ArtificialAnlys have popularized the 2-by-2s so stating the obvious here but while quality is king cost, latency and other factors also matter ... ultimately its all about the pareto frontier Harvey Tenet runs at $5.92 per LAB task against $5.62 for base Kimi K3 -- effectively flat -- while being much much cheaper per task than other frontier models
30
Replying to @BobBarroso @harvey
2 / track hill climbing and regressions at the same time Its easy to see improvements on the benchmarks you are training on (because you know overfitting), the hard part is making sure you are not breaking the model on other tasks Alongside LAB all-pass we ran benchmarks on Mercor's Apex Agents, Crosby's Redline Bench, CUAD, MUAD and other legal agent benchmarks The results are very nice, not only did out of distribution benchmarks not regress but we also saw signs of generalization Apex Agents 58.8 → 74.0, Redline Bench 49.3 → 55.5 over base Kimi K3
48
Replying to @BobBarroso @harvey
1 / evaluation methodology really matters We all see new models coming out on a weekly basis with shiny numbers on swebench, terminalbench, etc etc Harvey built LAB (Legal Agent Benchmarks) for that. the agent gets a sandboxed workspace with real docs and tools and has to produce what a partner would ask for Even then, small details decide if what you're measuring is real signal or noise. Judge prompt, batching specifics, etc can all add noise and make comparisons not apples-to-apples We spent a ton of time (like waaaay too much time) tightening how we benchmarked: different judges, batching, per-criterion scoring
1
112
Roberto Barroso Luque retweeted
Specialized intelligence is hot in case you didn’t notice: @harvey’s Tenet Cognition’s SWE Cursor’s Composer GenSpark’s DeepResearch … the list goes on Join the movement with @FireworksAI_HQ training platform
Introducing Tenet, our first model post-trained for legal. Tenet is a Kimi K3 base that we post-trained with @FireworksAI_HQ on a corpus of publicly available legal data, synthetic data, and human expert data simulating long-horizon legal work. Training increases Tenet's all-pass rate by 82% on LAB and 22% on LAB Contracts relative to the Kimi K3 base model. It achieves state-of-the-art performance on LAB Contracts and places second on LAB. These gains generalize to other leading agentic benchmarks including @mercor's Apex Agents - Corporate Law, @crosbylegal's Redline Bench, and @scale_AI's Professional Reasoning Bench. Tenet is also optimized for token efficiency, operating at less than a fourth the cost of leading foundation models. We additionally post-trained three specialist models for Tenet to use as subagents: 1) M&A Diligence: post-trained with @baseten on our LAB Diligence environment in an RLM harness, this model is optimized for high-scale, long-horizon tasks. 2) Review Tables: trained with @appliedcompute on our Review Table environment, this model is state-of-the-art and cost-effective at high-volume document review and structured data extraction. 3) Firm Knowledge: trained with @EngramLab on our synthetic law firm environment, this model is optimized to learn and search over a firm's knowledge via memory and structured notes. More details on model training, environment design, benchmarking, results, and more in the article by @gabepereyra below. What's next for Harvey’s research? - Scaling LAB to more jurisdictions, practice areas and workflows - Scaling compute to bring new generalist models and capabilities to Harvey More to come soon.
6
15
1
123
13,289
🐳 keeps on shipping, 🎆 keeps on shipping DSV4 Pro 0813 is up on @FireworksAI_HQ day 0 legoooooooo
DeepSeek-V4-Pro-0813 is live Day-0 on Fireworks! Built for agentic and tool-augmented workflows, it outperforms Opus 4.8 on quality benchmarks like Terminal Bench 2.1, Cybergym, and DeepSWE at up to 6x the cost efficiency. It is on par with leading frontier models on Vals Index across Coding and Legal benchmarks. What to select: - Use Pro-0813 for heavy reasoning & coding - Use Flash-0731 for budget, less-intensive tasks Start building: fireworks.ai/models/deepseek…
1
5
192
Roberto Barroso Luque retweeted
When should you start post-training your own models? @FireworksAI_HQ CEO @lqiao’s answer: after product-market fit. Not because it's hard... but because only after PMF is the data coming off your product surface worth training on. Lin joined us for our @sequoia "Own Your Intelligence" event to host a workshop on all things post-training; what works, what breaks, and how not to let the model outsmart you. Must listen!! 00:00 Introduction 00:37 What Fireworks sees across thousands of AI applications 02:47 Off-the-shelf APIs and the problem of keeping your taste 03:58 What "owning your intelligence" actually means 05:43 The progression: prompting → RAG → SFT → preferences → RL 07:20 Why this mirrors how humans learn 09:03 Matching the technique to the problem you actually have 10:46 Where teams get stuck: data quality and vibe evals 12:28 Reward hacking: the model that wrote zero lines of code 13:59 Training-to-serving alignment (and why quality drops) 15:55 Post-training in healthcare and security 17:31 From coding to every co-work domain 19:35 Incumbents, cost burden, and not scaling into bankruptcy 21:26 How much control do you want? 23:24 Q&A: What makes a good reward signal 25:00 Q&A: When to start thinking about post-training
11
16
2
168
28,344
Roberto Barroso Luque retweeted
Last week, I shared why @20vcFund invested $10m in Fireworks in 24h: Lessons from 40 reference calls, and why we believe they will be a >$100bn company 👇 First - I have never seen that level of advocacy across so many datapoints. 35 out of 40 were strong positives, and two things stood out: 1) *the customers do not talk like users of a commodity* - it matters, because it is the first pushback we tend to get here. Instead, they literally talk as if they found an unfair advantage, and do not want to give it back. So what about the specifics? - Our friends at Figma benchmarked it at 3x faster than Baseten in production, and now run their new AI "Assistant" on a small in-house model, on Fireworks, at >5x lower cost than a frontier model - The team at Uber ran its own benchmark, and said it plainly: Fireworks "crushes it" compared to anyone else. Specifically, they get to sub-100ms time to first token, and they are ~60% cheaper than Google - Engineer at HuggingFace we spoke to said that they were world leading on speed, latency and throughput - ElevenLabs scored them "at least a 9/10" and thought the same we did on the team: "they are the core PyTorch team, that is an unfair advantage" and "they are the best at fine tuning" - A large bank managed to get higher throughput than NVIDIA's own TensorRT, and watched Fireworks rewrite a kernel to support a brand-new model overnight I can go on for ages, but the theme is not "cheap tokens". It is instead: speed, reliability and customisation, that these teams cannot easily reproduce themselves. That is where @FireworksAI_HQ absolutely crushes it (and the competition at the same time). 2) *The part that's actually even more convincing*: the competition agrees. Here is what we discovered there: - a former head of marketing at one of their closest competitor told us any team looking at doing their best inference work should look at partnering with someone like Fireworks - an ex-engineer at another rival platform called them the leader in the space, and the company with most momentum. He specifically highlighted that they were converting developer adoption into a real, enterprise-scale business, as opposed to just running AI pilots - A former PM at the largest chip company in the world also admitted that they had a superior user experience and a better TCO (total cost of ownership) Those concrete examples (and there were many more) are why we see Fireworks as the un-disputed leader in the space. So while the pushback is real (open-source engines like vLLM and SGLang keep improving, and the hyperscalers would love to swallow this layer), there are two things we see: (a) across these calls, engineers still put Fireworks "several months ahead" of open source and rivals, so the gap is not closing as fast as the some would claims - if at all -, and (b) Fireworks keeps climbing up the stack, into fine-tuning, reinforcement learning and evals. I believe the latter is exactly where value moves as AI goes agentic. This leads me to the bigger idea I already shared: in AI, the model are the ones getting the limelights. But in the end, the layer that makes every open model fast, cheap and usable is where a lot of the durable value will sit. So to re-iterate: Invisible to users. Indispensable to developers. That is the bet. Thoughts @HarryStebbings @jaminball @GavinSBaker @EverettRandle @sahirazam @ericvishria?
13
18
3
106
66,269
Roberto Barroso Luque retweeted
one of the top cybersecurity models, post-trained from open weights by @depthfirstlabs on @FireworksAI_HQ long-horizon RL is as much an infra problem as a research one: 100+ turn rollouts, async/pipeline RL for high utilization, train-sampling numerical alignment Fireworks training platform takes care of infra, so research can move fast we're proud to support the ecosystem of cyber defenders like depthfirst as AI adoption accelerates
Today we're announcing dfs-large1, our newest cybersecurity model that achieves best-in-class performance on vulnerability detection tasks. Besides frontier AI labs, only a handful of companies have built specialized models that reach the state of the art in their domain. We're proud to be the first to do it for cybersecurity. dfs-large1 is built on GLM-5.2 and post-trained with reinforcement learning inside depthfirst's security infrastructure. We evaluated it on depthfirst-bench, our benchmark of long-horizon vulnerability discovery across complex repositories, where it achieves best-in-class performance. Training improvements have not plateaued yet and we expect additional performance gains as we continue training. A huge thank you to @FireworksAI_HQ for being an outstanding training partner. Their infrastructure enabled us run large-scale reinforcement learning efficiently and iterate much faster. dfs-large1 is now in preview within the @depthfirstlabs platform
3
5
38
3,777
Kimi K3 is live on @FireworksAI_HQ As per usual not only are we fast AF but we take quality as a P0 and never sacrifice quality for anything (only OSS provider to top Moonshot's evals across the board - see screnshot) Oh and one more nugget, Kimi K3 is also day 0 on our training platform for Serverless Lora Training (see fireworks.ai/blog/K3-LoRA-Tr…) excited to build specialized intelligence with our clients and partners! Thank you @Kimi_Moonshot team for this absolute gem, keep cooking my friends
Kimi K3 is the new SOTA for open models, and we partnered closely with Moonshot to make sure developers can have a great experience with K3 on Fireworks. Fireworks offers fast speeds and verified quality for production workloads. Give it a try: fireworks.ai/models/firework… @pirroh @Replit: "Kimi K3 on Replit Design exceeded our expectations. Unprecedented product design and UI capabilities, at a fraction of the cost we've come to expect from frontier models." Michael Haines from @mercor: "Kimi K3 on Fireworks delivered across the board. The performance is top-tier, the serverless scaling is seamless, and the Zero Data Retention assurance gives us full confidence at scale."
2
4
1,443
Roberto Barroso Luque retweeted
Kimi K3 is now live on Fireworks. Get started → fireworks.ai/models/firework…
11
20
9
341
434,099
Huge huge respect to @demishassabis and @GoogleDeepMind, they are the only frontier model to consistently release OSS models (Gemma) + released the OG transformer paper (Attention is all you need) And also lets not forget, @GoogleDeepMind also released AlphaGo and most importantly AlphaFold (to this day probably the biggest contribution to AI for science) Unrequested book recommendation on this topic - go read "The Infinity Machine" great read about Demis, DeepMind and the past 20 years of AI development
A strong and secure open ecosystem is important for the world to benefit from AI. We’ve always supported and contributed heavily to open source and science from Jax to Transformers to AlphaFold to Gemma open models which have now been downloaded 300M+ times. And the standards framework we’ve proposed supports responsible deployment of both open and proprietary models.
2
116
Preach. a couple of reasons why we should fight for open source: 1 / Our entire industry is built on open source (and the entire technology sector at large - however you define it) we would not exist without it. Linux, Kubernetes, Pytorch, Transformers, vLLM you name it all oss projects. 2 / Imagine if back in the day MSFT had managed to ban Linux, we would be without android, without cloud providers, ... you get my point. @BradSmi himself admitted they were on "the wrong side of history" (side note but kudos Brad for the humility + 100 points to you) 3 / Frontier labs themselves have benefited not only from open source but from the entirety of humanity's public knowledge - they have trained on the open web, built on top of open research and software and now have every incentive for regulatory capture. Not cool to pull back the ladder behind you just bc you want to make a couple of extra $B in revenue my guys @ClementDelangue said it best in his recent podcast with @a16z (I am paraphrasing a bit) "opennes is the antidote, you cant secure what nobody can inspect"
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models. images.nvidia.com/pdf/Open-W…
2
1
1
10
760
@FireworksAI_HQ front and center, and most importantly on the right side of history
36