Pinned Tweet
Give your agent a few hours with Pocket Model, and it can run dataset synthesis and optimization, evals, and training runs to generate a highly capable assistant for your specific home automation (or any custom API) that matches or beats a prompt-tuned big-lab model
Pocket Model retweeted
Reflection @reflection_ai Beam 1.0 vs DeepSeek V4.1 Flash 1017 on 90 hard specimens (BFCL tool calls, Omni-MATH 7+, SuperGPQA hard), one sample each, code grader on @pocketmodelai:
Beam 67/90
DeepSeek 66/90
Beam is 6x the cost. Maybe a lossy quant similar to my dsx experiment can help equalize the pricing here. They are similarly sized models.
Pocket Model retweeted
DeepSeek-V4.1-Flash-1017 on 2RTX 3090s and 128 GB DDR4 RAM at 30 tok/s on agentic coding
New quant, v3a: recalibrated on the model's own reasoning traces. Same size, speed and quality, and v2's runaway reasoning is gone.
v2 sometimes ran 3–4× longer than the full model; v3a stays in its range.
Better perplexity doesn't stop overthinking (up to a certain point). Less lossy quants in 2-3 bit range still reason 2–3× longer than baseline on hard math @pocketmodelai evals
Run it locally on your own:
github.com/lxe/dsx
huggingface.co/lxe/DeepSeek-…
Pocket Model retweeted
2 bit quant of DeepSeek-V4.1-Flash-1017 on two used RTX 3090s matches the quality of hosted full model on hard reasoning at the cost of thinking 2x longer
Preliminary eval on 26 problems ran on @pocketmodelai
Runner: github.com/lxe/dsx
Model: huggingface.co/lxe/DeepSeek-…
Pocket Model retweeted
DeepSeek 4.1 Flash 1017, a 552 billion parameter model (~748b w/engram) running at ~30 t/s on 2 asymmetric RTX 3090s (48 GB total VRAM) and 128 GB of DDR4 RAM:
Sol-level model on your old gaming PC
Pocket Model retweeted
Small models are making a comeback. Qualifying a dataset and finetuning a small checkpoint is almost always going to yield better results (and cost) than just using a generalized frontier cloud model.
For example, I was able to finetune an adapter for a small Qwen3.5-4B checkpoint that does receipt OCR semantic reconciliation and ship directly to a device.
Now I don't need to pay for infrastructure or tokens without sacrificing UX or result quality. Not to mention not having to deal with privacy and compliance nuances.
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"
I imagine we'll have some backend services that could be replaced entirely by this new architecture.
Pocket Model retweeted
DeepSeek V4.1 Flash 1017 came out today at $0.60 per million output tokens.
I ran it against Opus 5.5 and GPT-6.1 Sol on 250 hard problems (BFCL tool calls, Omni-MATH 7+, SuperGPQA hard), simple code grader on @pocketmodelai
Flash 76.4% at a 128k token limit
Opus 82.0%
Sol 74.8%
Flash ties Opus on tool calls at 1/35 the cost per correct call, and matches Sol everywhere.
It trails Opus by 11 points on hard knowledge, and takes about 2 minutes per hard math problem.
You don’t need to feed your sensitive logs and traces to costly inference providers to gather insights
Own your own model, eval and tune it continuously, and pay pennies for inference
Pocket Model retweeted
> 9b models can barely write a flappy bird game
challenge accepted
Pocket Model retweeted
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"
I imagine we'll have some backend services that could be replaced entirely by this new architecture.
Create your own AI that works for you. Stay tuned for more. pocketmodel.ai/















