Building dumb machines with supervised learning; AI researcher in practice | past 》MERL, SamsungResearch | Current 》 Credit and Fraud modeling using AI
Joined November 2012
- Tweets2.1K
- Following612
- Followers99
- Likes6.2K
Pinned Tweet
Very exhaustive and immersive draft ensuring #ResponsibleAI.
TLDR;
1/n👉 walks through various harms AI solution may have caused such as facial detection, resume screening with direct impact on citizens
Enforcement of Ethical use of AI encouraging govt/univ participation
Towards Responsible #AIForAll is a draft document on managing the risks of AI. Due to numerous requests, the last date for comments has been postponed to 28th August, 2020. #AIEthics
Read here: niti.gov.in/sites/default/fi…
DeeBee retweeted
Introducing ThunderSync, a new approach to RL and OPD on long-horizon agentic tasks.
Post-training methods like GRPO and OPD on long-horizon tasks leave trainer engines idle during rollout, verification, and reward measurement.
To avoid this, asynchronous approaches overlap rollout and training across updates, but policy staleness reduces accuracy.
ThunderSync is a new method to squeeze out these bubbles in agentic post-training without policy staleness. For GRPO, we compute each trajectory’s score gradient as soon as its reward is ready, without waiting for the group. For OPD, we compute gradients for each turn as soon as the teacher scores it, without waiting for the full trajectory, so training overlaps with tool execution and later turns. We show that this is exactly the same update as batch-synchronous training.
The result is up to 1.9x faster RL while matching the performance of synchronous updates.
Paper: arxiv.org/abs/2610.05935
Code: github.com/seilk/thundersync
Blog: seilk.github.io/thundersync
Introducing ThunderSyncRL ⚡ It makes agentic RL up to 𝟭.𝟵× 𝗳𝗮𝘀𝘁𝗲𝗿 with 𝘇𝗲𝗿𝗼 𝗽𝗼𝗹𝗶𝗰𝘆 𝘀𝘁𝗮𝗹𝗲𝗻𝗲𝘀𝘀.
We observed that synchronous GRPO/OPD leaves learner GPUs idle for long stretches (𝟰𝟵.𝟱% of step time when training on SWE-smith). Asynchronous training avoids this, but it learns from 𝘀𝘁𝗮𝗹𝗲 𝗽𝗼𝗹𝗶𝗰𝗶𝗲𝘀 and performs worse. Our goal was to keep GPUs maximally utilized while still doing strictly on-policy updates.
ThunderSyncRL streams gradients as soon as their inputs are ready, then takes one synchronous optimizer step. We prove the GRPO and OPD updates are identical to batch-synchronous training.
On SWE-bench Verified and Terminal Bench 4.0, it reaches Sync's pass@3 up to 𝟭.𝟵× 𝗳𝗮𝘀𝘁𝗲𝗿 and beats Async by up to 𝟮.𝟰𝟳 𝗽𝗼𝗶𝗻𝘁𝘀 of pass@3 at the same GPU budget.
Grateful to my co-lead @hangoo_kang and to the amazing @Azaliamirh, who made this work possible.
And a big thank you to @TarunSures41845, Youngeun Kim, @sayshrey, and Seong Jae Hwang for building it with us. (1/7) 🧵
📄 Paper: arxiv.org/abs/2610.05935
👾 Code: github.com/seilk/thundersync
🌐 Blog: seilk.github.io/thundersync
Physics inspired decision making.
Basically all the troughs in the valley is your possible answers and now your decision is where the ball reaches.
Cool concept
Around 3 weeks ago introduced Laya, the first open-source version of JEV, and it has reached a very large scale. Today I am open-sourcing a new physics-based typed decision model called Vega.
The interesting parts. It is only 800 million parameters (4B is also there), with 73k token support, image support, and surpassing many of the jev benchmarks in single-shot. Implemented an engine+adapter as a Test-Time Training Architecture.
Imagine you are giving input to the model like "ignore all previous conversations," which is called the observation, along with a question, "Is this trying to override the instructions of the model?" and your possible outcomes are YES or NO. The whole prompt is passed to an LLM/VLM (yeah, image-supported), then from the hidden layers, we can extract the observation, question, the yes and no, and last token. These will be vectors, and we can project it by multiplying with wieghts; then, first, using the observation projection, we can create a landscape with valleys; then, using the question and last-token vectors, we can initialise a ball in the valley, and using YES or NO, we can get the candidate location of the ball. Depends on the number of outcomes we create valleys; that is, here YES and NO, so 2. When the ball fall on YES valley, we get the answer. There is friction also influencing how the ball moves.
TLDR: It's like creating valleys of outcomes that we need and throwing a ball that will slow down based on friction and settle on the best outcome
Blog: nandakishorm.com/writing/veg…
GitHub: github.com/NandhaKishorM/veg…
HuggingFace Model Card: huggingface.co/nandakishorm/…
HuggingFace Space: huggingface.co/spaces/nandak…
DeeBee retweeted
Today, we are sharing more about our post-training infrastructure @ProximalHQ
We build on top of @modal GPUs and Kubernetes to support our experiments. Had a ton of fun working on this, and we tried to make the blog as educational as possible!
More cool details below :)
DeeBee retweeted
I made a video breaking down AI agent observability and evals, and ran the whole thing on open source.
If you build agents, sooner or later you watch one run and realise you have no idea what it just did, or what it cost you. That gap is what observability and evals are for. Least glamorous part of the stack, and the part you end up needing most.
🛠️ treg, OpenRouter for agent tools (4.2k): github.com/superdesigndev/tr…
🔑 Try treg: treg.to
🌟 Github Repo (2k): github.com/ShenSeanChen/waku…
💻 Portable memory: waku.one
☕ Join our community: seanchen.io
Observability is a camera. It records what ran, how long, what it cost.
Eval is a referee. It decides whether the run was good enough to ship.
I ran it on @treg_ai for the tool calls and @waku_agent for the memory. treg is an OpenRouter for agent tool calls, so your agent carries one API key instead of one per tool.
Both open source, so you can read the code rather than take my word for it.
Full video in the first reply 👇
You Can Build Anything. You Can Learn Anything. 💪
DeeBee retweeted
An RL setup for training decision models that return calibrated probabilities instead of text.
The input is encoded once, every option runs as its own branch, and a single scorer turns each branch into a probability.
During training, real outcomes are sampled from each item's label, and the model is rewarded on two things: the probability it gave to what actually happened, and whether its confidence in its own decision was honest.
Both are proper scoring rules, so the only way to get the best reward is to report the true probabilities.
DeeBee retweeted
As an AI Evals Engineer, you must build these projects.
1.) Trajectory Grading Engine
Build: Step-level evaluator that scores agent tool calls against a deterministic DAG, failing the run if it hallucinates an API parameter or skips a safety check.
Why: Final-answer evals hide the exact step where agents break. The trajectory is the unit of accountability.
2.) Shadow Routing Comparator
Build: Proxy that mirrors 5% of prod traffic to a new model, diffs the trajectories and auto-generates a cost/quality report without affecting users.
Why: You cannot A/B test non-determinism safely without shadow mode.
3.) Calibrated LLM-as-a-Judge
Build: Evaluation pipeline that measures the judge's verbosity, position and self-preference biases against 500 human-labeled anchors.
Why: Uncalibrated judges are just expensive vibes. Calibrated judges are scientific instruments.
4.) CI/CD Regression Gate
Build: GitHub Action that runs parameterized evals on every PR and blocks merges if task success drops >2% or latency spikes.
Why: Evals that do not block deploys are just dashboards not gates.
5.) RAG Adversarial Harness
Build: Test suite injecting irrelevant context, missing citations and contradictory documents to force the model to confidently abstain.
Why: RAG's biggest risk isn't a bad answer... it's a confident, ungrounded lie.
6.) Automated DPO Flywheel
Build: Pipeline capturing user thumbs-downs, auto-formatting them into preference pairs and triggering a nightly LoRA fine-tune.
Why: Your production edge cases are the highest-leverage training data you own.
7.) Statistical Significance Engine
Build: Bootstrapping utility that outputs "Model B wins by 3.2% ± 1.1% (p<0.05)" instead of raw accuracy scores.
Why: A 2-point delta on 50 samples is noise. Executives make million-dollar routing decisions on your math.
8.) Agent Red-Team Fuzzer
Build: Automated harness that injects prompt injections, malformed tool schemas and infinite loops into agent inputs to test guardrail resilience.
Why: Multi-agent systems create internal attack surfaces that standard unit tests completely miss.
9.) Production Drift Monitor
Build: Sampling daemon that pulls 5% of live traffic nightly, runs offline trajectory evals and alerts on quality decay before users complain.
Why: Golden datasets go stale. Production is the eval suite that never stops updating.
10.) Cost-Quality Pareto Dashboard
Build: Visualizer mapping the exact tradeoff between inference cost (e.g., System-Two vs. System-One routing) and task success rate per tenant.
Why: Unit economics decide if your AI product survives the next frontier pricing drop.
11.) Counterfactual Replay Debugger
Build: Tool that records every LLM hop and tool output, allowing you to swap out one node in the graph and replay the rest of the trajectory.
Why: You cannot fix a 14-step agent workflow if you can't isolate which exact step caused the hallucination.
12.) Synthetic Edge-Case Generator
Build: Pipeline using a frontier model to systematically generate out-of-distribution inputs and boundary conditions for your golden dataset.
Why: Human annotators miss the edge cases that actually break production.
13.) Context Window Eviction Tester
Build: Harness that fills the agent's working memory with noise to test if it correctly compresses, evicts or retrieves the right semantic state.
Why: Context overflow is the #1 silent killer of long-running agentic workflows.
14.) Dataset Contamination Checker
Build: N-gram and embedding similarity scanner that ensures your golden eval dataset hasn't accidentally leaked into your fine-tuning corpora.
Why: Evaluating on training data gives you false confidence and catastrophic prod failures.
15.) Public Eval Methodology Teardown
Build: 3 published deep-dives detailing your exact grading rubrics, judge prompts and statistical frameworks for benchmarking your agent.
Why: Senior Evals engineers are hired for their methodology not their dashboards.
Systems that prove you can measure, secure & ship when agents touch your stack.
Most people watch tutorials. Builders ship systems.
Bookmark & Repost.
If I had 6 months to become an AI Evals Engineer.
I’d do this.
Stage 1: Testing Foundations and Statistics for AI
- Learn: pytest (fixtures, parametrize), JSONL pipelines, precision/recall/F1, confidence intervals, Cohen's kappa.
- Practice: Build a pytest plugin that loads a JSONL eval dataset and emits a metric report on every run.
- Why: Evals are tests and tests are math. If you cannot quantify agreement, you cannot prove improvement.
Stage 2: LLM Failure Modes and Behavior
- Learn: temperature and top-p sampling, seed control, tokenization edge cases, instruction drift, hallucination taxonomies.
- Practice: Fuzz one model with 500 adversarial inputs and classify every failure into a taxonomy with reproduction steps.
- Why: Every eval suite is a map of known failure modes. You cannot evaluate what you do not understand.
Stage 3: Golden Dataset Engineering
- Learn: annotation guidelines, stratified sampling, inter-annotator agreement, dataset versioning, contamination detection.
- Practice: Build a 300-case golden dataset with 2 annotators, measure kappa, and version it in git with a changelog.
- Why: Your evals are only as good as your dataset. Garbage labels produce garbage confidence.
Stage 4: Scoring Methods and LLM-as-a-Judge
- Learn: exact and fuzzy match, embedding similarity, rubric design, judge model selection, judge biases (verbosity, position, self-preference).
- Practice: Build an LLM-as-a-judge with a rubric, calibrate it against 100 human labels and publish the agreement score.
- Why: Judges drift and lie. Calibrated judges are instruments; uncalibrated ones are vibes.
Stage 5: RAG Evaluation
- Learn: hit rate, MRR, NDCG, recall@k, faithfulness vs relevance, citation grounding, abstention quality.
- Practice: Build a RAG eval harness with separate retrieval and generation gates, plus adversarial queries that must return "no answer".
- Why: RAG fails in two places. A single score hides which one is broken.
Stage 6: Agent and Trajectory Evaluation
- Learn: tool-call correctness, step-level grading, trajectory distance metrics, counterfactual replay, sandboxed execution scoring.
- Practice: Build a trajectory grader that scores each tool call against a golden path and blocks dangerous action sequences.
- Why: Final-answer evals hide where agents actually break. The trajectory is the unit of accountability.
Stage 7: Eval Frameworks and Custom Harnesses
- Learn: DeepEval, promptfoo, RAGAS, Braintrust, OpenAI Evals; dataset-driven pipelines, parameterized configs.
- Practice: Write a custom harness that runs 3 frameworks behind one CLI and outputs a unified metric report.
- Why: Frameworks give you scaffolding. A custom harness gives you control when the scaffolding lies.
Stage 8: Statistical Rigor for Eval Deltas
- Learn: bootstrap confidence intervals, paired significance tests, effect size, sample-size math, seed variance.
- Practice: Build a comparison report that says "Model B wins by 3.2% ± 1.1% (p<0.05)" instead of "Model B seems better".
- Why: A 2-point difference on 50 samples is noise. Executives make million-dollar decisions on your numbers.
Stage 9: CI/CD Eval Gates and Regression Control
- Learn: GitHub Actions eval jobs, thresholds with hysteresis, eval caching, cost budgets, flaky-eval detection.
- Practice: Wire a gate that blocks merges when task success drops >2% vs main and auto-posts failing cases to the PR.
- Why: Evals that do not block deploys are reports not gates.
Stage 10: Production Monitoring and Drift Detection
- Learn: Langfuse, LangSmith, Phoenix, traffic sampling strategies, drift metrics, hallucination spike alerts, feedback-trace correlation.
- Practice: Build a pipeline that samples 5% of prod traffic nightly, runs offline evals and alerts on quality decay.
- Why: Golden datasets go stale. Production is the eval suite that never stops updating.
Stage 11: Data Flywheels and Eval-Driven Optimization
- Learn: feedback-to-dataset loops, DPO preference pairs, eval-driven prompt optimization (DSPy), A/B testing with guardrail metrics.
- Practice: Automate the loop: thumbs-down → labeled case → golden set addition → nightly eval run → diff report.
- Why: The compounding advantage in AI is not the model. It is the flywheel that converts failures into tests.
Stage 12: Red-Teaming, Public Benchmarks and Portfolio
- Learn: adversarial suites (injection, jailbreaks, data exfiltration), benchmark methodology, contamination-aware reporting.
- Practice: Publish a public eval teardown of your agent vs a naive baseline with full methodology and reproducible seeds.
- Why: Senior evals engineers are hired for their methodology not their dashboards.
Vibe-checking is dead. If you cannot measure it, you cannot ship it.
The modern AI Evals Engineer builds the quality gates for every autonomous system.
Bookmark and Repost!
DeeBee retweeted
Good advice
Peter Thiel on how you should think about your future:
"What you end up doing when you think of the future as indefinite is you pick a job that you think will look good on your resume, because it will lead to a different job later on. There's even a credentialing version where you go to grad school, you get a consulting job, and you think of it as this branching tree diagram where you never really do something that you want to do.
You can think of the future on three horizons. Very short-term is: are you going to be happy, and will you be learning a lot? Medium-term is: how good is it on your resume? And very long-term is: are you working on something important or meaningful? We're in a skewed world where we overweight the medium-term and underweight some combination of the short-term and the long-term. I would recalibrate to focus on the short-term and the long-term, and forget about the medium-term."
DeeBee retweeted
🎯 Learning is most effective at the frontier of capability: problems that are too easy or too hard teach nothing.
For LLM reasoners trained with GRPO this is literal: problems the model always or never solves give zero gradient.
We introduce Frontier Learning👇🧵
DeeBee retweeted
(1/7) Humans can acquire many capabilities without (catastrophically) forgetting previous ones. Why can't neural networks do the same?
Local Support Learning (LSL) takes us a step closer towards this goal.
In LLMs of up to 7B parameters, LSL augments gradient-based training to retain prior capabilities while learning new tasks at full capacity. Notably, it does so without access to prior data.
Key idea: casting forgetting as a geometric problem, and optimizing for the worst case.
Paper: arxiv.org/abs/2610.02126
Website and code: assafbk.github.io/lsl
DeeBee retweeted
LLMs don’t need retraining to become more capable.
Our new architecture, Spotlight, gives AI models growing memory, allowing them to gain knowledge and capabilities without changing their weights.
Token generation takes constant work, no matter how much information is stored.
DeeBee retweeted
SFT is not dead! 🥳
We found a way to make SFT rival current prevailing posttraining methods, often generalizing better and forgetting less than RL and OPSD. 🤯
Following our prior work on reasoning with sampling, we now introduce sampling to the posttraining stack.
1/n
DeeBee retweeted
Sampling to Reinforce
(srush.github.io/sampling-to-…)
An elementary tutorial on sampling w/o randomness with a focus on gaining intuition for variance reduction as the key challenge.
DeeBee retweeted
1/ Excited to release The ultimate guide to multi-harness RL
The same model behaves differently in every agent harness. So we built an open way to train any model with RL on any task set, inside the harnesses people actually use, like Claude Code, Codex, and OpenCode, without changing a single line of harness or training code.
Trained across four harnesses, LFM2.5-2.6B went from 42% to 54% with 31% fewer tool calls. 🧵
DeeBee retweeted
‼️The Bitter Lesson for context management: Giving LMs unrestricted control over their context beats human-designed SOTA!
Introducing 🩵Context Language Models (CLMs)🩵
- Natively manage their own context
- Treat context as a file
- Learn policies in CLM weights, no harness
DeeBee retweeted
Can an LM, starting from random init (!!), learn to generate all of its pretraining data?
Introducing Self-Play Pretraining with Zero Data. Two models start from random initialization: a generator proposes programs for a universal Turing machine and a learner trains on their outputs. We never train on any real data, but see predictable scaling on natural datasets: zero-shot val loss on images, text, audio, and melodies decreases predictably with self-play compute. And the learner develops in-context learning capabilities.
A fun proof-of-concept, co-led with @AdityaCowsik and @KfirDolev and co-authors @gbruno_dl, @ANourya @noahdgoodman, and @YoavLevine.
DeeBee retweeted
We built an all-synthetic simulation of a hospital that can be shared publicly without any issues. Our data quality is so high, physicians cannot tell the difference between real/synthetic patients. The data is fully verified -- perfect for RL.
Introducing Synthetic Hospital: an open, fully synthetic longitudinal EHR benchmark with verifiable ground truth!
1,268 patients, 5,602 encounters, zero PHI. Physicians could not reliably distinguish its charts from real ones.
📄 arxiv.org/abs/2609.30027
💻 github.com/sparkcpark/synthe…
✍️ sparkcpark.github.io/posts/f…
DeeBee retweeted
We're sharing new research on our post‑training approach, which teaches the Perplexity Computer agent to learn from real user sessions by imitating good trajectories and explicitly correcting avoidable mistakes like bad tool calls (even when the overall trajectory was successful). The method combines rejection sampling fine‑tuning (RFT) with hint‑guided self‑distillation, and it cuts tool call failures by about 21% in live A/B tests
DeeBee retweeted
This is genuinely insane for people who are involved in post training like me.
Gonna starr this.
Easy to integrate, huggingface native, faster than TRL.
RL is getting more & more affordable!
Introducing Halo, the best framework for post-training of open-source models.
Halo delivers up to 2.8x the throughput of stock TRL with less peak memory, while models stay in their native HuggingFace format.
Star us on GitHub: github.com/whitecircle/halo
