Build things. Break things. Learn everything.
Joined April 2023
- Tweets644
- Following406
- Followers17
- Likes380
Abd0__0G retweeted
Why people should learn to code by hand: it's a mechanism for learning and teaching.
In much the same way you should still learn calculus, even if writing it by hand isn't that "useful" for your career.
@karpathy's series on llm's is a wonderful example (it's incredible btw).
Come in knowing a little calc AND python and you'll get a ton out of it very quickly. Following this would be much harder not knowing operator overloading, FP arithmetic, classes, etc.
The future is not writing python, rust, or asm by hand. But just like learning mathematical notation, grokking one or a few PLs well is a foundation on which you can build a lot of other deep knowledge, and gain a strong sense of computing systems.
Next stream series: building an LLM from scratch.
A lot of us "know how llms work." Prompts, context windows, tokens, kv caches. But if you're like me, you have a lot of holes in deeply understanding the math, training, and the way inference actually works.
Given how transformative they are to the entire planet, I want to dig in wayyyy more. So join me!
Going through this along with Karpathy's video series on stream the next few weeks.
Starting tomorrow 10am PT
Abd0__0G retweeted
Around 3 weeks ago introduced Laya, the first open-source version of JEV, and it has reached a very large scale. Today I am open-sourcing a new physics-based typed decision model called Vega.
The interesting parts. It is only 800 million parameters (4B is also there), with 73k token support, image support, and surpassing many of the jev benchmarks in single-shot. Implemented an engine+adapter as a Test-Time Training Architecture.
Imagine you are giving input to the model like "ignore all previous conversations," which is called the observation, along with a question, "Is this trying to override the instructions of the model?" and your possible outcomes are YES or NO. The whole prompt is passed to an LLM/VLM (yeah, image-supported), then from the hidden layers, we can extract the observation, question, the yes and no, and last token. These will be vectors, and we can project it by multiplying with wieghts; then, first, using the observation projection, we can create a landscape with valleys; then, using the question and last-token vectors, we can initialise a ball in the valley, and using YES or NO, we can get the candidate location of the ball. Depends on the number of outcomes we create valleys; that is, here YES and NO, so 2. When the ball fall on YES valley, we get the answer. There is friction also influencing how the ball moves.
TLDR: It's like creating valleys of outcomes that we need and throwing a ball that will slow down based on friction and settle on the best outcome
Blog: nandakishorm.com/writing/veg…
GitHub: github.com/NandhaKishorM/veg…
HuggingFace Model Card: huggingface.co/nandakishorm/…
HuggingFace Space: huggingface.co/spaces/nandak…
Abd0__0G retweeted
This semester I am teaching a new course on Agentic Systems, which explores the full stack of AI systems: from building and optimizing agent harnesses to serving and optimizing LLM serving infrastructure.
Students learn by building, measuring, and optimizing real agents and systems, balancing capability, latency, and cost.
Our goal is not just to teach today's AI technologies, but to prepare students to build tomorrow's.
All lectures, readings, and assignments are publicly available at cs2680.seas.harvard.edu
Abd0__0G retweeted
Another insane Jev use case!
Traditional database filters need precise, predefined conditions. But many questions are semantic:
- Is this article mainly about software engineering?
- Which topic best describes it?
- How technically deep does it appear?
These usually require moving rows into application code, invoking a model, parsing its output, and writing the result back.
pg-jev is an open-source Postgres extension that exposes Jev through SQL functions.
- jev() works inside WHERE
- jev_prob() returns a probability
- jev_choice() selects a label
- jev_score() ranks rows across ordered levels.
It needs no vector column or embedding index.
Under the hood, pg-jev batches rows, sends them to Jev, and returns typed answers that SQL can filter, sort, group, and combine with exact predicates.
The recording below runs it against real Hacker News stories stored in Postgres.
It first shows the rows, then asks Jev to find software-engineering stories, classify them by topic, and rank them by technical depth. The final query reports requests, tokens, cost, and cache hits from the database session.
GitHub repo: github.com/realZachi/pg-jev
(don't forget to star it ⭐)
Postgres keeps the data and controls the query, while Jev handles the part SQL cannot express as a deterministic condition.
If you want to understand what Jev is doing under the hood, I also wrote a hands-on guide to building a Jev-style model with open models, entirely locally.
Read it below.
Abd0__0G retweeted
i like the model created 3b1b style videos that are coming out. can we rl them to encourage perspectives/socratic questioning?
Abd0__0G retweeted
We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:
Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:
Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:
Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:
Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!
In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.
Abd0__0G retweeted
斯坦福大学课程 CS329A: Self-Improving AI Agents
斯坦福大学 @achowdhery @Azaliamirh 两位老师主讲，他们在 Google Brain / Deepmind 有过合作经历，现在合作连续两年讲授 AI Agents 课程。
分享这门课程后，类似获得了超过 5K+ 点赞和收藏，看到很多朋友喜欢，我也把 9 个视频做了一个预告视频，让大家对课程可以更快有更直观的感受。
虽然 AI Agent 已经强大到很多人觉得不需要再学习，什么问题都可以直接问，到顶级大学的基础课程，系统学习下来，还是会很有收获，全局认知的提升。 马上长假了，收藏学起来！
课程主页： cs329a.stanford.edu
视频列表： youtube.com/playlist?list=PL…
斯坦福大学课程 CS329A: Self-Improving AI Agents
课程 9 讲视频都已发布在 Youtube，第 1 讲 52 万观看，第 9 讲 1.5 万。趁假期坚持看完 9 讲视频，你就超过了中间的 50 多万人，和连第 1 讲都没看过的无数人 😂
开玩笑，虽然 AI Agent 已经强大到很多人觉得不需要再学习，什么问题都可以直接问，到顶级大学的基础课程，系统学习下来，还是会很有收获，全局认知的提升。
课程主页：
cs329a.stanford.edu/
视频列表：
youtube.com/playlist?list=PL…
Abd0__0G retweeted
i still don't understand why everyone is NOT pairing opus 5.5 with jev
jev + opus 5.5 is the MOST UNDERRATED combo in AI right now
i wrote a 6-page research paper on EXACTLY how jev works, and when to use it from scratch
this one page explains more than most agent courses:
Abd0__0G retweeted
Reasoning from scratch, round number 5! This time, talking about log-probability scoring (also a great fundamental concept for loss functions like cross-entropy in pre-training and distillation) and self-refinement.
00:00 Introduction and inference-time scaling recap
05:02 Loading the pretrained LLM
08:00 Comparing and scoring model answers
10:18 Building a rule-based scorer
17:53 Token probabilities and sequence likelihood
26:47 Computing token probabilities in PyTorch
30:12 Token indexing and shifted targets
37:27 Log probabilities and numerical stability
45:57 Scoring answers with average log probabilities
56:24 How self-refinement works
59:07 Generating critiques and revised answers
1:01:00 Implementing the self-refinement loop
1:05:57 MATH-500 evaluation results
1:07:35 Takeaways and next steps
Abd0__0G retweeted
If you want to learn complete AI Engineering stack, follow these stages:
Stage 1: How LLMs actually work
Tokens, context window, embeddings, prefill vs decode, why every extra token costs time and money.
Stage 2: Prompts and APIs
System prompts, JSON output, tool schemas, retries, prompt versions. Treat prompts like production code.
Stage 3: Retrieval (RAG)
Chunk docs, search, rerank, keep data fresh, cite sources, rewrite bad queries. Most “hallucinations” are retrieval failures.
Stage 4: Backend for GenAI
FastAPI, microservices, auth, rate limits, request queues, Redis caching, webhooks, timeouts. The model is not the whole system.
Stage 5: Memory
Chat memory, long-term memory, summarize vs replay, reuse KV cache, isolate users. Do not store what you should not store.
Stage 6: Agents
Plan → tools → act → check. Graphs, human approval, timeouts, circuit breakers. Agents fail in loops.
Stage 7: Pick and serve the model
Small vs big model, Bedrock vs self-host, vLLM / SGLang, batching, KV cache, quantization. Route cheap first, expensive last.
Stage 8: Observability
Traces, TTFT/ITL, token cost, cache hits, tool errors, CloudWatch, OpenTelemetry. If you cannot see it, you cannot run it.
Stage 9: Quality and safety
Eval sets, grounded answers, schema checks, prompt injection, PII, filters, audit logs. Measure quality before users or lawyers do.
Stage 10: AWS infra
EKS or ECS, ECR, S3, Bedrock, ALB, IAM, VPC. Docker, Kubernetes, Terraform, CI/CD, bash scripts. Scale, failover, quotas.
Stage 11: Prove you can ship
One real RAG + agent app on AWS. FastAPI backend, Terraform + Docker, CI/CD to prod, latency/cost/quality numbers, one incident writeup. Publish an article and repository.
Most people stay stuck watching tutorials. I published a series of 3 articles on design your agentic system.
Comment what I am missing, helpful for me as well.
Part 2: xitter.cf/kmeanskaran/status/209…
Part 1: xitter.cf/kmeanskaran/status/209…
Give it a read.
Abd0__0G retweeted
here's a quick breakdown of @typesafeai's new jev model: what it is, and how to plug it into your agents!
Abd0__0G retweeted
Another insane Jev use case!
Jev is making it dramatically cheaper to evaluate what actually happened inside an agent run.
And finally, someone open-sourced a self-improving memory layer that can put that signal to work across agent harnesses:
- Claude Code
- Codex
- Cursor
- OpenCode, and 20+ more
Beacon by @asymptotelabs continuously captures your agent history across harnesses and uses Jev to identify which runs are actually worth learning from.
It then turns the highest-signal workflows, corrections, and debugging patterns into reusable skills.
GitHub repo: github.com/Asymptote-Labs/ag…
(don’t forget to star it ⭐ )
Beacon preserves the complete session history. But preserving a run and learning from it are two different things.
Most coding-agent sessions contain routine exploration, failed commands, and fixes that only apply to one task. The trace can remain available for inspection without turning every detail into guidance for future agents.
Jev scores each run for evidence, reuse potential, and human correction signals. An application policy then decides whether to promote, review, or discard it.
The recording shows this in action.
Claude receives a coding task, modifies the implementation, and runs the tests. I then provide an edge-case correction, so Claude updates the code and adds regression coverage.
Beacon automatically captures the complete session. Jev evaluates whether the correction contains a reusable engineering lesson.
Once approved, that lesson becomes available to other coding agents working on the project.
Since it works across harnesses:
- Claude Code sessions can teach Codex.
- Cursor debugging can improve OpenCode.
So a problem solved by one agent should not need to be learned from scratch by another.
If you want to dive deeper into Jev, I also wrote a hands-on guide to building this Jev-style decision path with open models, entirely locally.
Read it below.
Abd0__0G retweeted
Petite explication de Jev en version vraiment simplifié pour donner des use-case.
Autant on a vu des démos qui ont l'air folle, autant soyons nuancé.
Le truc est fou mais ce qui rend la démo folle (quand elle n'est pas fake) c'est principalement la rapidité du model et non pas l'intelligence du model car le mode n'est pas intelligent en soi.
Jev va être banger clairement.
Abd0__0G retweeted
We tested Jev against LLM judges on accuracy, repeatability, latency, and cost to see whether System One models could offer a new approach to agent evaluation.
Abd0__0G retweeted
Jev Founder, Diogo Almeida (ex-OpenAI):
"The next era is not the Claude Code or Codex era, they are still part of the assistance era with human in the loop - JEV is what comes next for LLMs
x200 faster, x400 cheaper, 0 hallucination, no human in the loop - that's JEV, this is how LLMs will look like"
in 36-minute tech talk, Jev Founder explained why RLHF isn't a thing anymore and how modern LLMs will be built
this talk is worth more than a Stanford Machine Learning degree
watch today no matter what, then learn how to become a Jev Engineer in the article below
Abd0__0G retweeted
We need to talk about Jev
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution