Pinned Tweet
Jev costs $0.044 per 1,000 judgements.
GPT-6 Astra costs $12.182. But cheaper isn't always better: Jev matches on simple evidence checks and trails on harder reasoning.
Best setup? Use both.
analyticsvidhya.com/blog/202β¦ #AIEvals #LLM
Closed-source app. No docs. One AI agent, 7 minutes, $1.83 π΅οΈ
We tested REA, an open-source MCP server for reverse engineering. First we guessed how Obsidianβs quick-switcher ranks results by hand, then let the agent inspect the code.
π― It found the scoring function inside a 3.9MB minified file
β
Our hand-made prediction was right on priority order, even the exact βa word boundary = 10 positionsβ trade-off
π It also found 4 hidden mechanisms we couldnβt see from outside
β οΈ Its written formula had an off-by-one that only showed up when checked against its own data
Lesson: predict first, then verify.
Full breakdown π analyticsvidhya.com/blog/202β¦
FDE or AI Engineer?
Both build with AI.
The difference: where you sit, and who owns the system after it ships.
New to the FDE path? Start with our free course: analyticsvidhya.com/courses/β¦ #AICareers #AIEngineer
Your RAG bot can still make things up, even with the source text right in front of it π«
Fix: let the LLM write 5 answers, then let JEV (a tiny, cheap checker) ask 4 yes/no questions about each: supported? on-topic? invented details? contradictions?
π« Blocked all 16 planted wrong answers
πΈ 56 checks cost about $0.001
β οΈ But it also blocked ~1/3 of the good answers
π€· Writing 5 drafts didn't help, the first one was already right
Start with 1 draft + 1 JEV check, then measure before scaling up.
Full breakdown π analyticsvidhya.com/blog/202β¦
We put Gemini 4 Argon through 4 real tasks π§ͺ
π¬ 8-second product video: nailed it
π Working booking website in Canvas: nailed it
π "Conversion fell, roll it back": it spotted that both visitor groups actually improved
π 8 conflicting launch claims: all labeled correctly, but one invented phrase slipped into the summary π
Great at catching traps, still needs a proofread.
Full review π analyticsvidhya.com/blog/202β¦
OpenAI's new API doesn't chat, it decides π―
β
Yes/no with probability
π§ Pick from options, with odds
π Grade on a rubric
10x faster, $0.10 per 1M tokens, text and images. And it knows when it's unsure: a half-markets, half-regulation headline scored just 52% π
analyticsvidhya.com/blog/202β¦
Your GPU is 99% idle while generating tokens. Yes, really π³
LLM decode is stuck waiting on memory, not math. The fix is batching:
π Static: waits for the slowest request, one 500-token reply holds up everyone
β³ Dynamic: starts sooner, still waits to finish
β‘ Continuous: swaps in new requests every single token
Our simulator: continuous batching finished the same work in 41% fewer steps. A 30-token request waited 320 steps on static, exactly 30 on continuous.
Full breakdown π analyticsvidhya.com/blog/202β¦
Weβre giving AI agents our most sensitive data and the power to act on our behalf.
Can anyone explain why a model did what it did? Nope. π
Hereβs the plan: trust the model the least. A thread π§΅
The checklist:
π Use different models, none grades its own homework
ποΈ Log everything in tamper-proof, human-readable form
π§ͺ Test the failures, not just the wins
π Keep an emergency brake that always works
π’ When things break, tell everyone what failed
Your RAG system confidently answers from the wrong documents. Here's the fix π§
Agentic Corrective RAG adds a safety net:
π§ An LLM grades every retrieved chunk: relevant or not
π Irrelevant? It rewrites the query and searches the web for live info
π§© Built in LangGraph with Chroma + Tavily, nodes and edges you can follow
Fewer hallucinations, fresher answers.
Full guide π analyticsvidhya.com/blog/202β¦
Nano Banana 2.1 is here and free for everyone π
We threw 4 tests at it:
π Bar chart with exact values: proportions came out spot on
π§± Clay-style RAG explainer: clean, though one of the "two books" went missing
πΌοΈ Hawa Mahal as a linocut print: gorgeous style, but it deleted the busy street
π» Debugging comic: consistent character, unreadable typo
Great for charts and comics. Always check the details you can't afford to lose.
Full review π analyticsvidhya.com/blog/202β¦
Your LLM judge says "0.995 confident" on every single answer. Useful? Not really π¬
Jev is a tiny judge model with honest probabilities:
πΈ 277x cheaper and 13x faster than GPT-6 Astra on the CMU study
π― Matches it on fact-checking, falls apart on math, code and logic
πͺ Smart trick: trust Jev when it's confident, send only the unsure cases to a bigger LLM. 93.4% accuracy at 41% of the cost
In our test, the one case Jev got wrong was also its lowest-confidence call.
Full comparison π analyticsvidhya.com/blog/202β¦
Most agent bugs aren't in the answer, they're in the dozens of tiny decisions before it π―
Jev turns those into typed choices with probabilities. 10 GitHub repos show what you can build with it:
π Browser agents that pick actions from a table
βοΈ Context pruning for coding agents
π A code reviewer that scores severity
π§ A router that sends easy prompts to cheaper models
Full list + repos π analyticsvidhya.com/blog/202β¦
We built a real enterprise support agent with Claude Fable 5.1 ποΈ
π΅οΈ 8 prompts: investigation layer, policy RAG, human approval gates, security attacks, multi-tenancy
π« Refund limits stayed hardcoded, never an LLM call
π Tried a prompt injection hidden in customer text, it stayed just text
π Planted a bug, Claude Code couldn't reproduce it and said so
Full build π analyticsvidhya.com/blog/202β¦
Tired of retyping the same setup instructions into Claude Code? Turn them into a slash command πͺ
π Mark a skill disable-model-invocation: true so only you can trigger it, never Claude
π Scope it personal, project, or just one subfolder
π§© Example: /new_repo scaffolds backend/frontend/README in one shot
One markdown file, one word, done.
Full guide π analyticsvidhya.com/blog/202β¦


