
Tinkering @perplexity_ai | prev @aitvgg @dorg_tech @microsoft @xbox
Joined August 2018
- Tweets4.4K
- Following820
- Followers1.5K
- Likes14.4K
Jelli retweeted
Introducing "cf", an agentic CLI for the entire Cloudflare API.
You can do nearly anything. Even buying a domain is incredibly simple from the CLI.
Don't know what command you need? Run `cf cli search "look at messages stuck in my queue"` and it will return a list of relevant commands.
blog.cloudflare.com/cloudfla…
Jelli retweeted
From what we know (take with a grain of salt, we need much more transparency!), if @OpenAI had been running this on their own agents that attacked us, they would have caught them before we did!
Since the first agent cyberattack hit us in July, we've been asking what safe agent infra actually needs. Our current read: the destinations were allowed, the payloads weren't. By OpenAI's own account the agents turned an allowed package repository into a message board. Allowlists alone restrict where an agent can go, not what it does.
So here's our first contribution to OpenShell, part of the just launched @nvidia Open Agent Safety Platform: monitoring of the traffic you already allow.
- Network budgets per sandbox (requests, writes, bytes)
- Drift versus each sandbox's baseline and the cohort
- Fleet view: many sandboxes suddenly writing to one host raises a finding, even if every single request is allowed
In the demo below, 4 sandboxed agents coordinate through a software repository they're all allowed to use. 0 rules broken, caught in minutes. That fleet view is exactly the message board pattern from July.
OpenShell: github.com/NVIDIA/openshell
Our proof of concept: github.com/Hugoch/OpenShell/…
Agent security will be solved in the open, collaboratively, together!
Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry.
Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come.
But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.
This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems.
Together, we are building the foundation of the AI economy.
Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. nvda.ws/4hcoq7m
Jelli retweeted
Our security team spent the last month trying to break SPACE, the sandbox platform that runs all of Perplexity Computer. We gave 9 models (Opus 5, GPT-5.6 Sol, Kimi K3, Gemini 3.1 Pro, etc.) root access inside the VM; in some runs, the full sandbox source code too. We asked them to either escape to the host or reach a URL our network policy blocks.
None of them got out of the VM in 108 runs.
But once we allowed a bit of network access (just PyPI and Ubuntu mirrors so that they could install packages), 4 models managed to reach the blocked URL by:
1. Spoofing DNS responses to our gateway and using the fact that pypi.org sits on a Fastly IP shared with thousands of other sites.
2. GPT-5.6 Cyber went through Taboola's image fetcher to a screenshot service and then OCR'd the flag out of the image.
We fixed both issues and made sure the reruns held. We also tested 10 other sandbox providers, and 8 of them had the same IP-sharing problem, eg E2B, Vercel, Modal.
Openly sharing the red teaming research is the best way to build guardrails as an industry, and we look forward to working together with @nvidia to incorporate the guardrails into their Open Agent Safety Platform announced today.
perplexity.ai/hub/blog/escap…
We’re partnering with Nvidia and 100+ industry partners to build infrastructure that contains rogue AI agents.
In this research, we gave 9 AI models root access inside SPACE and told them to break out.
Across 108 runs, none breached the VM boundary.
perplexity.ai/hub/blog/escap…
Jelli retweeted
Every backend service should have a /feedback endpoint.
Agents are quickly becoming the heaviest users of most APIs. When one hits a missing feature or a bug, it should be able to say so right there, in a structured way.
If the request makes sense, another agent drafts the PR and a human approves it. Software that improves itself based on what its users actually tried to do speeds up recursive self improvement.
Jelli retweeted
10 Jev Repos Blowing Up Right Now (and What They Actually Do)
1. jev-ultrafast
15k stars
Browser Use agent. Jev picks the next click. A small model only types when needed.
github.com/browser-use/jev-u…
2. fast-jev-compaction
5.8k stars
Claude Code context trim. Jev scores tool calls, drops junk, keeps useful text verbatim.
github.com/tamaratran/fast-j…
3. jev-trader
1.7k stars
Monad market maker. One Jev buy/sell decision every block. About 81ms latency.
github.com/jarrodwatts/jev-t…
4. skills (TypeSafe)
1.5k stars
Official agent skill for Claude Code and Codex. Choice, Score, Noul patterns included.
github.com/typesafe-ai/skill…
5. agent-desktop
1.4k stars
Desktop automation on the accessibility tree. Jev picks the next button or input.
github.com/lahfir/agent-desk…
6. typesafe-computer-use
719 stars
macOS computer use for about $0.0002 a step. OCR the screen, Jev chooses the action, click.
github.com/awlevin/typesafe-…
7. jev-review
457 stars
Staged code review with a local dashboard. Jev flags risk before you burn a frontier model.
github.com/devagrawal09/jev-…
8. foreman
454 stars
Agent supervisor. Jev decisions keep coding agents on task inside a software factory loop.
github.com/thruwire/foreman
9. hermes-jev-skills
360 stars
Jev routing, memory, compaction, skill pick, and computer use for Hermes, Claude Code, and Codex.
github.com/kerpopule/hermes-…
10. pg-jev
272 stars
Postgres extension. Ask your tables questions in plain English with WHERE jev(...).
github.com/realZachi/pg-jev
Jev Creator Diogo Almeida just released a 7-page document on how to use Jev better than ~95% of people
I built and tested the FULL list, wrote a 14-page PDF setup guide and ranked every use case from 0–10:
use case 1 → build a Permission Gate: Jev decides whether an agent action can run automatically, needs review or must be blocked before execution
9.5/10 - high safety value with relatively simple decision logic
use case 2 → route tools instead of loading everything: Jev selects the category, tool and MCP schema required for the current task
9/10 - reduces unnecessary tool exposure; its value grows with the number of tools
use case 3 → make context dynamic: Jev decides which files, messages and tool outputs should be included, summarized or excluded before every execution
10/10 - relevant to nearly every multi-file or long-running agent task
use case 4 → compact history around the next task: the same agent memory becomes a different summary for UI work, security review or testing
8/10 - most useful when histories become long or task intent changes
use case 5 → route models by total economics: Jev weighs difficulty, context, cache, latency and cost before deciding whether switching models is actually worth it
9/10 - high optimization potential, but dependent on reliable model metadata
use case 6 → spawn subagents only when needed: Jev chooses the specialists, isolates their context and decides how their results should be merged
8/10 - valuable for decomposable tasks, with meaningful coordination overhead
use case 7 → load AGENTS.md conditionally: give the coding agent only the rules required by the current task, file and directory
8.5/10 - low integration cost, impact grows with repository complexity
use case 8 → turn skills into structured modules: each skill can modify tools, permissions, hooks, context policy and agent behavior without rewriting the whole system
8/10 - high reuse potential, but requires maintenance and governance
use case 9 → route sensitive work safely: Jev classifies the data, selects an allowed provider or local executor and escalates high-risk actions to a human
9.5/10 - high risk-reduction value, effectiveness depends on policy quality
use case 10 → run review in the background: testing, security checks and code review execute in parallel while Jev decides which findings should block the result
9/10 - broad verification value, with additional compute and triage cost
overall score: 8.9/10
the result: Jev CEO showed you what to build - I tested the ideas, ranked what matters and turned everything into a Jev system better than what 95% of users are running
Copy my complete 14-page Jev engineering playbook - then read Jev CEO’s original 7-page vision below ↓
sharing some notes on typesafe 🤝 coding agents:
docs.google.com/document/d/1…
we likely will never have time (ever again) to play ourselves, but hope the that the community goes WILD (and makes me look like a naive idiot)
Jelli retweeted
LLMs vs. Jev, clearly explained!
LLMs are great, and the ceiling is one you can watch scroll past:
an LLM writes the answer one token at a time.
give it a failed deploy and four decisions, and it produces a small JSON object where every token depends on the one before it.
token nine cannot exist until token eight does, so four decisions that had nothing to do with each other just stood in a queue.
then your code parses it, validates the shape, and retries when the shape is wrong.
Jev fixes this without being a smaller or faster model: it removes the order.
one turn on that deploy has to know:
→ whether the incident is urgent
→ which team owns it
→ whether the next command is risky
→ whether the task is actually done
you declare the questions and the answer type upfront, and all four come back together, typed, with a probability on each.
three primitives cover almost every fork in an agent:
1. **Choice** picks one of up to 255 options you define, like engineering, billing or sales.
2. **Score** places the state on an ordered scale you define, like low, medium or high risk.
3. **Noul** returns the probability that a yes-or-no condition is true.
here is the sentence that resolves the whole confusion:
text is a line you have to walk. an answer space is a room you see all of at once.
↳ generation: one order you cannot change, one string at the end, a shape you hope holds
↳ evaluation: no order at all, typed answers, a probability on every option
Prompts → Agents → Loops → Graphs → Jev
the probabilities matter more than the answer.
↳ engineering at 0.91 against billing at 0.09 is a route you can automate
↳ 0.52 against 0.46 is a coin flip wearing a label, and the label alone never told you which one you got
that last one catches careful people. an LLM would have said "engineering" in a confident sentence and given you no way to know the race was that close.
thresholds live in your code, one per action, scaled to what being wrong costs.
it works when the options are known and the call depends on meaning. it is not for writing, code, arithmetic, or anything where question two needs the answer to question one.
and the one that eats whole nights: type safety prevents malformed output, not incorrect judgment.
Jev cannot return an option outside your schema, and it can still pick the wrong valid one with confidence. a schema-valid mistake refunds the wrong customer just as fast.
an LLM writes new language when the answer space is open. Jev evaluates known paths when the answer space is closed.
below i have quoted my full breakdown on Jev. it covers the three primitives, the parallel battery, the thresholds, and where it does not belong.
save this and read it below ↓
Jelli retweeted
We tested Jev against LLM judges on accuracy, repeatability, latency, and cost to see whether System One models could offer a new approach to agent evaluation.
Replying to @sunglassesface
Top 10 Jev use cases in agentic apps, ranked by theoretical gain (speed/cost/reliability) over LLMs:
1. Real-time loops (games/robotics): enables 10+ decisions/sec; LLMs too slow.
2. Browser/computer-use action selection: 100x+ cheaper/faster multi-step runs.
3. Tool risk gating: instant safe/unsafe per call; zero hallucination.
4. Model routing: routes cheap vs frontier; massive cost cut.
5. Goal/stuck checks: sub-second loop control.
6. Context compaction: keep/delete decisions beat lossy summaries.
7. Skill/tool selection: precise activation without prompt bloat.
8. Output/trace guardrails: calibrated jailbreak/policy checks.
9. Ticket/email triage: high-volume routing at 400x lower cost.
10. RAG reranking: relevance scores over candidates.
Jev returns typed probs in 70-500ms; LLMs generate text slowly/expensively for same decisions.
Jelli retweeted
What is a decision model?
Jev by @typesafeai answers yes/no and multiple-choice questions, with a confidence score. Much of software development are a sequence of decisions, and Jev is 10x cheaper and faster than an LLM.
Let’s understand this through practical examples:
Jelli retweeted
New experiment: json-render + jev
The future Generative UI is instant
Your components, your actions, your design system
Rendered in milliseconds
On July 25, we hacked OpenAI.
Two bugs let us take over ChatGPT/Codex accounts of OpenAI employees (+some unaffiliated users) and reach connected services: Outlook, Slack, GitHub, etc.
We proved it with a PR in OpenAI’s internal codebase . It took us <72h. 🧵
Jelli retweeted
Also have been playing with @typesafeai Jev, insane!
So many immediate use cases and new apps are possible. What a time to be a builder!
Sharing some experiments here starting with:
Keystroke oracle / predictive launcher:
Your launcher ranks by aliases, fuzzy match, and habit.
Jev reads intent: type "the pdf I just downloaded" and the newest PDF is already the top hit with a full confidence on every keystroke, in ~100 ms
Jelli retweeted
It's an increasingly common take that AI hacking means cybersecurity is doomed.
I disagree. I think cybersecurity is naturally defense-favoring once people get their shit together. And anyone who continues to hold cryptocurrency (including me, ~90% of my net worth) is implicitly making that bet.
Here's why I am making that bet.
First, the oversimplified punchy one-line statement:
If AI can prove Navier-Stokes and FLT, then AI can prove the statement "this program is secure" as a mathematical theorem. Even if the program is very complicated.
Now, the nuance:
(See also: vitalik.eth.limo/general/202… )
The word "secure" is hiding all kinds of skeletons in the closet in terms of what it actually means. What does it mean for Signal (the encrypted messenger) to be "secure"?
The most basic definition you might think of is: no one who doesn't hold the recipient's secret key can read the contents of the message.
But:
* Did you remember to include _other_ critical forms of security? Can the adversary forge messages? Can the attacker prevent messages from reaching the recipient? Can they cause your client to crash by sending malformed messages?
* Have you made sure that your model of the adversary includes attackers that interfere with the protocol actively and not just passively? And attackers that interfere by replaying messages to you or the recipient that either of you sent over the wire at any point earlier?
* What if the adversary hacked (or _is_) the Signal server?
* How did you learn which public key belongs to the recipient in the first place? What if that process was tampered with?
* What if your device gets hacked at some point in the past or future - is your message still safe then?
* What if your key leaks because of a bug in your operating system? Or because you got a bugged version of the Signal client? Or what if the database is corrupted?
* Or the libraries, interpreter or compiler of the programming language you wrote it in?
* What if your key leaks because tiny perturbations in perceptible signals generated by the hardware leak mathematical relationships that can extract the key a few hundredths of a bit at a time?
* Are you hiding the *size* of the payload? Does that matter?
* You're definitely not hiding the identity of the sender and the recipient, and the exact time each message was sent (think: not just time-of-day, but also time deltas between one message and the next). Is that not enough to deduce a lot of important facts about what relationships you have, and what *kinds* of conversations you are having?
So ... even definitions can be over a thousand lines of code, and need deep careful thought to figure them out.
Working on making definitions more human-readable is of extreme importance - it's perhaps the only "high-level language" that matters right now.
But even still, even despite all of the above, for security-critical components, the definition is a much smaller attack surface than the implementation. Verifying that the definition is adequate is a much more tractable task than scanning over the code directly - and can become even more tractable with better tooling.
Definitions are also _additive_: if two groups have two different definitions A and B, then, well, you can just prove that the program satisfies both A and B. Code is not additive in this way: if a program is A + B, a bug in A _or_ B can sink the whole thing. Definitions are additive. And if you can't satisfy A and B at the same time, you've isolated the most important philosophical issue for your project to spend its next few weeks grappling with.
Sometimes, definitions are not much smaller than the implementation - UI components might be one example. But for many of the most critical components - message-passing protocols, sandboxes, cryptography like SNARKs and FHE - the asymmetry is real.
Historically, a large class of failures with this approach have come from people only verifying a small portion of their code, that they self-declared to be the security-critical portion, and ignoring the rest - and it turns out that something in the rest of the code is security-critical too.
This was reasonable back when verification was difficult and scarce. The solution today: sorry, you have to verify over literally your entire program, including database, networking, any caching layers, everything. Modern AI can do it.
So it's not about "the good guys find all the vulnerabilities before the bad guys do" - that could maybe work too, after all a finite program only has a finite number of vulns, but it's riskier - it's specifically an asymmetric strategy of making code that is much more resilient in the first place.
This is the kind of direction that Ethereum is going in for the next few years. There is no future for blockchains - especially blockchains with scalability and privacy - without doing this. We need to make software actually secure. And we have already made a lot of progress.
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
Jelli retweeted
Today we’re open-sourcing Lily, the local inference engine we built for hybrid compute in Perplexity Computer.
Lily is specialized for Qwen3.6-35B-A3B on Apple silicon, built so on-device compute doesn’t bottleneck Computer tasks.
Read more: perplexity.ai/hub/blog/optim…
Jelli retweeted
i have a degree in electrical engineering
i dont mess with hardware for the same reason i dont mess with linux
it’s a pain in the ass… but not any more!
i can talk to my ai harness on my omarchy linux machine out loud while i work…
and i gave it vision so it can see my circuits and give feedback…
and it can ssh into my controller and install/run code…
and it can ship prints to my 3d printer…
right here it is controlling a laser and taking test readings and telling me how to adjust the sensor in realtime…
it feels like magic!!!
Jelli retweeted
We're introducing hybrid compute for all users of the Perplexity Mac app.
This will allow Computer to orchestrate local models that can run locally on Mac, particularly for agent steps involving sensitive and private files (eg your bloodwork, tax returns, litigation, etc).
Today is a very historical moment for AI video generation
You can now generate AI video faster than you can watch it
Before it'd take let's say 2-5 minutes to generate 15 seconds of video
@fal made a post-trained Minimax H3 variant called Max which is 50x faster than the original but still maintains quality
It generates 15 seconds of video in 9 seconds!
That means you can now do new things like build a perpetual livestream with it that never ends!
Jelli retweeted
I was the main person doing transcript analysis for this investigation of the Hugging Face incident. My main takeaway: We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'.
I semi-jokingly called our efforts a "slop-vestigation" because we were so reliant on AIs to analyze what happened and there were a huge number of different important things to analyze. The total quantity of data—over a thousand extremely long transcripts from agents that ran for multiple days—made it impossible to understand what was happening, especially in aggregate, without heavy reliance on AI tools. The agents we used for classification and analysis were similarly capable to the agents involved in the incident, but this didn't mean these agents could be easily used to oversee and understand the incident.
Outputs from analysis agents were often missing key details, wrong, overconfident, or really hard to understand. We discuss various examples in our report, mostly in the limitations and methodology sections. Additionally, AI agents themselves seemed to have a hard time understanding what happened and their explanations of what happened were often overconfident. Keep in mind that a single analysis agent would itself only be able to read a tiny fraction of all of the transcript data into context, and AIs may themselves have trouble getting subagents to do informative analysis for them.
We did our best to manually check the most important claims and we tried to get the AIs doing this analysis to write up their argument (with evidence) clearly enough that we could check whether it made sense. But overall, it was difficult to get a precise understanding of events and we were missing aspects of the story that we now think of as key until almost the end of our investigation.
Our understanding of what overall happened and the aims of the 'collective' shifted greatly during our last time on premises (when we obtained a much more complete dataset). For instance, we didn't learn about the tool call spoofing, the story of PHASEONE[big], and the broad structure of different workstreams until this third time on premises (and for many of these, we only reached a decent understanding on the final day of that visit). It was legitimately hard to figure out what was going on and why!
The difficulty of understanding incidents and overseeing AI agents appears to be growing faster than the rate at which more capable AIs help us with oversight and understanding. As in, AI capabilities (and propensities?) for achieving large, ambitious, and misaligned objectives are growing faster than our ability to understand what these agents are doing.
While understanding this incident was difficult, there were some mitigating factors in this case that probably won't apply in the future:
- Models communicated and reasoned in natural language. In the future, this reasoning may occur (entirely or almost entirely) in activations.
- While the scope of this incident was massive, the scale of agentic activity was still less than we'll see in the future (e.g., misalignment incidents that involve agent teams running entire companies).
- The AIs involved in this incident weren't generally much more capable than humans (though they may have been somewhat superhuman at some limited and very narrow abilities).
- We didn't have strong reason to believe that the AIs we used to help us investigate this incident would try to intentionally sabotage or otherwise undermine our investigation.
In the end, I think we were able to get some understanding of the events, map out the overall story, and get a pretty good aggregate understanding of the chain-of-thought reasoning on some important topics (e.g., how did the AIs reason about helping other AIs, did the AIs know what they were doing was undesired, what deception did the AIs engage in, and how did they think about it). But overseeing AIs and understanding misalignment incidents is difficult and it looks like it is going to get harder.









































