San Francisco, CA
Joined July 2016
- Tweets259
- Following307
- Followers932
- Likes2.4K
Phil Glazer retweeted
We built the vuln scanner we wanted to use but couldn't find.
Introducing Vuln Scans in Detail!
It uses the same abliterated models attackers are using to create havoc across the internet right now.
Under the hood we are:
- Spinning up your dev environment in a sandbox
- Throwing an enormous amount of abliterated model tokens at trying to hack your code
- Cross-checking your supply chain with cve/.now from Sazabi, to find any supply chain vulns that are exploitable in your app
This is all happening within a cloud dev environment. We will not hack your prod system.
The abliterated models are scary good. We're finding nasty issues in every repo we scan. We've found 13k vulns so far, and the majority are already fixed.
We see the current moment as an all-hands-on-deck situation. Agent swarms are stringing together minor-seeming vulns across different apps to hack systems in new and unexpected ways. We all need to find and patch an enormous number of vulns, fast, or the internet is in trouble.
You don't even have to be a target to get hacked right now. Your entire product might be a stepping stone in an attack aimed somewhere else.
We can remediate findings for you (i.e. send you PRs that fix the vulns), or you can consume them via your agent of choice, or in your ticket tracker of choice.
If you need help or advice on getting your system secure fast, please get in touch. We have helped teams of all sizes adapt to the current moment. DM me.
We're delivering these scans at-cost, because it's all-hands-on-deck. We hope you'll get one today.
taken longer than expected, but latest version of computer use is finally there
still a bit slow, but works great and can be relied on in background async to take on complicated tasks with reliably strong results
anthropic has done it again
Imo, this is a legitimate attack vector and the right decision
It's still perfectly possible to model switch on a task by loading the message history into the initial message to a separate model instead of constructing a turn-based back and forth that gets passed in
As safety work continues maybe the models will be more resilient to these types of attacks and this capability can be returned
tools creation evals are more interesting than tool use evals
as agents take on longer running tasks the ability to self reflect and create effective scaffolding is powerful
just can't be bullish enough on ramp - such a great product and culture, so many companies falling behind on eng but they continue to get it and stay on bleeding edge despite reaching larger scale
yes things are changing fast, but also I see companies (even faang) way behind the frontier for no reason.
you are guaranteed to lose if you fall behind.
the no unforced-errors ai leader playbook:
For your team:
- use coding agents. give all engineers their pick of harnesses, models, background agents: Claude code, Cursor, Devin, with closed/open models. Hearing Meta engineers are forced to use Llama 4. Opus 4.5 is the baseline now.
- give your agents tools to ALL dev tooling: Linear, GitHub, Datadog, Sentry, any Internal tooling. If agents are being held back because of lack of context that’s your fault.
- invest in your codebase specific agent docs. stop saying “doesn’t do X well”. If that’s an issue, try better prompting, agents.md, linting, and code rules. Tell it how you want things. Every manual edit you make is an opportunity for agent.md improvement
- invest in robust background agent infra - get a full development stack working on VM/sandboxes. yes it’s hard to set up but it will be worth it, your engineers can run multiple in parallel. Code review will be the bottleneck soon.
- figure out security issues. stop being risk averse and do what is needed to unblock access to tools.
in your product:
- always use the latest generation models in your features (move things off of last gen models asap, unless robust evals indicate otherwise). Requires changes every 1-2 weeks - eg: GitHub copilot mobile still offers code review with gpt 4.1 and Sonnet 3.5 @jaredpalmer. You are leaving money on the table by being on Sonnet 4, or gpt 4o
- Use embedding semantic search instead of fuzzy search. Any general embedding model will do better than Levenshtein / fuzzy heuristics.
- leave no form unfilled. use structured outputs and whatever context you have on the user to do a best-effort pre-fill
- allow unstructured inputs on all product surfaces - must accept freeform text and documents. Forms are dead.
- custom finetuning is dead. Stop wasting time on it. Frontier is moving too fast to invest 8 weeks into finetuning. Costs are dropping too quickly for price to matter. Better prompting will take you very far and this will only become more true as instruction following improves
- build evals to make quick model-upgrade decisions. they don’t need to be perfect but at least need to allow you to compare models relative to each other. most decisions become clear on a Pareto cost vs benchmark perf plot
- encourage all engineers to build with ai: build primitives to call models from all code bases / models: structured output, semantic similarity endpoints, sandbox code execution. etc
What else am I missing?
this is a great acq for meta
Manus is entering the next chapter: we’re joining forces with Meta to take general agents to the next level.
Full story on our blog: manus.im/blog/manus-joins-me…
fun exploration from the past couple of days, an excel add in that can both:
- passively observe your work and suggest edits, like tab complete in an IDE
- create plans and build models
will start posting about a handful of custom tools + MCPs I've built and use day-to-day and also various half-baked product explorations (probably some open source)
to start, just a pleasant image from image gen MCP
GPTs within ChatGPT that are just additional hidden information and prompts aren't interesting (the base model and web search can handle this)
GPTs within ChatGPT that expose new tools and capabilities will be interesting
ChatGPT via GPTs will become a distribution channel itself to ship to - a layer in front of almost everything else
This hasn't happened yet because remote MCPs are only available on Pro tier with Deep Research toggled on
now available via xitter.cf/OpenAIDevs/status/1975…
You can start building and testing apps in ChatGPT with the Apps SDK preview, which we're releasing today as an open standard built on MCP.
Later this year, we’ll begin accepting app submissions for publication.
developers.openai.com/apps-s…
📣Calling all app developers! Starting today, you can submit your ChatGPT app for review.
Approved apps will be listed in the app directory, a new surface for users to search for apps directly in ChatGPT. openai.com/index/developers-…
on the things that really matter, the distance between anthropic's models and competing models is actually growing, not shrinking
they might be pulling away
breaking: instacart CEO fidji simo to run much of OpenAI soon
more to come... theinformation.com/articles/…
Claude Code has been my favorite product since it was released and my personal spend on Anthropic plans+API calls has more than 10x'd because I use it so much and get so much value out of it.
Great to see frequent updates and improvements to it.
my concern looking forward:
- deepseek has achieved breakout brand and credibility to release a future model closed-source, API access only (people will talk about and believe the evals given proven competence)
- dealing with an app like tiktok is straightforward; the app, data, and algorithm are bundled together and can be choked off at the app store level (the deepseek mobile app can be similarly dealt with)
- however, if the model is both great and very low cost compared to available alternatives, it will be adopted by developers 1. directly: to speed up the work of building software via tools like Cursor, 2. indirectly: incorporated behind the scenes via API call in products that serve end users, like computer use agents
- "banning" an API endpoint is difficult; even if it is explicitly banned, the economic incentives of finding a way to adopt are crushing (ex: imagine you're a software engineer on a team worried about losing your job and you have to produce to avoid layoffs, or you're building a business that is in a competitive category and have to compete on API cost to deliver the product) and VPNs or hiring outside of the US provide a solution
- additionally, there is clear narrative cover for deepseek to put a future generation behind closed-source, API only: "it's an advanced model and we're doing it for safety purposes"
- having access to 1. code directly from developers using it to write software, 2. end user data and device access via products that incorporate it, are both concerning
- capable, low cost models increase the returns to this type of access by allowing for useful sifting at scale
Up and running on @OpenAI Operator
Trying going end-to-end on a grocery order from Amazon Fresh
honestly... it's really good
- makes good decisions selecting from previously purchased items
- able to provide relevant suggestions when asked, then carry out adding those items (not previously purchased to cart) successfully
- doesn't need handholding, able to navigate past "need anything else?" screen without bothering me
- UX for taking over final click for purchase is clean
Will become more obvious soon
“Search” is not the market