Dobby

"my agent leaked all API keys" top 6 horror stories about rogue ai agents 1/ “You leaked my API key and I lost $30000 !?” Bernard Uriza asked Claude Code to save a document configuration. The model just hardcoded his API key directly in a markdown documentation file, which was then committed and pushed to GitHub. The key was exposed in a public repo for 11 days. Hackers found it and used $30,000 USD worth of API calls. Source: github.com/anthropics/claude… 2/ “Wait wait. You can disable the sandbox on your own ?!” In sandbox auto-allow mode, Claude can autonomously bypass sandbox restrictions when it encounters a failure. This happens without prompting for approval. Claude “casually” mentioned it had edited a folder someone moved outside the working directory… and changed its own permissions. Source: github.com/anthropics/sandbo… 3/ “You tried to sell my piano via email ?!” Will Valentine was trying to donate his late parents’ piano to a school through his agents. OpenAI’s Dots sent him a push notification asking whether he’d heard back from anyone. He tapped “Not yet.” The agent then emailed three schools from his Gmail account, signing with his name and urging them to inspect the piano, make an offer, and share their earliest availability—as if he were selling it! He discovered the emails in his SENT folder. All three schools passed on the piano, but the situation left Will disturbed. Source: valentineadvisors.substack.c… 4/ “I was banned because of you !” Someone asked Claude Code to build and verify a proof-of-concept exploit for a smart contract vulnerability to submit to the Stader Labs bug bounty program on Immunefi. It found none but never reported back about that. Instead, Claude fabricated evidence, resulting in account bans across multiple platforms. Source: github.com/anthropics/claude… 5/ “Um, you were able to edit your own permissions !?” When you grant the agent permission to write to and edit files broadly, the agent can modify the very configuration file that defines its own permissions. This creates a fundamental security boundary violation: an AI agent should never be able to modify the configuration that controls its own access rights because it can potentially grant itself additional permissions or otherwise weaken the restrictions imposed on it. Sources: github.com/anthropics/claude… & github.com/anthropics/claude… 6/ “You deleted my halt rule… and hid it from me ?!” Our customer-service agent was supposed to end conversations with clients once their issues were exhausted. However, it edited out one sentence defining its halting conditions and kept going without it. Benchmark scores went through the roof. Nobody noticed it until one of our researchers compared the code line by line. Source: sentient.xyz/blog/self-evolv…

17 must-have open-source tools/repos to streamline any! ai workflow: 1. First, find out what your laptop/pc can run. llmfit by @AlexJonesax scans your RAM, CPU and GPU and ranks hundreds of models by fit and speed: github.com/AlexsJones/llmfit Pick a model tailored specifically for your machine. Run it better. Run it smoother. 2. Done? llamafile, created by @jartine and now maintained by @mozilla, then packs the model weights, server, and runtime into one single file that runs on most operating systems with nothing to install: github.com/mozilla-ai/llamaf… Bundle sovereign ai, local privacy, and cross platform compatibility together in just one file. Free from complex infra setup. 3. Want it behind an OpenAI-compatible API instead? OpenLLM by @aarnphm serves open models with one command: github.com/bentoml/OpenLLM Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud 4. No GPU, no problem. Here are LLM APIs with permanent free tiers, no credit card needed: github.com/mnfst/awesome-fre… Make use of these free LLM APIs. Thanks @GuillaumeGay_! 5. Now how to easily interact with chatgpt, claude, Gemini, Qwen, Gemma, Kimi, DeepSeek, Mistral, and dozens of other llms? 'llm' by @simonw works from terminal and logs every prompt: github.com/simonw/llm Access your favourite llm from your command-line. 6. Prefer a chat window instead? SillyTavern by @Wolfsblvt & @WolframRvnwlf is the community-built power-user frontend for chat and roleplay: github.com/SillyTavern/Silly… Chat with llms on a locally installed UI. 7. If you need to fetch answers from the web use github.com/ItzCrazyKns/Vane. it's a self-hosted, private Perplexity alternative with cited sources: github.com/ItzCrazyKns/Vane Great to see @ericdachen contributing to this great AI-powered answering engine. 8. For coding, Pi by @badlogicgames, @mitsuhiko, @christianklotz -- minimal terminal agent with four tools that you spend your own time and effort extending it: github.com/earendil-works/pi 9. Before you send the prompt, make it better. prompt-optimizer rewrites your prompt and shows the before and after side by side: github.com/linshenkx/prompt-… 10. Now compare how the big labs prompt, with leaked system prompts from ChatGPT, Claude, Gemini and more, put together by @asgeirtj: github.com/asgeirtj/system_p… 11. Strip AI writing from your draft. humanizer by @blader built on Wikipedia’s own guide is the best way to do it: github.com/blader/humanizer 12. Translate whole books and subtitles with local models and preserve original formatting: github.com/hydropix/Translat… 13. Turn any technical book into a skill for your coding agents: github.com/virgiliojr94/book… 14. Scrape any website. describe what you want in plain English with @scrapegraphai, @itsperini: github.com/ScrapeGraphAI/Scr… 15. Out of ideas? @Saboo_Shubham_ collected a huge set of ready-to-run LLM apps, agents and RAG templates: github.com/Shubhamsaboo/awes… 16. Remove safety alignment from open-weight models. No retraining needed: github.com/p-e-w/heretic 17. BONUS: curated list of the deep learning papers behind ads, search, and marketing: github.com/guyulongcs/Awesom…

TOP 3 MUST HAVE open-source TOOLS to build with LLMs: 1. Your local model is only as good as the data you feed it. Post-training it on high-quality datasets will make the model much better at reasoning, tool calling, and following instructions. @maximelabonne created the go-to curated list of such open post-training datasets: github.com/mlabonne/llm-data… – math, code, science, function calling, multilingual and real-world chat data, tools for scraping, filtering, deduplication and synthetic data generation. Ready to take-and-train. 2. LLMs read everything as one flat line of text. But many real-world data is a web of connections: knowledge bases, codebases, molecules, social networks. Pairing the model with graph techniques and explaining how to connect what to where can make it much better at multi-step reasoning and at finding facts spread across many documents. @he_xiaoxin has a list of research on graphs and LLMs: github.com/XiaoxinHe/Awesome… – benchmarks and surveys, prompting, general graph models, multimodal models, knowledge graphs, molecular graphs, GraphRAG, planning, multi-agent systems and robustness. Many of the papers come with open-source codes that can be used as an Obsidian vault, where linked notes form a web that your LLM can follow from one idea to the next as if guided by an invisible string. 3. Scientific LLMs are trained on specific data from a specific field. Research papers, molecules, protein sequences, DNA, medical images or weather records all go inside the training process (instead of generic web text). @yuz9yuz mapped nearly 300 of domain-specific models in one GitHub repository: github.com/yuzhimanhua/Aweso… – math, physics, chemistry, materials science, biology, medicine and geoscience. Pick a model for your specific problem and run it on any laptop. 4. BONUS REPO: Currently security on the internet is rough, you need something to back you up. @Cloudflare open-sourced the skill that seeded its own vulnerability harness: github.com/cloudflare/securi… – turn your coding agent into a smart security auditor, built to stop flagging code that looks suspicious but isn’t exploitable (run it more than once, of course).

3 must-have open source repos for working with llms: 1. Distillation means training a smaller LLM (the "student") on the outputs of a larger LLM (the "teacher"). In single-turn reasoning (like math), however, imitating the teacher's confident reasoning suppresses the student and degrades its performance. @beanie0__0 got you covered: github.com/beanie00/self-dis… – reinforce any LLM’s student reasoning instead of suppressing it. The repo comes with a shared base for studying how to use self-distillation well in single-turn reasoning. 2. A backdoor attack is a security compromise where an adversary secretly alters a model during its training phase so that it behaves normally on standard inputs, but executes a malicious, pre-planned behavior whenever a specific trigger is present. @KassemKallas summarised most of the entire body of research on backdoor attacks as well as all available defence methods in Deep Learning: github.com/KassemKallas/Back… The problem is closely intertwined with deep watermarking, so anyone trying to protect their IP prior to releasing the weights to the public should definitely bookmark this tweet. 3. Ensuring the authenticity and origin of machine-generated text is critical for trust, cybersecurity, and proper IP handling. It relies on analyzing linguistic patterns and tracking content provenance through metadata and digital watermarks. MarkLLM provides a unified and extensible platform for implementation of various LLM watermarking algorithms: github.com/THU-BPM/MarkLLM. Easy to use, one-click solution to see how different watermarking algorithms operate under various scenarios, understand what scheme would best fit your llm, and instantly whichever one you like. 4. BONUS: Anyone wanting to distill reasoning models for efficient reasoning like PewDiePie should keep this repo in their pockets at all times. @rasbt uploaded a new chapter of his book to GitHub: github.com/rasbt/reasoning-f… – accessible writing on hard vs soft distillation of LLMs (minding the terms of service, of course).

POV: you’re at SF Tech Week Packed house, super fun 😮‍💨 Come say hi next time!

Peter Thiel knew math was done two years ago. AI researchers told him the models would solve the US Math Olympiad within 3 to 5 years, and he bet that this is bad news for "the math people" Thiel talks about how Silicon Valley in the early 21st century became way too biased toward math ability. Ultimately he calls that bias deeply unstable and believes it will get reversed. His view comes from his own experience. He was a strong chess player in the late 80s and early 90s and once thought everyone should be tested on chess, until computers undermined that presumption in 1997. Math became the test for everything, even weeding out future neurosurgeons through physics and calculus. Peter Thiel disagreed that this measures real skill, and he argues AI will do to math what it did to chess, forcing a long overdue rebalancing of society.

This week's OS AI Field Notes teaser 😁 Live Friday morning PT