AI-pilled | AI Analyst

Paris, France
Joined February 2015
Benjamin Polge retweeted
New Cohere paper finds that a team of coding agents gets the most from extra agents when they only research and plan, and cheap models can do that job for a strong lead at far lower cost. A team of coding agents beats 1 agent working alone, but costs 3 to 7 times as much. More agents can also mean duplicate work and extra code to merge. The researchers built a lead agent that hands subtasks to workers, then varied the workers' role, model and count. On ProgramBench, agents rebuild a whole program from just a working executable and its docs. Workers that only plan matched the reward of workers used in every stage, at half the cost. A strong GPT-5.6 Sol lead with 16 cheap Luna planners came within 1.4 points of an all-Sol team at 70% lower cost. On tasks like this, put your strongest model in the lead and use cheap models only for planning, since cheap coders lowered reward.
45
14
4
135
9,082
Benjamin Polge retweeted
MiniMax M3.1 Flash: Space Performance + Bunny Speed 🚀🐰 Yes, Space Bunny - which has topped global leaderboards with 200T+ tokens - was an earlier version of M3.1 Flash. 💰Redefining the performance economics of agentic AI. M3.1 Flash achieves frontier-level intelligence across key tasks such as Cowork (Automation-Bench: 62%; ALE: 30%), Coding (DeepSWE: 74%) and Visual Reasoning (BabyVision: 91%), at 1/3 the cost of M3 and 3x the speed. API pricing (flat rate): $0.15 / $0.60 per 1M input/output tokens, with cache reads at $0.015 / 1M tokens. Model weights will arrive on Oct 23. API: platform.minimax.io M Plan: minimax.io/m-plan MiniMax Code for desktop: agent.minimax.io/download
375
580
570
6,616
931,156
Same list price, different bill. We tested OpenAI and Anthropic models on 4 enterprise tasks at JDN. Claude Sonnet 5.5 and GPT-6.1 Sol cost the same per token, yet Sonnet came out up to 1.7x pricier. Main culprit: Claude's tokenizer counts 46-63% more tokens for the same text.
1
45
Benjamin Polge retweeted
Today, we're publishing two new findings from inside Anthropic: how work and hiring are changing at a company at the frontier of AI adoption, and how Anthropic staff expect AI to change the economy. anthropic.com/research/expec…
69
140
67
1,423
272,161
Benjamin Polge retweeted
Anthropic’s new report is a wake-up call for job seekers. TL;DR: • Junior tech hires: 27% → 9%. • AI skills cited 3–4x more in hiring reviews • Build your own automations. Show the receipts • Judgment > narrow expertise • Autonomy > following instructions Your best credential is not what you know, it’s how much you can automate yourself.
14
16
1
568
97,917
Benjamin Polge retweeted
the life before this tweet:
today we launched ChatGPT. try talking with it here: chat.openai.com
137
23,383
93
291,797
3,218,292
Benjamin Polge retweeted
What can large-scale AI models bring to the growing complexity of particle physics? @CERN and Mistral join forces to explore this question. The ambition: a single Large Physics Model, trained on billions of simulated collisions.
132
325
101
2,418
107,626
Benjamin Polge retweeted
My 50 tips for using AI: 1. Data, data, data. Feed your AI as much good quality data as possible to fill in the context it hasn't got between its training data and what you want from it. With no context you'll get very generic responses. 2. Use chat history and chat memory. Build this up over time. Brain dump on it regularly. Have dedicated sessions to query the memory, fix it, and add to it. 3. Add custom instructions to fine-tune how you want it to respond and act, including the way it writes. 4. Clean up your local workspace. Add readmes to each folder to explain its context. Don't include obvious things any AI would be able to figure out. 5. Add an agents .md file to the root of your workspace. Again include context gaps only or regularly occurring mistakes. Keep it short. Use it like an if-then-else and point to an agent-docs folder with more instructions and context. 6. Use some form of task management tool, like Linear. Built-in tools are still pretty awful from my experience. Make sure to keep your task (issues) in a project. For more granular steps include checklists in your issues. 7. Use voice. Unless you're the fastest typer in the world, voice is always going to be quicker. It also lets you speak your mind. ChatGPT has great dictation, others are OK, but Copilot's particularly awful for some reason. 8. Use personal AI assistants or one session that can control all other sessions. The former is still early but is the end game for AI, get used to them. ChatGPT is really the only one that can do the latter properly at the moment. 9. When using your voice use a condenser mic that lets you talk incredibly quietly, thus not disturbing the people around you. 10. Give your AI various things, such as a master plan, principles, and an identity. 11. Ask your AI to keep a changelog and audit. 12. For schedules plan a way to have a backup so if you run out of tokens another AI can pick them up. 13. Use your AI on your mobile. It's surprising how much you can actually do. I can see a lot of us will go down to fewer and fewer and smaller and smaller screens. 14. Connect any and all relevant apps. I recommend GitHub for code work, Linear for tasks & project management, Notion for wiki and KB, Google Drive for storage and workspaces, Neon for the database, and perhaps something for video generation + image generation for Claude. 15. Use GitHub for coding. No you don't need to do any coding yourself. All control is managed by the AI. 16. And just don't be afraid of coding or building apps. You now have the power to build any app and tool you wish. There is no need to use proprietary interfaces anymore if you don't have to. 17. Ask your AI to set a schedule to review and research updates on all the tools and systems you use in your workspace. Ask it to find best practises, model releases, system card, updates, and prompting guides and ask it to suggest changes and improvements in your workspace after its review. 18. Stop doing admin. Give it to your AI. The only thing I would be wary of is managing finances. Though if you're in an area of the world that has a finance plugin such as ChatGPT in the US, use that. Other than that, from my experience, an AI can do everything else. 19. Markdown files are good but I'd recommend HTML files for explaining stuff. It's just better on the eye and gives richer content 20. Stop reading and using influencers prompts and guides (not including this one, of course 😜). They have no idea about you. You need to figure that out. 21. If you have any expertise, you are still better at that than people without that expertise so take advantage of that. AI does not make you an expert of something you are not, at least not yet or not instantly. 22. Though I'd argue coding and data analytics are solved, potentially a few other areas as well. 23. Stop paying for third-party tools and systems when AI can do that job. Ask your AI first. 24. Give it small things to do at first, then build up and up. Ask it to verify its output and check evidence. Ask it to ask you questions. 25. Give it examples and counterexamples; screenshots, files, and photos. If you're designing websites or tools, produce mockups with a relevant image generator. 26. There's no real need to start a new session nowadays but certainly create new sessions for new projects and put the AI in a new folder with project specific instructions so that session solely works on that. Use it like a worker. Keep it focused. 27. Never rely on a single model. You should have at least two chatbot plans. Use for second opinions and a backup for when one has an outage. Or just to run more work. 28. Document things you want the AI to permanently remember. Don't rely on memory and history for that. 29. Ask your agent to check your knowledge base and workspace for any conflicting info and work with it to fix it. 30. Make sure your knowledge base is easy to search, including the file names, with little explanation. 31. Build systems that last. If you're running repeatable tasks consider building a skill. I like to call these specialists. 32. Keep files and workspaces fresh. Remove any stale data. 33. Explain what "finished" and "complete" mean. 34. Prepare for mistakes. Always back up or have a way to recover. 35. AI gives you time to prototype over and over before you find your final solution. Long gone are the days of risking time on one idea for weeks or months. You can now test multiple ideas that only waste hours or days. You only need one of these to work. 36. When working with agents for the first time, start with all restrictions on and slowly but surely give it more and more freedom. They work better with fewer restrictions. 37. Only use AI if you need intelligence. Else use automation. Remember AI is still not 100% accurate, or 100% hallucination-free. Nor is it cheap. Tokens are precious. 38. So ask the AI to use scripts, formulas, and schedules where it can. 39. Schedules allow you to run your AI while you're not even at your desk, such as overnight, at weekends, and on all holidays. Set exactly what you want it to do: some AI can set fairly dynamic schedules (such as one piece of work during the week and a different piece of work during the weekend) or are reactive depending on the results 40. Ask your AI to run parallel sessions to get more work done quicker. Or if you're not using a personal AI assistant or orchestration, run multiple sessions at once yourself. 41. Use desktop apps for your AI. They give you so much more than the web versions. 42. Speak to your AI a lot. Most AI are powerful enough out of the box that a lot of prompting techniques are now dead. Talk to it like you talk to a person to get the best out of it. 43. AI is like a work colleague, be it one without context. Treat it as such. 44. Get it to question your assumptions, identify weaknesses, and present alternatives. Ask for pushback 45. Always fact-check important claims and ask for sources and dates. 46. Check what it can actually access. Don't presume what it tells you is correct. 47. No need to wait for the perfect use case. Just give it a problem right in front of you now. 48. Watch for silent failures such as schedules breaking. 49. This advice will get outdated very quickly. Always keep up to date with the latest news and techniques. 50. My top tip is just to start using it. The earlier the better. You'll start to understand how they work and their pros and cons. You'll also start to build up context, memories, and history, which are hugely vital. We're at the stage where we are now delegators and supervisors, get the AI to do most of the work as long as you give it good instructions at the start, check it thoroughly at the end and guide it in the middle.
46
120
15
991
58,353
Benjamin Polge retweeted
We've been working hard on making the Claude Code desktop app great for running lots of sessions side by side. It's now much quicker to start sessions, easier to split work across a few, and you can always see what each of them is doing. Here’s what's new:
938
908
778
20,575
2,246,823
This graph is all you need to know
The cost is dropping faster than any transformative technology in history, 54 times faster than electricity, and it's been dropping at this same rate since GPT-3.
66
288
25
3,903
403,228
Benjamin Polge retweeted
Today we are rolling out Gemini 4 Argon globally in the Gemini API, to Pro + Ultra in Gemini App, and Ultra subs in Antigravity! Argon is a huge step forward for Gemini, and a labor of love. This model has been battle tested by over 100,000 people at Google over the last month, with so much new innovation, ambition, and hard work baked into this release. The team has been heads down over the last 6 months working to make this possible across the stack (research, infra, safety, eng, etc). We rolled the model out to cyber defenders as part of our fairwind program and early access customers over the last 2 weeks, it has been amazing to see and talk to folks who are now using this model daily. Seeing the models writing capability, raw knowledge and persistence for coding, paired with new features like 1M token output when the model needs to work more has gotten me even more excited about our path ahead. This is just the start of the Gemini 4 era. As you saw with 3.5, 3.6, 3.7, and 3.8 Flash, our iteration speed with Gemini is fast : ) Now with Gemini 4 Argon to work from, I am cannot wait to see the progress continue, and most importantly see what this new model enables for the world. This is such a special moment in the arc of technology, as millions of people everyday tackle problems they could never have thought possible before. Gemini 4 Argon was built to help you tackle the hardest problems. I have personally felt this spark with Argon and hope you do too. Onwards!!
1,249
769
443
10,975
1,092,614
Benjamin Polge retweeted
This is Google’s Embedding Gemma 2 🤯 100% free and local running on Mac.
81
131
24
2,610
177,873
Benjamin Polge retweeted
Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel. - Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all. - Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract. - Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code. - And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while. The funnel is driven by a combination of factors: 1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel. 2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel. 3. Access. Free-tier, paid-tier, internal. So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
Judging by my tl there is a growing gap in understanding of AI capability. The first issue I think is around recency and tier of use. I think a lot of people tried the free tier of ChatGPT somewhere last year and allowed it to inform their views on AI a little too much. This is a group of reactions laughing at various quirks of the models, hallucinations, etc. Yes I also saw the viral videos of OpenAI's Advanced Voice mode fumbling simple queries like "should I drive or walk to the carwash". The thing is that these free and old/deprecated models don't reflect the capability in the latest round of state of the art agentic models of this year, especially OpenAI Codex and Claude Code. But that brings me to the second issue. Even if people paid $200/month to use the state of the art models, a lot of the capabilities are relatively "peaky" in highly technical areas. Typical queries around search, writing, advice, etc. are *not* the domain that has made the most noticeable and dramatic strides in capability. Partly, this is due to the technical details of reinforcement learning and its use of verifiable rewards. But partly, it's also because these use cases are not sufficiently prioritized by the companies in their hillclimbing because they don't lead to as much $$$ value. The goldmines are elsewhere, and the focus comes along. So that brings me to the second group of people, who *both* 1) pay for and use the state of the art frontier agentic models (OpenAI Codex / Claude Code) and 2) do so professionally in technical domains like programming, math and research. This group of people is subject to the highest amount of "AI Psychosis" because the recent improvements in these domains as of this year have been nothing short of staggering. When you hand a computer terminal to one of these models, you can now watch them melt programming problems that you'd normally expect to take days/weeks of work. It's this second group of people that assigns a much greater gravity to the capabilities, their slope, and various cyber-related repercussions. TLDR the people in these two groups are speaking past each other. It really is simultaneously the case that OpenAI's free and I think slightly orphaned (?) "Advanced Voice Mode" will fumble the dumbest questions in your Instagram's reels and *at the same time*, OpenAI's highest-tier and paid Codex model will go off for 1 hour to coherently restructure an entire code base, or find and exploit vulnerabilities in computer systems. This part really works and has made dramatic strides because 2 properties: 1) these domains offer explicit reward functions that are verifiable meaning they are easily amenable to reinforcement learning training (e.g. unit tests passed yes or no, in contrast to writing, which is much harder to explicitly judge), but also 2) they are a lot more valuable in b2b settings, meaning that the biggest fraction of the team is focused on improving them. So here we are.
2,099
5,824
2,224
43,440
6,636,232
Benjamin Polge retweeted
Introducing ChatGPT for Dishwashing GPT-6 achieves state-of-the-art results on dishwashing benchmarks for long-running jobs and difficult stains. The Navier–Stokes Millennium Prize Problem is a $1 million challenge about whether the equations describing fluid motion can break down. In this video, I accidentally found a counter-example while trying to optimize my dishwasher. But in reality, OpenAI used 10,000 AI agents to find a proof in 88 hours: openai.com/index/navier-stok… Parody of the original GPT-6 Astra ad @OpenAI #ChatGPT_Partner #chatgpt Credits: Written and Directed by Joma Co-Writer - Henry Connor Coan IV @coan_henry Production by Dauntlus Studios @dauntlus.studios Producer/Co-Director - Marcus Liew @dauntlus Camera Operator - Amos Lee @amoselijahlee Art Director - Alethea Soo @aletheasoo Art Assistant - Alicia Lim @aliciatkl_ Assistant Editor - GPT-6 Astra Music: Tensions Run High by Soundridemusic youtube.com/watch?v=Ly9H63SL…
424
960
341
10,783
1,122,994
Benjamin Polge retweeted
We urgently need more public traces of AI agents attacking and defending systems. Defenders can’t learn from what they can’t see. If you have traces and are being pressured to keep them private, my DMs are open. Let’s level the playing field and fight the asymmetry and lack of transparency in AI!
136
138
24
1,288
64,296
Mistral Large 4: a 1T-parameter MoE trained on just 3,800 Nvidia Grace Blackwell GPUs. We benchmarked it at JDN against GPT-6 Luna and GLM-5.3. Short version: strong on French, legal and security review, weak on agentic coding. Numbers below.
1
1
155
Pricing per million tokens (input / cached / output): $1.36 / $0.14 / $4.18, almost identical to GLM-5.3 ($1.40 / $0.26 / $4.40). GPT-6 Luna sits at $0.10 / $0.01 / $0.50. Mistral is 50% off for the first 15 days.
1
23