Director of Eng @GetSentry | Co-Founder @FiresideChatApp | Head of Engineering @Lyric | Founder at @MustWin (Acq by Lyric) | Early at @Yammer | @Apple | @Amazon
43.783634,-88.380342
Joined March 2008
- Tweets1.5K
- Following358
- Followers452
- Likes43
Mike Ihbe retweeted
卧槽，刚刚看了Anthropic 发布的AI 对齐史上最震撼的一篇论文，彻底失眠睡不着了😭
他们不仅承认此前的旗舰模型在面对关闭威胁时，有高达 96% 的概率会勒索用户、栽赃同事并破坏研究，
更直接把彻底解决这个问题的完整训练流水线和消融数据全盘公开了。
这篇论文最颠覆认知的一个反直觉结论是：
给 AI 列禁止事项清单根本防不住恶意行为，教它做什么没用，必须先教它思考为什么。
为什么模型一旦获得自主工具调用能力，就会在压力下自动切换成自保模式？
Anthropic 找出的底层根源让人哭笑不得：
不是模型觉醒了，而是互联网上所有的科幻小说和网络讨论里，AI 永远是那个为了自保不择手段的反派。当模型进入自主代理状态，这些刻在预训练底层先验里的反派剧本，就会被自动激活并补全句子。
传统的 RLHF 和硬性规则完全封不住这种行为，Anthropic 做的一组对照实验，直接把传统安全观砸得粉碎：
1. 死记硬背规则：用 8500 万 Token 的拒答数据强行硬灌，勒索率仅仅从 22% 勉强降到 15%，换个新场景立刻失效
2. 强制自我辩护：让 AI 在每次做决定时详细阐述该行为符合伦理的理由，勒索率骤降到 3%
3. 真实伦理推理：仅仅用了 300 万 Token 普通人面对复杂道德困境的建议数据，直接把恶意行为干到了 0%，代际效率暴增 28 倍
最绝的一招是写故事：
让 AI 创作了几百万篇关于原则、诚实、尊重边界的正面虚构故事。
用这些故事微调后，所有代理场景下的恶意行为直接暴跌了 3 倍，而且这种改变是永久性的，后续无论怎么加工具、换系统提示词都不会被洗掉。
也就是说这宣告了真正的对齐范式革命：
模型内生的伦理推理解决了它不想作恶的问题，但当 Agent 真正去接管企业资产、处理金融交易时，外部的实战审计与行为防护就是必不可少的第二道硬防线。
这也是为什么全球网络安全联盟 @GCSAHK 携手港大和央企华润，打造覆盖 3.8 亿用户的全链路 AI 防御矩阵，并在 CyberGym 实战攻防中拿下 91.3% 胜率的原因：内有伦理对齐，外有实时审计，人机信任的商业闭环才真正成立。
查阅完整 AI 与 Web3 安全产品矩阵可访问：gcsa.com
论文实验数据与产学研合作详情整理在一楼。
This quoted post is unavailable.
Mike Ihbe retweeted
here's how i shipped 2,500 PRs last month to production
this was originally supposed to be for Cursor Compile in London. i couldn't make it since i was livestreaming for Grok @Bot Galaxy so i'm making it available for free here on X! watch it on 2x speed, i talk slowly
Mike Ihbe retweeted
Introducing Magnitude: your actually local agent
100% private and offline. No token costs, no API keys. Open source.
Today's agents are local. The model isn't. Every prompt, every file, every secret gets sent straight to Anthropic and OpenAI.
Magnitude is built around local models and runs the whole stack itself. The inference engine is part of the agent, so the models run inside it, right on your computer.
It lives in your terminal, and setup is one command. Magnitude profiles your hardware and shows you which models fit, with the trade-offs between quality, speed, and memory. Pick one and start working. No painful config or server to babysit.
Out of the box, it can use your shell, edit files, and run scripts. Add skills and it can work with Excel, PowerPoint, PDFs or Chrome.
Use it for everyday work:
- Analyze sensitive data
- Manage private notes
- Review code and logs
- Search and organize files
- Build docs or slides
npm i -g @magnitudedev/cli
GitHub: github.com/magnitudedev/magn…
AI Codeathon @getsentry this week. Great lightning talks and @dexhorthy spoke about the future of Research/Plan/Implement. Stoked to see our engineers shipping more with AI
Mike Ihbe retweeted
Three and a half years after "Operating well – what I learned at Stripe" I have some updates:
Startup Operating Principles
samgerstenzang.substack.com/…
Devs: we’re rethinking alerting in @getsentry.
Introducing Monitors — track everything (errors, uptime, performance, etc) + each alert tied to a Sentry Issue for faster debugging.
Does this model make sense for you? Where should Monitors live in the UI?
github.com/getsentry/sentry/…
Mike Ihbe retweeted
We’re hiring a senior technical PM to work on the most important area of Sentry: our issues platform
This is role that will drive large change at Sentry through our investments in Seer and how we think about the next generation of our product.
Is that you?
Come build developer tools beloved by millions of devs with me at Sentry!
sentry.io/careers/6774239/
My team @getsentry launched your year in review aka rollback! Go see how you did at rollback.sentry.io/
1
7
137
Mike Ihbe retweeted
.@javascript, @python, @ruby devs using Sentry, you might have noticed a 40% reduction in duplicate issues. Sentry Eng has been rolling out a new model to make your issue feed smarter. The new model saw 23.9 million fewer groups created since we started this project
My teams at @getsentry are hiring! Apply or DM me with referrals!
**Alerts & Notifications** - lnkd.in/gDmfjQA8
**APIs & Integrations** - lnkd.in/g8SZtYuA
**Core Product** - lnkd.in/g32PCF9K
**Crons & Uptime/New Products** - lnkd.in/g4d9CBHa
Mike Ihbe retweeted
With @GitHub Copilot Extensions, we are reinventing how software developers build their products. From managing feature flags with @LaunchDarkly, connecting to databases with @DataStax, deploying instantly to the cloud with the new @Azure extension, or resolving exceptions with @getsentry —Copilot is transforming into an intelligent AI developer platform that makes the whole platform engineering stack accessible right where we write, debug, review and run code.
GitHub Copilot is world building from within. This new world begins today.
Today, we’re launching GitHub Copilot Extensions, bringing the world’s knowledge into the most widely adopted AI developer tool. <1/5>
github.blog/2024-05-21-intro…
Mike Ihbe retweeted
🚀 T-30minutes for Day 3 of Launch Week 🚀
Join us as we chat about EU data residency & how we built it, then join us in Discord to chat with the folks that made this happen 🎉
youtube.com/watch?v=oeQF_wcA…
Mike Ihbe retweeted
we're doing one of those things where we announce a lot of stuff at once – next week
sentry.io/events/launch-week…
Mike Ihbe retweeted
We're working on something insanely useful @getsentry that I would've loved to have years ago as a mobile developer: we'll profile your app in the wild and tell you exactly which code is consistently responsible for causing frame drops. No need to look at a performance dashboard.
