Fellow at Tenstorrent; believes in dynamic typing, first-class functions, the immortal essence of the human soul and tea. Tweets are my own.
Lübeck, Germany
Joined January 2010
- Tweets4.3K
- Following389
- Followers1.2K
- Likes6.8K
Mark retweeted
I didn't believe the video when I saw it yesterday because it lacked technical details in addition to being too attention grabbing.
But today I had Claude take apart the Android version of the game for me; Everything OP says is true.
There are lots of manipulations but the key one here involves what the code calls offer type ZHI_SI_TI (致死题, "lethal puzzle") or alternatively "algo_must_death".
The game generates puzzle pieces in sets of three, and has different algorithms that swap how the pieces are selected. Generators have behaviors like "random", "hard", "fill gaps" (to help player), "random but guaranteed not to kill you", etc. If a given algo fails to find a solution it passes to the next algo in an active set.
The "lethal puzzle" generator considers every possible move with a given set of three pieces, then only emits them if the number of placement possibilities is 0 (no solution, guaranteed death). This is activated in several situations, one apparently being that the app thinks your current play session is a little slow, and they want you to not get bored so they kill it early to start a new one.
What the video doesn't explain is the context. In the game, "rewarded" ads are ads you get some game reward for watching. The main reward is a revive/continue; If you get into an unwinnable state you can watch an ad to keep playing. You usually only get one revive per game but there are a bunch of server-activated properties to modify this constraint.
The policy at play here is that, if you're a high-value player, and there's a high-value ad queued for you, and you're in a play session nearing your high score, the game puts "lethal puzzle" to the top of the piece generator list to rig the game to fail, right when you're most likely to be highly engaged and watching the ad on the revive screen.
It doesn't end there though; The game remembers if the last game ended due to a hand dealt by ZHI_SI_TI, and triggers FatalPuzzleDeathCompensationTrait, which takes the edge off by letting you carry over your combo count from the previous session.
Like everything else this compensate-after-we-killled-you trait is an A/B feature controlled by the server. It's manipulations all the way down; there's like 5% actual game code and the rest is literally hundreds of remotely controlled toggles to control the game state for maximum engagement.
Mark retweeted
Лучшее, что я видел за последнее время — чел пилит ASCII MMORPG, в которой вся графика состоит из букв и символов.
Да, буквально весь мир, персонажи, окружение и спецэффекты нарисованы разноцветным текстом.
Автор сейчас экспериментирует с магией, визуалом и боссами. Я прям залип на то, как из нескольких десятков символов получается такая красота.
Буквально сны при температуре 37,2.
Via: tiktok.com/@Ansenjeo
Continuing, the AI war is.
完蛋了…..以后中国开源模型只能开源小参数的模型了….
商务部科技部就修订《中国禁止出口限制出口技术目录》征求意见，为期一个月，涉及人工智能（大模型、智能体、算力集群、信息处理等23项）生物医药、光伏3个领域。
1️⃣限制出口AI大模型：超过10^26计算量、或2T参数、或公开评估排名全球前10的大模型。
2️⃣限制出口智能体Agent：工具调用与系统操作技术，支持集成的第三方工具数量≥200个，工具选择准确率≥90%，且在需要调用3种及以上工具协同完成任务场景中，端到端任务平均成功率≥90%；自主规划与任务分解技术，可将自然语言指令自动拆解≥20步可执行子任务链，子任务分解精度≥80%，且子任务平均成功率≥90%；标准博弈基准超过90%的人类的博弈决策技术；得分75%以上的AI辅助漏洞挖掘技术。
限制出口算力集群技术：片间互联技术、分布式训练容错技术、通信调度技术、异构集群高效推理与调度。
征求意见期间，出口相关技术需向商务部填报登记表。已通过开源、发表学术论文等方式进入公共领域的技术，不纳入管理范围。
（中国商务部）
目前看来@Kimi_Moonshot 的K4、@MiniMax_AI 的M3 Pro、@Zai_org 的GLM5以及@deepseek_ai 的下一代模型都没法开源了…..
而且相关架构、训练优化和推理加速技术也都不行了。
Readers added context they thought people might want to know
该帖称中国大模型今后“没法开源了”、只能开源小参数模型，与官方文件不符。公告明确支持人工智能开源创新生态，对已通过开源等方式进入公共领域的技术不纳入管理范围，并对开源情形适用自动许可等便利措施（仍在征求意见中）。
mofcom.gov.cn/zwgk/zcfb/art/…
mofcom.gov.cn/syxwfb/art/202…
Mark retweeted
This is obviously the future of video games.
Streamers and their community building the game together, then playing it.
When Astra came out I streamed a call of duty clone, with 10 loadouts, vehicles, kill streaks etc, and in between matches people we're shouting in the voice chat (that we vibe coded into the game), what changes they wanted.
But i see a world where they can send in their prompts and the team of the streamer can moderate those prompts and send them through, and the agents will make 20 changes at once, and then 30 min later everyone could be playing the next version of the game.
And every month that goes by, as models improve, these games will look more and more professional. Some company will figure out how to make these games easily forkable, hostable, monetizable, social, fast, and fun.
I have spent 3 weeks and two claude max accounts vibe coding a 50-person battle royale.
Kong Royale.
We invented hive coding, chats messages prompt 20 agents working overnight to fix and improve the game.
and today we finally launch 🦍🍌
live playtest TODAY 1 PM ET. drop in with us 👇
youtube.com/watch?v=b5n1K_IA…
Mark retweeted
I had a similar reaction. Everyone should applaud performance improvements, but gauge against the “speed of light”, not just a sloppy competitor. If you are measuring in seconds, you may have already anchored yourself incorrectly.
Structure the UI so you can put locally cached media up just a few vblanks after the process launches.
Structure the server so it can return everything necessary for the dynamic UI in one single query that was kicked off in parallel with the Gfx API initialization, branching to a modern transport protocol for large media files.
You should be able to double click on the launcher and have it responsive before your muscle memory can slam the pointer to a location to click on.
This is hilarious and awesome.
Awesome because this has annoyed millions of peoples for years.
Hilarious because WHY DOES IT TAKE 5 SECONDS TO SHOW A MENU OF GAMES?!
Steam takes 10 SECONDS?!
Do you realize how much time that is for a CPU with a modern internet connection? Am I crazy!? What am I missing?
Mark retweeted
Our G1 robot found its way to the pantry, picked up a Pringles can, and handed it over. No pre-built map. No training for this task.
Behind it is a coding agent, writing, running, and fixing the robot's programs as it goes. And it self-evolves: what works is saved as code for the next task, no retraining needed. This is ASENA.
Joint work with @Isabella__Liu, @edmund_bu, Johan Bjorck, @yin_hongxu, @zhengyiluo, @jankautz, @DrJimFan, @yukez, and @Sifei30488L. 🧵👇
Mark retweeted
wow, amazing technical blog post about microsoft's new 3T pre-training with very rare and clear insights on how to train models of this size
first let's get an overview of the shape of this model, it's a sparser model than Kimi K3 (80B active vs 104B), same sliding window hybrid pattern, with this time more kv heads for the sliding window layers (a bit different from stepfun and poolside that add query head). they added a shared expert and also removed the interleaved MoE/Dense pattern of MAI 1
the scaling ladder is very similar to MAI 1 with the same mechanism, each rung depends only on the number of layers. they mention that one of the key improvements is how they handle the smaller rung data distribution where they make sure to have the same repetition as the final corpus. they already did that for MAI-1 but only for data ablation iirc
their prediction is almost identical to the final run loss, very impressive
they compare the model with different base models such as deepseek v4 pro and MAI 1, tho in the case of deepseek v4 pro i'd say it's quite hard to distinguish how much the data mixture "overfitting" to the scaling ladder has an impact here (they mention it's both), need to always be a bit careful about EG metrics for data in general
they have VERY NICE scaling laws for weight decay and learning rate, for the learning rate they express it as a function of tokens per parameter (N/D) and steps (hence depending on the batch size). they also do the same for dropout where it's a function of active params only, they also find the usual sqrt(k) scaling for learning rate and batch size (other scaling laws often also converge to that)
a bunch of other stability stuff, they use Muon this time and activation gradient clipping, the efficiency gain here is actually pretty crazy at larger scale. they also increase the training horizon to ~40T text (20T unique) + a new 10T (4.5T unique) multimodal since the model is natively multimodal
imo this is a safe but very clean scaling! very exciting for the tech report and MAI-base-3, congrats!!
We now have state-of-the-art pre-trained base models in the 3T-parameter weight class, built entirely in-house at MAI.
Post-training results coming soon -- stay tuned!
Lots of details in our technical blog post.
microsoft.ai/news/frontier-p…
Mark retweeted
Argon has a similar performance profile on ARC-AGI-3 as Astra. Google is back at the frontier.
Gemini 4 Argon from @Google on ARC-AGI (Verified):
- ARC-AGI-3: 97.5%, $4.8K (provider adapter harness), 49.8%, $5.8K (standard harness)
- ARC-AGI-2: 91.3%, $0.32/task
- ARC-AGI-1: 98.5%, $0.16/task
Google DeepMind is the second frontier AI lab to score above 95% on ARC-AGI-3.
Mark retweeted
I've been thinking a lot about data bottlenecks, and I'm not sure who else is screaming this from the hilltops. I feel like I'm taking crazy pills looking at the jagged frontier thinking it's smooth if you order by data volume.
the spikes in the jagged frontier are tasks with tons of data (either static or simulated; think math, programming, games), and the valleys are tasks with little or no data (cooking a lasagna, olfaction, manufacturing a passenger train). I don't know of any exceptions.
if you order the jagged frontier by data it looks like a spiral frontier. models are superintelligent where we have enormous data, and progressively less so as we have less data. some tasks may draw data from static lakes of data, some may be from limitless waterfalls from simulators that convert compute into data. but ultimately, intelligence on a task is proportional to the data absorbed by the model for that task.
scaling laws are real. if you are not bound by compute or algorithms (increasingly we are not), then intelligence is proportional to data. since we don't have a very good notion of task similarity, it's often difficult to know if data for one task will help you with another, without training a model and seeing. but, it doesn't seem like chess data helps much with math performance and vice versa. so when I refer to data, I refer to data that is task-relevant.
maybe we call this the 'no shortcuts hypothesis'. all the rules-of-thumb about learning are at the root, the same -- Malcom Gladwell's 10,000 hours, Moravec's Paradox, Scaling Laws, the Bitter Lesson, Age of experience. If your learning system (model, brain, whatever) is unconstrained by compute, intelligence is proportional to accumulated experience. no shortcuts. want to be superhuman at chess? play all the chess, more than the whole species combined (good thing a model can do that in 24 hours). want to be superhuman at math? build a math simulator and lock the model in it and run it faster than reality. superhuman at coding? first, collect all public examples of humans writing and responding to code ever, make the model learn from that first, and then lock the model in a simulator where it fixes bug after bug, faster than realtime.
superhuman at image generation? no shortcuts. but here, we don't have a simulator. so now we're moving around the spiral frontier away from closed-loop into large static lakes. still big enough to be superhuman though, which also applies to video, text, and audio. oh! and once you have a superhuman model, you can use it to bootstrap other tasks by making a superhuman model a judge on other tasks.
let's keep marching along the spiral frontier, out of the realm of data abundance, simulators and large static lakes.
biology? no shortcuts. go get a ton of data. and be freaking careful! you can get bad data in biology pretty easily. tilting a well plate 5 degrees the wrong way can ruin an experiment. so make sure you understand the domain, otherwise you're going to have to filter a lot out.
robotics? no shortcuts. get the data. if you can build a simulator, do it, but be mindful our simulators don't have perfect correspondence with reality. so, build an arm farm. fold 1M shirts, put 1B balls in cups. play ping pong forever.
olfaction? no shortcuts. smell the world. build a laboratory of chemical sensors, an international network of trained sensory panelists, a scent production factory, and link them all together. make millions of samples. smell millions of samples. take billions of chemical sensor readings.
I totally buy super intelligence as a concept because I feel it, I use it for coding, the breakthrough math results speak for themselves. I see RSI coming, because how could an all-digital task like ML research and model building not fall to the current paradigm? But the notion that AI will come for everything, all at once, irrespective of domain, feels silly, because there are no shortcuts. if you want a specific form of intelligence, you have to get the data.
looking at you @willdepue, I feel like we're both seeing the same thing
🤖 Made with AI
Mark retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Mark retweeted
when nick told me that he had some interp breakthroughs, he was uh, underselling it. i think this may be remembered as a major moment in scientific history. news so good even eliezer will have to lower his pdoom. we can start making alignment into a 'normal science'
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
