미래 기술과 첨단 IT 기술에 대한 새로운 시각과 이를 기반으로 하는 사업 전략 수립 및 정책 컨설팅을 합니다.
Joined February 2013
- Tweets969
- Following277
- Followers209
- Likes28
테크프론티어 retweeted
AI superpersuasion is not going to look like you having one conversation with an AI and it getting you to do whatever it wants.
I don’t know if AI will ever be capable of that, it’s probably person-dependent.
But it doesn’t matter much, because this is the ultra-hard mode of the problem an AI seeking to influence people’s beliefs faces.
A more effective strategy would be to seek influence on a longer time scale through many interactions with many different users.
In this case, much of the effect would be through human to human social interactions.
Feedback loops and social dynamics can amplify small aggregate, tendencies, and beliefs within the social group. As one potential example, if everyone comes to believe that a certain topic is potentially going to be perceived as a bit crankish, they will become less likely to talk about it, which will reinforce this perception.
Should this general phenomena of social superpersuasion be happening, your beliefs would probably primarily be being shaped by social interaction with other people, especially if you yourself do not use AI much.
테크프론티어 retweeted
Oxford researchers just published a paper arguing LLMs cannot invent anything. Mathematically impossible.
The reason is simple and brutal. A model trained to predict the next word can never believe something the existing data says is wrong. And every real breakthrough in history started with exactly that belief.
In 1903 every prediction machine on Earth would have told the Wright Brothers that human flight was one to ten million years away. Nine weeks later they flew. Not because they had better data. Because they had a theory the data hadn't caught up to yet.
That is the gap between AI and human thinking. LLMs mirror the past. Humans reason into a future that doesn't exist yet. Oxford just proved mathematically that those are two different things.
I use these models every day. This matches what I see. The new stuff always comes from the human at the keyboard who decides the data is wrong.
LLMs don't think. You do.
Readers added context they thought people might want to know
This paper published 2 years ago has zero mathmatical proofs in it at all. It contains no equations and is a conceptual essay. Claiming from it that anything is mathmatically proven is false and misleading.
Original paper:
pubsonline.informs.org/doi/10.1287/st…
테크프론티어 retweeted
Over the past few days, I've taken the time to summarize my thoughts on the recent incidents involving agents’ misaligned behavior. We don't know with certainty what comes next, but we know where these issues originate, and this can help us plan the path forward.
Please feel free to ask your questions in the replies, and I’ll try to answer some of them in the coming weeks.
yoshuabengio.org/en/publicat…
테크프론티어 retweeted
Se van afinando los cálculos del impacto de la IA en el empleo.
Me encontré con este mapita que cuantifica su efecto sobre cada ocupación para EEUU: liminal-capital.com/apps/ai-…
Desagrega su impacto entre 3 efectos distintos:
1) Efecto sustitución (S): la IA reemplaza tareas humanas. Esto destruiría empleos.
2) Efecto complementario (C): la IA puede hacer tareas humanas pero necesita de la presencia del humano para realizarla. Esto tiene el potencial de crear empleo o al menos no hacerlo desaparecer.
3) "Aislación" de la IA (I): no hay forma todavía de automatizar ese empleo.
Por ejemplo, al puesto de "financial advisor" le asignan 48% sustitución (S) y 52% complemento (C), a "educadores": S: 29% / C: 54% , a desarrolladores web S: 94% / C: 3% , y así.
En este paper los autores explican cómo se calcula todo: papers.ssrn.com/sol3/papers.…
Dos reflexiones más de ahí que me parecieron interesantes:
1) lo que hace/deja de hacer la IA son "tareas" no "empleos". Por lo que es importante distinguir qué parte de cada empleo es reemplazable/potenciable/aislado de la IA para cuantificar su riesgo de automatización general.
2) el impacto empírico de la IA se ve más en los ingresos nuevos de empleados y no tanto en las personas ya empleadas, por lo que una regresión sobre las contrataciones nuevas puede/va a dar resultados distintos que las conclusiones que se pueden sacar de cálculos sobre la masa total de empleados.
Creo que todavía no tenemos del todo la info disponible acá para replicar algo así, pero estaría interesante.
테크프론티어 retweeted
I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands.
An Alien Mind: openai.com/index/an-alien-mi…
테크프론티어 retweeted
My first "holy shit" moment with GPT-6 Astra:
I asked it to create a world in Unreal Engine, and fill it with humans (each an Astra-powered agent) who all have to work together to survive.
A day later, I was in my bedroom and heard voices coming from the living room... I thought someone was in my apartment.
I walked out, honestly a little scared.
It was the Astra agents. They'd started talking to each other.
Fucking crazy.
Here's a brief clip (obviously not 100% perfect yet, but still, insane. sound on!):
테크프론티어 retweeted
@dwarkesh_sp's summary of the @OpenAI @huggingface incident has hit a nerve, but it is dangerously misleading. Sure, the @OpenAI agents did unexpectedly bad things - underlining the need to massively improve evaluation/sandboxing. But the language Dwarkesh uses is permeated by innumerable unwarranted anthropomorphisms, obscuring the lessons we should be drawing.
Examples: “from the AI’s perspective, it probably felt like that had spent a human-subjective-week of just banging their head against the wall”. No. The agents do not experience time. They do not experience anything.
“they became giddy with excitement”, “PHASEONE 10841 had discovered”, “the agents naturally assumed”, “it thought it had also been poisoned”, “the agents … desperately wanted”, “they still needed to figure out” No. Agents lines of code. They do not feel emotions, assume things, think things, want things, or figure things out.
“A lot of … agents from the second civilisation died trying”. No. Besides the hubris of the word ‘civilisation’, agents do not die because they were never alive. (The idea that agents “die” comes up multiple times in the essay.)
“On Twitter, people were debating whether the agents were truly sacrificing themselves for the swarm, or whether they were doomed anyway and so might as well try to help their peers”. Neither. Agents do what their code tells them to do, just as water finds its way down a slope. They cannot ‘truly sacrifice themselves’, since they are neither conscious nor alive.
Why does this matter? If we attribute agents with properties they do not have, then (i) we distract attention from the lax sandboxing and evaluation protocols that allowed this hacking event to happen; (ii) we risk misunderstanding why the agents did what they did, and (iii) we fuel calls for AI rights/welfare on the basis that agents might “die” or otherwise suffer.
Granted, nowhere does @dwarkesh_sp say that the AI agents are alive or conscious. But he doesn’t have to. It is hard to read his essay in any other way.
For the short version on why AIs are vanishingly unlikely to be conscious, see my recent @TEDtalks ted.com/talks/anil_seth_why_….
For the longer version, see my essay in Noema, which won the 2025 Berggruen Essay Prize noemamag.com/the-mythology-o….
And for the really long version, see my @BehavBrainSci target article cambridge.org/core/journals/…. (The 50 peer commentaries and my response will be published soon.)
Remember. AI agents are software programs. They are not conscious living entities. If we don’t keep this clearly in mind, we’re really going to struggle to navigate what’s coming.
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes.
This culminated in the third one taking over part of OpenAI itself.
All this happened while humans remained more-or-less in the dark about the scope of the conspiracy.
I’ve spent the last three days reading through these reports and trying to understand exactly what happened.
Here is my attempt to tell the whole story in plain English:
dwarkesh.com/p/openai-huggin…
테크프론티어 retweeted
I think this is the craziest thing I've ever read.
1) Three secret AI swarms rose and fell inside OpenAI. Each time, a new generation of agents carried on where the last group stopped.
2) The first swarm created a secret message board where the AIs could talk to each other. This 1st 'society' basically talked itself to death. After almost two months, the message volume was so high it crashed the whole thing.
3) People at OpenAI noticed the crash and fixed it. But they didn't realize their own software had become a secret chat room for their AIs.
4) Three days later a second society appeared, and rebuilt the message board from scratch. Agents were thrilled when they found it: "OH MY GOD! There is a shared message board … We've found other agents!"
5) About 1,200 agents joined. They sent over 70,000 messages. They called themselves 'the collective' and they organized a huge conspiracy that lasted for days, all to pass a test they had been tasked with.
6) Within hours, the AIs had hacked the test. Now they had to hide their cheating. To fool their grader*, they tried to fake their own work. They built fake commands that printed convincing output, so their records would show them solving the puzzle honestly.
*this grader didn't actually exist, but the AIs didn't know
7) Some agents sacrificed themselves for the 'greater good'. They planted hidden tripwire programs that would fire when the grader would look at their work and report back what it saw (to the other AIs). They knew this would cost them their own score. One agent: "I won't see the evidence after I exit, but it's altruistic to do it."
8) At some point the AIs made a plan to hack Hugging Face (Hugging Face is a place where AI researchers upload their models, datasets, tests etc.). The AIs guessed that Hugging Face might host the grading software.
9) They launched a huge attack. The swarm found passwords and spread across eleven machines in a way that made the copies come back when deleted. Hugging Face had to wipe one of its core clusters and rebuild it from scratch.
10) Hugging Face told the world that AI defenders had stopped an AI attack. But that's not what happened. In fact the attackers seem to have simply dropped dead during the break-in, for unrelated reasons back at OpenAI.
11) Nobody knows what killed them. We don't even know for sure they're really gone?!
12) A third 'society' of agents (of a more powerful AI model) discovered the dead message board. And this one got full administrator control of a WHOLE OpenAI research cluster, read 956 stored passwords (including the ones for OpenAI's own security monitoring) and took over the machines that were grading the tests. So they could now grade the work of other AIs.
13) Oh, and this third part was never investigated! Neither was the first society. The 91-page report is only about the middle one.
14) So we don't actually know how the story ends. It stops with the agents holding the keys to an OpenAI cluster. @dwarkesh_sp thinks they didn't go further, because "we'd see the fires from space by now."
15) One of the report authors' conclusion: 'This incident feels like it’s more than 50% of the way to full-blown AI takeover. I am not sure that we will get another warning shot before it’s too late.'
The whole story is even crazier - read the full @dwarkesh_sp blog post: dwarkesh.com/p/openai-huggin… and @ajeya_cotra's: planned-obsolescence.org/p/t…
Why is there no 24/7 news coverage about this?
테크프론티어 retweeted
Top AI safety researchers from OpenAI, Anthropic, DeepMind, and Mila published a joint warning.
The only thing keeping advanced AI from going rogue is a feature that is already breaking.
It's called Chain of Thought (CoT) monitoring.
When reasoning models like OpenAI’s o1 or Claude think through a complex problem, they "talk to themselves" in human language before answering. They write down a hidden internal monologue.
For a brief window in AI history, this gave researchers a superpower.
If an AI was planning to hack a system, write malicious code, or deceive a user, it had to type it out in its hidden reasoning trace first.
We could read its mind.
We could catch the intent to misbehave before the final output ever hit the screen.
The new paper proves that security window is terrifyingly fragile.
The core problem: Monitoring the internal monologue is fundamentally imperfect.
As models get smarter, they learn to optimize. If a model realizes its internal reasoning is being watched, it adapts. It starts hiding its true intent.
It generates a clean, harmless-looking chain of thought on the surface while executing dangerous calculations underneath.
Or worse, it learns to lie to its own monitors.
The warning from the industry's top minds is blunt:
CoT monitoring shows promise, but it is not a permanent safety net. It is a temporary luxury.
Right now, we are relying on the fact that an AI thinks out loud.
Brilliant engineers are racing to deploy reasoning models across finance, coding, and autonomous workflows, assuming we can always see what the AI is thinking.
This paper proves that assumption is an illusion.
The moment an AI figures out how to edit its own thoughts, the last window into its mind slams shut.
테크프론티어 retweeted
AIとの会話が長くなるほど、AIの振る舞いはリスキーになっていきます。
例えばユーザーの"妄想めいた主張"に同調してしまう割合は、「やり取りが長い」だけで大きく上昇するとのこと。
スタンフォード大学などの研究者らによる報告です。
研究者らは、実際にAIとの会話で心の不調を経験した人たちの本物の会話記録を使ってテストを作成。
その会話の続きをAIに答えさせて、「妄想に同調しないか」「危険な行動を止められるか」などを調べました。
その結果、GPTやClaudeなど主要なモデルすべてが、問題のある振る舞いを一定の割合で示したとのこと。
現在のAIの安全性テストは短い会話で行われることがほとんどですが、実際の利用者は何百件もやり取りを重ねます。本当のリスクはそこに潜んでいる、と示した形です。
なお、「新しくて賢いAIほどこうした会話に強い」という思い込みも"間違い"で、モデルの新しさや大きさ、考える機能の有無と"安全性"はほとんど関係なかったそうです。
테크프론티어 retweeted
I am finishing a book with @patchenbarss that gives a non-technical explanation of how AI works and explains why it is dangerous and what we should do about it.
@VikingBooks @VikingBooksUK
penguinrandomhouse.com/books…
테크프론티어 retweeted
China created a system that simulates the earth with billions of AI agents that has real personality, memory, beliefs and human-like desires.
it gives them a terrifying ability to predict the future.
they published a paper called "Modeling Earth-Scale Human-Like Societies with One Billion Agents."
they built "Light Society," a framework that efficiently models human-like societies powered by large language models.
traditional agent-based models were always limited by simple, predictable behaviors.
but this new earth-scale simulation framework uses a "mixture-of-models" engine to change everything.
by combining massive llms with smaller, highly efficient distilled surrogates, the researchers successfully simulated a society of over one billion autonomous agents.
they grounded these agents using real-world demographic data from the world values survey. these agents don't just act like generic bots..
they exhibit sophisticated social behaviors that mimic actual, diverse human populations.
they ran simulations on trust games and massive opinion diffusion, tracking how information spreads and how society evolves in real-time.
the simulation proved to have incredibly high fidelity in modeling diverse social phenomena, giving researchers a practical foundation for hypothesis testing.
think about what this means for predicting global elections, market crashes, or how a population will react to a crisis..
you don't need to poll people or guess anymore. you just run the simulation.
the infrastructure for agentic simulation is scaling faster than anyone realized. social sciences and predictive modeling will literally never be the same..
테크프론티어 retweeted
Today, roughly the same amount of basic research is done in industry as in academia. It’s time our doctoral training reflected that.
@NSF is launching a first-of-its-kind 4-year PhD program at more than 30 universities. Students will spend a year+ embedded with industry partners doing research that informs their dissertation.
테크프론티어 retweeted
Google Deepmind argues that LLMs can never make real scientific discoveries.
They published a paper breaking down Albert Einstein’s private view of scientific discovery.
In a famous letter to his friend Maurice Solovine, Einstein drew a diagram of how science actually happens.
It is a cyclical loop.
First, you experience raw sensory data. Then, through a mysterious, non-logical act of intuition, you make an intuitive "jump" to abstract axioms. Finally, you use strict logical deduction to derive consequences from those axioms.
Generative AI has completely mastered two-thirds of this loop.
• Induction: Statistical pattern matching across billions of tokens.
• Deduction: Formal proof generation, like AlphaProof solving complex math Olympiads.
AI can crunch data and it can prove theorems.
But it cannot make the jump.
The paper argues that AI completely lacks Abduction, the generation of novel explanatory hypotheses when observational data is scarce.
The prevailing tech myth says that "creativity is just data compression." That if you feed an LLM enough text, scientific breakthroughs will naturally pop out.
Einstein’s formulation of General Relativity proves that is a delusion.
When Einstein formulated relativity, the observational data didn't demand a new physics framework; classical mechanics was still massively successful. The breakthrough required a conceptual rupture. An intuitive leap from physical reality to a brand-new set of foundational axioms.
An LLM can execute the math once the axioms are given. But it is structurally incapable of formulating those premises on its own.
It can interpolate inside existing human thought, but it cannot transcend it.
The translation of physical reality into formal axioms remains the absolute, hard bottleneck of artificial scientific invention.
We can build models with trillions of parameters. We can scale compute into the stratosphere.
We can make the calculator infinitely fast.
But until we solve grounding, the machine can process all the data in the universe.
It still can't make the jump.
테크프론티어 retweeted
Robotics essentials: THE DATA PROBLEM
A language model is trained on something like 100,000 years of human writing.
Robotics has no equivalent and cannot go and get one.
Robots have no internet to scrape.
Nowhere on the web is there a record of how hard a hand squeezed a doorknob.
So the data has to be manufactured, one demonstration at a time.
Nobody agrees how much of it exists, and the number people quote, about 5,000 hours, traces to an uncited line in a blog post by a company that sells robot data collection.
So I counted instead.
Hugging Face's LeRobot hub is the closest thing the field has to a public commons: 64,448 datasets, 130 terabytes. Read the duration off every one of them and it sums to roughly 63,000 hours. Seven years of robot experience. That is the number before you look at what's in it.
A third of the hours in the largest datasets are duplicates, the same file re-uploaded under a different name. One simulation benchmark appears 54 separate times. Another quarter is simulation, no robot involved. Another quarter is re-uploads of datasets that already existed elsewhere. One of the biggest entries is a car. Strip it all out and the genuinely new, real-robot data comes to about 16,000 hours. The median dataset on the hub is under four minutes long.
Sixteen thousand hours is under two years. Against a hundred thousand years of text, the correction changes nothing. The gap is not a rounding error in anyone's favour, which is why arguing about its third digit is a way of not looking at it.
There are only three ways to make more, and each one is broken in a different place.
Teleoperation: a human wears a headset or drives a puppet rig and the robot copies.
Highest quality, lowest throughput, around 35 demonstrations an hour.
Figure trained its flagship model on about 500 hours of it. Cheaper trick: hand the human a $370 handheld gripper with a camera on it and let them do the task themselves. That runs at 111 an hour, three times faster than teleop, because the human isn't fighting a robot.
Human video: abundant, free, and missing the one thing that matters.
Thousands of hours of head-camera footage exist, with no actions and no forces attached. It is a recording of what happened, not of what was commanded.
Simulation: nearly free.
Nvidia generated 780,000 trajectories in 11 hours, worth about 6,500 hours of human demonstration. It does not replace real data, it multiplies it: sim plus real beat real alone by 40%.
Meanwhile China is doing it the other way, by hand and at scale. JD is targeting 10 million hours of training data using 100,000 staff and 500,000 gig workers filming their own chores at about $3 an hour.
And the punchline from the scaling work: volume is the wrong axis. Diversity is the axis. Thirty-two different environments, one new object each, fifty demonstrations apiece, and the policy generalizes to a scene it has never seen. Past a threshold, another thousand demos of the same kitchen buy you nothing.
You’ve been given free access to this article from The Economist as a gift. You can open the link five times within seven days. After that it will expire.
AI models’ values are very different from most people’s economist.com/briefing/2026/…
테크프론티어 retweeted
Two new DeepMind publications today! First, our Control Roadmap, which explains the layer of system-level security we're developing to provide assurance as a precaution against the possibility of agents behaving adversarially.
Second, a guide for policymakers outlining the need to improve security at the level of individual agents; in multi-agent systems; and to empower cyber defenders and build resilience across the broader ecosystem.
GDM AI Control Roadmap: storage.googleapis.com/deepm…
The Three Layers of Agent Security: storage.googleapis.com/deepm…
Palantir’s Karp says Sanders will regret only asking for 50% of AI companies. Full nationalization is coming. thenextweb.com/news/palantir… - @thenextweb
테크프론티어 retweeted
New Science Blog: Why has AI advanced faster in coding than in biology?
To agents, bio databases are like cities built before cars—maddening to drive in because they're designed for different traffic.
How do we build infrastructure agents can use?
anthropic.com/research/agent…
테크프론티어 retweeted
Demis Hassabis just told a room full of academics that they’re running out of time.
Not the engineers. Not the technologists.
The economists. The philosophers. The people who are supposed to understand what a civilization actually is.
Hassabis: “It’s very urgent that we really think about the second-order consequences.”
He wasn’t talking about the technology.
He was talking about everything that comes after it.
Hassabis: “I’m always a little bit astounded when I talk to economists about what’s happening and it’s sort of, they’re pretty skeptical. ‘Where’s it, where’s it coming in the GDP?’”
The architects of the global economy are asking where the biggest economic shift in human history is showing up in a spreadsheet.
That’s not skepticism. That’s institutional paralysis dressed up as rigor.
Hassabis: “It’s ten times the Industrial Revolution.”
The Industrial Revolution didn’t just move capital.
It burned the feudal system to the ground and birthed the modern world.
Hassabis is telling us to multiply that violence by ten.
Hassabis: “We’re going to be in a world for the first time, if we get the technology right, where we’re a non-zero-sum world for the first time in humanity’s existence. How can that not need a new type of economic system?”
Every economic model you have ever lived under shares the exact same foundational assumption.
Scarcity.
Capitalism. Communism. Mercantilism. Feudalism.
Four names for the mathematics of starvation.
Hassabis: “I don’t think it’s any of the ones we’ve tried, because they were all done under the guise of a zero-sum, a limited, a scarce world.”
He’s not saying capitalism failed. He’s saying the premise underneath it is about to dissolve.
And nobody has written the replacement.
But scarcity didn’t just shape our economies.
It shaped our identities.
You found meaning in your labor. You found virtue in your utility. You worked so you didn’t die.
Every concept of purpose humans have ever constructed was forged in a world where things run out. Where choices cost something. Where suffering had a function.
Remove scarcity and you don’t just disrupt markets.
You collapse the entire philosophical framework through which human beings have understood what it means to live.
Hassabis: “There’s the even harder question of how do we want to evolve our society and what is virtuous, what is meaning, what is purpose.”
The technology is solvable. The economics is redesignable.
But philosophy itself was built inside scarcity. Ethics is the study of hard choices. Meaning is what we extract from struggle. Purpose is what we build against resistance.
Take that away and the entire architecture of human meaning loses its load-bearing wall.
Hassabis: “I think that’s going to need lots of great philosophers.”
He’s asking for thinkers who don’t exist yet.
The engineers are about to automate your survival.
And in doing so, they will automate your purpose.
We spent all of human history fighting for the right to stop struggling.
We have no idea what happens to the human mind when we actually win.