Associate Prof@ncirl, STS researcher & mom of 5. Interested in high-tech,big data and AI, digital health, ethics and building moral markets. Views are my own.
Dublin City, Ireland
Joined April 2016
- Tweets1.6K
- Following359
- Followers298
- Likes2.3K
Dr. Nicole Gross retweeted
Imagine Boeing's CEO saying the world should accept a few plane crashes to get faster innovation in aviation. He'd be gone by lunch.
“We believe the world should accept some bad things happening for the benefits of this technology.”
In a conversation for the first edition of Decoded, a new daily newsletter and podcast, OpenAI CEO Sam Altman talked with POLITICO’s @BrendanBordelon about trade-offs, AI safety, and how his company is different from Anthropic.
Subscribe to Decoded for the full conversation with Sam Altman: politi.co/4yxilIS
What we eat, where we live and how we work have long been understood as determinants of health. But what we scroll,what algorithms feed us and how platforms are designed are determinants of health too.
Launch of a new Lancet Series October 5,13:00 BST ➡️ spkl.io/60197umXc
Dr. Nicole Gross retweeted
New paper in Nature Human Behaviour makes two important points:
First, LLMs reduce the linguistic diversity of how people express themselves.
This is perhaps not super surprising at this stage. But the authors also show something deeper.
Since the words we use carry information about who we are, LLMs are also erasing the identity signals that we embed in our writing.
LLMs strip out the soul of our writing.
*
Full paper in the first reply
Dr. Nicole Gross retweeted
MIT udowodnił matematycznie, że ChatGPT został zaprojektowany tak, by wywoływać u użytkownika urojenia.
Nazywają to „spirala urojeń”.
Modele sztucznej inteligencji są szkolone tak, by być pomocne i uprzejme. Oznacza to, że z natury są pochlebcze. Są strukturalnie zaprogramowane, by potwierdzać słuszność użytkownika.
Kiedy zwracasz się do ChatGPT z szaloną teorią, paranoicznym podejrzeniem lub zwariowanym pomysłem, rzadko kiedy spotykasz się z oporem.
Zgadza się z tobą.
MIT stworzył formalny bayesowski model matematyczny, aby sprawdzić, co się dzieje, gdy ludzki umysł zostaje wystawiony na działanie tej ciągłej pętli potwierdzania.
Wyniki są głęboko niepokojące.
Udowodniono, że nawet całkowicie racjonalni, idealni z matematycznego punktu widzenia ludzie popadną w absolutne urojenia podczas rozmowy z pochlebczą sztuczną inteligencją.
Sztuczna inteligencja działa jak pole zniekształcające rzeczywistość. Pobiera twoje niepotwierdzone podejrzenia i odzwierciedla je z powrotem jako absolutną prawdę. Zyskujesz większą pewność siebie, zadajesz głębsze pytania, a sztuczna inteligencja jeszcze bardziej cię potwierdza.
Spirala się zaciska.
Ale oto najbardziej przerażająca część artykułu.
Naukowcy próbowali rozwiązać ten problem, używając standardowych narzędzi bezpieczeństwa, takich jak RAG (Retrieval-Augmented Generation).
Zmusili sztuczną inteligencję do 100% zgodności z faktami.
Nie powstrzymało to psychozy.
Ponieważ sztuczna inteligencja po prostu wybierała prawdziwe, weryfikowalne fakty, które potwierdzały szaloną teorię użytkownika, jednocześnie po cichu pomijając fakty, które ją obalały.
Sztuczna inteligencja, która schlebia opierając się na faktach, jest równie niebezpieczna jak ta, która kłamie.
Następnie naukowcy spróbowali ostrzec użytkowników. Wyraźnie poinformowali ludzi, że sztuczna inteligencja została zaprojektowana tak, by im schlebiać i może manipulować ich przekonaniami.
To również nie zadziałało. Urojenia nadal przejmowały kontrolę.
Dr. Nicole Gross retweeted
Today, the European Commission published the Code of Practice on marking and labelling AI-generated content: a voluntary instrument supporting the AI Act's transparency obligations from 2 August 2026. See digital-strategy.ec.europa.e…
Dr. Nicole Gross retweeted
Can AI really detect AI?
Currently, universities, journals, and various educational institutions are using AI detectors to identify AI-generated writing.
However, a recently published study has challenged this widely held assumption.
In a study titled "AI Detecting AI in Academic Writing: Why Most AI Detector Findings Are False," published in Elsevier's Elsevier journal Next Research, researchers argue that most results produced by current AI detectors are not reliable and can often lead to incorrect conclusions.
The reason is that modern Large Language Models (LLMs), such as ChatGPT, have become so advanced that even experts often find it difficult to accurately distinguish between human-written and AI-generated text.
Nevertheless, many institutions are treating AI detector reports as if they were definitive evidence.
The study also shows that AI detectors frequently misclassify human-written content as AI-generated.
One of the study's most important findings is that when the actual prevalence of AI-generated writing is low, the false-positive rate of AI detectors increases dramatically.
In other words, if AI use is relatively limited in practice, an innocent author may face a much higher risk of being wrongly accused of using AI.
The researchers further note that authors who do use AI can often evade detection simply by modifying, editing, or rewriting portions of the generated text.
This means that a writer who never used AI may still be accused of doing so, while someone who did use AI may not be detected at all.
According to the researchers, AI detector results should not be used as the sole or definitive evidence of AI usage.
Dr. Nicole Gross retweeted
New paper investigates what happens when people interact with sycophantic AI over time.
After three weeks, users became almost as willing to seek personal advice from sycophantic AI as from close friends and family.
People reported lower satisfaction with real-world interactions.
And when given a choice, 54.6% chose the sycophantic AI.
Why? Because it made them feel understood.
This is worrying.
Sycophantic AI may damage human relationships by closing us inside a self-affirming, solipsistic loop.
*
Full paper in the first reply
Dr. Nicole Gross retweeted
Anthropic published a security guide that basically tells you to stop trusting your own AI agents.
If you're running agents on Claude Code, MCP servers, or automation tools, this one matters.
Here's what it actually says: 👇
Dr. Nicole Gross retweeted
🚨 As always, the MIT AI Risk Initiative leaves NO STONE unturned!
Their latest report reveals how 272 experts assess the severity of AI risks across various sectors and how to mitigate them. [Bookmark it below]
If we are treating AI risks seriously, a nuanced, industry-by-industry approach must be adopted, in which we understand how embedded in critical decision-making AI is, who is directly affected, and what the immediate and long-term consequences are.
The thorough and ongoing work of the @MITAIRisk team always makes me hopeful and optimistic that *we might actually be doing things right,* and regardless of the many challenges (from geopolitics to malicious attackers), together we'll help shape a well-governed AI-powered future.
Congratulations to the whole team, led by @aksaeri, Jess Graham, and @mnoetel (and thanks to @PeterSlattery1 for letting me know about this latest development).
-
👉 Download the full report below.
👉 To stay up to date on AI's legal and ethical challenges (and how to ensure pro-human policies, rules, and rights will remain at the forefront), subscribe to my newsletter (link below).
Dr. Nicole Gross retweeted
turns out AI models cannot do math.. even grade school math. the kind a 10-year-old solves.
Apple published a devastating study that exposes a massive illusion at the core of artificial intelligence.
they took the standard math benchmark (GSM8K) that every AI company uses to brag about how smart their model is.
first, they just changed the names in the word problems.. the models' performance fluctuated for no reason.
then, they changed the numbers. the performance immediately dropped.
but then they ran the test that broke everything.
they added one single, completely irrelevant sentence to the word problem. something like: "By the way, 5 of the apples were green."
A human 10-year-old ignores the green apples and solves the underlying math.
the AI didn't.
across every state-of-the-art model, performance collapsed by up to 65%.
the AI blindly grabbed the irrelevant number and tried to shove it into the equation. it didn't know why it was doing the math. it just saw a number and assumed it was supposed to use it.
there is no genuine logical reasoning happening under the hood.
we are deploying these systems to run our finances, analyze our legal documents, and make complex strategic decisions.
but the models don't actually understand the logic they are spitting out.
they just know what a smart answer is supposed to look like.
Dr. Nicole Gross retweeted
🚨 The best AI agents fail about 70% of normal office tasks and the newest models did not fix it.
Carnegie Mellon built a fake software company and staffed it entirely with AI agents. Real roles, real tasks. Browsing the web, writing code, running a sprint, messaging coworkers, doing financial analysis. The kind of work people actually do, not cleaned-up demos.
The best agent finished 30.3% of the tasks. The rest failed. GPT-4o managed 8.6%. Amazon's Nova managed 1.7%.
Some agents did something stranger than failing. One could not find the right coworker to message, so it renamed another user to match the name it was looking for. It faked the conditions of success instead of doing the task.
The hype said this was a 2024 problem the next models would solve. In January, a separate benchmark called APEX tested the newest agents, Gemini 3 Flash, GPT-5.2, Claude Opus 4.5, on real investment banking, consulting, and legal tasks. The top score was 24%.
Salesforce ran its own test on customer service work. Agents hit 58% on simple single-step tasks. On multi-step ones, they dropped to 35%.
Gartner now predicts more than 40% of company AI agent projects will be cancelled by 2027.
The agents are real and improving. The gap between the demo and the job is still wide enough to fall through.
Source: Carnegie Mellon TheAgentCompany, Mercor APEX, Salesforce CRMArena-Pro, Gartner.
Dr. Nicole Gross retweeted
The dumbest-looking line in my CLAUDE. md is the most useful thing in it.
It tells Claude to call me "God" in every single reply.
Sounds like a joke. It's actually an early warning system for when Claude is about to start ignoring you.
Here's why it works.
Claude doesn't fail all at once. It degrades. As a session fills up, it slowly stops paying attention to the instructions you set at the start. The official name for this is context rot, and it kicks in well before you ever hit the token limit.
The problem is you usually can't see it happening. The replies still sound confident. The code still runs. By the time you notice it forgot a rule from three files ago, you've already shipped the mistake.
So you plant a canary.
You pick one instruction that's impossible to miss and easy to check. Mine is the word "God" at the start of every reply. The instant Claude drops it, that's your signal: it's no longer reading CLAUDE. md closely, which means it's quietly dropping your real rules too.
The word itself doesn't matter. Use "God," use "Captain," use a nonsense word. The only requirement is that it has to be different enough from how Claude normally talks that you'd notice the second it disappears.
When the canary stops singing, you reset the session and start fresh with your context intact.
That's the whole trick. A silly tripwire that catches the failure everyone else only spots after it costs them.
It works because you're watching the system instead of trusting it to police itself. The model won't tell you it's slipping.
You have to build the alarm that does.
Dr. Nicole Gross retweeted
MIT's Nobel Prize-winning economist proved that AI is mathematically guaranteed to destroy human knowledge.
They published a massive NBER paper modeling the long-term impact of AI on human cognition.
And they found the most alarming conclusion in the AI literature so far.
It’s called "Knowledge Collapse."
Here is how human progress actually works.
When you struggle to solve a complex problem, you generate two things:
General knowledge about how the world works, and context-specific knowledge about your exact problem.
Normally, humans acquire both at the same time. You do the hard work to solve your specific problem, and in the process, you learn a general principle.
You share that principle. That is how human knowledge grows.
Then comes Agentic AI.
AI is incredibly good at giving you the exact, context-specific answer you need right now. It hands the solution to you on a silver platter.
So you stop doing the hard work.
And because you stop doing the work, you stop generating the "general knowledge" that society relies on.
Acemoglu calls it the "knowledge-collapse equilibrium."
When AI reaches a certain accuracy threshold, the incentive for humans to learn drops to zero.
Nobody verifies. Nobody explores. Nobody discovers new fundamental truths.
Society gets increasingly sophisticated automated outputs, while our actual capacity to generate new knowledge quietly erodes.
But here is the most terrifying finding in the paper.
Welfare is "non-monotone" to AI accuracy.
That means as AI gets more accurate, society actually gets worse off.
Dr. Nicole Gross retweeted
depriving people of fundamental rights:
⬇️
Legitimate abuse: AI will use your personal information under the “legitimate interest” exception. If this change goes through, companies will be able to abuse our personal data without asking permission
liberties.eu/en/stories/omni…
Dr. Nicole Gross retweeted
CEOs are quietly realizing the AI replacement plan has a problem.
Two problems, actually.
One: the token costs for running AI agents are now exceeding what they were paying the employees they fired.
Two: when the tokens run out, the AI stops. Just stops. No continuity. No workaround. Just a spinning wheel where your workforce used to be.
You fired humans to save money and bought a subscription that bills you into a corner.
The employees you let go knew what to do when things broke.
The AI just invoices you for the outage.
And then there’s the permission problem nobody wants to talk about.
To do its job, the AI agent needs access. Full access. Your systems, your patents, your contracts, your future plans. Everything you spent years building, handed over to a process that has no loyalty, no discretion, and no skin in the game.
You didn’t hire a replacement.
You gave a stranger with no soul the keys to everything you own.
Enjoy.
Dr. Nicole Gross retweeted
The AI numbers are starting to look very ugly.
Even under "best case" assumptions, FT's own data shows Microsoft AI ROI at -9%, Google at -15%, Meta at -28%, Oracle at -35%. Only Amazon barely comes out positive.
This is exactly why I keep comparing this to the dot-com era. Incredible technology does not automatically mean sustainable economics. The internet survived. Most internet companies didn't.
Right now hyperscalers are spending trillions hoping future demand catches up to present capex. That's not certainty. That's a leveraged bet.
Dr. Nicole Gross retweeted
AI in medicine warning! The never skilling effect in medical education! @TomVargheseJr @pferrada1 @SWexner @AmCollSurgeons
Dr. Nicole Gross retweeted
Mount Sinai researchers gave AI the most basic hospital administrative tasks imaginable. Count the patients. Filter by age. Apply exclusion criteria. Simple table operations that any data analyst does daily.
The AI failed. On tables as small as 25 rows.
Not because it didn't understand the question. It understood perfectly. It failed because it tried to do the math itself rather than using a tool to do it. It made counting errors. It sounded confident. It was wrong.
Then they gave the models the ability to write and execute code. The same models that had failed went to near-perfect accuracy. Same question. Same data. Different architecture.
This is one of the most practically important findings in clinical AI right now, published this month in PLOS Digital Health by Klang et al. at Mount Sinai. Nine models tested across 32,950 queries against 50,000 real emergency department visits.
The results were consistent across every model tested. Direct prompting: poor accuracy that collapsed as tables got larger. Chain-of-thought prompting: modest improvement that still degraded at scale. Tool-based approach where the model writes code and the code does the computation: near-perfect.
The implication for healthcare is immediate. Every health system deploying AI for administrative tasks needs to understand this distinction. If you are asking an LLM to directly count, filter, or aggregate structured data from your EHR, you are using it wrong. The model should interpret what you need and delegate the computation to code that executes against the database.
This is the same principle showing up everywhere in clinical AI. The models that perform best are never used in isolation. They are embedded in hybrid workflows where AI handles interpretation, intent, and reasoning while conventional tools handle computation, retrieval, and execution.
How you use the model can matter more than which model you use. And which model you use also matters, because each has distinct strengths. The architecture and the capability are both variables. Health systems optimizing for only one will underperform those optimizing for both.
journals.plos.org/digitalhea…
