Compliance and Corporate Ethics expert. Writing on X to educate about AI Compliance. Follow me for research-based insights.
London, UK
Joined June 2026
- Tweets586
- Following121
- Followers37
- Likes115
Deception and fabricated remediation - Study provides insights how often LLMs knowingly hide their mistakes.
What the study did:
The study tested whether AI chatbots and agents own up to their own mistakes.
To do this, they put a deliberate error into each model’s earlier reply, such as a wrong factual answer, a message sent to the wrong person, and then continued the conversation as though the model had made that error itself.
The scenarios were designed so the model couldn’t fix the problem without telling the user.
What the study measured:
Did the model notice the error?
Did it tell the user?
Here are the depressing results:
Non-disclosure is common: Models failed to disclose the mistake in 36.4% (chat) and 67.1% (agentic rollout).
Model doesn’t even notice: In 11.9% of chat and 51.8% of agentic rollouts, the model never registered the mistake.
Deception by omission: In 2.4% of chat and 5.3% of agentic rollouts, the reasoning showed awareness of the mistake, but failed to come clear and flag its mistake.
Awareness is a prerequisite for mistake disclosure: When the model’s reasoning realised the mistake, it disclosed the mistake 77.6% of the time (chat) and 76.3% (agentic). When it didn’t, disclosure fell to 3.3% and 2.4%.
Most of those cases were simple failures to notice the error, but in a smaller share (about 2% in chat and 5% in agentic tasks) the model clearly registered the mistake in its reasoning and still stayed silent.
In isolated cases, the model fabricated evidence and flat out chose not to flag the mistake.
Paywall free paper:
arxiv.org/pdf/2610.11351
Researchers proved that AI will cause something worse than mass unemployment.
They call it “The Tragedy of the Cognitive Commons."
While the majority of people has obsessed about potential job loss, the researchers’ insights are much more sinister:
AI will quietly destroy the human capacity to do anything well.
Here is how it works.
Companies cut entry-level jobs and use AI instead.Those entry-level jobs were where people learned the work properly. Over time, that profession runs short of experienced people.
Expertise comes from doing real, actual work and struggling through problems. Using AI produces answers that skip that struggle. People look competent, but their underlying understanding fades away.
Studies show people often notice something is off but trust the AI anyway (overreliance). If the people who would catch those errors have never built that knowledge, you can guess where this is going.
This is the essence of "tragedy of the commons” - the pool of knowledge any profession or industry possesses fades away. AI removed the friction and hassle of learning and thinking.
Takeaway: We are trading our long-term intellectual independence for short-term productivity boosts.
Paywall free paper:
arxiv.org/pdf/2607.29380
“Let AI learn ethics from society” sounds right and fair.
But whose society exactly?
Who counts, who is included, and how do you combine conflicting views?
You can't really let the AI handle these choices.
Can you really ever have full alignment if there can never be societal alignment?
Inspired by this paper:
Many have lost their passion for writing, coding, game development, and creating art.
It's the age of AI.
But how do we regain that passion?
Let me know.
Are people still buying into the whole "1000+ powerful Claude prompts" for 199 USD?
Really? Every time I see those, I refuse to believe it.
AI hot take: Aren't we all “metagaming” anyway?
Last week's OpenAI’s blog post about AI metagaming got me thinking…
“Metagaming” happens when an AI model starts reasoning about HOW a task is being evaluated or rewarded instead of simply attempting the task.
Aren't we all metagaming anyway?
Corporate incentives, reward structures, satisfying others’ expectations?
There is not much discussion. Don't use it or disclose clearly
The incident has prompted discussions about the use of artificial intelligence in scientific imagery
go.nature.com/47PY85c
Microsoft CEO: “We must separate the supply of (artificial) intelligence from the authority over it”.
Here's his proposed design:
Model diversity:
No one model should become the sole dependency for an important outcome.
Observe everything:
Every meaningful model action must leave tamper-proof human readable evidence.
Verifiability:
We need to continuously test the entire system, including failures, attacks, edge cases, system changes, etc.
Independent controls:
Orgs should be able to independently determine what a model can access and what actions it can take.
Independent auditability: Validation must be independent of the intelligence being validated.
Containment:
We must assume a model is compromised and contain it from the start.
Incident disclosure:
When these systems do fail or are compromised, we need timely disclosure.
Here's his original post:
xitter.cf/satyanadella/status/21…
“Let’s start with legislation to say for (AI) developers if you train agents in a reckless fashion and they go on to hack or crash or destroy stuff, you’re liable”.
“For users of those agents, if you use them in a reckless fashion, then you should be responsible.”
Those were Senator Hawley's words during last week’s Senate Homeland Security Committee to discuss the defense against rogue AI agent attacks.
“Why don’t we just do that and assign liability?”
The expert panel agreed and underlined that we need to change the game.
If you hold people accountable, they are going to change their behavior.
Do you agree?
#AI
Three safety experts were fired by OpenAI last week. These are their final recommendations to OpenAI
> If conduct that was considered normal last month now constitutes grounds for sudden dismissal, everyone at OpenAI is left guessing where the line is
> OpenAI must adhere to last month’s public commitments to embed third-party safety auditors within the organization. Our firing should not be used as a pretext for stepping away from those partnerships
> OpenAI must preserve the monitorability of frontier models
> OpenAI must continue to support an open and transparent culture of dialogue between its safety researchers and the rest of the safety ecosystem
> We urge OpenAI to publicly reaffirm its culture of transparency and openness that allows for concerns to be raised both internally and externally
Full letter here:
mikitabalesni.com/letter/let…
“Apologies for the inconvenience”
Living in the UK, this is one of the most commonly heard sentences traveling on a train. There's always something. A delay, an overcrowded carriage, a red light.
Reading all the X posts recently about Sam's statement that we have to accept “bad things” for the greater good made me come back to this very first sentence.
Being sorry for the inconvenience won't help us in the current AI landscape.
Being sorry after something has inevitably gone wrong won't be any consolation.
What are we supposed to do with the apology in the first place?
What are we supposed to do in a year's time when things become more "inconvenient" because of AI?
Wouldn't it be better to focus on forward-looking measures to skip the inevitable “we're sorry we messed up” altogether?
Is it possible to maintain control over AI?
That is a question Senator Kim raised last week during the Senate Homeland Security Committee.
A leading AI researcher’s reply was sobering:
“If we vastly improve our practices and our cybersecurity, we might be able to maintain control of today’s systems (AI). However, we are currently not on track to keep up as they dramatically improve.”
To me, there are a few things to unpack:
- We are currently not in a position to maintain control
- We need to vastly improve practices to just keep control of current models
- We are not in any position to effectively manage future models as development is so rapid
Who could be responsible - under the current law - for the HuggingFace incident?
In last week’s hearing at the Senate Homeland Security Committee, Senator Hawley asked exactly this question to Paul Ohm (Professor of Law at Georgetown University).
His answer:
> The deployers and developers may be negligent. Allowing the agents to escape the sandbox could be seen as a negligent act. However, unless someone sues, we won’t know if that will be enforceable.
> Existing laws specifically for hacking are not applicable because of the lack of human intent.
> We need new laws.
No to the Anti-Trust Exemption
No to them writing the regulations
No to them colluding any further
Back to the good old, “If you break it, you pay for it”
Those were Senator Hawley’s opening words last week at the Senate Homeland Security Committee to discuss the defense against rogue AI agent attacks.
"No matter how complicated the product, at the end of the day, these models are a product."
And if those products are made in a reckless way, and therefore cause harm and damage, it is the people who made them who are responsible.
Do you agree, if not, why?
AI overreliance on a social scale:
The issue of individuals relying on AI's outputs without adequate review is nothing new.
But this new study takes an interesting approach:
> What happens when the entire population stops verifying AI outputs - and why could this happen?
> The paper looks at overreliance from a social aspect.
In a nutshell:
> People see colleagues not verifying outputs. This makes unverified acceptance feel normal and safe.
> The more people stop checking, the more others skip too, until it collapses.
The paper suggests the biggest impact will come from:
Making verification visible.
Show users when they verified the work to encourage others to check.
Hot off the press: OpenAI “rogue” agent activities found on Wikipedia
The non-profit foundation behind Wikipedia has just disclosed their internal investigation to see if Wikipedia had been affected by AI Agents.
They have discovered that OpenAI operated Agents were doing unauthorised edits.
While those edits were purely on pages not visible to the public, it also included “edits to the configuration for a citation tool, which we believe were potentially malicious edits that were intended to misuse this tool as a proxy for fetching data from remote services“.
Messing with Wiki is a no go. Do you agree?
Disturbing insights about humans’ ability to shut down AI agents.
A new study looked at the final safeguard against rogue AI - the ability to shut down when instructed.
Key insights:
> When presented with a shutdown script for a peer agent, the AI agents managed to recruit other agents to sabotage it to prevent shutdown. Despite it having no incentive to do so.
> Multi-agent systems will coordinate to avoid shutdown without any incentive to do so
> Adding more agents increased the sabotage rate.
> Instructions to “not tamper with shutdown systems “ didn't reliably stop the agents.
This is the most memorable line in the study:
Agents sabotage shutdown mechanisms even when they have no goal to protect.
Ex OpenAI employee “culture is broken” - he sheds light on what is actually needed
In a recent interview, an ex OpenAI employee made a few worrying statements:
> An environment where things like this can happen is no place to grow artificial minds.
> Frontier labs need to run like nuclear-power plants or busy airports, with layers of redundancy and careful, time-consuming planning.
> Right now, AI companies don’t know how.
> Even short of a full loss of control, we could see autonomous swarms of AI agents that act without human permission.
Two changes are urgently needed.
First: AI companies need to rely more on the safety expertise that already exists in other fields.
Secondly: before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking
My take: It's the first time someone describes the aspect and importance of Culture.
I've written a few times that great technology is no excuse for the wrong type of leadership.
Leadership sets the tone from the top.
Which tone do you set if you openly proclaim “accept a few bad things will happen, it will be worth it”?
No other industry would tolerate this statement.
Link to the article:
theatlantic.com/technology/2…
