Building Responsible Healthcare AI ventures
Palo Alto, CA
Joined August 2009
- Tweets364
- Following774
- Followers313
- Likes187
We measure AI agents on accuracy, latency, cost and task completion.
As autonomy rises, we need Oversight Load, the human attention required per successful action.
But does lower oversight mean safer autonomy or failures humans can no longer see ?
@NeelNanda5 @ch402
This is where interpretability should become operational.
Can internal signals predict when an agent is:
• Misrepresenting uncertainty
• Optimizing the wrong proxy
• Planning an irreversible action
• Avoiding human intervention
If the insight cannot change supervision, is it safety or explanation?
A possible benchmark:
Give an agent progressively greater autonomy and measure:
• Correct self-escalations
• Human-detected failures
• Silent failures
• Successful recoveries
• Safe actions per minute of human attention
Capability should determine what an agent can attempt.
Evidence should determine what it can do alone.
A starting definition:
Oversight Load = human review time + interventions + reversals, normalized per 100 agent actions.
But it must be weighted by consequence.
Reviewing an email is not equivalent to approving a payment, changing production code or making a clinical recommendation.
This is how Google and Meta made money even before the AI era. With AI the new thing is that users can feed more human context and deeper nuance for getting the answers or fulfill the tasks.
I’m skeptical on the net effect of post AI world on systems at large.
We see that Jobs are reorganizing with Human + AI.
But what’s the long term impact of AI on society ?
Will purpose be the new token succeeding Skills and knowledge ?
Healthcare AI is entering a new phase.
Phase 1: Administrative AI (scribes, coding, inbox)
Phase 2: Clinical AI (AI integrated into care delivery)
UpDoc’s FDA-cleared SaMD is one of the strongest signals yet that the industry is crossing that boundary.
We have spent years optimizing AI for compute.
Maybe that’s the wrong objective.
The metric I think will be more valuable is
Intelligence Density ,
how much useful intelligence an AI system delivers per unit of resource.
Per watt.
Per dollar.
Per millisecond.
Per liter of water.
The next frontier may not be bigger models.
It may be maximizing intelligence per unit of resource.
The future of AI won’t be won by whoever builds the biggest model.
It may be won by whoever delivers the same intelligence using 10x fewer resources.
Efficiency is becoming the new frontier.
Google’s latest sustainability reporting highlights that AI growth is increasing electricity use, water consumption, and emissions even as datacenters become more efficient , illustrating that efficiency gains and overall demand can both rise at the same time.
Microsoft now publicly reports both Power Usage Effectiveness (PUE) and Water Usage Effectiveness (WUE) for its datacenters, showing that water efficiency is becoming an engineering metric rather than just a sustainability report.
Kashyap Kompella retweeted
Indeed: if we actually had AGI we would not need forward deployed engineers.
Two simultaneous AI narratives right now:
1) You can now do the work of 20 people, learn anything and create anything with AI. Just learn how to use claude.
2) We are investing Billions in forward deployed engineers to help you implement AI b/c its too time consuming and hard for you to do it yourself.
I skew more towards #1, fwiw. But I find these narratives to be incongruent.
