Sam Muli retweeted
🚨 Medical AI has an evidence problem.
Not a performance problem.
An evidence problem.
A brilliant Nature Medicine editorial says what many people avoid saying:
Stop showing us AUROC.
Show us clinical value.
Because:
- sensitivity
- specificity
- calibration
- benchmark performance
…do not prove better patient care.
A model can be technically excellent and still be:
❌ clinically useless
❌ workflow-disruptive
❌ dangerously misleading
If:
- outputs arrive too late
- nobody acts on them
- clinicians can’t trust them
- they simply automate bad decisions
then the AI has zero medical value.
Even with a beautiful ROC curve.
This is the real problem with many “medical copilots” and guideline-centric tools like OpenEvidence:
- They optimize retrieval of what is already written.
- Not understanding of what is actually happening.
- They replicate consensus.
- Not reasoning.
- Medicine is not benchmark logic.
- It is uncertainty management.
The strongest line in the editorial:
“The stronger the claim, the stronger the evidence needed.”
Exactly.
If you claim:
👉 “Our AI improves outcomes”
you should need more than retrospective validation and polished dashboards.
Otherwise, that’s not innovation.
That’s marketing.
My provocative take:
We are not automating intelligence.
We are often automating:
- bad workflows
- rigid guideline thinking
- and weak clinical reasoning
…at scale.
Which means:
we are industrializing mediocrity.
The future of medical AI is not better models.
It is better proof.
Until then:
AI risks becoming beautifully engineered overconfidence.
Sam Muli retweeted
New presentation on things we often forget to consider when designing a clinical trial: fharrell.com/talk/qrct/ #Statistics #clinicaltrials
Sam Muli retweeted
Prediction: we’re about to enter a period of many negative “individualized treatment strategy” RCTs, because:
1. The need for “individualized treatment” is an appealing idea, so we want to believe its true in excess of the evidence supporting that it actually is true
2. Existing evidence of HTE is generally weak, because it usually misattributes the sources of variation (see citation below)
3. Even when there is latent HTE, our theories that guide individualized strategies are rudimentary and unreliable. We choose a rule so simplistic - so as to be operationalizable - that it has little chance of actually selecting people with atypical responses.
Sam Muli retweeted
Interesting, nuanced, and sometimes contentious discussion about the validity and interpretation of confidence intervals for observational research: discourse.datamethods.org/t/… #epidemiology #Statistics
Sam Muli retweeted
The author seems to be making the age-old error of using quantile groups to understand risk patterns. If the type of food consumed matters it will matter on an individual dose-response basis, not according to how many people are like you. sensible-med.com/p/the-60-ye…
Sam Muli retweeted
1/5 How reproducible are phase III oncology trials?
📊 With @AlexSherryMD @ErikVanZwet @ebludmir et al. We analyzed 632 RCTs (~496,000 pts). The data offer both reassurance and caution for clinicians and researchers. pubmed.ncbi.nlm.nih.gov/4062… 👇#MsaouelLab
Sam Muli retweeted
An elastic net (lasso + ridge regression) approach applied to the win ratio for variable selection and risk prediction for hierarchical composite outcomes.
Check lmaowisc.github.io/wrnet/ for R tools and demonstrations.
Paper accessible at doi.org/10.1186/s12874-025-0….
Sam Muli retweeted
Tutorials in Biostatistics
Guidelines and Best Practices for the Use of Targeted Maximum Likelihood and Machine Learning When Estimating Causal Effects of Exposures on Time‐To‐Event Outcomes. Talbot et al. Statistics in Medicine. onlinelibrary.wiley.com/doi/…
Sam Muli retweeted
Virtually none of the work in these meta-analyses controls for variation in fatty acid metabolism among humans, which is substantial.
In fact, there is good evidence that the FADS genes (encoding fatty acid desaturases) have undergone intense selection, generating substantial genotypic variation in just the past ~12K years (likely related to dietary shifts in PUFAs in response to agriculture).
It is no wonder that the results in these studies are often variable and conflicting.
Examples of genotype-dependent effects in PUFA metabolism: xitter.cf/trikomes/status/192278…
This was an interesting Umbrella review on “seed oils,” or – more precisely – omega-6 fatty acids. It included analyses of meta-analyses of observational studies, covering 150 publications in total and 230 studies. The first sentence of the abstract’s conclusion reads, “Higher dietary intake and circulating levels of omega-6 fatty acids were associated with lower risks of CVDs, cancers, and all-cause mortality.”
But what popped out to me when reading:
1. Notes a remarkable increase in dietary linoleic acid increase since the 1960s: average intake went from ~2.7g/d to 4.9-21g/d in Western countries.
2. Large degree of heterogeneity in the data. The words “inconsistent” and “inconclusive” appear 13 times in the main text.
3. Diverse outcomes in the umbrella review for cancer. Omega-6 intake was linked to lower risk of lung and prostate and increased risk of ovarian and endometrial cancers. And remarks on recent studie(s) reporting increased risk of colon cancer. (More on that in a video in a couple of weeks)
4. Included meta-analysis of observational studies but excluded meta-analysis of interventional studies.
5. Questionable methodological quality of the base data: “of the 11 meta-analyses reviewed, six were rated as having low methodological quality, while the remaining five were rated as critically low”
6. Protective associations were largely absent in those with pre-existing health conditions. This is consistent with a strong element of healthy user bias, among other explanations.
The authors write in the discussion the health effects of dietary omega-6 fatty acids appear to be a “double edged sword.” So why if the data seem so uncertain and inconsistent does social media seem so split and certain on “seed oils?"
My two cents: "Absence of evidence isn't evidence of absence." I don't think fear of all omega-6 rich foods is reasonable, but neither is it reasonable to disregard concerns about seed oils as "nutribullocks"
Sam Muli retweeted
You’ve been told to fear:
Egg yolks
Ghee / Butter
Paneer
Red meat
But never questioned:
Pizza
Kachori, samosa, vada
“Diabetic” biscuits full of maltodextrin
Sugary Cereal and bread sold as ‘healthy’
This isn’t nutrition. It’s confusion.
Low-carb challenges the noise and brings back real nourishment
Sam Muli retweeted
Advocating for progress in mental health isn’t anti-psychiatry.
It’s not a fight against clinicians.
It’s not a fight against science.
Instead…
It’s a fight *for* science.
*For* innovation.
And most importantly, *for* better outcomes and better lives for patients.
Sam Muli retweeted
New paper from @ChrisPalmerMD in the British Journal of Psychiatry Open!
Thank you for highlighting our pilot study work, which was greatly inspired by your pioneering clinical work and publications on ketogenic diet in psychiatry.
👉cambridge.org/core/journals/…
💬The quote below from the paper is an important insight considering that several medications used in the treatment of bipolar were originally developed as anti-seizure medications (e.g. Lamotrigine, Carbamazepine, Valproate).
🧠Many anti-seizure treatments act on cerebral glutamate, and we saw significant reductions in glutamate in brain imaging on a ketogenic diet.
Sam Muli retweeted
People don’t always need answers.
Sometimes, they just need to feel seen, heard, and understood.
Your presence can sometimes be more healing than your advice.
Sam Muli retweeted
Can a treatment for a metabolic problem also be a cause of metabolic problems?
Yes
Just look at the side effects of insulin.
Sam Muli retweeted
You were born sinless, you are not suffering from any generational curse from your ancestors. Your ancestors were not demons, they were great Africans.
Sam Muli retweeted
First person in the family to become an academic:
Cons:
- No money
- No investments
- Questionable career prospects
- Wearing a backpack
Pros:
- more peer-reviewed articles than siblings






















