Looking deeper into the the shared embedding space, it has clinically meaningful structure: 🟢 Subtypes cluster together 🟤 Inflammatory dermatoses separate cleanly from melanocytic lesions Slide embeddings mirror text embedding organization, showing alignment across modalities, not just within them.

Quantitative gains over image-only aMIL classifiers: 📊 Dermatopathology (52 subclasses, 3,388 eval slides): ~4–6% relative improvement 📊 GI pathology (102 subclasses, 7,857 eval slides): ~8–10% relative improvement

🔬 Moving Beyond Pixels: Advancing Multimodal Pathology Pathology AI has long relied on images alone. What happens when you add language? We combined PLUTO-4 vision embeddings with rich histological descriptions to build a joint vision-language space for disease classification, and the results are promising. 🧵 #AI #MachineLearning #Pathology #MultimodalAI #FoundationModels

Beyond public benchmarks, PLUTO-4 shows real-world impact — 🩺 ~10 % improvement across multiple PathAI products, with strong gains in dermatopathology specimen classification. These advances bring us closer to robust, generalizable FMs for pathology applications. #Dermatology #HealthcareAI

Large-scale multi-node training is often communication-bound. We optimized ViT-G training with DDP + GPUDirect RDMA, tuning parameters (bucket_cap_mb, gradient_as_bucket_view) to saturate InfiniBand bandwidth. Result: 3× throughput improvement and near-linear scaling across 4 H200 nodes.

Comparison of Patch-Sizes 🧩 4S (FlexiViT) lets us tune patch size per task ps-8 excels at tasks needing fine-grained features (ex: nuclear segmentation) ps-16 suits coarse pattern tasks (Gleason patterns) #FlexiViT #FoundationModels

PLUTO-4 is trained on a diverse dataset of 551,164 WSIs from 137,144 cases across 50 institutions. The dataset spans 40+ organs, 60+ diseases, and 100+ stain variants, capturing real-world variation in tissue, staining, and scanning systems.

🚀 Excited to share PLUTO-4, our new state-of-the-art foundation models for pathology! 🔬 We’re seeing SoTA performance across multiple public benchmarks (EVA and HEST) — surpassing other leading pathology foundation models. (1/6) #AI #MachineLearning #Pathology #FoundationModels #HealthcareAI

🔍 Interpretable concepts found using SAE - SAE trained on PLUTO embeddings disentangled polysemantic features. Single dimensions captured distinct concepts: ✅ Cell types (e.g., cancer cells, red blood cells) ✅Geometric features (e.g. edge of tissue) ✅ Artifacts (surgical ink)

Feature evolution across layers - SAEs trained on PLUTO’s intermediate layers revealed: Early layers → Low-level color/texture features 🎨 Later layers → Pathology-relevant biological features 🔬 (e.g., monosemantic plasma cell dimension).

Monosemantic representations - Single SAE dimensions correlate with counts of single cell types. For example, SAE-1736 represents plasma cell abundance exclusively - The findings generalized to: ✅ Out-of-domain datasets (CPTAC) ✅ Different stains (H&E, IHC) ✅ Various scanners