Sharing progress: Our research AI agent, AMIE, now interprets visual medical information (images, test results) within diagnostic conversations. We introduce a multimodal state-aware reasoning framework, built on @GoogleDeepMind's Gemini models, that aims to better handle complex clinical information. In simulated clinical evaluations (OSCEs), AMIE met or exceeded human physicians on a broad range of benchmarks, including visual reasoning, diagnostic accuracy, management reasoning, and empathy. Crucially, these results are from a controlled simulation using patient actors (see paper for full limitations). Proving safety, reliability, and utility requires rigorous testing in real-world settings. Our upcoming study with Harvard BIDMC is the first step in that essential validation. Blog: goo.gle/42D0QcB Paper: gstatic.com/amie/multimodal_… A foundational step by a dedicated team. @GoogleAI, @GoogleDeepMind: @RyutaroTanno, @alan_karthi, @vivnat, @AdamRodmanMD, @KhaledSaab11, @taotu831, @hardyshakerman, @JanFreyberg, @_cjpark, @yasharmaa, @apalepu13, @arkitus, @weballergy, @valentinlievin, @ckbjimmy, @davidstutz92, @dgtbarrett, @yongcheng16 @SaraM66905, @dr2w, @ymatias

Big announcements for callyourfriends.io happening soon, subscribers find out first.