AMIE: Advancing medical AI for video consultations

When you visit a doctor, a consultation extends far beyond words — a physician notices a cough, observes gait, or registers visible signs of discomfort. Today, Google Research and Google DeepMind are advancing AMIE, our research medical AI system, toward real-time clinical video consultations, with a first-of-its-kind demonstration of expert-level AI capabilities in this setting. Built on Gemini and Project Astra using a multi-agent architecture, AMIE now interprets visual and auditory cues, guides virtual physical exams, and reasons diagnostically in real time.

In a randomized study using simulated consultations with patient actors and a group of primary care physicians, clinical evaluators assessed AMIE favorably across core clinical competencies, including history-taking thoroughness, diagnostic accuracy, management appropriateness and communication quality. Patient actors also preferred the video experience over text chat.

AMIE remains a research system and more research is needed before responsible real-world clinical deployment, but it offers an exciting glimpse into the future of health AI. Read more on the Google Research blog.