OpenAI's AI Reasoning Model Matches or Outperforms Doctors in Diagnostic Tests
Similar Articles
Ontario AI Medical Scribes Show Accuracy Issues in Provincial Test
AI Model Detects Pancreatic Cancer Years Earlier on Routine CT Scans
AI Firms Brief Congress on Advanced Cybersecurity Models and Risks
AI Model 'Centaur' Faces Challenge Over Its Ability to Simulate Human Cognition
Stanford Study Finds Affirming AI Models Can Reduce Apology and Self-Reflection
A new study published in Science finds an AI reasoning model developed by OpenAI matched or often outperformed doctors in clinical acumen tests. The research, conducted by Harvard Medical School and Beth Israel Deaconess Medical Center, tested the model on its ability to diagnose patients and manage care using text-based data from real cases and established benchmarks.
Facts First
- An OpenAI-developed AI reasoning model matched or outperformed doctors in clinical diagnostic tests.
- The model was tested on real patient cases, including a pulmonary embolism case from a Boston emergency department.
- Researchers graded the AI's diagnostic accuracy from the triage stage through to hospital admission.
- The study used only text-based data, excluding images, sounds, and nonverbal cues.
- The research was conducted by teams from Harvard Medical School and Beth Israel Deaconess Medical Center.
What Happened
Researchers from Harvard Medical School and Beth Israel Deaconess Medical Center tested an OpenAI-developed AI reasoning model on its ability to diagnose patients and manage care. The study, published in the journal Science, found the model matched or often outperformed doctors and the previous AI model, Chat GPT-4, in clinical acumen tests. Experiments used actual cases, including a patient at the Beth Israel emergency department in Boston who had a pulmonary embolism and a suspected history of lupus. The AI model outperformed two experienced physicians using only electronic health records and the limited information available to the physicians at the time.
Why this Matters to You
This development suggests AI could one day serve as a powerful diagnostic aid in healthcare settings, potentially helping to reduce diagnostic errors and improve the speed of care. For you, this may lead to more accurate initial assessments during emergency room visits or hospital admissions in the future. The research was conducted using text-based data alone, which means its current application is limited but points toward a specific, integrable tool for doctors.
What's Next
The study authors, including Dr. Adam Rodman and Raj Manrai, have established a performance benchmark. Further research is likely needed to test the model with more complex data, including images and other clinical inputs, before it could be deployed in real-world settings. The results may accelerate investment and development in clinical AI tools, which could begin appearing in pilot programs at hospitals in the coming years.