AI Matches Mid-Level Clinicians in Skin Lesion Diagnosis in Realistic Settings, Finds JAMA study
France: A diagnostic study found that a modern artificial intelligence (AI) foundation model outperformed clinicians with less than 3 years of experience and achieved diagnostic accuracy comparable to that of clinicians with 3–10 years of experience in identifying skin lesions. However, it remained less accurate than dermatology experts with more than 10 years of experience, underscoring both the potential and current limitations of AI in dermatologic diagnosis.
The findings were published in JAMA Dermatology by Julien Anriot from Claude Bernard University Lyon 1, Lyon, France, and colleagues.
Artificial intelligence has shown promising results in detecting skin cancers under controlled research conditions. However, questions remain about how these systems perform in everyday clinical practice, where physicians encounter a wide range of common, uncommon, and atypical skin lesions. To address this gap, the researchers compared the diagnostic performance of AI models with that of physicians possessing different levels of dermatology experience.
For the study, investigators conducted a multi-institutional diagnostic assessment using a dataset of 1,117 dermatological cases that reflected routine clinical scenarios. The cases included clinical and dermoscopic images along with relevant patient information. The study evaluated three AI systems: a first-generation convolutional neural network (CNN), a unimodal foundation model called PanDerm, and a multimodal PanDerm model. Their performance was compared with that of 652 physicians whose dermatology experience ranged from less than one year to more than ten years.
The study revealed the following findings:
- A total of 652 physicians participated in 1,092 diagnostic testing sessions.
- All physician groups outperformed the first-generation CNN model.
- Human readers achieved a mean diagnostic accuracy of 65.9%, compared with 56.7% for the CNN.
- The unimodal foundation model was the best-performing AI system, achieving an accuracy of 72.2%.
- The unimodal model outperformed clinicians with less than three years of experience, whose average diagnostic accuracy was 68.2%.
- The unimodal foundation model demonstrated performance comparable to dermatologists with 3–10 years of experience.
- Dermatologists with more than 10 years of experience achieved the highest diagnostic accuracy at 74.2%.
- Expert dermatologists outperformed all AI models evaluated in the study.
- The multimodal foundation model achieved a diagnostic accuracy of 66.3%.
- The CNN model showed the lowest performance among all human and AI evaluators, with an accuracy of 56.7%.
The findings suggest that current AI tools can support less experienced clinicians in skin lesion diagnosis but still fall short of the performance achieved by highly experienced dermatologists in real-world clinical settings.
The researchers concluded that while modern foundation models can match the diagnostic accuracy of mid-career clinicians, expert dermatologists remain superior. They emphasized that future practice may benefit from human-AI collaboration, with AI assisting in decision-making, triage, and reducing fatigue-related diagnostic errors.
Reference:
Anriot J, Yan S, Coste C, et al. Limits of Artificial Intelligence Models for Skin Cancer Diagnosis in Realistic Settings. JAMA Dermatol. Published online June 03, 2026. doi:10.1001/jamadermatol.2026.1492
Next Story