🔬OpenAI MentalHealthBench: 80+ Clinicians Grade AI Replies
TL;DR
OpenAI released MentalHealthBench, an open benchmark scoring AI replies in realistic mental health chats. More than 80 licensed experts from 22 countries wrote weighted rubrics spanning everyday stress to emergencies, filling a gap left by crisis-only evals.
OpenAI released MentalHealthBench, an open benchmark scoring AI replies in realistic mental health chats. More than 80 licensed experts from 22 countries wrote weighted rubrics spanning everyday stress to emergencies, filling a gap left by crisis-only evals.
Key Points
Built with 80+ psychologists and psychiatrists across 22 countries and 19 languages
Rubric criteria are weighted from -10 to +10, kept only if 2 of 3 expert reviewers agree
Covers adults, teens aged 13-17, caregivers and clinicians across three acuity levels
GPT-5.6 Sol serves as the automated grader against expert-written criteria
A side study of 44 users found people value practical next steps and tone more than experts do
Why It Matters
Anyone shipping a chatbot that touches wellbeing now has an open, clinician-built yardstick, though a lab grading with its own model deserves outside replication.
Quick Facts
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,564 builders reading daily.