Abstract
IMPORTANCE: Despite increasingly widespread use of artificial intelligence (AI)-driven ambient scribes in medicine, the extent to which they are associated with clinician practice is not well studied.
OBJECTIVE: To characterize differences in documentation and treatment of psychiatric symptoms in primary care outpatient notes generated using ambient scribes compared with human or no scribes.
DESIGN, SETTING, AND PARTICIPANTS: This cohort study used a matched retrospective case-control design to evaluate primary care annual visit notes from the Massachusetts General and Brigham and Women's Hospital systems between February 2023 and February 2025. A random sample of notes from 4 types of visits, matched 1:1 using sociodemographic and clinical features, was used: those using an ambient scribe, those using a human scribe, those occurring during the same period without a scribe (contemporaneous), and those occurring prior to scribe deployment. Data analysis was performed from April 25 to May 1, 2025.
EXPOSURE: Use of an AI ambient scribe.
MAIN OUTCOMES AND MEASURES: Neuropsychiatric symptom documentation, in terms of estimated Research Domain Criteria (RDoC), using a Health Insurance Portability and Accountability Act-compliant large language model (GPT-4o version gpt-4o-11-20; OpenAI); antidepressant prescriptions and diagnostic codes; and referral for mental health follow-up.
RESULTS: Among 20 302 notes, the mean (SD) age of the patients was 48 (14) years and 11 960 (59%) were for visits by female patients; 1026 (5%) met criteria for moderate or greater depressive symptoms by Patient Health Questionnaire-9 score. Estimated levels of RDoC symptoms in all 6 domains were significantly greater in the AI-scribed notes compared with other groups. In a multiple logistic regression model, likelihood of a psychiatric intervention (referral, new diagnosis, or antidepressant prescription) was significantly lower among AI-scribed visits compared with contemporaneous unscribed visits (adjusted odds ratio, 0.83; 95% CI, 0.72-0.95), but not for human-scribed visits compared with contemporaneous unscribed visits (adjusted odds ratio, 0.97; 95% CI, 0.85-1.11).
CONCLUSIONS AND RELEVANCE: In this retrospective cohort study using a matched case-control design examining outpatient primary care notes, incorporation of AI ambient scribes in primary care was associated with greater levels of neuropsychiatric symptom documentation but lesser likelihood of documented management of psychiatric symptoms. Further study will be required to determine whether these changes are associated with differential outcomes.