BACKGROUND: Administrative billing codes commonly misclassify complex diseases like heart failure (HF) in the electronic health record (EHR). Artificial intelligence (AI) approaches, including natural language processing (NLP), may improve disease adjudication.
OBJECTIVES: We validated an existing NLP model and specifically examined accuracy across HF subtypes (preserved vs reduced ejection fraction: HFpEF, EF ≥50%; HFrEF, EF <50%) in an EHR-based sample.
METHODS: We identified individuals who sought primary care between 2004-2014 at Beth Israel Deaconess Medical Center. Incident HF hospitalizations were identified using International Classification of Diseases (ICD) codes, and gold-standard HF adjudication was performed in 500 random cases. We applied a preexisting NLP model to classify each case and then further fine-tuned the model. We calculated diagnostic metrics for overall HF and by subtype.
RESULTS: Among 156,409 individuals, 5014 incident HF cases were identified. The fine-tuned NLP model outperformed primary ICD codes, yielding a superior kappa (0.73 vs 0.49), negative predictive value (93% vs 73%), and sensitivity (89% vs 52%), with similar positive predictive value (79% vs 87%) and specificity (86% vs 94%). Notably, NLP performance was robust across HF subtypes, whereas primary ICD codes had worse performance for identification of HFpEF. When applied to the full cohort, NLP identified 84% more HFrEF and 99% more HFpEF candidate cases than primary ICD codes.
CONCLUSION: NLP accurately classifies HF cases across subtypes, with improved accuracy after fine-tuning. AI-based approaches represent a critical opportunity to improve disease ascertainment at scale, nearly doubling the number of HF cases identified in the EHR.