North Texas Research Forum 2026
Files
Download Full Text (419 KB)
Division
North Texas
Hospital
Medical City Fort Worth
Specialty
Internal Medicine
Document Type
Poster
Publication Date
2026
Keywords
cardiogenic shock, large language model, artificial intelligence, AI, clinical notes
Disciplines
Cardiology | Cardiovascular Diseases | Internal Medicine | Medicine and Health Sciences
Abstract
Background Automated adjudication of cardiogenic shock (CS) from unstructured clinical documentation can facilitate clinical case finding and research trial/registry enrollment. While Large Language Models (LLMs) hold considerable promise, commercial cloud-based solutions raise data privacy and cost concerns. We assessed the performance of an open-source, locally run LLM for the adjudication of CS from unstructured progress notes.
Methods Patients from a 22-hospital healthcare system from 2021 to 2024 were identified using ICD-10 codes to form an enriched CS evaluation cohort with a ~1:1 case/non-case ratio. We selected gpt-oss-20b, an open-source LLM capable of running locally on a consumer-grade computer, and tested it without task-specific fine tuning. A total of 105 de-identified cardiology progress notes from distinct patients were independently reviewed by two physicians and the LLM to determine whether the patient had CS or not. Inter-rater agreement was assessed pairwise among reviewers.
Results Inter-reviewer agreement for CS classification between the two physicians was 90% (94/105 notes). Agreement between the LLM and two physicians were 79% and 82%, respectively, in the whole cohort. In a consensus subset (n = 94) where both physicians agreed, the LLM achieved 84% agreement with the human consensus. Among these cases, physicians identified CS in 44 (47%) patients, whereas the LLM classified 49 (52%) as having CS, indicating a slight over-estimation.
Conclusions An open-source, locally run LLM without fine-tuning demonstrated strong concordance with human consensus in identifying CS from unstructured clinical notes. This approach offers a scalable, privacy-safe, and cost-effective alternative to manual adjudication for clinical and research case finding and enrollment.
Original Publisher
HCA Healthcare Graduate Medical Education
Recommended Citation
Adusumilli, Devika; Taylor, Luke; Mahr, Claudius; and Li, Song, "Automated Adjudication of Cardiogenic Shock from Unstructured Clinical Notes Using an Open-Source Large Language Model" (2026). North Texas Research Forum 2026. 43.
https://scholarlycommons.hcahealthcare.com/northtexas2026/43