February 2026

ORTHOPAEDICS

TESTING LLM FUNCTIONALITY FOR SPINE CONSULTS

At the recent CSRS Annual Meeting, Srikanth Divi, MD, presented his work evaluating whether large language models (LLMs) can function as a digital spine consultant, accurately synthesizing complex clinical data and supporting surgical triage for cervical spine consults.
 
Using a decade of real-world inpatient consult data with assessments and plans masked, the authors benchmarked multiple LLMs against clinically relevant decision-making metrics.
 
Study Design
The study evaluated three LLMs:
  • ChatGPT‑5
  • DeepSeek‑R1
  • Llama‑4‑Maverick
 
The study gave each LLM clinical tasks:
  • Summarize patient presentation.
  • Determine need for surgery.
  • Recommend a specific procedure (if indicated).
  • Provide rationale for operative vs. nonoperative management.
 
The study graded LLM performance on:
  • Clinical accuracy
  • Correct surgical recommendation
  • Quality and nuance of nonoperative reasoning
 
Results
ChatGPT‑5 consistently outperformed DeepSeek and Llama-4-Maverick across all primary endpoints. It achieved the highest mean accuracy scores with statistically significant separation from the other models.
 
ChatGPT‑5 provided the correct operative recommendation in roughly 78% of cases, compared with DeepSeek (around 53%) and Llama-4-Maverick (around 43%).
 
In addition, ChatGPT‑5 demonstrated more nuanced and surgeon‑level rationale, particularly for nonoperative management decisions.
 
What Worked Well
  • Clear organization of comorbidities, risk factors and longitudinal disease course
  • Effective synthesis of imaging findings with clinical presentation
  • Consistent inclusion of next steps, follow‑up considerations and symptoms to monitor
 
Key Limitations
  • The LLMs performed worse in cases involving myelopathy and cord compression, and those involving geriatric cervical trauma, particularly odontoid fractures.
  • All LLMs tended to over-recommend surgery, often driven by radiology findings.
 
Takeaway
While not a replacement for surgeon expertise, LLMs may help improve consistency and support decision‑making when thoughtfully implemented. This study showed that they have the ability to support triage and consult efficiency, not replace surgeon judgment.
 
What’s Next
The study team plans to:
  • Evaluate other LLMs, such as Claude and Google Gemini
  • Refine prompts and multi‑reviewer validation
  • Provide a broader analysis across all cervical spine consults
Srikanth Divi, MD, is an assistant professor of Orthopaedic Surgery and Neurological Surgery at Northwestern Medicine

Refer a Patient

Northwestern Medicine welcomes the opportunity to collaborate with you in caring for your patients. ​

You May Also Like