Clinical prediction models for spine surgery applications are on the rise, with an increasing reliance on machine learning (ML) and deep learning (DL). Many of the predicted outcomes are uncommon; therefore, to ensure the models’ effectiveness in clinical practice it is crucial to properly evaluate them. This systematic review aims to identify and evaluate current research-based ML and DL models applied for spine surgery, specifically those predicting binary outcomes with a focus on their evaluation metrics. Overall, 60 papers were included, and the findings were reported according to the PRISMA guidelines. A total of 13 papers focused on lengths of stay (LOS), 12 on readmissions, 12 on non-home discharge, 6 on mortality, and 5 on reoperations. The target outcomes exhibited data imbalances ranging from 0.44% to 42.4%. A total of 59 papers reported the model’s area under the receiver operating characteristic (AUROC), 28 mentioned accuracies, 33 provided sensitivity, 29 discussed specificity, 28 addressed positive predictive value (PPV), 24 included the negative predictive value (NPV), 25 indicated the Brier score with 10 providing a null model Brier, and 8 detailed the F1 score. Additionally, data visualization varied among the included papers. This review discusses the use of appropriate evaluation schemes in ML and identifies several common errors and potential bias sources in the literature. Embracing these recommendations as the field advances may facilitate the integration of reliable and effective ML models in clinical settings.

Limitations in Evaluating Machine Learning Models for Imbalanced Binary Outcome Classification in Spine Surgery: A Systematic Review

OBJECTIVE
In recent years, machine learning models for clinical prediction have become increasingly prevalent in the neurosurgical literature. However, little is known about the quality of these models, and their translation to clinical care has been limited. The aim of this systematic review was to empirically determine the adherence of machine learning models in neurosurgery with standard reporting guidelines specific to clinical prediction models.


METHODS
Studies describing the development or validation of machine learning predictive models published between January 1, 2020, and January 10, 2023, across five neurosurgery journals (Journal of Neurosurgery, Journal of Neurosurgery: Spine, Journal of Neurosurgery: Pediatrics, Neurosurgery, and World Neurosurgery) were included. Studies where the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines were not applicable, radiomic studies, and natural language processing studies were excluded.


RESULTS
Forty-seven studies featuring a machine learning-based predictive model in neurosurgery were included. The majority (53%) of studies were single-center studies, and only 15% of studies externally validated the model in an independent cohort of patients. The median compliance across all 47 studies was 82.1% (IQR 75.9%-85.7%). Giving details of treatment (n = 17 [36%]), including the number of patients with missing data (n = 11 [23%]), and explaining the use of the prediction model (n = 23 [49%]) were identified as the TRIPOD criteria with the lowest rates of compliance.


CONCLUSIONS
Improved adherence to TRIPOD guidelines will increase transparency in neurosurgical machine learning predictive models and streamline their translation into clinical care.

/pdf/machine-learning-predictive-models-in-neurosurgery-an-1bnaxans.pdf

Machine learning predictive models in neurosurgery: an appraisal based on the TRIPOD guidelines. Systematic review.

Artificial intelligence and machine learning (ML) can offer revolutionary advances in their application to the field of spine surgery. Within the past 5 years, novel applications of ML have assisted in surgical decision-making, intraoperative imaging and navigation, and optimization of clinical outcomes. ML has the capacity to address many different clinical needs and improve diagnostic and surgical techniques. This review will discuss current applications of ML in the context of spine surgery by breaking down its implementation preoperatively, intraoperatively, and postoperatively. Ethical considerations to ML and challenges in ML implementation must be addressed to maximally benefit patients, spine surgeons, and the healthcare system. Areas for future research in augmented reality and mixed reality, along with limitations in generalizability and bias, will also be highlighted.

Machine Learning in Spine Surgery: A Narrative Review.

Machine Learning (ML) and Deep Learning (DL) models show potential in surpassing traditional methods including generalised linear models for healthcare predictions, particularly with large, complex datasets. However, low interpretability hinders practical implementation. To address this, Explainable Artificial Intelligence (XAI) methods are proposed, but a comprehensive evaluation of their effectiveness is currently limited. The aim of this scoping review is to critically appraise the application of XAI methods in ML/DL models using Electronic Health Record (EHR) data. In accordance with PRISMA scoping review guidelines, the study searched PUBMED and OVID/MEDLINE (including EMBASE) for publications related to tabular EHR data that employed ML/DL models with XAI. Out of 3220 identified publications, 76 were included. The selected publications published between February 2017 and June 2023, demonstrated an exponential increase over time. Extreme Gradient Boosting and Random Forest models were the most frequently used ML/DL methods, with 51 and 50 publications, respectively. Among XAI methods, Shapley Additive Explanations (SHAP) was predominant in 63 out of 76 publications, followed by partial dependence plots (PDPs) in 11 publications, and Locally Interpretable Model-Agnostic Explanations (LIME) in 8 publications. Despite the growing adoption of XAI methods, their applications varied widely and lacked critical evaluation. This review identifies the increasing use of XAI in tabular EHR research and highlights a deficiency in the reporting of methods and a lack of critical appraisal of validity and robustness. The study emphasises the need for further evaluation of XAI methods and underscores the importance of cautious implementation and interpretation in healthcare settings. 

The application of explainable artificial intelligence (XAI) in electronic health record research: A scoping review

Leveraging its ability to handle large and complex datasets, artificial intelligence can uncover subtle patterns and correlations that human observation may overlook. This is particularly valuable for understanding the intricate dynamics of spinal surgery and its multifaceted impacts on patient prognosis. This review aims to delineate the role of artificial intelligence in spinal surgery. A search of the PubMed database from 1992 to 2023 was conducted using relevant English publications related to the application of artificial intelligence in spinal surgery. The search strategy involved a combination of the following keywords: "Artificial neural network," "deep learning," "artificial intelligence," "spinal," "musculoskeletal," "lumbar," "vertebra," "disc," "cervical," "cord," "stenosis," "procedure," "operation," "surgery," "preoperative," "postoperative," and "operative." A total of 1,182 articles were retrieved. After a careful evaluation of abstracts, 90 articles were found to meet the inclusion criteria for this review. Our review highlights various applications of artificial neural networks in spinal disease management, including (1) assessing surgical indications, (2) assisting in surgical procedures, (3) preoperatively predicting surgical outcomes, and (4) estimating the occurrence of various surgical complications and adverse events. By utilizing these technologies, surgical outcomes can be improved, ultimately enhancing the quality of life for patients. 

Revolutionizing spinal interventions: a systematic review of artificial intelligence technology applications in contemporary surgery

Robust Prediction of Non-home Discharge After Thoracolumbar Spine Surgery with Ensemble Machine Learning and Validation on a Nationwide Cohort.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Eric E. Geng

Author Tools

Chat about Author

Papers

Robust Prediction of Non-home Discharge After Thoracolumbar Spine Surgery with Ensemble Machine Learning and Validation on a Nationwide Cohort.