Unravelling intubation challenges: a machine learning approach incorporating multiple predictive parameters

Abstract

Background

To protect patients during anesthesia, difficult airway management is a serious issue that needs to be carefully planned for and carried out. Machine learning prediction tools have recently become increasingly common in medicine, frequently surpassing more established techniques. This study aims to utilize machine learning techniques on predictive parameters for challenging airway management.

Methods

This study was cross-sectional. The Shahid Beheshti University of Medical Sciences in Iran’s Loghman Hakim and Shahid Labbafinezhad hospitals provided 622 records in total for analysis. Using the forest of trees approach and feature importance, important features were chosen. The Synthetic Minority Oversampling Technique (SMOTE) and repeated edited nearest neighbor under-sampling were used to balance the data. Using Python and 10-fold cross-validation, seven machine learning algorithms were assessed: Logistic Regression, Support Vector Machines (SVM), Random Forest (INFORMATION-GAIN and GINI-INDEX), Decision Tree, and K-Nearest Neighbors (KNN). Metrics like F-measure, AUC, Recall, Accuracy, Specificity, and Precision were used to evaluate the performance of the model.

Results

Twenty-four important features were chosen from the original 32 features. The under-sampling strategy produced better results than SMOTE. Among the algorithms, KNN (Euclidean, Minkowski) had better performance than other algorithms. The highest values ​​for accuracy, precision, recall, F-measure, and AUC were obtained at 0.87, 0.88, 0.82, 0.85, and 0.87, respectively.

Conclusion

Algorithms for machine learning provide insightful information for anticipating challenging airway management. By making it possible to forecast airway difficulties more accurately, these techniques can potentially improve clinical practice and patient outcomes.

Introduction

A difficult airway scenario occurs when a healthcare provider with adequate airway management skills faces challenges with laryngoscopy, intubation of the trachea, and more especially, ventilating and oxygenating the lungs. These challenges may be predicted based on the patient’s anatomic or pathologic characteristics or may arise unexpectedly while the procedure is being performed. Additionally, resistance to tube placement may occur in some patients [1]. Unexpectedly difficult airway situations increase the risk of brain damage and death [2]. It follows to reduce these risks that preoperative identification of patients at risk for airway difficulties is warranted [3, 4]. Nevertheless, respiratory distress prediction remains a challenging task, and existing predictive accuracy is still quite low [5]. In addition, existing comprehensive clinical evaluation approaches for difficult airways, including the modified LEMON criteria and the Simplified Airway Risk Index (SARI) model, cannot achieve the desired prediction performance in predicting airway management difficulty [6, 7]. Difficult airway management requires a structured procedure, careful evaluation, and a focused effort to maintain oxygenation and minimize complications [8]. Artificial intelligence (AI) encompasses a diverse array of technologies, principles, and methodologies aimed at fostering the development of systems that can replicate and enhance human cognitive abilities. Algorithms are central to AI’s functionality, which drives reasoning, problem-solving, and analytical tasks [9]. AI rapidly transforms various aspects of our lives, and its impact on the medical field is particularly profound. As AI technologies continue to evolve, their potential to revolutionize healthcare delivery is becoming increasingly evident [10]. Machine learning (ML) is a subfield of AI that enables systems to learn from data without being explicitly programmed. ML algorithms can identify patterns, predict, and adapt to new information [11,12,13]. Some research has addressed difficult airway prediction and management using machine learning (Table 1).

Table 1 Related work

These studies show that Machine learning algorithms hold significant promise for predicting airway management challenges and the need for intubation, yet challenges such as selecting more comprehensive predictive features, enhancing data quality, and conducting more rigorous model evaluations persist.

Despite its clinical and research significance, no studies have utilized machine learning approaches to predict a difficult airway from this study’s collated and localized data. Therefore, a data balancing technique was applied to this dataset, and significant features were selected. Then, machine learning algorithms were utilized to predict difficult airway management.

Methods

Study design, setting, and participants

This cross-sectional study assessed 622 adult patients scheduled for elective surgery and referred to Loghman Hakim and Labbafinezhad Hospitals, Tehran, Iran. Inclusion criteria: (1) Patient scheduled for elective surgery requiring general anesthesia and tracheal intubation, (2) The patient should not have any contraindications to the administration of neuromuscular blocking agents, (3) Age greater than 18 years, (4) Ability to open the mouth with inter incisor distance greater than 2 centimeters (for laryngoscope placement), (5) absence of recent cervical spine trauma (< 2 months), (6) No contraindications to anesthesia induction before airway placement. Exclusion criteria: (1) The development of new-onset acute neck pain or injury between the clinic visit and surgery, (2) Patient’s consent withdrawal from the study at any time. Demographic information and a history of medical comorbidities were preoperatively collected from the patient using a structured questionnaire (Appendix). Airway assessments were performed and documented in standardized forms according to the predefined variables. A metal inclinometer and a mobile inclinometer app (Goniometer Records, Orthopedic Research Group Initiative, Indian Orthopedic Research Group, Last updated September 2018) are the two primary tools for measuring the cervical range of motion angles. Anesthesia induction was tailored to the patient’s condition following a complete pre-operative assessment. Ventilation was initiated using an ambu bag, and the bag-mask ventilation score (Grade I: easy to ventilate, Grade II: requiring nasopharyngeal or oral airway to ventilate, Grade III: difficult to ventilate or require two providers, Grade IV: unable to ventilate using all the above maneuvers. Grades III and IV are defined as difficult mask ventilation) [19] was documented. Subsequently, laryngoscopy was performed using a Macintosh laryngoscope with an appropriately sized blade based on the patient’s anatomy. The Cormack-Lehane score was recorded based on the observations and reports of the intubating personnel. In the event of a difficult airway at any stage, the necessary equipment or personnel were utilized based on the patient’s condition. Data were entered daily into an Excel spreadsheet by an anesthesia assistant. These data were then analyzed using the Python programming language, and machine learning algorithms were applied and evaluated.

Data preprocessing

First, features that had the same value in most records were removed. Then the issue of missing data in continuous features was addressed using imputation by mean. Continuous features were then normalized using the MIN-MAX formula, while categorical features were converted to binary representations using One-Hot Encoding. This concluded the data preprocessing stage.

Data balancing

Before data balancing, the 10-fold cross-validation method separated the training and testing data. Due to the imbalanced nature of the data, various balancing techniques were employed in this step. This phase was carried out in two distinct steps:

• Step 1: The SMOTE algorithm was applied to the training data in each fold, augmenting the records of the minority class to match the size of the majority class records. This procedure resulted in a balanced training dataset. The prepared training data was then utilized for the subsequent phase of the experiment.

• Step 2: This step employed the Repeated Edited Nearest Neighbor (RENN) under-sampling technique to address the data imbalance. This method involves reducing the majority class data points to match the size of the minority class. Following data balancing, the hold-out method was utilized to divide the balanced dataset into training (70%) and testing (30%) sets. Feature importance was assessed using a forest of trees.

Selecting data mining algorithms

In this phase, various classification algorithms, including Logistic Regression, SVM (RBF), SVM (POLY), Random Forest (INFORMATION-GAIN), Random Forest (GINI-INDEX), Decision Tree, and KNN (Euclidean, Manhattan, Cosine, Minkowski), were employed for data mining on both the balanced datasets obtained from step 1 and step 2. Below is a brief description of each algorithm:

Logistic regression is a statistics technique that is utilized to model the relationship between a binary outcome variable and one or more predictor variables. In logistic regression, the estimate of the probability of the outcome variable is based on the values of the predictor variables [20]. SVM stands out as one of the most efficacious algorithms in machine learning, especially for classification issues. SVMs can do this by constructing an optimal hyperplane that effectively partitions the data points that belong to different classes. This is done by maximizing the margin between the classes, which ultimately leads to a model with a superior ability to generalize [21].

Decision trees provide a tree-like hierarchical structure for use in machine learning models. This structure features inner nodes representing data attributes, branches representing decision rules based on those attributes, and terminal leaf nodes representing the predicted outcome or class label [22]. Random Forest is an ensemble learning technique that uses decision trees as its base learners. It is operated by constructing a large number of decision trees. Their predictions are then aggregated to improve the final classification or regression output. The core principle of Random Forest is to generate a collection of decision trees during the training phase, with each tree producing a distinct prediction. In the predicting phase, the Random Forest algorithm achieves the result by combining the predictions of all the constituent decision trees [23]. KNN is a common nonparametric machine learning algorithm that can be applied to classification and regression problems. Its function is the identification of the k-nearest data points (neighbors) of a new data point. Then, based on the majority vote (classification) or the average (regression) of the labels or values of its K-nearest neighbors, the class label or value of the new data point is predicted [24].

Algorithm evaluation

In this phase, the performance of the algorithms was assessed using a suite of evaluation metrics, including accuracy, specificity, precision, recall, F-measure, and the area under the ROC curve (AUC). These metrics provided comprehensive insights into the performance of each algorithm. We define the evaluation indices as:

$$\:\text{A}\text{c}\text{c}\text{u}\text{r}\text{a}\text{c}\text{y}=\frac{TP+TN}{TP+TN+FP+FN}$$

(1)

$$\:\text{S}\text{p}\text{e}\text{c}\text{i}\text{f}\text{i}\text{c}\text{i}\text{t}\text{y}=\frac{TN}{TN+FP}$$

(2)

$$\:\text{P}\text{r}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}=\frac{TP}{TP+FP}$$

(3)

$$\:\text{R}\text{e}\text{c}\text{a}\text{l}\text{l}=\frac{TP}{TP+FN}$$

(4)

$$\:\text{F}-\text{m}\text{e}\text{a}\text{s}\text{u}\text{r}\text{e}=\frac{\left(2\:\text{x}\:\text{P}\text{r}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}\:\text{x}\:\text{R}\text{e}\text{c}\text{a}\text{l}\text{l}\right)}{(\text{P}\text{r}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}+\:\text{R}\text{e}\text{c}\text{a}\text{l}\text{l})}$$

(5)

In Eq. (1) to (4), TP, TN, FP, and FN denote the True-Positive, True-Negative, False-Positive, and False-Negative, respectively. The result of Eq. (5) is calculated using the results of Eq. (3) and Eq. (4). Figure 1 shows the step-by-step breakdown of the methodology employed in this study.

Fig. 1

Shows the step-by-step breakdown of the methodology

Results

A total of 622 patient records were collected. The mean age of the patients was $\:3.09\pm\:14.08$ years. Out of 32 features in the collected data, 24 important features were identified. 20 features had discrete values and 4 features were continuous. Sternomental Distance was the most important (0.10) among the features. Table 2 shows the features with a discrete value, the importance, mean, and standard deviation. Table 3 shows the continuous features, their importance, and their classes.

Table 2 Features that have a discrete value

Table 3 Features that have a continuous value

Cormack-Lehane score was the target class. This classification system is based on the visualization of structures during laryngoscopy and is divided into four grades: grade 1: Complete visualization of the laryngeal entrance, grade 2: Partial visualization of the posterior aspect of the laryngeal entrance, grade 3: Only the epiglottis is visible, grade 4: No part of the epiglottis or larynx is visible. If Cormack-Lehane grade 3 is observed, it indicates a difficult laryngoscopy (class 2). Otherwise, it is labeled as Class 1. Class 1 included 531 patients and class 2 included 91 patients. In this study, there were no patients with grade 4.

Figures 2 and 3 illustrate scatter plots depicting the relationships between two variables and their corresponding class labels, both before and after data balancing. These scatter plots provide visual insights into the distribution of data points and the potential existence of class imbalances.

Fig. 2

Scatter plot of selected variables concerning two classes after balancing (under-sampling based on the repeated edited nearest neighbor method)

Fig. 3

Scatter plot of selected variables for two classes before balancing

To evaluate the data’s normality, Q-Q plots were employed in addition to scatter plots. These charts provide a comparison between the observed data’s quantiles and a theoretical normal distribution. The data points in the Q-Q plots deviated from the expected diagonal line, indicating deviations from normalcy prior to balancing. On the other hand, the Q-Q plots improved once the data was balanced, with the data points aligning closer to the diagonal, indicating a more normal distribution of the data. Because many machine learning models, including some utilized in this study, perform better when the data is closer to a normal distribution, this enhanced normality is significant.

The data balancing approach, as demonstrated by the scatter plots and Q-Q plots, not only resolved the problem of class imbalance but also enhanced the dataset’s statistical qualities, which enabled the machine learning models to produce predictions that were more precise and trustworthy. Finally, the robustness of the study’s conclusions was enhanced by these visual aids, which offered important proof that the preprocessing procedures were successful. Figures 4 and 5 present Q-Q plots to assess the normality of the data both before and after balancing.

Fig. 4

Normality of selected variables after balancing (under-sampling based on the repeated edited nearest neighbor method)

Fig. 5

Normality of selected variables before balancing

Table 4 shows a comparative analysis of various machine learning algorithms evaluated after applying two data balancing techniques: SMOTE and the Under-sampling Based on the Repeated Edited Nearest Neighbors (RENN). The KNN algorithms with K = 3 generally outperformed those with K = 5. Minkowski and Euclidean metrics with K = 3 yielded the highest accuracy (0.87). SVM (POLY and RBF) achieved respectable accuracies of 0.82 and 0.80, respectively. The Decision Tree achieved a moderate accuracy of 0.77. Random Forest had slightly better accuracy when using Information Gain (0.76) than the Gini index (0.74). Logistic Regression, with an accuracy of 0.71, demonstrated the weakest performance compared to the other models.

Algorithms in the RENN method generally outperformed SMOTE across all metrics. When using the SMOTE technique, only Random Forest was able to produce the same level of accuracy as RENN.

Table 4 Performance of different classification algorithms after data balancing

Figure 6 shows the ROC curves for different classification algorithms when the data has been balanced using the Under-sampling Based on the RENN Method. The best performance was achieved when the number of neighbors was optimally set to K = 3 and the Minkowski metric was used, which allowed greater flexibility in calculating the distance. The SVM models with RBF and Polynomial kernels showed acceptable performance and the Polynomial kernel was slightly better. Moderate performance was seen by the Random Forest (Information-Gain and Gini-Index). The Decision Tree, with an AUC of 0.77, outperformed Logistic Regression but still demonstrated lower accuracy and discriminative power compared to other models.

Fig. 6

ROC curves of different classification algorithms after balancing (under-sampling based on the repeated edited nearest neighbor method)

According to the findings presented in Table 4; Fig. 6, the KNN algorithm with three neighbors and Euclidean, Minkowski, and Cosine distance metrics outperformed the other algorithms.

Discussion

This study aims to determine how machine learning algorithms can be used to predict outcomes in challenging airway management, a critical area of anesthesia. Several algorithms, including KNN, Random Forest, Logistic Regression, and SVM were trained using data from 622 patient records gathered from two hospitals in Iran. 24 significant features were found when the data was preprocessed and balanced, with Sternomental Distance being the most significant. The KNN method yielded the best accuracy of 0.87, especially when Euclidean and Minkowski metrics were used. These findings imply that machine learning can dramatically improve prediction accuracy in clinical settings, improving patient outcomes and enhancing the efficacy of airway control techniques.

Zhou’s research showed that machine learning models, particularly gradient boosting algorithms, were very successful in predicting difficult airway intubations, which is similar to the current study. With an AUC of more than 0.80 and a flawless precision score, the Gradient Boosting Machine (GBM) proved to be the most effective algorithm in their investigation [25]. The utilization of physical metrics like BMI and Sternomental Distance and the excellent performance of ML models like GBM were constant across both investigations, although KNN was not the best model in Zhou’s investigation. Wang et al. showed that the performance of machine learning algorithms, namely Naïve Bayes and Random Forest, was best for predicting difficult tracheal intubation (DTI) and difficult laryngoscopy (DL) with AUC values of 0.95 and 0.90 respectively. Linear models such as Logistic Regression and Decision Tree are less accurate and sensitive to predicting intubation difficulties than these models. The results of the present study are consistent with these findings, especially for the superior performance of complex models over simpler ones [26]. Hayasaka et al. found that Convolutional Neural Networks (CNN) can predict intubation difficulty with high accuracy (AUC = 0.86) using facial images [27]. The closeness of the AUC values between the CNN model and the KNN model in this study indicates how machine learning techniques can be applied broadly to a variety of methods, such as facial image analysis and measurements of the neck circumference and mouth opening. Xia et al. trained a neural network using facial images and the Light Gradient Boosting Machine algorithm, achieving a commendable Area Under the Curve (AUC) of 0.77 for predicting intubation difficulty [28]. Although this AUC was slightly lower than in the current study, probably because the feature sets and models were different, it nevertheless shows how consistently machine learning models outperform conventional techniques in terms of prediction accuracy. Cuendet et al.‘s study, utilizing Random Forest algorithms, yielded an AUC of 0.77, which while promising, was lower than the AUC of the KNN model employed in the current study [29]. Nonetheless, to improve prediction accuracy, both research stresses how crucial it is to combine various features. Although the ensemble aspect of Random Forest contributed to its good performance, KNN’s use of distance-based metrics appeared to provide better results in this instance.

In summary, machine learning techniques show promising results for predicting difficult airway management. The choice of physical parameters used to train the algorithm impacts the accuracy of these algorithms. With the right choice of parameters, machine learning algorithms can significantly impact clinical practice by improving the prediction of difficult airways, which can also lead to better patient outcomes.

Limitations and future research directions

Using only one type of anesthesia technique makes the results non-generalizable to other anesthesia methods. The model could become more accurate in many situations if we consider other more diverse techniques. Furthermore, cases of emergency surgery were not included in this study. However, this may limit the model’s applicability in emergencies since emergencies need faster diagnoses and better models. Variables used may not always be available in clinical settings or may not be easy to measure. Such a limitation would restrict the use of the model in general environments or emergencies. The exclusion of patients of different age groups (children or elderly) may make the findings unreliable for some age groups (e.g. children, elderly). The different physiological characteristics in these groups may require specific algorithms. Given that the data was collected by two specialists, there is a potential for human error, which could affect the accuracy of the data and, ultimately, the model’s accuracy. It is recommended that in future research, additional features, including physiological and biological characteristics (such as cardiovascular or respiratory health status), be added to the model. This could enhance prediction accuracy in various clinical conditions.

Incorporating imaging data, such as radiology images or CT scans of the airway, could assist machine vision algorithms in providing more accurate predictions in more complex conditions.

Examining the impact of specific characteristics of different age groups (children, elderly) or particular medical conditions (such as patients with chronic respiratory diseases) could help improve the model’s performance in special conditions. Using deep learning algorithms and more complex neural networks could help better simulate complex airway patterns and improve prediction accuracy.

Conclusions

These studies show that machine learning, particularly KNN models, can provide accurate predictions in clinical contexts, assisting anesthesiologists in anticipating and better managing problematic airways. These algorithms are useful tools for enhancing patient outcomes and safety because of their capacity to examine several complex variables at once. The study concludes that machine learning has a lot of promise to improve preoperative planning by helping doctors identify patients who are more likely to experience difficulties in the airway and, as a result, reduce unfavorable outcomes following surgery. Applying such predictive models may result in more accurate and customized airway management techniques in practice, which would eventually lower the risk of problems and raise the standard of anesthesiology treatment as a whole.