Assessing the efficacy of difficult airway alerts in electronic medical records: a quality improvement study

Abstract

Study design

Retrospective analysis of difficult airway alerts in a major tertiary centre.

Objective

To investigate the completeness, utility and efficacy of difficult airway alerts and their impact on future airway management episodes.

Design, data sources and methods

Patients who had a “difficult airway” entry on electronic medical records (EMR) from 2011 to 2021 were included. Each alert was analyzed by a team of anesthetists with expertise in difficult airway management for its quality, appropriateness, and impact on future airway management episodes. Alert quality was defined as to whether the content of the alert contained pertinent information for emergent airway management. Alert appropriateness was defined as to whether the experts would be unhappy to perform a rapid sequence induction and intubation, if required, following review of all available documentation.

Results

141 patients were included for this study, with a mean age of 58.6 +/- 15.3 years. Ninety-three (66%) alerts were created by medical staff, of which 52 were recorded by consultant anesthetists. 117 alerts (83%) were deemed to be appropriate by the airway expert team, but only 40 alerts (28%) were found to have sufficient quality to be helpful in emergent airway management. Sixty-five patients (47%) had at least one subsequent airway management episode, of which 35 patients (56%) underwent a change of management following alert creation. We proceeded to modify 103 alerts (73%) to improve their quality to aid future encounters.

Conclusion

Difficult Airway encounters are an uncommon event in anesthesia, but clear, comprehensive and effectively communicated documentation is required to minimize the risk in future encounters. In our institution, while most difficult airway alerts were appropriate, we found significant heterogeneity in the quality of this documentation, which limits the clinical utility of the alert system. We have taken measures to improve local processes of difficult airway documentation and considered the implications of our project for the broader airway management community.

Trial registration

Not applicable.

Introduction

Difficult and failed airway management are important contributors to patient morbidity and mortality [1,2,3]. Previous airway difficulty is an important predictor of future airway difficulty [4, 5]. Real time access to previously created difficult airway communication has been recognized as an effective way of reducing risk for future airway encounters [2, 6, 7].

Difficult airway communications describe clinical features predictive of difficulty or descriptions of previous airway management attempts [3, 8]. Traditionally, this information has been communicated using anesthesia records and difficult airway letters. However, this information may not be readily available to subsequent clinicians, especially in time critical situations [5]. Complicated and poorly developed processes for information dissemination contribute to this problem [2, 9, 10].

Many institutions, including our own, utilize a “difficult airway alert” feature within the electronic medical record (EMR) to assist in readily identifying patients known to have predicted or previous airway management difficulty [11]. While these alerts are easy to create, the data collected is unstructured and may not be useful to future airway operators. More recently, a Difficult Airway Alert form has been developed by Queensland Health, which has clear criteria for alert creation, and facilitates structured information collection. This form has since been endorsed by the Australian and New Zealand College of Anaesthetists (ANZCA), and has been implemented across Australia [8, 12, 13].

In this audit and quality improvement project, we aimed to determine the circumstances under which difficult airway alerts are created in our institution; the appropriateness and quality of this documentation; and whether the entry of these alerts informed subsequent airway management. We hypothesized that while most difficult airway alerts would be considered appropriate, many would lack sufficient quality as to be useful in the event of emergent airway management. We further aimed to update the content of these alerts with relevant information found within the medical record, and to use our findings to inform future efforts to improve difficult airway communication within our institution. We hope that our project can help further inform other institutions within Australia and around the world in the optimization of difficult airway documentation.

Methods

Design & setting

We conducted a retrospective analysis in a major tertiary public hospital in Melbourne, Australia. Our hospital is a regional referral centre for head and neck pathologies and complex airway surgery. We have an experienced multi-disciplinary team consisting of ENT, Faciomaxillary, and Thoracic Surgeons, Anesthetists, Intensivists, Emergency Physicians and Allied Health staff with considerable experience in complex airway assessment and management. This quality improvement project was granted ethical approval by the hospital’s ethics committee with a waiver of patient consent granted (RiskmanQ Number: 41033).

Difficult airway alert system

Our EMR (Cerner, Kansas City, Missouri, USA) includes a clinical alert function, which allows “difficult airway alerts” to be recorded and displayed in the banner of the patient’s record. These alerts contain a text field for recording additional information, including reasons for difficulty, and can reference the existence of other relevant documents within the medical record. The clinician creating the alert, and the timestamp are automatically recorded on alert creation (Fig. 1).

Fig. 1

Difficult Airway Alert ExampleScreenshot of the difficult airway alert implementation (green) from the electronic medical record, with patient information redacted (red). Quality improvement has been performed on this particular alert

Difficult airway alerts were created at the discretion of individual clinicians. Throughout the study period, our institution did not have any specific criteria that mandated the placement of a difficult airway alert. Instructions on entering difficult airway alerts are available in our institutional policy and protocol repository and are explained to critical care staff during orientation. However, difficult airway alerts can be created by any clinician with access to the electronic medical record, including both: medical staff, including anesthetists, intensivists, emergency physicians and trainees; and non-medical staff, including perioperative and ward nursing staff, allied health clinicians and administrative staff. Difficult airway alerts are occasionally (but inappropriately) recorded as a generic, free-text alert, which may not appear in the alerts banner of the EMR.

Patient selection

Patients who had a difficult airway alert created between January 2011 and March 2021 were eligible for inclusion. To ensure both standardized and non-standardized difficult airway alerts were captured in this analysis, a structured search of the medical record was performed to identify all patient encounters where any alert field contained any of the keywords: “airway”, “intubation”, “fiberoptic”, “ventilation”, “CICO”, and “laryngoscopy”. Alerts that did not indicate “difficulty” were subsequently excluded (Appendix: Table 1). All patients undergoing elective or emergent airway management at our centre were included.

Data collection

The EMR was reviewed to ascertain the demographics of the patient at the time of alert creation; the role of the clinician creating the alert; the type and contents of the alert; any available documentation regarding airway management and/or difficulty prior to the placement of the alert; and any available documentation regarding subsequent airway management following alert placement.

Outcomes

Our two primary outcomes were the appropriateness and the quality of each difficult airway alert.

Alert appropriateness was judged by expert consultant anesthetists with an interest and expertise in difficult airway management and defined as to whether the experts would be unhappy to perform a rapid sequence induction and intubation, if indicated, based on all available documentation [3]. This definition was selected because it reflected the prevailing practice within our institution throughout the study period. Two experts independently judged each alert, and a third expert was used as an arbitrator.

Alert quality was defined as whether the alert would be useful in the event of emergent airway management by the following composite criteria: whether the alert utilized the standard alert type; and whether the information within the alert contained at least one “suggested indication” for creation of a difficult airway alert by ANZCA-endorsed Queensland Health guidelines (Fig. 2) [8]. This permitted an objective assessment of the pertinence of the information contained within the alert to airway difficulty. Reference to previous anesthetic records or letters was also recorded.

Fig. 2

Queensland Health Difficult Airway Alert – Suggested Indications [8]

Secondary outcomes included the associations between: alert appropriateness and alert quality; alert appropriateness and the role of clinician creating the alert; alert quality and the role of the clinician creating the alert; and changes in subsequent airway management following alert placement. Clinician role was classified into medical and non-medical disciplines, and further subclassified into specialties. Subsequent airway management with endotracheal intubation was described as an “escalation” or “de-escalation” of laryngeal visualization or instrumentation with reference to the initial management (direct laryngoscopy < video laryngoscopy < awake tracheal intubation < awake tracheostomy).

Quality improvement

We performed quality improvement on the identified alerts by creating an addendum to the original alert with relevant information found through this review, where applicable. We did not invalidate or delete any difficult airway alerts, even with evidence to the contrary, out of an abundance of caution. We did not perform quality improvement on any patient who had since died.

Statistical analysis

All data were reviewed for collection or coding errors prior to analysis. No assumptions were made regarding missing data. Descriptive statistics were used to report data relating to alert creation, alert appropriateness and quality, and initial and subsequent airway management. The number of events (N) and percentage (%) of all events are reported for binary variables. The mean (µ) and standard deviation (SD) are reported for normally distributed variables. The median and interquartile range (IQR) are reported for non-normally distributed variables.

Contingency table and chi-squared analysis were performed to explore associations between alert appropriateness and quality. Univariate logistic regression models were constructed to explore the associations between both alert appropriateness and quality with the role of the clinician creating the alert; and changes in subsequent airway management following alert placement. The odds ratio (OR), 95% confidence interval (95%CI) and P-value are presented for each model. A two-tail p-value less than 0.05 was considered statistically significant. Statistical analyses were performed using Stata/SE version 15.1 (StataCorp, College Station, TX, USA).

Results

Patient cohort

A total of 143 patients had difficult airway alerts recorded during the ten-year study period. Two alerts had previously been invalidated, one due to a wrong-patient error, and another due to unspecified reasons. The remaining 141 patients with active difficult airway alerts were analyzed (Fig. 3).

Fig. 3

Patient Flow Diagram

The mean patient age was 58.5 ± 15.3 years at the time of alert creation, and 95 patients were male (67%). Documentation of a difficult airway encounter was found in 128 patients (91%), of which, a consultant anesthetist was present in nearly all encounters (N = 123, 96%). Despite reported difficulty, endotracheal intubation under anesthesia was eventually achieved in 110 patients (86%). Alternative airways were achieved by awake tracheal intubation in 11 patients (9%) and supraglottic airway placement (SGA) in 5 patients (4%). There were 8 reports of impossible endotracheal intubation, 2 reports of impossible SGA insertion, and 2 reports of impossible BMV. There were no cases where all three modalities were impossible, and there were no cases that required emergency front of neck access.

Ninety-three (66%) alerts were recorded by medical staff, of which 52 were recorded by consultant anesthetists, 32 were recorded by anesthesia trainees, 7 were recorded by intensive care physicians, and 2 were recorded by emergency physicians. Forty-three (30%) alerts were recorded by nursing staff, and two (1%) alerts were created by other health staff.

Appropriateness & quality

Of the 141 patients, a total of 117 alerts (83%) were deemed to be appropriate, while 24 alerts (17%) were deemed to be inappropriate. However,, only 40 alerts (28%) were found to have sufficient quality as to be helpful in emergent airway management. Amongst all patients with a difficult airway alert, high quality alerts had a high positive predictive value (90%) for predicting alert appropriateness, but low negative predictive value (20%), sensitivity (42%), and specificity (24%) (Table 1).

Table 1 Difficult airway alert appropriateness vs. quality

There was no association between alert appropriateness and alert quality, or any component of our definition of alert quality (Table 2). The standardized difficult airway alert was utilized in 90 patients (64%), while the remainder were recorded using a free-text alert. In 105 patients (74%), the alert text field was populated with information related to the difficult airway encounter, however only 65 alerts (46%) were found to contain information that met at least one of the Queensland Health suggested indications for providing a difficult airway alert (Fig. 2). Only 29 alerts (23%) contained clear instructions on how to access this documentation within the EMR.

Table 2 Difficult airway alert appropriateness vs. quality components

Clinician role

The associations between clinician role and both alert appropriateness and quality is shown in Table 3. Appropriate alerts were more likely to have been entered by medical staff than non-medical staff (OR = 3.42, 95%CI = 1.38–8.44, p = 0.008). There was no difference between alert appropriateness between consultant anesthetists and anesthetic trainees (p = 0.224). Quality alerts were more likely to have been entered by medical staff than non-medical staff (OR = 4.03, 95%CI = 1.55–10.5, p = 0.004), and more likely to have been entered by anesthetic trainees compared to consultant anesthetists (OR = 3.40, 95%CI = 1.33–8.67, p = 0.01).

Table 3 Difficult airway alerts appropriateness and quality, stratified by recording clinician role

Subsequent airway management

Sixty-five patients (47%) had at least one subsequent airway management encounter following the creation of their difficult airway alert. A change in subsequent airway management was observed in 35 of these patients (54%), of which 19 patients underwent endotracheal intubation with escalation of laryngeal airway visualization or instrumentation, 5 patients underwent endotracheal intubation with de-escalation of laryngeal airway visualization or instrumentation, 10 patients were managed with SGA, and 1 patient was managed without a definitive airway using jet ventilation. There was no association between a change in airway management and either difficult airway alert appropriateness (p = 0.148) or quality (p = 0.684).

Quality improvement

Quality improvement was performed on a total of 103 alerts (73%), where the comments field was amended to improve the content of the alert, including details of previous or subsequent airway management, and references to anesthetic charts or letters. Only 18 alerts (13%) were found to be sufficiently detailed and did not warrant modification or revision. Quality improvement was not performed in 20 patients that were deceased at the time of review.

Discussion

Key findings

In this retrospective analysis in a major metropolitan hospital, we identified 141 active difficult airway alerts over a 10-year period. Although over 8 in 10 alerts were found to be appropriate, only 1 in 4 alerts were found to be useful in the setting of emergent airway management. While high quality alerts were nearly always appropriate, most poor-quality alerts were also appropriate, and therefore cannot be ignored. We updated the content of nearly three quarters of alerts to improve their quality.

Despite the near ubiquitous presence of a consultant anesthetist during the initial difficult airway encounter, we found that a significant proportion of alerts were created by non-medical staff. The appropriateness and quality of these alerts were lower than alerts created by medical staff. Entry by anesthetic trainees was associated with greater alert quality, potentially due to greater familiarity with the EMR and experience in airway documentation.

Comparison with the literature

While several studies have explored the utility of difficult airway alert letters and cards [14, 15], or the implementation of a new difficult airway alert system into the EMR [11, 16], no studies have specifically looked at the performance of an established difficult airway alert system in an existing EMR, particularly with respect to the appropriateness or quality of the documentation created.

A significant barrier preventing the successful implementation of a difficult airway alert system is clinician compliance in creating the alert once a difficult airway had been identified. In a New Zealand cohort, only 12% of patients with a known difficult airway had a difficult alert placed within their hospital EMR [9]. Automatic alert creation using rule-based criteria from electronic anesthetic records has been shown to significantly improve alert creation compliance but may result in false-positive alert creation, where no difficulty was encountered with subsequent airway management [17].

Implications for practice

Our study has confirmed that the presence of a difficult airway alert should be taken seriously amongst airway operators as the vast majority were found to be appropriate after review by expert clinicians. However, we have identified barriers that prevent difficult airway alerts from being clinically useful in an airway emergency in our institution, as well as solutions that may be applicable to other institutions. These fall into four areas:

Firstly, criteria for difficult airway alert creation were not clear and specific. Advice for determining for whom a difficult airway communication should be created has varied [12], ranging from clinician discretion [18] to a specific list of indications or criteria [3, 8, 12, 13]. While most active alerts were found to be appropriate, the lack of specific criteria for alert creation raises the possibility that some patients with actual airway difficulty may not have received a difficult airway alert.

Secondly, there was great heterogeneity in the quality of the alerts created, which limits the utility of the system in a time critical scenario. Our current alert system relies on airway information to be entered within a free-text field, which is frequently left incomplete.

Thirdly, the use of an EMR provides both efficiencies and barriers to communication. In our institution, the patient’s record states that an alert has been “Recorded” following difficult airway alert creation but does not reveal the nature or content of the alert until the “Dx Problems/Alerts” tab is selected (Fig. 1). Without specific experience or training, these steps may not be performed, particularly in a time critical scenario.

Finally, there is currently no effective system for distributing difficult airway communication between individual institutions in Victoria. While national and state-wide difficult airway alert databases have been established elsewhere [3, 7, 12], further work is needed to develop an effective solution across Australia as part of the National Digital Health Strategy and broader efforts to streamline digital health information [12, 13]. We expect that many health institutions around the world face a similar challenge.

In response to the first three barriers identified, we have implemented a local version of the Queensland Health Difficult Airway Alert form (Fig. 4) which provides clear indications for creating difficult airway alerts, and a structured way to communicate this information to future airway operators. It was technically impossible to incorporate this form into our primary EMR. We therefore incorporated it into our scanned medical record system (Clinical Patient Folder, InfoMedix, Melbourne, VIC, 3000). While this potentially provides another barrier to access, the form is displayed prominently when filtering for anesthesia records. An organizational-wide education campaign was conducted to inform clinicians of this change with accompanying instructions on the hospital’s intranet. This documentation is best completed by medical staff with detailed knowledge of the reason for airway difficulty.

Fig. 4

Difficult airway alert documentation revised implementation

Strengths and limitations

Our analysis has many strengths, namely that we have comprehensively appraised the difficult airway alert system in a major tertiary centre with a high throughput of at-risk patients over a decade. We identified factors that may improve the performance of a difficult airway alert system, and in turn, patient care. Furthermore, we were able to update many of our own alerts to make these more clinically useful should these patients re-present to our institution in the future.

Our study has several limitations. Our sample size is relatively small, which likely reflects that a truly difficult airway is a relatively rare event [19]. As our analysis was retrospective, causation cannot be determined, and our conclusions may be the result of an unmeasured confounder. This audit includes analysis of all reported difficult airway events but does not enable us to draw any conclusions about whether airway alerts are completed in all patients in whom they were indicated. Unfortunately, our EMR does not capture this information, and it is therefore likely that some events were missed. While our findings may be specific to our institution and not be perfectly generalizable, our EMR is widely used in the public health sector in Australia, and it is likely that our insights will be directly relevant for many other institutions worldwide.

Conclusion

While most difficult airway alerts entered into our EMR were appropriate, we found that most difficult airway alerts lacked sufficient quality to be useful for emergent airway management. We have since implemented a new system for documenting difficult airway management. However, despite our best efforts, we still lack a perfect system for communicating difficult airway events within our institution. Within Australia, we aspire towards a national strategy to minimize adverse outcomes from subsequent difficult airway management, and we believe that health systems and institutions around the world should aim to achieve the same.