Delirium is a psychiatric disorder presenting as an acute deterioration of brain function leading to a fluctuating level of consciousness and an inability to maintain attention.1 It is a common complication after cardiac surgery with a prevalence ranging from 3-47% of patients.2-5 This large variation in the reported incidence of postoperative delirium depends on the study design, the patient population studied, and the sensitivity and specificity of the methods used for the clinical diagnosis of delirium.

Postoperative delirium presents a unique and complex clinical condition. The diagnosis of delirium is based solely on the clinical assessment of patients and is often missed during routine clinical practice. This situation may potentially lead to lack of proper documentation and a false representation of the true incidence of postoperative delirium. It has previously been reported that retrospective medical record review has been inaccurate in establishing the diagnosis of delirium in medical, cardiac, and orthopedic patients.6-9 Importantly, medical record reviews are the basis for data extraction and coding for administrative hospital databases. Although the administrative electronic databases are highly specific for postoperative complications, they are known to be relatively insensitive;10,11 however, the magnitude of this discrepancy has not been clearly addressed.

The objective of this study was to determine the incidence of postoperative delirium in a cardiac surgical patient population using a targeted prospectively collected database and to compare this result with the incidence identified in the hospital administrative database. We hypothesized that administrative databases significantly underestimate the real delirium rates when compared with prospectively collected data based on validated screening tools.

Methods

Following Research Ethics Board approval, we compared delirium rates in the same patient cohort identified in two different databases: 1) prospectively collected data in a research database, and 2) retrospectively entered data in an administrative hospital database using ICD-10 coding for the diagnosis.

Assessment of delirium

In the research database, delirium was assessed every 12 hr postoperatively during the patients stay in the intensive care unit using the Confusion Assessment Method in the Intensive Care Unit (CAM-ICU). The CAM-ICU allows monitoring for symptoms of delirium in patients with ventilated lungs and extubated trachea.12 It is based on the Diagnostic Statistical Manual of Psychiatric Disorders criteria and includes a four-step algorithm assessing 1) an acute onset of changes or fluctuations in mental status, 2) inattention, 3) disorganized thinking, and 4) an altered level of consciousness. The patient is determined to be delirious (CAM positive) if he/she manifests both features 1 and 2 and either feature 3 or 4.

The nursing staff performed the CAM-ICU measurements, and nurse-educators trained all nurses in the use of the CAM-ICU method in patients with ventilated or non-ventilated lungs. This training consisted of an introductory lecture and a series of in-service training sessions. The CAM-ICU was not performed after patients were discharged from the ICU.

The research database was created to gather information about postoperative delirium in the cardiac surgical patient population of a large academic hospital. The data entered included the patients’ demographic characteristics, type of surgery, the number of CAM-ICU assessments performed, and the onset and duration of delirium. One of the study investigators collected the CAM-ICU data sheets every week and entered results into the research database.

The administrative database contained the International Classification of Diseases version 10 (ICD-10) codes for patient diagnoses. These codes were entered by professional coders and abstractors who review information in the patient chart following discharge. The information reviewed includes discharge summaries, consultation reports, clinical notes, and interventional reports. The diagnostic code for delirium was based on documentation by a physician. For each patient in the research database, the ICD-10 codes were extracted from the administrative database and checked for the presence of the code for delirium.

We evaluated the charts of the patients who were defined as having delirium in the administrative hospital database but did not have diagnosis of delirium in the research database.

Statistical analysis

The delirium rates were compared between the two groups with the Chi square test for differences in probabilities in a 2 x 2 contingency table. Confidence intervals (CIs) for proportions were calculated at 95%. A probability value < 0.05 was considered significant. Inter-observer agreement between the two delirium scoring methods was determined using Cohen’s kappa. The sensitivity and specificity of detecting delirium cases retrospectively using the administrative database were calculated using the prospective data collection as the gold standard for case finding. Statistical analysis was conducted with the use of SPSS computer software.

Results

Data from a cohort of 1,528 patients were analyzed. The demographic data, surgical characteristics, and postoperative outcomes of this cohort of patients are shown in the Table 1.

Table 1 Demographic data and surgical characteristics

Postoperative delirium was identified in 182 (11.9%) patients (95% CI, 10.3-13.5%) in the prospectively studied research cohort, while only 46 (3%) patients (95% CI, 2.2-3.8%) were coded for diagnosis of delirium in the administrative dataset (P < 0.001). The total number of CAM-ICU assessments performed is reflected in Fig. 1.

Fig. 1
figure 1

Number of Confusion Assessment Method in the Intensive Care Unit (CAM-ICU) assessments

A total of 33 (2.2%) patients with delirium were identified in both datasets. The onset and duration of delirium in these patients were similar (Fig. 2). Interestingly, there were 13 (0.85%) patients that were coded for delirium in the administrative database but not identified in the research dataset. The median onset of delirium in these patients was significantly delayed (4 [3-9] days) when compared with patients identified in both datasets (Fig. 3).

Fig. 2
figure 2

The onset and duration of delirium in the research and administrative databases. Columns 1 and 2 represent the onset of delirium; columns 3 and 4 represent the duration of delirium in both the research and administrative databases

Fig. 3
figure 3

The onset of delirium in the research and administrative databases. Shown are the onset of delirium in patients identified by both the research and administrative databases; the research database only; and the administrative database only; P = 0.007

The inter-observer agreement between the two methods was κ = 0.25. The sensitivity of the retrospective administrative database review to detect delirium was 18.1% (13.2-24.5%). The specificity of the retrospective administrative database review to detect delirium was 99% (98.4-99.5%). Positive predictive value of the administrative data was 71.7% (56.3-83.5%). Negative predictive value was 90.0% (88.3-91.4%).

In the research database, the diagnosis of delirium was made in 105 (58%) patients on the first postoperative day, in 62 (34%) patients from postoperative day two to six, and in the remaining 15 (8%) patients on the seventh postoperative day or later. The duration of delirium in these patients was one day in 111 (61%) patients, two days in 29 (16%) patients, three days in 18 (10%) patients, four days in seven (4%) patients, five days in two (1%) patients, six days in nine (5%) patients, and more than seven days in six (3%) patients.

Discussion

In this observational study we demonstrated that the incidence of postoperative delirium documented in the hospital administrative database was significantly lower when compared with the same cohort of patients in the prospectively collected research database.

The current study confirmed the previous concerns about the reliability of using administrative databases as a source for reporting delirium rates in different patient populations.6-9,13 Using an administrative database as the only source of identification of postoperative delirium could considerably underestimate the real incidence rates14 and lead to conclusions that might conflict with the studies based on validated screening tests for postoperative delirium.15,16

It is likely that the discrepancy in delirium rates between our two databases is based on the practice and methodology of establishing the diagnosis of delirium. It could be a reflection of inadequate data abstraction, poor communication among the care team, and/or incomplete charting by physicians.

Delirium is a clinical diagnosis that cannot be confirmed by laboratory results or neuro-imaging tests. It is based on the clinical evaluation of a patient aimed at identifying the essential clinical features of delirium. The clinical picture of delirium varies and includes hyperactive, hypoactive, and mixed types with symptoms that can differ significantly. Delirium has a fluctuating course; therefore, it can be missed easily during routine clinical examination. Furthermore, the routine clinical practice in the early postoperative period after cardiac surgery includes the administration of sedatives and narcotic analgesics that may mask symptoms of delirium. The fluctuating level of consciousness, the inability to maintain attention, and disorganized thinking (the very symptoms that are used to define delirium) could be interpreted as the side effects of general anesthesia, sedation, and postoperative analgesia. Consequently, the diagnosis of delirium might not be documented clearly in the clinical and consultation notes, and the diagnosis would be omitted in the discharge summaries that are used for diagnostic coding in the administrative database. Even when the diagnosis of delirium is identified in the consultation reports and clinical notes, it would commonly lack the indicators of onset or course of delirium.

Timing of the onset of delirium with respect to the postoperative course is an important factor in understanding the mechanisms of postoperative delirium. Risk factors that are associated with delirium in the immediate postoperative period are likely different from the risk factors triggering delirium days or weeks after surgery. This information is particularly important in evaluating potentially modifiable risk factors of delirium and developing preventive treatment strategies to reduce postoperative delirium. In the current study, the prospectively collected research dataset contained the delirium rates that were established using the CAM-ICU assessments performed every 12 hr after surgery. The CAM-ICU is a validated screening instrument for delirium based on the Diagnostic Statistical Manual of Psychiatric Disorders with high specificity and sensitivity and low inter-rater variability.12 In our research database, the incidence of delirium after cardiac surgery was similar to previous reports in comparable patient populations.5,17

The current study showed that the onset and duration of delirium were similar in patients who were identified by both datasets. However, a small number of patients who were identified only by the administrative database had delayed onset of delirium with a median of four days after surgery. This finding is consistent with the current clinical practice of following the postoperative course of patients undergoing cardiac surgery. The duration of ICU stay after low-risk cardiac surgery rarely exceeds a 24-hr period. Patients are transferred to the surgical floors until discharge from the hospital. The CAM-ICU assessments were limited to the postoperative period in ICU and were not administered on surgical floor. This was the main limitation of the research database in determining the variability in the postoperative course of delirium. Even though the administrative database identified only 13 (0.85%) patients developing late (> four days) delirium, it demonstrated the importance of following patients beyond the immediate postoperative period.

In conclusion, this report found that delirium rates after cardiac surgery were 11.9% in patients identified by the prospective research compared with 3% in the same cohort of patients identified by the administrative hospital databases. This finding reinforced concerns of using an administrative database as a reliable source for identifying delirium rates after cardiac surgery. Caution should be exercised in interpreting results reported from administrative databases, particularly if different treatment and management strategies are employed in an attempt to reduce postoperative delirium. Recording the results of delirium screening tests as separate entities and facilitating advancements in the electronic dataset collection would improve detection and reporting of delirium in the administrative databases. Furthermore, documenting the onset time and duration of delirium would further improve the credibility of administrative databases and make them more consistent sources of information.