Mortality As a Measure of Treatment Effect in Clinical… : Critical Care … – LWW Journals

Critically ill patients admitted to an ICU face a substantial risk of death (1); thus, all-cause mortality is a common measure of treatment effect in ICU-based randomized clinical trials (RCTs) (24). Mortality is patient-centered, objective, and definitive integrating the net benefits and harms of life-sustaining therapies (5).

Because of ICU support, however, the risk of dying unfolds over time and is impacted by factors other than the immediate acute illness. Death sometimes results from a catastrophic eventdevastating brain injury, uncontrollable hemorrhage, or cardiac arrestand occurs early during the ICU course (6). More commonly, it is the outcome of a contingent series of complications of the presenting illness that lead to a decision to discontinue life support measures (7). Both the timing of this decision and the likelihood of making it are influenced by factors such as age, comorbidities, prior wishes, and cultural and religious mores (8).

The interpretation of a mortality signal, or its absence, in a clinical trial can also be uncertain (9). Well-conducted and adequately powered clinical trials using survival as a measure of efficacy have yielded divergent conclusions about interventions as the use of corticosteroids (10,11), the management of hyperglycemia (12,13), and the use of adjuvant therapies for sepsis (14,15), leading to pessimism about negative trials in the ICU setting (16).

Much about the performance of an all-cause mortality endpoint in clinical trials in critical illness is unknown, or based on untested assumptions, leaving important questions unanswered. When do these deaths occur, and how does this vary with the population studied? When an intervention shows benefit or harm, when is that differential effect maximal, over what time frame is it evident, and for how long does it persist? How do landmark time point estimates (e.g., mortality at day 28) differ from process-based endpoints (e.g., survival to hospital discharge)? And when a mortality signal is demonstrated in a clinical trial, is that information incorporated into clinical practice?

We sought to understand the performance properties of all-cause mortality as a measure of treatment effect in ICU-based RCTs and to address these questions through a systematic review and meta-analysis of large RCTs conducted over the past 3 decades.

We conducted a systematic review of RCTs that recruited critically ill adult patients and were published in one of five high-impact general medical journals (New England Journal of Medicine, JAMA, Lancet, Annals of Internal Medicine, and British Medical Journal) or eight subspecialty journals (American Journal of Respiratory and Critical Care Medicine, Critical Care Medicine, Intensive Care Medicine, Chest, Critical Care, Journal of Critical Care, Shock, and Journal of Trauma, Infection, and Critical Care) between 1990 and 2018; the search strategy is summarized in the Supplemental Methods (https://links.lww.com/CCM/H244).

We included trials that enrolled at least 100 patients and that reported mortality outcomes at two or more prespecified time points: ICU discharge, hospital discharge, 7 days (range, 510 d), 14 days, 28 or 30 days, 60 days, 90 days, 180 days, and greater than or equal to 1-year. Risk of bias was adjudicated using the Cochrane Collaborations published tool (17).

We extracted the following data from each study: year of publication; trial size; study population (sepsis, acute respiratory distress syndrome [ARDS], or other); risk of bias (allocation concealment, blinding of outcome assessors, number of centers, whether analysis was by intention to treat, whether the trial was stopped early for benefit, and number of randomized patients lost to follow-up); duration of ICU and hospital length of stay; and mortality outcomes (deaths in each randomized group at each prespecified time point). Mortality outcomes for each time point were recorded when reported explicitly or estimated from survival curves.

We hypothesized that by analyzing the trajectory of mortality data from multiple trials, pooled using meta-analytic techniques, we could identify a time point where a differential signal was maximal and one beyond which a differential effect was no longer evident.

To determine the temporal course of a differential mortality signal, we calculated the change in mortality over successive time intervals7, 14, 2830, 60, 90, and 180 daysfor all trials where these data were available. For each trial, the incremental absolute risk difference [RD] is the change in mortality risk between successive time points for each study group A and B, where A was the intervention with the higher mortality at the first time point that mortality was measured:

IncrementalRD=D2,AD1,ANAD1,AD2,BD1,BNBD1,B

Where D2,A and D2,B are the cumulative number of deaths at the second time point, D1,A and D1,B are the cumulative number of deaths at the first time point, and NA and NB are the total numbers of patients randomized to study groups A and B. The numerators are the number of additional deaths that occurred between the two time points in each randomized group, while the denominators reflect the number of patients alive at the earlier time point. Further details and an example are available in the Supplemental Methods (https://links.lww.com/CCM/H244).

Because we are aggregating data from studies evaluating multiple interventions in heterogeneous patient populations, we calculated the incremental RD at the trial level and pooled these estimates using a meta-analytic random effects model (18). We analyzed data using Review Manager (RevMan Version 5.2; Cochrane Collaboration, Oxford, United Kingdom). Statistical heterogeneity among trials was assessed using both the I2 statistic (19) and tau, the sd of the between-study distribution of effects derived from random effects meta-analysis. We conducted sensitivity analyses, analyzing RCTs enrolling patients with sepsis or systemic inflammatory response syndrome, RCTs enrolling patients with acute lung injury (ALI) or ARDS, and RCTs enrolling patients without either of these conditions and used the Z test to calculate interaction p values comparing RDs between these three different pairs of subgroups of RCTs. To study the potential confounding effects of transfer to long-term acute care facilities in the United States on location-based mortality measures, for comparisons involving ICU or hospital mortality, we stratified RCTs by the proportion of enrolled patients originating from the United States versus other countries, categorized as high (> 90%), intermediate (1090%), or low (< 10%) percentage of Americans.

We further evaluated trial impact by analysis of whether the conclusions resulted in regulatory approval of a novel therapy or incorporation into clinical practice guidelines, including the Surviving Sepsis Campaign guidelines (20), the European Respiratory Society guidelines on noninvasive ventilation (21), the American Heart Association/American Stroke Association recommendations for the management to cerebral infarction (22), the Brain Trauma Foundation guidelines for the management of severe traumatic brain injury (23), and the Kidney Disease Improving Global Outcomes guidelines for the management of acute renal failure (24).

Baseline data are presented as means and sds, or medians and interquartile ranges (IQRs). Differences in means were assessed using an unpaired t test, and differences in medians using the Mann-Whitney U test. We considered a (two-sided) p value of less than 0.05 as statistically significant.

There was no funding source for this study.

From 2,592 citations identified through the search strategy, we retrieved 813 articles for a detailed review. After excluding trials that reported mortality at a single time point only, trials that randomized fewer than 50 patients per group, and cluster (n = 13), pediatric (n = 16), and neonatal RCTs (n = 26), we included 346 publications in our analysis (Fig. 1). These 346 publications reported 343 separate RCTs comparing 350 trial interventions (for details see Supplemental Table 1, https://links.lww.com/CCM/H244), and recruited 228,784 patients. Both the number of trials and the duration of follow-up within each trial increased over each decade (Supplemental Fig. 1, https://links.lww.com/CCM/H244).

Preferred Reporting Items for Systematic Review and Meta-Analyses diagram of search results. From 2,592 citations identified through our research strategy, we ultimately included 343 unique trials comparing 350 interventions in our review. RCT = randomized clinical trial.

Most trials (78%) were multicenter studies; they recruited a median of 324 subjects (IQR, 188735.5; Table 1). Risk of bias was low; most trials reported allocation concealment (93%), use of an intention to treat analysis (99%), recruitment of planned sample size (94%), and a loss to follow-up of less than 5% (91%). Outcome assessment was blinded in 47% of included RCTs.

IQR = interquartile range.

Mortality was the primary outcome measure in 50% of the trials and was most commonly measured at 2830 days after randomization (268/343 [78%] of the included RCTs; Table 1). Mortality was explicitly reported at days 2830 in 214 trials (80%); it was measured from survival curves in the remaining 54 (20%) trials. Mortality could be ascertained at 7 days in 231 studies (36 by direct report and 195 estimated from survival curves), at 14 days in 228 studies (direct report in 44, from survival curves in 184 studies), at 60 days in 135 studies (57 direct report, 78 from survival curves), at 90 days in 143 studies (109 direct report, 34 from survival curves), at 180 days in 69 studies (63 by direct report and six from survival curves), and at 1 year in 24 studies (22 by direct report). Unadjusted 2830 days control group mortality declined from a median of 38% in the 1990s to 27% in the 2010s (p < 0.001). Similar declines in control group mortality occurred in trials recruiting patients with sepsis, ALI/ARDS, and other diagnoses (Supplemental Fig. 2, https://links.lww.com/CCM/H244).

We analyzed time to death among patients who died during trial follow-up. Most deaths occurred early following randomization: approximately one third of all patients who had succumbed by 180 days had died before day 7, and one-half by day 14 (Fig. 2). On average, 77% (IQR, 6786%) of in hospital deaths, and 71% of deaths occurring by 90 days, occurred prior to ICU discharge. The median number of days till death was slightly longer for patients recruited to trials of interventions for ARDS; however, the shape of the survival curves was similar for all diagnoses, with an early steep rise in mortality, followed by a more gradual rise through 90 days, and a smaller rise over the ensuing 90 days (Supplemental Fig. 3, https://links.lww.com/CCM/H244).

Temporal trajectory of mortality following randomization. Mortality rates rose rapidly over the first 14 d but continued to increase as the duration of follow-up increased. Data include only trials that report mortality both at 180 d and at each respective time point (7 d [56 trials], 14 d [54 trials], 2830 d [65 trials], 60 d [50 trials], and 90 d [57 trials]). Rates are expressed as a percentage of deaths at each time point relative to 180 d deaths, pooled for each group of trials for each pairwise estimate, and reported as medians interquartile range.

The conclusion of an RCT that an intervention is effective arises through demonstration of a statistically significant difference in rates of the primary endpoint. Among the trials included in our study, we sought to determine when following randomization, this differential effect was greatest and for how long a differential effect persists. We anticipated that relatively few trials would report significant effects at conventional levels of (p < 0.05); therefore, we decided a priori to include trials for which there was a differential mortality effect of p value of less than 0.2 at one or more time points in our primary analysis. As shown in Figure 3, for the 162 trials for which there was a differential mortality effect of p value of less than 0.2 at one or more time points, the RD between treatment arms continued to increase through day 60. Further separation between study arms was evident between days 60 and 90; however, the difference was no longer statistically significant (p = 0.07). The differences persisted for at least 6 months and likely up to 1 year, although only eight studies reported follow-up to a full year. We conducted two sensitivity analyses using only trials for which there was a differential mortality of p value of less than 0.05 at one or more time points during the trial and all trials regardless of p value. The same temporal course in incremental RD was seen in both analyses (Supplemental Fig. 4, https://links.lww.com/CCM/H244). A further sensitivity analysis by trial focussepsis, ARDS, or othershowed that the temporal course for differential treatment effect was similar across diagnoses (Supplemental Fig. 5, https://links.lww.com/CCM/H244).

Incremental risk difference between control and study arms over time. For all trials where the p value comparing study arms was at least < 0.2 on at least 1 occasion (n = 162) we calculated the pooled incremental risk difference between interventions over time. The risk difference is calculated as the percentage of patients in each study arm who were alive at the start of the interval, but had died by the end of that interval. A positive value indicates further separation of the groups, and hence a presumed treatment effect. Numbers beside each point estimate indicate the number of trials upon which the comparison was based. A statistically significant difference (p < 0.05) was seen up to 60 d; by 90 d, this difference was no longer significant. The upper panel shows the risk difference over successive time intervals, and the lower panel, the cumulative difference over time.

Both overall mortality and the differential mortality between treatment groups increased between 2830 and 90 days. For all trials reporting a mortality difference at an level of p value of less than 0.2 at one or more time points, and for which mortality data were available at both 2830 and 90 days (n = 72), median mortality in the arm with lower mortality increased from 24% (IQR, 1531%) at 2830 days to 31% (IQR, 2235%) at 90 days, while the differential mortality signal over this time period increased by 0.94% (95% CI, 0.311.58%).

The number of deaths at ICU discharge approximated the number of 2830-day deaths (95% [IQR, 86106%]), while the number of deaths at hospital discharge approximated the number of 60-day deaths (99% [IQR, 94104%]) (Fig. 4). Location-based outcomes (discharge from the ICU or hospital) are potentially confounded by differences in the organization of care, particularly by the widespread availability of long-term ventilator facilities in the United States (25). Consistent with this, the mean ICU length of stay in trials with fewer than 10% American patients was 156 days compared with 62 days in trials with more than 90% American patients (p < 0.001) and mean hospital length of stay in trials with fewer than 10% American patients was 2710 days compared with 155 days in trials with more than 90% American patients (p < 0.001).

Survival to ICU (top) and hospital (bottom) discharge. Mortality rates at ICU discharge approximated those at 28 to 30 d, whereas those at hospital discharge approximated mortality rates at 60 d. Numbers adjacent to data points indicate number of studies for which the comparison was available; data are median interquartile range.

Of the 350 comparisons in our review, 51 (15%) reported benefit for one study intervention at one or more time points at an level of p value of less than 0.05, and 15 (4%) reported an increased risk of harm (Supplemental Table 2, https://links.lww.com/CCM/H244). Only 12 of 43 (28%) unique interventions reported to reduce mortality have been incorporated into practice guidelines. A trial (26) that led to a change in the formulation of the sedative agent, propofol (https://www.accessdata.fda.gov/drugsatfda_docs/label/2017/019627s066lbl.pdf) was also considered to have impacted practice, although it was not cited in guidelines. With the exception of drotrecogin alpha activated (Xigris) that was briefly licensed for the treatment of sepsis (14), none of the seven studies of novel sepsis therapies that reported a mortality benefit (2732) resulted in the regulatory approval of a new therapy. Similarly, no novel pharmacologic agents have been introduced for the treatment of ARDS, and the benefit apparent in trials of mechanical ventilation has reflected minimization of the harms of positive pressure ventilation (3336).

We examined the performance characteristics of a mortality endpoint through a systematic review of 343 trials published over 28 years. Our work has four main findings.

First, independent of the diagnostic group studied, mortality in ICU-based trials displays a characteristic trajectory, with half of all deaths occurring in the first two weeks following randomization and three quarters by 28 to 30 days, but with additional deaths continuing to accrue over the next 6 months. On average, aggregate mortality increases from 29% on days 2830 to 37% on days 90. The time to death is slightly longer in studies of ARDS, but the trajectory is similar. Three factors may explain this convergence. First, ICUs provide support rather than treatment. ICU survival depends on success in weaning from mechanical ventilation and in avoiding complications such as acute kidney injury or ventilator-associated pneumonia that are common to all critically ill ICU patients. Second, ICU diagnoses are syndromic, and there is substantial overlap in these. A patient admitted with community-acquired pneumonia may meet criteria for a trial in sepsis or ARDS and also be a candidate for a trial of stress ulcer prophylaxis. Finally, ICU patients typically have other life-limiting comorbidities, and death is less often a result of an inability to provide support than of a conscious acceptance that prolonged support is inappropriate (37). A recent trial showed a spike in deaths at day 29 (38), suggesting that these decisions may be delayed so that they will not impact trial conclusions.

Second, using a meta-analytic derived incremental RD calculation, we found that a differential mortality risk is evident 60 or more days following randomization. The conventional landmark time point for measuring mortality in ICU-based trials has been 28 days, based on Food and Drug Administration from 3 decades ago (39). Although a differential mortality signal occurs early, a longer time horizon appears necessary to capture the full survival impact of ICU-based interventions. A higher event rate and a larger differential event rate at a more distal time point have important implications for sample size calculations. Pooling data from the 72 trials that reported an level of p value of less than 0.2 for mortality, median mortality was 24% and 29% for the lower and higher mortality arms at days 2830, and 31% and 37% at day 90. To demonstrate these differences at an level of p value of less than 0.05 and a power of 80% would require 2,444 patients for the earlier time point compared with 1,954 patients for the later.

Third, we found that for trials conducted outside the United States, 2830day mortality approximates ICU mortality, while 60-day mortality approximates hospital mortality. Follow-up at a landmark time point may require in-person or telephone contact with study subjects, while ICU or hospital discharge can be determined from the hospital record, an important consideration in an era of large, multinational, but modestly funded investigator-initiated trials (40,41).

Finally, although mortality was the primary outcome in the majority of trials in our review, a mortality benefit was inconsistently associated with adoption of the intervention into clinical practice. Conclusions were incorporated into clinical guidelines for only 13 of 43 studies (30%) showing a mortality reduction. None of these are novel pharmacologic therapies.

Our study has limitations that may impact its interpretation. Even though the overall number of included RCTs was large, some of the binary comparisons were based on very small numbers. In particular, very few U.S.-based RCTs reported ICU or hospital mortality, and very few of the RCTs reported mortality rates beyond 90 days, resulting in low statistical power to detect further separation of survival curves at later time points. We only included trials for which mortality was available at two or more time points, so the sample was not necessarily representative of all large ICU-based trials. Further, in pooling mortality trajectories across a diverse group of interventions and study contexts, we may have obscured subtle differences in course based on where the study was conducted or what was compared.

In summary, incremental differences in mortality rates between intervention and control groups occur until at least day 60. The number of hospital discharge deaths is similar to deaths at 60 days, suggesting that hospital discharge mortality can be used as an alternate to day 60 mortality. Finally, while mortality is commonly used as the primary endpoint for ICU trials when an effect was seen, the interventions were inconsistently adopted into clinical practice, suggesting that factors other than evidence of survival benefit impact clinician uptake of trial results (3).

We acknowledge with gratitude the efforts of Wilson Kwong and Kamya Kommaraju who identified and abstracted some of the articles included in this article.

Excerpt from:
Mortality As a Measure of Treatment Effect in Clinical... : Critical Care ... - LWW Journals