Author + information
- Received September 14, 2015
- Revision received November 5, 2015
- Accepted November 13, 2015
- Published online April 1, 2016.
- Lily E. Johnston, MD, MPHa,b,
- Joshua C. Grimm, MDc,
- J. Trent Magruder, MDc and
- Ashish S. Shah, MDd,∗ ()
- aBloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland
- bDivision of Thoracic and Cardiovascular Surgery, University of Virginia, Charlottesville, Virginia
- cDivision of Cardiac Surgery, The Johns Hopkins Hospital, Baltimore, Maryland
- dDepartment of Cardiac Surgery, Vanderbilt University Medical Center, Nashville, Tennessee
- ↵∗Reprint requests and correspondence:
Dr. Ashish S. Shah, Department of Cardiac Surgery, Vanderbilt Medical Center East, 1215 21st Avenue South, Suite 5025, Nashville, Tennessee 37232.
Objectives The aim of this study was to develop a risk index specific to patients on mechanical circulatory support that accurately predicts 1-year mortality after orthotopic heart transplantation using the United Network for Organ Sharing database.
Background Few clinical tools are available to aid in the decision between continuing long-term device support and performing transplantation in patients bridging with mechanical circulatory support.
Methods Using a prospectively collected, open cohort, 6,036 patients receiving mechanical circulatory support who underwent orthotopic heart transplantation between 2000 and 2013 were evaluated and randomly separated into derivation (80%) and validation (20%) groups. Multivariate logistic regression models were constructed using variables that improved the explanatory power of the model, which was determined using multiple methods. Points for a simple additive risk index were apportioned on the basis of relative effect on odds of 1-year mortality.
Results A 75-point scoring system was created from 9 recipient and 4 donor variables. The average score in the validation cohort was 14.4 ± 7.7, and scores ranged from 0 to 57; these values were similar to those in the derivation cohort. Each 1-point increase predicted an 8.3% increase in the odds of 1-year mortality (odds ratio: 1.08; 95% confidence interval: 1.06 to 1.11). Low (0 to 10), intermediate (11 to 20), and high (>20) risk score cohorts were created, with predicted average 1-year mortalities of 8.6%, 12.8%, and 31%, respectively, in the validation cohort.
Conclusions The investigators present a novel, internally cross-validated risk index that accurately predicts mortality in bridge-to-transplantation patients.
Although bridge to transplantation (BTT) with mechanical circulatory support (MCS) has become increasingly prevalent (1–4), several studies evaluating the role of destination therapy have demonstrated acceptable mid- and long-term outcomes (5–8). Therefore, the decision to proceed with orthotopic heart transplantation (OHT) after a period of mechanical support is not straightforward. Furthermore, current models do not sufficiently predict the risk for mortality in these bridged cohorts (9).
The objective of this study was to use United Network for Organ Sharing (UNOS) data to create and validate a risk index for MCS patients on the basis of both recipient and donor factors that would accurately predict 1-year mortality after OHT. We aimed to generate a score that accurately described the data, predicted 1-year mortality with reasonable accuracy, and was simple enough to use at the bedside. The goal was to aid practitioners in assessing the risks of transplanting a given recipient with a specific donor organ in order to inform prognosis, raise suspicion for adverse postoperative events in high-risk patients, and potentially tailor decisions regarding organ allocation.
Standard Transplant Analysis and Research files were provided by UNOS and based on Organ Procurement and Transplantation Network data current through December 31, 2013. These files contain de-identified patient-level records that include information on recipients, donors, and candidates on the waiting list. The UNOS registry captures all transplantations in the United States since October 1, 1987. No patient or center identifiers were included, and the study was exempted from human subjects research review by the Institutional Review Board at the Johns Hopkins Bloomberg School of Public Health.
Study design and definitions
For studies involving transplantation patients, we examined all primary OHT patients ages 18 years and older who underwent transplantation after 2000. Patients with missing follow-up information, those with missing MCS information, and those receiving simultaneous transplantation of other organs were excluded (Figure 1). The sample was then randomly divided into an 80% model derivation cohort and a 20% validation cohort.
MCS was defined as having an intracorporeal device at the time of transplantation and was stratified by anatomic position (left ventricular assist device [LVAD], right ventricular assist device, biventricular assist device, total artificial heart, or unspecified). Survival time began at the date of transplantation. The primary outcome was 1-year mortality; secondary endpoints included 90-day and 3-year mortality as well as overall survival time. Standard UNOS variables and definitions were used. Prior dialysis included any dialysis requirement at or since listing. Recent infection was any infection treated within the 2 weeks before transplantation.
The dataset contained 680 variables, including both donor and recipient data. All plausible variables for predicting 1-year mortality were evaluated using univariate logistic regression in the derivation sample. The multivariate regression model was built in a hierarchical fashion with variable inclusion on the basis of 3 criteria: 1) variables were included if they had previously been found to be significant in relevant published research (10,11); 2) variables were included if they were hypothesized to be clinically relevant; and 3) variables that demonstrated modest (p < 0.20) relationships with 1-year mortality in univariate analysis were included. Models were tested using the continuous forms of all variables; once selected, continuous variables were dichotomized or binned to maximize the simplicity, parsimony, and clinical utility of the model. Cut points for bins were determined using fitted splines, and multiple cut points were tested for each variable to maximize its performance in the model. Model diagnostics, particularly the Akaike information criterion, the Hosmer-Lemeshow goodness-of-fit test, and area under the receiver-operating characteristic curve were used to determine the explanatory power of a given variable and therefore its inclusion or exclusion in the next iteration of the model. Mixed manual forward and backward selection methods on the basis of model Akaike information criterion were used to select only those variables that contributed substantially to model performance; random stepwise algorithms were not used given their poor sensitivity (12). Variables with more than 15% missing data were excluded from the multivariate model because of case-wise deletion of records with missing data in model construction. Multicollinearity and interactions were examined in detail. A model using imputed data from a multiple imputation procedure was fit and compared with the original model to assess the importance of missing data. Once the final model was complete, points were assigned to each variable to generate the transplantation risk index in patients with MCS (TRIP-MCS) score. Several point systems were tested, but the allocation that best correlated with the model was an additive scale in which points were assigned in proportion to the odds ratios of the variables in the final model. The performance of the scoring system was evaluated by comparing observed and expected 1-year mortality as a function of score. The model was run using the secondary endpoints of 90-day and total 3-year mortality to delineate the timing and longevity of the incurred risk. Overall survival was examined using Kaplan-Meier survival curves stratified by risk grouping, and differences in risk groupings were tested using the log-rank test for survival equality.
A total of 6,036 patients constituted the overall sample. The mean age was 51.2 ± 12.7 years, and 18.3% of the cohort were women (n = 1,107). There were 1,556 total deaths in the study period, representing an incidence of 7.33 deaths per 100 person-years. The Kaplan-Meier survivor function at 1 year was 85.6% (95% confidence interval [CI]: 84.6% to 86.4%). Random allocation to derivation and test cohorts yielded an 80% sample of 4,829 patients and a 20% sample of 1,207 patients. Key variables were similar between the derivation and test cohorts (Table 1).
Variables selected from the derivation cohort
Of the approximately 50 variables that were screened in univariate logistic regression, 20 were at least modestly associated with 1-year mortality and were candidates for inclusion in the final model. All variables improved the Akaike information criterion of the model, and all demonstrated significant likelihood ratio test results (p < 0.05). The C statistic of the final model was 0.69. Goodness of fit was assessed using the Hosmer-Lemeshow test with 10 groups, which resulted in a chi-square value of 14.29 (p = 0.074), suggesting an appropriate fit.
Recipient variables that strongly predicted 1-year mortality included age, body mass index (BMI), intensive care unit admission, need for preoperative mechanical ventilation, renal and hepatic function, recent infection, and type of support device. Significant donor variables included donor age, sex mismatch with the recipient, ischemic time, and donor glomerular filtration rate (GFR) (Table 2). Complete data were available to calculate scores for 90% of patients in the derivation cohort (n = 4,358), and TRIP-MCS scores ranged from 0 to 60, with a mean of 14.5 ± 8.0. Each point increased the 1-year odds of death by 8.5% (odds ratio [OR]: 1.08; 95% CI: 1.07 to 1.10) in this cohort.
In the validation cohort (n = 1,207), complete data were available for 90% (n = 1,085) of patients, and TRIP-MCS scores centered on a mean of 14.4 ± 7.7 and ranged from 0 to 57 (Figure 2); these values were similar to those in the derivation cohort. The score demonstrated good predictive accuracy when examined as a continuous variable, with each 1-point increase predicting an 8.3% increase in the odds of 1-year mortality (OR: 1.08; 95% CI: 1.06 to 1.11), and a C statistic of 0.66. In both the derivation and validation cohorts, predicted mortality conformed to observed mortality up to a score of approximately 40, at which small sample size and outliers skewed the relationship between predicted and observed 1-year mortality (Figure 3). We also explored stratification of patients into high-, medium-, and low-risk groups on the basis of clinically meaningful differences in expected mortality. Patients were separated into 3 groups: score ranges of 0 to 10, 11 to 20, and >20. Within the derivation cohort, the low-, intermediate-, and high-risk groups had predicted 1-year mortality rates of 7.6 ± 2.7% (n = 1,438), 14.6 ± 4.1% (n = 1,984), and 31.0 ± 12.7% (n = 936) respectively (Bonferroni-adjusted p < 0.001 for all comparisons). These predictions were similar to those obtained within the validation cohort of 8.6 ± 3.5% (n = 360), 12.8 ± 4.2% (n = 517), and 31.0 ± 17.9% (n = 208) for low-, intermediate-, and high-risk groups, respectively (Bonferroni-adjusted p < 0.001 for all comparisons) (Figure 4).
To ensure that the model was valid for latest generation ventricular assist device technology, most specifically the HeartMate II (Thoratec, Pleasanton, California), we conducted a subset analysis examining only patients who underwent transplantation after 2009. This time-frame proxy has been used by other investigators for the same purpose, and its validity has been corroborated by another group (4,13). In this scenario, the score remained a robust predictor of 1-year mortality in both derivation (OR: 1.08; 95% CI: 1.06 to 1.09) and test (OR: 1.09; 95% CI: 1.05 to 1.13). To establish whether the score was more broadly generalizable, we tested its classification ability against both 90-day and 3-year mortality as well. Looking at 90-day mortality, each point was associated with a 9% increase in the odds of 90-day mortality (OR: 1.09; 95% CI: 1.08 to 1.10) in the derivation cohort and 9% in the test cohort as well (OR: 1.09; 95% CI: 1.06 to 1.11). The C statistics for these models were 0.71 and 0.68, respectively. For 3-year mortality, we also found a significant relationship between risk score and mortality: in the derivation group, each point corresponded to a 7% increase in the odds of 3-year mortality (OR: 1.07; 95% CI: 1.06 to 1.08). In the test group, the increase in odds per point was 6% (OR: 1.06; 95% CI: 1.04 to 1.08). Taken together, the results over these 3 time frames suggests that the score is predictive of early risk for mortality that affects overall survival at both 1 and 3 years post-transplantation.
To better understand the balance between risk incurred from donor and recipient variables, we broke the scores down by risk source and examined what percentage of patients could move to a lower risk group given the selection of an alternative organ. In the moderate-risk group, 36% (95% CI: 34% to 38%) of patients in the derivation cohort and 36% (95% CI: 32% to 40%) of those in the validation cohort could move to the low-risk group with a lower risk organ. In the high-risk group, 49% (95% CI: 46% to 52%) and 46% (95% CI: 39% to 53%) of patients in the derivation and validation cohorts, respectively, could move down at least to the moderate group with a different organ, and 1.5% (95% CI: 0.82% to 2.5%) and 1.4% (95% CI: 0.30% to 4.2%), respectively, could potentially move from high-risk to low-risk groups with alternative organ selection.
Given the robust performance of the TRIP-MCS score at multiple time points, we explored the overall survival function as stratified by the risk score groups described earlier. Using Kaplan-Meier analysis, we demonstrated accuracy of the model in the derivation cohort and predictive accuracy in the test cohort with respect to survival over time (Figure 5). Specifically, the high-risk group in the derivation cohort had 21% lower cumulative survival at 1 year compared with the low-risk group (72.5% vs. 93.5%, pairwise log-rank p < 0.001). Findings were similar in the test group, with the high-risk group having 19.6% lower cumulative survival at 1 year than the low-risk group (73.3% vs. 92.9%, pairwise log-rank p < 0.001).
In the first study to model 1-year mortality risk in patients bridging with MCS to OHT, we found that the TRIP-MCS, a simple additive score, accurately predicted risk for 1-year mortality after transplantation. Many recipient variables that strongly predicted 1-year mortality are consistent with those identified in non-MCS-specific models, such as the Quantitative Recipient Risk Index for Mortality Prediction After Cardiac Transplantation (IMPACT) score (11), including renal function and the need for preoperative ventilatory support. Recipient age, recent infection, and total bilirubin are also predictive in both the IMPACT and TRIP-MCS scores. Recipient variables unique to TRIP-MCS include BMI, intensive care unit admission, and type of circulatory support required. Our score is also the first to incorporate both donor variables—including age, sex mismatch with the recipient, ischemic time, and donor GFR—as well as recipient variables. In particular, recipient GFR <50 mg/ml/1.73 m2, BMI of 35 kg/m2 or greater, and mechanical ventilation at time of transplantation are especially poor prognostic factors.
Increases in MCS utilization and technological improvements have paradoxically heightened the complexity of clinical decision making for BTT patients. For patients with minimal complications related to their ventricular assist devices and reasonable quality of life, the question of whether to accept a given donor offer may not be straightforward. Clinicians must be able to weigh the risks of transplantation against the inherent risks associated with ongoing MCS. This score may serve to add structure to shared decision-making conversations between patients and providers by quantitating the short- and midterm risks of transplantation. From society’s perspective, organs remain a scarce resource, and it may be reasonable to question the utility of performing transplantation in a stable, or an especially high-risk, MCS patient. Understanding the risk for mortality after transplantation in this population is an important piece of this puzzle, but the TRIP-MCS score is intended to facilitate such discussions rather than provide threshold values for individual or policy decisions.
This study is the first to explicitly include patients with devices other than LVADs and total artificial hearts in the analysis. Although LVAD patients certainly represent the majority of patients receiving MCS, our study supports the general clinical intuition that patients with other devices are at increased risk for mortality after transplantation. Extrapolation of findings and outcomes from LVAD patients may underestimate the risks to patients with non-LVAD support. Although several groups have convincingly argued that at least some LVAD patients are too stable to merit the additional status 1A time they accrue, this may not hold for patients with other devices (2,13–15).
Our model is largely consistent with the findings of other groups investigating risks of OHT broadly and the use of MCS more specifically. As with the IMPACT score, we found that patient support leading up to transplantation was an important predictor of 1-year mortality, as was evidence of noncardiac end-organ compromise, including renal and hepatic dysfunction (11). Looking at the effect of MCS on graft survival, Maltais et al. (16) found that higher pulmonary vascular resistance, longer ischemic time, increased donor age, donor-to-recipient sex mismatch, and lower donor-to-recipient BMI ratio were all associated with increased likelihood of graft failure after controlling for device type. They also found that patients bridged with total artificial hearts were at higher risk for graft failure than those bridged with LVADs.
The distinction among low-, middle-, and high-risk groups as identified by our risk score helps put a few model variables with high point values into context. Three of the recipient variables are assigned 10 points each because of their large effect sizes. Having any of these risk factors alone, including BMI >35 kg/m2, requirement for mechanical ventilation at the time of transplantation, and GFR <50 mg/ml/1.73 m2, will promote all but the lowest risk patients from low- to medium-risk groups or from medium- to high-risk groups, and each subsequent risk group represents a near doubling of predicted mortality at 1 year.
Last, we note variables that failed to predict mortality risk in our model.
We originally hypothesized that patients with ventricular assist devices were more likely to be sensitized because of previous transfusions associated with device implantation (17) and that this would increase 1-year mortality, but neither transfusions nor percentage reactive antigens added significant explanatory power in our model. We also found that in the setting of MCS and accounting for device type, the etiology of the patient’s heart failure did not improve the model.
MCS represents relatively new technology, and we have limited follow-up data for these patients; this is especially true for those patients using the HeartMate II and other current-generation devices. Our model robustly predicts 1-year mortality in patients who underwent transplantation after 2009, but the number of patients is reduced and follow-up time is limited, making time-to-event analysis less meaningful for this cohort of patients. The rapid evolution of MCS technology is also a limitation: although the TRIP-MCS score works well for current devices, it is unclear whether it will continue to do so as technology evolves. However, technological evolution is a limitation shared by many areas of surgical research, and this risk model fills an important gap in our ability to make informed clinical decisions about these patients at present.
Second, model construction and validation are always subject to scrutiny. Our modeling decisions were based on 3 goals: maintaining accuracy, both in describing the derivation cohort as well as in forecasting 1-year mortality in the validation cohort and in developing a simple, clinician-friendly bedside tool. Using more variables and leaving all continuous variables as such rather than dichotomizing them would have generated a model with somewhat greater statistical power, but at the cost of clinical facility and utility. We were unable to validate this model with an independent sample from outside the United States, but we used multiple strategies to ensure appropriate model selection and minimization of bias. These methods included cross-validation with a random subset of the UNOS dataset, multiple imputation to check for bias resulting from missing data, and checking variable selection using multiple selection methods. Additionally, this model by itself does not take into account the risks incurred by remaining on the waiting list. However, using this model to help evaluate the counterfactual decision of remaining on MCS instead of undergoing transplantation is a topic of ongoing investigation by our group.
On the basis of a cohort of more than 6,000 patients who underwent transplantation while receiving MCS, we have derived an accurate and predictive risk model of 1-year mortality that incorporates both donor and recipient variables. This model in turn was translated into a simple scoring system that accurately predicted 1-year mortality in both our derivation and our validation samples, and this risk affects survival up to 3 years from the time of transplantation. This index may help inform prognosis, organ allocation, and the complex decisions regarding BTT versus bridge-to-destination therapy.
COMPETENCY IN MEDICAL KNOWLEDGE: A combination of donor and recipient variables can be used in a simple scoring system to estimate the risk for 1-year mortality after transplantation in patients bridged with MCS. Recipient BMI, renal function, and need for mechanical ventilation are the strongest predictors of risk in this model.
TRANSLATIONAL OUTLOOK: Future studies should compare the risk of transplantation relative to the risks of remaining on long-term MCS. Additionally, the risks for patients with non-LVAD devices need to be more clearly defined. A clearer understanding of the role of cardiac transplantation in the setting of adequate MCS will inform shared clinical decision making as well as policy regarding transplantation priority status and organ allocation.
The data reported here were supplied by the United Network for Organ Sharing as the contractor for the Organ Procurement and Transplantation Network. The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy of or interpretation by the Organ Procurement and Transplantation Network or the U.S. government. Dr. Grimm is the Sharp Fellow in cardiac surgical research at Johns Hopkins. Dr. Magruder is the Piccinini Fellow in cardiac surgical research at Johns Hopkins. This work was supported in part by grant UM1 HL088925, Network for Cardiothoracic Surgical Investigations in Cardiovascular Medicine. The authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- body mass index
- bridge to transplantation
- confidence interval
- glomerular filtration rate
- Quantitative Recipient Risk Index for Mortality Prediction After Cardiac Transplantation
- left ventricular assist device
- mechanical circulatory support
- orthotopic heart transplantation
- transplantation risk index in patients with mechanical circulatory support
- United Network for Organ Sharing
- Received September 14, 2015.
- Revision received November 5, 2015.
- Accepted November 13, 2015.
- American College of Cardiology Foundation
- Schulze P.C.,
- Kitada S.,
- Clerkin K.,
- Jin Z.,
- Mancini D.M.
- Wever-Pinzon O.,
- Drakos S.G.,
- Kfoury A.G.,
- et al.
- Jorde U.P.,
- Kushwaha S.S.,
- Tatooles A.J.,
- et al.
- Dardas T.,
- Mokadam N.A.,
- Pagani F.,
- Aaronson K.,
- Levy W.C.