Author + information
- Received May 5, 2016
- Revision received July 13, 2016
- Accepted July 20, 2016
- Published online December 1, 2016.
- Luigi Adamo, MD, PhDa,
- Yuanyuan Tang, PhDb,
- Michael E. Nassif, MDa,b,
- Eric Novak, MSa,
- Philip G. Jones, MSb,
- Shane LaRue, MD, MPHSa,
- John A. Spertus, MDb,c and
- Douglas L. Mann, MDa,∗ ()
- aCardiovascular Division, Department of Medicine, Washington University School of Medicine, St. Louis, Missouri
- bSaint Luke’s Mid America Heart Institute, Kansas City, Missouri
- cUniversity of Missouri, Kansas City, Missouri
- ↵∗Reprint requests and correspondence:
Dr. Douglas L. Mann, Division of Cardiology, Washington University School of Medicine, 660 South Euclid Avenue, Campus Box 8086, St. Louis, Missouri 63110.
Objectives This study sought to assess the performance of the HeartMate Risk Score (HMRS) in a large multicenter cohort, with a focus on its performance as a function of disease severity.
Background The HMRS has been proposed as a simple tool for risk stratification of LVAD recipients, but subsequent studies have challenged its validity.
Methods We performed a retrospective, longitudinal, comparative study using the INTERMACS (Interagency Registry for Mechanically Assisted Circulatory Support) database. The HMRS was calculated for each patient and its association with mortality was assessed using Cox models, including a pre-specified interaction by INTERMACS profile groups (1 vs. 2 vs. 3 vs 4+).
Results Among 10,847 patients with a mean age of 57.0 ± 12.9 years, 78.9 % were male; and 14.1%, 37.4%, 30.4%, and 18.2% were in INTERMACS profile groups 1, 2, 3, and ≥4, respectively. The HMRS showed moderate discrimination for both short-term (90-day, C-index 0.62) and long-term (2-years, C-index 0.60) mortality, with no significant difference between axial and centrifugal devices. Patients in the highest HMRS group had a relative risk of 90-day mortality 2.8 times greater than those in the lowest HMRS group (13.0% vs. 4.7%; p < 0.001). Importantly, the relative risks of higher HMRS scores were similar across INTERMACS profile groups, with subgroups of patients in INTERMACS profile 1 and 2 having comparable or lower mortality than some in INTERMACS profile 4+.
Conclusions The HMRS is a valid means of risk-stratifying patients across all INTERMACS profiles and may be superior to traditional INTERMACS classification. Risk stratification with the HMRS showed that patients within each INTERMACS profile groups have a wide spectrum of mortality risk and low INTERMACS profiles should therefore not be considered a contraindication to mechanical support.
Mechanical circulatory support with left ventricular assist devices (LVAD) is rapidly becoming a prevalent therapeutic option for patients with advanced heart failure with reduced ejection fraction (1,2). However, placement of an LVAD is a complicated and expensive procedure that not only provides great benefit to some patients but also is associated with significant morbidity and mortality in others (3). Pre-operative risk estimation is therefore a cornerstone for patient selection and shared decision making with patients and their caregivers (4,5).
Several studies have sought to define quantitative risk scores that could help heart failure specialists more objectively and consistently estimate the periprocedural and long-term outcomes of LVAD implantation (5,6). Among these is the HeartMate Risk Score (HMRS), a simple clinical prediction rule based on patients’ age, international normalized ratio, albumin, creatinine, and the implant center’s volume (7). The HMRS was recently derived in a large cohort of patients enrolled in the HMII trials and was shown to be superior to other existing risk scores within that cohort. Nevertheless, it’s applicability to real-world populations remains controversial. In a single-center study, the HMRS had good discriminatory ability and was able to successfully risk stratify patients in INTERMACS (Interagency Registry for Mechanically Assisted Circulatory Support) profile 1 (8). Another single-center study, however, found that the HMRS was poorly correlated with mortality after implantation (9). In addition, the applicability of the score to centrifugal devices is unknown, because all prior studies only assessed its performance in axial flow devices. Given the need to identify a valid risk score and the unclear utility of the HMRS in a real-world setting, we assessed its performance in the INTERMACS database, with special attention to its performance as a function of disease severity and device type.
We performed a retrospective, longitudinal, comparative study using the data collected in the INTERMACS database between 2006 and June 2015. The INTERMACS database is a North American registry for patients who are receiving durable mechanical circulatory support device therapy to treat advanced heart failure. Launched in June 2006 with 15 U.S. hospitals, the INTERMACS registry collects clinical data relevant to mechanical circulatory support devices starting at the index hospitalization. The registry has progressively grown to include most LVAD implanting centers in North America. We queried the INTERMACS database and included for analysis all recipients of a primary continuous-flow LVAD with pre-implant data available to calculate the HMRS. We excluded patients with continuous-flow LVAD implant after March 2015 to ensure a minimum of 90 days of follow-up, patients with biventricular VAD, and patients with continuous-flow LVAD implant to replace a previously placed VAD.
The HMRS was calculated by the INTERMACS Data Coordinating Center at the University of Alabama at Birmingham using exact age at the time of implant and volume of VAD implants in the year before that of each implant. The HMRS was subsequently categorized into low (<1.58), mid (1.58 to 2.48), and high (>2.48), as previously described, to maintain consistency with current clinical practice and previously published studies (7). Survival while on mechanical circulatory support was calculated from the date of implant to death and patients were censored as alive at the time of cardiac transplantation. Baseline characteristics were compared among patients with low-, mid-, and high-HMRS using 1-way analysis of variance for continuous variables and chi-square tests (or Fisher exact test) for categorical variables. Kaplan-Meier survival curves were developed to calculate the mortality risk at 90 days, 1 year, and 2 years for all the patients identified in our cohort and were compared using the log-rank test. The association between HMRS and mortality was also investigated using Cox proportional hazards models by including HMRS as both a categorical and as a continuous variable within each INTERMACS profile group (1 vs. 2 vs. 3 vs. 4+) and by including all profiles with an interaction term for HMRS and INTERMACS profiles to define whether the association of HMRS with mortality differed across INTERMACS profiles. To support clinical interpretability of these results, the Cox model was also used to compare similar HMRS groups from different profile groups. Discrimination of the models was summarized with the C statistic. The impact of adding the HMRS to the INTERMACS classification was evaluated by C statistic and calculating the category-free Net Reclassification Improvement (an index that quantifies how well a new model reclassifies subjects, either appropriately or inappropriately, as compared with an old model) (10). To explore the applicability of the HMRS score to centrifugal devices, we built a Cox model including the variable axial versus centrifugal device type and we studied the interaction between HMRS and this variable. All analyses were performed using SAS version 9.3 (SAS Institute, Cary, North Carolina), and a p value of <0.05 was used to indicate statistical significance.
Within the INTERMACS database we identified 10,847 patients meeting our inclusion/exclusion criteria. The cohort was composed predominantly of white males (78.9%). The average age was 57.0 ± 12.9 years and 88% received an axial device. A total of 14.1% were classified as INTERMACS profiles 1 at the time of implantation, 37.3% as INTERMACS profile 2, 30.4% as INTERMACS profile 3, and 18.2% as INTERMACS profile 4+. There was a wide distribution of patients across the 3 HMRS levels of risk, with 24.2% having a low-HMRS risk, 38.4% having an intermediate risk, and 37.5% having a high-HMRS. Mortality for the entire cohort was 9.2% at 90 days, 19.1% at 1 year, and 29.9% at 2 years (Table 1, Online Table 1). Although all INTERMACS profile groups were represented in each HMRS level, the prevalence of lower INTERMACS profiles increased with increasing HMRS category (Online Table 1). When compared with the original HMRS derivation cohort our cohort was a little younger (57.03 vs. 58.9; p ≤ 0.001) and had a slightly lower creatinine (1.40 vs. 1.47 mg/dl; p = 0.009) but it was similar in terms of sex, albumin, and international normalized ratio (Online Table 2).
Association of HMRS risk score with mortality
The HMRS, as a continuous variable, was strongly associated with 90-day mortality (hazard ratio: 1.32; 95% confidence interval [CI]: 1.26 to 1.38, for each unit increase in HMRS). Patients in the highest HMRS group had a relative risk of 90-day mortality 2.8 times greater than those in the lowest HMRS group (unadjusted rates: 12.9% vs. 4.6%; p < 0.001) and 1.6 times greater than those in the mid-HMRS group (unadjusted rates: 12.9% vs. 8.2%; p < 0.001) (Figure 1). We found similar results for 1-year (hazard ratio: 1.27; 95% CI: 1.23 to 1.32) and 2-year mortality (hazard ratio: 1.25; 95% CI: 1.21 to 1.29) (Online Figures 1 and 2).
Association of HMRS with mortality across INTERMACS profile groups
The HMRS had similar performance across INTERMACS profile groups (1 vs. 2 vs. 3 vs. ≥4; p = 0.85). The hazard ratio for 90-day mortality among INTERMACS profile 1 patients was 1.29 (95% CI: 1.17 to 1.42) for each unit increase in the HMRS, among INTERMACS profile 2 patients was 1.27 (95% CI: 1.18 to 1.36), among INTERMACS profile 3 patients was 1.30 (95% CI: 1.18 to 1.44), and among INTERMACS profile 4+ patients was 1.34 (95% CI: 1.18 to 1.50). We found similar results for 1- and 2-year mortality (Online Table 3). Importantly, after stratifying patients within each INTERMACS profile by HMRS class, we found that INTERMACS profile 1 patients with a low-HMRS had a 90-day mortality risk similar to that of INTERMACS profile 3 patients with a high-HMRS (8.4% vs. 8.8%; p = 0.82) (Figure 2) and similar to that of INTERMACS profile 4+ patients with a high-HMRS (8.4% vs. 10.9%; p = 0.27) (Figure 2). INTERMACS profile 2 patients with low-HMRS had a risk of 90-day mortality lower than that of INTERMACS profile 4+ patients with high-HMRS (5.4% vs. 10.9%; p < 0.001). The ability of HMRS to identify patients from low INTERMACS profile with mortality risk lower or similar to that of patients with higher INTERMACS profile was similar when examining 1- and 2-year mortality (Figures 3 and 4). Importantly, there was no difference in 1- or 2-year mortality across patients of different INTERMACS profiles with low-HMRS (16% for profile 1, 13.5% for profile 2, 11.2% for profile 3, 11.9% for profile 4+; p = 0.11 at 1 year) (Figure 3) (23.7% for profile 1, 21.8% for profile 2, 21.1% for profile 3, 20.7% for profile 4+; p = 0.36 at 2 years) (Figure 4) or with mid-HMRS (17.8% for profile 1, 17.9 % for profile 2, 17.1% for profile 3, 15.6% for profile 4+; p = 0.28 at 1 year) (Figure 3) (25.0% for profile 1, 29.2% for profile 2, 29.5% for profile 3, 26% for profile 4+; p = 0.36 at 2 years) (Figure 4) and at 2 years the mortality rate of both INTERMACS profile 1 and profile 2 patients with low-HMRS was lower than that of INTERMACS group 4+ patients with high-HMRS (23.7% and 21.8%, respectively vs. 34.2%; p < 0.05) (Figure 4).
HMRS risk stratifies mortality risk post-LVAD better than INTERMACS classification
When directly comparing INTERMACS classification and HMRS in terms of their ability to stratify mortality risk (Table 2), the HMRS had better discrimination for mortality than INTERMACS profile groups (HMRS C statistic = 0.62 at 90 day, 0.60 at 1 year and 2 years; INTERMACS profile C statistic = 0.58 at 90 day, 0.55 at 1 year, and 0.54 at 2 years). Addition of the HMRS to INTERMACS classification improved the C statistic of the INTERMACS classification but did not produce a C statistic superior to that of the HMRS alone. Calculation of a Net Reclassification Index showed that addition of the HMRS to INTERMACS classification led to appropriate reclassification of about 25% to 30% of events (29% of 90-day outcomes, 27% of 1-year outcomes, 25% of 2-year outcomes), mostly through appropriate reclassification of nonevents (Table 2).
HMRS has similar performance in axial and centrifugal devices
As shown in Table 3, the HMRS as a continuous variable had similar correlation with 90-day mortality for both centrifugal and axial LVADs. Hazard ratio for 90-day mortality was 1.329 (95% CI: 1.270 to 1.392) for axial devices and 1.277 (95% CI: 1.120 to 1.456) for centrifugal devices with a p value for the interaction between HMRS and device type of 0.577. We found similar results analyzing 1- and 2-year mortality (Table 3).
Pre-operative HMRS correlates with mortality post-LVAD
Risk stratification of patients with severe heart failure is critically important to better estimate the potential outcomes of LVAD therapy. Although several risk models have been developed for this purpose, traditionally this has been done by categorizing patients into different INTERMACS profiles. Although the HMRS was a promising approach to risk stratification, subsequent reports have questioned its utility. In the largest validation study to date, we explicitly examined the performance of the HMRS with short- (90-day) and long-term (1- and 2-year) survival. In a multicenter cohort of almost 11,000 patients, we found that the HMRS validly risk-stratified patient survival across all INTERMACS profile groups, and in both axial and centrifugal devices. We were also able to demonstrate that the HMRS was much better at estimating short- and long-term prognosis than INTERMACS profile groups.
This study significantly extends prior insights about the validity of the HMRS (7). Whereas Thomas et al. (9) suggested that the HMRS did not stratify risk well, our data from this study support our earlier findings that the discrimination of the HRMS was reasonable (C statistic = 0.62) and comparable with that of the original HRMS validation cohort (8). Not only were we able to confirm this across a large number of hospitals and to demonstrate its superiority in estimating prognosis, as compared with INTERMACS profiles, our data are the first to show the prognostic significance of HMRS in centrifugal devices.
Low intermacs profile should not be considered a contraindication to LVAD implant
Our study also has important clinical implications. Although the INTERMACS profile was not initially designed as a tool for risk stratification (11), INTERMACS profiles at the time of LVAD implant, were shown to correlate with mortality post-implant (12), with the most recent estimates reporting hazards ratios for post-LVAD mortality in INTERMACS profile 1 and 2 of 1.55 and 1.37 (1,13,14). In light of this, several authors have advocated a reduction of LVAD implants in the most critically ill INTERMACS profiles (4,15). INTERMACS profile 1 was described as a contraindication to mechanical circulatory support with the result that the relative number of implants in patients in INTERMACS class 1 and 2 has been progressively decreasing (4), although the HMRS scores of patients denied treatment are unknown. However, the INTERMACS classification reflects a subjective rapid assessment of the clinical trajectory of a patient. Our own experience and the data we extracted from the analysis of our own cohort of LVAD recipients (8) suggest that INTERMACS classification does not reliably discriminate between patients with preserved organ function and patients with long-standing multiorgan dysfunction, and that it can therefore overestimate the mortality risk in certain groups of patients (8). Data from this study suggest that the HMRS can more effectively risk stratify patients’ prognosis than INTERMACS profile groups. In fact, many patients with low INTERMACS profile had a post-implant mortality risk lower than some patients with higher INTERMACS profiles. Conversely, INTERMACS profile 4+ patients with high-HMRS had a similar risk of 90-day mortality than INTERMACS profile 1 patients with low-HMRS, and they had a higher risk of 90-day mortality than INTERMACS profile 2 patients with low-HMRS. When looking at the 2-year time points, the mortality of INTERMACS profile 3+ with high-HMRS was approximately 40% higher than that of INTERMACS profile 1 and INTERMACS profile 2 patients in the low-HMRS category. Collectively, these data suggest that restricting LVAD implantation in patients with low INTERMACS profiles might inadvertently prevent some patients with a good prognosis after implantation from being offered a life-sustaining treatment.
Our findings should be interpreted in the context of several potential limitations. First, our study is based on analysis of data from the INTERMACS registry and therefore has all the limitations of retrospective cohort studies. However, unlike the original carefully selected clinical trial population in which the HMRS was developed, the broad, multicenter data from almost 11,000 patients provides robust evidence of the performance of the HMRS in a real-world population. A second concern is that we were not able to include all patients and those in whom an HMRS could not be calculated may have introduced some selection biases. Third, there may be other variables that could further improve the performance of the HMRS and we did not explore opportunities to further improve the discrimination of the model. Fourth, in our analysis we made multiple comparisons but did not adjust our p value and therefore our analysis has an inherent increased risk of type 1 errors. Fifth, it should be recognized that the C statistic of the HMRS was only modest (0.6 to 0.62), and may not be sufficient for rigorous clinical application. However, we would argue that much of the outcomes are often conditional on events that could not be known at the time of a treatment decision (e.g., post-procedure strokes or bleeding events). Hayward et al. (16) have suggested that a C statistic >0.60 adds value to clinical decision-making and we have shown previously that prospective implementation of a risk model with only a slightly higher C statistic was able to improve clinical outcomes (17). Although it would be ideal to have a score with better predictive value than that of the HMRS, our data suggest that risk assessment models based on a quantitative assessment of physiologic parameters can predict mortality risk post-LVAD implant better than the subjective assessment of clinical trajectory made by treating physicians at the time of LVAD implant and that using INTERMACS profiles as a tool for pre-VAD risk stratification might be unwarranted.
We have validated the prognostic significance of the HMRS in short- and long-term survival among patients undergoing LVAD implantation, either with centrifugal or axial devices. Patient selection and patient education before implantation of mechanical circulatory devices are critical aspects in the care of patients with advanced heart failure. Quantitative risk scores, such as the HMRS, could be of great help in this effort. Using such risk models, especially when supplemented with risk estimates of quality of life outcomes, estimates of periprocedural risks and estimates of these outcomes with medical therapy alone, might improve care and shared medical decision-making. However, risk scores with discriminatory ability superior to that of the HMRS needed to realize the full potential of quantitative risk stratification of LVAD candidates.
COMPETENCY IN MEDICAL KNOWLEDGE: Patients in INTERMACS profiles 1 and 2 have been shown to have increased risk of mortality post-LVAD. INTERMACS profile 1 is therefore considered a contraindication to durable mechanical circulatory support and several authors have argued that LVADs should be placed in clinically stable patients to improve outcomes. Here we show that the HMRS identifies patients with similar mortality risk across INTERMACS classes and it stratifies mortality risk post-LVAD better than INTERMACS profile groups. These data demonstrate that low INTERMACS profile should not be considered a contraindication to LVAD implant and that objective measures of organ function, as those used to calculate the HMRS, correlate with post-LVAD outcomes better than clinical status at the time of implant.
TRANSLATIONAL OUTLOOK: LVAD therapy has revolutionized the care of patients with advanced heart failure but too many LVAD recipients still harvest only small benefits from this invasive and expensive therapy, mostly because of complications or early mortality. Many authors have looked at better patient selection as a potential solution to this problem. In this manuscript, through the analysis of a large multicenter cohort, we provide new information to help clinicians risk stratify LVAD recipients. Importantly, we show that clinical status at the time of LVAD implant is not a reliable indicator of mortality post-LVAD because it underperformed the HMRS, a score based on age, implant volume, and biochemical indices of nutritional status, hepatic function, and renal function. However, the correlation of the HMRS with mortality was modest (C statistic = 0.6 to 0.62). Further work is therefore needed to identify additional variables that determine post-LVAD outcomes and to develop more powerful tools for risk stratification of LVAD recipients.
The authors thank the INTERMACS investigators, coordinators, and participating institutions for the data they have provided for this registry.
For supplemental tables and figures, please see the online version of this article.
Data collection for this work was supported in whole or in part by the National Heart, Lung and Blood Institute, National Institutes of Health, Department of Health and Human Services under Contract No. HHSN268201100025C. The content is solely the responsibility of the author(s) and does not necessarily represent the official views of the INTERMACS or the National Institutes of Health. Dr. Spertus is the Founder and an equity holder in Health Outcomes Sciences. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- confidence interval
- HeartMate Risk Score
- Interagency Registry for Mechanically Assisted Circulatory Support
- left ventricular assist device
- Received May 5, 2016.
- Revision received July 13, 2016.
- Accepted July 20, 2016.
- American College of Cardiology Foundation
- Mancini D.,
- Colombo P.C.
- Long E.F.,
- Swain G.W.,
- Mangi A.A.
- Miller L.W.,
- Guglin M.
- Ravichandran A.K.,
- Cowger J.
- Levy W.C.
- Cowger J.,
- Sundareswaran K.,
- Rogers J.G.,
- et al.
- Adamo L.,
- Nassif M.,
- Tibrewala A.,
- et al.
- Spertus J.A.,
- Decker C.,
- Gialde E.,
- et al.