Author + information
- Received December 4, 2017
- Revision received February 2, 2018
- Accepted March 28, 2018
- Published online August 27, 2018.
- Manreet K. Kanwar, MDa,∗ (, )
- Lisa C. Lohmueller, PhDb,
- Robert L. Kormos, MDc,
- Jeffrey J. Teuteberg, MDd,
- Joseph G. Rogers, MDe,
- JoAnn Lindenfeld, MDf,
- Stephen H. Bailey, MDa,
- Colleen K. McIlvennan, ANPg,
- Raymond Benza, MDa,
- Srinivas Murali, MDa and
- James Antaki, PhDb
- aCardiovascular Institute, Allegheny Health Network, Pittsburgh, Pennsylvania
- bDepartment of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania
- cHeart and Vascular Institute, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania
- dDepartment of Cardiovascular Medicine, Stanford University Medical Center, Stanford, California
- eDivision of Cardiology, Duke University School of Medicine, Durham, North Carolina
- fCardiovascular Medicine, Vanderbilt University Medical Center, Nashville, Tennessee
- gCardiovascular Institute, University of Colorado, Aurora, Colorado
- ↵∗Address for correspondence:
Dr. Manreet Kanwar, Allegheny Health Network, 320 East North Avenue, 16th Floor, South Tower, Pittsburgh, Pennsylvania 15212.
Objectives This study investigates the use of a Bayesian statistical models to predict survival at various time points in patients undergoing left ventricular assist device (LVAD) implantation.
Background LVADs are being increasingly used in patients with end-stage heart failure. Appropriate patient selection continues to be key in optimizing post-LVAD outcomes.
Methods Data used for this study were derived from 10,277 adult patients from the INTERMACS (Inter-Agency Registry for Mechanically Assisted Circulatory Support) who had a primary LVAD implanted between January 2012 and December 2015. Risk for mortality was calculated retrospectively for various time points (1, 3, and 12 months) after LVAD implantation, using multiple pre-implantation variables. For each of these endpoints, a separate tree-augmented naïve Bayes model was constructed using the most predictive variables.
Results A set of 29, 26, and 31 pre-LVAD variables were found to be predictive at 1, 3, and 12 months, respectively. Predictors of 1-month mortality included low Inter-Agency Registry for Mechanically Assisted Circulatory Support profile, number of acute events in the 48 h before surgery, temporary mechanical circulatory support, and renal and hepatic dysfunction. Variables predicting 12-month mortality included advanced age, frailty, device strategy, and chronic renal disease. The accuracy of all Bayesian models was between 76% and 87%, with an area under the receiver operative characteristics curve of between 0.70 and 0.71.
Conclusions A Bayesian prognostic model for predicting survival based on the comprehensive INTERMACS registry provided highly accurate predictions of mortality based on pre-operative variables. These models may facilitate clinical decision-making while screening candidates for LVAD therapy.
Left ventricular assist devices (LVADs) are being increasingly used to support patients with advanced heart failure (HF) and are known to improve survival in this critically ill population (1). LVADs are now considered standard of care for end-stage HF patients, who are becoming too ill to await a donor organ (bridge to transplantation), for those ineligible for transplant (destination therapy), or for those patients whose eventual transplant candidacy is uncertain (2,3). Although outcomes have continued to improve, proper patient selection remains key to successful outcomes (1,4,5).
Although various risk stratification models have been proposed to predict survival/mortality post-LVAD, they all have limited applications in “real-life” decision-making due to their derivation from a limited number of variables in small datasets, or analyzing a specific pump (6–9). It is recognized that accurate predictions of outcomes after LVAD implantation depend on a complex and dynamic interplay of multiple pre-operative variables that may be inadequately captured by traditional multivariate statistical modeling. Bayesian network (BN) algorithms have been proposed to predict mortality, gastrointestinal bleeding, and right ventricular failure in populations with a LVAD (10,11). These BN algorithms can account for dynamic, nonlinear interactions between clinical and nonclinical variables and their interdependency in influencing outcomes. In this way, they mimic complex human decision-making, while drawing their diagnostic algorithms from thousands of patients. Moreover, these models can predict outcomes at different time points after LVAD implantation by recognizing the time-varying importance of relevant variables. We, therefore, sought to develop a Bayesian-based prognostic model of mortality at multiple time points after implantation of a continuous-flow LVAD, using data from the INTERMACS (Inter-Agency Registry for Mechanically Assisted Circulatory Support).
This study was approved by the INTERMACS Data, Access, Analysis, and Publication Committee. The Data Coordinating Center at the University of Alabama at Birmingham provided de-identified patient data for LVAD implantations undertaken between April 2006 and December 2016 (n = 20,216). Modeling was performed using pre-implant patient information from January 2012 to December 2015, for adults (>18 years of age) who received an initial primary continuous-flow LVAD or LVAD and right ventricular assist device (RVAD) in combination (n = 10,277). We chose this time frame to include current-generation, continuous-flow LVADs with the least amount of missing data and derived from more than 160 U.S. hospitals. Total artificial heart recipients, pulsatile flow LVAD, and RVAD-only receipts were excluded from this study. Patients who received device exchanges (n = 800) were included in the study, with total time on pump calculated across the multiple implants. Patients who were recovered or underwent heart transplantation were included and indicated as alive in modeling outcomes up until that time point and censored for subsequent time points. All variables (e.g., clinical events and interventions during hospitalization) used to create the models were replicated from INTERMACS users guide and limited to pre-operative interventions only. Mortality after LVAD implantation was modeled at 1 month, 3 months, and 12 months.
The INTERMACS dataset includes more than 400 pre-implant variables, with of varying levels of data completion. Bayesian model construction requires no missing data in the training set. To prepare the data for modeling, we categorized the missing data into 2 sets: those missing in specific patterns (missing, not at random) and those that were truly unknown (missing at random). An example of data missing, not at random, was if a patient did not complete a quality-of-life questionnaire because he or she was too sick, then the answers for all the questionnaire response variables were filled in as not applicable. A missing at random example was if a patient did not perform a 6-min walk test and no reason why was documented, then the result of that test (distance walked) was classified as missing rather than left blank. Variables with more than 40% data missing were excluded from the analysis (n = 42). Additionally, variables with <1% positive responses (e.g., previous Dor procedure done in only 8 patients) were removed from the analysis (n = 16). Some variables in INTERMACS capture information across a series of binary (yes/no) answers. To reduce the fields and improve the predictive power of variables, some fields were collapsed into multilevel variables. For example, INTERMACS has 2 variables for every comorbidity: contraindication limiting transplant (yes/no) and contraindication but not limiting transplant (yes/no). These were collapsed into contraindication yes, yes-limiting transplant, or no. In this way, the number of variables for modeling was reduced. Fields for past medical interventions were combined into total counts of events, while keeping the individual binary information. For example, a patient with a coronary artery bypass grafting and dialysis during hospitalization was captured as coronary artery bypass grafting = yes, dialysis = yes, and total event count = 2. Variables with many levels were broken into subsets to identify important features. For instance, primary diagnosis (a 31-level variable) was divided in to ischemic etiology, restrictive myopathy, dilated myopathy, and congenital disease. After variable pre-processing, 203 pre-implant variables were used in the model construction.
The BN classifiers were derived for each time point of interest using a training dataset consisting of 80% of the data records selected at random (using Weka test/train split function). The remaining 20% of data were held aside as the test set for the final model validation. This resulted in 6 sets of data across 3 time points: 3 training sets (1 for each model), and 3 test sets. The 3 training sets were each used for discretization and feature selection independently.
Discretization of continuous variables
The Bayesian modeling approach used in this study (Bayesian multinomial network modeling) requires that all variables be categorical, therefore continuous variables must be discretized (12). To this end, 4 different methods of discretization were explored: expert binning (cutpoints determined for ventricular assist device implant guidelines, established risk tools, and normal ranges), supervised binning (MDL method in Weka), equal frequency binning, and equal width binning. Using training data, information gain was measured for each variable using each method, and results were compared. Choosing the method that yielded the greatest information gain for each variable, a hybrid approach of expert binning, equal frequency, and equal width binning was used in to discretize the variables. This was performed for all 3 models.
To select variables for inclusion in the model, information gain was run in a 10-fold cross-validation on the training data, with the recurring top variables being selected for model inclusion. The cutoff for selection was information gain was >0.003 for all 3 model time points. This resulted in a set of 29, 26, and 31 variables for the 1-, 3-, and 12-month models, respectively.
The BNs process individual patient data in a dynamic and nonlinear fashion to predict probable outcomes. The selected features from the training sets were used to learn both tree-augmented naïve Bayes (NB) and NB graphical models with GeNIe software (BayesFusion, Pittsburgh, Pennsylvania). Each model was optimized by running 10-fold cross-validation and removing or adding variables that either had low diagnostic value (as calculated in GeNie) or were on the cusp of the information gain cutoff. At all 3 time points, the NB models had superior performance, as measured by the area under the receiver operator characteristics (ROC) curve. The final NB models had 28, 26, and 21 predictive variables for the 1-, 3-, and 12-month outcomes, respectively. Variables were grouped into 3 categories: demographics/patient status, medical history, and test results (laboratory, exercise, and imaging). These models were collectively entitled as Cardiac Outcomes Risk Assessment (CORA) models.
Models were validated using the 3 test datasets, which had not been used in the prior model learning. ROCs were plotted in R (R Project, Vienna, Austria). In addition, we also report accuracy, sensitivity, and specificity (assuming a 50% threshold) of the Bayesian model’s performance.
A total of 10,277 patients met the inclusion criteria (Figure 1). The majority were between 50 and 69 years of age (n = 6,174; 60%); 8,044 (78%) were male; 3,811 patients (35%) received the LVAD as destination therapy, and 5,528 patients (54%) received the device as bridge to transplantation. Ischemic disease was listed as the cause for cardiomyopathy in 4,637 patients (41.5%). At the time of implantation, 1,671 (16%) were categorized as INTERMACS profile 1, 3,458 (35%) as INTERMACS profile 2, and 3,318 (32%) as INTERMACS profile 3. In the training sets (n = 8,222), the 1-month mortality was 5% (n = 426), the 3-month mortality was 9% (n = 776), and the 1-year mortality was 18% (n = 1,459) after LVAD implantation. In the test sets (n = 2,055), the 1-month mortality was 6% (n = 114), the 3-month mortality was 10% (n = 200), and the 1-year mortality was 19% (n = 390) after LVAD implantation.
Models and test validation
Bayesian models for 1, 3, and 12 months after LVAD implantation are illustrated in Figures 2, 3, and 4⇓⇓⇓. Variables are color-coded according to 3 categories: demographics/patient status, medical history, and results. The ROCs, accuracy, sensitivity, specificity, and area under the curve ROC are summarized in Figure 5 and Table 1. Accuracy ranged between 76% and 87%, and area under the ROC curve ranged between 70% and 71%.
Mortality at 1 month after LVAD implantation
This NB model contains 29 variables directly connected to the mortality outcome (Figure 2). Although the order of influence changes as variables are observed or specified (i.e., while calculating the risk for a specific patient), the variables most predictive of early post-LVAD mortality are concomitant RVAD implant, total number of events (as described in INTERMACS) during the implant hospitalization, low platelet count, high bilirubin levels, high aspartate aminotransferase level, and lower INTERMACS profile.
Mortality at 3 months after LVAD implantation
The NB model for mortality at 3 months after LVAD implantation had 26 variables, with concomitant RVAD implant, older age, elevated blood urea nitrogen, low hemoglobin, and lower INTERMACS profiles being highly predictive of higher mortality risk (Figure 3).
Mortality at 12 months after LVAD implantation
The NB model for mortality at 12 months after LVAD implantation had 21 variables, with older age, elevated blood urea nitrogen, low hemoglobin, device strategy (destination therapy), and concomitant RVAD implant being highly predictive of mortality (Figure 4).
Unique variables across time points
There were several variables that influenced the risk of mortality across all time points, such as old age, dialysis during index hospitalization, previous cardiac operations, albumin, platelet count, and blood urea nitrogen. Similarly, there were several highly predictive variables that predicted short-term mortality that were distinct from those predicting the 1-year risk of death. These included lower INTERMACS profile, pre-operative ventilator dependence, and hepatic and renal function affecting 1-month mortality, whereas ischemic etiology, history of chronic renal disease, and frailty contributed more to the 1-year mortality.
Appropriate patient selection is key to optimal outcomes after LVAD therapy. There is a critical need for an accurate predictive model that is derived from a comprehensive database across multiple clinical sites, incorporates the impact of a large number of clinical variables to account for the heterogeneity of patients with end-stage HF, and is up-to-date with the evolving technological innovation in mechanical circulatory support devices. In other words, a successful predictive tool would mimic human decision-making, while drawing on data from tens of thousands of patients who have undergone LVAD implantation. BN algorithms can provide the necessary tools to achieve this, as demonstrated in our analysis. Our models present a percent probability for patient survival at each time point. When assessing the model performance using a 50% threshold (i.e., a patient with 60% survival at 12 months is indicated as alive), there is a lower sensitivity (30% to 33%) but a much higher specificity (87% to 90%). Using this threshold, clinicians would be urged to take a much higher note of caution in their decision-making for patients deemed high risk (with a very high degree of specificity) by the model. Yet, they will be given full freedom to interpret the patient’s survival probability when making the decision to implant, especially those deemed to be at a lower risk.
Bayesian analyses revealed a variety of risk factors from disparate categories (e.g., demographics, medical history, and laboratory test results) that influence post-LVAD survival. Many of the variables that were found to be predictive in these models have previously been recognized as high-risk factors in separate analyses (13–15). Rather than trying to combine a multiplicity of factors by using a weighted summation, Bayesian models provide a dynamic incorporation of many variables, yielding a more robust ROC value than previously published scores (6,11). The 90-day and 1-year HeartMate Risk Score stratifications had an area under the curve of 61% and 59%, respectively, whereas the Bayesian 90-day and 1-year predictions exhibited area under the curve of 71% and 70%, respectively (15).
The usefulness of the BN methodology has only been recognized within the past 25 years, with the more recent application of BN-based decision support being published in a variety of medical disciplines (16–19). We have previously published Bayesian-derived algorithms in predicting clinical outcomes after LVAD implantation, including short-term mortality, right ventricular failure, and other adverse events (10,11). They have been shown to have superior predictive performance over traditional statistical methods. Traditional statistical methods are composed of weighted combinations of independent variables resulting in “black box” risk scores, which can only account for a restricted number of variables that often fail to adequately represent the complexity of a patient with end-stage HF. Comparatively, BNs provide the advantages of a rigorous probabilistic framework that uses inference of multiple variables and a visual representation that is interactive and easy to interpret.
In the present analysis, there were several variables found to have a significant impact on the predicted mortality at different time points after LVAD implantation. These included both clinical and non–patient-related variables, both of which play a vital role in the decision-making that occurs on a day-to-day basis with these often critically ill patients. An example of a non–patient-related variable is the number of LVAD implants performed at a site annually, which has been shown to influence patient outcomes (13). The final BN models included both nonmodifiable/historical variables (such as patient age and surgical intervention history) and modifiable variables (such as nutritional assessment and renal function). We recognize that long-term mortality after LVAD implantation is likely more influenced by post-operative adverse events (such as stroke, infections, and right ventricular failure) than pre-operative markers, which is reflected by a slight decrease in the ROC at the 1-year mark.
BN analyses have the ability to show how clinical variables (e.g., hepatic and renal dysfunction) affect the predicted class value (mortality), without requiring that every variable be entered to give a prediction. With CORA models, the algorithms become more accurate in prediction as additional data points are entered, but a reasonable (albeit with a wider range of confidence) prediction can be assessed if only a few key variables are available. This is another advantage over existing risk scores, which are rendered unusable if any of the parameters are not available for input. This also allows a user to input these various scenarios and calculate the changes in predicted mortality and other adverse events in a highly interactive fashion. For example, if high creatinine is felt to be a particular high-risk variable for a specific patient, the clinician can input the impact a lower value (after optimization) to assess its impact on proposed outcome (right ventricular failure and mortality) on the same web interface, without having to re-enter data in individual risk score models. We can also demonstrate that although certain pre-operative variables are individually associated with greater mortality, having several of these risk factors compounded their impact in an exponential fashion. For example, a 65-year-old, male patient with an INTERMACS 3 profile on dialysis had a predicted 10% risk of death at 90 days after implant (scenario 1), which increased to 51% if they also required extracorporeal membrane oxygenation (scenario 2) (Figure 6).
The ability to recognize the impact of different variables in predicting mortality at various time points after implantation is important, given that many high-risk variables (e.g., acute cardiorenal failure) that could influence short-term mortality may reverse with time and be less relevant in predicting long-term outcomes. Although there are some high-risk features that influence both short- and long-term risks of mortality, their impact may change over time. Extrapolating data from 90-day models to predict 1-year mortality as was done in HeartMate Risk Score comes with its own inherent assumptions (14), which are overcome by this Bayesian methodology.
The CORA models demonstrated a remarkable improvement over existing models with respect to accuracy, specificity, and ROC. The models in this study have an ability to: 1) learn from prior probability; 2) apply to the most recent patient mix and device technology; and 3) be more tolerant to missing data elements when calculating predictions. In addition, BNs reflect the natural clinical decision-making process compared with traditional risk scores and, therefore, provide greater confidence as a tool for those making medical decisions.
We acknowledge that this study has several important limitations, including missing data pertaining to the independent variables. Although the INTERMACS database is large and representative, it suffers from sparsity of many data elements. This factor affects the analysis by having patients used in both the model training and validation who have up to 40% of their data missing. If missing data per patient is related to the health of the patient (e.g., patient was too sick or emergent and tests could not be done), the analysis will be skewed toward healthy patients. Additional limitations include the uneven distribution of many continuous variables and skew of categorical variables; inherent retrospective bias (all patients were already chosen to receive a ventricular assist device); and finally, only U.S. Food and Drug Administration–approved ventricular assist devices were included in registry. Furthermore, the preponderance of the INTERMACS dataset is derived from the Heart Mate II (Thoratec Corporation, Pleasanton, California) and, therefore, might not accurately reflect outcomes of competitive devices.
The BN models show great promise as reliable and accurate risk stratification tools for clinical decision making. The potential usefulness of the CORA models is to assist the medical team in decision-making with patients for whom the merits or contraindications to LVAD implantation are not immediately clinically apparent. Accordingly, we hope that CORA will promote the appropriate and perhaps judicious use of LVAD therapy by providing clinicians and patients a more informed decision regarding potential short-term and long-term outcomes.
COMPETENCY IN MEDICAL KNOWLEDGE: LVADs are being used in an increasing number of patients with end-stage HF across the world. Clinical decision-making in predicting outcomes in these patients remains an art, which can be supplemented by the science of Bayesian analysis.
TRANSLATIONAL OUTLOOK: We recognize that the usefulness of these BN models, containing more than 20 variables each, will depend greatly on the ease with which they can be calculated and accessed. For this reason, ongoing work aims to allow the incorporation of electronic health record data into decision support tools for physicians and patients engaged in LVAD discussions and decision-making. The online application for CORA will be provided in the form of an interactive interface accessible on smartphones and other computing devices. It will provide the user access to all the Bayesian models derived for predicting adverse events and mortality in a patient being evaluated for an LVAD. (The reader is invited to contact the corresponding author for access to the beta version of the Web-based application).
The authors thank the Data Access, Analysis, and Publications Committee of INTERMACS for allowing us to use their registry for the study. The authors also thank Susan Meyers and Grant Studdard for administrative, database, and statistical assistance with INTERMACS. We are also grateful for contributions from Dr. Marek Druzdzel and Decision Systems Laboratory, University of Pittsburgh.
Funding for this work was provided by National Institutes of Health, Division of National Heart, Lung, and Blood Institute grants R41 HL120428 STTR Phase I Cardiac Health Risk Stratification System and R01 HL122639, CORA: A Personalized Cardiac Counselor for Optimal Therapy. Data for this study were provided by the International Registry for Mechanical Circulatory Support (INTERMACS), funded from the National Heart, Lung, and Blood Institute, National Institutes of Health, under Contract No. HHSN268201100025C. The authors have reported that they have no relationships relevant to the contents of this paper to disclose.
Barry Greenberg, MD, served as Guest Editor for this paper.
- Abbreviations and Acronyms
- Bayesian network
- Cardiac Outcomes Risk Assessment
- heart failure
- Inter-Agency Registry for Mechanically Assisted Circulatory Support
- left ventricular assist device
- naïve Bayes
- receiver operator characteristics
- right ventricular assist device
- ventricular assist device
- Received December 4, 2017.
- Revision received February 2, 2018.
- Accepted March 28, 2018.
- 2018 American College of Cardiology Foundation
- Kirklin J.K.,
- Pagani F.D.,
- Kormos R.L.,
- Stevenson L.W.,
- Blume E.D.,
- Myers S.L.,
- et al.
- Yancy C.W.,
- Jessup M.,
- Bozkurt B.,
- Butler J.,
- Casey D.E. Jr..,
- Colvin M.M.,
- et al.
- Miller L.W.,
- Guglin M.
- Kanwar M.K.,
- Lohmueller L.C.,
- Kormos R.L.,
- Loghmanpour N.A.,
- Benza R.L.,
- Mentz R.J.,
- et al.
- Ravichandran A.K.,
- Cowger J.
- Loghmanpour N.A.,
- Kormos R.L.,
- Kanwar M.K.,
- Teuteberg J.J.,
- Murali S.,
- Antaki J.F.
- Cowger J.A.,
- Stulak J.M.,
- Shah P.,
- Dardas T.F.,
- Pagani F.D.,
- Dunlay S.M.,
- et al.
- Cowger J.A.,
- Castle L.,
- Aaronson K.D.,
- Slaughter M.S.,
- Moainie S.,
- Walsh M.,
- et al.
- Cowger J.,
- Sundareswaran K.,
- Rogers J.G.,
- Park S.J.,
- Pagani F.D.,
- Bhat G.,
- et al.
- Quintana M.,
- Viele K.,
- Lewis R.J.
- Johnson S.R.,
- Granton J.T.,
- Tomlinson G.A.,
- Grosbein H.A.,
- Le T.,
- Lee P.,
- et al.
- Khan S.U.,
- Winnicka L.,
- Saleem M.A.,
- Rahman H.,
- Rehman N.