Author + information
- Received December 20, 2016
- Accepted December 22, 2016
- Published online February 27, 2017.
- Milton Packer, MD∗ ()
- ↵∗Address for correspondence:
Dr. Milton Packer, Baylor Heart and Vascular Institute, Baylor University Medical Center, 621 North Hall Street, Dallas, Texas 75226.
Many physicians boast that there is no reason to read or keep up with the medical literature. Why? If it is true that 1 trial is never enough to change clinical practice, clinicians can always claim that they are waiting for confirmatory evidence and for critical review by regulatory authorities and peer groups. Such thinking greatly simplifies the challenge of maintaining medical knowledge. Practitioners need not become familiar with any scientific study; they need only to wait for guideline statements, which will determine when a critical mass of evidence has been compiled. Sadly, some clinicians will also wait to see what practices and procedures are reimbursed, because this—not scientific evidence—has become a major factor influencing medical care.
This current approach to medical practice is egregiously wrong. The belief that there is no need to be up-to-date on medical evidence—because no observation is valid unless it has been replicated—represents the worst possible excuse for intellectual lethargy and clinical inertia. Furthermore, it allows the practitioner to relegate the process of distilling and deciphering the totality of evidence to others, who may have their own limitations or conflicts. The present state of affairs also fosters efforts to generate replicative evidence by artificial (and often ethically dubious) means, in order to be able to state categorically that an observation has been confirmed.
A History of Heart Failure Trials Showing a Survival Benefit
It is scientifically unnecessary and ethically impossible to replicate a trial that shows (with a high degree of confidence) that an intervention prolongs life. Such a situation, however, must be distinguished from the observations of early development trials that report imbalances in the reported risk of death between 2 treatment groups. When conclusions about drug effects are based on a small number of total events (e.g., <200), treatment differences (although large in magnitude) are determined more by the play of chance that by a true effect of the intervention. The reliability of a finding in an underpowered trial depends not on its nominal p value, but on the narrowness of the confidence intervals, and thus, its likelihood of replication. When a survival benefit is reported in a trial not designed to examine the effect of the intervention on the risk of death, the findings (with respect to survival) are unreliable (because they are based on sparse information) and often, are not confirmed by subsequent trials (Table 1). This is true even though the patient population being studied in the definitive trial is precisely the same as the one that originally was the basis of the reported survival benefit.
This situation, however, differs dramatically from the finding of a survival benefit in a trial that has been specifically designed to evaluate the influence of a new treatment on the risk of death (Table 2). Such trials are rare; during the past 40 years, fewer than 10 trials that have been designed to evaluate the effect of a drug on mortality in patients with chronic heart failure have reported a survival benefit. The number of events supporting the conclusion of a mortality risk reduction in such trials is generally larger than 200 and is substantially greater than the number studied in the early development trials (Table 1).
However, as is seen in Table 2, no effort has ever been made to replicate the results of a definitive finding of a survival benefit in the same population of patients with heart failure that was originally studied. In the case of angiotensin-converting enzyme (ACE) inhibitors, the 3 trials evaluated entirely distinct nonoverlapping groups of patients. Several large-scale trials reported similar effects of different beta-blockers on mortality in chronic heart failure, but the trials were carried out at the same time or were carried out in patients that were excluded from earlier trials. The 3 large-scale outcome trials with mineralocorticoid receptor antagonists studied nonoverlapping patient populations. With respect to the combination of hydralazine and isosorbide dinitrate, the second trial focused on patients who were not studied in the earlier trial (i.e., women and African Americans receiving neurohormonal antagonists).
Hence, in the field of heart failure, there has never been replication of the result of a survival benefit if it was reported in a definitively designed trial. When the benefits of a drug have been convincingly demonstrated in a specific patient population, subsequent trials (if carried out) always focus on studying a distinct and nonoverlapping group of patients. A similar population is targeted for confirmation only when the initial report of a survival difference is generated as a result of an unreliable estimate in a trial reporting a small number of events (Table 1). This point is often missed in the development of heart failure guidelines: a Class IA recommendation for a survival benefit is never based on replication of the results with the same drug in the same clinical setting. Instead, a Class IA recommendation for mortality reduction (when based on sequentially executed trials) is based on experience in clinically distinct populations. The results of a survival benefit observed in a single adequately designed heart failure trial are always considered definitive. They are never replicated; they are complemented. Therefore, a Class IA recommendation does not represent greater strength of evidence or a higher level of confidence than a Class IB recommendation; it simply indicates a broader spectrum of therapeutic activity.
The Dilemma Posed by a Trial With a Survival Benefit That Is Granted a Class IB Recommendation
In the PARADIGM-HF (Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure) trial, the angiotensin neprilysin inhibitor sacubitril/valsartan was shown to reduce the risk of cardiovascular death in a persuasive manner (1); the advantage of the drug over a conventional inhibitor of the renin-angiotensin system was based on more than 1,200 events. The data from the trial have been examined by numerous regulatory agencies, and the drug is approved throughout the world as a replacement for ACE inhibitors and angiotensin receptor blockers. Yet, because there is only 1 trial, the drug carries a 1B (rather than a 1A) recommendation, even though the strength of evidence is greater than any previous trial or combination of trials (2). Uptake of the medication by practitioners has been slow because third-party payers have impeded access to an expensive drug. However, it would be wrong to conclude that uptake should be slow because the results of the PARADIGM-HF trial have not been (and need to be) replicated.
Given that we cannot (and do not) replicate the results of a trial with a definitive survival benefit, how can we give physicians the comfort that a Class IA recommendation for the drug would provide? A variety of creative approaches have been used to distinguish the findings of the PARADIGM-HF trial. First, the U.S. guidelines provide sacubitril/valsartan with a unique Class IB-R recommendation, indicating that the strength of evidence supporting the use of sacubitril/valsartan is more persuasive than the usual Class 1B recommendation. Second, a meta-analysis (3) that combined the results of trials with 2 different neprilysin inhibitors (omapatrilat and sacubitril/valsartan) was carried out solely for the purpose of supporting a Class IA recommendation, even though the dosing regimens used for the 2 drugs were not comparable. Third, a new trial (PARADISE-MI [Prospective ARNI vs ACE Inhibitor Trial to Determine Superiority in Reducing Heart Failure Events After MI]) (4) has been launched to determine whether sacubitril/valsartan is superior to an ACE inhibitor in patients with left ventricular dysfunction following an acute myocardial infarction.
Other approaches to producing evidence that might complement the results of the PARADIGM-HF trial have been considered. One could conceive of a trial that compares very high and very low doses of sacubitril/valsartan on the survival of patients with class II to IV heart failure, but such a proposal has serious ethical concerns. Others have suggested the evaluation of patients who were specifically excluded from the PARADIGM-HF trial, for example, hospitalized patients. However, hospitalized patients evolve into outpatients very rapidly, and given the benefits seen in the PARADIGM-HF trial, outpatients with chronic heart failure cannot be ethically deprived of treatment with sacubitril/valsartan for meaningful periods of time. Consequently, an ongoing study of the drug in hospitalized patients (PIONEER-HF [Comparison of Sacubitril/Valsartan Versus Enalapril on Effect on NTpro-BNP in Patients Stabilized From an Acute Heart Failure Episode]) (5) is truncating follow-up after only 8 weeks.
This paper describes the confusion we cause when we claim that the survival benefits in heart failure trials need be replicated. In truth, we require replication only when the trial reporting favorable results cannot possibly yield reliable data (Table 1), due to the sparse amount of events. By contrast, we fully accept the results of single trials when definitively designated and well-executed studies reliably demonstrate a meaningful mortality reduction. Under such circumstances, we do not ask for replication, but we are pleased when we see complementary findings that broaden the scope of a drug’s benefits. However, the search for a Class IA recommendation is not a search for the certainty of a therapeutic benefit, and it has practical and ethical limits.
For supplemental references, please see the online version of this article.
Dr. Packer has consulted for Admittance, Amgen, AstraZeneca, Bayer, BioControl, Boehringer Ingelheim, Cardio3, Cardiokinetix, Cardiorentis, Cytokinetics, Daiichi Sankyo, GlaxoSmithKline, Novartis, Takeda, and ZS Pharma.
- Received December 20, 2016.
- Accepted December 22, 2016.
- 2017 American College of Cardiology Foundation
- Packer M.
- Solomon S.D.,
- Claggett B.,
- McMurray J.J.,
- Hernandez A.F.,
- Fonarow G.C.