Author + information
- Published online February 26, 2018.
- aDepartment of Laboratory Medicine and Pathology, Mayo Clinic and Foundation, Rochester, Minnesota
- bDepartment of Cardiovascular Medicine, Mayo Clinic and Foundation, Rochester, Minnesota
- ↵∗Address for correspondence:
Dr. Allan S. Jaffe, Division of Cardiovascular Diseases, Department of Internal Medicine, Mayo Clinic College of Medicine, 200 First Street SW, Rochester, Minnesota 55905.
Meta-analyses commonly are done to synthesize data and summarize areas of clinical interest to help support specific postulates where only multiple modest to small-sized studies exist. The paper from Evans et al. (1) in this issue of JACC: Heart Failure provides such an analysis in regard to the use of high-sensitivity cardiac troponin (hs-cTn) values to predict which patients will develop heart failure (HF). This area is particularly fertile; other studies such as STOP-HF (Screening to Prevent Heart Failure) (2) and PONTIAC (NT-proBNP Selected PreventiOn of cardiac eveNts in a populaTion of dIabetic patients without A history of Cardiac disease), using biomarkers (natriuretic peptides in these studies) identified at-risk patients potentially in need of treatment. Treatment in both instances reduced the frequency of subsequent HF. These studies, however, as in some of the studies included in the Evans et al. (1) meta-analysis, relied on values lower than the typically used clinical cutoff values (the 99th percent upper reference limit [URL]) to define risk. For hs-cTn this is a good strategy, because the comorbidities that lead to cardiovascular diseases tend to increase hs-cTn values, although often they may not exceed the 99th percentile URL recommended to diagnose acute myocardial infarction.
Meta-analyses often are good summaries of the data in the field of interest and the one from Evans et al. is no exception. They comprehensively reviewed the published data and selected papers of good quality to include, and then analyzed the hazard ratios for the studies and concluded that values of hs-cTn can be used to identify patients who initially do not have evidence of symptomatic HF but who will progress and eventually develop HF. This was determined predominantly by HF admissions, but also by other adverse clinical events. The analysis, although it supports the hypothesis, reveals substantial unexplained heterogeneity to the designs and testing of the selected studies. So, rather than helping to clarify how to proceed based on the findings of the meta-analysis, uncertainty is extended. This marked heterogeneity may be why the increase in C-statistic for the analysis is so small (0.03). We believe we can offer explanations for the heterogeneity they report and in doing so sensitize clinicians to the importance of some of the analytical aspects of biomarker testing. These include the following:
1. All troponin assays including hs-cTn assays are not of equal sensitivity. The International Federation of Clinical Chemistry committee has suggested that hs-cTn assays should meet the criterion of detecting values in at least 50% of putatively normal subjects (3). This definition is thought by the expert consensus to be a rough estimator of assay sensitivity. However, the assays involved differ markedly in terms of their ability to detect hs-cTn in normal subjects. One of the assays used in many of the studies in this meta-analysis is the Roche hs-cTnT assay, which often fails to meet this threshold (4). If one compares and blends studies with those that use hs-cTn assays like the Singulex or Abbott assays, where nearly 100% of normal individuals have detectable values, then there is a mix of studies that rely on assays that may have very different sensitivities.
2. A second important consideration is what values are used to define risk. Some studies used the limit of detection (LoD), a very low value, whereas some used the higher 99th percentile URL value. However, in other studies the limit of the assay blank, which is an even lower value than the LoD, was used. Analytically, one might prefer the LoD, but some have argued that there is information at still lower values (5). Similar considerations exist in regard to the 99th percentile URL. It is clear with hs-cTn assays that the more rigorous the screening, the lower the detected values will be (5). If screening questionnaires, history, and physician examinations, imaging, and biomarkers such as N-terminal pro–B-type natriuretic peptide, hemoglobin A1c, and the estimated glomerular filtration rate are used as criteria for normality, the hs-cTn values are substantially lower than if such covariates are not used. In addition, the statistics used to generate those data vary. Thus, different 99th percentiles could be used for the same assay depending how the normal value studies were done. Because of these and other factors, there can be substantial differences when comparing assays. Add in the controversy about how best to develop 99th percentile values and the contribution of skeletal muscle false-positive results, which can occur with hs-cTnT assays and this issue is further complicated (4).
3. Many of the studies cited used different cut-off values for the 99th percent URL and the LoD, and for risk analysis. Some of this is due to identifiable institutional biases, where the recommended 99th percentile values for standard troponin and many high-sensitivity assays are not used (5). Often, the values used are substantially higher and spring from physician concerns of difficulty in explaining some increases in troponin. Thus, even robust attempts to collect data about the assays might fall short of providing ideal combinations of studies for meta-analyses, unless the cut-off values that institutions use are uncovered. However, this factor in and of itself might not be adequate, because different studies use different metrics. For example, in the study by McKie et al. from the Mayo Clinic (6), we used the 80th percentile of hs-cTnI (not hs-cTnT as in Table 1 in the paper by Evans et al. ) and the 80th percentile of N-terminal pro–B-type natriuretic peptide using sex and age specific cut-offs to predict HF because it provided optimal separation of those with a propensity to develop HF. Other studies have used values just above the LoD; that is, extremely low values, and others values above the 99th percentile (abnormal values). The higher the value, the more likely the individual is to be harboring significant underlying occult cardiovascular comorbidity. However, it is clear that there is gradation from lowest to highest values in terms of cardiovascular risk, and this is likely because risk factors (hypertension, hyperlipidemia, diabetes, obesity, and so on), when present, cause modest synergistic increases in hs-cTn.
4. An additional confounder to this process is the concept of imprecision, which is important in making sure that one value is truly different from another. Assay values that are very low are associated with higher degrees of imprecision. Categorizing patients whether it is with continuous data or in tertiles as done in this meta-analysis might allow substantial overlap, given the lack of precision causing the confidence intervals to overlap substantially. Thus, the strategy chosen and how low and/or different the values are is a key to proper implementation. If values are chosen that are too close together, the ability to distinguish groups might be difficult. If so, less robust results would be presented than if larger differences were used. This is a critical issue in many studies assessing primary and secondary prevention and needs to be kept in mind in all studies (5). There are, however, ways to mitigate these issues such as companies improving low-end imprecision. Other strategies would be to repeat the samples multiple times to reduce the variation around any given value, and, as suggested by these authors, one can reevaluate hs-cTn values over time to see if values are changing. This strategy may be valuable, with the idea that increasing values would be more prognostic. However, the change criteria used need to be greater than the reference change value, which takes into account the biologic and analytic variation of the biomarker value (5). Also, given that imprecision is worse at low values, it depends in part on the baseline or starting value. Therefore, it is important to be circumspect about minor changes. This is one reason why we have been concerned about the advocacy for tiny differences in some ACS protocols.
5. Finally, for some studies and some assays, sex-specific cutoffs seem to help in the area of primary and secondary prevention. This is not something delved into in the present meta-analysis, but other studies have found different optimums based on sex (5).
These issues do not invalidate the present meta-analysis, which suggests that hs-cTn should be helpful in identifying patients who are at risk for the development of HF. However, this and other meta-analyses should call attention to the need for biomarker studies to improve the quality of their summary analyses. Such efforts might include the following:
1. A tabulation of the metrics of the assay used in the study.
2. A tabulation of the specific metrics used by each study and whether they were sex specific.
3. An evaluation of imprecision to be sure that the categorization of patients really provides adequate separation based on the characteristics of any given assay.
4. When change values are used, an analysis to indicate whether they are significantly above the reference change value.
5. Given the differences in assays, separating studies into those using one versus another assay.
As advocates for biomarker use in patients with HF and from our previous studies, we appreciate the interesting proof of concept presented by Evans et al. (1), and with which we concur in regard to establishing estimates of those at risk for HF going forward. However, we also have tried to use this opportunity to suggest that the field needs to spend additional time and effort refining the specific techniques that might more fully optimize the ability to use summary and meta-analyses that include biomarkers.
↵∗ Editorials published in the JACC: Heart Failure reflect the views of the authors and do not necessarily represent the views of JACC: Heart Failure or the American College of Cardiology.
Dr. Jaffe has reported that he either presently or in the past has consulted for Beckman, Abbott, Siemens, Sphingotec, Becton-Dickinson, ET Healthcare, Singulex, and Novartis. Dr. Miller has reported that he has no relationships relevant to the contents of this paper to disclose.
- 2018 American College of Cardiology Foundation
- Evans J.D.W.,
- Dobbin S.J.H.,
- Pettit S.J.,
- Di Angelantonio E.,
- Willeit P.
- Apple F.S.,
- Jaffe A.S.,
- Collinson P.,
- Mockel M.,
- Ordonez-Llanos J.,
- Lindahl B.,
- et al.,
- on behalf of the International Federation of Clinical Chemistry (IFCC) Task Force on Clinical Applications of Cardiac Bio-Markers
- Apple F.S.,
- Sandoval Y.,
- Jaffe A.S.
- Wu A.H.B.,
- Christenson R.H.,
- Greene D.N.,
- et al.
- McKie P.M.,
- AbouEzzeddine O.F.,
- Scott C.G.,
- et al.