Author + information
- Christopher M. O’Connor, MD, Editor-in-Chief, JACC: Heart Failurea,∗ ( and )
- Dirk J. van Veldhuisen, MD, Former Editor-in-Chief, European Journal of Heart Failureb
- aAmerican College of Cardiology, Washington, DC
- bDepartment of Cardiology, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands
- ↵∗Address for correspondence:
Dr. Christopher M. O’Connor, Editor-in-Chief, JACC: Heart Failure, American College of Cardiology, Heart House, 2400 N Street NW, Washington, DC 20037.
Recent efforts by the National Institutes of Health, Food and Drug Administration, and other entities have promoted greater data transparency and availability of clinical trial databases to the general public. The National Institutes of Health has significant regulations requiring publicly funded clinical trials to become available 2 years after publication of the primary results. With this has come an exponential availability of datasets and analyses at independent centers and the publication of new information from these previously unavailable studies. Despite this lofty goal in a world of greater transparency, the results thus far of providing clinical trial databases to independent investigators have been mixed, with several limitations. These limitations have become apparent to primary investigators of these clinical trials and to journal editors who are receiving the papers.
It is apparent that when these databases move to new investigative sites, the investigators have been reluctant to involve the original investigators. This has been an unfortunate result of these new policies. Investigators who spent 5 to 10 years accumulating, aggregating, analyzing, providing surveillance, and optimizing the quality of the database have been left out of the subsequent analyses. Furthermore, the statisticians who have been involved in establishing the rules of evidence, deriving and developing predictive models, and establishing definitions have also been left out of these analyses. In some trial databases, it is estimated that over 50% of the papers developed by independent investigators have significant errors in the methodology. These errors of methodology involve inaccurate sample size, inaccurate definition of variables, poorly developed models, unestablished rules of covariate definition, improperly developed primary and secondary hypothesis, no correction for multiple testing, and inconsistent analyses. The result is that we have been inundated with poorly developed papers advocating discovery of new knowledge, which are found to be inaccurate. We have seen a number of papers on the same topic with different results, causing confusion as to which information is correct or erroneous. A recent relevant case is the analysis of digoxin in patients with atrial fibrillation from the AFFIRM (Atrial Fibrillation Follow-up Investigation of Rhythm Management) study, in which separate investigators using propensity analysis found different outcomes (1–3).
Data transparency efforts are laudable and important, but we must establish rules of engagement for the use of publicly available clinical trial datasets for independent investigators.
As editors, we propose some basic rules:
1. The development of hypotheses and analysis plans from a clinical trials dataset should be run by any existing publication committee chair or previous steering committee principal investigator for up to 10 years post-trial. By having this oversight, duplicate publications on the same topic can be avoided, and if such publications occur, authors should explain why they have taken a second, more or less similar angle on the subject and explain potential differences and similarities.
2. The independent investigators should involve the principal investigators and the primary statistician in the development of hypotheses and an analytical plan. Independent investigators should harmonize all statistical methodology, endpoint definitions, and modeling techniques for their scientific investigations.
3. The principal investigators and the primary biostatistician should be offered the opportunities to be coauthors in these investigations.
4. Independent investigators should understand what has previously been published from these databases and what analyses have been conducted, but not published, so that they are not moving forward with blind analyses. When results differ, the investigators should be able to explain the reason for differences.
5. Independent investigators should obtain and review the original statistical analytical plan for the trial and cite it as a reference.
Despite this proposed guidance, a number of questions remain for journal editors. What should we do when papers are submitted from the same database, with similar conclusions, but not exactly the same numbers? We sometimes become aware of this by serving as reviewers for other journals. As an editor, should we let it go, and let both papers be published? Should we notify the authors? If the first one gets in, does the second one lose priority? What if they are submitted at the same time?
Although we support the importance of data transparency, freedom of investigation, and publicly available databases, including privately sponsored studies, we must have rules of engagement to protect from noise, inaccurate information, conflicting data, duplicative data, and data that appear to be nonsensical because of a lack of understanding of the studies in their ascertainment, aggregation, management, and surveillance of information that has been so carefully conducted over a 5- to 10-year period. This conflict of information could be harmful to patients and clinical science.
Let us move forward with data transparency and data freedom with enthusiasm, but caution, and improve the ultimate goal of advancing knowledge and not advancing noise.
- 2017 American College of Cardiology Foundation