At oncology meetings, nearly one third of oral presentations on cancer clinical trials where the primary end point was not met presented a positive, or "not negative," conclusion about a formally negative result, according to results of a quality improvement analysis now published in JAMA Oncology.
"Presentations at medical meetings have great resonance within the scientific community, especially following the diffusion of social media," write the investigators, led by first author Massimo Di Maio, MD, Associate Professor of Medical Oncology at the University of Turin, Italy. "Oral presentations are not subject to peer review, and some authors' conclusions may not be completely justified by the results. This is particularly critical when, despite the formally negative trial result, the authors' conclusions are not negative."
Seeking to determine how often this phenomenon occurs, Dr. Di Maio and colleagues analyzed oral presentations of phase 3 randomized clinical trials presented at American Society of Clinical Oncology (ASCO) and European Society for Medical Oncology (ESMO) meetings from 2017 through 2019. They excluded noninferiority trials from their analysis. The researchers defined a trial's result as either negative or "not negative" based on whether it met the trial's primary end point. The investigators then compared the trial results to the conclusions presented, considering the conclusions of negative trials to be "not negative" when the authors "consider the possibility of using the experimental treatment in that setting, without clear conclusions about the study negativity."
Of the 208 randomized clinical trials included in the analysis, 91 were formally negative. Of these, 26 trials (29%) presented a not-negative conclusion. The proportion of negative trials with not-negative conclusions varied by year, with 22% in 2017 and 13% in 2018 rising dramatically to 47% in 2019. Not-negative conclusions were slightly more common in nonprofit studies than in for-profit studies (30% vs 26%). In 50% of the negative studies with not-negative conclusions, the authors emphasized a numerically better outcome in the experimental arm. A positive result in one or more subgroups was emphasized in 46%, while 38% emphasized a positive result in one or more secondary end points. In 27% of negative trials with not-negative conclusions, the authors did a post hoc interpretation of the study as a noninferiority design.
"Each of the reasons used to underline positive aspects of a formally negative result should be presented with caution," note Dr. Di Maio and colleages. "Positive subgroup analysis of negative trials can be misleading, should not support treatment adoption, and, at best, should be hypothesis generating… Positive analyses of secondary end points in trials with a negative primary analysis carry the same risk of false-positive results inherent in multiple testing. Furthermore, the primary end point is the measure used for the study hypothesis… When a trial is designed to test the superiority of an experimental treatment, post hoc interpretation of noninferiority is methodologically debatable."
How, then, should negative trial results be portrayed?
"We believe that more attention should be paid to the statements included in the conclusions of oral presentations at meetings, and the discussants' role is crucial," conclude Dr. Di Maio and colleagues. "When the primary end point is not met, the word 'negative' should be explicitly used."
For More Information
Di Maio M, Audisio M, Cardone C, et al (2020). The use of not-negative conclusions to describe results of formally negative trials presented at oncology meetings. JAMA Oncol. [Epub ahead of print] DOI:10.1001/jamaoncol.2020.0475
Image credit: Stephen Rose. Licensed under CC BY-SA 4.0