Dynamic Chiropractic

Dynamic Chiropractic Facebook Twitter
Dynamic Chiropractic
Find
Advanced Search
Wellness Blog
Dynamic Chiropractic
Dynamic Chiropractic PracticeINSIGHTS
Current Graphic
Facebook

Outcome Switching in Scientific Publication

Much of my career has been spent working in part as a journal editor, as a scientist and researcher, and as a teacher. In my current teaching position, I teach a course on evidence-based chiropractic practice. Part of that course is given over to critical appraisal of literature. The reason for placing emphasis on critical appraisal is simple: it allows you, a reader, to extract meaningful information from a paper, once you have judged that the paper has sufficient rigor for you to feel comfortable using that information. In our internet age, it is possible to find a plethora of information, but not all information is equal, and some is not worth using. This is true even if the finest of bioscientific journals.

Which brings me to the work of medical gadfly Ben Goldacre. I call him a gadfly because the term has traditionally been used to denote someone who launches incisive attacks on the establishment. Goldacre came to fame with a series of TED talks he gave, in which he discussed poor and misleading reports of medical literature. He authored two books, Bad Science and Bad Pharma and has a third coming out. He has been outspoken on the concept of publication bias, the failure to report research with negative findings. As you can imagine, this might have significant issues in pharmaceutical research.

Goldacre has now turned his attention to what he refers to as “outcome switching.” This is a subtle problem in scientific publication, wherein the authors of a research paper report different outcomes for their work than the ones which they indicate they will look at in their protocol. Goldacre notes in a recent editorial for the journal Nature that “those conducting clinical trials are supposed to publicly declare what measurements they will take to assess the relative benefits of the treatment being compared.” (1) And that this does not always happen is a growing concern. We must keep in mind that research protocols specify levels of statistical significance based on those primary outcomes to be measured; changing out the outcomes may have effects on the statistics, making a study look strong when it is not, for example.

But Goldacre has decided that simply noting this problem is not enough. He has developed COMPare-trials.org, at the Centre for Evidence-Based Medicine at Oxford, UK. His group has gone so far as to take papers published in the top 5 medical journals and examine their reported outcomes against their submitted protocols. And when they identify outcome switching, they then send a letter to the editor of the journal to point that out. Corrections have been issues as a result. As well as controversy with at least one major journal, which is referred to in the editorial I cite above.

I raise this issue here insofar as it is a measure of how self-correcting good science can be, to note that no matter the quality of the journal, you still need to read carefully, and to signal to you to be aware of some of the subtle pitfalls that may occur.

 

References

  1. Goldacre B. Make journals report clinical trials properly. Nature 2016;530:7

Leave a Reply