DREaM event 5: Closing Keynote
Dr Ben Goldacre presented the closing keynote at the final DREaM conference, in which he will discuss “Research, evidence bases, decision making and policy.”
Ben provided a preview of this session in a short interview.
This video is password protected and only accessible to members of the DREaM online community. If you wish to watch the video, please first apply for membership of the community. The video will be accessible to DREaM online community members until Wednesday January 3rd 2013.
Goldacre began by clarifying that although the title of his book and column “Bad Science” seems really mean, science is all about unpicking evidence. So, by taking apart the claims made by science in popular culture, he is doing what science is supposed to do. He used this as the starting point for a presentation that described how data can be corralled and misused, and to emphasise how these issues are very much at LIS researchers’ end of the scientific process.
It is easy to forget that there is a real world application for the data produced… It can change the way people actually behave.
Goldacre believes that fluke findings are often disproportionately reported by both the media and academic journals, whilst negative articles and attempts at experimental replications are often overlooked for publication. This publication bias is distorting. Roughly half of the studies of the effectiveness of drugs never get published, whilst studies with positive results are twice as likely to get published. This means people get put at risk because data is withheld, and people can die as a result.
He discussed the importance of phase one or “first in human” clinical trial results, which are rarely published. He noted that there is no legal obligation for pharmaceutical companies to publish these results, and they are often considered commercially sensitive. However, this means people taking part in such trials can be put at risk. This is because those conducting such studies do not have access to information about previous, similar trials.
Goldacre also observed that it is perfectly legal for pharmaceutical companies not to publish negative studies, and to focus on outcomes that show their drug in a favourable light, rather than the results that pertain to the original primary aims of the experiment. He argued that every single paper should be checked to make sure that the primary outcome reported is the same as the original aims in the trial registers.
There have been some regulatory efforts made to address these issues. Goldacre highlighted the FDA Amendment Act 2007, which states that phase two and three clinical trials must be published within a year of finishing the study. However, this has not been effectively monitored and the reporting rate is still only one in five.
Goldacre argued that the problem is essentially one of the information architecture. He introduced a side project he is running to address some of the issues called alltrials.org. The project involves collecting information about clinical studies and matching data sources about those trials, including the record on the trial register, conference presentations, press releases and so on. This work will help draw a thread between all the digital entities that exist for one piece of research. He also intends to track how people search PubMed to help identify search strategies that are more or less likely to find the paper required to follow up on a particular trial on the trial register, with a view to automating the process in the future. Goldacre appealed to the LIS community to get involved with these activities and suggest ways in which they could be made more efficient. If you are interested, please e-mail Ben at email@example.com.
Finally, Goldacre observed how we fail to harness the data we already have available that could be repurposed for other means at no or little expense. He cited an example of trial by HMRC to see how the wording of their letters affects response times for tax returns. This trial used existing administrative systems, so was extremely low cost, and yielded really useful results. Goldacre argued that there are many more opportunities to do similar (non-medical related) work by harnessing existing data sets.
Ben then returned to the medical realm to emphasise the importance of randomisation in sample selection in a discussion of “channelling”. To illustrate, doctors selectively put their sicker patients on what looks like the safer of a choice of two drugs. This means the drug appears to perform badly in comparison with the alternative drug – but this could be explained by the fact that the drug is routinely prescribed to sicker patients in the first place. Conducting fully randomised trials is the only way around this.
In the conclusion to his talk Goldacre complained that knowledge has such a low priority for people that they can’t see that small, cheap techniques to extract real meaning from chaotic data are really important. LIS researchers have the opportunity to use their skills to change this.
If you would like to comment on this presentation, please join the discussion in the DREaM online community.