Can Peer Review Solve Conflicts of Interest?

March 4, 2009

As I wrote recently, Stephen Quake has been writing about conflicts of interest in research over at The Wild Side blog. He proposes solving these problems with peer review. I like the article, and he has many thoughtful things to say on the topic, but I don't really understand this proposal.

He doesn't give a lot of details as to how this would work. For instance:

When this bureaucracy asked me for a plan to manage conflicts in my own research, I wrote one that described all of the steps involved in peer review – and the COI committee sent it back as “too much.” In their view the process that scientific publications go through was more rigorous than necessary.

This reads as if the committee thought it would be too much work for him, but it sounds like too much work for the peer reviewers. Peer reviewers are not professional peer reviewers: they are typically volunteers from within the community who review papers partly out of a sense of social responsibility. Any proposal for expanding peer review has to keep in mind that the reviewers might not want the extra responsibilities.

I have not seen his proposal, but I'm not sure if it would work even in principle. As I wrote previously, nobody really knows what the data are but the researcher. Analyses can always be run in many different ways, potentially giving different results -- a problem whether your concern is conscious bias or unconscious bias. It's just not clear what peer review is supposed to do about that.

(Image borrowed from wi.mit.edu; Paradigm Magazine)

Permalink