Hypothesis Overdrive? Posted by Dr. Jon Lorsch on March 26, 2014 Historically, this blog has focused on news you can use, but in the spirit of two-way communication, for this post I thought I would try something that might generate more discussion. I m sharing my thoughts on an issue I ve been contemplating a lot: the hazards of overly hypothesis-driven science. When I was a member of one study section, I often saw grant applications that began, The overarching hypothesis of this application is. Frequently, these applications were from junior investigators who, I suspect, had been counseled that what study sections want is hypothesis-driven science. In fact, one can even find this advice in articles about grantsmanship. Despite these beliefs about what study sections want, such applications often received unfavorable reviews because the panel felt that if the overarching hypothesis turned out to be wrong, the only thing that would be learned is that the hypothesis was wrong. Knowing how a biological system doesn t work is certainly useful, but most basic research study sections expect that a grant will tell us more about how biological systems do work, regardless of the outcomes of the proposed experiments. Rather than praising these applications for being hypothesis-driven, the study section often criticized them for being overly hypothesis-driven. Many people besides me have worried about an almost dogmatic emphasis on hypothesis-driven science as the gold standard for biomedical research (e.g., see Jewett, 2005; Beard and Kushmerick, 2009; Glass, 2014 ). But the issue here is even deeper than just grantsmanship, and I think it is also relevant to recent concerns over the reproducibility of scientific data and the correctness of conclusions drawn from those data. It is too easy for us to become enamored with our hypotheses, a phenomenon that has been called confirmation bias. Data that support an exciting, novel hypothesis will likely appear in a high-impact journal and lead to recognition in the field. This creates an incentive to show
that the hypothesis is correct and a disincentive to proving it wrong. Focusing on a single hypothesis also produces tunnel vision, making it harder to see other possible explanations for the data and sometimes leading us to ignore anomalies that might actually be the key to a genuine breakthrough. In a 1964 paper, John Platt codified an alternative approach to the standard conception of the scientific method, which he named strong inference. In strong inference, scientists always produce multiple hypotheses that will explain their data and then design experiments that will distinguish among these alternative hypotheses. The advantage, at least in principle, is that it forces us to consider different explanations for our results at every stage, minimizing confirmation bias and tunnel vision. Another way of addressing the hazards of hypothesis-driven science is to shift toward a paradigm of question-driven science. In question-driven science, the focus is on answering questions: How does this system work? What does this protein do? Why does this mutation produce this phenotype? By putting questions ahead of hypotheses, getting the answer becomes the goal rather than proving a particular idea. A scientific approach that puts questions first and includes multiple models to explain our observations offers significant benefits for fundamental biomedical research. In order to make progress, it may sometimes be necessary to start with experiments designed to give us information and leads Who are the players? or What happens when we change this? before we can develop any models or hypotheses at all. This kind of work is often maligned as fishing expeditions and criticized for not being hypothesis-driven, but history has shown us just how important it can be for producing clues that eventually lead to breakthroughs. For example, genetic screens for mutations affecting development in C. elegans set the stage for the discovery of microrna-mediated regulation of gene expression. Is it time to stop talking about hypothesis-driven science and to focus instead on question-driven science? Hypotheses and models are important intermediates in the scientific process, but should they be in the driver s seat? Let me know what you think.
Google + This entry was posted in Director s Messages and tagged Rigor and Reproducibility. Bookmark the permalink [https://loop.nigms.nih.gov /2014/03/hypothesis-overdrive/]. 59 comments on Hypothesis Overdrive? John Rodgers on April 19, 2014 at 12:42 PM said: While I agree with the critique of hypothesis by Glass and by Glass & Hall, I believe that basing research funded by the public dollar on questions misses the point. What is intrigues a researcher in the form of a question may be very hard to explain to a layperson, or even a colleague in the same field. I prefer to explain to my students that research proposals must be problem-solving. These are typically mechanistic or design/engineering or targeted description (essentially the same as Freedman s taxonomy of research problems -Freedman, P. (1960). The Principles of Scientific Research. Oxford, Pergamon Press. ), but inevitably the solution to scientific problems involves all three modes of research. Problems are solved by proposed design, hypotheses or search strategies, all of three of which can be collected under the rubric of testable solutions. Hypotheses are tools for solving problems; and we should dispense with any hypothesis as soon as it no longer is helping us solve the problem. I would suggest that questions and models (more or less as described by Glass and Hall) are also tools for problem solving. Scientists should love hypotheses or questions no more than a journeyman loves her hammer. Now, I suggest, that anyone can come to understand
a problem, even if they never understand the hypothesis. Now, problems can be posed as questions, but not all questions identify problems. How does Toxoplasma infection cause mice to lose their fear of cats?, refers to a problem but Does Toxoplasma damage fear-processing neurons in the amygdala refers to an hypothesis that MIGHT solve the problem. Similarly, we can identify a designengineering problem as a question ( How can we send humans to Mars and bring them back alive ) or as a statement ( The problem is to send humans to Mars and bring them back alive ). Similarly, we can express the targetted description of the human genome as a questions ( What is the sequence of the human genome? ) or as a mechanistic question ( How can we explain the genetic basis of human disease?) or as search stragety ( Sequencing the human genome should reveals details critical to explaining the genetic basis of human disease ). Leigh on June 13, 2014 at 6:19 PM said: NIH grants: Focus on questions, not hypotheses http://www.nature.com/nature/journal/v507/n7492 /full/507306d.html Radio on July 18, 2014 at 4:26 AM said: Perhaps it could be framed as goal driven science. For instance, one could say, my goal is to understand the function of the MafP1 signalling system in mammalian development. Then you are not bound by the questions you can think to ask. You can just keep
asking and answering questions until you reach your goal.