Download icon

Point of View: How should novelty be valued in science?

  1. Barak A Cohen Is a corresponding author
  1. Washington University School of Medicine, United States
Feature Article
Cited
0
Views
4,511
Comments
0
Cite as: eLife 2017;6:e28699 doi: 10.7554/eLife.28699

Abstract

Scientists are under increasing pressure to do "novel" research. Here I explore whether there are risks to overemphasizing novelty when deciding what constitutes good science. I review studies from the philosophy of science to help understand how important an explicit emphasis on novelty might be for scientific progress. I also review studies from the sociology of science to anticipate how emphasizing novelty might impact the structure and function of the scientific community. I conclude that placing too much value on novelty could have counterproductive effects on both the rate of progress in science and the organization of the scientific community. I finish by recommending that our current emphasis on novelty be replaced by a renewed emphasis on predictive power as a characteristic of good science.

https://doi.org/10.7554/eLife.28699.001

Introduction

“(T)he primary novelty of this work is the ability to make a prediction about drug sensitivity. Reviewers felt that the predictive ability would be very hard to generalize, however, reducing the impact of this novel feature. This concern about novelty… was the driving factor in this decision.”

-excerpt from a rejection letter received by the author

A mere 48 years separates the discovery of the double-helix structure of DNA (Watson and Crick, 1953) from the announcements that the human genome had been sequenced (Lander et al., 2001; Venter et al., 2001). The pace and regularity with which important discoveries have been made in molecular biology is remarkable. Molecular biologists have had an uncanny knack of homing in on the small irregularities that lead to large breakthroughs. It was irregularly colored ears of corn that revealed the existence of mobile genetic elements known as transposons (McClintock, 1950). Many of the most important regulators of human development first surfaced as mutations that slightly alter the rows of bristles on the undersides of fruit fly larvae (Nüsslein-Volhard and Wieschaus, 1980). Scientists studying tiny roundworms that age in odd ways helped uncover micro RNAs (Lee et al., 1993; Wightman et al., 1993), which are now thought to regulate a large fraction of human genes. Again and again molecular biologists have seized on these sorts of minutiae to gain enormous insight into the inner workings of cells. Looking back over the last 60 years one feels a great sense of pride in being part of a tradition that is undoubtedly one of the most productive in the history of science.

Given the winning formula molecular biologists appear to have hit on, it is interesting that there are large changes occurring in our community. As the size of the molecular biology community continues to grow, competition for limited funding has become much more intense. With the completion of the human genome has come immense pressure to “translate” basic research findings into new treatments for disease. In the United States our institutional leaders at the National Institutes of Health (NIH) openly worry about data showing that the rate of discovery in the biomedical sciences no longer reflects the size of their investments (Cook et al., 2015; Fortin and Currie, 2013; Gallo et al., 2014; Lauer et al., 2015; Doyle et al., 2015). Undoubtedly these pressures influence the trajectories of research programs. What we do not know yet is how these pressures impact the overall productivity of our community.

One manifestation of these changes is an increasing emphasis on “novelty” in science. Our scientific establishment – through our funding agencies, review panels and editorial boards – are clearly putting a higher and higher premium on research that is deemed novel. Research programs that lack a “high degree” of novelty struggle for support and “incremental” findings are relegated to publication in second- and third-tier journals. NIH grant proposals now have an “Innovation” section where investigators must explicitly list the attributes of their research that make it novel. While funding agencies seek novelty in their grant portfolios, they are also increasingly looking for "feasibility" as resources become scarce, and this appears to put novel research programs at a disadvantage (Alberts et al., 2014). As investigators struggle to walk a nearly impossible line between feasibility and novelty, the definition of novelty itself becomes blurred. Novelty can now mean anything from demonstrating a well-established phenomenon in a new system to testing a hypothesis with no precedent in the literature. Even though we cannot strictly define what is and is not novel, the message is still clear; novelty equates with good research.

Perhaps this emphasis on novelty is not really new at all, but only a codifying of something we already value implicitly. Even so, we should consider the effects that an explicit emphasis on novelty might have on the properties of scientific research that have made molecular biology so successful. These properties include our system of peer review, our scientific standards of proof and falsification, and the organization of the scientific community. Increasing the value we place on novelty will likely affect each of these factors.

Lessons from the philosophy of science

For working scientists Karl Popper is almost certainly the most influential philosopher of science. Most of us at least pay lip service to Popper’s philosophy when we recite the mantra that hypotheses can never be proved, only disproved. For many scientists the distinction between what is disprovable and what is not demarcates the line between what is and is not science, an idea taken directly from Popper’s writings. According to Popper, scientists propose new hypotheses about how the world works, and any hypotheses that are subsequently falsified by empirical observation are relegated to the scrap heap (Popper, 1963). This framework of hypothesis generation and refutation is widely accepted by scientists.

What is less well appreciated is how utterly Popper rejected the notion of confirmation. Popper was adamant that the survival of a hypothesis in the face of empirical challenge says nothing about its validity, only that that the hypothesis has yet to be falsified. However, Popper’s strict adherence to this idea became difficult to defend and, to be practical, most scientists do allow that empirical evidence can either support or falsify a hypothesis.

What if anything can we infer about the value of novelty from Popper's ideas on hypotheses and falsification? Because Popper believed that hypotheses can never be proved, he stressed that hypotheses must be subjected to repeated testing, even after they have survived several empirical challenges. In this sense he valued follow-through over novelty. However, because Popper believed that “good tests kill flawed theories”, new tests must be more than trivial variations of previous experiments. The philosopher Imre Lakatos argued that good research programs are "progressive" (Lakatos, 1970), and that scientists should constantly seek to expand their hypotheses into new areas of observation. Today, however, review panels are likely to tag progressive research programs as lacking in novelty because the scientists who pursue these programs seek to expand old hypotheses into new realms, rather than develop new hypotheses altogether. This is misguided. Scientists following progressive research programs require ingenuity and creativity to devise the tests that expand the reach of their hypotheses beyond the obvious. According to Popper the novelty of a new hypothesis is beside the point, unless and until the hypothesis it is meant to replace is falsified.

It appears then that nothing in the ideas of Popper or Kuhn particularly values novelty for its own sake.

Thomas Kuhn, a contemporary of Popper, was in many ways Popper’s opposite. Kuhn emphasized the importance of “paradigms”, coherent collections of claims, methodologies, and teaching practices that govern scientific inquiry. In his hugely influential book The Structure of Scientific Revolutions he explains that the purpose of a paradigm is to provide a guide for investigating the right questions (Kuhn and Hacking, 2012). Here Kuhn’s philosophy sharply contrasts with Popper’s. While Popper advocated abandoning a theory the moment it was falsified, Kuhn emphasized that paradigms can tolerate a good deal of “anomalies” and still remain valid. The flexibility of paradigms allows scientists to continue working in a productive framework long after falsification would have dictated a change. If scientists had to drop their paradigms every time they encountered a problem then nothing would ever get done. Only a critical mass of anomalies requires a “paradigm shift”.

It appears then that nothing in the ideas of Popper or Kuhn particularly values novelty for its own sake. Both Popper and Kuhn emphasized the need for scientists to stick doggedly with their hypotheses, Popper because hypotheses must be challenged continually no matter how often they have been confirmed, and Kuhn because only a critical mass of anomalies can force a paradigm shift. Ironically, over time the effect of Kuhn's book has been to weaken scientists’ belief in their paradigms. Many investigators now actively search for paradigm shifts. This conflicts with Kuhn’s description of progress in which scientists cling tightly to their paradigms, giving them up only grudgingly after the weight of anomalous results renders the paradigm unsupportable. Despite their differences, novelty seeking is not a key component in the philosophies of either Popper or Kuhn.

Many scientists have a visceral reaction to philosophies that cast them as mechanically pursuing their hypotheses. Kuhn in particular was attacked for seeming to endorse a grinding and boring type of science, and he did not help his case by referring to work done in the context of a paradigm as “normal” science.

But we need not explicitly value novelty to keep science from being a dull grind. Peter Godfrey-Smith writes that Popper painted an appealing picture of scientists as “hard-headed cowboys, out on the range, with a Stradivarius tucked in their saddlebags” (Godfrey-Smith, 2003). Hard-headed because they must have the determination to stick with their hypotheses, and packing a Stradivarius because they need inspiration when devising tests that expand their hypotheses into new realms. Kuhn too seemed in awe of the ability of normal science to hone in on “miniscule” findings that end up revealing deep truths about the world. Think of the little tails on the electron micrographs of the RNA:DNA hybrids that revealed the phenomenon of intron splicing (Berget et al., 1977), or the examples given at the start of this article. While normal science might seem a derogatory term for what most investigators do, Kuhn saw it as requiring imagination.

Even still, as working scientists we know that much of day-to-day science involves painstaking and often repetitive work. Science succeeds because powerful social incentives help us push through the less glamorous aspects of research. Godfrey-Smith writes that the most significant reactions to the philosophies of both Popper and Kuhn emphasized the importance of social forces in science. For example, in his later writings Popper struggled with the question of exactly when an observation counts as a refutation. His solution was to shift from describing the proper methodologies of science to describing the proper social behavior of scientists. For Kuhn, paradigms highlighted the importance of the social aspects of science, including the indoctrination of students and the collective adherence to particular claims among investigators working under the same paradigm. In the next section I discuss how the increasing emphasis on novelty might influence the social structure of science.

Lessons from the sociology of science

An important question for sociologists of science – and also for scientists and funding agencies – is: What distribution of people across rival research programs is best for science? The immediate impact of emphasizing novelty might be to distribute researchers over the widest possible range of research programs, as each investigator seeks to maximize the novelty of their own research program. This might seem an efficient way of exploring the widest possible range of theories but such a distribution also raises problems. Kuhn wrote extensively of the necessity of having large groups of researchers organized around a particular set of theories. Placing too much emphasis on novelty may result in a distribution of effort that is too diffuse to enable efficient progress. But scientists consider an array of incentives besides novelty when choosing their research programs.

Robert Merton laid the foundations of the sociology of science with his discussion of reward systems in science (Merton, 1957). Merton argued that recognition is the main form of reward in science. In particular the “priority rule”, which awards the most recognition to the first investigator to support a hypothesis, is an especially powerful incentive in science. To support his idea Merton showed that the history of science is chock full of disputes over priority (for example, Isaac Newton battled Gottfried Leibniz over priority for the invention of calculus (Hall, 1980)). One benefit of an incentive system that rewards priority is that it encourages original thought and novel lines of investigation. One might argue that this means that novelty seeking is already baked directly into the social fabric of science.

Hull viewed the success of science as a result of a delicate balance between competition and cooperation, creativity and skepticism, trust and doubt, and open-mindedness and dogmatism. Placing too much emphasis on novelty could upset this equilibrium in ways that are not optimal for scientific progress.

Some sociologists argue that the priority incentive coupled with the individual quest for credit is what produces good outcomes in the scientific community. These authors envision something like the “invisible hand” that guides free market capitalism in Adam Smith’s Wealth of Nations (Smith, 2000). Scientists must balance risk versus reward when choosing between competing hypotheses to explore. The priority incentive prevents all investigators from working on the hypothesis with the highest probability of success. The argument is that credit is a pie of fixed size that can be shared either equally (Kitcher, 1990) or unequally (Strevens, 2003), but only by investigators who work on the winning hypothesis. When too many scientists work on the same hypothesis there is an incentive to work on novel hypotheses, even ones where the chance of success might be smaller, but where the share of credit would be larger (Laudan, 1977). In this way the priority rule balances cooperation and competition between scientists, and divides individual effort between different research programs.

David Hull argued that science is particularly good at portioning effort in a way that maximizes good outcomes for the community (Hull, 1988). Hull agreed with Merton that the priority rule helps to maintain a balance between cooperation and competition in science. However, he also recognized the importance of the rivalries between scientists that encourage investigators to check the validity of their competitors’ work, especially results they may want to use in their own research. This checking, along with the priority rule, helps to maintain a balance between creativity and skepticism, which Hull believed was an essential feature of science. Scientists can become overly attached to their ideas, and most are reluctant to kill their pet theories, especially theories with creative panache. To counterbalance this tendency science relies on the incentive rival scientists have to vigorously check work that may be useful to them, or results that challenge their own dogma.

Hull might have been wary about introducing an explicit incentive for novelty into the scientific community. For one thing, along with most other sociologists of science, he thought that the priority incentive already provided a powerful motivation for scientists to test novel theories. But more than others Hull viewed the success of science as a result of a delicate balance between competition and cooperation, creativity and skepticism, trust and doubt, and open-mindedness and dogmatism. Placing too much emphasis on novelty could upset this equilibrium in ways that are not optimal for scientific progress.

In particular, an explicit emphasis on novelty might perturb the balance between the incentive for scientists to check their rivals’ theories and the priority rule. The priority rule provides a powerful incentive for scientists to publish their work quickly. This is good for the community because new ideas get disseminated rapidly, where they can be incorporated into other research programs. However, there is an equally powerful incentive to be correct when publishing because scientists know that other investigators who want to build on their results are likely to uncover any mistakes that make it into print. If we value novelty too much then scientists will be incentivized to publish too quickly, without imposing the rigor they might normally demand of themselves. Progress would slow to a crawl as other scientists waste time trying to build on flawed results.

Indeed, some in the scientific establishment have already warned of a “crisis in reproducibility” (Errington et al., 2014; Baker, 2016). Not surprisingly this crisis follows an explosion in papers reporting weak claims of novelty (Henikoff and Levis, 1991; Friedman and Karlsson, 1997). Others have argued that the reward system in modern molecular biology incentivizes statistically underpowered research designs (Higginson and Munafò, 2016). To counteract this trend some of the leaders in our field now advocate funding centralized efforts to validate published studies (Collins and Tabak, 2014). This suggests that priority and checking have become unbalanced in the general scientific community. Those leaders advocating for centralized checking efforts might do well to ask themselves what role their emphasis on novelty has played in precipitating this so-called crisis.

Another consequence of emphasizing novelty might be to increase the tenacity with which scientists attack their rivals’ hypotheses. Novel results are particularly likely to be attacked, in part because scientists who can lay claim to novelty enjoy so many advantages over other scientists. Rival scientists are thus incentivized to use anomalous results to discredit novel hypotheses. This is unfortunate because as Kuhn emphasized, hypotheses must be allowed to tolerate some anomalous results before they are discarded, otherwise the community cannot exploit the utility of working models. Ironically, novel research programs have a very difficult time surviving when novelty is so highly coveted.

Perhaps our obsession with novelty is a sort of communal nostalgia for the good old days, when important foundational discoveries came fast and furious.

An emphasis on novelty could also break the cohesion between scientists working within research programs. Cooperation is essential to scientific progress, and this cooperation is balanced by competition from investigators who are willing to challenge rival theories. If scientists must maximize the novelty of their research then they are more likely to pursue avenues as different as possible from their colleagues. We risk producing a community in which no single paradigm has the critical mass of supporters required to function effectively. This is a serious problem because current paradigms, imperfect though they might be, often have great utility, even though they may eventually be revised or even discarded.

Conclusions

When an area of science experiences rapid advancement over a short interval of time it may be followed by a period in which novel discoveries are harder to come by. After Mendeleyev articulated the concept of the periodic table there was an exciting period in which novel elements were rapidly discovered. As time passed it became more and more difficult to isolate the remaining elements. Perhaps molecular biology is also in a lull after a period of virtually unprecedented achievement. Almost 50 years ago Gunther Stent argued that there were no new principles left to discover in molecular biology (Stent, 1969). All that scientists could look forward to would be the tedious grind of filling in details. These sorts of pronouncements have a way of being undone by events. For example, Stent’s prediction came before the discovery of splicing, reverse transcription, and micro RNAs. Even so, it may well be true that most of the foundational principles of molecular biology have already been discovered. Perhaps our obsession with novelty is a sort of communal nostalgia for the good old days, when important foundational discoveries came fast and furious.

It might also be that our desire to reward novelty stems from the frustration that research in molecular biology is not “translating” into new practical applications as fast as some might wish. The endless overpromising of novel therapeutics from our institutional leaders only makes this matter worse. Why don’t discoveries in molecular biology translate more quickly into practical applications? Is it because we are missing large chunks of basic theory? Probably not, and those who go searching for novelty and paradigm shifts are likely to be disappointed.

Instead, we face a very different set of problems. While our models are generally quite good at explaining the basic mechanisms underlying molecular biology, it is also the case that most of our models lack a quantitative formulation. Even when we know the underlying molecular mechanisms at work in a given system or process, in most cases we lack the ability to make quantitative predictions about the effects that specific perturbations will have on that system or process. We have a mountain of facts about how transcription initiates and beautiful cartoon models of this process, but we cannot predict the effects that genetic variants will have on transcription rates, whether these variants reside in cis-acting DNA sequences or in trans-acting protein factors. We know the identities of virtually all the proteins involved in apoptosis, and which of their post-translational modifications are pro- or anti-apoptotic. Yet we cannot use quantitative measures of the levels of these proteins in any cell type to make an accurate prediction of whether that cell will die or not. We understand the principles that drive peptide sequences to fold into secondary and tertiary structures, yet we cannot predict the shape any given amino acid sequence will adopt. Seen through the lens of predictive power, it is clear that the vast majority of models in molecular biology are inadequate for solving real world problems.

If we want to solve important practical problems then progressive research programs that expand and refine the predictive power of existing models are at least as important as research programs focused on novel hypotheses. One suggestion would be to replace the current emphasis on novelty with an emphasis on predictive power, particularly quantitative predictions. Research that results in models that reliably and quantitatively predict the outcomes of genetic, biochemical, or pharmacological perturbations should be valued highly, and rewarded, regardless of whether such models invoke novel phenomena.

The increasing emphasis placed on novelty brings significant dangers. As it becomes more and more important for scientists to be “the first to demonstrate” some claim, the influence of the priority rule will increase and more scientists will feel pressure to sacrifice rigor for speed of publication. We are also likely to see an increase in distasteful disputes over priority. The cohesion between competing groups may also be in jeopardy as the drive for novelty distorts the balance between competition and cooperation that has characterized the success of molecular biology over the past several decades.

Science as we practice it today is a relatively recent development. Our system of peer review, the priority rule, and the organization of scientists into cooperative social demes that compete against other groups of scientists all trace their origin to decisions made by the Royal Society in the late 1600s. For most of history humans acquired knowledge outside of what we would recognize as a scientific framework. It would be unwise to assume that science is a permanent feature of our society or that it can withstand deep structural changes and remain an efficient engine of discovery. The explicit value we now place on novelty in molecular biology is a change we should approach with caution if we are to safeguard the essential features of science that have made our field so successful.

References

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
    Criticism and the Growth of Knowledge
    1. I Lakatos
    (1970)
    Falsification and the methodology of scientific research programmes, Criticism and the Growth of Knowledge, Cambridge, Cambridge University Press, 10.1017/CBO9781139171434.009.
  19. 19
    Initial sequencing and analysis of the human genome
    1. ES Lander
    2. LM Linton
    3. B Birren
    4. C Nusbaum
    5. MC Zody
    6. J Baldwin
    7. K Devon
    8. K Dewar
    9. M Doyle
    10. W FitzHugh
    11. R Funke
    12. D Gage
    13. K Harris
    14. A Heaford
    15. J Howland
    16. L Kann
    17. J Lehoczky
    18. R LeVine
    19. P McEwan
    20. K McKernan
    21. J Meldrim
    22. JP Mesirov
    23. C Miranda
    24. W Morris
    25. J Naylor
    26. C Raymond
    27. M Rosetti
    28. R Santos
    29. A Sheridan
    30. C Sougnez
    31. Y Stange-Thomann
    32. N Stojanovic
    33. A Subramanian
    34. D Wyman
    35. J Rogers
    36. J Sulston
    37. R Ainscough
    38. S Beck
    39. D Bentley
    40. J Burton
    41. C Clee
    42. N Carter
    43. A Coulson
    44. R Deadman
    45. P Deloukas
    46. A Dunham
    47. I Dunham
    48. R Durbin
    49. L French
    50. D Grafham
    51. S Gregory
    52. T Hubbard
    53. S Humphray
    54. A Hunt
    55. M Jones
    56. C Lloyd
    57. A McMurray
    58. L Matthews
    59. S Mercer
    60. S Milne
    61. JC Mullikin
    62. A Mungall
    63. R Plumb
    64. M Ross
    65. R Shownkeen
    66. S Sims
    67. RH Waterston
    68. RK Wilson
    69. LW Hillier
    70. JD McPherson
    71. MA Marra
    72. ER Mardis
    73. LA Fulton
    74. AT Chinwalla
    75. KH Pepin
    76. WR Gish
    77. SL Chissoe
    78. MC Wendl
    79. KD Delehaunty
    80. TL Miner
    81. A Delehaunty
    82. JB Kramer
    83. LL Cook
    84. RS Fulton
    85. DL Johnson
    86. PJ Minx
    87. SW Clifton
    88. T Hawkins
    89. E Branscomb
    90. P Predki
    91. P Richardson
    92. S Wenning
    93. T Slezak
    94. N Doggett
    95. JF Cheng
    96. A Olsen
    97. S Lucas
    98. C Elkin
    99. E Uberbacher
    100. M Frazier
    101. RA Gibbs
    102. DM Muzny
    103. SE Scherer
    104. JB Bouck
    105. EJ Sodergren
    106. KC Worley
    107. CM Rives
    108. JH Gorrell
    109. ML Metzker
    110. SL Naylor
    111. RS Kucherlapati
    112. DL Nelson
    113. GM Weinstock
    114. Y Sakaki
    115. A Fujiyama
    116. M Hattori
    117. T Yada
    118. A Toyoda
    119. T Itoh
    120. C Kawagoe
    121. H Watanabe
    122. Y Totoki
    123. T Taylor
    124. J Weissenbach
    125. R Heilig
    126. W Saurin
    127. F Artiguenave
    128. P Brottier
    129. T Bruls
    130. E Pelletier
    131. C Robert
    132. P Wincker
    133. DR Smith
    134. L Doucette-Stamm
    135. M Rubenfield
    136. K Weinstock
    137. HM Lee
    138. J Dubois
    139. A Rosenthal
    140. M Platzer
    141. G Nyakatura
    142. S Taudien
    143. A Rump
    144. H Yang
    145. J Yu
    146. J Wang
    147. G Huang
    148. J Gu
    149. L Hood
    150. L Rowen
    151. A Madan
    152. S Qin
    153. RW Davis
    154. NA Federspiel
    155. AP Abola
    156. MJ Proctor
    157. RM Myers
    158. J Schmutz
    159. M Dickson
    160. J Grimwood
    161. DR Cox
    162. MV Olson
    163. R Kaul
    164. C Raymond
    165. N Shimizu
    166. K Kawasaki
    167. S Minoshima
    168. GA Evans
    169. M Athanasiou
    170. R Schultz
    171. BA Roe
    172. F Chen
    173. H Pan
    174. J Ramser
    175. H Lehrach
    176. R Reinhardt
    177. WR McCombie
    178. M de la Bastide
    179. N Dedhia
    180. H Blöcker
    181. K Hornischer
    182. G Nordsiek
    183. R Agarwala
    184. L Aravind
    185. JA Bailey
    186. A Bateman
    187. S Batzoglou
    188. E Birney
    189. P Bork
    190. DG Brown
    191. CB Burge
    192. L Cerutti
    193. HC Chen
    194. D Church
    195. M Clamp
    196. RR Copley
    197. T Doerks
    198. SR Eddy
    199. EE Eichler
    200. TS Furey
    201. J Galagan
    202. JG Gilbert
    203. C Harmon
    204. Y Hayashizaki
    205. D Haussler
    206. H Hermjakob
    207. K Hokamp
    208. W Jang
    209. LS Johnson
    210. TA Jones
    211. S Kasif
    212. A Kaspryzk
    213. S Kennedy
    214. WJ Kent
    215. P Kitts
    216. EV Koonin
    217. I Korf
    218. D Kulp
    219. D Lancet
    220. TM Lowe
    221. A McLysaght
    222. T Mikkelsen
    223. JV Moran
    224. N Mulder
    225. VJ Pollara
    226. CP Ponting
    227. G Schuler
    228. J Schultz
    229. G Slater
    230. AF Smit
    231. E Stupka
    232. J Szustakowki
    233. D Thierry-Mieg
    234. J Thierry-Mieg
    235. L Wagner
    236. J Wallis
    237. R Wheeler
    238. A Williams
    239. YI Wolf
    240. KH Wolfe
    241. SP Yang
    242. RF Yeh
    243. F Collins
    244. MS Guyer
    245. J Peterson
    246. A Felsenfeld
    247. KA Wetterstrand
    248. A Patrinos
    249. MJ Morgan
    250. P de Jong
    251. JJ Catanese
    252. K Osoegawa
    253. H Shizuya
    254. S Choi
    255. YJ Chen
    256. J Szustakowki
    257. International Human Genome Sequencing Consortium
    (2001)
    Nature 409:860–921.
    https://doi.org/10.1038/35057062
  20. 20
    Progress and Its Problems: Toward a Theory of Scientific Growth
    1. L Laudan
    (1977)
    Berkeley: University of California Press.
  21. 21
  22. 22
  23. 23
  24. 24
    The Sociology of Science: Theoretical and Empirical Investigations
    1. RK Merton
    (1957)
    Priorities in scientific discovery, The Sociology of Science: Theoretical and Empirical Investigations, Chicago, University of Chicago Press.
  25. 25
  26. 26
    Conjectures and Refutations: The Growth of Scientific Knowledge
    1. KR Popper
    (1963)
    London and New York: Routledge & Kegan Paul.
  27. 27
    The Wealth of Nations
    1. A Smith
    (2000)
    New York: Modern Library.
  28. 28
    The Coming of the Golden Age: A View of the End of Progress
    1. GS Stent
    (1969)
    New York: The Natural History Press.
  29. 29
  30. 30
    The sequence of the human genome
    1. JC Venter
    2. MD Adams
    3. EW Myers
    4. PW Li
    5. RJ Mural
    6. GG Sutton
    7. HO Smith
    8. M Yandell
    9. CA Evans
    10. RA Holt
    11. JD Gocayne
    12. P Amanatides
    13. RM Ballew
    14. DH Huson
    15. JR Wortman
    16. Q Zhang
    17. CD Kodira
    18. XH Zheng
    19. L Chen
    20. M Skupski
    21. G Subramanian
    22. PD Thomas
    23. J Zhang
    24. GL Gabor Miklos
    25. C Nelson
    26. S Broder
    27. AG Clark
    28. J Nadeau
    29. VA McKusick
    30. N Zinder
    31. AJ Levine
    32. RJ Roberts
    33. M Simon
    34. C Slayman
    35. M Hunkapiller
    36. R Bolanos
    37. A Delcher
    38. I Dew
    39. D Fasulo
    40. M Flanigan
    41. L Florea
    42. A Halpern
    43. S Hannenhalli
    44. S Kravitz
    45. S Levy
    46. C Mobarry
    47. K Reinert
    48. K Remington
    49. J Abu-Threideh
    50. E Beasley
    51. K Biddick
    52. V Bonazzi
    53. R Brandon
    54. M Cargill
    55. I Chandramouliswaran
    56. R Charlab
    57. K Chaturvedi
    58. Z Deng
    59. V Di Francesco
    60. P Dunn
    61. K Eilbeck
    62. C Evangelista
    63. AE Gabrielian
    64. W Gan
    65. W Ge
    66. F Gong
    67. Z Gu
    68. P Guan
    69. TJ Heiman
    70. ME Higgins
    71. RR Ji
    72. Z Ke
    73. KA Ketchum
    74. Z Lai
    75. Y Lei
    76. Z Li
    77. J Li
    78. Y Liang
    79. X Lin
    80. F Lu
    81. GV Merkulov
    82. N Milshina
    83. HM Moore
    84. AK Naik
    85. VA Narayan
    86. B Neelam
    87. D Nusskern
    88. DB Rusch
    89. S Salzberg
    90. W Shao
    91. B Shue
    92. J Sun
    93. Z Wang
    94. A Wang
    95. X Wang
    96. J Wang
    97. M Wei
    98. R Wides
    99. C Xiao
    100. C Yan
    101. A Yao
    102. J Ye
    103. M Zhan
    104. W Zhang
    105. H Zhang
    106. Q Zhao
    107. L Zheng
    108. F Zhong
    109. W Zhong
    110. S Zhu
    111. S Zhao
    112. D Gilbert
    113. S Baumhueter
    114. G Spier
    115. C Carter
    116. A Cravchik
    117. T Woodage
    118. F Ali
    119. H An
    120. A Awe
    121. D Baldwin
    122. H Baden
    123. M Barnstead
    124. I Barrow
    125. K Beeson
    126. D Busam
    127. A Carver
    128. A Center
    129. ML Cheng
    130. L Curry
    131. S Danaher
    132. L Davenport
    133. R Desilets
    134. S Dietz
    135. K Dodson
    136. L Doup
    137. S Ferriera
    138. N Garg
    139. A Gluecksmann
    140. B Hart
    141. J Haynes
    142. C Haynes
    143. C Heiner
    144. S Hladun
    145. D Hostin
    146. J Houck
    147. T Howland
    148. C Ibegwam
    149. J Johnson
    150. F Kalush
    151. L Kline
    152. S Koduru
    153. A Love
    154. F Mann
    155. D May
    156. S McCawley
    157. T McIntosh
    158. I McMullen
    159. M Moy
    160. L Moy
    161. B Murphy
    162. K Nelson
    163. C Pfannkoch
    164. E Pratts
    165. V Puri
    166. H Qureshi
    167. M Reardon
    168. R Rodriguez
    169. YH Rogers
    170. D Romblad
    171. B Ruhfel
    172. R Scott
    173. C Sitter
    174. M Smallwood
    175. E Stewart
    176. R Strong
    177. E Suh
    178. R Thomas
    179. NN Tint
    180. S Tse
    181. C Vech
    182. G Wang
    183. J Wetter
    184. S Williams
    185. M Williams
    186. S Windsor
    187. E Winn-Deen
    188. K Wolfe
    189. J Zaveri
    190. K Zaveri
    191. JF Abril
    192. R Guigó
    193. MJ Campbell
    194. KV Sjolander
    195. B Karlak
    196. A Kejariwal
    197. H Mi
    198. B Lazareva
    199. T Hatton
    200. A Narechania
    201. K Diemer
    202. A Muruganujan
    203. N Guo
    204. S Sato
    205. V Bafna
    206. S Istrail
    207. R Lippert
    208. R Schwartz
    209. B Walenz
    210. S Yooseph
    211. D Allen
    212. A Basu
    213. J Baxendale
    214. L Blick
    215. M Caminha
    216. J Carnes-Stine
    217. P Caulk
    218. YH Chiang
    219. M Coyne
    220. C Dahlke
    221. A Mays
    222. M Dombroski
    223. M Donnelly
    224. D Ely
    225. S Esparham
    226. C Fosler
    227. H Gire
    228. S Glanowski
    229. K Glasser
    230. A Glodek
    231. M Gorokhov
    232. K Graham
    233. B Gropman
    234. M Harris
    235. J Heil
    236. S Henderson
    237. J Hoover
    238. D Jennings
    239. C Jordan
    240. J Jordan
    241. J Kasha
    242. L Kagan
    243. C Kraft
    244. A Levitsky
    245. M Lewis
    246. X Liu
    247. J Lopez
    248. D Ma
    249. W Majoros
    250. J McDaniel
    251. S Murphy
    252. M Newman
    253. T Nguyen
    254. N Nguyen
    255. M Nodell
    256. S Pan
    257. J Peck
    258. M Peterson
    259. W Rowe
    260. R Sanders
    261. J Scott
    262. M Simpson
    263. T Smith
    264. A Sprague
    265. T Stockwell
    266. R Turner
    267. E Venter
    268. M Wang
    269. M Wen
    270. D Wu
    271. M Wu
    272. A Xia
    273. A Zandieh
    274. X Zhu
    (2001)
    Science 291:1304–1351.
    https://doi.org/10.1126/science.1058040
  31. 31
  32. 32

Decision letter

  1. Peter Rodgers
    Reviewing Editor; eLife, United Kingdom

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your manuscript "How should novelty be valued in science?" to eLife for consideration as a Feature Article. Your manuscript has been reviewed by two peer reviewers and the eLife Features Editor (Peter Rodgers). The following individuals involved in review of your submission have agreed to reveal their identity: Yitzhak Pilpel (Reviewer #1) and Angela H DePace (Reviewer #2).

The reviewers have discussed the reviews with one another and the Features Editor has drafted this decision to help you prepare a revised submission. Most of the major revisions requested are optional (we feel the article would be improved if you addressed them, but it is not essential that you do).

Summary:

The paper is an impressive scholarly work. It is broad, deep and methodological. It is very well written (though perhaps could be shortened). It studies the value of novelty in science through several angles, including philosophy of science (the excellent survey and comparison of Popper's vs. Kuhn's teachings as well as other less well-known thinkers is used here very effectively to deliver the notion that both falsification as well as paradigm establishment and shifting require more than purely "novelty-science"); it considers very effectively social and cultural aspects of science (the role of fame and recognition in the process, competition etc.); it touches upon the emotional aspects of doing science, and it very effectively also touches upon science organization and policy aspects such as in funding and granting of research projects (where the call for funding, not only individualistic research is refreshing and, in a way novel, in the current atmosphere).

Major revisions:

1) The solution presented at the end (to focus on quantitative prediction as a gauge of novelty) is only one of many possible solutions, and it would be good if the author could discuss other possible solutions, although we should not insist on this.

I would argue that another solution would be including some description of the sociology of science in graduate and undergraduate education, such that the value of novelty and reproducibility/extension at the community level are more clear to people. Right now we almost exclusively lift up isolated geniuses as scientific heroes; is it no wonder that everyone chases some paradigm shift of their own? I'm sure there are other solutions as well.

2) A common complaint I hear is that the competitive nature of modern science means that authors often over-sell their findings in papers in order make them seem more novel than they really are. Again, it would be good if the author could briefly discuss this phenomenon.

3) In addition to the relationship between novelty and philosophical and sociological factors it would be good to discuss how competition for funding and jobs seems to be reducing novelty – as outlined, for example, in the following passage from Alberts et al. 2014. Rescuing US Biomedical Research from its Systemic Flaws. PNAS 111:5773-5777:

"Competition in pursuit of experimental objectives has always been a part of the scientific enterprise, and it can have positive effects. However, hypercompetition for the resources and positions that are required to conduct science suppresses the creativity, cooperation, risk-taking, and original thinking required to make fundamental discoveries.

Now that the percentage of NIH grant applications that can be funded has fallen from around 30% into the low teens, biomedical scientists are spending far too much of their time writing and revising grant applications and far too little thinking about science and conducting experiments. The low success rates have induced conservative, short-term thinking in applicants, reviewers, and funders. The system now favors those who can guarantee results rather than those with potentially path-breaking ideas that, by definition, cannot promise success. Young investigators are discouraged from departing too far from their postdoctoral work, when they should instead be posing new questions and inventing new approaches. Seasoned investigators are inclined to stick to their tried-and-true formulas for success rather than explore new fields.

One manifestation of this shift to short-term thinking is the inflated value that is now accorded to studies that claim a close link to medical practice […]".

It would be good to discuss these matters (in just a paragraph or two) in part 1 or part 4 of the article, but this is not essential.

4) I would consider swapping the order of sections 2 and 3. Section 3 is the stronger of the two, in my opinion, and describes one ideal version of how the scientific community functions that many of us are familiar with, at least in the abstract. It thus may serve as more of a common starting point. (Although it may be worth noting that some aspects of this ideal might not serve us well either. For example it is highly individualistic and competitive in its framing; the same goals of novelty seeking and cross-checking might be achieved by other more collaborative social structures). The segue to section 2 can then be that novelty-seeking is a requirement of the social structure described in the previous section, as is independently validating or extending results in new areas. Both of these activities can be accommodated in the philosophical frameworks presented, but there is a clear second-tier status assigned to validating or extending results in some of them. Thus the dominant influence of Kuhn's work can be seen to be somewhat destructive in the overall goals of science. (Everyone constantly seeking poorly-defined paradigm shifts isn't necessarily productive).

https://doi.org/10.7554/eLife.28699.002

Author response

As directed in the decision letter I have addressed some, but not all, of the major points as the letter indicated that addressing these points was optional.

Major revisions:

1) The solution presented at the end (to focus on quantitative prediction as a gauge of novelty) is only one of many possible solutions, and it would be good if the author could discuss other possible solutions, although we should not insist on this.

I would argue that another solution would be including some description of the sociology of science in graduate and undergraduate education, such that the value of novelty and reproducibility/extension at the community level are more clear to people. Right now we almost exclusively lift up isolated geniuses as scientific heroes; is it no wonder that everyone chases some paradigm shift of their own? I'm sure there are other solutions as well.

2) A common complaint I hear is that the competitive nature of modern science means that authors often over-sell their findings in papers in order make them seem more novel than they really are. Again, it would be good if the author could briefly discuss this phenomenon.

This point is addressed in the ninth paragraph of the section “Lessons from the sociology of science”. I cite to papers documenting the exponential rise in claims to novelty.

3) In addition to the relationship between novelty and philosophical and sociological factors it would be good to discuss how competition for funding and jobs seems to be reducing novelty – as outlined, for example, in the following passage from Alberts et al. 2014. Rescuing US Biomedical Research from its Systemic Flaws. PNAS 111:5773-5777:

"Competition in pursuit of experimental objectives has always been a part of the scientific enterprise, and it can have positive effects. However, hypercompetition for the resources and positions that are required to conduct science suppresses the creativity, cooperation, risk-taking, and original thinking required to make fundamental discoveries.

Now that the percentage of NIH grant applications that can be funded has fallen from around 30% into the low teens, biomedical scientists are spending far too much of their time writing and revising grant applications and far too little thinking about science and conducting experiments. The low success rates have induced conservative, short-term thinking in applicants, reviewers, and funders. The system now favors those who can guarantee results rather than those with potentially path-breaking ideas that, by definition, cannot promise success. Young investigators are discouraged from departing too far from their postdoctoral work, when they should instead be posing new questions and inventing new approaches. Seasoned investigators are inclined to stick to their tried-and-true formulas for success rather than explore new fields.

One manifestation of this shift to short-term thinking is the inflated value that is now accorded to studies that claim a close link to medical practice […]".

It would be good to discuss these matters (in just a paragraph or two) in part 1 or part 4 of the article, but this is not essential.

I now address this point in the Introduction (fourth paragraph) and cite the Alberts et al. (2014) paper.

https://doi.org/10.7554/eLife.28699.003

Article and author information

Author details

  1. Barak A Cohen

    Edison Family Center for Genome Sciences and Systems Biology and Department of Genetics, Washington University School of Medicine, Saint Louis, United States
    Contribution
    BAC, Conceptualization, Writing—original draft, Writing—review and editing
    For correspondence
    cohen@wustl.edu
    Competing interests
    The author declares that no competing interests exist.
    ORCID icon 0000-0002-3350-2715

Acknowledgements

I thank Rob Mitra, Mark Johnston, Siqi Zhao, Max Staller, Michael White, Zach Pincus, and Dana King for critical readings of the manuscripts and engaging discussions.

Reviewing Editor

  1. Peter Rodgers, Reviewing Editor, eLife, United Kingdom

Publication history

  1. Received: May 17, 2017
  2. Accepted: July 11, 2017
  3. Version of Record published: July 25, 2017 (version 1)

Copyright

© 2017, Cohen

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 4,511
    Page views
  • 465
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: PubMed Central, Scopus, Crossref.

Comments

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading