Studies using animal models are important for understanding the physiopathological and therapeutic basis of human diseases. However, the translation of scientific findings from animal models to humans is far from straightforward, and more than 80% of potential therapeutics fail in human clinical trials after being successful in animal models (Perrin, 2014). There is a clear need, therefore, for animal research to become more reliable and reproducible (van der Worp et al., 2010).

The failure to translate from animal models to humans stems from various factors, and in recent years there have been growing concerns over the lack of reproducibility of results in certain areas of biomedical research (Begley and Ellis, 2012; Prinz et al., 2011; Collins and Tabak, 2014). One factor contributing to this observed lack of reproducibility may be inadequate reporting of experimental methods and materials (Landis et al., 2012; Moher et al., 2008; van der Worp and Macleod, 2011). It has been estimated that spending on preclinical research that is not reproducible amounts to $28 billion per year in the United States (Freedman et al., 2015).

Guidelines from the International Committee of Medical Journal Editors (ICMJE) state that the methods section of a paper "should aim to be sufficiently detailed such that others with access to the data would be able to reproduce the results" (ICMJE, 2013). In experiments using animals, for instance, the sex and age of the mice should be reported because they affect morphological, physiological, immunological and behavioral parameters and, hence, they influence the outcomes of experiments (Diedrich et al., 2007; Wizemann and Pardue, 2001). Sex and age are inextricably linked: it has been proposed that under natural conditions sexual selection has profound effects on the lifespan of organisms (Bale and Epperson, 2015; Maklakov and Lummaa, 2013). Considering some taxa exceptions, the general conclusion is that in many animals (including humans), males have shorter lifespans than females (Clutton-Brock and Isvaran, 2007). Furthermore, from an evolutionary standpoint, these sex differences in lifespan depends to a great extent on sexually dimorphic life-history strategies (such as mating systems; Maklakov and Lummaa, 2013), and on genetic architecture, including both the sex chromosomes (Nguyen and Disteche, 2006) and the mitochondrial DNA (Gemmell et al., 2004).

Regarding preclinical and clinical studies, sex and age play key roles in disease phenotypes, modifying their susceptibility, presentation and response to treatment (Arnold, 2010). Some pathologies exhibit a clear sexual dimorphism (Ober et al., 2008). Using stroke as an example, it is known that its incidence is higher in men than women during their lifespan (Mozaffarian et al., 2015). However, recent evidence suggests that after the age of 60 years and thus post-menopause, women have more severe strokes than men (Dehlendorff et al., 2015). In the case of animal models, sex- and age-dependent differences in protein expression profiles were observed in the heart proteome of female and male C57BL/6 mice of two distinct age groups (14 and 100 weeks) (Diedrich et al., 2007). This evidence implies that sex differences must be studied across the entire lifespan in order to bring new insights into the pathogenesis of the diseases and identify targets for new drugs for both sexes and different times of life.

Guidelines, such as ARRIVE (Animal Research: Reporting In Vivo Experiments; Kilkenny et al., 2010), have been developed to improve the reporting of animal-based research. However, although these guidelines have been endorsed by many journals, not all papers in these journals comply with the ARRIVE guidelines (Baker et al., 2014). Recent years have also seen more emphasis being placed on the need for animal-based experiments to comply with the principles of the 3Rs (Replacement, Reduction and Refinement; Burden et al., 2015).

In this study, we have used large-scale text mining to evaluate the reporting of information about the sex and age of the mice used in a set of over 15,000 articles. In the last decade, there has been a significant amount of research in the identification of targeted biomedical information in the scientific literature via text mining (Cohen and Hersh, 2005; Fleuren and Alkema, 2015). In particular, efforts have been made to recognize protein and gene names in text (Settles, 2005) or other biomedical entities of interest such as electronic health records (Meystre et al., 2008). Our approach differs from this work in that it addresses a significantly more diverse literature space while focusing on what should be the standard information in a paper about animal-based biomedical research.

Previously we have shown that important experimental details are repeatedly omitted from papers in parasitology (Florez-Vargas et al., 2014) and in studies of colitis (Bramhall et al., 2015). Here we use syntactic rules and simple dictionary matching to extract key characteristics (such as sex and age) from papers reporting the results of experiments on mice. We investigate questions of whether the sex and age of mice are reported, the use of animals of each sex in six different areas of preclinical research, and the use of animals of each sex in four subgroups (genetics, immunology, physiopathology and therapy) for these six areas.

Whilst this work constitutes the largest analysis of the reporting of sex and age data for mouse-based research to date, our sample does not represent the entire biomedical literature: not all journals are found in the PubMed Central Open Access subset, and some of the journals that deposit their contents into PubMed Central include only some of their articles in the Open Access subset. This is undoubtedly a limitation of our study.

We selected the mouse as a model because it is probably the most comprehensive and well-characterized model in the life sciences. Researchers rely on mouse models to mimic human disease conditions for several reasons. One of the main reasons is that mouse and human genomes are genetically similar – about 90% of human genes have direct orthologues with mice (Yue et al., 2014). Moreover, as animal models, mice are convenient due to their small size, short lifespan (up to two years), and quick generation time; three weeks for gestation and from 6 to 8 weeks to reach sexual maturity. Therefore, they can be easily housed and maintained, can be genetically manipulated to define gene function in a whole body system and a large number of mice can be studied in a relatively short period of time. This, for instance, allows scientists to study cell/cell interactions in the tissue environment and thus cause and effect relationships in a controlled situation.

Despite the implications for interpretation and reproducibility of experimental findings, the sex and age of the experimental subjects are often not recorded in scientific reports (Kilkenny et al., 2009). In agreement with previous reports, the evidence presented in this study showed that the lack of reporting of key methodological parameters in mouse experiments is still a cause of concern; only about half of the papers published in 2014 stated both sex and age of the mice as experimental variables (Figure 1c). The reason why these variables are not described is unclear, since this simple information is always available to researchers. We do not believe it is a space issue, because it is possible to describe them, including mice number and mouse strain, in about 40 characters of text (e.g. ten C57BL/6 female mice (6-8-weeks old)). Whilst an improvement in the reporting of mouse sex and age has been observed over time, this is not solely attributable to the introduction of journal guidelines, because this improvement started before the publication of the ARRIVE guidelines in 2010 (Kilkenny et al., 2010). In fact, a follow-up study in 2012 showed that while sex and age reporting had improved post-ARRIVE, journals that enforced the ARRIVE guidelines as a condition of publication still failed to publish sex and age in all cases (Baker et al., 2014). The observed improvements may also therefore be a result of a growing recognition of the importance of sex and age as experimental factors that may affect study outcomes.

An analysis of the scientific literature leads to the general conclusion that the males in both human and other animals are studied much more than their female counterparts. This conclusion is based mainly on the results of two studies that manually surveyed a set of biomedical articles (Beery and Zucker, 2011; Taylor et al., 2011). However, our results showed otherwise in mouse-based models: 31.84% and 23.38% of all papers assessed were on studies performed on female and male mice, respectively (Figure 1d). This could be explained by some practical advantages of using female rather than male mice: they are cheaper to house; they are less aggressive to each other and to experimenters; and they are smaller, requiring less weight-administered drug. In addition, the apparent contradiction between this observation and the previous reports might be related to the sample size and study design; our sample size was larger and we surveyed a much broader range of disciplines. In addition, although rodent models (that is, mouse and rat models) accounted for, respectively, 50% and 80% of the papers considered in these other studies (Beery and Zucker, 2011; Taylor et al., 2011), they also included other species, such as cat, dog and monkey models: moreover, these two studies did not explore sex bias by species.

In preclinical studies, furthermore, we noted an important sex- and age-bias in mouse-based disease models (Figure 2b and 2c). Among the main preclinical research topics assessed, we observed the strongest male-bias in cardiovascular disease models (2.25:1) and the strongest female-bias in infectious disease models (3.54:1) (Figure 2b). This situation still persists: between 2012 and 2014, about 70% and 77% of research articles assessed on these two disease models are still biased towards male and female mice, respectively. These pathologies and many others, exhibit important sexual dimorphisms, which are not only inherent to genetic differences, but also to hormonal influence (Case et al., 2013; Gilks et al., 2014). For example, in the study of hypertension, one of the major risk factors for cardiovascular disease, a greater increase in blood pressure was reported in gonad-intact XY males than XX females using the four core genotype in the MF1 mouse model. However, the mean arterial pressure was greater in gonadectomized XX mice compared with XY mice regardless of whether the mice were born with testes or with ovaries (Ji et al., 2010). On the other hand, in the case of infectious diseases, females have a more robust immune system than males – both the innate and adaptive immune responses, which makes them less susceptible to developing many infections (mainly Th1-type infections), although it increases the risk of developing autoimmune diseases due to their trend to develop a stronger pro-inflammatory response (Pennell et al., 2012). Interestingly, we also observed that the sex-bias could change in a particular disease mouse model according to the biomedical study. Diabetes disease mouse models exemplified this situation. From a global point of view, this disease was found to be male-biased (1.57:1) (Figure 2b). However, in studies related with the immunology of diabetes, there was a strong female-bias (7.87:1) (Figure 3b); a change that remains in the study of diabetes mellitus type 2 (Figure 4).

In order to balance sex of animals and cells in preclinical studies, the National Institutes of Health (NIH) have proposed a multi-dimensional initiative, which includes, among other things, extramural training on experimental design and data analysis by sex (Clayton and Collins, 2014). Regarding this initiative, new ideas have been proposed to achieve, and sustain, the sex balance in biomedical research (McCullough et al., 2014). In this context, our study provides an implementation of text mining to assess reporting of experimental factors. By knowing where there is imbalance for a particular variable, it is possible to address it in a cost-effective manner. This not only directly contributes to the comparability of experimental work, but also to the reproducibility of findings. To address this problem some journals are already introducing editorial measures and methods checklists in order to improve the quality of scientific reporting (Nature, 2013). Nevertheless, whilst journal checklists may make reference to species strain, sex and age of animals, the main focus is on statistical issues to ensure repeatability, rather than biological factors that modify the outcomes.

We hope that our text mining strategy can be used to explore other aspects of how experimental methods and materials are reported in the literature, and thus contribute to efforts to improve the reproducibility of biomedical research. It might also be possible to employ text mining to pre-screen manuscripts when they are submitted to journals.

1. 1. Ananiadou S
   2. Kell DB
   3. Tsujii J

   (2006) Text mining and its potential applications in systems biology

   Trends in Biotechnology 24:571–579.

   https://doi.org/10.1016/j.tibtech.2006.10.002

   • Google Scholar
2. 1. Arbogast P
   2. Glösmann M
   3. Peichl L

   (2013) Retinal cone photoreceptors of the deer mouse peromyscus maniculatus: development, topography, opsin expression and spectral tuning

   PLoS ONE 8:e80910.

   https://doi.org/10.1371/journal.pone.0080910

   • Google Scholar
3. 1. Arnold AP

   (2010) Promoting the understanding of sex differences to enhance equity and excellence in biomedical science

   Biology of Sex Differences 1:1.

   https://doi.org/10.1186/2042-6410-1-1

   • Google Scholar
4. 1. Baker D
   2. Lidster K
   3. Sottomayor A
   4. Amor S

   (2014) Two years later: journals are not yet enforcing the ARRIVE guidelines on reporting standards for pre-clinical animal studies

   PLoS Biology 12:e1001756.

   https://doi.org/10.1371/journal.pbio.1001756

   • Google Scholar
5. 1. Bale TL
   2. Epperson CN

   (2015) Sex differences and stress across the lifespan

   Nature Neuroscience 18:1413–1420.

   https://doi.org/10.1038/nn.4112

   • Google Scholar
6. 1. Beery AK
   2. Zucker I

   (2011) Sex bias in neuroscience and biomedical research

   Neuroscience and Biobehavioral Reviews 35:565–572.

   https://doi.org/10.1016/j.neubiorev.2010.07.002

   • Google Scholar
7. 1. Begley CG
   2. Ellis LM

   (2012) Drug development: Raise standards for preclinical cancer research

   Nature 483:531–533.

   https://doi.org/10.1038/483531a

   • Google Scholar
8. 1. Bramhall M
   2. Flórez-Vargas O
   3. Stevens R
   4. Brass A
   5. Cruickshank S

   (2015) Quality of methods reporting in animal models of colitis

   Inflammatory Bowel Diseases 21:1248–1259.

   https://doi.org/10.1097/MIB.0000000000000369

   • Google Scholar
9. 1. Burden N
   2. Chapman K
   3. Sewell F
   4. Robinson V

   (2015)

   Pioneering better science through the 3Rs: an introduction to the national centre for the replacement, refinement, and reduction of animals in research (NC3Rs)

   Journal of the American Association for Laboratory Animal Science : JAALAS 54:198–208.

   • Google Scholar
10. 1. Case LK
    2. Wall EH
    3. Dragon JA
    4. Saligrama N
    5. Krementsov DN
    6. Moussawi M
    7. Zachary JF
    8. Huber SA
    9. Blankenhorn EP
    10. Teuscher C

    (2013) The Y chromosome as a regulatory element shaping immune cell transcriptomes and susceptibility to autoimmune disease

    Genome Research 23:1474–1485.

    https://doi.org/10.1101/gr.156703.113

    • Google Scholar
11. 1. Clayton JA
    2. Collins FS

    (2014) Policy: NIH to balance sex in cell and animal studies

    Nature 509:282–283.

    https://doi.org/10.1038/509282a

    • Google Scholar
12. 1. Clutton-Brock TH
    2. Isvaran K

    (2007) Sex differences in ageing in natural populations of vertebrates

    Proceedings of the Royal Society B: Biological Sciences 274:3097–3104.

    https://doi.org/10.1098/rspb.2007.1138

    • Google Scholar
13. 1. Cohen AM
    2. Hersh WR

    (2005) A survey of current work in biomedical text mining

    Briefings in Bioinformatics 6:57–71.

    https://doi.org/10.1093/bib/6.1.57

    • Google Scholar
14. 1. Collins FS
    2. Tabak LA

    (2014) Policy: NIH plans to enhance reproducibility

    Nature 505:612–613.

    https://doi.org/10.1038/505612a

    • Google Scholar
15. 1. Cunningham H
    2. Tablan V
    3. Roberts A
    4. Bontcheva K

    (2013) Getting more out of biomedical documents with GATE's full lifecycle open source text analytics

    PLoS Computational Biology 9:e1002854.

    https://doi.org/10.1371/journal.pcbi.1002854

    • Google Scholar
16. 1. Dehlendorff C
    2. Andersen KK
    3. Olsen TS

    (2015) Sex disparities in stroke: women have more severe strokes but better survival than men

    Journal of the American Heart Association 4:.

    https://doi.org/10.1161/JAHA.115.001967

    • Google Scholar
17. 1. Diedrich M
    2. Tadic J
    3. Mao L
    4. Wacker MA
    5. Nebrich G
    6. Hetzer R
    7. Regitz-Zagrosek V
    8. Klose J

    (2007) Heart protein expression related to age and sex in mice and humans

    International Journal of Molecular Medicine 20:865–874.

    https://doi.org/10.3892/ijmm.20.6.865

    • Google Scholar
18. 1. Fleuren WW
    2. Alkema W

    (2015) Application of text mining in the biomedical domain

    Methods 74:97–106.

    https://doi.org/10.1016/j.ymeth.2015.01.015

    • Google Scholar
19. 1. Flórez-Vargas O
    2. Bramhall M
    3. Noyes H
    4. Cruickshank S
    5. Stevens R
    6. Brass A

    (2014) The quality of methods reporting in parasitology experiments

    PloS One 9:e101131.

    https://doi.org/10.1371/journal.pone.0101131

    • Google Scholar
20. 1. Freedman LP
    2. Cockburn IM
    3. Simcoe TS

    (2015) The Economics of Reproducibility in Preclinical Research

    PLoS Biology 13:e1002165.

    https://doi.org/10.1371/journal.pbio.1002165

    • Google Scholar
21. 1. Gemmell NJ
    2. Metcalf VJ
    3. Allendorf FW

    (2004) Mother's curse: the effect of mtDNA on individual fitness and population viability

    Trends in Ecology & Evolution 19:238–244.

    https://doi.org/10.1016/j.tree.2004.02.002

    • Google Scholar
22. 1. Gilks WP
    2. Abbott JK
    3. Morrow EH

    (2014) Sex differences in disease genetics: evidence, evolution, and detection

    Trends in Genetics 30:453–463.

    https://doi.org/10.1016/j.tig.2014.08.006

    • Google Scholar
23. 1. Herlyn M
    2. Fukunaga-Kalabis M

    (2010) What is a good model for melanoma?

    Journal of Investigative Dermatology 130:911–912.

    https://doi.org/10.1038/jid.2009.441

    • Google Scholar
24. 1. Hotho A
    2. Numberger A
    3. Paab G

    (2005)

    A brief survey of text mining

    LDV Forum – GLDV Journal for Computational Linguistics and Language Technology 20:19–62.

    • Google Scholar
25. Website

    1. ICMJE

    (2013) Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publications

    http://www.icmje.org/icmje-recommendations.pdf
26. 1. Ji H
    2. Zheng W
    3. Wu X
    4. Liu J
    5. Ecelbarger CM
    6. Watkins R
    7. Arnold AP
    8. Sandberg K

    (2010) Sex chromosome effects unmasked in angiotensin II-induced hypertension

    Hypertension 55:1275–1282.

    https://doi.org/10.1161/HYPERTENSIONAHA.109.144949

    • Google Scholar
27. 1. Kilkenny C
    2. Browne WJ
    3. Cuthill IC
    4. Emerson M
    5. Altman DG

    (2010) Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research

    PLoS Biology 8:e1000412.

    https://doi.org/10.1371/journal.pbio.1000412

    • Google Scholar
28. 1. Kilkenny C
    2. Parsons N
    3. Kadyszewski E
    4. Festing MF
    5. Cuthill IC
    6. Fry D
    7. Hutton J
    8. Altman DG

    (2009) Survey of the quality of experimental design, statistical analysis and reporting of research using animals

    PloS One 4:e7824.

    https://doi.org/10.1371/journal.pone.0007824

    • Google Scholar
29. 1. Landis SC
    2. Amara SG
    3. Asadullah K
    4. Austin CP
    5. Blumenstein R
    6. Bradley EW
    7. Crystal RG
    8. Darnell RB
    9. Ferrante RJ
    10. Fillit H
    11. Finkelstein R
    12. Fisher M
    13. Gendelman HE
    14. Golub RM
    15. Goudreau JL
    16. Gross RA
    17. Gubitz AK
    18. Hesterlee SE
    19. Howells DW
    20. Huguenard J
    21. Kelner K
    22. Koroshetz W
    23. Krainc D
    24. Lazic SE
    25. Levine MS
    26. Macleod MR
    27. McCall JM
    28. Moxley RT
    29. Narasimhan K
    30. Noble LJ
    31. Perrin S
    32. Porter JD
    33. Steward O
    34. Unger E
    35. Utz U
    36. Silberberg SD

    (2012) A call for transparent reporting to optimize the predictive value of preclinical research

    Nature 490:187–191.

    https://doi.org/10.1038/nature11556

    • Google Scholar
30. 1. Li J
    2. Kannan M
    3. Trivett AL
    4. Liao H
    5. Wu X
    6. Akagi K
    7. Symer DE

    (2014) An antisense promoter in mouse L1 retrotransposon open reading frame-1 initiates expression of diverse fusion transcripts and limits retrotransposition

    Nucleic Acids Research 42:4546–4562.

    https://doi.org/10.1093/nar/gku091

    • Google Scholar
31. 1. Lin CQ
    2. Dempsey PJ
    3. Coffey RJ
    4. Bissell MJ

    (1995)

    Extracellular matrix regulates whey acidic protein gene expression by suppression of TGF-alpha in mouse mammary epithelial cells: studies in culture and in transgenic mice

    Journal of Cell Biology 129:1115–1126.

    • Google Scholar
32. 1. Maklakov AA
    2. Lummaa V

    (2013) Evolution of sex differences in lifespan and aging: causes and constraints

    BioEssays 35:717–724.

    https://doi.org/10.1002/bies.201300021

    • Google Scholar
33. 1. McCullough LD
    2. de Vries GJ
    3. Miller VM
    4. Becker JB
    5. Sandberg K
    6. McCarthy MM

    (2014) NIH initiative to balance sex of animals in preclinical studies: generative questions to guide policy, implementation, and metrics

    Biology of Sex Differences 5:15.

    https://doi.org/10.1186/s13293-014-0015-5

    • Google Scholar
34. 1. Meystre MS
    2. Savova KG
    3. Kipper-Schuler CK
    4. Hurdle FJ

    (2008)

    Extracting information from textual documents in the electronic health record: a review of recent research

    Methods of Information in Medicine 47:128–144.

    • Google Scholar
35. 1. Moher D
    2. Simera I
    3. Schulz KF
    4. Hoey J
    5. Altman DG

    (2008) Helping editors, peer reviewers and authors improve the clarity, completeness and transparency of reporting health research

    BMC Medicine 6:13.

    https://doi.org/10.1186/1741-7015-6-13

    • Google Scholar
36. 1. Mozaffarian D
    2. Benjamin EJ
    3. Go AS
    4. Arnett DK
    5. Blaha MJ
    6. Cushman M
    7. de Ferranti S
    8. Després JP
    9. Fullerton HJ
    10. Howard VJ
    11. Huffman MD
    12. Judd SE
    13. Kissela BM
    14. Lackland DT
    15. Lichtman JH
    16. Lisabeth LD
    17. Liu S
    18. Mackey RH
    19. Matchar DB
    20. McGuire DK
    21. Mohler ER
    22. Moy CS
    23. Muntner P
    24. Mussolino ME
    25. Nasir K
    26. Neumar RW
    27. Nichol G
    28. Palaniappan L
    29. Pandey DK
    30. Reeves MJ
    31. Rodriguez CJ
    32. Sorlie PD
    33. Stein J
    34. Towfighi A
    35. Turan TN
    36. Virani SS
    37. Willey JZ
    38. Woo D
    39. Yeh RW
    40. Turner MB
    41. American Heart Association Statistics Committee and Stroke Statistics Subcommittee

    (2015) Heart disease and stroke statistics--2015 update: a report from the American Heart Association

    Circulation 131:e29.

    https://doi.org/10.1161/CIR.0000000000000152

    • Google Scholar
37. Website

    1. Nature

    (2013) Reporting Checklist for Life Sciences Articles

    http://www.nature.com/authors/policies/checklist.pdf
38. 1. Nguyen DK
    2. Disteche CM

    (2006) Dosage compensation of the active X chromosome in mammals

    Nature Genetics 38:47–53.

    https://doi.org/10.1038/ng1705

    • Google Scholar
39. 1. Ober C
    2. Loisel DA
    3. Gilad Y

    (2008) Sex-specific genetic architecture of human disease

    Nature Reviews Genetics 9:911–922.

    https://doi.org/10.1038/nrg2415

    • Google Scholar
40. 1. Pennell LM
    2. Galligan CL
    3. Fish EN

    (2012) Sex affects immunity

    Journal of Autoimmunity 38:J282–291.

    https://doi.org/10.1016/j.jaut.2011.11.013

    • Google Scholar
41. 1. Perrin S

    (2014) Preclinical research: Make mouse studies work

    Nature 507:423–425.

    https://doi.org/10.1038/507423a

    • Google Scholar
42. 1. Prinz F
    2. Schlange T
    3. Asadullah K

    (2011) Believe it or not: how much can we rely on published data on potential drug targets?

    Nature Reviews Drug Discovery 10:712–713.

    https://doi.org/10.1038/nrd3439-c1

    • Google Scholar
43. 1. Ross AP
    2. Mansilla MA
    3. Choe Y
    4. Helminski S
    5. Sturm R
    6. Maute RL
    7. May SR
    8. Wójcicki P
    9. Mostowska A
    10. Davidson B
    11. Adamopoulos IE
    12. Pleasure SJ
    13. Murray JC
    14. Zarbalis KS

    (2013) A mutation in mouse Pak1ip1 causes orofacial clefting while human PAK1IP1 maps to 6p24 translocation breaking points associated with orofacial clefting

    PLoS ONE 8:e69333.

    https://doi.org/10.1371/journal.pone.0069333

    • Google Scholar
44. 1. Settles B

    (2005) ABNER: an open source tool for automatically tagging genes, proteins and other entity names in text

    Bioinformatics 21:3191–3192.

    https://doi.org/10.1093/bioinformatics/bti475

    • Google Scholar
45. 1. Taylor K
    2. Vallejo-Giraldo C
    3. Schaible NS
    4. Zakeri R
    5. Miller VM

    (2011) Reporting of sex as a variable in cardiovascular studies using cultured cells

    Biology of Sex Differences 2:11.

    https://doi.org/10.1186/2042-6410-2-11

    • Google Scholar
46. 1. van der Worp HB
    2. Howells DW
    3. Sena ES
    4. Porritt MJ
    5. Rewell S
    6. O'Collins V
    7. Macleod MR

    (2010) Can animal models of disease reliably inform human studies?

    PLoS Medicine 7:e1000245.

    https://doi.org/10.1371/journal.pmed.1000245

    • Google Scholar
47. 1. van der Worp HB
    2. Macleod MR

    (2011) Preclinical studies of human disease: time to take methodological quality seriously

    Journal of Molecular and Cellular Cardiology 51:449–450.

    https://doi.org/10.1016/j.yjmcc.2011.04.008

    • Google Scholar
48. Website

    1. WHO

    (2014) The top 10 causes of death

    http://www.who.int/mediacentre/factsheets/fs310/en/
49. Book

    1. Wizemann TM
    2. Pardue ML

    (2001)

    Exploring the Biological Contributions to Human Health: Does Sex Matter?

    Washington DC: National Academy Press.

    • Google Scholar
50. 1. Yue F
    2. Cheng Y
    3. Breschi A
    4. Vierstra J
    5. Wu W
    6. Ryba T
    7. Sandstrom R
    8. Ma Z
    9. Davis C
    10. Pope BD
    11. Shen Y
    12. Pervouchine DD
    13. Djebali S
    14. Thurman RE
    15. Kaul R
    16. Rynes E
    17. Kirilusha A
    18. Marinov GK
    19. Williams BA
    20. Trout D
    21. Amrhein H
    22. Fisher-Aylor K
    23. Antoshechkin I
    24. DeSalvo G
    25. See LH
    26. Fastuca M
    27. Drenkow J
    28. Zaleski C
    29. Dobin A
    30. Prieto P
    31. Lagarde J
    32. Bussotti G
    33. Tanzer A
    34. Denas O
    35. Li K
    36. Bender MA
    37. Zhang M
    38. Byron R
    39. Groudine MT
    40. McCleary D
    41. Pham L
    42. Ye Z
    43. Kuan S
    44. Edsall L
    45. Wu YC
    46. Rasmussen MD
    47. Bansal MS
    48. Kellis M
    49. Keller CA
    50. Morrissey CS
    51. Mishra T
    52. Jain D
    53. Dogan N
    54. Harris RS
    55. Cayting P
    56. Kawli T
    57. Boyle AP
    58. Euskirchen G
    59. Kundaje A
    60. Lin S
    61. Lin Y
    62. Jansen C
    63. Malladi VS
    64. Cline MS
    65. Erickson DT
    66. Kirkup VM
    67. Learned K
    68. Sloan CA
    69. Rosenbloom KR
    70. Lacerda de Sousa B
    71. Beal K
    72. Pignatelli M
    73. Flicek P
    74. Lian J
    75. Kahveci T
    76. Lee D
    77. Kent WJ
    78. Ramalho Santos M
    79. Herrero J
    80. Notredame C
    81. Johnson A
    82. Vong S
    83. Lee K
    84. Bates D
    85. Neri F
    86. Diegel M
    87. Canfield T
    88. Sabo PJ
    89. Wilken MS
    90. Reh TA
    91. Giste E
    92. Shafer A
    93. Kutyavin T
    94. Haugen E
    95. Dunn D
    96. Reynolds AP
    97. Neph S
    98. Humbert R
    99. Hansen RS
    100. De Bruijn M
    101. Selleri L
    102. Rudensky A
    103. Josefowicz S
    104. Samstein R
    105. Eichler EE
    106. Orkin SH
    107. Levasseur D
    108. Papayannopoulou T
    109. Chang KH
    110. Skoultchi A
    111. Gosh S
    112. Disteche C
    113. Treuting P
    114. Wang Y
    115. Weiss MJ
    116. Blobel GA
    117. Cao X
    118. Zhong S
    119. Wang T
    120. Good PJ
    121. Lowdon RF
    122. Adams LB
    123. Zhou XQ
    124. Pazin MJ
    125. Feingold EA
    126. Wold B
    127. Taylor J
    128. Mortazavi A
    129. Weissman SM
    130. Stamatoyannopoulos JA
    131. Snyder MP
    132. Guigo R
    133. Gingeras TR
    134. Gilbert DM
    135. Hardison RC
    136. Beer MA
    137. Ren B
    138. Mouse ENCODE Consortium

    (2014) A comparative encyclopedia of DNA elements in the mouse genome

    Nature 515:355–364.

    https://doi.org/10.1038/nature13992

    • Google Scholar

Thank you for submitting your article "Bias in the reporting of sex and age as experimental factors across biomedical research on mouse models" to eLife for consideration as a Feature Article. Your article has been reviewed by two peer reviewers, and the evaluation has been overseen by a guest Reviewing Editor (Chris Mungall), a Senior Editor (Fiona Watt) and the Features Editor (Peter Rodgers).

The following individuals involved in review of your submission have agreed to reveal their identity: Terry Meehan (peer reviewer) and Nicole Vasilevsky (peer reviewer).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

The Features Editor will also contact you separately about some editorial issues that you will need to address.

Summary:

The authors have developed a text mining approach to assess bias in the reporting of sex and age of mouse model experiments reported in the scientific literature over the last two decades. The manuscript provides sufficient background as to why reporting this information is important. The methods and results are sound. The authors find while there is general improvement in reporting these variables, about half of the papers published in 2014 still lack these variables. An interesting gender-bias is noted in different fields of study.

Quantifying the problem will, it is hoped, drive journals and authors to include this information in the biomedical literature. This work could also be used by journals to help standardize descriptions of age and sex in the literature. It would also be good if the authors could rerun this analysis five years from now to see if the situation has changed.

Before publication, the following comments should be addressed.

Essential revisions:

1) In this analysis, you examined approximately 600 journals and 70 of those journals covered 30 or more articles of the corpus. In those 70 journals, do the instructions to authors require or request that sex and age are reported in the manuscripts? Are the authors complying with the journal reporting requirements?

2) I like how you show that there is an increase in the reporting of sex and age over time. In the Discussion, can you speculate as to why this is? Did journal reporting requirements get stricter, or did the ARRIVE guidelines come out at the time point when we started to see an increase?

3) Please explain why text mined articles were restricted to those with "Mice/Mouse" in the title. This eliminates many research articles. (For example, Nature is included in Figure 1—figure supplement 1, but Science is not: I suspect this is because of editorial differences in title rules, which may bias conclusions from this study). There is probably a compelling reason (e.g. automated sex and age detection breaks down when humans and mice data are described in the same article) but please tell the reader why, and note this as a limitation in the discussion.

4) In the Discussion, conjecture on why there is gender bias for females in many mouse studies. There are a number of references on mouse housing density – is gender assessed in these (e.g. some strains of male mice fight each other when caged, necessitating separate cages and driving up costs)?

Essential revisions: 1) In this analysis, you examined approximately 600 journals and 70 of those journals covered 30 or more articles of the corpus. In those 70 journals, do the instructions to authors require or request that sex and age are reported in the manuscripts? Are the authors complying with the journal reporting requirements?

We checked this and found that 50 out of the 70 journals requested the reporting of sex and age via the ARRIVE guidelines, and only one (Diabetologia) requires the reporting of these factors in the author’s guidelines. Accordingly, we have modified Supplementary Figure 1 in order to show this information.

2) I like how you show that there is an increase in the reporting of sex and age over time. In the Discussion, can you speculate as to why this is? Did journal reporting requirements get stricter, or did the ARRIVE guidelines come out at the time point when we started to see an increase?

We appreciate your comment, and we have addressed this point in the second paragraph in the Discussion section. In addition, we have also indicated in Figure 1 the year in which ARRIVE was published.

3) Please explain why text mined articles were restricted to those with "Mice/Mouse" in the title. This eliminates many research articles. (For example, Nature is included in Figure 1—figure supplement 1, but Science is not: I suspect this is because of editorial differences in title rules, which may bias conclusions from this study). There is probably a compelling reason (e.g. automated sex and age detection breaks down when humans and mice data are described in the same article) but please tell the reader why, and note this as a limitation in the discussion.

We have taken on board your comment and amended the Discussion and Methods sections respectively. The terms Mice/Mouse were used in the title prevented articles that made only passing references to mouse work from entering the dataset and ensured a high quality corpus for analysis. On the other hand, the PMC dataset we have used (Feb 2015 release) indeed did not have any Science papers. Although their articles are PMC Open Access, Science is not included in the PMC journal list. We note in the Open Access Subset website that “Many of the journals that deposit their complete contents into PMC include some or all of their articles in the Open Access subset” (http://www.ncbi.nlm.nih.gov/pmc/tools/openftlist/). Science has released only one issue from 2015 as open access.

4) In the Discussion, conjecture on why there is gender bias for females in many mouse studies. There are a number of references on mouse housing density – is gender assessed in these (e.g. some strains of male mice fight each other when caged, necessitating separate cages and driving up costs)?

We have addressed this point in the third paragraph of the Discussion section.

Author details

1. Oscar Flórez-Vargas

   Bio-health Informatics Group, School of Computer Science, The University of Manchester, Manchester, United Kingdom

   Contribution

   OF-V, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article

   For correspondence

   florezvo@cs.manchester.ac.uk

   Competing interests

   The authors declare that no competing interests exist.

2. Andy Brass

   Bio-health Informatics Group, School of Computer Science, The University of Manchester, Manchester, United Kingdom

   Contribution

   AB, Conception and design, Analysis and interpretation of data, Drafting or revising the article

   For correspondence

   andy.brass@manchester.ac.uk

   Competing interests

   The authors declare that no competing interests exist.

3. George Karystianis

   Text Mining Group, School of Computer Science, The University of Manchester, Manchester, United Kingdom

   Contribution

   GK, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

4. Michael Bramhall

   Bio-health Informatics Group, School of Computer Science, The University of Manchester, Manchester, United Kingdom

   Contribution

   MB, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

   "This ORCID iD identifies the author of this article:" 0000-0001-5938-158X

5. Robert Stevens

   Bio-health Informatics Group, School of Computer Science, The University of Manchester, Manchester, United Kingdom

   Contribution

   RS, Conception and design, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

6. Sheena Cruickshank

   Manchester Immunology Group, Faculty of Life Science, The University of Manchester, Manchester, United Kingdom

   Contribution

   SC, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

7. Goran Nenadic

   1. Text Mining Group, School of Computer Science, The University of Manchester, Manchester, United Kingdom
   2. Manchester Institute of Biotechnology, University of Manchester, Manchester, United Kingdom

   Contribution

   GN, Conception and design, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

• 4,999

  views

• 682

  downloads

• 87

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.