Analysis of NIH K99/R00 Awards and the Career Progression of Awardees

  1. Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
  2. Department of Biology, Tufts University, Medford, MA, USA
  3. Department of Health, Nutrition, and Food Sciences, Florida State University, Tallahassee, FL, USA
  4. Section of Genetic Medicine, Department of Medicine, University of Chicago, Chicago, IL, USA
  5. Department of Systems Pharmacology and Translational Therapeutics, University of Pennsylvania, Philadelphia, PA, USA
  6. Division of Molecular and Cellular Pathology, Department of Pathology, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA

Peer review process

Revised: This Reviewed Preprint has been revised by the authors in response to the previous round of peer review; the eLife assessment and the public reviews have been updated where necessary by the editors and peer reviewers.

Read more about eLife’s peer review process.

Editors

  • Reviewing Editor
    Peter Rodgers
    eLife, Cambridge, United Kingdom
  • Senior Editor
    Peter Rodgers
    eLife, Cambridge, United Kingdom

Reviewer #1 (Public Review):

Summary and strengths
This is an interesting, timely and informative article. The authors used publicly available data (made available by a funding agency) to examine some of the academic characteristics of the individuals recipients of the National Institutes of Health (NIH) k99/R00 award program during the entire history of this funding mechanism (17 years, total ~ 4 billion US dollars (annual investment of ~230 million USD)). The analysis focuses on the pedigree and the NIH funding portfolio of the institutions hosting the k99 awardees as postdoctoral researchers and the institutions hiring these individuals. The authors also analyze the data by gender, by whether the R00 portion of the awards eventually gets activated and based on whether the awardees stayed/were hired as faculty at their k99 (postdoctoral) host institution or moved elsewhere. The authors further sought to examine the rates of funding for those in systematically marginalized groups by analyzing the patterns of receiving k99 awards and hiring k99 awardees at historically black colleges and universities.

The goals and analysis are reasonable and the limitations of the data are described adequately. It is worth noting that some of the observed funding and hiring traits are in line with the Matthew effect in science (Merton, 1968: https://www.science.org/doi/10.1126/science.159.3810.56) and in science funding (Bol et al., 2018: https://www.pnas.org/doi/10.1073/pnas.1719557115). Overall, the article is a valuable addition to the research culture literature examining the academic funding and hiring traits in the United States. The findings can provide further insights for the leadership at funding and hiring institutions and science policy makers for individual and large-scale improvements that can benefit the scientific community.

Weaknesses
The authors have addressed my recommendations in the previous review round in a satisfactory way.

Reviewer #2 (Public Review):

Summary and strengths
Early career funding success has an immense impact on later funding success and faculty persistence, as evidenced by well-documented "rich-get-richer" or "Matthew effect" phenomena in science (e.g., Bol et al., 2018, PNAS). In this study the authors examined publicly available data on the distribution of the National Institutes of Health's K99/R00 awards - an early career postdoc-to-faculty transition funding mechanism - and showed that although 89% of K99 awardees successfully transitioned into faculty, disparities in subsequent R01 grant obtainment emerged along three characteristics: researcher mobility, gender, and institution. Men who moved to a top-25 NIH funded institution in their postdoc-to-faculty transition experienced the shortest median time to receiving a R01 award, 4.6 years, in contrast to the median 7.4 years for women working at less well-funded schools who remained at their postdoc institutions.

Amongst the three characteristics, the finding that researcher mobility has the largest effect on subsequent funding success is key and novel. Other data supplement this finding: for example, although the total number of R00 awards has increased, most of this increase is for awards to individuals moving to different institutions. In 2010, 60% of R00 awards were activated at different institutions compared to 80% in 2022. These findings enhance previous work on the relationship between mobility and ones' access to resources, collaborators, or research objects (e.g., Sugimoto and Larivière, 2023, Equity for Women in Science (Harvard University Press)).

These results empirically demonstrate that even after receiving a prestigious early career grant, researchers with less mobility belonging to disadvantaged groups at less-resourced institutions continue to experience barriers that delay them from receiving their next major grant. This result has important policy implications aimed at reducing funding disparities - mainly that interventions that focus solely on early career or early stage investigator funding alone will not achieve the desired outcome of improving faculty diversity.

The authors also highlight two incredible facts: No postdoc at a historically Black college or university (HBCU) has been awarded a K99 since the program's launch. And out of all 2,847 R00 awards given thus far, only two have been made to faculty at HBCUs. Given the track record of HBCUs for improving diversity in STEM contexts, this distribution of awards is a massive oversight that demands attention.

At no fault of the authors, the analysis is limited to only examining K99 awardees and not those who applied but did not receive the award. This limitation is solely due to the lack of data made publicly available by the NIH. If this data were available, this study would have been able to compare the trajectory of winners versus losers and therefore could potentially quantify the impact of the award itself on later funding success, much like the landmark paper by Bol et al. (PNAS; 2018) that followed the careers of an early career grant scheme in the Netherlands. Such an analysis would also provide new insights that would inform policy.

Although data on applications versus awards for the K99/R00 mechanism are limited, there exists data for applicant race and ethnicity for the 2007-2017 period, which were made available by a Freedom of Information Act request through the now defunct Rescuing Biomedical Research Initiative (https://web.archive.org/web/20180723171128/http://rescuingbiomedicalresearch.org/blog/examining-distribution-k99r00-awards-race/). These results are highly relevant given the discussion of K99 award impacts on the sociodemographic composition of U.S. biomedical faculty. During the 2007-2017 period, the K99 award rate for white applicants was 31% compared to 26.7% for Asian applicants and 16.2% for Black applicants. In terms of award totals, these funding rates amount to 1,384 awards to white applicants, 610 to Asian applicants, and 25 to Black applicants. However, the work required to include these data may be beyond the scope of the study.

The conclusions are well-supported by the data, and limitations of the data and the name-gender matching algorithm are described satisfactorily.

Reviewer #3 (Public Review):

Summary
The researchers aim add to the literature on faculty career pathways with particular attention to how gender disparities persist in the career and funding opportunities of researchers. The researchers also examine aspects of institutional prestige that can further amplify funding and career disparities. While some factors about individuals' pathways to faculty lines are known, including the prospects of certain K award recipients, the current study provides the only known examination of the K99/R00 awardees and their pathways.

Strengths
The authors establish a clear overview of the institutional locations of K99 and R00 awardees and the pathways for K99-to-R00 researchers and the gendered and institutional patterns of such pathways. For example, there's a clear institutional hierarchy of hiring for K99/R00 researchers that echo previous research on the rigid faculty hiring networks across fields, and a pivotal difference in the time between awards that can impact faculty careers. Moreover, there's regional clusters of hiring in certain parts of the US where multiple research universities are located. Moreover, documenting the pathways of HBCU faculty is an important extension of the study by Wapman et al. (2022: https://www.nature.com/articles/s41586-022-05222-x), and provides a more nuanced look at the pathways of faculty beyond the oft-discussed high status institutions. (However, there is a need for more refinement in this segment of the analyses). Also, the authors provide important caveats throughout the manuscript about the study's findings that show careful attention to the complexity of these patterns and attempting to limit misinterpretations of readers.

Weaknesses
The authors have addressed my recommendations in the previous review round in a satisfactory way.

Author Response

The following is the authors’ response to the original reviews.

Public Reviews:

Reviewer #1 (Public Review):

This is an interesting, timely and informative article. The authors used publicly available data (made available by a funding agency) to examine some of the academic characteristics of the individuals recipients of the National Institutes of Health (NIH) k99/R00 award program during the entire history of this funding mechanism (17 years, total ~ 4 billion US dollars (annual investment of ~230 million USD)). The analysis focuses on the pedigree and the NIH funding portfolio of the institutions hosting the k99 awardees as postdoctoral researchers and the institutions hiring these individuals. The authors also analyze the data by gender, by whether the R00 portion of the awards eventually gets activated and based on whether the awardees stayed/were hired as faculty at their k99 (postdoctoral) host institution or moved elsewhere. The authors further sought to examine the rates of funding for those in systematically marginalized groups by analyzing the patterns of receiving k99 awards and hiring k99 awardees at historically black colleges and universities.

The goals and analysis are reasonable and the limitations of the data are described adequately. It is worth noting that some of the observed funding and hiring traits are in line with the Matthew effect in science (https://www.science.org/doi/10.1126/science.159.3810.56) and in science funding (https://www.pnas.org/doi/10.1073/pnas.1719557115). Overall, the article is a valuable addition to the research culture literature examining the academic funding and hiring traits in the United States. The findings can provide further insights for the leadership at funding and hiring institutions and science policy makers for individual and large-scale improvements that can benefit the scientific community.

Thank you for these comments. We have incorporated the articles referenced on the Matthew effect into the first paragraph of the Discussion our revised preprint.

Reviewer #2 (Public Review):

Early career funding success has an immense impact on later funding success and faculty persistence, as evidenced by well-documented "rich-get-richer" or "Matthew effect" phenomena in science (e.g., Bol et al. 2018, PNAS). Woitowich et al. examined publicly available data on the distribution of the National Institutes of Health's K99/R00 awards - an early career postdoc-to-faculty transition funding mechanism - and showed that although 85% of K99 awardees successfully transitioned into faculty, disparities in subsequent R01 grant obtainment emerged along three characteristics: researcher mobility, gender, and institution. Men who moved to a top-25 NIH funded institution in their postdoc-to-faculty transition experienced the shortest median time to receiving a R01 award, 4.6 years, in contrast to the median 7.4 years for women working at less well-funded schools who remained at their postdoc institutions. This result is consistent with prior evidence of funding disparities by gender and institution type. The finding that researcher mobility has the largest effect on subsequent funding success is key and novel, and enhances previous work showing the relationship between mobility and ones' access to resources, collaborators, or research objects (e.g., Sugimoto and Larivière, 2023, Equity for Women in Science (Harvard University Press)).

These results empirically demonstrate that even after receiving a prestigious early career grant, researchers with less mobility belonging to disadvantaged groups at less-resourced institutions continue to experience barriers that delay them from receiving their next major grant. This result has important policy implications aimed at reducing funding disparities - mainly that interventions that focus solely on early career or early stage investigator funding alone will not achieve the desired outcome of improving faculty diversity.

The authors also highlight two incredible facts: No postdoc at a historically Black college or university (HBCU) has been awarded a K99 since the program's launch. And out of all 2,847 R00 awards given thus far, only two have been made to faculty at HBCUs. Given the track record of HBCUs for improving diversity in STEM contexts, this distribution of awards is a massive oversight that demands attention.

At no fault of the authors, the analysis is limited to only examining K99 awardees and not those who applied but did not receive the award. This limitation is solely due to the lack of data made publicly available by the NIH. If this data were available, this study would have been able to compare the trajectory of winners versus losers and therefore could potentially quantify the impact of the award itself on later funding success, much like the landmark Bol et al. (2018) paper that followed the careers of winners of an early career grant scheme in the Netherlands. Such an analysis would also provide new insights that would inform policy.

Although data on applications versus awards for the K99/R00 mechanism are limited, there exists data for applicant race and ethnicity for the 2007-2017 period, which were made available by a Freedom of Information Act request through the now defunct Rescuing Biomedical Research Initiative: https://web.archive.org/web/20180723171128/http://rescuingbiomedicalresearch.org/blog/examining-distribution-k99r00-awards-race/. These results are not presently discussed in the paper, but are highly relevant given the discussion of K99 award impacts on the sociodemographic composition of U.S. biomedical faculty. From 2007 to 2017, the K99 award rate for white applicants was 31.0% compared to 26.7% for Asian applicants and 16.2% for Black applicants. In terms of award totals, these funding rates amount to 1,384 awards to white applicants, 610 to Asian applicants, and 25 to Black applicants for the entire 2007-2017 period. And in terms of R00 awards, or successful faculty transitions: whereas 77.0% of white K99 awardees received an R00 award, the conversion rate for Asian and Black K99 awardees was lower, at 76.1% and 60.0%, respectively. Regarding this K99-to-R00 transition rate, Woitowich et al. found no difference by gender (Table 2). These results are consistent with a growing body of literature that shows that while there have been improvements to equity in funding outcomes by gender, similar improvements for achieving racial equity are lagging.

The conclusions are well-supported by the data, and limitations of the data and the name-gender matching algorithm are described satisfactorily.

One aspect that the authors should expand or comment on is the change in the rate of K99 to R00 conversions. Since 2016, while the absolute number of K99 and R00 awards has been increasing, the percentage of R00 conversions appears to be decreasing, especially in 2020 and 2021. This observation is not clearly stated or shown in Figure 1 but is an important point - if the effectiveness of the K99/R00 mechanism for postdoc-to-faculty transitions has been decreasing lately, then something is undermining the purpose of this mechanism. This result bears emphasis and potentially discussion for possible reasons for why this is happening.

Thank you for these insightful comments. We now calculate a rolling conversion rate for K99 to R00 awards which shows there is not as much of a decline in conversion from K99 to R00 (Fig 1B). We still see a slight decline in 2021 and 2022. 468 K99 awards are from 2020 or later so they may still convert to the R00 phase. Thus it is difficult to draw conclusions about 2021/2022 yet. As more time passes, we may better be able to determine whether or not significant alteration from normal occurred in these years, presumably due to pressures from the Covid-19 pandemic. We also thank you for providing the details of the FOIA request. We have included a discussion of these data in the discussion.

Reviewer #3 (Public Review):

The researchers aim add to the literature on faculty career pathways with particular attention to how gender disparities persist in the career and funding opportunities of researchers. The researchers also examine aspects of institutional prestige that can further amplify funding and career disparities. While some factors about individuals' pathways to faculty lines are known, including the prospects of certain K award recipients, the current study provides the only known examination of the K99/R00 awardees and their pathways.

Strengths:

The authors establish a clear overview of the institutional locations of K99 and R00 awardees and the pathways for K99-to-R00 researchers and the gendered and institutional patterns of such pathways. For example, there's a clear institutional hierarchy of hiring for K99/R00 researchers that echo previous research on the rigid faculty hiring networks across fields, and a pivotal difference in the time between awards that can impact faculty careers. Moreover, there's regional clusters of hiring in certain parts of the US where multiple research universities are located. Moreover, documenting the pathways of HBCU faculty is an important extension of the Wapman et al. study (among others from that research group), and provides a more nuanced look at the pathways of faculty beyond the oft-discussed high status institutions. (However, there is a need for more refinement in this segment of the analyses as discussed further below.). Also, the authors provide important caveats throughout the manuscript about the study's findings that show careful attention to the complexity of these patterns and attempting to limit misinterpretations of readers.

Weaknesses:

The authors reference institutional prestige in relation to some of the findings, but there's no specific measure of institutional prestige included in the analyses. If being identified as a top 25 NIH-funded institution is the proximate measure for prestige in the study, then more justification of how that relates to previous studies' measures of institutional prestige and status are needed to further clarify the interpretations offered in the manuscript.

The identification of institutional funding disparities impacting HBCUs is an important finding and highlights another aspect of how faculty at these institutions are under resourced and arguably undervalued in their research contributions. However, a lingering question exists: why compare HBCUs with Harvard? What are the theoretical and/or methodological justifications for such comparisons? This comparison lends itself to reifying the status hierarchy of institutions that perpetuate funding and career inequalities at the heart of the current manuscript. If aggregating all HBCU faculty together, then a comparable grouping for comparison is needed, not just one institution. Perhaps looking at the top 25 NIH funded institutions could be one way of providing a clearer comparison. Related to this point is the confusing inclusion of Gallaudet in Figure 6 as it is not an officially identified HBCU. Was this institution also included in the HBCU-related calculations?

Thank you for this comment. We agree this comparison perpetuates the perception of the prestige hierarchy and is problematic. We now compare all institutions in the top 25 NIH funding category to all HBCUs. Thank you also for identifying our error in mis-coding Gallaudet as an HBCU. We have corrected this in the current version.

There is a clear connection that is missed in the current iteration of the manuscript derived from the work of Robert Merton and others about cumulative advantages in science and the "Matthew effect." While aspects of this connection are noted in the manuscript such as well-resourced institutions (those with the most NIH funding in this circumstance) hire each others' K99/R00 awardees, elaborating on these connections are important for readers to understand the central processes of how a rigid hierarchy of funding and career opportunities exist around these pathways. The work the authors build on from Daniel Larremore, Aaron Clauset, and their colleagues have also incorporated these important theoretical connections from the sociology of knowledge and science, and it would provide a more interdisciplinary lens and further depth to understanding the faculty career inequalities documented in the current study.

Reviewer #1 (Recommendations For The Authors):

Comments to authors:

  1. For the benefit of general reader, it would be informative to mention the amount of annual NIH investment in the k99 funding mechanism in the text (230 awards representing a ~ 230 million US dollars investment).

Thank you for this suggestion. We have added that this is ~$25 million investment annually.

  1. It is worth noting that some of the observed funding and hiring traits resemble the Matthew effect, discussed in: The Matthew effect in science: https://www.science.org/doi/10.1126/science.159.3810.56

The Matthew effect in science funding: https://www.pnas.org/doi/10.1073/pnas.1719557115

It would be of value to cite these for further context for the readers.

Thank you for this suggestion. We have included these references and briefly discussed the Matthew effect in the first paragraph of the Discussion.

  1. Figs 3, 6 and Fig S1 are hard to read without zooming in due to their format and don't work great within a letter size page but can work if they are also linked to a zoomable web version. It would make sense to have an online navigable/searchable/selectable version. But when the reader zooms out, there are patterns that reflect what points the authors are making (though those could be illustrated differently). These figures are really made for online webapp visualization (such as Shiny in R).

We agree with this comment and have used the “googleVis()” package in R to put together interactive Sankey diagrams. These can be found at: https://dantyrr.github.io/K99-R00-analysis/ and they are referenced in the manuscript.

  1. The abstract states 85% of awardees get R00 awards. That appears to come from 198/234 (page 6) though it's not explicitly stated, and other ratios give different answers (e.g., 1-304/3475 = 91%) but the 85% seems to be the right one. That first paragraph of the results could be clearer. Also, in the middle of page three the number given is 90% so something is inconsistent. For Figure 1A, given the methodology it should be possible to calculate a rolling conversion rate as "R00(t) / K99(t-1)" (and a similarly-calculated cumulative rate).

Thank you for catching these errors. These were introduced because there are R00 awardees that did not have extramural K99 awards. These are intramural NIH K99 awardees but there is no public data on these awardees. The correct number is 78% of K99 awardees that transitioned to the R00 phase. We have also calculated the rolling conversion rate which is 89% if you exclude the first 2 years of the program (when the first awardees were within the 2-yr K99 period) and final 2 years (when most recent K99 awardees were still within their first 2 years of the K99 period).

  1. Assuming that 85% is the correct number, is there any information/insight into why ~1/6 of awardees do not continue to R00, which seems high given that only two years passes - that's a lot of awardees not getting R00 positions.

We are unsure of why these don’t convert. In the revised version of the manuscript, we speculate on this in the 4th paragraph of the discussion:

The factors that prevented the other 302 K99 awardees from 2019 and earlier unable to convert their K99-R00 grants is cause for concern within our greater academic community. Possible explanations include leaving the biomedical workforce, accepting tenure-track positions or other positions abroad, or by simply not successfully securing a tenable tenure-track offer.

  1. It looks like perhaps a non-zero number of K99s are just one year and not two (e.g., see 2006 in Fig 1A, which should not appear if all 2006 awards were 2 years). What is the typical percentage of K99s not activated for a second year, and is this a sizable % of the 15% not converting to R00?

This is an interesting question. We didn’t originally look into this and the dataset that we originally downloaded from NIH reporter included a significant number of duplicates for the grants because year 1 of the K99 was listed on its own line and year 2 was listed on a different line. The first step in curating the data was to delete the duplicate values so we only had one entry per person. Unfortunately based on sorting of the data tables, sometimes the year 1 appeared above year 2 and at other times year 2 appeared before year 1. Because none of the data we were interested in are benchmarked to K99 start date, we removed the duplicate values non-specifically. With the dataset we currently have, we would not be able to tell which individuals dropped out (didn’t convert to R00) during the first or second year of the K99. In order to do this we would have to download the raw data from NIH reporter again and curate it again. We may do this in the future but for the purpose of publishing the current manuscript we prefer to focus our efforts on other aspects of the revision.

  1. Further down page 3, the authors state that "men typically experience 2-3% greater funding success rates" is ambiguous, as rates are themselves a percentage. So, is it 2-3% greater as in 23% vs 20%, or is it 2-3% greater as in 20.6% vs 20%? Please clarify the language.

Thank you for asking for this clarification. We have updated the text here to reflect that we mean “23% vs 20%”.

  1. Metrics such as time to first R01 are compared internally within the study set, which yields interesting insights, but more could be done to benchmark these metrics to non-K99 scientists.

We agree with the reviewer that this would be ideal; however, we feel that it is out of the scope of this manuscript. We may examine this in the future.

  1. In the text, several times percentages are being referred to when the figures cited do not show percentages. For example (page 6) 'proportion of awardees that stayed at the same institution declined to about 20% where it has remained consistent (Fig 1B)' - Figure 1B does not show percentages, instead the reader would need to work out from the raw numbers what the pattern of percentages might look like. It's fine (great even) to provide the raw numbers, but would be great to show the percentages as well. This happened for multiple graphs.

Thank you for this comment. We agree that showing the percentage would be beneficial so we have included the percentages in Figure 1 for the conversion rate. We also added a standalone figure panel for the rolling conversion rate for Figure 1. For Figure 4, we have also included a right Y-axis to better indicate the % women.

  1. Figure 4 - putting the %women on a 0-250 scale makes it difficult to see the changes in that curve. Please replot it as a separate graph with an appropriate scale (30-50%? 30-70%?)

Thank you for this comment. We have made this edit.

  1. Figure 5 - The table appears inconsistent - the Moved/Stayed HR is 1.411 suggesting that moving is better for reducing time to R01, but then Woman/Man is 1.208, so one of these pairs needs to be written in the opposite order to have the table make sense (intended to be listed as 'better/worse'?)

Thank you for noticing this. In the revised manuscript we have re-run the cox proportional hazard model using the R package “survival” and the function “coxph()”. There were minor differences in the hazard ratios using this package instead of Graphpad prism; however, the R package is much more widely used compared to prism for these types of analysis. We present the new data in the table in Figure 5B in the revised manuscript. We now present the “detrimental” cox hazard value for each variable (i.e. 0.7095 for the mobility [moved/stayed]). We also underlined the variable which was detrimental to receiving an R01 award earlier.

  1. Figure 5's graph appears strange. All the lines have an appearance of stochasticity but are actually multiples of each other, rising exactly in sync. Are these actually modeled lines? If so, why not instead actually draw the lines based on the real data from the real groups depicted, and give the n for each group?

Thank you for picking this up. The software we originally used to plot the graphs did plot modeled lines instead of the actual data. We have re-run the cox proportional hazard model using the R “survival” package v3.5-5 and the coxph() and survfit() functions. The updated data are in Figure 5 of the revised manuscript.

  1. Table 1 should note that each column sums to 100%.

This is a good suggestion. In the revised manuscript, we have added a row to the table to indicate the column total N and %.

  1. The authors discuss how k99/R00 grant reviewing process may have to change but the k99 awards also impact the faculty hiring ecosystem as well. There are faculty hiring job ads explicitly requesting or indicating preference towards k99 holders and the results described in this article show that k99 awarding is biased towards particular demographics at select wealthy institutions. Of course, collective/central action is almost always more effective/impactful (especially in shorter time line) than individual elective action. In other words, NIH changing granting patterns would likely work better than encouraging faculty searches to change the weight they give to K99s, because there are many searches and just one NIH. But these are not mutually exclusive and individual action can still help when central action isn't done (if the NIH does not change the k99/R00 grant review process for more inclusive funding and does not increase the number of annual k99 awards hence the annual budget for this award mechanism) and it would be good to have this discussed in the manuscript.

Thank you for this comment and thoughtful insights. We have included additional discussion on this in the final paragraph of the discussion.

Reviewer #2 (Recommendations For The Authors):

Thank you for conducting this important work. On top of some thoughts I have described in the public review (in particular, Chris Pickett's FOIA data on K99/R00 outcomes by applicant race and ethnicity), I only have a few comments for potential improvements to this paper:

  1. The comparison of K99-R00 transition rates by gender was interesting. However, I missed the analysis on the K99-R00 transition rates by institution (by type or by top-25 NIH funded institution versus not). I think this analysis may be buried somewhere in the more nuanced descriptions about faculty flows from one institution type to another, but I was not able to locate it. I wonder if the authors could consider dedicating a subsection to specifically describing the transition rate by institution type, creating a table equivalent to Table 2. This section would probably fit best somewhere before the authors dive into the nuances of self-hires and faculty flows.

Said another way: As I was reading, I felt I was missing an answer to a simple question - are there differences in conversion rates by institution type (however you define institution type, as an MSI or non MSI, or top-25 NIH funded versus not)?

Thank you for this suggestion. We have created the table (Table 3 and Table 4) in the revised manuscript. We also made a new figure (now figure 5 in the revised manuscript). This was an interesting way to look at the data and it is very clear that the number of K99 and R00 awards is heavily concentrated within the institutions that have the highest NIH funding. We have added a paragraph in the results in a new section entitled “K99 and R00 awards are concentrated within the highest funded institutions”.

  1. Regarding the comparison of HBCUs and Harvard: this analysis was elucidating, but I am not sure if the framing of this analysis as pertaining to "systematically marginalized groups" - see second sentence in the section, "Faculty doctorates differ between Harvard and HBCUs" is appropriate. While it is true that proportionally more faculty at HBCUs are from marginalized groups, there are also many faculty at HBCUs who are from privileged or advantaged backgrounds (e.g., white, men, educated at elite institutions). It would be more accurate to rephrase the second sentence to say something along the lines of, "We sought to examine the rates of funding for those at historically under-funded institutions." I recommend that the authors comb the paper for any other potential places in the text that conflate systemic marginalization with institution type, and rephrase as needed for accuracy.

Thank you for pointing this out. This is an extremely important point and we have removed any instances we could find where we conflate systemically marginalized groups with institution type.

  1. I strongly recommend Sugimoto and Larivière (2023)'s new book, Equity for Women in Science, which has an entire section dedicated to previous work investigating how researcher mobility impacts access to resources, collaborations, et cetera (Chapter 5 on Mobility; other chapters on Funding are also relevant but I hone in on Mobility since this is such a key result of this work). I think this chapter would provide significant food-for-thought and background that could strengthen the Discussion section of the paper.

Thank you for this suggestion. We have added some discussion of mobility in the first paragraph of the Discussion.

  1. I appreciated the subsection headings that described key results (e.g., "Institutions with the most NIH funding tend to hire K99/R00 awardees from other institutions with the most funding"; "K99/R00 awardee self-hires are more common at institutions with the top NIH funding.") This paper structure made it easier for me to ensure that I was getting the intended takeaway from a figure or section. But partway through the paper, the subheadings changed to being less declarative and therefore less informative (e.g., "Gender of K99/R00 awardees"; "Factors influencing K99/R00 awardee future funding success"). It would be great to rephrase these boilerplate subsection headers to be more declarative, like earlier subsection headings. For example, maybe say "Men receive the majority of K99 awards" or "No gender difference in the rate of conversion from K99 to R00" or something to that effect, depending on what result the authors wish to emphasize.

Thank you for this comment. This is a very good point. We have re-worded the more generic headings in the revised version.

  1. Lastly, I would like to share a question that came to my mind that involves an additional analysis, but is work that is (probably) out-of-the-scope of this paper, but could instead be a separate paper or product. Circling back to Chris Pickett's FOIA-ed data on K99/R00 funding outcomes by applicant race and ethnicity (https://web.archive.org/web/20180723171128/http://rescuingbiomedicalresearch.org/blog/examining-distribution-k99r00-awards-race/): Given that Pickett's numbers provide incontrovertible information on the number of awards to various racial and ethnic groups, I wonder if it is possible to use this information as an "answer key" to (1) check the accuracy of an algorithm that assigns race based on name for applications in your analysis but for 2007-2017 period, and, (2) if the results are reasonable, then examine the dataset with race and ethnicity information. Some recent papers performing large-scale bibliometric analyses have applied such algorithms (e.g., see Kozlowski et al. 2022 PNAS Intersectional inequalities in science) and I wonder if they could be useful, or at least tested, here. Again, Pickett's data would serve as the benchmark to see if the algorithm produces numbers that are consistent with the actual funding outcomes; if they're not wildly off, or perhaps accurate for some groups but not others, there might be something here.

This is a really insightful comment. We have discussed whether we could assign ethnicity based on an algorithm and check based on Chris Pickett’s data. We agree that it is beyond the scope of this article, but has potential for future research.

Reviewer #3 (Recommendations For The Authors):

-In the methods section, it would be helpful to provide an overview of the number of universities, departments, and faculty represented in the data analyzed in the study.

Thank you for this comment. We agree with the reviewer. We have added a section to the results discussing the distribution of different types of institutions. We also added Table 3 and Table 4 and a new Figure 5 describing these. Regarding the faculty, we have discussed the demographics of the K99 and R00 awardees as best as we could. We do not have data on which faculty laboratories the K99 awardees were in when they received their awards. This information is not available through NIH reporter.

-I would consider incorporating, or at least citing, Jeff Lockhart and colleagues' recent paper Nature Human Behavior article "Name-based demographic inference and the unequal distribution of misrecognition" about to provide readers with an additional resource and more information about the likelihood of misattribution and general cautionary notes about using gender and race/ethnicity ascription/imputation approaches and tools for research.

Thank you for bringing this reference to our attention. We have incorporated this into the methods section describing our name-based gender determination.

-In the next to last sentence under the final paragraph of the methods section, there looks to be a typo as it should read "K99 or R00," not "K00" as currently written.

Thank you for catching this. We have now corrected it.

-Clarifying some of the data and measures used are necessary to limit confusion and misinterpretations of the study's findings.

Thank you. We have significantly updated the revised manuscript and hope that it is more clear.

-Elaborating more on the gender inequality notable in the Cox proportional hazard model would strengthen the authors' point about persistent gender inequalities within the K99/R00 funding mechanism and pathways. In its current iteration, the findings are somewhat buried by the discussion of institutional differences, but when we look at the findings and the plot associated with the model, we notice that men have more advantages than women in funding and institutional location.

Thank you for highlighting this. This is true and we have elaborated on the gender inequality in the revised version of the manuscript.

-Also for the Cox proportional hazard model, I would consider exploring the inclusion of data that can further clarify the biomedical research infrastructure of institutions. For example, in the conversation about the differences between Princeton and other universities including other Ivies, it's important to note that Princeton does not have a medical school. Moreover, other institutions do not operate or are affiliated with a hospital. Adding more data to the model that can better contextualize the research infrastructure around researchers with NIH awards beyond the size of the NIH portfolio can shed light on possibly other important institutional differences that undergird these inequalities.

Thank you for this comment. We have added additional details about the institutional type; however, to examine whether institutions are attached to a hospital (or are themselves as hospital like MGH etc.) or whether institutions include a medical school may be difficult. We would have to manually code these and then determine whether or not the award recipient was affiliated with a department within that entity or not. We believe that this is a fascinating question but that it is out of the scope of the present manuscript. This is something that we will look into for potential future publications.

-Throughout the manuscript there's usage of "elite" and "prestigious" that are somewhat ambiguous regarding what exactly they are referring to about institutional characteristics. This is a common issue in the literature, but trying to clarify what these terms specifically mean for the current study and checking for consistent usage with limited interchangeability that can add confusion for readers about what is being referred to would give added strength to the conversation provided by the authors.

Thank you for this suggestion. Based on these comments and those by the other reviewers, in the revised version of the manuscript, we have limited the use of “elite” and “prestigious” to describe institutions in order not to perpetuate biases toward certain institutions.

-In relation to the discussion at the end of the manuscript of the longer time to award noted for researchers who stay at the same institutions, another possibility for the disparity could be their reliance for service work (e.g., hiring committees, departmental committees, supporting graduate students through mentoring and/or dissertation committee work, etc.) in their institutions given their knowledge of and experience within it.

Thank you for this suggestion. We have added 2 sentences to the discussion reflecting this possibility.

-Engaging with how STEM professional cultures can perpetuate these funding disparities and related hiring and career outcomes could enhance the contributions of the study. In relation to STEM professional cultures, engaging with the work of Mary Blair-Loy and Erin Cech in their recent book, Misconceiving Merit, could help provide additional insights for readers.

Thank you for these comments. We have incorporated edits to the revised manuscript reflecting the work of Erin Cech and Mary Blair-Loy.

  1. Howard Hughes Medical Institute
  2. Wellcome Trust
  3. Max-Planck-Gesellschaft
  4. Knut and Alice Wallenberg Foundation