Illustration of the Matthew effect and the early-career setback effect.

An academic, say Alice, first applies for early-career funding in 2015. She received early-career funding, and goes on to reapply for later-career funding in 2020. According to the Matthew effect, the chances of Alice receiving later-career funding is higher when she received early-career funding. Alice showed a high Mean Field Citation Rate (MFCR) before receiving her early-career funding, and similarly a high MFCR in between the early-career and later-career applications. According to the early-career setback effect, had she instead not received funding, her “Between” MFCR would have been higher. We not only study the “Between” MFCR, we also study the MFCR after the early-career application (“Post (early)”) and after the later-career application (“Post (later)”).

Illustration of Bayesian model.

The latent quality affects whether a researcher applies for funding, the review score (s)he receives and his or her citation impact. The funding decision at time t affects the decision to apply, the review score and the citation impact at time t + 1. The effect of time is modelled using a survival analysis approach and also considers right censoring of the observations. Field and affiliation effects in the funding decision are not illustrated here for simplification, but are included in the model.

The Matthew effect replicates and generalises.

In (a) we show that funded applicants are more likely to receive later-career funding than unfunded applicants, both overall (p < 2.5 · 10–5, n = 40369) and limited to near-hit/miss applications (p = 0.087, n = 1699). In (b) we show that previously funded applicants are more likely to submit later applications than unfunded applicants (θ = 0.26 ± 4.5 · 10–3, n = 109624). In contrast, (c) shows that prior funding does not increase review scores, if anything, the effect is even slightly negative (λ = −0.058 ± 1.6 · 10–3, n = 109624). The illustrated effects are calculated using the mean inferred quality ((g) = 3.51) as a baseline. The insets show the posterior distributions of the estimated effect coefficients.

The early-career setback effect replicates but does not seem to generalise.

In (a) we show that funded applicants have a higher MFCR in all periods (p < 0.05 across all periods, n = 40369). In (b) we show that among near-hit/miss applicants, the “Between” MFCR is higher for unfunded applicants (p = 0.019, n = 1699) while the MFCR is nearly the same across all other periods (p > 0.24 across all periods, n = 1699). In (c) we show that the Bayesian model suggests a positive effect of funding on later MFCRs (γ = 0.91 ± 0.038, n = 109624). We illustrate this effect using the mean inferred quality as a baseline (‹q› = 3.51), and we display the distribution of the MFCR of 30 publications (i.e. using the mean over 30 publications). The inset shows the distribution of the estimated effect coefficient.

Illustration of the potential collider in the early-career setback effect based on a simulation.

We show quality and MCFR values for near-hit (in blue) and near-miss (in red) applicants from a simulation. Simulated applicants who reapply later are opaque, while those who do not are translucent. The marginal distributions of quality and MFCR are similar for funded and unfunded applicants (shown as translucent filled areas in the marginal plots). However, in our simulation, unfunded applicants only reapply when they have a high MFCR, while funded applicants reapply irrespective of their MFCR. The marginal distribution of the MFCR clearly shows that the MFCR for unfunded applicants is higher than for funded applicants who reapply (shown as a solid line in the marginal plots). Due to some association between quality and citations in our simulation, there is also a smaller but visible difference in the marginal distribution for quality between funded and unfunded applicants who reapply. This illustrates that results similar to an early-career setback effect may appear due to a collider bias, and need not represent a causal effect. Moreover, it also shows that the “conservative removal” procedure does not properly address this collider bias, because there are simply not as many funded applicants as unfunded applicants with equally high MFCRs.

Overview of funders, funding programmes and number of applications per programme.

Review scores for different funding programmes. Scores are ordered from least to most likely to be funded. In some cases, higher review scores are less likely to be funded.

Time between successive applications.

Number of funding applications over time.

Distribution of review scores normalised to the [0,1] range. The peaks are due to some funding programmes not having continuous scores, but being rounded to .5.

Distribution of review scores normalised so that 0 corresponds to the funding threshold.

Comparison of near-hit and near-miss applicants for (a) year the PhD was awarded; (b) year of birth; (c) the number of publications before the early-career application; and (d) the MFCR before the early-career application.

Overall probabilities to receive any later-career funding.

Matthew effect across funders.

Probabilities to receive any later-career funding for near-hit and near-miss applicants.

Matthew effect across funders for near-hit and near-miss applicants.

Probabilities to receive any later-career funding for near-hit and near-miss applicants who reapplied.

Matthew effect across funders for near-hit and near-miss applicants who reapplied.

Inferred effect of previous funding decision on the application rate. The hue shows the effect, while the size shows the precision of the estimates.

Inferred effect of previous funding decision on the review score. The hue shows the effect, while the size shows the precision of the estimates.

Number of reapplications for all applications.

Mean-field citation rate (MFCR) for all applications.

Number of reapplication for near-hit/miss applications.

Mean-field citation rate (MFCR) for near-hit/miss applications.

Early-career setback across funders.

Early-career setback across funders from near-hit/miss applications.

Mean-field citation rate (MFCR) for near-hit/miss applicants who reapplied.

Early-career setback across funders from near-hit/miss applicants who reapplied.

Mean-field citation rate (MFCR) for near-hit/miss applicant who reapplied with conservative removal.

Early-career setback across funders from near-hit/miss applicants who reapplied with conservative removal.

Inferred effect of previous funding decision on the MFCR. The hue shows the effect, while the size shows the precision of the estimates.

Scatter plot illustrating the association between the inferred quality in the Bayesian model, the “Pre” MFCR and the review score.