1. Introduction

Citations are often used as a proxy for how well a researcher disseminates their work, which is important both for spreading knowledge and establishing a scientific reputation (1). Furthermore, citation counts and other metrics like the h-index are critical for hiring and promotion in an increasingly tenuous academic job market (2–4), necessitating a thorough examination of citation practices across research fields. Existing investigations of citation practices have found, for instance, an establishment of unfounded authority through citation distortions (5) and a false inflation of impact factors by specific journals (6). Others have demonstrated under-citation of racial and ethnic minority groups (7) and women (8–10), including three studies specific to Neuroscience literature (7, 8, 10). These examples of citation manipulations and biases underscore the importance of comprehensively investigating citation practices in the broader Neuroscience literature.

Self-citation, or how frequently authors cite themselves, remains an understudied citation practice in the Neuroscience literature. Self-citation can be calculated from two different perspectives: 1) as the proportion of an author’s total citations that come from their own works (11, 12), or 2) as the proportion of an author’s references on which they are also an author (13). Since the former accounts for the total number of times an author cites themselves (across all papers) divided by the total number of citations the author has received, it helps identify when a particular author only accumulates citations from themselves (11). However, in this manuscript we defined self-citation as the latter because one cannot control how much others cite their works. As such, the second definition of self-citation rate may more closely reflect intention in self-citing and will allow for more self-reflection about self-citation practices.

Self-citations may often be appropriate. For example, in a direct follow-up work, a researcher will need to cite their previous work. Yet, h-index can be strategically manipulated via self-citation (14), and some scientists may practice extreme and unnecessary cases of self-citation (11). While certain citation metrics can be adjusted to remove self-citations, the effect of a single self-citation extends beyond adding one additional citation to an author’s citation count. In a longitudinal study of self-citation, Fowler and Aksnes(15) found that each self-citation leads to approximately three additional citations after five years. Given the potential effects of self-citations on various citation metrics that influence career trajectories, a detailed analysis of self-citation rates and trends in the Neuroscience literature could benefit the field.

This work summarizes self-citation rates in Neurology, Neuroscience, and Psychiatry literature across the last 21 years, 63 journals, 157,001 articles, and 8,270,200 citations. We then build upon these calculations by exploring trends in self-citation over time, by seniority, by country, by gender, and by different subfields of research. We further develop two models of self-citation: a first modeling self-citation at the level of each citing/cited pair, and a second modeling highly self-citing articles. Finally, we discuss the implications of our findings in the Neuroscience publishing landscape and share a tool for authors to calculate their self-citation rates: https://github.com/mattrosenblatt7/self_citation.

2. Results

2.1 Data

We downloaded citation information from 157,001 papers from Scopus across 63 different journals representing the top Neurology, Neuroscience, and Psychiatry journals based on impact factor (Table S1) from 2000-2020. These 157,001 papers included a total of 8,270,200 citations.

2.2 Metrics

Using the Scopus database and Pybliometrics API (16), we calculated three metrics for each individual paper: First Author self-citation rate, Last Author self-citation rate, and Any Author self-citation rate, where self-citation rate is defined as the proportion of cited papers on which the citing author is also an author. As an example, consider a hypothetical paper by Author A, Author B, and Author C that cites 100 references.

• ● If Author A is an author on 5 of those references, then the First Author self-citation rate is 5/100=5%.

• ● If Author C is an author on 10 of those references, then the Last Author self-citation rate is 10/100=10%.

• ● If at least one of Author A, Author B, OR Author C is an author on 18 of the references, then the Any Author self-citation rate is 18%.

We will use the above definitions of self-citation throughout the remainder of the paper. Furthermore, our estimations via Python code of the above three metrics showed strong agreement with 906 manually scored articles from a subset of Psychiatry journals (r’s>0.9) (Figure S1).

We performed 1,000 iterations of bootstrap resampling on the article level (i.e., resample articles with replacement) to obtain confidence intervals for all analyses. We additionally performed 10,000 iterations of permutation testing to obtain two-sided P values for all significance tests. All P values are reported after applying the Bonferroni correction, unless otherwise specified.

First, we showed raw trends and group differences in self-citation rates. We then developed a probability model of self-citation that controls for numerous covariates, which allowed us to obtain significance estimates for each variable of interest.

2.3 Self-citation rates in 2016-2020

In the last five years of our dataset (2016–2020), the overall self-citation rates were 4.50% (95% CI: 4.44%, 4.56%) for First Authors, 7.95% (95% CI: 7.89%, 8.02%) for Last Authors, and 14.42% (95% CI: 14.30%, 14.53%) for Any Authors (Table 1). In all fields, the Last Author self-citation rates were significantly higher than that of First Author self-citation rates (P’s<0.004). Neuroscience had a significantly lower self-citation rate than Neurology and Psychiatry for First, Last, and Any Authors (P’s<0.004). We found a significantly lower First Author self-citation rate for Neurology compared to Psychiatry (P=0.004), and significantly higher Last Author and Any Author self-citation rates for Neurology compared to Psychiatry (P’s<0.004). Although there is no clear rule for what levels of self-citation are “acceptable,” a histogram of self-citation rates (Figure 1a) and a table of self-citation percentiles (Table S2) both provide insight into the self-citation levels that are typical in the Neuroscience literature.

Figure 1.

Table 1.

2.4 Temporal trends in self-citation rates

Furthermore, self-citation rates have changed since 2000 (Figure 1). For example, First Author self-citation rates were 6.53% (95% CI: 6.34%, 6.75%) in 2000 and 4.18% (95% CI: 4.07%, 4.29%) in 2020. First Author self-citation rates decreased at a rate of −1.05% per decade (95% CI: −1.11%, −1.00%), Last Author self-citation rates increased at a rate of 0.02% per decade (95% CI: −0.05%, 0.09%), and Any Author self-citation rates increased at a rate of 0.54% per decade (95% CI: 0.43%, 0.64%). Corrected and uncorrected P values for the slopes are available in Table S5. Further details about yearly trends in self-citation rate by field are presented in Figure S2.

2.5 Author seniority and self-citation rate

We also considered that the self-citation rate might be related to seniority. To test this, we calculated each author’s “academic age” as the years between the publication of their first paper (in any author position) and their current paper. For example, if the Last Author of a 2017 paper published their first paper in 1995, their academic age would be 22. We averaged the self-citation rates across each academic age, only including those ages with at least 50 papers in the dataset, and found marked increases in self-citation rate with greater academic age (Figure 2a). For instance, at ten years, the self-citation rate for First Authors is about 5%, while this number jumps to over 10% at 30 years. Academic age appears to be a more robust indicator of self-citation than authorship position, as, for a given academic age, First Author and Last Author self-citation rates are comparable (Figure 2b).

Figure 2.

2.6 Geographic location and self-citation rate

In addition, we used the country of the affiliated institution of each author to determine the self-citation rate by institution country over the last five years (2016–2020). We averaged First Author and Last Author self-citation rates by country and only included countries with at least 50 papers. This analysis is distinct from country self-citation rate because we calculated self-citation at the level of the author, then averaged across countries. In contrast, previous studies have operationalized country self-citation rates as when authors from one country cite other authors from the same country (17). The results are shown on a map of the world using GeoPandas (18) (Figure 3; see Methods) and also presented in table format (Table S4). Self-citation rates in the highest self-citing countries were nearly double that of the lowest for the First and Last Authors. For instance, the First Author self-citation rate in the United States is 4.73%, while in China, it is 2.41%.

Figure 3.

2.7 Self-citation rates by subtopic

We next investigated how self-citation rate varies within subfields of Neuroscience research. Based on Scopus abstract data for papers from 2016-2020, we developed a topic model using latent Dirichlet allocation (LDA). In LDA, each abstract is modeled as a distribution of topics, and each topic contains probabilities for many different words.

We assigned each paper to the topic with the highest probability to determine “subtopics” for each paper. The topic number was chosen as 13 with a parameter search (Figure S4), and results with seven topics are also presented in the SI (Figures S6-7). Based on the most common words of each topic (Figure S5), we assigned 13 overall themes: 1) Aging & development, 2) Animal models, 3) Cellular, 4) Clinical research, 5) Clinical trials, 6) Dementia, 7) Depression & anxiety, 8) Functional imaging, 9) Mechanistic, 10) Pain, 11) Schizophrenia, 12) Social Neuroscience, 13) Stroke. We then computed self-citation rates for each of these topics (Figure 4).

Figure 4.

We generally found that clinical trial research had the highest self-citation rates for First Authors at 7.34% (95% CI: 7.22%, 7.44%), whereas animal research had the lowest self-citation rate at 3.32% (95% CI: 3.28%, 3.36%). For Last Authors, self-citation rates were highest for Dementia research at 10.78% (95% CI: 10.67%, 10.89%) while Social Neuroscience had the lowest self-citation rate at 5.85% (95% CI: 5.79%, 5.90%). For Any Author, Clinical trials once again had the highest self-citation rate at 22.37% (95% CI: 22.19%, 22.55%), and Social Neuroscience had the lowest self-citation rate at 10.03% (95%CI: 9.94%, 10.11%). One possible explanation for the differing self-citation rates by field is a different number of authors per paper, as there were moderate to strong correlations between the number of authors per paper in a given field with self-citation rate (Spearman’s r=0.1484, 0.6648, 0.7747 for First Author, Last Author, and Any Author self-citation rates, respectively; corrected and uncorrected P values are available in Table S5).

2.8 Self-citation by gender

Several previous works have explored gender differences in self-citation practices. King et al. (19) found that men self-cited 70% more than women from 1991-2011, but this work did not control for the number of previous papers that the authors had due to limitations of the dataset. More recent works demonstrated that gender differences in self-citation largely disappear when accounting for the number of possible works an author may self-cite (i.e., number of previous publications) (8, 20, 21). While Dworkin et al. (8) specifically explored citation by gender in the Neuroscience literature, we here expand the analysis to a wider range of journals to better represent field-wide self-citation rates (63 journals in this work compared to five in the previous work).

For each paper, we assigned a probability of a particular name belonging to a woman or a man using the Genderize.io API. We retained only authors with >80% probabilities. There are clear limitations to these types of packages, as described in Dworkin et al. (8), because they assume genders are binary and do not account for authors who identify as nonbinary, transgender, or intersex. As such, the terms “women” and “men” indicate the probability of names being that gender as opposed to a specific author identifying as a man or woman. Despite these limitations, we believe these tools can still help broadly uncover gender differences in self-citation rates.

We calculated the proportion of men and women First and Last Authors since 2000 (Figure 5a). Although the authorship proportions have begun to converge to be equal by gender, the gender disparity among the Last Authors was more notable than among the First Authors. Based on linear fits, we estimated that men and women would be equally represented as First and Last Authors in 2025 (95% CI: 2024, 2026) and 2052 (95% CI: 2050, 2054), respectively.

Figure 5.

In 2020, there were significant differences between First Author self-citation rates of men and women (P<0.004). First authors who were men had average self-citation rates of 5.12% (95% CI: 4.93%, 5.31%), while women authors had average self-citation rates of 3.34% (95%CI: 3.19%, 3.49%), which is significantly different (P<0.004). Similarly, in 2020, Last Authors who were men had significantly higher self-citation rates than those who were women (P<0.004), with self-citation rates of 8.18% (95%CI: 7.99%, 8.36%) and 6.80% (95% CI: 6.56%, 7.04%), respectively.

In addition, men persistently had higher self-citation rates than women since 2000 (Figure 5b), though the gap has slowly decreased. Linear fits were used to estimate that self-citation rates for men and women would be equal for First Authors in the year 2105 (95% CI: 2071, 2171) and equal for the Last Authors in 2053 (95% CI: 2041, 2072). Furthermore, we calculated the ratio of men to women self-citations over the past two decades (Figure 5c). For First Authors, men have consistently cited themselves more than women by 42.5-67.8% depending on the year. Among Last Authors, there was a steep decrease in 2002, but since then, men have cited themselves 17.8-36.5% more than women.

Seniority may account for gender differences in self-citation rate, as there are gender disparities in faculty positions and ranks (22–25). To explore the effect of seniority, we investigated self-citation rates by academic age and gender (2016–2020). Gender disparities for the same academic age emerged early in an academic career and were relatively persistent throughout most of the career (Figure 5d-e). For instance, in the previous five years (2016–2020), there were 12,558 papers by early-career women authors and 14,574 by early-career men authors. Women authors had 781,096 references and 17,836 self-citations (2.28% self-citation rate), while men authors had 884,331 references and 28,560 self-citations (3.23% self-citation rate). This equated to a 41.4% higher self-citation rate for men than women during the first ten years of their careers (P<0.004).

We considered two factors that might contribute to the gender discrepancy in self-citation rate by academic age: the number of papers published for authors of a given academic age, which is greater for men at all career stages (20, 21, 26, 27), and the self-citation rate for a given number of papers. We compared the number of papers for men and women at a given academic age (Figure 5f-g) and found that men had a higher number of papers. This trend started early in the career (academic age<=10 years), where men had significantly more papers than women (P<0.004). For example, at an academic age of 10 years, men were authors on an average of 44 papers, and women authored 30 papers on average. In addition, we divided the number of papers into groups (Figure S3) and computed self-citation rate by gender for each group. Although the effect was small, men had significantly higher self-citation rates for 0-9 papers (P<0.004), 10-19 papers (P=0.0111), 20-29 papers (P=0.0185), and 40-49 (P<0.004) papers. All other differences were not statistically significant. We further investigate the role of gender by adjusting for various other covariates in sections 2.9 and 2.10.

2.9 Exploring confounds and interactions with citing/cited pairs model

Investigating the raw trends and group differences in self-citation rates is important, but several confounding factors may explain some of the differences reported in previous sections. For instance, gender differences in self-citation patterns can be attributed to men having a greater number of previous papers available to self-cite (8, 20, 21). As such, controlling for various author- and article-level characteristics can improve the interpretability of self-citation rate trends. To allow for inclusion of author-level characteristics, we only consider “First Author” and “Last Author” self-citation in these models.

Following the methodology of Mishra et al. (20), we used logistic regression to model the probability of a single citation pair (citing article to cited article) being a self-citation. The terms of the model included several article characteristics (year of citing article, time lag between citing and cited article, document type, number of references, field, and number of authors), as well as author characteristics (academic age, number of previous papers, gender, and affiliation continent). All continuous terms except “year” were positively skewed and thus were transformed with the inverse hyperbolic sine transform (21). Model performance (pseudo-R²) and odds ratios for each predictor are shown in Table 2.

Table 2.

First, we considered several temporal variables. Consistent with prior works (20, 21), the number of previous papers had a strong positive association with self-citation rate, and the time lag had a negative association with self-citation rate. Similarly, increasing academic age was related to higher odds of self-citation. Moreover, the interaction [Last Author x year] term was significant. With each increase in the year, the odds ratio of the Last Author term increased by 1.01, which translates to a 1.19 (∼1.01²⁰) times increase from 2000 to 2020 in the odds of Last Author compared to First Author self-citation.

Then, we considered affiliation continent, our primary geographic predictor. The odds of self-citation for authors affiliated with an institution in Africa, the Americas, Europe, and Oceania were 1.07, 1.29, 1.24, and 1.19 times that of Asia.

Next, in an analysis by field, despite the raw results suggesting that self-citation rates were lower in Neuroscience, the odds of self-citation in Neuroscience were 1.20 times greater than Neurology and 1.10 times greater than Psychiatry. This discrepancy may reflect age differences between authors in Neuroscience versus Neurology and Psychiatry journals in our database, where Neuroscience authors tended to be slightly more junior. The median academic age of Neuroscience authors was 14, compared to 15 for Neurology and Psychiatry, and the median number of previous papers was 36 in Neuroscience, compared to 49 and 48 in Neurology and Psychiatry, respectively.

Finally, our results agreed with previous findings that the odds of self-citation for men and women are nearly equivalent, or even show a slightly reversed effect from what is expected, where the odds of self-citation of men is 0.97 times that of women. Since raw data showed evidence of a gender difference in self-citation that emerges early in the career but dissipates with seniority, we incorporated an interaction term between gender and sinh^-1(academic age) into the model (Table 2). While most other coefficients remained unchanged, the odds of self-citation for men relative to women changed from 0.97 to 1.39. The exponential of the interaction term [Gender(man) x sinh^-1(academic age)] was 0.90. Together, these results suggest that men may self-cite at a greater rate than women early in their careers, but the difference between men and women disappears–and even reverses–in later career stages. A trajectory of the odds ratio of self-citation of men to women as academic age progresses and as the number of previous papers increases is shown in Figure S8a-b.

2.10 Exploring confounds and interactions with highly self-citing model

While modeling factors that affect self-citation at the level of a single reference is important, identifying factors that are associated with “highly self-citing” articles arguably provides greater insight into excessive self-citations. Highly self-citing articles were classified as those with a self-citation rate of 25% or greater, which was 3.3% of articles in our dataset. The 25% threshold was selected based on previous comments indicating that self-citation rates above 25% may warrant further investigation (28). However, we recognize that not all of these articles necessarily reflect excessive self-citation, as self-citation rates of 25% or greater may be appropriate in certain situations. Articles with between 15-25% self-citation rates were discarded, so the model classified between articles with <15% (“typical”) and >25% (“high”) self-citation rates.

Once again, the number of previously published works had a strong relationship with extreme self-citation. Notably, highly self-citing articles were significantly more likely in the Americas (odds=2.17), Europe (odds=1.61), and Oceania (odds=1.52) compared to Asia (odds=1). Furthermore, odds of high self-citing articles were greater in Neuroscience (odds=1.70) but less in Psychiatry (odds=0.77) as compared to Neurology (odds=1). The odds of Last Author high self-citation were 0.42 times that of First Authors, but the exponential of the interaction term was 1.02, which, after 20 years, equated to a 1.52 times increase in odds for Last Authors relative to First Authors. Finally, the odds of highly self-citing articles from men were 0.96 times that of women. However, when including a [Gender(Man) x sinh^-1(Academic age)] interaction term, the odds of men having highly self-citing articles compared to women increased to 4.66. The exponential of the interaction term was 0.66, reflecting a decrease and potential reversal of the trend in later career stages. As in section 2.9, a trajectory of the odds ratio of highly self-citing articles for men and women as academic age progresses and as the number of previous papers increases is shown in Figure S8c-d.

2.11 Self-citation code

We provide code for authors to evaluate their own self-citation rates at the following link: https://github.com/mattrosenblatt7/self_citation. Please note that this code requires access to Scopus, which may be available through your institution. The code may also be adapted for journal editors to evaluate the author self-citation rates of published articles in their journal. Further details about the outputs of the code are described in Figure S9 and Figure S10.

3. Discussion

This work analyzed self-citation rates in 157,001 peer-reviewed Neurology, Neuroscience, and Psychiatry papers, with over eight million total citations, to dissect factors that contribute to self-citation practices. First, self-citation rates over the past two decades increased for Any Authors (by 1% since 2000), remained constant for Last Authors, and decreased for First Authors (by 2% since 2000). Even when controlling for other covariates, since 2000, the odds of Last Author self-citations and highly self-citing articles are increasing relative to those of First Authors. Second, we characterized differences in self-citation by country. When breaking down the data by continent, authors who were affiliated with North or South American institutions had the highest odds of self-citation and of publishing highly self-citing articles. Third, men cited themselves more than women in the early career stage. Also, men tend to self-cite more and be more likely to publish “highly self-citing” papers early in their careers and less later in their careers. Fourth, differences in self-citation rate in three major fields (Neurology, Neuroscience, Psychiatry) as well as 13 different subfields exist, suggesting that there is no universal level of “typical” or “acceptable” self-citation, but rather it varies by field.

3.1 Temporal trends in self-citation rates

Increasing collaborations and expanding author lists in recent years likely explains the increase in Any Author self-citation rates. A more concerning trend is the decrease in First Author relative to Last Author self-citations since 2000. The decreasing age of First Authors and increasing age of Last Authors since 2000 partially explains the observed differences. Still, even when accounting for age and seniority, the significant [author position x year] interaction term suggests that the self-citation tendencies of Last Authors are increasing relative to First Authors since 2000. In the Neurosciences, First Authors are typically early-career researchers (e.g., graduate students, postdoctoral fellows) who perform the majority of the experiments and analysis, whereas Last Authors are typically professors who oversee the project and secure funding. As a result, these changes in citation practices could make it harder for early-career scientists to advance in their academic careers, warranting further investigation and monitoring. Another possible explanation is that an increasing number of early career researchers are moving away from academia (29). Thus, early-career researchers may be less incentivized to self-promote (e.g., self-cite) for academic gains compared to 20 years ago.

Furthermore, our analysis of academic age provided more evidence of differences between early- and late-career researchers’ self-citation practices. The finding of increasing academic age and number of papers relating to increased self-citation rates is not surprising because, as one continues in their career, they contribute to more papers and are more likely to cite themselves. In addition, researchers may often become more specialized throughout their career, which may necessitate higher self-citation rates later in the career. However, these results demonstrate a “snowball effect,” whereby senior authors continually accumulate a disproportionate number of self-citations. For example, an author with 50 years of experience cites themselves approximately twice as much as one with 15 years of experience on average. Both authors have plenty of works that they can cite, and likely only a few are necessary. As such, we encourage authors to be cognizant of their citations and to avoid excessive self-citations.

3.2 Geographic differences in self-citation rates

Odds ratios by continent for the citing/cited pairs model were significant and moderate in effect size. For example, authors affiliated with the Americas had 1.28 times the odds of self-citation as authors affiliated with Asia. Much larger effect sizes were observed for “high self-citation,” where authors affiliated with the Americas had 2.65, 2.17, 1.35, and 1.43 times the odds of publishing a highly self-citing article as authors affiliated with Africa, Asia, Europe, and Oceania, respectively. Several possible explanations for such drastic differences in odds exist, including broader cultural differences or academic culture differences. For instance, an analysis of management journals previously found that self-citation rates of authors from individualist cultures were higher than that of authors from collectivist cultures (30). In addition to broader cultural norms affecting the tendency to self-cite, differences in academic norms likely play a major role as well. Researchers in the United States, for instance, rated their feeling of pressure to publish papers higher compared to other countries (31). Pressure to publish stems from pressure to advance one’s career. Similar pressures that vary by geographic region may drive researchers to unnecessarily self-cite to improve their citation metrics and make them more competitive candidates for hiring, promotion, and funding.

While hiring and promotion almost universally depend on citation metrics to some extent, an example of a recent policy in Italy demonstrates how rules regarding hiring and promotion can influence self-citation behavior. This policy was introduced in 2010 and required researchers to achieve certain citation metrics for the possibility of promotion, which was followed by increases of self-citation rates throughout Italy (32). Ideally, authors, journals, and policymakers would work together to establish self-citation guidelines and discourage a “game the system” mindset. However, requiring all institutions and countries to follow similar values regarding citation metrics is not practical, so awareness of possible differences in metrics by geographic region due to self-citation differences is the next best alternative.

3.3 Field differences in self-citation rates

Initially, it appeared that self-citation rates in Neuroscience are lower than Neurology and Psychiatry, but after controlling for various confounds, the self-citation rates are higher in Neuroscience. This discrepancy likely emerges because authors in Neuroscience journals in our dataset tended to be more junior (fewer number of previous papers, slightly lower academic age) compared to Neurology and Psychiatry, giving the illusion of lower field-wide self-citation rates. The field-wide differences in self-citation rate likely depend on both necessity and opportunity. In some research fields, a researcher may need to reference several of their previous works to properly explain the methodology used in the present study, thus having a high necessity of self-citation. Depending on the nature of the work across various fields, researchers may publish more or less frequently, which will affect their number of previous works and thus their opportunity to self-cite.

In addition, while not included in the model to limit the number of terms, the 13 subtopics under examination had different raw self-citation rates, and “acceptable levels’’ of self-citation may vary depending on the subfield. For example, clinical trials had the highest self-citation rate, which may relate to the relatively high number of authors per paper in clinical trial research or the fact that clinical trial research often builds upon previous interventions (e.g., Phase 1 or 2 trials). Overall, these field and subfield differences highlight the importance of editors and researchers understanding common self-citation rates in their specific fields to ensure that they are not excessively self-citing.

3.4 Self-citation rates by gender

The higher self-citation rate of men compared to women without considering any other confounds aligns with previous self-citation literature (8, 19–21). Similar to prior works (8, 20, 21), we found that the largest difference in self-citing is explained by the number of previous papers (i.e., number of citable items) as opposed to differences in self-citation behavior itself. This result overall points toward a more general underrepresentation of women in science, such as in publication counts (26, 27), collaboration networks (33, 34), awards (35), editorial boards (36), and faculty positions (37–39). In other words, women have a lower self-citation rate than men in the Neuroscience literature because they are not given the same opportunity, such as through prior publications, to self-cite.

Still, even when the opportunity to self-cite is controlled for, our model suggests that men self-cite more than women early in the career, though this trend disappears and reverses later into one’s career. Azoulay and Lynn (21) found a somewhat similar trend, where their linear probability model shows that being a woman is associated with 0.5% lower self-citation rates when accounting for an interaction between gender and seniority (Azoulay Table 3). Establishing field-wide influence and scientific prominence may be most crucial in early career stages, since soon thereafter decisions will be made about hiring, early-career grants, and promotion. Furthermore, the same trend holds for “highly self-citing” articles, where men are much more likely to publish such an article in their early career compared to women. A major remaining question is whether gender differences in early-career self-citations manifest into differences in career trajectories. Azoulay and Lynn (21) did not find any gender differences in returns to self-references, as measured by future productivity and awards, in 3,667 life scientists who received a prestigious postdoctoral fellowship. Yet, those results may not generalize due to ascertainment bias in the sample (i.e., including only highly-achieving researchers in the life sciences)(21) and lack of consideration of the possible career benefits of publishing “highly self-citing” articles. Investigating the nature and effects of researchers who tend to publish “highly self-citing” articles early in their careers is a promising technique that may provide insight into the differences in academic career trajectories between men and women in future studies.

3.5 Limitations

There were several notable limitations of this study. First, our analyses were restricted to the top-ranked Neurology, Neuroscience, and Psychiatry journals, and the generalizability of these findings to a wider variety of journals has yet to be determined. Citations of a journal’s articles directly affect the journal’s impact factor. As such, it is possible that the selection of journals based on high impact factor skews the results toward higher self-citation rates compared to the entire field of Neuroscience. Second, we calculated differences between Neurology, Neuroscience, and Psychiatry journals by assigning each journal to only one field (Table S1). As some journals publish across multiple fields (e.g., both Neuroscience and Psychiatry research), this categorization provides a gross estimate of differences between fields. Third, we reported averages of self-citation rates across various groups (e.g., academic ages), but there is a wide inter-author and inter-paper variability in self-citation rate. Fourth, as described above, we evaluated gender differences with gender assignment based on name, and this does not account for nonbinary, transgender, or intersex authors. Fifth, selecting subtopics using LDA was subjective because we assigned each subtopic name based on the most common words. Sixth, our modeling techniques are not useful for prediction due to the inherently large variability in self-citation rates across authors and papers, but they instead provide insight into broader trends. Finally, our analysis does not account for other possible forms of excessive self-citation practices, such as coercive induced self-citation from reviewers (40). Despite these limitations, we found significant differences in self-citation rates for various groups, and thus we encourage authors to explore their trends in self-citation rates.

3.6 Self-citation policies

According to The Committee on Publication Ethics (COPE), “citations where the motivations are merely self promotional…violates publication ethics and is unethical” (41). Excessive and unnecessary self-citations can possibly be limited by using appropriate citation metrics that cannot be easily “gamed” (32, 40). Furthermore, while COPE suggests that journals and editors should make policies about acceptable levels of self-citation (41), many journals have no such policy. For example, only 25% of General Surgery (42) and 14.3% of Critical Care (43) journals had policies regarding self-citation, most of which were policies discouraging “excessive” or “inappropriate” self-citations. Although the self-citation policies in the investigated journals had no significant effect on self-citation rate (42, 43), a more appropriate consideration might be whether these policies significantly reduce excessive self-citations. Self-citation practices are not typically problematic, but excessive self-citations may falsely establish community-wide influence (44). As such, we believe that the “highly self-citing” model presented in this work could serve as a useful guide in identifying potential cases of excessive self-citation. In practice, there should be more nuance than a binary threshold of acceptable/unacceptable levels of self-citation, as some fields may have atypical self-citation patterns (44) or specific articles may require high levels of self-citation.

3.7 Conclusions

Overall, we identified trends in self-citation rates by time, geographic region, gender, and field, though the extent to which this reflects an underlying problem that needs to be addressed remains an open question. We do not intend to argue against the practice of self-citation, which is not inherently bad and in fact can be beneficial to authors and useful scientifically (15, 40). Yet, self-citation practices become problematic when they differentially benefit various groups or are used to “game the system.” We hope that this work will help to raise awareness about factors influencing self-citation practices to better inform authors, editors, funding agencies, and institutions in Neurology, Neuroscience, and Psychiatry.

4. Methods

We collected data from the 25 journals with the highest impact factors, based on Web of Science impact factors, in each of Neurology, Neuroscience, and Psychiatry. Some journals appeared in the top 25 list of multiple fields (e.g., both Neurology and Neuroscience), so 63 journals were ultimately included in our analysis.

4.1 Dataset collection

The data were downloaded from Scopus API in 2021-2022 via http://api.elsevier.com and http://www.scopus.com. We obtained information about research and review articles in the 63 journals from 2000-2020. We downloaded two sets of .csv files: 1) an article database and 2) a reference database. For each year/journal, the article database contains last names and first initials of the authors, title, year, and article EID (a unique identifier assigned by Scopus) of all research and review articles. The reference database contains the same information for all articles referenced by any article in the article database.

4.2 Python code using Pybliometrics API

We used the Pybliometrics API (16) to access citation information for each entry in the article database. First, we used the article EID to retrieve a detailed author list, which included full names and Scopus Author IDs, and a list of references for each article. The list of references included author names with the format “Last Name, First Initial.” To reduce the number of calls to the API, we only retrieved more information about references that included the author of interest by last name and first initial. For all these references with matching “Last Name, First Initial,” we retrieved more detailed information, including full author names and Scopus Author IDs.

To count as a self-citation, we required that the Scopus Author IDs matched exactly or that the first and last names matched exactly. While we recognize that some authors may have the exact same first and last names, the vast majority of self-citations with matching names will be correctly counted. If both Scopus Author IDs and full names were not available on Scopus, then we matched only based on last name and first initial.

Our self-citation metrics included First Author, Last Author, and Any Author self-citation rates. For First (Last) Author self-citation rates, we computed the proportion of reference papers on which the citing First (Last) author is also an author. We considered papers with only a single author as both First Author and Last Author self-citations. For Any Author self-citation rates, we found the proportion of papers for which at least one of the citing authors (any authorship position) was also an author. For the analyses in this paper, we reported total (or weighted average) self-citation rates for different groups. For example, in Figure 1, the reported self-citation rate for the year 2000 is the total number of self-citations in 2000 across all papers divided by the total number of references in 2000 across all papers.

Other data we collected from Scopus and Pybliometrics included the affiliation of the authors, the number of papers published by the First and Last Authors before the current paper, and academic age of the First and Last Authors, which we defined as the time between the author’s first publication and their current publication.

4.3 Country affiliation

For both First and Last Authors, we found the country of their current institutional affiliation. We then calculated the total First Author and Last Author self-citation rate by country, only including countries that had at least 50 First Author or Last Author papers in these select journals from 2016-2020. We then projected the self-citation rates onto a map using Geopandas (18), specifically using the map with coordinate systems EPSG:6933 (https://epsg.io/6933).

4.4 Topic modeling

Latent Dirichlet Allocation (LDA) (45, 46) was implemented with the Gensim package (47) in Python. LDA is a generative probabilistic model that is commonly used in natural language processing to discover topics in a large set of documents. In LDA, each document is modeled as a distribution of latent topics, and each topic is represented as a distribution of words. Based on the data provided, in this case abstracts from all articles in our dataset from 2016-2020, the model finds distributions of topics and words to maximize the log likelihood of the documents. Further details about LDA are available in (45–47).

For our implementation, we first removed all special characters and numbers from the abstract data. Then, we lemmatized the words using the Natural Language Toolkit (48). We excluded words that appeared in less than 20 documents, words that appeared in over 50% of the documents, common stop words (e.g., “the”, “you”, etc.), and some additional words that we felt would not meaningfully contribute to the topic model (e.g., “associated”, “analysis”, “effect”, etc.). In addition, we allowed for bigrams (two consecutive words) and trigrams (three consecutive words) in the model, as long as they appeared at least 20 times in the dataset. Our total corpus included 41,434 documents with 16,895 unique tokens (words + bigrams + trigrams). We used 90% of the corpus to train our LDA model, and left out 10% to evaluate the perplexity, where a lower perplexity demonstrates better performance, as described in (45). For the a-priori belief on document-topic distribution, we used Gensim’s “auto” option. We trained models with a number of topics ranging from 2-20, passing through the entire train corpus 30 times for each number of topics we evaluated. The number of topics was picked based on two evaluation metrics. First, we selected 13 topics as the topics that seemed most meaningful, as assessed qualitatively by word clouds for each topic. Second, we selected seven topics as the number of topics with the lowest validation perplexity.

Finally, we assigned each paper a discrete topic by choosing the topic with highest probability. Since we do not necessarily care about the generalization of this model and are instead using it to determine topics of a specific set of papers, we determined topics on the same data on which the model was trained.

4.5 Name gender probability estimation

To compute gender probabilities, we submitted given names of all First and Last Authors to the Genderize.io API. Each name was assigned a probability of a name belonging to a woman or man, and we only used names for which Genderize.io assigned at least an 80% probability. Details about the Genderize.io database used to calculate probabilities is available at this link: https://genderize.io/our-data.

There are clear limitations to probabilistically assigning genders to names with packages such as Genderize.io, as described in (8), because they assume genders are binary and do not account for authors who identify as nonbinary, transgender, or intersex. As such, the terms “women” and “men” indicate the probability of a name being that gender and not that a specific author identifies as a man or woman. However, these tools are still useful to explore broad trends in self-citation rates for women and men.

4.6 Self-citation rate for a particular author

We also calculated the self-citation rate for a particular author, in this case Dustin Scheinost, in Figure S9. Here, we defined Scheinost-Scheinost self-citation rates as the proportion of references with Dr. Scheinost as one of the authors. Notably, Dr. Scheinost can be in any author position on the citing or cited article. In Figure S9c, we calculated the Any Author self-citation rate for all of Dr. Scheinost’s papers.

4.7 Confidence Intervals

Confidence intervals were computed with 1000 iterations of bootstrap resampling at the article level. For example, of the 157,001 articles in the dataset, we resampled articles with replacement and recomputed all results. The 95% confidence interval was reported as the 2.5 and 97.5 percentiles of the bootstrapped values.

4.8 P values

P values were computed with permutation testing using 10,000 permutations, with the exception of regression P values and P values from model coefficients. For comparing different fields (e.g., Neuroscience and Psychiatry) and comparing self-citation rates of men and women, the labels were randomly permuted to obtain null distributions. For comparing self-citation rates between First and Last Authors, the first and last authorship was swapped with 50% probability for each paper.

In total, we made 40 comparisons (not including the models of self-citation). All P values described in the main text were corrected with the Bonferroni correction. With 10,000 permutations and 40 tests, the lowest P value after correction is P<0.004, which indicates that the true point would be the most extreme in the simulated null distribution. Further details about each comparison and P values can be found in Table S5.

4.9 Exploring confounds and interactions with citing/cited pairs model

For these analyses, we saved a dataset with every possible combination of citing/cited articles. For each, we had a binary indicator for first authors and for last authors, where a “1” reflected the presence of self-citation. Unlike previous sections, we required that the Scopus Author IDs exactly matched and did not count citations where only names matched as self-citations. In addition, papers were excluded when data were possibly incorrect. For example, articles with authors with an academic age of 90 or greater were excluded, and citing/cited pairs with a time lag of more than 150 years were excluded. Then, after removing all articles with only one author and additional entries with missing values 12,898,465 citing/cited pairs remained (counting separately for First Authors and Last Authors).

Next, we developed a logistic regression model, with a dependent variable being a binary outcome as to whether a specific citing/cited pair includes a self-citation. The predictor variables included several article characteristics and author characteristics. The article characteristics were document type (article/review), year of the citing article, time lag (year of cited minus year of citing), number of citing authors, number of citing references, and field of study (Neurology, Neuroscience, or Psychiatry). The author characteristics were academic age, number of prior papers, affiliation continent, gender, and author position (First Author or Last Author). Interaction terms included an interaction between the citing year and author position, as well as an interaction between gender and academic age. All continuous variables were transformed with the sinh^-1 transform, as in (21), which is similar to the logarithmic transform but allows for zero-values. To implement logistic regression, we used the logit model in the statsmodels Python package version 0.13.2 (49). Logistic regression results were reported as the exponential of the coefficients (i.e., exp(𝛽)), which can be interpreted as the odds ratio for all non-interaction terms.

4.10 Exploring confounds and interactions with highly self-citing model

We followed similar steps as above to build models of highly self-citing articles. In this case, we included only articles with at least 30 references. This exclusion was designed to remove shorter articles with a small number of self-citations but also a small number of references, thus leading to an inflated self-citation rate. The dependent variable was whether an article was considered “highly self-citing”. Highly self-citing articles were deemed as those with a self-citation rate of 25% or greater. The 25% threshold was selected based on previous comments indicating that self-citation rates above 25% may warrant further investigation (28). Articles with self-citation rates between 15-25% were discarded, and articles with self-citation rates less than or equal to 15% were considered “not highly self-citing.” After removal of articles with 15-25% self-citation rates and additional exclusion for missing values, 224,210 articles remained (counting First Authors and Last Authors separately). The predictor variables were all the same as above in section 4.9, except time lag was excluded because it is not possible to evaluate when considering self-citation rates at the level of an article.

4.11 Data Availability

The dataset is unavailable due to data-sharing restrictions. However, you may use our code, with appropriate access to Scopus, to gather self-citation data about yourself: https://github.com/mattrosenblatt7/self_citation.

4.12 Code Availability

The code used in our primary analyses of 63 journals in Neurology, Neuroscience, and Psychiatry is available at: https://github.com/mattrosenblatt7/self_citation. Steps to download the necessary data are also included on the GitHub page.

4.13 Citation Diversity Statement

Recent work in several fields of science has identified a bias in citation practices such that papers from women and other minority scholars are under-cited relative to the number of such papers in the field(7–10, 50–54). Here we sought to proactively consider choosing references that reflect the diversity of the field in thought, form of contribution, gender, race, ethnicity, and other factors. First, we obtained the predicted gender of the First and Last Author of each reference by using databases that store the probability of a first name being carried by a woman(8, 55). By this measure (and excluding self-citations to the First and Last Authors of our current paper), our references contain 12.53% woman(first)/woman(last), 19.27% man/woman, 13.17% woman/man, and 55.03% man/man. This method is limited in that a) names, pronouns, and social media profiles used to construct the databases may not, in every case, be indicative of gender identity and b) it cannot account for intersex, non-binary, or transgender people. Second, we obtained predicted racial/ethnic category of the First and Last Author of each reference by databases that store the probability of a first and last name being carried by an author of color(56, 57). By this measure (and excluding self-citations), our references contain 7.46% author of color (first)/author of color(last), 17.45% white author/author of color, 14.81% author of color/white author, and 60.29% white author/white author. This method is limited in that a) names, Census entries, and Wikipedia profiles used to make the predictions may not be indicative of racial/ethnic identity, and b) it cannot account for Indigenous and mixed-race authors, or those who may face differential biases due to the ambiguous racialization or ethnicization of their names. We look forward to future work that could help us to better understand how to support equitable practices in science.

Supplement

S1. All journals included in these analyses

Table S1 shows all 63 journals included in our dataset. We categorized each journal as belonging to Neurology, Neuroscience, or Psychiatry. While we recognize that some journals belong to overlapping fields (e.g., Neurology and Neuroscience), we attempted to select the most relevant field for each journal.

Table S1.

S2. Manual scoring and self-citation percentiles

We manually scored the self-citation rates of 906 articles and compared them to the output of our code.

Figure S1.

In addition, amongst all papers in the dataset from 2016-2020, we computed percentiles of self-citation rates.

Table S2.

S3. Temporal trends in self-citation rate by field

We repeated the analysis in Figure 1b after separating the papers into Neurology, Neuroscience, and Psychiatry. In addition, correlations and slopes between year and self-citation rate are reported in Table S3. Notably, Last Author and Any Author self-citation rates are increasing in Neurology and Psychiatry but decreasing in Neuroscience.

Figure S2.

Table S3.

S4. Self-citation rates by country

Table S4.

S5. Comparison of self-citation rates by gender for a given number of papers

We categorized authors based on the number of previous papers they had at the time of publication. We then evaluated the self-citation rates by the number of papers for women and men. This included a binned evaluation (Figure S3a) and an evaluation using a moving average window (Figure S3b).

Figure S3.

S6. Latent dirichlet allocation

Figure S4.

Figure S5.

Figure S6.

The results for self-citation rates with seven topics show similar trends as the results for 13 topics. For example, both Clinical trials and Dementia have high self-citation rates whether using seven or 13 topics.

Figure S7.

S7. Self-citation rates models

The below plots show relative odds of self-citation of men to women as academic age and number of papers vary, assuming all other terms at their reference values.

Figure S8.

S8. Self-citation tool

Along with evaluating self-citation rates by topic, we also investigated self-citation rates for a particular author, in this case Dustin Scheinost. Dr. Scheinost permitted us to use his name and self-citation data in this work. We show a histogram of self-citations by paper (Figure S9a), the self-citation rates over time (Figure S9b), and the histogram of Any Author self-citation rates for all of Dr. Scheinost’s papers (Figure S9c).

Figure S9.

Self-citation rates for particular authors may be of interest for authors to evaluate and regulate their self-citations and to better understand individual trajectories in self-citation rates. Furthermore, these methods can be extended to evaluate self-citation rates at the level of a country, institute, or journal. For instance, we compared self-citation rates in Nature Neuroscience to the overall field of Neuroscience (Figure S10). In general, Last Author and Any Author self-citation rates were higher in Nature Neuroscience compared to the field. First Author self-citation rates used to be lower in Nature Neuroscience (e.g., Year 2000) but are now approximately equal to that of the field.

Figure S10.

S9. Summary of all comparisons

Table S5.