In 2020 we introduced the NIH BRAIN Initiative’s (Brain Research Through Advancing Innovative Neurotechnologies®) experiment in supporting functional team-research neuroscience at a scale commensurate with the complexity of a mechanistic understanding of how the brain works as perhaps the most complex machine in the universe (David et al, 2020). Here, on the 10^th anniversary of the NIH BRAIN Initiative, progress and impact of the program is presented to better understand our funding efforts in large-scale, team-research neuroscience from fiscal year 2014 (FY2014) through fiscal year 2023 (FY2023).

The team-approach program was initially issued in 2014 as 3-year funding opportunity (U01), Integrated Approaches to Understanding Circuit Function in the Nervous System. In 2017, the program evolved into the larger-scale, multi-component TeamBCP funding opportunity (U19), and in 2018 the original U01 mechanism was repurposed as Exploratory Team-Research BRAIN Circuit Programs – eTeamBCP, an exploratory stage to enable advancement to the larger, more elaborated BCP U19. For FY2014-FY2023, the total expenditures of the 3-year eTeamBCP (including the original ‘Integrated Approaches’ U01 was $118.7M (Table 1); the total expenditures for the TeamBCP U19 was $280.7M (Table 1). Note that these expenditures in BRAIN team-science approaches were balanced with parallel, small-project awards, the Targeted BRAIN exploratory and elaborated programs (R34, R01) at $348M (Table 1). In FY2023, the expenditures in this team-science approach (U01 and U19) in systems and computational neuroscience was $129.4M which was 19% of the total BRAIN Initiative expenditures.

Table 1.

Table 1 summarizes some statistics of the BRAIN exploratory (eTeamBCP) and elaborated (TeamBCP) programs, compared to the BRAIN small-project circuits programs (R34 and R01). We report the total number, cost, and number of publications per funding stream for FY2014 through 2023. As measures of impact, we report the median for the total cost per award and the medians of the total publications, and the citations and Relative Citation Ratio (RCR) per program. The Relative Citation Ratio (RCR) is a normalized index of impact, which is a measure of citations relative to the topic neighborhood for the year of publication (Hutchins et al., 2016). An RCR of 1.0 represents a median citation count relative to that year and topic neighborhood.

Normalized by per dollar investment, the two programs appear to have similar rates of publication. However, the citation and RCR impacts per dollar are higher for the team-based programs, demonstrating the team-research approach to be fiscally efficient.

All TeamBCP and the original Integrative BCP awardees joined two consortia for awardees from all years to explore common challenges and best practices: a leadership consortium and a data science consortium. The leadership consortium produced topic maps of 1) Domains of study (Fig 1A) and, 2) Scales of research with respect to conceptual models (Fig 1B). As investigator-initiated research topics, the total collection of awards covers a broad portfolio of systems and behaviors. The modeling efforts of the collective U19s (David et al., 2020) span broad scales and levels of mechanistic focus.

To measure the success of the peer-reviewed ‘pathway’ of competing award from exploratory to elaborated projects and programs, we looked at the ‘flow’ of the original U01 projects using Sankey plots. The Sankey plot tracks all the original 3-year U01 awards (Fig. 1C, dark blue). Of 17 U01s in 2014-15, 59% went through successful peer-reviewed, competing award to a U19 (Fig. 1C, orange). Of the 20 total TeamBCP U19s, 11 (55%) came from novel (non-U01) sources, including 3 (15%) transitioning from non-NIH funding sources. By including successful competing award to any NIH mechanism (Fig. 1C, yellow), the advancement of the original U01s to any competing NIH award is 76%. By comparison, the success rate (successful award including type 1 new applications and type 2 competing revisions) in 2023 of NIH research project grants (RPG, mostly R21 and R01) was 21.3%. We conclude the funding strategy of exploratory awards that proceed to larger, more elaborated projects through competing peer-review was a highly successful mechanism of continuity for highly meritorious research.

Another measure of team-building success is the confederation of multiple projects to a collective new award. The TeamBCP has 5 examples (20%) of two or more awards converging into one, mostly from non-BRAIN Initiative sources (4 of the 5 examples).

Culture Change and Best Practices

In addition to these productivity measures, the program has facilitated a cultural change of fundamental, quantitative, and investigative systems neuroscience through these multi-disciplinary, team-science approaches (Miller et al., 2024). Below, we offer some useful detail of how this team-research program has addressed our original guiding principles for Brain Circuit programs (David et al., 2020).

• A culture of collaboration: Bridging of disciplines is embedded in the team-research approaches of the multi-component U19 programs. This has brought different disciplines together, which is reflected, in part, by a demographic survey from the 2022 BRAIN Initiative Investigator’s Meeting. This survey of meeting attendees found an emphasis on integration of experimentalists and theorists (Statistics, Computational Biology, Computer Science; 31% of 520 respondents), and research spanning different scales and species (see below). We have also brought in tool developers, usually from physics and/or engineering backgrounds (42%), as well as neuroethicist expertise (5%). All of the U19 awards that include humans in the NIH definition of Clinical Trials (L2L, Osmonauts) had neuroethics components built into peer review. In addition, the topics supported have expanded to include the impact of non-neuronal components on brain circuit dynamics (A-Team, USARhythms, BrainflowzZZ).

  We have supported projects that are approaching complex questions at multiple scales, through bridging of disciplines, and across species. In addition, the high productivity of the TeamBCP projects demonstrate that the impact scales with investment and is well above the normalized RCR = 1.0. The range of the RCR for the top 5 eTeamBCP and TeamBCP awards was 57.3-150.0 and 136.9-218.0, respectively.

  Furthermore, the high-risk approaches such as those leveraged from Team BCP’s awardees can be measured by the high publication rate and a median RCR/$M of 5.86 (Table 1). Exploring this further with Altimetric’s index of what garners the highest media attention reveals an array of studies with high public interest, such as SARS-CoV-2 and anosomia (Brann et al., 2020); the discovery of possible fourth meningeal membrane in the brain that helps protect and bathe the brain (Møllgård et al., 2023); and elucidation of the neural circuitry underlying maternal responses to infant cries (Valtcheva et al., 2023).

• Consortia-level collaboration: The U19 consortia specifically are charged with taking on community interests larger than the individual awards. Despite vast differences in research topics and methods/approaches, the consortia of awardees have built a research community that develops and shares best practices in team management, data science (Schottdorf et al., 2024), and has promoted many confederated efforts across awards. Several U19s with similar research topics exchange data and best practices by joint laboratory meetings (e.g., aABC & SCC, BrainflowzZZ & USARhythms). In 2023, the U19 Data Consortium convened an international Digital Atlas Interest Group on common challenges for digital atlas development. Through the consortia, PIs were introduced to recent insights from the field of Science of Team Science, strengthening connections between theoretical and practical considerations of the practice of team science.

• A change in the workforce: The resource cores offer support for staff scientists and a rich training environment that is producing multi-faceted trainees. We also note the value of program managers and thoughtful processes/policies as key to effective coordination and fair governance – a point particularly well addressed by the highly standardized International Brain Laboratory. Trainees from team science awards are exposed to an expanded universe of scientific collaborations/interactions and have greater access to resources, mentors, and muti-disciplinary expertise available to them (David et al., 2020). However, team science is not without challenges (National Research Council 2015), such as programmatic management of disparate disciplines, authorship policies, and credit assignment (David et al., 2020). Annual site visits diligently watch for concern with these challenges (see below). TeamBCP trainees have recently established a working group to focus on trainee outcomes. This group is addressing best practices for trainee-related concerns and to optimize trainee team-science research experiences. Additionally, NINDS Office of Programs to Enhance the Neuroscience Workforce (OPEN) recently hosted a webinar on perspectives in training in team science. The webinar highlighted the work of a few TeamBCP-supported early career stage scientists and elaborated on the best practices, lessons learned, and lasting impacts of engaging in collaborative research. As well, the U19 data science consortium recognized through their own survey the diverse expertise requirements for creating robust infrastructure and tools for supporting the U19 scientific pipeline from theory to experiment and analysis.

• Spawning new team-science research programs in neuroscience: BRAIN Initiative Team science approaches have inspired innovative non-BRAIN Initiative funding opportunities for team science, notably the NINDS Collaborative Opportunities for Multidisciplinary, Bold, and Innovative Neuroscience (COMBINE) program RM1 program. Distinct from the NINDS P01, and parent R01, this RM1 program draws from experiences with the BRAIN TeamBCP program and emphasizes tight integration across team expertise and approaches.

• Integrated data science and experimental science: The required Data Science Cores of the TeamBCP awards were designed to promote flexible and focused solutions to sharing complex data from diverse disciplines, formats, and scales between component laboratories (Chapuis and Winter, 2024). The TeamBCP site visits (see below) revealed various and unique data sharing challenges and solutions that were effectively addressed by the specific data science expertise and resources of the Data Science Cores. The dedicated personnel with data science expertise were highly successful and ensured resources to solve complex data science challenges without burdening the productivity efforts of the research staff. We find this ‘bottom-up’ approach of project-specific data management complements well ‘top-down’ BRAIN Initiative Data Science and Informatics programs and Data Archives. A self-evaluation by the data science consortium revealed, however, that improvements can be made to adopt more standards for organizing, documenting, processing, evaluating and sharing data and software. Particularly salient for the early-career scientists was how to recruit and incentivize data science efforts that are not as easily measurable in traditional neuroscience publication and citation.

• Site visits: Annual site visits are arranged for the awarded U19s to monitor progress and offer mid-award guidance by an External Advisory Board (EAB). The EAB is composed of an NIH or NSF scientific Program Officer with subject matter expertise, a scientist from other team-based BRAIN awards, and an invited scientist at large. The EAB is charged to help the Program Officers assess progress, offer expert constructive critique to the U19 team, to explore scientific opportunities, and highlight the achievements of the awardees and trainees. These site visits, which specifically emphasize guided self-evaluation through a Strengths, Weaknesses, Opportunities, and Threats (SWOT) format, have proven to be one of the most valuable factors in the TeamBCP program for both the scientific goals and progress management.

• Enabling cross-species comparative studies: BRAIN supports investigative and tool development studies that take advantage of and/or enable the special abilities and power of diverse species and comparative experimental systems to address specific questions. Species of study among the eTeamBCP and TeamBCP awards include Drosophila fruit flies, Danio rerio zebrafish, Mus and Rattus rodents, Rhesus and Marmoset monkeys, Berghia nudebranch, octopus, voles, bats and humans. Through these species, these programs have revealed conserved and diverse mechanisms by which the nervous system produces behavior.

• Combining experimental and computational approaches: The extensive cross-scale experimental modeling in Figure 1B attests to the power of bringing quantitative approaches to experimental neurobiology. Greater integration of quantitative, predictive modeling into research projects has led to more cross-talk between theoretical modelers and experimentalists, and has enhanced collaborations between mathematical and modeling communities (David et al., 2020), revealing the importance of a dynamic interaction between theory and experimentation. Like the NSF/NIH CRCNS program, the engagement of special emphasis review panels for the TeamBCP applications effectively recruited well-tuned expertise for the specific, and sometimes complex, integration of computational and experimental methodologies.

How are we shaping the next generation of scientists?

Based on observations from our TeamBCPs, the BRAIN-awarded trainees, and job advancement from early career investigators, the next generation of scientists are more multi-faceted and can comfortably wear different technology hats and speak multiple scientific languages. From an attendee survey of 722 respondents to 2022 BRAIN Initiative Investigators meeting registration, the percentage of primary or secondary fields was highest for Engineering, Psychology or Behavioral Science, Biochem/Molec/Cellular Biology, Physiology or Systems Biology, and Neuroimaging/Radiology (28% to 18% in descending order), This was followed by Computer Science, Computational Biology, Genetics, Statistics (from 11% to 10% in descending order). Also represented was Clinical Science, Physics, Ethics and Chemistry (from 8% to 3% in descending order). For example, trainees are able to move with ease across the different scales of inquiry, from cellular to systems, to quantitative approaches. The data science consortium found through their self-evaluation that theorists and experimentalists are similarly engaged in data science training, though the level of expertise required to enforce standards in research practice remains high (Schottdorf, 2024). Nonetheless, the culture reflects an openness to innovative and integrative approaches in the neuroscience research workforce.

Looking to the future

By conceptual design, team-research programs are successful to the extent that they foster collective efforts that could not have happened as effectively as individually funded projects. Traditionally, NIH standard study sections review applications with the mindset of a single lab’s success, or competition with one another. But with team science, they are reoriented to think in terms of synergy, co-publication, leveraging collective expertise and resources, and collective capacity to achieve the goals they set out to achieve. We believe the rubric of the funding program description and the review criteria drives an ecosystem of team-research that seeks to cross boundaries of interdisciplinary collaboration at scales beyond the limitations of the “single-principal-investigator (PI)-led projects” common in biology in the past (Olds, 2016). The TeamBCP review criteria emphasize synergy as a qualifying element, and the annual site visits monitor this closely. This report finds that awardees have used the TeamBCP program to engage more adventurous and necessarily-collaborative programs of high impact, on topics that transcend interests of all the ‘neuro’ NIH Institutes. We note that the BRAIN Initiative is part of an effort commissioning NASEM to produce a consensus study about team science and its opportunities and challenges: https://www.nationalacademies.org/our-work/research-and-application-in-team-science.

Conclusions

Following the 10^th year of our experiment in supporting team neuroscience, we offer that the BRAIN TeamBCP has been a leading agent for changing the culture of fundamental neuroscience research to consider collective, team efforts of high impact scientifically and socially. The TeamBCP program offers a successful, collective-research approach that complements other collaborative models in neuroscience (David et al, 2020). We expect that the TeamBCP approach will continue to be an enduring model of bold and effective team-research approaches in fundamental neuroscience discovery at NIH. Useful guidance in how to advance academic implementation of team research approaches can be found in the EMBO reports white paper (van Helden et al., 2024).

Methodology

The budget data for fiscal years 2014 to 2023 of BRAIN TeamBCP were downloaded using an NIH-internal database search engine (Query View Report, QVR) that can recapitulate the public, awarded grant information from a download available from NIH RePORTER as linked in this report. Comparison to ‘small project awards’ were made to a similar download as linked in this report. Bibliometric data were extracted from the publicly accessible iCITE database as a snapshot from the end of fiscal year 2023 and through custom Python scripts by the BRAIN Initiative Office of Budget.

For the BRAIN TeamBCP Circuit Domains of Study and Models, awarded investigators were requested to self-identify the categories that were most representative of their research and were then plotted in accordance with the designations. These diagrams were updated from the figures in previous publication (David et al., 2020).

The Sankey continuity analysis utilized a fingerprint process in QVR that is is equivalent in NIH RePORTER through the “similar projects” tab to find NIH awards that shared prior or subsequent similarity of scientific content by the award investigators of the TeamBCP awards. Non-NIH seed funding was identified from awarded investigators and can be verified publicly.

The qualitative measures of culture change and best practices are cited from BRAIN Initiative resources and publicly available citations from various operations of the TeamBCP awardee consortia, as linked throughout this report.