Research Culture: A survey of early-career researchers in Australia
Abstract
Early-career researchers (ECRs) make up a large portion of the academic workforce and their experiences often reflect the wider culture of the research system. Here we surveyed 658 ECRs working in Australia to better understand the needs and challenges faced by this community. Although most respondents indicated a ‘love of science’, many also expressed an intention to leave their research position. The responses highlight how job insecurity, workplace culture, mentorship and ‘questionable research practices’ are impacting the job satisfaction of ECRs and potentially compromising science in Australia. We also make recommendations for addressing some of these concerns.
Introduction
Advances in science, technology, engineering, mathematics and medicine (STEMM) have revolutionized virtually every facet of modern life. In Australia the government is relying on further advances in these fields to underpin future economic prosperity (Innovation and Australian Government, 2017). Australia has also become the largest provider of education to international students in the Organization for Economic Co-operation and Development (OECD) nations (Sá and Sabzalieva, 2018), with a rise in the number of PhD students accounting for a large portion of this increase: indeed, the number of students (domestic and international) completing a PhD in 2019 was more than twice the number for 2000 (McCarthy and Wienk, 2019).
Two international surveys conducted in 2015 (Ghaffarzadegan et al., 2015) and 2017 (Woolston, 2017) indicated that nearly 78% and 75% of PhD candidates, respectively, aspired to obtain a job in academia, despite the global lack of such job opportunities. Not all PhD graduates need work in academia, but the advanced industries that typically employ highly skilled workers are less developed in Australia than, say, the United States or Germany (Christopherson et al., 2014; Weller and O'Neill, 2014). Australian graduates are therefore more dependent on academia as an employer than graduates from other OECD nations. A previous survey of 284 postdoctoral researchers in Australia revealed that more than half (52%) took their position hoping to transition to a full-time research role in academia (Hardy et al., 2016). The majority of respondents (54%) felt that structural, rather than personal limitations would prevent them from realizing a long-term research career. In addition to concerns about the international so-called ‘glut’ of PhD students (Woolston, 2017; Woolston, 2014; Woolston, 2019) there have been concerns about the reproducibility of published findings in a number of research areas (Baker, 2016; Begley and Ellis, 2012).
Early-career researchers (ECRs) represent the transition stage between PhD and senior academic positions, and their well-being provides insight into the health of the industry. In this study, we surveyed ECRs in STEMM disciplines in Australia to better understand the pressures impacting them and their career development. We defined ECRs as being less than 10 years since PhD completion, similar to the definition used by the Global Young Academy in their study of how to best support young scientists on a global scale, and another important survey of the STEMM workforce conducted in Australia (Pain, 2014; Bell and Yates, 2015). Data were collected from respondents employed in research institutions or universities via an on-line survey (n = 658), which was developed based on previously published questions and through focus group discussions (Supplementary file 1). From our survey we quantified job satisfaction, likelihood of continuing to work in research in Australia, views on mentoring and career planning, and observation of questionable research practices (see material and methods for more detail on how topics were selected for the survey).
In addition to fraud, John et al., popularised the notion that questionable research practices included less egregious practices, such as data exclusion, may in fact be more prevalent and more damaging to the academic enterprise (John et al., 2012). Their findings warned that the frequency of questionable practices may be so prolific that they are becoming ‘the norm’ in research. Note that we did not define ‘questionable research practices’ in our survey. However, our data suggests that the systemic pressures compromising the training and career progression of ECRs in Australia may also contribute to a decline in research quality. It is time to carefully consider if the support and career advancement options available to ECRs in STEMM subjects is aligned with Australia’s scientific aspirations. As many of the documented pressures highlighted in this study are common global problems, these data likely highlight important considerations relevant to the international research community.
Results
Demographic of respondents
Of the 658 respondents, 65.8% identified as female and 34.2% as male. The two most common age brackets were 31–35 years old (42.7%) and 36–40 years old (25.9%), with most respondents having completed their PhD 2–4 years earlier (37.8%) or 5–7 years earlier (25.3%). The four most common countries of birth were Australia (50.6%), England (6.2%), India (4%) and China (2.5%). Of the respondents, 48% identified as being in the medical and health sciences and most (63.2%) were employed in a research only position. Recent data from the Australian Research Council (ARC) indicates that 38.9% of Australia’s STEMM workforce is employed in the medical and health sciences (Table 1; Australian Research Council, 2019). Comparison of our survey demographics with this ARC data indicates that our sample and the target population were not statistically different by discipline (chi square = 16.344, df = 9, p=0.06), and our survey population can be considered representative. A more detailed summary of respondent demographics is provided in Figure 1.
Characteristics that influence ECR job satisfaction
We attempted to identify workplace characteristics that influenced ECR job satisfaction and career progression. Using a non-biased approach, we used text responses to generate a word cloud (Figure 2), and tabulated the most common words associated with satisfaction or dissatisfaction in the context of the responses to survey question 76 (Why do you stay?). Respondents almost universally noted their ‘love’ of research and the job fulfilment it provides (Figure 2—source data 1 and Table 2). ECRs reported that they derived fulfilment from research, mentoring, teaching and the general sense that they are making a meaningful contribution to society, while job security and challenges associated with the job remain major concerns (Table 2). One respondent said, ‘I love it! I am passionate about my work and driven to make a difference. I will keep going as long as I can’.
Satisfaction with workplace culture and intention to leave academia
We queried ECRs regarding satisfaction with their workplace culture. Academic workplace culture, which encompasses interactions between colleagues and professional norms (Faulkner, 2009), has evolved with corporate pursuits and hypercompetitive funding environments (Edwards and Roy, 2017). Figure 3 shows that 51.0% of respondents indicated that they were satisfied or very satisfied with their workplace culture, while a concerning 31.9% were somewhat or very dissatisfied with their workplace culture. Overall, the survey data indicated that the most significant barrier to job satisfaction and career advancement was job insecurity (48.9%). A poor workplace culture (31.9% dissatisfied or very dissatisfied), lack of support from institutional superiors (60.1% a problem or significant problem), poor leadership and management (33.1% dissatisfied or very dissatisfied), and lack of recognition (22.6% disagree or strongly disagree they feel valued) and lack of work life balance (38.1% disagree or strongly disagree they can manage the demands of work and home life) were further major influences compromising satisfaction. When respondents positively described job satisfaction, this was associated with responses indicating good leadership and management (47.3% were satisfied or very satisfied), feeling valued (61.9% agree or strongly agree they feel valued) and confidence in job prospects (35% agree or strongly agree they have good job prospects).
Some respondents indicated that attitudes to their gender (13.1% reported a problem or major problem), ethnic background (5.8%) or sexual orientation (1.4%) may have impacted their career advancement. However, when we made detailed queries regarding workplace challenges detailed in Question 36, we observed effectively no significant difference in the responses from male or female researchers, nor from researchers who were Australian born or non-Australian born. We were unable to draw conclusions regarding the impact sexual orientation might have on workplace challenges, as we did not ask respondents to identify their sexual orientation other than with respect to their living arrangements.
Those who were more than 4 years post-PhD were less likely to be satisfied with their job (55.7%) compared to those who were 4 years or less post-PhD (66.9%). Similarly, those who were more than 4 years post-PhD tended to indicate a higher frequency of being negatively impacted by lack of support from institutional supervisors (increase of 13.4%), questionable research practices of colleagues within their institution (increase of 14.5%), and harassment based on power position (increase of 5.6%). In addition, less than 40% express satisfaction with leadership and management in their workplace compared to 53.4% of respondents who are less than 4 years post-PhD. In responses to Question 74–1 ‘This is a poor time for any young person to begin an academic career in my field’, more senior postdoctoral researchers indicated that this was not a good time to be in science, and were less willing to recommend science as a career (73.2% compared to 58.5% of junior researchers).
We compared our survey respondents’ satisfaction data with previous survey data from academics in Australia (Supplementary file 2; Bell and Yates, 2015; Coates et al., 2009; Bexley et al., 2011; NTEU State of the Uni Survey, 2017). Each of these studies used one or more of the ‘job satisfaction’ questions from our survey in their own survey of the academic workforce in Australia. It can be seen respondents from the current study are more concerned about job security than respondents in any of the other studies. Our respondents also indicated a higher level of personal stress (52%) than those in all the other studies (28%–43%) and agree most strongly (65%) ‘this is not a good time for any young person to aspire to an academic career’. Their reported job satisfaction is low (62%). The combination of answers to these questions for current ECRs relative to those for the other studies indicates a situation about which there should be grave concern.
Previous studies have identified diversity and inclusion as factors that have impact on senior academics’ dissatisfaction (Zimmerman et al., 2016; Professionals Australia, 2014), including the career progression for female academics (Potvin et al., 2018; Else, 2019; Gewin, 2018). However, in our survey of ECRs working in Australia, most identified as satisfied or at least unconcerned, regarding discrimination with respect to age (87.2% satisfied or not concerned), gender (85.9% satisfied or not concerned), ethnic background (93.8% satisfied or not concerned) or sexual orientation (98.2% satisfied or not concerned). Low levels of concern regarding attitude to ethnicity could reflect an under-representation of respondents from minority backgrounds. Similarly, low rates of reported concern about attitudes to sexual orientation may reflect the small number of respondents (3%) who identified as living with a same sex partner.
When asked to what extent they agreed with the statement ‘I am satisfied with my workplace’s commitment to a diverse and inclusive workplace’, 6.4% strongly disagreed, 11.5% disagreed, 20% neither agreed or disagreed, 41.2% agreed and 21% strongly agreed. Gender did not appear to influence ECR’s perception of their workplace, with satisfaction rates being similar between male and female respondents (50.3% satisfied or very satisfied versus 51.9%, Figure 3—figure supplement 1A). However, workplace position did influence satisfaction rates, with those on teaching only positions reporting the highest levels of satisfaction (23.1%), and none reporting being very dissatisfied (Figure 3—figure supplement 1B). Those in research and teaching positions reported high levels of dissatisfaction (25.4% somewhat dissatisfied and 12.7% very dissatisfied).
We did not identify an obvious influence on country of birth on job satisfaction (Table 3 and Figure 3—figure supplement 1C). Both Australian and non-Australian born researchers reported finding their work rewarding (78.0% and 76.4%), were satisfied or very satisfied with their current job (65.2% and 59.5%), but also categorised their job as a considerable source of personal strain (56.2% and 45.9%) and indicated a lack of support from institutional supervisors (63.5% and 55.6%). We regard the consistency in reporting as not necessarily indicating that country of birth has no impact on job satisfaction, but rather that systemic workplace culture issues dominate the concerns of both Australian and non-Australian born researchers.
We asked if ECRs had considered a major career or position change in the previous 5 years. The majority (78.3%) of respondents had considered a major career change, while only 21.7% had not (Figure 3B). Many considered leaving academia all together (19.1%) or moving overseas (17.4%) in order to progress their career path. For each group of respondents that indicated that they had considered a major career change, we quantified how satisfied they were with their current work environment (Figure 3C). Interestingly, within the population of ECRs who had not considered a career change, the largest group (34.5%) were dissatisfied with their current workplace. By contrast, within the population of ECRs who indicated that they had recently considered moving to another area within or outside of science, the largest group (41.9%) were very satisfied with their current workplace. These data suggest that there might be populations of ECRs who are unhappy in their current workplace, but feel trapped, while there is another population of ECRs who are very happy in their current workplace, but feel changing jobs would be beneficial. More generally, ECR’s satisfaction with their current position does not appear to significantly bias their consideration of major career changes.
Influence of gender, country of origin and appointment type on workplace challenges
Workplace and career progression challenges are displayed in Table 4. Data were sorted based on gender and subsequently sorted based on appointment types, which were categorized as ‘research only’, ‘research and teaching’, or ‘clinician researcher’. Those with a teaching or clinical appointment are likely to be less dependent on research funds for their salary, and thus their perspectives may differ. Greater than 50% of both male (52.4%) and female (63.8%) ECRs indicated that they felt they had been negatively impacted by a lack of support from institutional leaders (Table 4A). Female ECRs indicated higher rates of inequitable hiring practices (40.0% females versus 35.4% males) and harassment from those in a position of power (31.7% females versus 25.9% males). Interviews with ECRs conducted in another part of this project and the focus group which evaluated the questionnaire for this survey, as well as survey responses, suggest instances where senior academics (both male and female) were regarded as bullies (the results of this part of the project will appear in a separate publication). When asked if they feel safe in the work environment, overall 12.5% felt unsafe with an unexpected bias of males (15.6%) to females (11.0%) reporting this problem (Table 4A). We further delineated these data based on researchers who were either Australian or non-Australian born, finding that non-Australian born researchers reported being marginally less safe at work (15.1%) than respondents born in Australia (10.1%). Similarly, slightly more non-Australian born researchers reported inequitable hiring practices being a significant problem in their career advancement (14.5% versus 11.8%).
Impact of inappropriate behaviours
Particularly concerning was the number of female and male ECRs who identified that their wellbeing, or their career had been impacted by questionable research practices within their institution (41.4% of females and 30.7% of males) or external to their institution (33.6% of females and 28.9% of males). While some respondents would have been cautious not to reveal specifics regarding questionable research practices, even in a confidential survey, a number of comments did provide reasonably detailed examples of concerning behaviour (Table 5): ‘…what they wanted to see result-wise wasn’t what I was seeing. And so, I was being accused of misconduct because I wasn’t seeing what they wanted me to see, and I wouldn’t change that’.
When the data was re-sorted based on appointment type, it was possible to estimate the influence that different appointments and contract stability may have on ECR job satisfaction and/or career progression (Table 4B). The majority of clinician researchers (79.0%) reported having been impacted by lack of support from institutional superiors, compared with research and teaching ECRs (64.3%), and research only ECRs (54.5%). This pattern was replicated with respect to inequitable hiring practices reported more frequently by clinician researchers (52.6%), followed by research and teaching ECRs (46.8%), and research only ECRs (33.4%). These data may indicate that ECRs employed across multiple research, teaching and clinical departments struggle more to find unified institutional support, or to access what they perceive to be equitable hiring/recruitment practices. While clinician researchers we surveyed faced a number of challenges, we note that our survey only captured data from 19 such respondents. These ECRs, in many cases, rely primarily on their clinical appointment as a source of income, and so are potentially less sensitive to job insecurities felt by research only ECRs. Only two (10%) of clinician researcher ECRs reported feeling unsafe at work, compared with 39 (12.6%) research only ECRs, and 19 (14.1%) research and teaching ECRs. Similarly, clinician researchers reported less impact of harassment based on power positions (26.3%), compared to research only (31.7%) and research and teaching ECRs (41.3%). It is possible that the job security benefits realised by clinician researchers manifests itself in actual or perceived reductions in feeling unsafe at work, and reduced harassment from those in a position of power.
The frequency that questionable research practices had negatively impacted ECRs declined incrementally from those who were research-only (37.3% internally and 33.8% externally), clinician researchers (36.8% internally and 36.9% externally) and research and teaching (34.1% internally and 27.8% externally). These data suggest that greater research time commitment may increase the frequency of exposure to questionable research practices, but that the stability associated with salary funding from a teaching or clinical position does not obscure the perception that this is a major problem.
The need to relocate
Many academics relocate to capture job opportunities, and many appreciate the opportunity to move internationally with their career. However, we observed that the academic culture promotes a perceived need to relocate during the ECR years, and that many ECRs who wished to remain in academia considered moving as part of their career development process. To better understand this phenomenon, we asked more detailed questions regarding decisions to move. The answers to these questions indicated that moves to new institutions can be stressful, are frequently made without financial compensation, and can be challenging for families and for careers (Table 6).
This problem was highlighted in recent article published in Science, which described the struggles of a tenure-track academic on a work visa in the United States who was unable to gain financial approval to purchase a home (Evaristo, 2020). While a tenure-track academic can make long-term decisions, this is virtually impossible for many ECRs. Most (68.1%) respondents reported that they had already changed location in order to advance their careers. Of these, 28.6% of ECRs had moved once, 20.1% had moved twice and 19.5% had moved more than twice. Commonly expressed consequences, noted in interviews and in text-based responses, were that relocation was associated with stress, separations from family, loss of support network, personal cost and loss of career momentum. Within the small number of respondents that reported a chronic health condition (12%) some indicated that relocation was challenging.
Mentorship and career guidance
To better understand ECRs concerns regarding support from institutional leaders, respondents were asked to describe their mentorship and career guidance. A definition of a of mentor was provided with the questions: “A mentor is someone who is there to assist you achieve your personal, academic and career exploration goals. This person is not necessarily your supervisor’. In our survey, 61.9% of ECRs reported having a mentor, while 38.1% did not. We asked ECRs to indicate what aspects of mentoring they valued most, and these data are summarized in Figure 4A. ECRs valued advice on career decisions (81.7%) as the most important contribution from mentors. This was followed by integration into networks (77.2%), and direct influence on their gaining employment (56.7%). Ranked less significant, but still important, were skill training on methodologies (60.3%), fundraising (50.8%), and scientific writing (59.7%). Of those with a mentor, the quality of the mentoring was often described as inadequate, and some indicated that they paid for external mentoring. From the survey data (n = 322), those who did receive mentoring (Question 44 of our survey) described it as follows; 15.1% neutral, 7.5% not beneficial, 32.8% highly beneficial, or 44.6% beneficial.
With respect to supervision, as opposed to mentoring, only 68.3% of respondents had a performance review in the past two years, indicating that 31.7% had not. While half of the 31.7% respondents with no performance review indicated that they had recently been appointed or were on probation (not unusual in an environment where short term contracts are commonplace), the other half had not been offered a review. Many who did have a performance review did not find the process useful (41.6%; Figure 4B). There was no opportunity given to provide an explanation for these answers, however, respondents identified the primary utility of performance reviews as being (1) a review of personal progress (57.1%), (2) identifying strengths and achievements (50.7%), (3) help focusing on career aspirations (50.4%), and (4) to highlight issues (44.2%). ECRs identified performance reviews as least useful in leading to changes in their work practices. Given that performance reviews are often used to influence work practices, it is useful to know that this process is frequently viewed as ineffective.
Intention to leave
Finally, we circled back and considered if the positions ECRs held were similar to what they had anticipated, and if they intended to remain in or leave these positions (Figure 5A). Relatively few (14.5%) found their current position to better or much better than expected. Regardless of their perception of the position, many ECRs indicated their intention to leave. There was a trend (regression analysis, p=0.0234) indicating a greater bias to leave the position depending on how it had met expectations (Figure 5B). However, even in instances where the current position was much better than expected, nearly 40% more (61.5%) ECRs intended to leave the position rather than remain (38.5%).
As most ECR positions are short-term contracts, including those supported by ‘soft money’ (where all expenses for that researcher, including salary, are covered by fixed-term grants), it might be rational to expect to have to leave a position even if the position had met or exceeded expectations. If ECRs were to leave their current academic position, we asked what the primary motivation would be (Figure 5C). Cumulatively, two of the possible responses, lack of funding (28.2%) and job insecurity (48.9%), accounted for 77% of likely motivations for ECRs leaving their current position. Establishing an independent research group is the goal of many ECRs. Lack of independent positions was cited as the motivation 11.8% of ECRs would use to justify leaving their current position. While in Table 5 many respondents list poor institutional support as problematic, only 1.4% of respondents cite interpersonal relationships with their supervisor as a potential motivation for leaving their current position. We found that family/carer responsibilities were cited by 9.6% of ECRs as a reason to exit academia. Similar to a previous survey of postdoctoral researchers in Australia, the burden of family/carer responsibilities is heavy on both male and female ECRs, suggesting that young parents (male or female) and their families are not sufficiently accommodated by the current system. In interviews, we did identify young mothers on parental leave struggled to continue to run their laboratories, knowing that their staff depend on them, and continued to write publications while on leave out of fear of falling behind. Quotes in Table 7 provide insights into stresses felt by ECRs in Australia; we leave the comments to speak for themselves.
Given the many challenges faced by ECRs, their persistence in their endeavours to remain in the academic research workforce is impressive. However, their perceived commitment to academia in Australia may be confounded by limited number of alternative (perceived and actual) employment opportunities outside of academia. A number of comments made by ECRs (Table 8), indicate that they consider themselves to be inadequately trained for alternative careers, that there are limited alternatives available, or that they regard leaving academia as a failure. One respondent said, ‘I constantly think about leaving academia/research (from necessity not choice) but don't know how and am not qualified for any other jobs.'
Discussion
It is common vernacular to say that ‘ECRs are the future’. If this is factually true, then are we content with how we are shaping this future? We suggest that this survey data provides reason to be concerned. While ECRs in our survey overwhelmingly and repeatedly indicated that they ‘loved’ their work, only 51.0% of ECRs indicated that they were satisfied with their workplace culture. More than half of ECRs felt they had been negatively impacted by a lack of support from institutional leaders. This is in agreement with previous studies which showed that academics loved their work and realised intellectual satisfaction, but were frequently discontented with their own institution and wonder if they would be happier somewhere else, in a different profession or industry (Johnsrud and Rosser, 2002; Smith, 2020).
Many ECRs in our survey indicated that they did not have a mentor (38.1%), nor performance review (31.7%). Superficially, these data suggest that allocation of a mentor and performance review would lead to considerable improvements. However, a number of respondents (41.6%) indicated that they did not find the performance review useful. When mentoring and reviews were provided, ECRs valued career advice most, followed introduction to important networks, and the capacity of their mentor to directly help them find employment. Ranked less significant, but still important, were skill training on methodologies, fundraising, and scientific writing. These preferences may seem surprising, but a previous survey of postgraduate researchers in Australia found that the quality of supervision did not positively influence initial job attainment, but that ‘nurturing networking and careers advice’ did (Jackson and Michelson, 2015). This pattern may remain robust among STEMM ECRs in Australia, where ‘who you know’ could play a significant role in employment outcomes. Our data suggest that ECRs believe this is a factor, and many report being impacted by inequitable hiring practices (40.0%, females and 35.4%, males). Job stress in the sector is likely causing similar patterns to evolve in jurisdictions around the world (see discussion on social networks and so call ‘gate keepers’) and academic recruitment (van den Brink and Benschop, 2014).
We do not dismiss the value of good mentoring and recommend that group leaders consider investing time into training and mentoring strategies (see, for example, Lee et al., 2007). It was reported recently that ECRs who co-author publications with highly-cited scientists have greater probability of repeatedly co-authoring additional publications with top-cited scientists, and, ultimately, a higher probability of becoming top-cited scientist themselves (Li et al., 2019). While this does not directly constitute mentorship, it does provide an indication of the value of being able to follow or mimic an established research leader.
We consider the most concerning of all of our results to be the high rate at which ECRs (41.4% of females and 30.7% of males) claimed that questionable research practices within their institutions had negatively impacted their careers. We did not define ‘questionable research practices’ in our survey, but this terminology is commonly used to describe activities ranging from fraud to less egregious practices, such as data exclusion or p-Hacking (John et al., 2012). A 2019 survey conducted by the National Health and Medical Research Council (NHMRC) of Australia found that 54% of all survey participants were aware of researchers feeling tempted or under pressure to compromise on research quality, and that junior researchers were most likely to be aware of such instances, while ethics committee members were least likely (NHMRC Australia, 2020). Given the very high stress on individual ECRs and on the system, it is rational to expect that rates of questionable research practice could be on the rise. In 2005, Ioannidis reasoned that ‘most published research findings are false’, discussing the influence of data selection bias and financial pressures on data interpretation and reported outcomes (Ioannidis, 2005).
Global research pressures have not declined since 2005, and in 2016 Nature published the results from a survey of 1576 researchers on the so-called reproducibility crisis (Baker, 2016). This survey found that pressure to publish and selective reporting were perceived to contribute to greater than 60% of reproducibility problems. Our interpretation of the survey data we collected, where ~ 35% of respondents indicated that questionable research practices had impacted their careers, is that the full extent of known misconduct or data reproducibility problems is likely underestimated. Given that ECRs are both sufficiently trained to identify problems, and often in the laboratory enough to observe these problems, concern from this cohort should be viewed as genuine. Future surveys should ask respondents to characterise what types of questionable research practices they believe are most prevalent, and which are the most harmful.
Our findings highlight the need for institutional and national consideration regarding how pressures are playing out in the Australian STEMM research eco-system. We do not blame institutions or individual ECR mentors for these problems. Few ECRs (1.4%) indicated that they would leave their current position because of poor interpersonal relationships with their supervisor. Rather, we consider that the challenges experienced by ECRs in Australia reflect systemic problems. Most ECRs (78.3%) had considered a major career change in the past five years, including leaving academia all together or moving overseas. If ECRs left their current positions it would be primarily because of lack of funding and job security (total 77.1%). When the ECR responses were delineated based on years post PhD, those who were greater than 4 years post PhD were less satisfied than those who were 4 years or less post PhD. Our observations parallel a previous study that observed that job satisfaction was greater for those who had more recently started their first postdoctoral appointment (Miller and Feldman, 2015).
In our survey, female ECRs indicated experiencing higher rates of inequitable hiring practices and harassment from those in a position of power than their male counterparts. In contrast, more males felt unsafe in their work environment. We also found that both male and female ECRs were concerned about parental/carer responsibilities, knowing that delayed research productivity could compromise their career prospects. Men were more concerned about this than women, possibly reflecting recent efforts to accommodate mothers, but not necessarily families.
Challenges for researchers are not isolated to Australia. A survey by the Wellcome Trust in 2020 of over 4,000 researchers (mostly in the UK) paralleled many of our observations (Wellcome Trust, 2020; Abbott, 2020). While 84% of researchers were proud to work in the research community, only 29% felt secure in pursuing a research career, and 23% of junior researchers and students suggested that they had felt pressured by their supervisor to produce a particular result. In agreement with our findings on ‘questionable research practices’, 43% of respondents in the Wellcome survey believed that their workplace puts more value on meeting research metrics than the quality of the research. It is clear that these are global challenges that will require intervention at all levels of the research community.
Compounding inefficiencies and suggestions for change
Many current problems in the field could be viewed as inefficiencies. Solving these problems may contribute to improvements at both the personal and community level, thus justifying investment into solutions. For example, a study published in 2015 estimated that $28bn per year is ‘spent on preclinical research that is not reproducible – in the United States alone’ (Freedman et al., 2015). Although we have not seen similar estimates for Australia, findings from the recent NHMRC survey suggests there are reasons to be concerned (NHMRC Australia, 2020).
A second major source of inefficiency is the low grant funding success rates, coupled with evidence that current mechanisms for ranking applications is unreliable (Graves et al., 2011; Pier et al., 2018; Forscher et al., 2019). In 2012, researchers spent an average 34 days preparing NHMRC grant proposals, only 21% of which were successful; this means that out of the 550 years invested into all applications (3,272 applications), 435 years were spent on unfunded applications (Herbert et al., 2013). These problems are not isolated to the NHMRC; the time-cost of preparing and reviewing grants, coupled with poor ranking reliability, have motivated many to propose transition to outright lotteries (Adam, 2019).
Poor funding rates, and the inherent risk that an individual’s salary might not be funded for the next calendar year drive high attrition rates. While the constant flow of eager new PhD graduates into the workforce offers a mechanism to replace those who have exited the system, high turnover should be viewed as another potential source of community level inefficiency. While the less expensive labour of PhD students may help to balance the budgets of individual laboratories, the process of training many individuals for brief careers in science represents an inefficiency likely to negatively impact national research budgets and output. This workforce inefficiency is almost certainly linked to inefficiencies associated with irreproducible science.
Lastly, our data showed that job security (52%) was the number one reason that Australian STEMM ECRs said they might leave their current position, in agreement with the Wellcome survey which also identified this as a major concern. As a community we need to work to improve job security (take care of our people) and the quality of research data (our product). Below we set out a list of national and international recommendations that could help tackle some of these problems.
Recommendations for Australia
With the goal of stabilising the careers of early-career researchers in science, technology, engineering, mathematics and medicine and maintaining research quality, Australia should consider: (1) An increase in GDP expenditure on research and development to align with other Organization for Economic Co-operation and Development (OECD) nations. (2) Trim PhD completion numbers to better align with current workforce demands. While PhD students offer a sizable and inexpensive workforce, a long-term view of graduate contributions are likely to benefit the field. (3) Research funds should be distributed through smaller and more consistent grants with the goal of supporting the long-term career development of ECRs. Innovation and innovators are rare, and time is required to test ideas and develop gifted researchers. (4) Finally, Australia should establish an independent research ombudsman to oversee research integrity issues. Need for an independent research ombudsman has been discussed previously (Vaux, 2013; Brooks et al., 2016).
Institutional recommendations
At the institutional level, around the world, the research environment for early-career researchers in STEMM disciplines could be improved by: (1) Training mentors to manage the career development of ECRs. (2) Aiming to provide greater career stability through longer contracts. (3) Developing skills training programs that prepare PhD candidates and early-career scientists for employment outside of academia for when long-term academic employment is not viable, and a culture for attending this training. (4) Supporting the development of a research culture that counters questionable research practices by encouraging all academics to ask questions, challenge hype and report honestly.
Limitations of the study
Our survey captured the opinion of 658 early-career researchers working in Australia in STEMM disciplines. It has proven difficult to determine the precise number of such ECRs. There were approximately 23,000 higher education staff in Level A and B positions (all disciplines) in 2019 (Department of Education, Skills and Employment, 2019), but it is not known how many of these were in the STEMM disciplines. Previous work estimated the number of postdoctoral researchers employed in Australia as 6,000 (Hardy et al., 2016). It is likely our survey captured opinion from 5–10% of the target population. As the survey participants were self-selected, it is possible that we attracted a disproportionate number of dissatisfied respondents. Surveys were distributed by third parties at research institutes, or recruited via social media, potentially limiting or biasing distribution and preventing calculation of response rate. This process and its limitations have been reported briefly in Research Ethics Monthly (Christian et al., 2019).
In our survey, we collected some demographic data which could be used to measure diversity. These data included country of birth, language spoken at home, country of PhD, whether respondents lived with a partner (no, heterosexual, same sex or prefer not to say) and chronic disability. However, only 20 (3%) respondents indicated they were in a living with a partner of the same sex, and 3% preferred not to say. Furthermore, we did not collect data on the ethnicity of respondents and therefore cannot know how this may have influenced the findings of our survey. Further research should examine how the challenges identified in this study may vary between diverse groups. We did not observe indications of cultural bias, but this could be because our survey was conducted in English, perhaps selecting for those whose language skill made them well equipped to complete the survey.
Materials and methods
This survey formed one part of a mixed methods research project which explored challenges faced by early-career scientists at universities and at independent research institutes in Australia. The primary research questions from which the survey questions were derived were; (1) What are the relationships between ECR job satisfaction or dissatisfaction and their likelihood of staying in STEMM? (2) What are the principal factors that shape the ECR experience of various cohorts in the STEMM in Australia? (3) What are the motivations for ECRs leaving their research position? (4) What are the specific features of the experiences and environment of those ECRs who remain in STEMM? The definition of ‘early-career researcher’ for the purpose of this project included holding a PhD or equivalent, awarded no more than ten years prior and employment in an Australian university or independent research institute in a STEMM discipline.
Survey
Request a detailed protocolSurvey questions are included in Supplementary file 1. Quantitative data was collected from 658 respondents in an on-line survey of ECRs working in a scientific environment in universities and research institutes across Australia. Individuals employed in private enterprise/industry, not-for profit entities or in government funded organisations were excluded from this project as their research environments are considered different. The conceptual framework for the study was built on frameworks for job satisfaction for academics developed by Rosser, 2004 and Basak and Govender, 2015, which identified important factors as workload, job security, job satisfaction, challenges, mentoring and supervision, career planning, intention to leave, career breaks and expectations about the career. Survey questions were selected to explore these factors and were supplemented with questions seeking demographic information which included the institution type, research discipline, country of origin, family situation and work arrangements. In addition, we held focus group discussions which enabled us to identify other important questions, and to optimise our approach. The questionnaire for the survey was developed by first compiling questions, often used in a broader or international context, from research literature including questions from Australian Council of Education Research, The EMCR Forum at the Australian Academy of Science, Federation of Australian Scientific and Technological Societies (FASTS), Global Young Academy, National Science Foundation, Nature Research and Vitae (Christopherson et al., 2014; Hardy et al., 2016; Coates et al., 2008; Coussens et al., 2017; Nature Research and Penny, 2017; Bell and Yates, 2015; Phou, 2015; Vitae, 2018).
In order to cover all the themes identified in the literature as matters relating to job satisfaction or dissatisfaction. Some additional questions were created if no suitable question was identified elsewhere. Questions were combined and modified to create a question bank for this survey relevant to the research questions and the Australian context and further informed by data collected from a focus group of ECRs, after which the survey was pilot tested. In keeping with the conceptual framework for the study, matters investigated include inequity, bias or discrimination with respect to age, gender, inequitable hiring practices and harassment based on different power positions, mentoring and supervision, career planning, training and professional development and work life balance. The data from these questions were supplemented by questions seeking demographic information which included the institution type, research discipline, country of origin, family situation and work arrangements.
The invitation to take part in the survey was distributed via email after direct contact with the institutions, via social media or ‘umbrella groups’ such as EMCR Forum (Australian Academy of Science, 2020) and The Australian Society for Medical Research (ASMR, 2020) with members or affiliates drawn from the STEMM community who were likely to include the target group.
A focus group discussion attended by seven ECRs on January 30, 2019 evaluated the questionnaire prior to the survey and participants in the focus group offered additional insights. These seven focus group participants were ECRs from five STEMM disciplines and four institutions based in Sydney, Australia who responded to an email invitation that was circulated within Sydney institutions and who were considered to be broadly representative of ECRs in STEMM. All provided informed consent. Once the survey was established, a pilot study (n = 22) permitted testing for understanding and clarity and to check for technical difficulties. The pilot survey ran from February 14 to February 28, 2019. The National survey followed, and the data from the survey is discussed in this paper. The survey ran from March 5 to June 14, 2019. The survey was conducted online using LimeSurvey (v2.01). Eligibility to participate was determined by the initial questions in the survey.
Data availability
The data set for this paper has been uploaded to Federation University Figshare, which is an open access data base. DOI 10.25955/5f98c272a6ef5. These data has been purged of institution name, country of birth and identifying statements made in open text responses. Most of the remaining survey data is likewise be available. Some project data is subject to embargo to protect the anonymity of participants however it may be shared subject to the approval of the Federation University of Australia Human Research Ethics Committee. See https://federation.figshare.com/projects/Challenges_Faced_by_Early-Career_Researchers_in_the_Sciences_in_Australia_and_the_Consequent_Effect_of_those_Challenges_on_their_Careers_a_Mixed_Methods_Project/90317.
-
FigshareChallenges for ECRs in STEMM - data supporting eLife manuscript "A survey of early-career researchers in Australia".https://doi.org/10.25955/5f98c272a6ef5
References
-
WebsiteThe state of Australian university research 2018–19, ERA National ReportAccessed December 21, 2020.
-
Theoretical framework of the factors affecting university academics job satisfactionInternational Business & Economics Research Journal 14:317–326.https://doi.org/10.19030/iber.v14i2.9167
-
WebsiteWomen in the science research workforce: identifying and sustaining the diversity advantageAccessed December 21, 2020.
-
Australia needs an ombudsman or office for research integrityInternal Medicine Journal 46:1233–1235.https://doi.org/10.1111/imj.13211
-
Reindustrialising regions: Rebuilding the manufacturing economy?Cambridge Journal of Regions, Economy and Society 7:351–358.https://doi.org/10.1093/cjres/rsu023
-
WebsiteThe attractiveness of the Australian academic profession: a comparative analysisAccessed December 21, 2020.
-
Academic research in the 21st century: maintaining scientific integrity in a climate of perverse incentives and hypercompetitionEnvironmental Engineering Science 34:51–61.https://doi.org/10.1089/ees.2016.0223
-
A note on PhD population growth in biomedical sciencesSystems Research and Behavioral Science 32:402–405.https://doi.org/10.1002/sres.2324
-
Factors influencing the employment of Australian PhD graduatesStudies in Higher Education 40:1660–1678.https://doi.org/10.1080/03075079.2014.899344
-
Faculty members' morale and their intention to leaveThe Journal of Higher Education 73:518–542.https://doi.org/10.1353/jhe.2002.0039
-
Early coauthorship with top scientists predicts success in academic careersNature Communications 10:5170.https://doi.org/10.1038/s41467-019-13130-4
-
Isolated in the lab: examining dissatisfaction with postdoctoral appointmentsThe Journal of Higher Education 86:697–724.https://doi.org/10.1353/jhe.2015.0029
-
WebsiteSurvey of research culture in Australian NHMRC-funded institutions: key messagesAccessed December 21, 2020.
-
WebsiteAustralia: National Tertiary Education Union's State (NTEU) of the Uni Survey 2017Accessed December 21, 2020.
-
Faculty members' intentions to leave: a national study on their worklife and satisfactionResearch in Higher Education 45:285–309.https://doi.org/10.1023/B:RIHE.0000019591.74425.f1
-
The race to the bottom and the route to the topNature Chemistry 12:101–103.https://doi.org/10.1038/s41557-019-0410-y
-
Gender in academic networking: the role of gatekeepers in professorial recruitmentJournal of Management Studies 51:460–492.https://doi.org/10.1111/joms.12060
-
WebsiteBackground for careers in research online survey, (Realising the potential of Researchers)Accessed December 21, 2020.
-
De-industrialisation, financialisation and Australia's macro-economic trapCambridge Journal of Regions, Economy and Society 7:509–526.https://doi.org/10.1093/cjres/rsu020
Decision letter
-
Peter RodgersSenior Editor; eLife, United Kingdom
-
Julia DeathridgeReviewing Editor; eLife, United Kingdom
-
Rebeccah S LijekReviewer; Mount Holyoke College, United States
-
Harry RolfReviewer; ATSE, Australia
-
Tanvir HussainReviewer; University of Nottingham, United Kingdom
In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.
Thank you for submitting your article "Survey of Australian STEMM Early Career Researchers raises concerns about research culture" to eLife for consideration as a Feature Article. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by two members of the eLife Features team (Julia Deathridge and Peter Rodgers) The following individuals involved in review of your submission have agreed to reveal their identity: Rebeccah S Lijek (Reviewer #2); Harry Rolf (Reviewer #3); Tanvir Hussain (Reviewer #4).
The reviewers and editors have discussed the reviews and we have drafted this decision letter to help you prepare a revised submission. We hope you will be able to submit the revised version within two months. Please also note the following
i) if your article is accepted, we would like to publish some of the relevant code with the article: please let me know if this will be a problem.
ii) eLife does not permit figures in supplementary materials: instead we allow primary figures to have figure supplements. I can suggest how to accommodate the figures in your supplementary materials if your article is accepted.
Summary
In this article, the authors surveyed 658 Australian early-career researchers (ECRs) about their job satisfaction, mentorship, working environment, and motivations to stay or leave science. The results provide a comprehensive picture of the culture of research in Australia and make an important contribution to ongoing discussions around the challenges faced by ECRs.
Major comments to address
1) It would be useful to report relevant data on ethnicity, disability and sexual orientation, if available. Given that a lack of DEI efforts was a previously documented source of job dissatisfaction (Zimmerman, Carter-Sowell and Xu, 2016; Professionals Australia, 2014; Potvin et al., 2018; Else, 2019 and Gewin, 2018), any/all survey data on DEI should be included rather than saying data not shown. It is concerning that 1 in 5 ECRs (17.8%) are dissatisfied with their workplace's approach to inclusion of diverse populations, and this finding is underplayed in the current manuscript, especially without any racial/ethnic data on survey respondents. Is it possible that the 17.8% of respondents who are dissatisfied with DEI are the scientists of colour, and the remaining survey respondents are the white scientists who are privileged to be "unconcerned" with issues of DEI in the workplace? That discrimination is a minority problem is exactly the problem. Especially since Table 3A/3B shows that ~30-40% of respondents have suffered harassment, it seems important to break this down by other demographic info (like race/ethnicity, age, parental status etc.). Discrimination and harassment is a serious problem that deserves further analysis and discussion in the manuscript.
If the data on the respondents' ethnicity and sexual orientation is not available, this should be discussed as a limitation of the study. Furthermore, it would be interesting to include the data and analysis from Qs 36 in the survey, asking how people's attitudes towards their gender, sexual orientation, or ethnicity impacted their career advancement.
2) The rationale for selecting the topics investigated in the survey is unclear. Please provide justification for the topics selected for the survey and why other relevant pressures on STEM ECRs were omitted (e.g. racism in STEM and demographic data on race/ethnicity). Additionally, please add an explanation of what is new about this study compared to previous surveys . Specifically, describe the findings of the reference cited in the Introduction, from which the survey questions used in your study were sourced.
3) Please make the following changes to the “Suggestions for Change” section:
– The current recommendations provided, although valid, are very high-level and do not provide enough guidance for senior managers in the sector who want to implement changes. It would be great to have more practical recommendations similar to the ombudsman recommendation. For example, the recommendation "aim to provide greater career stability", would benefit from having more information on how institutes should do this, such as longer contracts or ideas on how to make performance reviews more effective.
– The recommendations suggest several systematic changes at an international, national and institutional level. While changes like increased funding, regulation, a greater focus on research quality/accountability or reducing the number of PhDs may improve the research system, the section does not clearly explain how adopting these recommendations will improve the experience of ECRs or how the survey results led to these being necessary/desirable changes. For example, in Australia PhDs account for 57% of the R&D human resources dedicated to research and development, reducing the number would have significant unintended consequences to the research system the consideration of which goes beyond the scope of the article (https://www.abs.gov.au/ausstats/abs@.nsf/mf/8111.0). Making such broad and poorly considered recommendations will detract from the quality of the article and its core message.
– This section does not mention who is responsible for implementing recommendations at the different levels. The recommendations could instead pay attention to research culture and associated issues identified by the survey results and consider the challenges and opportunities available at different levels to address them. i.e. what are the benefits of a national or international mentoring scheme as opposed to an institutional one, why is one approach preferred and where does responsibility sit? This would produce recommendations useful to policy makers interested in pursuing the suggestions for change.
4) Please provide a much clearer definition of participant inclusion and exclusion criteria, especially with regard to job categories. Sometimes the authors equate this study population with postdocs but I was under the impression that their definition of ECR as 0-10yrs post PhD could also include independent investigators (like PIs, faculty, or non-trainee staff scientists). The text seems to suggest some respondents are running their own laboratories and are no longer trainees. I would appreciate some clarity on the Australian system and whether faculty/PIs were excluded or included in this sample. Much of the data interpretation relies on knowing what jobs respondents have and so analyses should include a stratification of survey responses by job (e.g. trainee/postdoc vs. independent PI/staff scientist).
5) In regards to the word cloud in Figure 1, please use the “thematic analysis” approach as described here [https://www.scribbr.com/methodology/thematic-analysis/], and an appropriate quantitative text analysis approach to support/explain the themes. One possible approach for this quantitative analysis would be to apply the algorithm used to generate the word cloud to visualise adjectives/verbs in the text responses or other approaches such as word co-occurrence – see the open source software KH Coder for examples https://khcoder.net. Please avoid the inclusion of words which are not relevant to the data you are trying to extract from the question such as “research”, “want” or “Australia”. The methodology used to analyze the text response to question 76, should also be included in the text.
6) The authors state that "publications [should] outline the limitations of their studies" yet there is no discussion of their own study's limitations. Please include a section outlining the limitations of this study. This section should mention (in addition to other limitations):
– Without any comparator groups for non-ECRs, it is difficult to say that any of these analyses are specific to the ECR population vs. a general phenomenon common to the entire STEM workforce.
– The article would benefit from some reflection by the authors on the limitations of the survey method (noting a statistically significant sample was not achieved) and the interpretive analysis/description by the researchers (i.e. no follow-up with participants is mentioned, why were results not explored in more depth / were there plans to do so?). It could also discuss any steps taken to address limitations along with lessons learnt.
7) The data on ECRs' considerations of job change (Figure 2B,C) does not seem to separate out attrition/stagnation (leaving the job because a contract runs out or lateral moves to another training position) vs. promotion into more senior roles like π (the stated desired outcome of said PhD/postdoctoral training). Doesn't the survey response "Yes, to take another position in the same field of science in Aus./overseas" include both negative and positive career outcomes? As a result, I find it challenging to interpret these data as "job insecurity" and question why the authors only interpret this as a negative. The same critique can be applied to the data that career moves are common without disambiguation of moves due to career stagnation vs. career advancement
8) I share the authors' concern about the reproducibility crisis in science and the high rates of "questionable research practices" reported in this study. Yet I find it hard to connect their data to that crisis because of the lack of definition of the term "questionable research practices" in the survey. The authors rightly mention this lack of definition. Respondents seem to think that a "questionable research practice that negatively impacted my career" includes the (abhorrent) bullying described in Table 4 Quote 4 – but the fact that the authors confound this with the data fraud (e.g. falsification or exclusion) that leads to the reproducibility crisis is overinterpretation. I don't doubt that data fabrication and exclusion occurs, and is a major problem in STEMM; I just am not convinced this survey question captures it with enough precision to interpret these data that way. I think it's important to provide evidence of research misconduct accurately and specifically, because otherwise it can lead to fear-mongering, anti-science sentiment that disproportionately undermines trust in the scientific endeavour.
Please can you remove the section in the discussion linking together the result from the survey on questionable research practices and the reproducibility crisis, or add a few sentences explaining how the lack of definition of the term “questionable research practices” makes this link difficult to determine directly, and what further work would be needed to research this finding further.
9) The Introduction focuses on the issue of employability and research quality as drivers for the work. While these two issues do emerge in the survey results, and are good reasons to be concerned about the position of ECRs in the research workforce, the section would benefit from a broader discussion of research culture to situate the findings of the survey. Discussion about perceptions of research quality could consider results from a related survey by the NHMRC the NHMRC 2019 Survey of research culture in Australian NHMRC-funded institutions https://www.nhmrc.gov.au/research-policy/research-quality
10) Please can you add a methods and procedure section before the results. This should include the section on how Survey was conducted and respondents were recruited which can be found in the Materials and methods. It would also be good to include how the questions for the survey were chosen and the steps taken to ensure there were no biases in the questionnaire.
Please can you also include the first section of the results, “The respondent demographics”. The Results section would then start with, “Characteristics that influence ECR job satisfaction”.
11) Where are the data described in subsection “Impact of Inappropriate behaviours”? I think this stratification is very interesting and may demonstrate burn-out and accumulated years of being undervalued in the workforce. If this data is not already shown, please can you include it as a figure in the manuscript.
https://doi.org/10.7554/eLife.60613.sa1Author response
Major comments to address
1) It would be useful to report relevant data on ethnicity, disability and sexual orientation, if available. Given that a lack of DEI efforts was a previously documented source of job dissatisfaction (Zimmerman, Carter-Sowell and Xu, 2016; Professionals Australia, 2014; Potvin et al., 2018; Else, 2019 and Gewin, 2018), any/all survey data on DEI should be included rather than saying data not shown. It is concerning that 1 in 5 ECRs (17.8%) are dissatisfied with their workplace's approach to inclusion of diverse populations, and this finding is underplayed in the current manuscript, especially without any racial/ethnic data on survey respondents. Is it possible that the 17.8% of respondents who are dissatisfied with DEI are the scientists of colour, and the remaining survey respondents are the white scientists who are privileged to be "unconcerned" with issues of DEI in the workplace? That discrimination is a minority problem is exactly the problem. Especially since Table 3A/3B shows that ~30-40% of respondents have suffered harassment, it seems important to break this down by other demographic info (like race/ethnicity, age, parental status etc.). Discrimination and harassment is a serious problem that deserves further analysis and discussion in the manuscript.
If the data on the respondents' ethnicity and sexual orientation is not available, this should be discussed as a limitation of the study. Furthermore, it would be interesting to include the data and analysis from Qs 36 in the survey, asking how people's attitudes towards their gender, sexual orientation, or ethnicity impacted their career advancement.
We collected some demographic data which could be used to measure diversity. These data included country of birth, language spoken at home, country of PhD, whether respondents lived with a partner (no, heterosexual, same sex or prefer not to say) and chronic disability. We found little difference between responses about challenges in the workplace from respondents who are Australian born compared with people from elsewhere. In addition to the data already contained in Figure 3—figure supplement 1C, we have added data in Table 2.
We did not directly ask about sexual orientation other than in Q11 Do you live with a partner or spouse? where an option was “yes with a same sex partner”. This option was included at the suggestion of Science in Australia Gender Equality (SAGE), with the aim of capturing whether parents in a same sex relationship had different or additional issues with respect to childcare or parental leave when compared with heterosexual parents. Only 20 (3%) respondents indicated they were in a live-in same sex relationship and there were no comments made which provided further information for people from this sector.
We did not ask respondents about disability, however Q72 Do you have a long term health condition or disability that restricts you in your everyday activities and has lasted, or is likely to last, for more than 6 months? could have captured this data. We did learn from comments that it was difficult for people with a chronic health condition to relocate.
While an examination of the difficulties of diverse populations is certainly merited for further research, we did not investigate experiences for this sector any further in this project.
We have now included a comment regarding the impact of chronic health conditions.
“Within the small number of respondents that reported a chronic health condition (12%) some indicated that relocation was challenging.”
2) The rationale for selecting the topics investigated in the survey is unclear. Please provide justification for the topics selected for the survey and why other relevant pressures on STEM ECRs were omitted (e.g. racism in STEM and demographic data on race/ethnicity). Additionally, please add an explanation of what is new about this study compared to previous surveys. Specifically, describe the findings of the reference cited in the Introduction, from which the survey questions used in your study were sourced.
The conceptual framework for the study was built on frameworks for job satisfaction for academics developed by Rosser, 2004 and Basak and Govender, 2015 which identified important factors as workload, job security, job satisfaction, challenges, mentoring and supervision, career planning, intention to leave, career breaks and expectations about the career. These questions were supplemented with questions seeking demographic information which included the institution type, research discipline, country of origin, family situation and work arrangements. In addition, we held focus group discussions which enabled us to identify other important questions, and to optimise our approach.
The following text has been added to the Materials and methods to clarify these details:
“The conceptual framework for the study was built on frameworks for job satisfaction for academics developed by Rosser, 2004 and Basak and Govender, 2015 which identified important factors as workload, job security, job satisfaction, challenges, mentoring and supervision, career planning, intention to leave, career breaks and expectations about the career. Survey questions were selected to explore these factors and were supplemented with questions seeking demographic information which included the institution type, research discipline, country of origin, family situation and work arrangements. In addition, we held focus group discussions which enabled us to identify other important questions, and to optimise our approach.”
3) Please make the following changes to the “Suggestions for Change” section:
– The current recommendations provided, although valid, are very high-level and do not provide enough guidance for senior managers in the sector who want to implement changes. It would be great to have more practical recommendations similar to the ombudsman recommendation. For example, the recommendation "aim to provide greater career stability", would benefit from having more information on how institutes should do this, such as longer contracts or ideas on how to make performance reviews more effective.
– The recommendations suggest several systematic changes at an international, national and institutional level. While changes like increased funding, regulation, a greater focus on research quality/accountability or reducing the number of PhDs may improve the research system, the section does not clearly explain how adopting these recommendations will improve the experience of ECRs or how the survey results led to these being necessary/desirable changes. For example, in Australia PhDs account for 57% of the R&D human resources dedicated to research and development, reducing the number would have significant unintended consequences to the research system the consideration of which goes beyond the scope of the article (https://www.abs.gov.au/ausstats/abs@.nsf/mf/8111.0). Making such broad and poorly considered recommendations will detract from the quality of the article and its core message.
– This section does not mention who is responsible for implementing recommendations at the different levels. The recommendations could instead pay attention to research culture and associated issues identified by the survey results and consider the challenges and opportunities available at different levels to address them. i.e. what are the benefits of a national or international mentoring scheme as opposed to an institutional one, why is one approach preferred and where does responsibility sit? This would produce recommendations useful to policy makers interested in pursuing the suggestions for change.
We are grateful for the opportunity to provide further comment on how systems can be improved. While we cannot dictate policy, we feel that we can add value by making the inefficiencies in the current research system at the personal and community level clear. With this objective, we have added text to subsections “Compounding inefficiencies and suggestions for change”, “Recommendations for Australia” and “Institutional recommendations”
4) Please provide a much clearer definition of participant inclusion and exclusion criteria, especially with regard to job categories. Sometimes the authors equate this study population with postdocs but I was under the impression that their definition of ECR as 0-10yrs post PhD could also include independent investigators (like PIs, faculty, or non-trainee staff scientists). The text seems to suggest some respondents are running their own laboratories and are no longer trainees. I would appreciate some clarity on the Australian system and whether faculty/PIs were excluded or included in this sample. Much of the data interpretation relies on knowing what jobs respondents have and so analyses should include a stratification of survey responses by job (e.g. trainee/postdoc vs. independent PI/staff scientist).
ECRs were defined as being less than 10 years since PhD completion, similar to the definition used by the Global Young Academy in their study of how to best support young scientists on a global scale. This information was provided in the Introduction, and we have added this information to the Materials and methods section as well. It is correct that the eligible researchers can fall into the range of raw trainee to independent investigator: years postdoctoral is the prime definition for inclusion. As shown in the demographic data their positions include research only, research and teaching, teaching only and clinician researcher.
The following text was added to the Materials and methods:
“We defined ECRs as being less than 10 years since PhD completion, similar to the definition used by the Global Young Academy in their study of how to best support
young scientists on a global scale (Pain, 2014; Bell et al., 2015).”
5) In regards to the word cloud in Figure 1, please use the “thematic analysis” approach as described here [https://www.scribbr.com/methodology/thematic-analysis/], and an appropriate quantitative text analysis approach to support/explain the themes. One possible approach for this quantitative analysis would be to apply the algorithm used to generate the word cloud to visualise adjectives/verbs in the text responses or other approaches such as word co-occurrence – see the open source software KH Coder for examples https://khcoder.net. Please avoid the inclusion of words which are not relevant to the data you are trying to extract from the question such as “research”, “want” or “Australia”. The methodology used to analyze the text response to question 76, should also be included in the text.
The word cloud was generated using NVivo v12 for Mac, an accepted analysis tool which helps researchers analyse data. The query facility in NVivo 12 for Mac which permits a count of word frequencies and generation of word cloud images was used to provide illustrations of the themes which were emerging from open text responses for selected questions in the on-line survey and from a section of the focus group discussion. Images were generated for the top 500 words with a minimum length of three letters long and including stemmed words. In addition to the standard “stop words”, the names of interview participants were excluded. Because this is not a sophisticated portion of the analysis, we have simply cited the software in the figure caption, and made clear in the results from which question the data was harvested.
While we appreciate the suggestion to remove words such as “research” or “Australia”, we feel that these words are specifically important in the context of our respondents. In an effort to meet the suggestions of the reviewers we have now introduced Table 2 in addition to the word cloud, where the words are limited to only those words which indicate satisfaction or dissatisfaction.
6) The authors state that "publications [should] outline the limitations of their studies" yet there is no discussion of their own study's limitations. Please include a section outlining the limitations of this study. This section should mention (in addition to other limitations):
Thank you for the opportunity to discuss the study limitations, and suggestions for what future studies might consider.
The following text has been added to the Discussion.
“Limitations of the study
Our survey captured the opinion of 658 early-career researchers working in Australia on STEMM subjects. […] This process and its limitations have been reported briefly in Research Ethics Monthly (Christian et al., 2019).”
– Without any comparator groups for non-ECRs, it is difficult to say that any of these analyses are specific to the ECR population vs. a general phenomenon common to the entire STEM workforce.
To provide context for our survey observations, we have included a summary table that describes outcomes from other Australian surveys, and the following text in the Results section.
“We compared our survey respondents’ satisfaction data with previous survey data from Australian academics (Supplementary file 2; Coates et al., 2009; Bexley et al., 2011; Bell et al., 2015; National Tertiary Education Union, 2017). Each of these studies used one or more of the “job satisfaction” questions from our survey in their survey of the Australian academic workforce. It can be seen respondents from the current study are more concerned about job security than respondents in any of the other studies. Our respondents also indicated a higher level of personal stress (52%) than those in all the other studies (28%-43%) and agree most strongly (65%) “this is not a good time for any young person to aspire to an academic career”. Their reported job satisfaction is low (62%). The combination of answers to these questions for current ECRs relative to those for the other studies indicates a situation about which there should be grave concern.”
– The article would benefit from some reflection by the authors on the limitations of the survey method (noting a statistically significant sample was not achieved) and the interpretive analysis/description by the researchers (i.e. no follow-up with participants is mentioned, why were results not explored in more depth / were there plans to do so?). It could also discuss any steps taken to address limitations along with lessons learnt.
7) The data on ECRs' considerations of job change (Figure 2B,C) does not seem to separate out attrition/stagnation (leaving the job because a contract runs out or lateral moves to another training position) vs. promotion into more senior roles like π (the stated desired outcome of said PhD/postdoctoral training). Doesn't the survey response "Yes, to take another position in the same field of science in Aus./overseas" include both negative and positive career outcomes? As a result, I find it challenging to interpret these data as "job insecurity" and question why the authors only interpret this as a negative. The same critique can be applied to the data that career moves are common without disambiguation of moves due to career stagnation vs. career advancement
We agree it is possible that the outcomes in this section could be either positive or negative for the respondent. In Figure 4C we provide a breakdown of why respondents might leave a research position. The most commonly cited reason was job insecurity. Job security is a problem even for those who are happy and productive in their current position. The reviewer is of course correct that some movement or change of jobs will be motivated by efforts to advance careers, or to capture opportunities to travel with career development. To make this clear, we have added additional text to the section of the manuscript that discusses this issue.
“The need to relocate
Many academics relocate to capture job opportunities, and many appreciate the opportunity to move internationally with their career. However, we observed that the academic culture promotes a perceived need to relocate during the ECR years, and that many ECRs who wished to remain in academia considered moving as part of their career development process.”
8) I share the authors' concern about the reproducibility crisis in science and the high rates of "questionable research practices" reported in this study. Yet I find it hard to connect their data to that crisis because of the lack of definition of the term "questionable research practices" in the survey. The authors rightly mention this lack of definition. Respondents seem to think that a "questionable research practice that negatively impacted my career" includes the (abhorrent) bullying described in Table 4 Quote 4 – but the fact that the authors confound this with the data fraud (e.g. falsification or exclusion) that leads to the reproducibility crisis is overinterpretation. I don't doubt that data fabrication and exclusion occurs, and is a major problem in STEMM; I just am not convinced this survey question captures it with enough precision to interpret these data that way. I think it's important to provide evidence of research misconduct accurately and specifically, because otherwise it can lead to fear-mongering, anti-science sentiment that disproportionately undermines trust in the scientific endeavour.
Please can you remove the section in the discussion linking together the result from the survey on questionable research practices and the reproducibility crisis, or add a few sentences explaining how the lack of definition of the term “questionable research practices” makes this link difficult to determine directly, and what further work would be needed to research this finding further.
We understand the reviewer’s concerns, and do not want to promote fear-mongering or anti-science sentiment. We have modified the statements made in the conclusion both with caveats, and additional data from another recent Australian survey. At the end of this paragraph we recommend that future surveys ask respondents for further feedback on questionable research practices. We feel that the recent survey data from National Health and Medical Research Council (NHMRC) of Australia does indicate that broad concern in the community about these issues, and we hope that our data and analysis will motivate further conversations about how to manage these problems. While we do not expand on this in the manuscript, it is conceivable that these problems are being amplified by job insecurity driven by the COVID-19 crisis.
“We did not define “questionable research practices” in our survey, but this terminology is commonly used to describe activities ranging from fraud to less egregious practices, such as data exclusion or p-Hacking (John et al., 2012). […] Future surveys should ask respondents to characterise what types of questionable research practices they believe are most prevalent, and which are the most harmful.”
9) The Introduction focuses on the issue of employability and research quality as drivers for the work. While these two issues do emerge in the survey results, and are good reasons to be concerned about the position of ECRs in the research workforce, the section would benefit from a broader discussion of research culture to situate the findings of the survey. Discussion about perceptions of research quality could consider results from a related survey by the NHMRC the NHMRC 2019 Survey of research culture in Australian NHMRC-funded institutions https://www.nhmrc.gov.au/research-policy/research-quality
Thank you for pointing us to these survey results. We have discussed and highlighted some of their observations in the manuscript Discussion. These comments are as described for the question above. Additionally, we have included Appendix 1—table 1 to enable job satisfaction data from our survey to be compared to other National surveys.
10) Please can you add a methods and procedure section before the results. This should include the section on how Survey was conducted and respondents were recruited which can be found in the Materials and methods. It would also be good to include how the questions for the survey were chosen and the steps taken to ensure there were no biases in the questionnaire.
The Materials and methods section has been moved to before the Results as requested. Further, additional text has been added to the Materials and methods to provide clarity.
“The conceptual framework for the study was built on frameworks for job satisfaction for academics developed by Rosser, 2004 and Basak and Govender, 2015 which identified important factors as workload, job security, job satisfaction, challenges, mentoring and supervision, career planning, intention to leave, career breaks and expectations about the career. Survey questions were selected to explore these factors and were supplemented with questions seeking demographic information which included the institution type, research discipline, country of origin, family situation and work arrangements. In addition, we held focus group discussions which enabled us to identify other important questions, and to optimise our approach.”
Please can you also include the first section of the results, “The respondent demographics”. The Results section would then start with, “Characteristics that influence ECR job satisfaction”.
This minor change has been made.
11) Where are the data described in subsection “Impact of Inappropriate behaviours”? I think this stratification is very interesting and may demonstrate burn-out and accumulated years of being undervalued in the workforce. If this data is not already shown, please can you include it as a figure in the manuscript.
We have added an additional table.
https://doi.org/10.7554/eLife.60613.sa2Article and author information
Author details
Funding
National Health and Medical Research Council (APP1130013)
- Michael R Doran
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
Katherine Christian is supported by an Australian Government Research Training Program (RTP) Fee-Offset Scholarship through Federation University Australia. MRD is supported by a NHMRC Fellowship (APP1130013). The Translational Research Institute is supported by Therapeutic Innovation Australia (TIA). The Australian Government supports TIA through the National Collaborative Research Infrastructure Strategy (NCRIS) program. The Authors would like to thank Dr. Kathryn Futrega for critical discussion and figure design.
Ethics
Human subjects: This study has been conducted according to the guidelines of the ethical review process of Federation University Australia and the National Statement on Ethical Conduct in Human Research (Approval Number 18/139A). Voluntary and informed participants contributed to these data. All provided informed consent.
Publication history
- Received:
- Accepted:
- Version of Record published:
Copyright
© 2021, Christian et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 11,370
- views
-
- 726
- downloads
-
- 71
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.