Peer review process
Revised: This Reviewed Preprint has been revised by the authors in response to the previous round of peer review; the eLife assessment and the public reviews have been updated where necessary by the editors and peer reviewers.
Read more about eLife’s peer review process.Editors
- Reviewing EditorCharlotte CecilErasmus MC, Rotterdam, Netherlands
- Senior EditorMa-Li WongState University of New York Upstate Medical University, Syracuse, United States of America
Reviewer #1 (Public Review):
This study by Park et al. describes an interesting approach to disentangle gene-environment pathways to cognitive development and psychotic-like experiences (PLEs) in children. They have used data from the ABCD (Adolescent Brain Cognitive Development) study and have included phenotypes polygenic scores (PGS) of educational attainment (EA) and cognition, environmental exposure data, cognitive performance data and self-reported PLEs. The study has several strengths, including its large sample size, interesting approach and comprehensive statistical model,
One remaining concern is the authors' conflation of PLEs and schizophrenia. They stated, for example, that it is necessary to adjust for schizophrenia PGS. Even though studies have found a statistical relationship between schizophrenia PGS and PLEs, this relationship is not very strong (although statistically significant) and other studies have found no relationship. Similarly, having PLEs increases the risk of developing psychosis, but that does not necessarily mean that this risk is substantial or specific. I think this needs more nuance in the manuscript and the term 'schizophrenia' should be used sparsely and very carefully as the paper has focused on PLEs.
Reviewer #2 (Public Review):
This paper tried to assess the link between genetic and environmental factors on psychotic-like experiences, and the potential mediation through cognitive ability. This study was based on data from the ABCD cohort, including 6,602 children aged 9-10y. The authors report a mediating effect, suggesting that cognitive ability is a key mediating pathway in the link between several genetic and environmental (risk and protective) factors on PLEs.
Strengths of the methods
The authors use a wide range of validated (genetic, self- and parent-reported, as well as cognitive) measures in a large dataset with a 2-year follow-up period. The statistical methods have potential to address key limitations of previous research.
Weaknesses of the methods
The methodological advantage of the method (Integrated generalized structured component analysis, IGSCA) over the standard method (Structural equation modeling, SEM) is not fully clear.
Not all methods are fully explained (how genetic components were derived; how cognition was assessed in Lee et al., 2018).
Not the largest or most recent GWAS (Genome-wide association studies) were used to generate PGS.
Strengths of the results
The authors included a comprehensive array of analyses.
Weaknesses of the results
Some factor loadings presented in Figure 3 seem counterintuitive/inconsistent.
Supplementary tables are difficult to assess. Unclear significance statement / p-values in Table 2.
Appraisal
The authors suggest that their findings provide evidence for policy reforms (e.g., targeting residential environment, family SES (social economic status), parenting, and schooling).
Impact
Immediate impact is limited given the short follow-up period (2y), possibly concerns for selection bias and attrition in the data, and some methodological concerns. The authors are transparent about most of these limitations.
Author Response
The following is the authors’ response to the original reviews.
eLife assessment:
This study presents a useful inventory of the joint effects of genetic and environmental factors on psychotic-like experiences, and identifies cognitive ability as a potential underlying mediating pathway. The data were analyzed using solid and validated methodology based on a large, multi-center dataset. However, the claim that these findings are of relevance to psychosis risk and have implications for policy changes are only partially supported by the results.
We appreciate the feedback and insightful suggestions from the editor and reviewers, which aided us to improve the manuscript. We believe the concerns initially raised were mostly due to areas that needed further clarification, which we have now clarified in this revised version. Our primary contribution lies in our meticulous analytical approach aimed at minimizing confounding effects and providing more precise estimates of the genetic and environmental impact on children's cognition and psychology. This method differs from the widely used general linear modeling in the field, which, in our opinion, may not be the optimal strategy for large-scale data analysis. Our comprehensive, tutorial-style description of the methods might serve as a valuable resource for the community.
Regarding the critique that our findings 'partially support the relevance to psychosis risk,' we have updated our manuscript to more accurately reflect this feedback. We have altered the narrative to indicate that psychotic-like experiences (PLE) are associated with the risk for psychosis, a connection substantiated by prior studies cited in our manuscript.
Similarly, in response to the comment that our findings 'partially support implications for policy changes,' we have nuanced our conclusion. However, we would like to emphasize our discovery that a negative genetic predisposition impacting cognitive development (i.e., low polygenic scores for cognitive phenotypes) can be counteracted by a positive school and familial environment. We believe that this finding could have meaningful implication for policy making and is robustly supported by our analyses.
We hope this revised manuscript more accurately reflects our research findings and its significances. Lastly, we would like to express our gratitude for your fair and detailed review process. Our experience working with eLife has been incredibly rewarding, and we commend your dedication to an encouraging and progressive publishing culture.
Public Reviews:
Reviewer #1
This study by Park et al. describes an interesting approach to disentangle gene-environment pathways to cognitive development and psychotic-like experiences in children. They have used data from the ABCD study and have included PGS of EA and cognition, environmental exposure data, cognitive performance data and self-reported PLEs. Although the study has several strengths, including its large sample size, interesting approach and comprehensive statistical model, I have several concerns:
- The authors have included follow-up data from the ABCD Study. However, it is not very clear from the beginning that longitudinal paths are being explored. It would be very helpful if the authors would make their (analysis) approach clearer from the introduction. Now, they describe many different things, which makes the paper more difficult to read. It would be of great help to see the proposed path model in a Figure and refer to that in the Method.
We clarified the longitudinal paths tested in this study in Intro [line 149~159]. We also added a figure of the proposed path model (Figure 1) [Methods: line 231~238].
- There is quite a lot of causal language in the paper, particularly in the Discussion. My advice would be to tone this down.
We adjusted and moderated the use of causal languages throughout the manuscript.
- I feel that the limitation section is a bit brief, and can be developed further.
We clearly specified the limitations of our study. These included concerns about the representativeness of the ABCD samples, of the limited scope of longitudinal data, and the use of non-randomized, observational data [line 524~544].
- I like that the assessment of CP and self-reports PEs is of good quality. However, I was wondering which 4 items from the parent-reported CBCL were used and how did they correlate with the child-reported PEs? And how was distress taken into account in the child self-reported PEs measurement? Which PEs measures were used?
Thanks for the clarification question. We report the Pearson’s correlation coefficients between the PLEs [line 198~200]. (The Reviewer #1 may have referred to the prior version of our manuscript submitted elsewhere, for this point has been already addressed in our initial submission to eLife).
- What was the correlation between CP and EA PGSs?
The Pearson’s correlation between CP and EA PGS was 0.4331 (p<0.0001). We added the statistics to the manuscript. [line 214]
- Regarding the PGS: why focus on cognitive performance and EA? It should be made clearer from the introduction that EA is not only measuring cognitive ability, but is also a (genetic) marker of social factors/inequalities. I'm guessing this is one of the reasons why the EA PGS was so much more strongly correlated with PEs than the CP PGS. See the work bij Abdellaoui and the work by Nivard.
We appreciate the reviewer’s insightful feedback. Acknowledging the role of both CP and EA PGSs in our study, we agree with the observation that EA PGS goes beyond gauging cognitive aptitude—it also serves as an indicator of societal influences and inequalities. The multifaceted nature of EA PGS could be the reason underlying the stronger correlation with PLEs compared to CP PGS. In response to this feedback, we revised our introduction to articulate the multifaceted role of EA PGS in more precise terms. For supporting our assertions, we have included references to prior studies (Abdellaoui et al., 2022) [line 131~142].
Abdellaoui, A., Dolan, C. V., Verweij, K. J. H., & Nivard, M. G. (2022). Gene–environment correlations across geographic regions affect genome-wide association studies. Nature Genetics. doi:10.1038/s41588-022-01158-0
- Considering previous work on this topic, including analyses in the ABCD Study, I'm not surprised that the correlation was not very high. Therefore, I don't think it makes a whole of sense to adjust for the schizophrenia PGS in the sensitivity analyses, in other words, it's not really 'a more direct genetic predictor of PLEs'.
We thank the reviewer for the thoughtful comments. We acknowledge that the correlation between schizophrenia PGS and PLE may not be exceedingly high, as evidenced by previous work, including analyses from the ABCD study. However, we would like to emphasize our rationale for adjusting schizophrenia PGS in the sensitivity analyses. Our study design stemmed from the established associations between PLEs and increased risk for schizophrenia. Existing studies have reported significant associations between schizophrenia PGS and cognitive deficits in both psychosis patients (Shafee et al., 2018) and people at risk for psychosis (He et al., 2021). Notable, the PGS for schizophrenia has shown significant associations with PLEs, arguably more so than PGS for PLEs itself (Karcher et al., 2018). Our updated manuscript has incorporated these references to improve clarity. [line 307~309]. By adding this layer of adjustment, we believe that our mixed linear model more precisely examines the relationship between the cognitive phenotype PGS and PLEs, in terms of both sensitivity and specificity.
He, Q., Jantac Mam-Lam-Fook, C., Chaignaud, J., Danset-Alexandre, C., Iftimovici, A., Gradels Hauguel, J., . . . Chaumette, B. (2021). Influence of polygenic risk scores for schizophrenia and resilience on the cognition of individuals at-risk for psychosis. Translational Psychiatry, 11(1). doi:10.1038/s41398-021-01624-z
Karcher, N. R., Paul, S. E., Johnson, E. C., Hatoum, A. S., Baranger, D. A. A., Agrawal, A., . . . Bogdan, R. (2021). Psychotic-like Experiences and Polygenic Liability in the Adolescent Brain Cognitive Development Study. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. doi:https://doi.org/10.1016/j.bpsc.2021.06.012
Shafee, R., Nanda, P., Padmanabhan, J. L., Tandon, N., Alliey-Rodriguez, N., Kalapurakkel, S., . . . Robinson, E. B. (2018). Polygenic risk for schizophrenia and measured domains of cognition in individuals with psychosis and controls. Translational Psychiatry, 8(1). doi:10.1038/s41398-018-0124-8
- How did the FDR correction for multiple testing affect the results?
Please note that we have clarified our FDR correction in the methods
As detailed in the method section [line 254~255], we applied False Discovery Rate (FDR) correction for multiple testing across nine key variables in the study: PGS (CP or EA), family income, parental education, family’s financial adversity, Area Deprivation Index, years of residence, proportion of population below -125% of the poverty line, positive parenting behavior, and positive school environment. An exception was made in our additional sensitivity analysis, where we included schizophrenia PGS in the linear mixed model for adjustment, thus the FDR correction was applied across ten key variables instead. Overall, the application of FDR correction had minimal impact on our findings. Most associations between the key variables and the outcomes that were originally marked as highly significant sustained their significance after the FDR correction.
Overall, I feel that this paper has the potential to present some very interesting findings. However, at the moment the paper misses direction and a clear focus. It would be a great improvement if the readers would be guided through the steps and approach, as I think the authors have undertaken important work and conducted relevant analyses.
We express our appreciation to the reviewer for the positive feedback and constructive suggestions, which only serve to improve and strengthen our manuscript. We have incorporated the suggested corrections and clarifications in response to the reviewer's suggestions. We believe that these changes will not only enhance the overall readability but also more effectively emphasize the significance and implication of our work.
Reviewer #2 (Public Review):
This paper tried to assess the link between genetic and environmental factors on psychotic-like experiences, and the potential mediation through cognitive ability. This study was based on data from the ABCD cohort, including 6,602 children aged 9-10y. The authors report a mediating effect, suggesting that cognitive ability is a key mediating pathway in the link between several genetic and environmental (risk and protective) factors on psychotic-like experiences.
While these findings could be potentially significant, a range of methodological unclarities and ambiguities make it difficult to assess the strength of evidence provided.
Strengths of the methods:
The authors use a wide range of validated (genetic, self- and parent-reported, as well as cognitive) measures in a large dataset with a 2-year follow-up period. The statistical methods have the potential to address key limitations of previous research.
Weaknesses of the methods:
The rationale for the study is not completely clear. Cognitive ability is probably a more likely mediator of traits related to negative symptoms in schizophrenia, rather than positive symptoms (e.g., psychosis, psychotic-like symptom). The suggestion that cognitive ability might lead to psychotic-like symptoms in the general population needs further justification.
We appreciate the reviewer’s concern regarding the role of cognitive ability in relation to schizophrenia symptoms. We are aware that cognitive ability often serves as a mediator of psychotic-like experiences. However, to our best knowledge, a growing body of research has proposed that cognitive ability can mediate positive symptoms in schizophrenia including psychotic-like experiences. The studies by Howes & Murray (2014) and Garety et al. (2001) suggested that deficits in cognitive ability can potentially contribute to the manifestation of positive symptoms such as psychotic-like experiences. We have elaborated on this aspect in the Introduction section [line 104-115].
Howes, O. D., & Murray, R. M. (2014). Schizophrenia: an integrated sociodevelopmental-cognitive model. The Lancet, 383(9929), 1677-1687. doi:https://doi.org/10.1016/S0140-6736(13)62036-X
Garety, P. A., Kuipers, E., Fowler, D., Freeman, D., & Bebbington, P. E. (2001). A cognitive model of the positive symptoms of psychosis. Psychological Medicine, 31(2), 189-195. doi:10.1017/S0033291701003312
Terms are used inconsistently throughout (e.g., cognitive development, cognitive capacity, cognitive intelligence, intelligence, educational attainment...). It is overall not clear what construct exactly the authors investigated.
We thank the reviewer’s feedback regarding the consistency of terminology in our manuscript. Per the suggestion, we standardized the use of ‘cognitive capacity’ and now consistently refer to it as ‘cognitive phenotypes’ throughout our manuscript. Furthermore, we explicitly stated in the Introduction section that our two PGSs of focus will be termed ‘cognitive phenotypes PGSs’, aligning with terminology used in prior studies (Joo et al., 2022; Okbay et al., 2022; Selzam et al., 2019) [line 140~142].
Joo, Y. Y., Cha, J., Freese, J., & Hayes, M. G. (2022). Cognitive Capacity Genome-Wide Polygenic Scores Identify Individuals with Slower Cognitive Decline in Aging. Genes, 13(8), 1320. doi:10.3390/genes13081320
Okbay, A., Wu, Y., Wang, N., Jayashankar, H., Bennett, M., Nehzati, S. M., . . . Young, A. I. (2022). Polygenic prediction of educational attainment within and between families from genome-wide association analyses in 3 million individuals. Nature Genetics, 54(4), 437-449. doi:10.1038/s41588-022-01016-z
Selzam, S., Ritchie, S. J., Pingault, J.-B., Reynolds, C. A., O’Reilly, P. F., & Plomin, R. (2019). Comparing Within- and Between-Family Polygenic Score Prediction. The American Journal of Human Genetics, 105(2), 351-363. doi:https://doi.org/10.1016/j.ajhg.2019.06.006
Not the largest or most recent GWASes were used to generate PGSes.
We appreciate the reviewer’s observation. Indeed, we were unable to utilize the most recent or the largest GWAS for cognitive performance, educational attainment, and schizophrenia due to the timeline of our study. Regrettably, the commencement of our study preceded the publication of the ‘currently’ the largest or most recent GWAS studies by Okbay et al. (2022) and Trubetskoy et al. (2022). Our research was conducted with the best available data at that time, which was the GWAS of European-descent individuals for educational attainment and cognitive performance (Lee et al, 2018). To eliminate any potential confusion, we adjusted the text to specify that our study used 'a GWAS of European-descent individuals for educational attainment and cognitive performance' rather than the largest GWAS [line 206~208].
It is not fully clear how neighbourhood SES was coded (higher or lower values = risk?). The rationale, strengths, and assumptions of the applied methods are not fully clear. It is also not clear how/if variables were combined into latent factors or summed (weighted by what). It is not always clear when genetic and when self-reported ethnicity was used. Some statements might be overly optimistic (e.g., providing unbiased estimates, free even of unmeasured confounding; use of representative data).
Thank you for pointing this out. Consistent with the illustration of neighborhood SES in the Methods, higher values of neighborhood SES indicate risk [line 217~228]. In the original Figure 2, higher value of neighborhood SES links to lower intelligence (direct effects: β=-0.1121) and higher PLEs (indirect effects: β=-0.0126~ -0.0162). We think such confusion might have been caused by the difference between family SES (higher values = lower risk) neighborhood SES (higher values = higher risk). Thus, we changed the terms to ‘High Family SES’ and ‘Low Neighborhood SES’ in the corrected figure (Figure 3) for clarification.
Considering that shorter duration of residence may be associated with instability of residency, it may indicate neighborhood adversity (i.e., higher risk). This definition of the ‘years of residence’ variable is in line with the previous study by Karcher et al. (2021).
During estimation, the IGSCA determines weights of each observed variable in such a way as to maximize the variances of all endogenous indicators and components. We added this explanation in the description about the IGSCA method [line 266~268].
We deleted overly optimistic statements like ‘unbiased estimates’ and used expressions such as ‘adjustment for observed/unobserved confounding’ instead, throughout our manuscript.
Karcher, N. R., Schiffman, J., & Barch, D. M. (2021). Environmental Risk Factors and Psychotic-like Experiences in Children Aged 9–10. Journal of the American Academy of Child & Adolescent Psychiatry, 60(4), 490-500. doi:10.1016/j.jaac.2020.07.003
It appears that citations and references are not always used correctly.
We thoroughly checked all citations and specified the references for each statement: We deleted Plomin & von Stumm (2018) and Harden & Koellinger (2020) and cited relevant primary studies (e.g., Lee et al., 2018; Okbay et al., 2022; Abdellaoui et al., 2022) instead. We also specified the references supporting the statement that educational attainment PGS links to brain morphometry (Judd et al., 2020; Karcher et al., 2021). As Okbay et al. (2022) use PGS of cognitive intelligence (which mentions the analyses results in their supplementary materials) as well as educational attainment, we decided to continue citing this reference [line 131~141].
Strengths of the results:
The authors included a comprehensive array of analyses.
We thank the reviewer for the positive comment.
Weaknesses of the results:
Many results, which are presented in the supplemental materials, are not referenced in the main text and are so comprehensive that it can be difficult to match tables to results. Some of the methodological questions make it challenging to assess the strength of the evidence provided in the results.
As you rightly identified, we inadvertently failed to reference Table S2 in the main text. We have since corrected this omission in the Results section for the IGSCA (SEM) analysis [line 376]. The remainder of the supplementary tables (Table S1, S3~S7) have been appropriately cited in the main manuscript. We recognize that the quantity of tables provided in the supplementary materials is substantial. However, given the comprehensiveness and complexity of our analyses, which encompass a wide array of study variables, these tables offer intricate results from each analysis. We deem these results, which include valuable findings from sensitivity analyses and confound testing, too significant to exclude from the supplementary materials. That said, we are open to, and would greatly welcome, any further suggestions on how to present our supplementary results in a more clear and digestible format. Your guidance in this matter is highly valued.
Appraisal:
The authors suggest that their findings provide evidence for policy reforms (e.g., targeting residential environment, family SES, parenting, and schooling). While this is probably correct, a range of methodological unclarities and ambiguities make it difficult to assess whether the current study provides evidence for that claim.
We believe that with the improvement we made in this revised manuscript, this concern may have been successfully mitigated.
Impact:
The immediate impact is limited given the short follow-up period (2y), possibly concerns for selection bias and attrition in the data, and some methodological concerns.
We appreciate the feedback provided in the reviewer's impact statement. We added as study limitations [line 524~544] that the impact of our findings may be limited due to the relatively short follow-up period, the possibility of sample selection bias, and the problems of interpreting results from an observational study as causality (despite the novel causal inference methods, designed for non-randomized, observational data, that we used).
As responded above (and also in more detail in the Reviewer #2’s Recommendations For The Authors section below), we made necessary corrections and clarifications for the points suggested by the reviewer. As we are willing to make additional revisions, please feel free to give comments if you feel that our corrections are insufficient or inappropriate.
Nevertheless, we would like to discuss some points. We sincerely hope this following response does not come across as argumentative to the reviewer and the editor. We fully understand the reviewer's perspective on this matter, and we agree that the issues raised about the ABCD study are absolutely valid. However, when evaluating the overall impact of a study, other factors, such as how the field has been assessing the impact of similar studies, should also be considered.
Firstly, the potential selection bias and attrition in the ABCD data may not necessarily limit the conclusions of this study. While recognizing the potential issues with the ABCD data is important, we feel that judging the impact of our findings as "limited" based on these issues may not be entirely fair. This is because no study, particularly those of a nationwide scale such as the UK Biobank, IMAGEN, HEAL, HBCD, etc., is completely free of limitations. Typically, the potential limitations of the data don't undermine the impact of individual studies' findings. Numerous studies using ABCD data have been published in top-tier journals—despite the limitations of the ABCD study—underscoring the scientific merit of the findings. For example, the study by Tomasi, D., & Volkow, N. D. (2021), entitled "Associations of family income with cognition and brain structure in USA children: prevention implications," published in Molecular Psychiatry, might be highly relevant to the limitations of the ABCD study raised by the reviewer. The scientific community, including editors, reviewers, and readers, may have appreciated the impact of this study despite the acknowledged limitations of the ABCD data.
Secondly, the two-year time window of our longitudinal analysis might not impact the aim of this study—an iterative assessment of the associations between genetic and environmental variables with cognitive intelligence and mental health, with a focus on PLE, in preadolescents. Had we aimed to test the developmental trajectory from childhood to adolescence, perhaps a longer timeframe would have made more sense. So, we do not agree with the reviewer’s assessment that the short time window limits the impact of our study.
Suggested revisions based on the combined reviewer feedback:
- The terminology used should be carefully reviewed and revised
• Please use the correct terminology for the key concepts assessed in this study. For example, authors sometimes conflate PLEs and psychosis, two related but separate constructs. Furthermore, the terms 'good parenting' and 'good schooling' are vague and subjective.
• The authors use multiple terms to refer to cognitive ability (cognitive capacity, intelligence, cognitive intelligence, etc). The term 'cognitive development' in the title and manuscript does not seem to be justified given the focus on different measures of cognitive ability at a single time point (i.e. baseline).
• Please avoid causal language and using statements that cannot be entirely substantiated (e.g. unbiased estimates, free from unmeasured confounding)
Thank you for suggesting this point. We revised all key terminologies used throughout our manuscript.
Per your suggestion, we specified that PLEs indicate the risk of psychosis and often precede schizophrenia. We checked all misused cases of the term ‘psychosis’ and corrected them as ‘PLEs’. We also changed the terms 'good parenting' and 'good schooling' to ‘positive parenting behavior’ and ‘positive school environment’.
We changed the term ‘cognitive development’ to ‘cognitive ability’ throughout our manuscript. We also changed the title to ‘Gene-Environment Pathways to Cognitive Intelligence and Psychotic-Like Experiences in Children’ because we used ‘cognitive intelligence’ for NIH toolbox variable in the text.
We corrected and tone-downed all causal languages used in our manuscript. As mentioned by the reviewers, we deleted statements like ‘unbiased estimates’ and used expressions such as ‘adjustment for observed/unobserved confounding’ instead.
- A stronger rationale for the focus on PLEs, and the potential mediating role of cognitive ability in genetic and environmental effects on PLES, should be provided
We appreciate the raised concerns that cognitive ability may serve as a mediator of psychotic-like experiences. To our best knowledge, it has been proposed that cognitive ability can be a mediator of positive symptoms in schizophrenia (including psychotic-like experiences), as well as negative symptoms. This mediating role of cognitive ability was proposed in several prior studies on cognitive model of schizophrenia/psychosis. Per your suggestion, we included an additional justification in Intro [line 104~115] where we highlighted that cognitive ability has been proposed as a potential mediator of genetic and environmental influence on positive symptoms of schizophrenia such as psychotic-like experiences. We refer to studies conducted by Howes & Murray (2014) and Garety et al. (2001).
Howes, O. D., & Murray, R. M. (2014). Schizophrenia: an integrated sociodevelopmental-cognitive model. The Lancet, 383(9929), 1677-1687. doi:https://doi.org/10.1016/S0140-6736(13)62036-X
Garety, P. A., Kuipers, E., Fowler, D., Freeman, D., & Bebbington, P. E. (2001). A cognitive model of the positive symptoms of psychosis. Psychological Medicine, 31(2), 189-195. doi:10.1017/S0033291701003312
- As described in more detail by the reviewers, more information should be provided about the measures used in the study and how they relate to one another (e.g. correlations between PQ-BC and CBCL; PGS-CA and PGS-EA).
Thank you for your suggestion. Although this information was already provided in our initial submission, it appears that the Reviewer #1’s might have referred to the prior version of our manuscript submitted elsewhere before eLife.
To clarify, our findings reveal significant Pearson’s correlation coefficients between PLEs across all time-points (baseline year: r=0.095~0.0989, p<0.0001; 1-year follow-up: r=0.1322~0.1327, p<0.0001; 2-year follow-up: r= 0.1569~0.1632, p<0.0001) and we added this information in the Method section [line 198~200]. We also added the Pearson’s correlation between the two PGSs (r=0.4331, p<0.0001) in the Methods for PGS [line 214].
- More details are needed regarding the analytical strategies used (e.g. how imputation was performed, why PGS were not based on the largest and most recent GWASes, whether latent or observed variables were examined, what exactly the supplementary materials show and how they relate to information provided in the main text).
We appreciate your feedback. We acknowledge the concerns about the GWAS sources utilized for the study. Unfortunately, our study commenced prior to the publication of the ‘currently’ most recent or largest GWAS by Okbay et al. (2022) and Trubetskoy et al. (2022). Our research was conducted with the best available data at that time, which was the largest GWAS of European-descent individuals for educational attainment and cognitive performance (Lee et al, 2018). We have now clarified this point in the manuscript. [line 206~208]
Also, we specified the use of composite indicators for the PGS, family SES, neighborhood SES, positive family and school environment, and PLEs, while latent factors were used for cognitive intelligence [line 269~285].
We highly appreciate the reviewer’s comments regarding the supplementary materials. We regret overlooking the citation of Table S2 in the main manuscript, and this has now been rectified in the Results section for the IGSCA (SEM) analysis [line 376]. The remaining supplementary tables (Table S1, S3~S7) have been correctly referenced within the manuscript. We acknowledge that the supplementary materials are extensive due to the comprehensive array of study variables and intricate results from each analysis. However, given that our analyses encompass a wide array of study variables, these tables offer intricate results from each analysis. We deem these results, which include valuable findings from sensitivity analyses and confound testing, too crucial to exclude from the supplementary materials. That said, we are open to any further suggestions to make our supplementary results more accessible and digestible. In order to improve the accessibility and clarity of our presentation, we are fully committed to making any necessary changes and look forward to any further recommendations.
- The limitation section should be expanded and statements regarding the implications of the study findings should be qualified accordingly (e.g. short follow-up period, potential for attrition and selection bias, reverse causation, etc)
We specified additional potential constraints of our study, including limited representativeness, limited periods of follow-up data (baseline year, 1-year, and 2-year follow-up), possible sample selection bias, and the use of non-randomized, observational data [line 524~544].
- Please ensure that the references provided support the statements in the text to which they are linked to.
Thank you for pointing this out. We thoroughly went over all citations and corrected the inaccurately or vaguely cited references for each statement.
Reviewer #2 (Recommendations For The Authors):
- Please use terms consistently and correctly. E.g., 'cognitive capacity' is not the same as 'educational attainment'.
We thank the reviewer’s feedback regarding the consistency of terminology in our manuscript. Per the suggestion, we standardized the use of ‘cognitive capacity’ and now consistently refer to it as ‘cognitive phenotypes’ throughout our manuscript. Furthermore, we explicitly stated in the Introduction section that our two PGSs of focus will be termed ‘cognitive phenotypes PGSs’, aligning with terminology used in prior studies (Joo et al., 2022; Okbay et al., 2022; Selzam et al., 2019) [line 140~142].
Joo, Y. Y., Cha, J., Freese, J., & Hayes, M. G. (2022). Cognitive Capacity Genome-Wide Polygenic Scores Identify Individuals with Slower Cognitive Decline in Aging. Genes, 13(8), 1320. doi:10.3390/genes13081320
Okbay, A., Wu, Y., Wang, N., Jayashankar, H., Bennett, M., Nehzati, S. M., . . . Young, A. I. (2022). Polygenic prediction of educational attainment within and between families from genome-wide association analyses in 3 million individuals. Nature Genetics, 54(4), 437-449. doi:10.1038/s41588-022-01016-z
Selzam, S., Ritchie, S. J., Pingault, J.-B., Reynolds, C. A., O’Reilly, P. F., & Plomin, R. (2019). Comparing Within- and Between-Family Polygenic Score Prediction. The American Journal of Human Genetics, 105(2), 351-363. doi:https://doi.org/10.1016/j.ajhg.2019.06.006
- The authors study 'cognitive performance using seven instruments', but it is not clear how fluid and crystalline intelligence was defined/operationalized.
Thank you for pointing this out. We specified the NIH Toolbox tests used for composite scores of fluid and crystallized intelligence, respectively. “We utilized baseline observations of uncorrected composite scores of fluid intelligence (Dimensional Change Card Sort Task, Flanker Test, Picture Sequence Memory Test, List Sorting Working Memory Test), crystallized intelligence (Picture Vocabulary Task and Oral Reading Recognition Test), and total intelligence (all seven instruments) provided in the ABCD Study dataset” [line 180~187].
- I don't think Lee 2018 is the largest GWAS for educational attainment. That would be Okbay 2022. It needs to be described how cognitive performance was defined in Lee 2018. Why did the authors not use the Trubetskoy 2022 schizophrenia GWAS?
Thank you for mentioning this point. The reason why we were not able to use the largest GWAS for CP, EA and schizophrenia is because (unfortunately) our study started earlier than the point when the GWAS studies by Okbay et al. (2022) and Trubetskoy et al. (2022) were published. We corrected that our study used ‘a GWAS of European-descent individuals for educational attainment and cognitive performance’ instead of the largest GWAS [line 206~208].
- It is unclear how neighbourhood SES was coded. The authors seem to suggest that higher values indicate risk, but Figure 2 suggests that higher values links to higher intelligence and lower PLE.
Thank you very much for pointing this out. Consistent with the illustration of neighborhood SES in the Methods section, higher values of neighborhood SES indicate risk. In the original Figure 2, higher values of neighborhood SES links to lower intelligence (direct effects: β=-0.1121) and higher PLEs (indirect effects: β=-0.0126~-0.0162). We think such confusion might have been caused by the difference between family SES (higher values = lower risk) neighborhood SES (higher values = higher risk). Thus, we changed the terms to ‘High Family SES’ and ‘Low Neighborhood SES’ in the corrected figure (Figure 3) for clarification.
- Also, the 'year of residence' variable is unclearly defined. Does this mean that a shorter duration of residency (even in a good neighbourhood) indicate risk?
Thank you for mentioning this point. Considering that shorter duration of residence may be associated with instability of residency, it may indicate neighborhood adversity (i.e., higher risk). This definition of the ‘years of residence’ variable is in line with the previous study by Karcher et al. (2021).
Karcher, N. R., Schiffman, J., & Barch, D. M. (2021). Environmental Risk Factors and Psychotic-like Experiences in Children Aged 9–10. Journal of the American Academy of Child & Adolescent Psychiatry, 60(4), 490-500. doi:10.1016/j.jaac.2020.07.003
- Please provide information on how correlated the two PGSes were.
Thank you for your suggestion. We added the Pearson’s correlation between the two PGSs (r=0.4331, p<0.0001) in the Methods section for PGS [line 214].
- Information on the outcome variable in the 'linear mixed models' section is missing. I assumed it was PLE.
Thank you for notifying us of this point. We added the information on the outcome variables in the section for linear mixed models [line 242~244].
- In the 'Path Modeling' section, please explain what 'factors and components' concretely refer to. How is this different from a standard SEM with latent factors?
Thank you for your comment on the need to elaborate the IGSCA method. We added that different from standard SEM methods which only uses latent factors, the IGSCA method can use components as well as latent factors as constructs in model estimation. This allows the IGSCA method to control bias more effectively in estimation compared to the standard SEM [line 261~268].
- The sentence starting line 229 is unclear. Does this mean variables were not used to generate latent factors. And if not, what weights were used to create a 'weighted sum'?
Thank you for mentioning this point. The sentence means that we treated PGSs, family SES, neighborhood SES, positive family and school environment, and PLEs as composite indicators (derived from a weighted sum of relevant observed variables), while general intelligence was represented as a latent factor.
It has been suggested from prior studies that these variables (PGSs, family SES, neighborhood SES, positive family and school environment, and PLEs) are less likely to share a common factor and were assessed as a composite index during analyses. For instance, Judd et al. (2020) and Martin et al. (2015) analyze genetic influence of educational attainment and ADHD as composite indicators. Also, as mentioned in Judd et al. (2020), socioenvironmental influences are often analyzed as composite indicators. Studies on psychosis continuum (e.g., van Os et al., 2009) suggest that psychotic disorders are likely to have multiple background factors instead of having a common factor, and notes that numerous prior research uses composite indices to measure psychotic symptoms. Based on this literature, we used components for these variables.
The IGSCA determines weights of each observed variable to maximize the variances of the endogenous indicators and components [added in line 265~268].
On the other hand, we treated general intelligence as a latent factor/variable underlying fluid and crystallized intelligence. This is based on the extensive literature of classical g theory of intelligence [added in line 269~284].
Judd, N., Sauce, B., Wiedenhoeft, J., Tromp, J., Chaarani, B., Schliep, A., ... & Klingberg, T. (2020). Cognitive and brain development is independently influenced by socioeconomic status and polygenic scores for educational attainment. Proceedings of the National Academy of Sciences, 117(22), 12411-12418.
Martin, J., Hamshere, M. L., Stergiakouli, E., O'Donovan, M. C., & Thapar, A. (2015). Neurocognitive abilities in the general population and composite genetic risk scores for attention‐deficit hyperactivity disorder. Journal of Child Psychology and Psychiatry, 56(6), 648-656.
van Os, J., Linscott, R., Myin-Germeys, I., Delespaul, P., & Krabbendam, L. (2009). A systematic review and meta-analysis of the psychosis continuum: Evidence for a psychosis proneness–persistence–impairment model of psychotic disorder. Psychological Medicine, 39(2), 179-195. doi:10.1017/S0033291708003814
- It is overall not clear when genetically and when self-reported information of ethnicity was used. This needs to be clearer throughout.
Thank you for mentioning this point. We only used genetically defined ethnicity, and we have not mentioned that we used self-reported ethnicity. Per your suggestion, we clarified that we used ‘genetic ethnicity’ throughout the paper.
- The sentence starting line 253 is also unclear. How is schizophrenia PGS a 'more direct genetic predictor of PLE' and compared to what other measure?
Thank you for pointing this out. Please note that our adjustment (or sensitivity analyses) was based on the reported associations between PLEs and the risk for schizophrenia: schizophrenia PGS is associated with a cognitive deficit in psychosis patients (Shafee et al., 2018) and individuals at-risk of psychosis (He et al., 2021), and psychotic-like experiences (more so than PGS for psychotic-like experiences) (Karcher et al., 2018). We added these references for clarification [line 307~309]. We believe that because of the adjustment our results from the mixed linear model show the sensitivity and specificity of the association between cognitive phenotype PGS and PLEs.
He, Q., Jantac Mam-Lam-Fook, C., Chaignaud, J., Danset-Alexandre, C., Iftimovici, A., Gradels Hauguel, J., . . . Chaumette, B. (2021). Influence of polygenic risk scores for schizophrenia and resilience on the cognition of individuals at-risk for psychosis. Translational Psychiatry, 11(1). doi:10.1038/s41398-021-01624-z
Karcher, N. R., Paul, S. E., Johnson, E. C., Hatoum, A. S., Baranger, D. A. A., Agrawal, A., . . . Bogdan, R. (2021). Psychotic-like Experiences and Polygenic Liability in the Adolescent Brain Cognitive Development Study. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. doi:https://doi.org/10.1016/j.bpsc.2021.06.012
Shafee, R., Nanda, P., Padmanabhan, J. L., Tandon, N., Alliey-Rodriguez, N., Kalapurakkel, S., . . . Robinson, E. B. (2018). Polygenic risk for schizophrenia and measured domains of cognition in individuals with psychosis and controls. Translational Psychiatry, 8(1). doi:10.1038/s41398-018-0124-8
- Please include a statement on the assumptions made when using the method used in this study and developed by Miao 2022, explain what evidence you have to support these assumptions and how this method, which I believe was developed for RCTs, can be applied to observational data.
We specified the assumptions for the causal inference method proposed by Miao et al. (2022) and why it is applicable to our study. Also, we noted that this novel method was developed to identify the causal effects of multiple treatment variables within non-randomized, observational data [line 309~319].
- Some of the statements are potentially misleading. E.g., I would be very cautious to claim that the methods applied allowed the authors to estimate 'unbiased associations again potential (even unobserved) confounding variables'. There are many concerns such as selection bias, attrition, reverse causation, genetic confounding, etc that cannot be addressed satisfactorily using these data and methods.
Thank you for pointing this out. We deleted statements like ‘unbiased estimates’ and used expressions such as ‘adjustment for observed/unobserved confounding’ instead.
Nevertheless, please note that due to some limitations in the data (e.g., confounders), an analytic approach should be robust enough to handle potential violations of assumptions. This was the point we wanted to emphasize--In contrast to the majority of studies using the ABCD study, which employ simplistic GLM or conventional SEM with only latent variable modeling, our study provides less biased, thus more accurate, estimates through the use of sophisticated modeling for confounding effects (instead of simplistic GLM) and IGSCA (instead of conventional simplistic SEM). We hope our study may help improve our analytical approach in this field.
- I would be equally cautious to claim that the ABCD study is representative. Please add information on the whole ABCD cohort to Table 1 and describe any relevance with respect to attrition effects or representativeness.
Thank you for highlighting this issue. We previously characterized the ABCD Study as representative of the US population, given its aim to ensure representativeness by recruiting from a broad range of school systems located near each of its 21 research sites, chosen for their geographic, demographic, and socioeconomic diversity. Using epidemiological strategies, a stratified probability sample of schools was selected for each site. This procedure took into account sex, race/ethnicity, socioeconomic status, and urbanicity to reduce potential sampling biases at the school level. Based on these strategies, previous research (e.g., Thompson et al., 2019; Zucker et al., 2018) has referred to the ABCD Study as ‘representative.’ However, we overlooked the fact that “not all 9-year-old and 10-year-old children in the United States had an equal chance of being invited to participate in the study,” and therefore, it should not be deemed fully representative of the US population (Compton et al., 2019). Heeding your suggestion, we have removed all descriptions of the ABCD Study being representative.
Compton, W. M., Dowling, G. J., & Garavan, H. (2019). Ensuring the Best Use of Data: The Adolescent Brain Cognitive Development Study. JAMA Pediatrics, 173(9), 809-810. doi:10.1001/jamapediatrics.2019.2081
Thompson, W. K., Barch, D. M., Bjork, J. M., Gonzalez, R., Nagel, B. J., Nixon, S. J., & Luciana, M. (2019). The structure of cognition in 9 and 10 year-old children and associations with problem behaviors: Findings from the ABCD study’s baseline neurocognitive battery. Developmental Cognitive Neuroscience, 36, 100606. doi:10.1016/j.dcn.2018.12.004
Zucker, R. A., Gonzalez, R., Feldstein Ewing, S. W., Paulus, M. P., Arroyo, J., Fuligni, A., . . . Wills, T. (2018). Assessment of culture and environment in the Adolescent Brain and Cognitive Development Study: Rationale, description of measures, and early data. Developmental Cognitive Neuroscience, 32, 107-120. doi:https://doi.org/10.1016/j.dcn.2018.03.004
- The imputation methods need to be explained in more detail / more clearly. What concrete variables were included? Why was 50% of the sample excluded despite imputation? How similar is the study sample to the overall ABCD cohort - and to the US population in general (i.e., is this a representative dataset)?
Thank you for mentioning this point. We clarified the method and detailed processes of the imputation (e.g., R package VIM, number of missing observations for each study variables such as genotypes, follow-up observations, and positive environment) [Methods; line 167~176].
The final samples had significantly higher cognitive intelligence, parental education, family income, and family history of psychiatric disorders, lower Area Deprivation Index, percentage of individuals below -125% of the poverty level, and family’s financial adversity (p<0.05). As you have noted above, these results also show the limited representativeness of the data used in our study. We fully acknowledge that our study sample, as well as the overall ABCD cohort, is not representative of the US population in general.
- There are a range of unclear statements (e.g., 'Supportive parenting and a positive school environment had the largest total impact on PLEs than genetic or environmental factors' - isn't parenting an environmental factor?).
Thank you for mentioning this point. We clarified seemingly vague expressions and unclear statements. We corrected the sentence you noted as ‘Supportive parenting and a positive school environment had the largest total impact on PLEs than any other genetic or environmental factors’ [line 57~58].
- The authors' conclusion (that these findings have policy implications for improving school and family environmental) are not fully supported by the evidence. E.g., genetic effects were equally large.
Thank you for pointing this out. Our description should be clearer. Our models consistently show that the combined environmental effects of positive family/school environment, and family/neighborhood SES exceeds the genetic effects. We suggest that these findings may have policy implications for “improving the school and family environment and promoting local economic development” [line 62~64].
To clarify, we newly added “Despite the undeniable genetic influence on PLEs, when we combine the total effect sizes of neighborhood and family SES, as well as positive school environment and parenting behavior (∑▒〖|β|〗=0.2718~0.3242), they considerably surpass the total effect sizes of cognitive phenotypes PGSs (|β|=0.0359~0.0502)” [line 510~513]. Based on these results, we suggest that our findings hold potential policy implications for “preventative strategies that target residential environment, family SES, parenting, and schooling—a comprehensive approach that considers the entire ecosystem of children's lives—to enhance children's cognitive ability and mental health” in the Discussion [line 507~510].
Admittedly, our results do not directly demonstrate a causal effect wherein an intervention in the school or family environmental variables would necessarily lead to a significantly meaningful positive impact on a child's cognitive intelligence and mental health. We do not make such a claim in this paper. However, we anticipate that further integrative analyses akin to ours might help identify potential causal or prescriptive effects. We hope this perspective will be recognized as one of the contributions of our study. We leave the final decision to the discerning judgment of the editors and reviewers.
- Many citations do not support the statements made and are sometimes used rather vaguely. For example, I believe Judd 2020 and Okbay 2022 did not use a PGS of cognitive capacity, but of educational attainment. Plomin 2018 and Harden 2020 are reviews, but the primary studies should be cited instead. Which reference exactly is supporting the statement that cognitive capacity PGS links to brain morphometry?
Thank you very much for your precise observations. We thoroughly checked all citations and updated the references for each statement.
We deleted Plomin & von Stumm (2018) and Harden & Koellinger (2020) and cited relevant original research articles (e.g., Lee et al., 2018; Okbay et al., 2022; Abdellaoui et al., 2022) instead. We also specified the references supporting the statement that educational attainment PGS links to brain morphometry (Judd et al., 2020; Karcher et al., 2021). As Okbay et al. (2022) used the PGS of cognitive intelligence (which presented the analyses results in their supplementary materials) as well as educational attainment, we decided to continue citing this reference [line 131~141].
- Citations are formatted inconsistently.
We apologize for the inconsistency of the citation formatting. We formatted all citations in APA 7th style, using EndNote v20. We checked that all citations maintain consistency according to the reference style.
- Re line 281, I believe effect sizes are 'up to twice as large', but not consistently twice as large as suggested in the text.
Thank you for mentioning this point. We corrected the sentence as ‘The effect sizes of EA PGS on children's PLEs were larger than those of CP PGS’ [line 342~343].
- Please add to the results a short statement on what covariates these analyses were controlled for.
Thank you for giving us this comment. We added that we used sex, age, marital status, BMI, family history of psychiatric disorders, and ABCD research sites as covariates in the Results section [line 329~331].
- Cho 2020 does not provide recommendations on FIT values (line 315). Please provide another reference and explain how these FIT values should be interpreted.
Thank you for mentioning this point. We added the correct reference for FIT values (Hwang, Cho, & Choo, 2021). We also added that the FIT values range from 0 to 1, and a larger FIT value indicates more variance of all variables is explained by the specified model (e.g., FIT=0.50 denotes that the model explains 50% of the total variance of all variables) [line 291~293].
- Regarding Figure 2, please add factor loadings to this figure and explain what the difference between the hexagon and circular shapes are. Please also add the autocorrelations between the 3 PLE measures. I assume these were also modelled statistically, given the strong correlations between time points?
Figure 2B needs reworking.
It is unclear what the x-axis of Figure 2C represents. Proportion of R2 or effect size? SM table 2 provides key information, which should be added to Figure 2.
Thank you for pointing this out. We added factor loadings to the corrected figure (Figure 3A and 3B). We also added that the X-axis of Figure 3C represents standardized effect sizes.
- I suggest adding units directly to Table 1, not in the legend. Was genetic or self-reported ethnicity used in this table? List age in years, not months?
Thank you for your suggestion. We added the units of age and family history of psychiatric disorders directly inside Table 1. We used genetic ethnicity in Table 1, as we only used genetic ethnicity (but not self-reported ethnicity) throughout our study. This is noted on the last row of Table 1. We listed age in chronological months, which is how each child’s age at each point of data collection is coded in the ABCD Study.
- Please include exact p-values in Table 2.
Thank you for your suggestion. We highly appreciate the reviewer’s comment on the importance of showing exact p-values in the analysis results. Unfortunately, we cannot estimate the standard errors based on normal-theory approximations to obtain the exact p-values of our IGSCA model results. This is described in detail in the original paper of the IGSCA method (Hwang et al., 2021): “Like GSCA and GSCAM, IGSCA is also a nonparametric or distribution-free approach in the sense that it estimates parameters without recourse to distributional assumptions such as multivariate normality of indicators. As a trade-off of no reliance on distributional assumptions, it cannot estimate the standard errors of parameter estimates based on asymptotic (normal-theory) approximations. Instead, it utilizes the bootstrap method (Efron, 1979, 1982) to obtain the standard errors or confidence intervals of parameter estimates nonparametrically.”
Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7, 1–26. http://dx.doi.org/10.1214/aos/1176344552
Efron, B. (1982). The jackknife, the bootstrap and other resampling plans. Philadelphia, PA: SIAM. http://dx.doi.org/10.1137/1.9781611970319
Hwang, H., Cho, G., Jung, K., Falk, C. F., Flake, J. K., Jin, M. J., & Lee, S. H. (2021). An approach to structural equation modeling with both factors and components: Integrated generalized structured component analysis. Psychological Methods, 26(3), 273-294. doi:10.1037/met0000336
- There are way too many indigestible tables presented in the supplementary materials, which are also not referenced in the main manuscript.
We appreciate your insightful observation. As you rightly identified, we inadvertently failed to reference Table S2 in the main text. We have since corrected this omission in the Results section for the IGSCA (SEM) analysis [line 376]. The remainder of the supplementary tables (Table S1, S3~S7) have been appropriately cited in the main manuscript. We recognize that the quantity of tables provided in the supplementary materials is substantial. However, given the comprehensiveness and complexity of our analyses, these tables offer intricate results from each analysis. We deem these results, which include valuable findings from sensitivity analyses and confound testing, too significant to exclude from the supplementary materials. That said, we are open to, and would greatly welcome, any further suggestions to ensure clarity and ease of comprehension. Your guidance in this matter is highly valued.
- Figure S1 is unclear, possibly due to the journal formatting. Is this one figure presented on two pages? Clarify which PGS is listed in Figure S1 and in any case, please add both PGSs.
Thank you for mentioning this point. Figure S1 presents two correlation matrices: the first one is the correlation matrix of component / factor variables in the IGSCA model and the second one is the that of observed variables used to construct the relevant component / factor variables in the IGSCA model. We noted each matrix as Figure S1-A and Figure S1-B. We also corrected the figure legend as “A. Correlation between all component / factor variables of the IGSCA model. B. Correlation between all observed variables used to construct the relevant component / factor variables in the IGSCA model.” Since Figure S1-A presents correlations between the components and latent factors, it lists a single PGS variable constructed from the CP PGS and EA PGS. On the other hand, Figure S1-B presents correlations between the observed variables. Thus, both CP PGS and EA PGS are listed in this correlation matrix.