Generative power of a protein language model trained on multiple sequence alignments

  1. Damiano Sgarbossa
  2. Umberto Lupo  Is a corresponding author
  3. Anne-Florence Bitbol  Is a corresponding author
  1. Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland

Abstract

Computational models starting from large ensembles of evolutionarily related protein sequences capture a representation of protein families and learn constraints associated to protein structure and function. They thus open the possibility for generating novel sequences belonging to protein families. Protein language models trained on multiple sequence alignments, such as MSA Transformer, are highly attractive candidates to this end. We propose and test an iterative method that directly employs the masked language modeling objective to generate sequences using MSA Transformer. We demonstrate that the resulting sequences score as well as natural sequences, for homology, coevolution and structure-based measures. For large protein families, our synthetic sequences have similar or better properties compared to sequences generated by Potts models, including experimentally-validated ones. Moreover, for small protein families, our generation method based on MSA Transformer outperforms Potts models. Our method also more accurately reproduces the higher-order statistics and the distribution of sequences in sequence space of natural data than Potts models. MSA Transformer is thus a strong candidate for protein sequence generation and protein design.

Data availability

Python code for generating sequences using the iterative masking procedure is available in our GitHub repository: https://github.com/Bitbol-Lab/Iterative_masking.Raw data was collected from two public sources: 1) MSAs from the Pfam database (https://pfam.xfam.org/); 2) further MSAs from https://github.com/matteofigliuzzi/bmDCA. We generated sequences with bmDCA using code publicly available at https://github.com/ranganathanlab/bmDCA.

Article and author information

Author details

  1. Damiano Sgarbossa

    Institute of Bioengineering, Ecole Polytechnique Federale de Lausanne (EPFL), Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7878-6061
  2. Umberto Lupo

    Institute of Bioengineering, Ecole Polytechnique Federale de Lausanne (EPFL), Lausanne, Switzerland
    For correspondence
    umberto.lupo@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.
  3. Anne-Florence Bitbol

    Institute of Bioengineering, Ecole Polytechnique Federale de Lausanne (EPFL), Lausanne, Switzerland
    For correspondence
    anne-florence.bitbol@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-1020-494X

Funding

European Research Council (851173)

  • Damiano Sgarbossa
  • Umberto Lupo
  • Anne-Florence Bitbol

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Copyright

© 2023, Sgarbossa et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 5,596
    views
  • 750
    downloads
  • 22
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Damiano Sgarbossa
  2. Umberto Lupo
  3. Anne-Florence Bitbol
(2023)
Generative power of a protein language model trained on multiple sequence alignments
eLife 12:e79854.
https://doi.org/10.7554/eLife.79854

Share this article

https://doi.org/10.7554/eLife.79854

Further reading

    1. Computational and Systems Biology
    2. Genetics and Genomics
    Jia-Ying Su, Yun-Lin Wang ... Chien-Ling Lin
    Research Article

    Untranslated regions (UTRs) contain crucial regulatory elements for RNA stability, translation and localization, so their integrity is indispensable for gene expression. Approximately 3.7% of genetic variants associated with diseases occur in UTRs, yet a comprehensive understanding of UTR variant functions remains limited due to inefficient experimental and computational assessment methods. To systematically evaluate the effects of UTR variants on RNA stability, we established a massively parallel reporter assay on 6555 UTR variants reported in human disease databases. We examined the RNA degradation patterns mediated by the UTR library in two cell lines, and then applied LASSO regression to model the influential regulators of RNA stability. We found that UA dinucleotides and UA-rich motifs are the most prominent destabilizing element. Gain of UA dinucleotide outlined mutant UTRs with reduced stability. Studies on endogenous transcripts indicate that high UA-dinucleotide ratios in UTRs promote RNA degradation. Conversely, elevated GC content and protein binding on UA dinucleotides protect high-UA RNA from degradation. Further analysis reveals polarized roles of UA-dinucleotide-binding proteins in RNA protection and degradation. Furthermore, the UA-dinucleotide ratio of both UTRs is a common characteristic of genes in innate immune response pathways, implying a coordinated stability regulation through UTRs at the transcriptomic level. We also demonstrate that stability-altering UTRs are associated with changes in biobank-based health indices, underscoring the importance of precise UTR regulation for wellness. Our study highlights the importance of RNA stability regulation through UTR primary sequences, paving the way for further exploration of their implications in gene networks and precision medicine.

    1. Computational and Systems Biology
    2. Medicine
    Hong Yang, Cheng Zhang ... Adil Mardinoglu
    Research Article

    Excessive consumption of sucrose, in the form of sugar-sweetened beverages, has been implicated in the pathogenesis of metabolic dysfunction‐associated fatty liver disease (MAFLD) and other related metabolic syndromes. The c-Jun N-terminal kinase (JNK) pathway plays a crucial role in response to dietary stressors, and it was demonstrated that the inhibition of the JNK pathway could potentially be used in the treatment of MAFLD. However, the intricate mechanisms underlying these interventions remain incompletely understood given their multifaceted effects across multiple tissues. In this study, we challenged rats with sucrose-sweetened water and investigated the potential effects of JNK inhibition by employing network analysis based on the transcriptome profiling obtained from hepatic and extrahepatic tissues, including visceral white adipose tissue, skeletal muscle, and brain. Our data demonstrate that JNK inhibition by JNK-IN-5A effectively reduces the circulating triglyceride accumulation and inflammation in rats subjected to sucrose consumption. Coexpression analysis and genome-scale metabolic modeling reveal that sucrose overconsumption primarily induces transcriptional dysfunction related to fatty acid and oxidative metabolism in the liver and adipose tissues, which are largely rectified after JNK inhibition at a clinically relevant dose. Skeletal muscle exhibited minimal transcriptional changes to sucrose overconsumption but underwent substantial metabolic adaptation following the JNK inhibition. Overall, our data provides novel insights into the molecular basis by which JNK inhibition exerts its metabolic effect in the metabolically active tissues. Furthermore, our findings underpin the critical role of extrahepatic metabolism in the development of diet-induced steatosis, offering valuable guidance for future studies focused on JNK-targeting for effective treatment of MAFLD.