TY - JOUR TI - Transformer-based deep learning for predicting protein properties in the life sciences AU - Chandra, Abel AU - Tünnermann, Laura AU - Löfstedt, Tommy AU - Gratz, Regina A2 - Dötsch, Volker VL - 12 PY - 2023 DA - 2023/01/18 SP - e82819 C1 - eLife 2023;12:e82819 DO - 10.7554/eLife.82819 UR - https://doi.org/10.7554/eLife.82819 AB - Recent developments in deep learning, coupled with an increasing number of sequenced proteins, have led to a breakthrough in life science applications, in particular in protein property prediction. There is hope that deep learning can close the gap between the number of sequenced proteins and proteins with known properties based on lab experiments. Language models from the field of natural language processing have gained popularity for protein property predictions and have led to a new computational revolution in biology, where old prediction results are being improved regularly. Such models can learn useful multipurpose representations of proteins from large open repositories of protein sequences and can be used, for instance, to predict protein properties. The field of natural language processing is growing quickly because of developments in a class of models based on a particular model—the Transformer model. We review recent developments and the use of large-scale Transformer models in applications for predicting protein characteristics and how such models can be used to predict, for example, post-translational modifications. We review shortcomings of other deep learning models and explain how the Transformer models have quickly proven to be a very promising way to unravel information hidden in the sequences of amino acids. KW - deep learning KW - transformers KW - life sciences KW - protein property prediction KW - machine learning JF - eLife SN - 2050-084X PB - eLife Sciences Publications, Ltd ER -