1. Introduction

Protein engineering, situated at the nexus of molecular biology, bioinformatics, and biotechnology, focuses on the design of proteins to introduce novel functionalities or enhance existing attributes.[1–3]. With the exponential growth of biological data and computational power, protein engineering has experienced a significant shift towards advanced computational methodologies, particularly deep learning, to expedite the design process and unravel complex protein-function relationships[4–9]. The zero-shot prediction of large-scale protein language models[10–13] can predict mutants without any prior experimental data, based on the self-supervised learning of numerous protein sequences to understand the law of protein sequence composition in natural[14–16]. However, a pressing challenge confronting protein engineering endeavors in industrial settings pertains to the exigent demand for proteins with inherent resistance to extreme conditions, such as high temperature and extreme pH environments (acidic or alkaline)[17, 18]. Unlike proteins in natural ecosystems, those used in industrial processes often encounter harsh physical and chemical conditions, necessitating exceptional resilience to maintain functionality[19, 20]. Most proteins in nature typically operate under ambient temperature and pressure. Even organisms thriving in extreme chemical environments harbor proteins shielded by cellular membranes, thereby sheltering them from extreme pH environment[21, 22]. Previous efforts to enhance protein resistance have often relied on rational design and mutant library screening. These methods are typically labor-intensive, inefficient, and yield limited improvements[23–26]. Consequently, the industrial demand for proteins resilient to harsh environments poses a notable absence within the training datasets of Artificial Intelligence (AI) models. Exploring whether AI can achieve the evolution of protein resistance to extreme environments is crucial for broadening protein applications and improving modification efficiency.

Here, we employed a LLM (large language model) developed by our group, the Pro-PRIME model[27], to predict dozens of mutants of a nano-antibody against growth hormone (a VHH antibody), and examined their fitness, including alkali resistance and thermostability, to evaluate their performance under extreme environments. The architecture and performance of the Pro-PRIME model can be found in Ref. [27]. It has been shown that the Pro-PRIME model can provide single-site mutants with improved fitness for distinct engineering indices. The VHH antibody, derived from heavy chain-only antibodies, serves as the antigen-binding fragment[28]. Capitalizing on its advantageous attributes including compact size, monomer state, structural robustness, and easy tailoring, VHH has emerged as a pivotal asset in medical research and the development of clinical antibody-based therapeutics[29], which have been utilized as an affinity ligand to selectively purify biopharmaceutical, for example prothrombin, tetrabromobisphenol A, intercellular adhesion molecule 1 and so on[30–32]. In practical production, the clean-in-place (CIP) method involves cleaning with 0.5 M NaOH for 24 hours to eliminate accumulation of impurities and contaminants. Therefore, to enable the application of VHH antibodies in efficient and low-cost biopharmaceutical pro[26, 33]. Although AI-guided protein design has been applied to predict the mutants with greater thermostability and higher activity[34–36], it is unexplored whether an AI model based on the natural protein information can find the mutants that adapt the unnatural extreme environments, such as the alkaline solution with the pH value higher than 13.

In this study, we utilized the Pro-PRIME model to score saturated single-point mutations of the VHH in a zero-shot setting, and selected the top 45 mutants for experimental testing. Some mutants exhibited improved alkali resistance, while others demonstrated higher thermal stability or affinity. Subsequently, we fine-tuned the Pro-PRIME model to predict dozens of multi-point mutations. As a result, we obtained three multi-point mutants with enhanced alkali resistance, higher thermostability, as well as strong affinity to the targeted protein. Also, the dynamic binding capacity of the selected mutant did not show significant decline after more than 100 cycles, making it suitable for practical application in industrial production. The selected mutant has been used in practical production and lower the cost for over one million dollars in a year. To the best of our knowledge, this is the first protein product developed by a LLM that has been successfully applied in mass production. Due to the Pro-PRIME model’s ability to achieve precise predictions of multi-point mutations with reliance on a small amount of experimental data, our two-round design process involved experimental validation of only 65 mutants in two months, demonstrating remarkable efficiency. Furthermore, we performed a systematic analysis of these findings and determined that the model can yield more valuable predictive outcomes while remaining consistent with rational design principles. Specifically, within the framework of multi-point combinations, the model’s incorporation of negative single-point mutations into the combinatorial space led to exceptional results, showcasing its capacity to capture epistatic interactions. Notably, in striving for global optimum, deep learning methods offer distinct advantages over traditional rational design approaches.

2. Results and Discussion

2.1. Wet experimental testing of single-point mutants

Currently, the prevailing approach in protein engineering involves rational design or high-throughput screening to identify positive single-point mutations, followed by their combination into more effective multi-point mutations through greedy search methods[37]. However, these approaches are inclined to trap in a local optimum and unable to avoid the negative epistasis[38, 39]. To overcome these limitations, we employed the Pro-PRIME model to screen single-point mutations in the first round, enabling rapid inference of scores for saturated mutations. An efficient strategy involves validating only a select number of top-ranked mutants, thereby saving considerable time and reducing economic costs compared to rational design and high-throughput experiment. Moreover, we posit that deep learning models can capture nuanced features during training, and recommend single-point mutations that expert experience may not cover. These mutants expand possibilities for subsequent combination of multi-point mutations and mitigate the risk of converging into local optima.

Specifically, we made use of the Pro-PRIME model to score the single-site saturated mutants, and selected the top 45 single-point mutants of the VHH antibody for testing. Pro-PRIME is a deep learning-based methodology developed to guide protein engineering, utilizing masked language modeling (MLM) and multi-task learning to study and comprehend the semantic and grammatical features inherent in protein sequences, and further capture the temperature traits associated with these sequences. A higher score corresponds to a greater appearance probability of the residue at this site in the Pro-PRIME model[27]. The alkali resistance of a mutant was determined by its half maximal effective concentration (EC50) after treatment with alkali. A lower EC50 represents that antibodies have stronger affinity. Firstly, the 45 single-point mutants were treated with 0.3 M NaOH for 24 hours, and evaluated by their thermal stability and affinity. The results revealed that 15 mutants exhibited higher alkali resistance, and 35 mutants displayed higher melting temperatures (T_m) (Figure 1A). We also noted that 8 out of 45 single-site mutants designed by Pro-PRIME showed improved affinity as compared to the WT before alkali treatment (Figure S1). It is worth noting that six mutants (A57D, A15P, V113D, P117Q, R20T, and L12K) exhibited enhancements in all three properties, i.e., higher alkali resistance, greater thermal stability, and stronger affinity before alkali treatment. Additionally, eight mutants (P29T, N85Q, G134R, N78S, R110E, L49V, T58K, and W112F) exhibited improved alkali resistance and T_m, albeit at the moderate cost of affinity. This trade-off is acceptable, as excessively high affinity could complicate the separation of growth hormone from the VHH. Furthermore, we subjected single-point mutants with remarkable alkali resistance to treatment with 0.5 M NaOH, and measured their affinity after 24 hours. Our findings demonstrated that four mutants (L49V, A15P, W112F, and R20T) exhibited higher affinity than the wild type following the same treatment protocol (Figure 1B). Interestingly, these four single-point mutations all showed improvements in T_m, and their affinities before alkali treatment were either similar to or lower than those of the wild type. However, this does not imply that enhancing thermal stability will concurrently improve alkali resistance. The spearman correlation between the EC50 values after 0.3 M NaOH treatment and T_m values of single-point mutants is -0.29, indicating that these two properties of the antibody are only weakly correlated (Figure S3). Therefore, achieving multi-point mutants that enhance both alkali resistance and thermal stability remains a challenge.

Figure 1.

2.2. Multi-point mutation design driven by experimental data

In the process of combining multiple single-site mutations, common greedy algorithms typically proceed by sequentially adding the most effective single-point mutations and gradually increasing the number of stacked mutations. However, this approach often demonstrates low efficiency and is susceptible to becoming trapped in local optima. Importantly, the most effective multi-point mutations in practical scenarios may not necessarily include the most effective single-point mutations[39]. Moreover, in the traditional approach of incrementally stacking mutations from single points to multiple points, only paths that yield better results at each step are retained[38]. Nevertheless, combinations that are discarded due to decreased effectiveness may significantly enhance performance in subsequent stacking step, a phenomenon unforeseeable in traditional methods.

To better search the global optimium, we chose the Pro-PRIME model to predict the results of multi-point mutants due to its remarkable performance on the design of diverse proteins[27]. We trained two Pro-PRIME models using the alkali resistance and T_m data of single-point mutants, respectively, to score these two properties of mutants. During the prediction of multi-point mutations, we prioritized mutants with higher alkali resistance scores while ensuring T_m scores were not lower than those of the wild type. Ultimately, we selected 20 multi-point mutants for experimentation (see details in Supporting Information, SI). Experimental results indicated that the alkali resistance of five multi-point mutants (A57D;P29T, A57D;D114Y;W112F, A15P;R20T, A15P;P117Q, and A57D;G134R;D114Y) surpassed that of the best single-point mutant. The T_m of multi-point mutants was generally higher, with the highest (P29T;A15P) exceeding that of the wild type by approximately 10 °C (Figure 1B). Although we did not optimize for the affinity of the VHH antibody, certain multi-point mutants (A15P;R20T, A15P;G134R, and P29T;G134R) exhibited affinity levels close to or even exceeding those of the wild type (Figure S2). Although experimental results from single-point mutations suggested that simultaneously improving the alkali resistance and thermal stability of VHH might be challenging, we successfully designed multi-point mutants that balance multiple properties. This demonstrates the excellent multi-objective optimization capability of the Pro-PRIME model.

2.3. Complex epistatic effects generated by the combination of single-point mutations

To understand the principle of the evolution, we analyzed the improvement in multi-point mutants across three different indicators. As illustrated in Figure 2B, the evolution of thermal stability in the second round was successful, with nearly all multi-point mutants exhibiting an increase in T_m compared to the best single-point mutant they contain. Three multi-point mutants demonstrated higher affinity than the wild type, but only P29T and G134R exhibited positive epistasis, resulting in the creation of a double-point mutant with superior affinity. Although A15P;R20T and A15P;G134R showed higher affinity than the wild type, these combinations did not contribute to affinity improvement because the A15P single-point mutation they contained exhibited stronger affinity compared to these double-point mutants.

Figure 2.

As shown in Figure 2A, single-point mutations produced complex results during the combination process due to epistatic effects, which were unpredictable using traditional methods. A57D;P29T exhibited remarkably high alkali resistance, despite A57D and P29T being negative mutants individually. This “double negative yields positive” phenomenon also occurred in terms of affinity, where the EC50 value of P29T;G134R was approximately one-third of that of the wild type, despite both P29T and G134R caused a decrease in affinity individually, with the EC50 value of P29T being larger than that of the wild type by a factor of five. These mutation pairs are distantly located on the VHH antibody structure, making it challenging to infer the reasons for the enhanced properties after combination (Figure 3B). Not all combinations resulted in improved mutant effects. For instance, while A15P showed high affinity and alkali resistance, adding R20T or P29T led to decrease in affinity or alkali resistance, respectively. Additionally, combining A15P with P117Q, two single mutations capable of individually enhancing affinity, resulted in a decrease in affinity.

Figure 3.

In traditional methods, only positive single-point mutations are selected to compose the multi-point mutants and negative single-point mutations are typically avoided in multi-point combinations. However, the number of negative mutations often far exceeds that of positive mutations, thus limiting the sequence space that can be explored by traditional methods. Large language models can leverage negative mutations to generate positive multi-point mutants, surpassing the capabilities of rational design and significantly expanding the design space in protein engineering. Therefore, the Pro-PRIME method manifests significant advantages in exploring sequence space, being less susceptible to local optima, and having greater potential to find the global optimum. As depicted in Figure 2C, the 20 multi-point mutations we identified consisted of combinations of nine different single-point mutations. Although the distribution was biased, we incorporated some negative mutations that experts might consider disadvantageous, implying that the Pro-PRIME method balances the exploitation of local optima and the exploration of global optimum.

2.4. Reevaluating the AI predictions from a rational design perspective

In addition to interpret the multi-point mutations based on the single-site mutations, we investigated the design of VHH antibodies from a rational design perspective to understand the AI predictions. The experimental results from SDS-PAGE revealed a significant degradation of the VHH antibody following alkali treatment, with small bands accounting for up to 61.1%. This phenomenon could be attributed to the poor alkali resistance of the VHH antibody. Mass spectrometry analysis identified specific breakage sites at Q2, Q4, G9, G10, G16, S22, S23, S26, A57, N78, N85, F101, and G134, highlighting these as ideal targets for rational design. Introducing mutations at these sites to prevent the breakdown of VHH antibodies could effectively enhance their alkali resistance. The predictions of the Pro-PRIME model align with this rationale, with 4 out of the top 10 performing single-point mutations occurring at these breakpoint sites. Experimental evidence showed that our single-point mutations could effectively reduce the proportion of small bands, and not all positive mutations locate at the breakage sites, such as P29T and A15P (Figure 3A). Mutations identified by the model at other locations may have non-local effects, contributing to overall protein stability. In the second round of experiments, multi-point mutants exhibited an overall lower proportion of small bands, reaching as low as 13.6% (A57D;G134R;D114Y), though we did not use the experimental results of SDS-PAGE to train the model and make predictions. Thus, the improvement in the second round suggests a positive correlation between the alkali resistance of the VHH antibody and the degree of breakage after alkali treatment. Employing a rational design approach, it is plausible to identify single-point mutants with improved alkali resistance at these breakpoints. However, determining the optimal mutated residues requires conducting multiple single-point saturation mutation experiments, which is time consuming and costly. Deep learning methods can directly predict mutation types at each site, enhancing design efficiency while remaining aligned with rational design principles.

Furthermore, we analyzed the VHH antibody and homologous sequences from the perspective of evolutionary sequence alignment (Figure 3C). The results revealed that some critical single-point mutations (P29T, L49V) changed original residues to more conservative ones, aligning with the “consensus sequence” method commonly used in rational design[40]. The Pro-PRIME model’s ability to predict such mutations is expected, given its training set contains information from homologous proteins. However, the advantage of deep learning methods extends beyond providing suggestions aligned with rational design principles; they can predict mutations contrary to expert experience. For instance, both A15 and R20 are conservative residues according to homologous sequence alignment (see Figure 3C). Moreover, mutating A15 to proline reduces hydrogen bonds, and changing R20 to threonine contradicts empirical physicochemical properties, since arginine is more likely to form salt bridges and hydrogen bonds, benefiting stability. However, both A15P and R20T contribute to increased alkali resistance and play positive roles in subsequent multi-point combinations. This exceptional capability beyond the traditional rational-design principles expands the exploration space of deep learning models, aiding to approach the global optimum.

2.5. Improved acid and salt resistance simultaneously

According to the rational design principles, a protein with enhanced intramolecular interactions can resist various stress, including high temperature, strong alkali, strong acid, concentrated salt, etc.[41, 42] Hence, we expected that the mutants with higher thermal stability and alkali resistance can tolerate acidic and saline environments as well. We evaluated the binding affinity of the mutants to the target protein under elution with saline (1 M NaCl) or acidic (20 mM citric acid) solutions to characterize their salt-induced and acid-induced dissociation abilities. Additionally, we assessed the acid resistance of the mutants by measuring their affinity after a 48-hour treatment with 1 M ethanoic acid (see details in SI). Figure 4 shows the acid and salt resistance of the selected multi-point mutants with higher thermal stability and stronger alkali resistance. It is noteworthy that the Pro-PRIME model can provide us the mutants, such as A57D;P29T and A15P;R20T, with strong resistance on acid and salt (Figure 4A) as well as high temperature and alkali resistance (Figure 2B). Our design enhances protein stability while retaining other properties such as affinity, thereby endowing the mutant with the potential for application in industrial production.

Figure 4.

2.6. Application in industrial production

The Pro-PRIME method can predict multi-site mutants using a small amount of experimental data, and the entire design process can be completed within two months. This method has the potential to empower industrial production due to its relatively low economic and time costs. Our mutants have already been widely applied in the purification process of growth hormones. Figure 4B shows the ratio of the residual dynamic binding capacity (DBC) after 0.5 M NaOH treatment for 6 hours and 24 hours to the residual DBC before treatment for various mutants and the wild type. The wild type experienced a 74.1% loss in the residual DBC after 6 hours of alkali treatment, and only retained 15.2% of its residual DBC after 24 hours of alkali treatment. In contrast, the designed multi-site mutants retained 60-90% of their residual DBC after 6 hours of alkali treatment, and even after 24 hours of treatment, some mutants maintained over 50% of their residual DBC, which is remakably more stable than the wild type in the alkaline environment of industrial production. Figure 4C shows the variation in yield during affinity chromatography across multiple cycles after employing our mutant. After more than 140 cycles, the yield of our mutant did not exhibit a significant downward trend. In contrary, the binding affinity of wild type is unable to sustain after 60 cycles. Therefore, our designed mutants maintain activity after more cycles of reuse compared to the wild type, substaintially reducing the production cost of growth hormones. Consequently, the selected mutant designed by the LLM has been applied in mass production scale up to 5,000 liters.

3. Conclusions

Through two rounds of evolution, we successfully designed a VHH antibody with strong resistance to extreme environments and enhanced affinity using the Pro-PRIME model. Although rare case can tolerate the extreme pH and saline conditions in our pre-training dataset, the Pro-PRIME model showed impressive performance after supervised learning with limited data, especially on capturing the epistatic effects. The analysis of these 65 mutants revealed that the Pro-PRIME model is adept at exploring the large space of protein fitness, being less susceptible to local optima, and having greater potential to find the global optimum. Our efficient method of designing mutants that consider multiple properties improvement holds promise for industrial application of proteins. Specifically, the VHH antibody has been deployed in practical production and significantly enhancing the efficiency of the entire production line after our design. This work demonstrates that the AI model can not only guide the evolution of protein thermal stability, enzymatic activity, ligand affinity, etc., but also enable to develop the mutants adapting the harsh unnatural environments, such as extreme pH and concentrated salt, largely expanding its application. The great potential of AI model is expected to significantly accelerate the development of medicine, agriculture, bioengineering, etc.

Additional information

CRediT authorship contribution statement

Liqi Kang: Writing – original draft, Data curation, Conceptualization, Methodology. Banghao Wu: Data curation, Methodology. Bingxin Zhou: Writing – review & editing. Pan Tan: Writing – review & editing. Yun (Kenneth) Kang: Data curation. Yongzhen Yan: Data curation. Yi Zong: Data curation. Shuang Li: Data curation. Zhuo Liu: Writing – review & editing, Supervision. Liang Hong: Writing – review & editing, Supervision.

Conflicts of interest

Patents have been filed in 2023 for all these mutations in the present work. GeneScience Pharmaceuticals, a company that discovered the native sequence of VHH, own the patent of VHH.

Data availability

Data will be made available on request.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (12204302), the Computational Biology Program of Shanghai Science and Technology Commission (23JS1400600), the Shanghai Pujiang Program (Grant No. 22PJ1406900), the Startup Fund for Young Faculty at SJTU (SFYF at SJTU), the Oceanic Interdisciplinary Program of Shanghai Jiao Tong University (Project No. SL2022MS018), the Natural Science Foundation of Shanghai (Grant No. 23ZR1431700), Shanghai Jiao Tong University Scientific and Technological Innovation Funds (21X010200843), Science and Technology Innovation Key R&D Program of Chongqing (CSTB2022TIAD-STX0017), the Student Innovation Center at Shanghai Jiao Tong University, and Shanghai Artificial Intelligence Laboratory.