Deep Batch Active Learning for Drug Discovery

Michael Bailey; Saeed Moayedpour; Ruijiang Li; Alejandro Corrochano-Navarro; Alexander Kötter; Lorenzo Kogler-Anele; Saleh Riahi; Christoph Grebner; Gerhard Hessler; Hans Matter; Marc Bianciotto; Pablo Mas; Ziv Bar-Joseph; Sven Jager

doi:10.7554/eLife.89679.2

eLife assessment

This valuable study reports novel active learning batch selection methods that have been applied to optimization tasks related to ADMET and affinity properties relevant within the drug discovery field. While the evidence is solid, the paper could have benefited from a clearer and deeper description of methods as well as interpretation of the obtained models, and a wider comparison to existing methods. The article will be of general interest to scientist working in the field of drug discovery and, in general, to researchers within the fields of machine learning and data analysis.

https://doi.org/10.7554/eLife.89679.2.sa2

Significance of findings

valuable: Findings that have theoretical or practical implications for a subfield

landmark
fundamental
important
valuable
useful

Strength of evidence

solid: Methods, data and analyses broadly support the claims with only minor weaknesses

exceptional
compelling
convincing
solid
incomplete
inadequate

During the peer-review process the editor and reviewers write an eLife assessment that summarises the significance of the findings reported in the article (on a scale ranging from landmark to useful) and the strength of the evidence (on a scale ranging from exceptional to inadequate). Learn more about eLife assessments

Abstract

A key challenge in drug discovery is to optimize, in silico, various absorption and affinity properties of small molecules. One strategy that was proposed for such optimization process is active learning. In active learning molecules are selected for testing based on their likelihood of improving model performance. To enable the use of active learning with advanced neural network models we developed two novel active learning batch selection methods. These methods were tested on several public datasets for different optimization goals and with different sizes. We have also curated new affinity datasets that provide chronological information on state-of-the-art experimental strategy. As we show, for all datasets the new active learning methods greatly improved on existing and current batch selection methods leading to significant potential saving in the number of experiments needed to reach the same model performance. Our methods are general and can be used with any package including the popular DeepChem library.

Introduction

The process that leads from a small molecule that shows some activity against the target of interest to a candidate for clinical development involves complex multi-parameter optimization. In this process, in addition to the activity against the target itself, the ADMET profile of the molecule, i.e. its Absorption, Distribution, Metabolism, Excretion, and Toxicity properties are optimized. Accurate in silico models for the desired properties are required to speed up and improve decision making and reduce the number of necessary experimentsHessler and Baringhaus (2018); Grebner et al. 2022); Wu et al. (2020). Amongst other machine learning (ML) techniques, deep learning models, and more specifically (graph) neural neural networks, have been used successfully in this fieldXiong et al. (2021).

Scale-effective ADMET and affinity prediction methods require an abundance of labeled training data due to their complexity and the need to cover the enormous molecular design space. This can be a bottleneck in cost, time, and experimental resourcesChuang et al. (2020). Various approaches have been proposed for improving data acquisition and the models, including transfer learning Weiss et al. (2016), data augmentation Cai et al. (2020) and active learning Cohn et al. (1996a).

Current optimization approaches often work in cycles. In each cycle a set of molecules are tested, the model is revised and, based on the revised model, a new set is selected for testing, and so on until the model reaches the desired performance.

Active learning Cohn et al. (1996b) is an approach for selecting molecules for each of the cycles. Unlike traditional approaches that test the most promising candidates in each round Cohn et al. (1994) in active learning samples are selected to optimize the model rather than the cycle result. In such approach optimization samples are prioritized by their ability to improve model performance when labeled. Active learning has been studied in sequential mode, where samples are labelled one-at-a-time,and batch mode, where samples are selected for labelling in batches Settles (2012). Batch mode is both more realistic for small molecule optimization and more challenging computationally. The main problem is that samples are not independent (sharing chemical properties that will influence model parameters) and so selecting a set based on marginal improvement does not reflect well the improvement provided by the entire batch Ash et al. (2021).

Active learning methods have been utilized to predict and optimize the physiochemical and biological properties of molecular systems. For instance, batch active learning (BMDAL) was used to enhance the model accuracy in predicting the conformations, energetics, and interatomic forces of small organic compounds where the diversity of the training data is a limiting factor.Zaverkin et al. (2022). Active learning was also combined with pairwise difference regression (PADRE)Tynes et al. (2021) to predict molecular properties including redox free energy. Reker et al developed an active learning based tool to screen and select top candidates for ligand-target binding prediction Reker et al. (2017). Thompson et al.Thompson et al. (2022) employed Active learning framework to build a package for the binding free energy of to TYK2 Kinase. In another study model uncertainty for unlabeled pharmacokinetics was used to set an active learning pipeline for plasma exposure prediction Ding et al. (2021). Pertusi et al Pertusi et al. (2017), used an active learning data selection method for characterizing enzyme promiscuity. Naik et al Naik et al. (2016) used active learning to build models to predict molecule effects on subcellular protein localization. However, while some of these methods worked well, they were not applied to the more advanced deep learning methods that have become the tool of choice for small molecule modeling.

On the theoretical side, a number of Batch Active learning methods have recently been developed though these have not been used in the drug design space. For example, BAITAsh et al. (2021) uses a probabilistic approach for the learning procedure that optimally selects (using greedy approximation) a set of samples that maximizes the likelihood of the model parameters (last layer) as defined by Fisher informationAsh et al. (2021). Others have proposed using the local approximation to estimate the maximum of the posterior distribution over the batch. This is achieved through computation of the inverse Hessian of the negative log posterior Daxberger et al. (2021). Recently, GeneDiscoMehrjou et al. (2021) was published as an open source library of benchmarking data pertaining to transcriptomics active learning work. However, these methods have not been extensively tested for small molecules optimization. Specifically, to date, popular in silico design suits, including ChemML Haghighatlari et al. (2019) and DeepChem dee (2016), do not support active leaning strategies.

To address these shortcoming, we developed a new innovative and generalizable strategy that can be used with any deep learning ADMET methods. Our methods are inspired by the Bayesian deep regression paradigm, where estimating the model uncertainty is tantamount to obtaining the posterior distribution of the model parametersKendall and Gal (2017). Model uncertainty is determined using innovative sampling strategies, with no extra model training required.We next select batches that maximize the joint entropy, i.e., the log-determinant of the epistemic covariance of the batch predictions. This enforces batch diversity by rejecting highly correlated batches.

To evaluate our methods and compare them to state of the art approaches we have assembled a large collection of datasets. As we show, our active learning algorithm consistently leads to the best performance when compared to prior methods suggested for this task.

We also generated new data for internal candidates and show that our methods can significantly save cost and time compared to current industry optimization approaches for these datasets.

Results

We developed and tested several batch active learning selection approaches which quantify the uncertainty over multiple samples. Given these uncertainties our method aims to select the subset of samples with maximal joint entropy (i.e., information content) (Figure 1). Specifically, we use multiple methods to compute a covariance matrix, C, between predictions on unlabeled samples, 𝒱. Then, using an iterative and greedy approach, the method selects a submatrix C_B of size B × B from C with maximal determinant. Such approach takes into account both the “uncertainty” (which is manifested in the variance of each sample) and the “diversity” (which is reflected in the covariance). See Methods for details.

Method overview. a) The active learning loop. We consider the entire dataset as the pool, and the oracle as the entity that holds the corresponding labels. During each iteration of the active learning loop, we select a batch of points from the pool and obtain their corresponding labels from the oracle. We then update our model using the selected batch and evaluate its performance on a holdout set. This process is repeated until the desired level of performance is achieved. b) Prediction of binding affinity is the target function for the ChEMBL and Sanofi-Aventis datasets c) Active learning batch selection. At the last layer of our model, we use either Laplace approximation or Monte Carlo dropout to compute covariances (COVLAP and COVDROP), from which an ensemble of predictions is generated. With the derived covariance matrix, we optimize batches iteratively based on their information content.

Evaluating active learning method on ADMET and affinity related data

We used several public drug design datasets to test and compare the performance of our methods. For comparison, in addition to the two methods we developed (MC dropout and Laplace Approximation, COVDROP and COVLAP, respectively) we also compared to methods that have been previously suggested (k-meansNguyen and Smeulders (2004) and BAITAsh et al. (2021) and to a random ordering of the experiments (i.e. no active learning). Batch size was set to 30 for all methods. During each iteration of the loop, each model (e.g., k-Means, BAIT, Random, COVDROP, or COV-LAP) selects a batch consisting of a fixed number of samples from the unlabeled pool. This iterative process is repeated until all labels in the Oracle are exhausted. In the retrospective setting, the pool includes all samples from the relevant dataset, while the oracle retains the corresponding labels.

Our evaluation datasets included a cell permeability dataset with 906 drugs Wang et al. (2016), aqueous solubility dataset, which comprises 9,982 small molecules Sorkun et al. (2019); Huang et al. (2022), and the lipophilicity data for 1200 small molecules Wenlock and Tomkinson (2015); Wu et al. (2018). We also included 10 large affinity datasets, 6 from ChEMBL and 4 new internal datasets. Details for all datasets are provided in Methods and in Table 1.

Number or required experiments for reaching model error with 10% higher than the minimum calculated RMSE for the whole training set across different methods. NC - Number of compounds in this dataset. Chron - analysis performed with batch selection using the actual order in which the data was profiled (only available for internal data). % gain estimates the improvement over the Random selection computed from (1 − N_COVDROP/N_Random) × 100. Where N_COVDROP as well as N_Random represents the average number of experiments out of 25 active learning cycles executed and a batch size of 32.

Comparison of active learning methods for solubility datasets

Results for a selected subset of the benchmarked datasets are presented in Figure 2. In these figures we present the accuracy of models when using the different active learning methods as a function of the iteration. As can be seen, in most cases the COVDROP method very quickly leads to better performance when compared to other methods.

Performance of different methods for optimizing models for ADMET and affinity related datasets and batch selection strategies: a) the cell effective permeability b) lipophilicity. Panels c–f are affinity measurements to various targets proteins: c) GSK3β (ChEMBL) d) MMP3 (ChEMBL) e) Renin (Sanofi-Aventis) f) FXa (Sanofi-Aventis)

The overall shape of the RMSE profiles is impacted by the statistics of the target values in each dataset. For instance, for the plasma protein binding rate (PPBR) dataset, one can observe that all methods are suffering from large RMSE values early on. Specific to this case, there is an extreme imbalanced distribution for the target value of the source, as illustrated in Figure 8. Using a small number of compounds, the model gets a good insight of the most representative range, with a small peak following within the 300-400 samples, indicating a lack of training in underrepresented regions in early stages. In contrast to PPBR, hydration free energy (HFE) and effective cell permeability (Caco-2) the target distributions suffer less from skewness and spreadiness, as shown in Figure 8. For HFE and Caco-2 datasets COVDROP is the clear winner reaching RMSE within 10% of the final RMSE after testing only 36 and 450 compounds, respectively.

We also tested much larger datasets. For example, the aqueous solubility dataset (AS) (Figure 1 a) is significantly larger than those presented in Figure 2. There we see much slower convergence of RMSE values which we attribute to the normal distribution of the target values (Figure 8) and larger size of the training data. This RMSE profile indicates that for the Solubility dataset, the BAIT method exhibits inferior performance compared to the other methods, while COVDROP demonstrates the smallest root mean square error (RMSE) on the full dataset compared to all other methods. Additionally, COVDROP outperforms all other methods starting at 400 compounds.

Performance on affinity data

We next evaluated the methods on several affinity datasets from ChEMBL and Sanofi-Aventis with different protein targets. For each of these, a diverse set of ligands is screened for their affinity to a target protein, e.g. MMP-8. Figure 2c depicts the retrospective experiment for modeling the affinity to Glycogen synthase kinase-3β(GSK3β), and we observe a similar trend to the solubility experiments. COVDROP outperforms very early the other methods.

Figure 2d presents to the retrospective experiment predicting the activity of small molecules against Matrix-metalloprotease(ChEMBL) with COVDROP gain the best option.

The internal datasets provide the opportunity to compare batch selection methods with the actual order used to experimentally test the compounds. We observe that all batch selection methods (though not random) outperform the current human based ordering (Figure 2 e-f and 1 h-i). For instance, for Renin and MMP-8, COVDROP significantly outperforms the chronological batch selection by requiring 62 and 58 % less training data points.

Similar to the ChEMBL data, we observed that for most of the molecules tested within the Sanofi-Aventis datasets COVDROP significantly outperforms other methods. However, for this dataset we also observed very good performance of COVLAP on the FXa dataset (Figure 2f). Furthermore, the number of experiments to achieve the 10% higher than minimum RMSE threshold is 750, while other selection methods require at least 12% more experiments to obtain the same metric (Tabel 1). Figure 2e showcases the retrospective experiment on the Renin dataset. Similar to our previous observations, we observed that COVDROP significantly outperformed the other methods. Notably, the method exhibited a small root-mean-square error (RMSE) after the initial batch selection in comparison to the final RMSE attained, which was achieved by the other methods after over 200 experiments and multiple iterations.

To further quantify the improvement we present in Table 1 the number of experiments required by each method for reaching model with an error at most 10% higher than the minimum RMSE obtained using all the data across all selection methods. As the table shows, both of our methods outperform all other methods for most datasets, in some cases significantly so. For example, we observed that for smaller datasets COVDROP can improve the results very quickly leading to much better performance. For most of the datasets performance improvements are greater than 10% vs. Random selection. As mentioned before for PPBR dataset due to the imbalance nature of the data COVDROP underperforms and BAIT selection produces the beset result. Even when compared to the chronological order in which the internal compounds were tested (i.e. to the actual experimental cycle) we observe improvements of up to 62%. This holds true if we change the stopping criteria to 20% or 5% difference (Table 4 & 5).

Methods

Datasets

To benchmark the various batch selection methods, we have collected both private (Sanofi-owned) and public datasets that represent a diverse range of some of the most important molecular properties that scientists need to address when developing small-molecule drugs. We list below the benchmark datasets used in this work covering the properties related to the absorption and distribution pharmacokinetic processes. In addition to the ADMET related properties, the benchmark datasets include four Sanofi and six public datasets recording the affinity of small drug molecules to ten target proteins, such as kinase and GCPRs. Table 1 provides detailed information on all of these datasets.

Affinity datasets

Affinity measures the strength of binding between the ligands and biological targets, such as proteins. It is a critical molecular property that determines the drug efficacy. Therefore, to validate our active learning strategy for building statistical models, several datasets from the public database ChEMBL Gaulton et al. (2012) and internal sources were used.

ChEMBL datasets

To retrieve suitable datasets from ChEMBL (version 31, https://www.ebi.ac.uk/chembl/), a collection of six proteins representing multiple target families (Table 1, column: Class) was identified using Uniprot IDs and ChEMBL target IDs. These targets include the alpha-1a adrenergic and dopamine D2 receptors as members of the GPCR family, glycogen synthase kinase-3 beta (GSK3β) from the kinase family, Matrix-metalloprotease 3, also known as MMP3 or stromelysin as metalloprotease, the sodium channel Nav1.7 as ion channel and the peroxisome proliferator-activated receptor delta (PPARδ) as member of the nuclear hormone receptor (NHR) family. See Supporting Methods for details on how these datasets were derived.

Sanofi datasets

Four internal sets with structure-activity relationship (SAR) data were used. These allow us to overcome limitations of public SAR datasets which may merge assay data from different sources and which do not provide specific information on the order in which experiments were carried out. Such information allows direct comparison of the active learning solution and current best practices. The first dataset comprises compounds for inhibiting serine protease factor Xa (FXa)Nazaré et al. (2004); Nazare et al. (2012); Matter et al. (2002); Nazaré et al. (2005); Matter et al. (2005). The second dataset is for inhibiting the aspartyl protease ReninScheiper et al. (2010); Matter et al. (2011); Scheiper et al. (2011) The third dataset is for the Matrix-metalloprotease 8, MMP-8, (human neutrophil collagenase) and the final dataset provides is for agonists for the nuclear hormone receptor PPARδ (peroxisome proliferator-activated receptor δ).

Each data point in the datasets comprises the chemical structure of the molecule, represented as a Simplified Molecular Input Line Entry System (SMILES) Weininger (1988), and the target value, which denotes the molecular property as a scalar. Table 1 provides details on the target and size of the each of these datasets.

Active learning for small molecule optimization

We use active learning to optimize experiment selection. We assume a setup whereby a user can select from among a pool of unlabelled samples to query for their labels, i.e., to test experimentally. Furthermore, batches of experiments are performed in rounds.

In a given round, we denote the set of possible samples to select for labelling as 𝒱, and the batch size as B. The ideal active learner will select a subset, ℬ ⊂ 𝒱, |ℬ | = B, of these experiments such that, after the experiments are performed and the labelled samples are added to the training data, upon averaging it is expected that the overall loss of the resulting models is minimized.

In the original active learning frameworks, only one sample is queried at a time Lewis and Gale (1994) Dagan and Engelson (1995). In this case, a straightforward and effective strategy is to query the sample on which the current model’s prediction has the highest epistemic uncertainty. While this works well for single queries, it is hard to use the same approach for batch active learning. In many cases, the most uncertain queries will be similar to each other, and so their uncertainties are not independent, and so choosing as a batch the B most uncertain samples will be redundant, and not the most effective use of the queries Azimi et al. (2012).

In this paper we tested several different methods for selecting batches, including two new methods we developed. These two new methods are similar to “select the least certain samples” in spirit, selecting batches which maximize the total covariance of their uncertainty (estimated in different ways).

Problem Formulation: Batch Active Learning for Deep Learning Regression Models

We consider a batch active learning scenario with multiple rounds of selection, {1, 2, …, T }. At the t-th round, let ℒ^(t) be the labeled dataset and 𝒱 ^(t) be the unlabeled dataset. To be clear, 𝒱 ^(t) and ℒ^(t) are determined only after the selection method has had a chance to examine 𝒱 ^(t−1) and ℒ ^(t−1) .

Since in our analysis we usually consider the selection problem in a single round, we omit superscripts ^(t)where it won’t cause confusion.

Let f_θ ∶ 𝒳 → ℝ be the trained model, determined by its parameter θ ∈ Θ. As usual for regression problems, θ is chosen to approximately solve the following optimization problem,

where is the L²-regularization term for the weights θ. The optimization problem is (approximately) solved with a variation of SGD (in our case the popular ADAMKingma and Ba (2014)), by iterating over multiple mini batches from the training set ℒ.

In Active Learning, a selection method S is a function

That is, B samples are selected from the unlabeled dataset 𝒱 by the active learning algorithm, and the selection may use the information of the given sets 𝒱 and ℒ, as well as the current supervised model f_θ, which is trained using the labeled dataset ℒ.

Typically, selection is done by (approximately) solving an optimization problem,

where the acquisition function, a, depends on the selection method. In a single selection round, ℒ, 𝒱 and θ are given to the selector, and from the selector’s point of view are fixed; therefore, in this case a depends only on the choice ℬ.

Predictive uncertainty

Estimating predictive uncertainty is essential to our batch selection methods. Predictive uncertainty can be broadly divided into: epistemic uncertainty and aleatoric uncertainty, which may be roughly understood as model uncertainty and noise, respectively.

Aleatoric uncertainty is predictive uncertainty due to the inability of the model to give a precise prediction, even when the model parameters are uniquely specified. If θ are the model parameters, then we denote this distribution p_Al (y|x, θ).

Epistemic uncertainty is predictive uncertainty due to uncertainty in the model’s parameters. This is precisely the uncertainty which could be reduced by more labelled data, and therefore is the relevant quantity for active learning. In the strict Bayesian setting, if f_θ ∶ 𝒳 → ℝ is the model corresponding to parameters θ, and p(θ) is the posterior distribution given the training data 𝒟, this distribution is

Calculating this distribution exactly is intractable, and so we use approximations, described in Section 16.

Batch selection via determinant maximization

In the batch setting, selecting only those points with the epistemic uncertainty may lead to a batch in which uncertainties are highly correlated, and thus wasted experiments. I.e., if a sample x has the highest estimated variance, then small perturbations of x will have similarly high variance, and a variance-only selection method will give a high score to this batch of similar (and thus correlated) samples.

We thus seek to select the batch with the highest total independent uncertainty. This can be computed by selecting the batch with the highest joint entropy, which, under the assumption of a normal distribution, is the highest log-determinant of the (epistemic) covariance.

Even if we already know the epistemic covariance for any batch (a problem we address in the next section), finding the strict maximum of our acquisition function is NP-hard Ohsaka (2022). Therefore we use an approximate maximization technique: We randomly generate a collection of batches as starting points, each containing N distinct samples independently chosen from a distribution proportional to the quantile of the variances. Then we select the best M < N of these batches as starting points for optimization. Then, for each starting point, we optimize the batch element-wise, i.e., changing the first element to optimize the covariance, then changing the second, and so on, doing several passes until the process reaches equilibrium. Then, we select the highestscoring final batch.

The log-determinant is computed using the Cholesky decomposition, which is a bit better than 𝒪(N³). However, the optimization step (substituting a new point into a batch one at a time) is just a rank-1 update to the batch’s covariance, and there are O(N²) rank-1 updates to the Cholesky decomposition, which we use.

Algorithm

Batch optimization

This leaves open the question of how to approximate predictive posterior of the model, and hence its covariance matrix over the sample pool. We address this in the following sections.

Approximation of the posterior distribution

A straightforward way to get a distribution of predictions is to train an ensemble of models Lakshminarayanan et al. (2017), and to use the outputs of these models to give an ensemble of predictions. However, this approach involves multiple rounds of retraining, which is resource intensive. Furthermore, this ensemble of models will not sample from the Bayesian posterior, and there is no guarantee that it will be diverse.

Instead, we take a more economical approach by leveraging only one trained model. The idea behind this method is that the optimal parameters of a trained deep regression model, i.e. θ^∗, are the maximum a posteriori (MAP) estimation of an equivalent Bayesian deep regression problem.

Thus, the Bayesian inference of the posterior of θ is approximated by leveraging the computed MAP estimation of θ^∗. As it is shown in Eq. (1), the optimal θ^∗ could be treated as the MAP estimation of the probabilistic model of {Y, θ} where Y = {y}_y∈ℒ.

Two approximations for computing the epistemic covariance

We adapted two different methods to approximate the epistemic covariance:

Monte Carlo Dropout

Monte Carlo dropout is a well-known technique for training neural nets. The usual approach is to turn off dropout during inference. However, if random dropout is applied during inference, then the model predictions will follow a probability distribution. It has been observed Gal and Ghahramani (2016) that this distribution is an approximation to the true Bayesian posterior. Therefore, we may sample an ensemble of predictions for a single sample, and use this distribution as a measure of the uncertainty of the prediction.

Dropout Hinton et al. (2012) is a well-known technique for better training of neural networks. With dropout, in the training stage, each of the neural network’s weights are randomly set to 0 with a certain probability (as defined by the drop out ratio, r). Previous work has demonstrated that models built with dropout are less prone to overfitting and that training with dropout is quite similar to variational inference of the probabilistic models Gal and Ghahramani (2016); Lawrence (2001) when assuming the following form of the posterior approximation q(θ),

where r is the dropout ratio, M, the masks, are the {0, 1} variables of same shape as θ, and θ^∗ are the weights obtained after training with dropout. We thus use dropout to obtain S predictions by sampling S masks {M_s}_{s∈1,2,⋯,S} according to Eq. (8). We then use the new network to predict sample labels and construct the covariance matrix based on these predictions. Monte Carlo dropout can be defined as follows:

where ŷ is the predicted output of the model, S is the number of forward passes with dropout, f_θ is the model with parameters θ, x is the input to the model, and m_s is a binary mask that determines which units are dropped out during the s-th forward pass. In this equation, the model is run with dropout S times, and the predicted output is computed as the average of the outputs of each of these forward passes. This average is used as an estimate of the model’s predicted output and can be used to approximate the model’s uncertainty. Monte Carlo Dropout approach aligns with the principles of Bayesian Neural Networks but without the associated computational burden. Although this requires multiple forward passes during inference, the trade-off is beneficial, especially when understanding the uncertainty is paramount.

Laplace Approximation

In addition to dropout, we also employed the Laplace approximation for estimating the posterior. This method assumes that the posterior of the model parameters is a multi-variate normal distribution centered at the MAP value, and with variance equal to the Fisher information, which can be found through a straightforward calculation. This simplifying assumption makes the approximation fast to use, as the computation relies only on MAP estimation and differentiation of the loss function.

We assume the following for the posterior probability distribution, q(θ):

By matching 0^th and 2^nd derivatives, we find Λ, and thus our covariance matrix Λ^∗−1. The covariance matrix Λ^∗−1 does not have to be represented as a full matrix of shape Q × Q, which is usually very large for neural networks. For example, Laplace Redux Daxberger et al. (2021) further approximate the multi-variate normal distribution in Eq. (10) significantly reducing the computation cost. See Supporting Methods for additional details.

Comparisons to other methods

To compare our proposed approach with prior approaches, for batch active learning we looked at three previous methods suggested for this task: Random selection Settles (2009), optimizing uncertainty and diversity based on information theory (the BAIT method), and an unsupervised method that is optimizing only for diversity Nguyen and Smeulders (2004) (a method we term k-means sampling). See Supporting Methods for details on these prior methods and how we used them here.

Evaluation Experiments

For the retrospective experiment an existing labelled data set is selected to simulate an active learning experiment.

Thus we indirectly validated our posterior approximation by measuring the error with the respective method. The results from these retrospective experiments provided evidence for the accuracy and reliability of our approximation in representing the true posterior distribution.

We start by selecting a random subset (which is used as the initial set for all comparisons as well).

For our data the chemical structures in X are represented as molecular graphs using the MolGraphConvFeaturizer from the DeepChem library Kearnes et al. (2016). For each active learning cycle, models are trained using DeepChem dee (2016), a library which provides the implementation of the Graph Convolutional Neural Networks (GCNN) for molecular systems Kipf and Welling (2016).

For accuracy as well as model performance we use the root mean squared error (RMSE). See Supporting Methods for details on how the RMSE is computed.

Neural network architecture

For all our datasets we use the “neural fingerprints” class of models described in Duvenaud et al. (2015), as implemented in the DeepChem library GraphConvModel contributors (2022). See Supporting Methods for complete details. Although our method is compatible with, PyTorch, TensorFlow, and Keras frameworks, the benchmarks presented here are performed using the Keras framework within Deepchem suite. In order to enforce deterministic behaviour of the models for each selection methods and active learning rounds, during the training the model weights were manually loaded at the beginning of each active learning loop. We perform multiple runs of the retrospective experiments for each methods.

Code and Data availability

Code and all data sets used in this study are available as a python package at Sanofi-Public GitHub page. The package is called ALIEN (Active Learning in data Exploration) and can be downloaded: https://github.com/Sanofi-Public/Alien

References

2016Democratizing Deep-Learning for Drug Discovery, Quantum Chemistry, Materials Science and Biology. GitHub; 2016. https://github.com/deepchem/deepchem.
1. Ash J
2. Goel S
3. Krishnamurthy A
4. Kakade S.
5. Ranzato M
6. Beygelzimer A
7. Dauphin Y
8. Liang PS
9. Vaughan JW
2021Gone Fishing: Neural Active Learning with Fisher EmbeddingsAdvances in Neural Information Processing Systems Curran Associates, Inc
1. Azimi J
2. Fern A
3. Zhang-Fern X
4. Borradaile G
5. Heeringa B.
2012Batch active learning via coordinated matchingarXiv preprint arXiv:12066458
1. Cai C
2. Wang S
3. Xu Y
4. Zhang W
5. Tang K
6. Ouyang Q
7. Lai L
8. Pei J.
2020Transfer Learning for Drug DiscoveryJournal of Medicinal Chemistry 63:8683–8694https://doi.org/10.1021/acs.jmedchem.9b02147
1. Chuang KV
2. Gunsalus LM
3. Keiser MJ
2020Learning Molecular Representations for Medicinal ChemistryJournal of Medicinal Chemistry 63:8705–8722https://doi.org/10.1021/acs.jmedchem.0c00385
1. Cohn D
2. Atlas L
3. Ladner R.
1994Improving generalization with active learningMachine learning 15:201–221
1. Cohn DA
2. Ghahramani Z
3. Jordan MI
1996Active learning with statistical modelsJournal of artificial intelligence research 4:129–145
1. Cohn DA
2. Ghahramani Z
3. Jordan MI
1996Active learning with statistical modelsJournal of artificial intelligence research 4:129–145
1. contributors D, DeepChem Documentation - Keras Models - GraphConvModel
2022contributors D, DeepChem Documentation - Keras Models - GraphConvModel; 2022. https://deepchem.readthedocs.io/en/latest/api_reference/models.html#graphconvmodel, [Online; accessed 27-February-2023].
1. Dagan I
2. Engelson SP
3. Prieditis A
4. Russell S
1995Committee-Based Sampling For Training Probabilistic ClassifiersMachine Learning Proceedings 1995 San Francisco (CA: Morgan Kaufmann :150–157https://doi.org/10.1016/B978-1-55860-377-6.50027-X
1. Daxberger E
2. Kristiadi A
3. Immer A
4. Eschenhagen R
5. Bauer M
6. Hennig P.
2021Laplace Redux–Effortless Bayesian Deep LearningIn: NeurIPS
1. Ding X
2. Cui R
3. Yu J
4. Liu T
5. Zhu T
6. Wang D
7. Chang J
8. Fan Z
9. Liu X
10. Chen K
11. Jiang H
12. Li X
13. Luo X
14. Zheng M.
2021Active Learning for Drug Design: A Case Study on the Plasma Exposure of Orally Administered DrugsJournal of Medicinal Chemistry 64:16838–16853https://doi.org/10.1021/acs.jmedchem.1c01683
1. Duvenaud DK
2. Maclaurin D
3. Iparraguirre J
4. Bombarell R
5. Hirzel T
6. Aspuru-Guzik A
7. Adams RP
8. Cortes C
9. Lawrence N
10. Lee D
11. Sugiyama M
12. Garnett R
2015Convolutional Networks on Graphs for Learning Molecular FingerprintsAdvances in Neural Information Processing Systems Curran Associates, Inc
1. Gal Y
2. Ghahramani Z.
3. of Proceedings of Machine Learning Research New York
4. Balcan MF
5. Weinberger KQ
2016Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep LearningProceedings of The 33rd International Conference on Machine Learning New York, USA: PMLR :1050–1059
1. Gaulton A
2. Bellis LJ
3. Bento AP
4. Chambers J
5. Davies M
6. Hersey A
7. Light Y
8. McGlinchey S
9. Michalovich D
10. Al-Lazikani B
11. et al.
2012ChEMBL: a large-scale bioactivity database for drug discoveryNucleic acids research 40:D1100–D1107
1. Grebner C
2. Matter H
3. Hessler G.
4. Heifetz A
2022Artificial Intelligence in Compound DesignNew York, NY: Springer US :349–382https://doi.org/10.1007/978-1-0716-1787-8_15
1. Haghighatlari M
2. Vishwakarma G
3. Altarawy D
4. Subramanian R
5. Kota BU
6. Sonpal A
7. Setlur S
8. Hachmann J.
2019ChemML: A Machine Learning and Informatics Program Package for the Analysis, Mining, and Modeling of Chemical and Materials DataChemRxiv https://doi.org/10.26434/chemrxiv.8323271.v1
1. Hessler G
2. Baringhaus KH
2018Artificial Intelligence in Drug DesignMolecules 23https://doi.org/10.3390/molecules23102520
1. Hinton GE
2. Srivastava N
3. Krizhevsky A
4. Sutskever I
5. Salakhutdinov RR
2012Improving neural networks by preventing co-adaptation of feature detectorsarXiv preprint
1. Huang K
2. Fu T
3. Gao W
4. Zhao Y
5. Roohani Y
6. Leskovec J
7. Coley CW
8. Xiao C
9. Sun J
10. Zitnik M.
2022Artificial intelligence foundation for therapeutic scienceNature Chemical Biology 18:1033–1036https://doi.org/10.1038/s41589-022-01131-2
1. Kearnes S
2. McCloskey K
3. Berndl M
4. Pande V
5. Riley P.
2016Molecular graph convolutions: moving beyond finger-printsJournal of Computer-Aided Molecular Design 30:595–608https://doi.org/10.1007/s10822-016-9938-8
1. Kendall A
2. Gal Y.
2017What uncertainties do we need in bayesian deep learning for computer vision?Advances in neural information processing systems 30
1. Kingma DP
2. Ba J.
2014Adam: A method for stochastic optimizationarXiv preprint
1. Kipf TN
2. Welling M.
2016Semi-Supervised Classification with Graph Convolutional Networks
1. Lakshminarayanan B
2. Pritzel A
3. Blundell C.
2017Simple and scalable predictive uncertainty estimation using deep ensemblesAdvances in neural information processing systems 30
1. Lawrence ND
2001Variational Inference in Probabilistic ModelsUniversity of Cambridge
1. Lewis DD
2. Gale WA
3. Croft BW
4. van Rijsbergen CJ
1994A Sequential Algorithm for Training Text ClassifiersSIGIR ‘94 London: Springer London :3–12
1. Matter H
2. Defossa E
3. Heinelt U
4. Blohm PM
5. Schneider D
6. Müller A
7. Herok S
8. Schreuder H
9. Liesum A
10. Brachvogel V
11. Lönze P
12. Walser A
13. Al-Obeidi F
14. Wildgoose P.
2002Design and Quantitative Structure-Activity Relationship of 3-Amidinobenzyl-1H-indole-2-carboxamides as Potent, Nonchiral, and Selective Inhibitors of Blood Coagulation Factor XaJ Med Chem 45:2749–2769
1. Matter H
2. Scheiper B
3. Steinhagen H
4. Böcskei Z
5. Fleury V
6. McCort G.
2011Structure-based design and optimization of potent renin inhibitors on 5-or 7-azaindole-scaffoldsBioorganic & Medicinal Chemistry Letters 21:5487–5492https://doi.org/10.1016/j.bmcl.2011.06.112
1. Matter H
2. Will DW
3. Nazaré M
4. Schreuder H
5. Laux V
6. Wehner V.
2005Structural Requirements for Factor Xa Inhibition by 3-Oxybenzamides with Neutral P1 Substituents: Combining X-ray Crystallography, 3D-QSAR, and Tailored Scoring FunctionsJournal of Medicinal Chemistry 48:3290–3312https://doi.org/10.1021/jm049187l
1. Mehrjou A
2. Soleymani A
3. Jesson A
4. Notin P
5. Gal Y
6. Bauer S
7. Schwab P.
2021GeneDisco: A Benchmark for Experimental Design in Drug Discovery
1. Naik AW
2. Kangas JD
3. Sullivan DP
4. Murphy RF
2016Active machine learning-driven experimentation to determine compound effects on protein patternseLife 5https://doi.org/10.7554/eLife.10047
1. Nazaré M
2. Essrich M
3. Will DW
4. Matter H
5. Ritter K
6. Urmann M
7. Bauer A
8. Schreuder H
9. Dudda A
10. Czech J
11. et al.
2004Factor Xa inhibitors based on a 2-carboxyindole scaffold: SAR of neutral P1 substituentsBioorganic & medicinal chemistry letters 14:4191–4195
1. Nazare M
2. Matter H
3. Will DW
4. Wagner M
5. Urmann M
6. Czech J
7. Schreuder H
8. Bauer A
9. Ritter K
10. Wehner V.
2012Fragment Deconstruction of Small, Potent Factor Xa Inhibitors: Exploring the Superadditivity Energetics of Fragment Linking in Protein-Ligand ComplexesAngewandte Chemie International Edition 51:905–911https://doi.org/10.1002/anie.201107091
1. Nazaré M
2. Will DW
3. Matter H
4. Schreuder H
5. Ritter K
6. Urmann M
7. Essrich M
8. Bauer A
9. Wagner M
10. Czech J
11. Lorenz M
12. Laux V
13. Wehner V.
2005Probing the Subpockets of Factor Xa Reveals Two Binding Modes for Inhibitors Based on a 2-Carboxyindole Scaffold: A Study Combining Structure-Activity Relationship and X-ray CrystallographyJournal of Medicinal Chemistry 48:4511–4525https://doi.org/10.1021/jm0490540
1. Nguyen HT
2. Smeulders A.
2004Active Learning Using Pre-ClusteringIn: NeurIPS ICML ༴ New York, NY, USA: Association for Computing Machinery https://doi.org/10.1145/1015330.1015349
1. Ohsaka N
2022On the Parameterized Intractability of Determinant Maximizationhttps://doi.org/10.48550/ARXIV.2209.12519
1. Pertusi DA
2. Moura ME
3. Jeffryes JG
4. Prabhu S
5. Walters Biggs B
6. Tyo KEJ
2017Predicting novel substrates for enzymes with minimal experimental effort with active learningMetabolic Engineering 44:171–181https://doi.org/10.1016/j.ymben.2017.09.016
1. Reker D
2. Schneider P
3. Schneider G
4. Brown J.
2017Active learning for computational chemogenomicsFuture Medicinal Chemistry 9:381–402https://doi.org/10.4155/fmc-2016-0197
1. Scheiper B
2. Matter H
3. Steinhagen H
4. Böcskei Z
5. Fleury V
6. McCort G.
2011Structure-based optimization of potent 4- and 6-azaindole-3-carboxamides as renin inhibitorsBioorganic & Medicinal Chemistry Letters 21:5480–5486https://doi.org/10.1016/j.bmcl.2011.06.114
1. Scheiper B
2. Matter H
3. Steinhagen H
4. Stilz U
5. Böcskei Z
6. Fleury V
7. McCort G.
2010Discovery and optimization of a new class of potent and non-chiral indole-3-carboxamide-based renin inhibitorsBioorganic & Medicinal Chemistry Letters 20:6268–6272https://doi.org/10.1016/j.bmcl.2010.08.092
1. Settles B.
2009Active Learning Literature SurveyUniversity of Wisconsin–Madison
1. Settles B.
2012Active Learning. Synthesis Lectures on Artificial Intelligence and Machine LearningMorgan & Claypool Publishers
1. Sorkun MC
2. Khetan A
3. Er S.
2019AqSolDB, a curated reference set of aqueous solubility and 2D descriptors for a diverse set of compoundsScientific Data 6https://doi.org/10.1038/s41597-019-0151-1
1. Thompson J
2. Walters WP
3. Feng JA
4. Pabon NA
5. Xu H
6. Goldman BB
7. Moustakas D
8. Schmidt M
9. York F.
2022Optimizing active learning for free energy calculationsArtificial Intelligence in the Life Sciences 2https://doi.org/10.1016/j.ailsci.2022.100050
1. Tynes M
2. Gao W
3. Burrill DJ
4. Batista ER
5. Perez D
6. Yang P
7. Lubbers N.
2021Pairwise Difference Regression: A Machine Learning Meta-algorithm for Improved Prediction and Uncertainty Quantification in Chemical SearchJournal of Chemical Information and Modeling 61:3846–3857https://doi.org/10.1021/acs.jcim.1c00670
1. Wang NN
2. Dong J
3. Deng YH
4. Zhu MF
5. Wen M
6. Yao ZJ
7. Lu AP
8. Wang JB
9. Cao DS
2016ADME Properties Evaluation in Drug Discovery: Prediction of Caco-2 Cell Permeability Using a Combination of NSGA-II and BoostingJournal of Chemical Information and Modeling 56:763–773https://doi.org/10.1021/acs.jcim.5b00642
1. Weininger D.
1988SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rulesJournal of Chemical Information and Computer Sciences 28:31–36https://doi.org/10.1021/ci00057a005
1. Weiss K
2. Khoshgoftaar TM
3. Wang D.
2016A survey of transfer learningJournal of Big Data 3https://doi.org/10.1186/s40537-016-0043-6
1. Wenlock M
2. Tomkinson N
2015Experimental in vitro DMPK and physicochemical data on a set of publicly disclosed compounds
1. Wu F
2. Zhou Y
3. Li L
4. Shen X
5. Chen G
6. Wang X
7. Liang X
8. Tan M
9. Huang Z.
2020Computational Approaches in Preclinical Studies on Drug Discovery and DevelopmentFrontiers in Chemistry 8https://doi.org/10.3389/fchem.2020.00726
1. Wu Z
2. Ramsundar B
3. Feinberg EN
4. Gomes J
5. Geniesse C
6. Pappu AS
7. Leswing K
8. Pande V.
2018MoleculeNet: a benchmark for molecular machine learningChemical science 9:513–530
1. Xiong G
2. Wu Z
3. Yi J
4. Fu L
5. Yang Z
6. Hsieh C
7. Yin M
8. Zeng X
9. Wu C
10. Lu A
11. Chen X
12. Hou T
13. Cao D.
2021ADMETlab 2.0: an integrated online platform for accurate and comprehensive predictions of ADMET propertiesNucleic Acids Research 49:W5–W14https://doi.org/10.1093/nar/gkab255
1. Zaverkin V
2. Holzmüller D
3. Steinwart I
4. Kästner J.
2022Exploring chemical and conformational spaces by batch mode deep active learningDigital Discovery 1:605–620https://doi.org/10.1039/D2DD00034B

Article and author information

Author information

Michael Bailey
R&D Data & Computational Science, Sanofi,, Cambridge, MA, United States
ORCID iD: 0009-0004-8425-993X
Saeed Moayedpour
R&D Data & Computational Science, Sanofi,, Cambridge, MA, United States
Ruijiang Li
Digital Data, Sanofi, Shanghai, China
Alejandro Corrochano-Navarro
R&D Data & Computational Science, Sanofi,, Cambridge, MA, United States
Alexander Kötter
Synthetic Molecular Design, Integrated Drug Discovery, Sanofi-Aventis Deutschland GmbH, Industriepark Höchst, Building G838, 65926 Frankfurt am Main, Germany
Lorenzo Kogler-Anele
R&D Data & Computational Science, Sanofi,, Cambridge, MA, United States
Saleh Riahi
R&D Data & Computational Science, Sanofi,, Cambridge, MA, United States
Christoph Grebner
Synthetic Molecular Design, Integrated Drug Discovery, Sanofi-Aventis Deutschland GmbH, Industriepark Höchst, Building G838, 65926 Frankfurt am Main, Germany
Gerhard Hessler
Synthetic Molecular Design, Integrated Drug Discovery, Sanofi-Aventis Deutschland GmbH, Industriepark Höchst, Building G838, 65926 Frankfurt am Main, Germany
Hans Matter
Synthetic Molecular Design, Integrated Drug Discovery, Sanofi-Aventis Deutschland GmbH, Industriepark Höchst, Building G838, 65926 Frankfurt am Main, Germany
Marc Bianciotto
Molecular Design Sciences, Integrated Drug Discovery, Sanofi, Vitry-sur-Seine, 94403, France
Pablo Mas
Molecular Design Sciences, Integrated Drug Discovery, Sanofi, Vitry-sur-Seine, 94403, France
Ziv Bar-Joseph
R&D Data & Computational Science, Sanofi,, Cambridge, MA, United States
ORCID iD: 0000-0003-3430-6051
- For correspondence:⠀Ziv.Bar-Joseph@sanofi.com; sven.jager@sanofi.com
Sven Jager
R&D Data & Computational Science, Sanofi,, Cambridge, MA, United States
ORCID iD: 0000-0003-1410-9114
- For correspondence:⠀Ziv.Bar-Joseph@sanofi.com; sven.jager@sanofi.com

Author Notes

M.B.,R.L.,H.M., G.H.,C.G., Z.B. and S.J. designed research; M.B., S.M., R.L., A.C.N.,S.J., H.M., P.M., L.K.A., and S.R. performed research; M.B., S.M., A.C.N.,S.R., C. G., G.H., H.M., P.M., and A.K. analyzed data; M.B., S.M., A.C.N., S.J., and Z.B. wrote the paper

Version history

Sent for peer review: June 16, 2023
Preprint posted: July 29, 2023
Reviewed Preprint version 1: September 13, 2023
Reviewed Preprint version 2: January 19, 2024

Copyright

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Revised: This Reviewed Preprint has been revised by the authors in response to the previous round of peer review; the eLife assessment and the public reviews have been updated where necessary by the editors and peer reviewers.

Reviewing Editor
Alan Talevi
National University of La Plata, La Plata, Argentina
Senior Editor
Volker Dötsch
Goethe University Frankfurt, Frankfurt am Main, Germany

Reviewer #1 (Public Review):

The authors present a study focused on addressing the key challenge in drug discovery, which is the optimization of absorption and affinity properties of small molecules through in silico methods. They propose active learning as a strategy for optimizing these properties and describe the development of two novel active learning batch selection methods. The methods are tested on various public datasets with different optimization goals and sizes, and new affinity datasets are curated to provide up-to-date experimental information. The authors claim that their active learning methods outperform existing batch selection methods, potentially reducing the number of experiments required to achieve the same model performance. They also emphasize the general applicability of their methods, including compatibility with popular packages like DeepChem.

Strengths:

Relevance and Importance: The study addresses a significant challenge in the field of drug discovery, highlighting the importance of optimizing absorption and affinity properties of small molecules through in silico methods. This topic is of great interest to researchers and pharmaceutical industries.

Novelty: The development of two novel active learning batch selection methods is a commendable contribution. The study also adds value by curating new affinity datasets that provide chronological information on state-of-the-art experimental strategies.
Comprehensive Evaluation: Testing the proposed methods on multiple public datasets with varying optimization goals and sizes enhances the credibility and generalizability of the findings. The focus on comparing the performance of the new methods against existing batch selection methods further strengthens the evaluation.

Weaknesses:

Lack of Technical Details: The feedback lacks specific technical details regarding the developed active learning batch selection methods. Information such as the underlying algorithms, implementation specifics, and key design choices should be provided to enable readers to understand and evaluate the methods thoroughly.

Evaluation Metrics: The feedback does not mention the specific evaluation metrics used to assess the performance of the proposed methods. The authors should clarify the criteria employed to compare their methods against existing batch selection methods and demonstrate the statistical significance of the observed improvements.

Reproducibility: While the authors claim that their methods can be used with any package, including DeepChem, no mention is made of providing the necessary code or resources to reproduce the experiments. Including code repositories or detailed instructions would enhance the reproducibility and practical utility of the study.

Suggestions for Improvement:

Elaborate on the Methodology: Provide an in-depth explanation of the two active learning batch selection methods, including algorithmic details, implementation considerations, and any specific assumptions made. This will enable readers to better comprehend and evaluate the proposed techniques.

Clarify Evaluation Metrics: Clearly specify the evaluation metrics employed in the study to measure the performance of the active learning methods. Additionally, conduct statistical tests to establish the significance of the improvements observed over existing batch selection methods.

Enhance Reproducibility: To facilitate the reproducibility of the study, consider sharing the code, data, and resources necessary for readers to replicate the experiments. This will allow researchers in the field to validate and build upon your work more effectively.

Conclusion:
The authors' study on active learning methods for optimizing drug discovery presents an important and relevant contribution to the field. The proposed batch selection methods and curated affinity datasets hold promise for improving the efficiency of drug discovery processes. However, to strengthen the study, it is crucial to provide more technical details, clarify evaluation metrics, and enhance reproducibility by sharing code and resources. Addressing these limitations will further enhance the value and impact of the research.

https://doi.org/10.7554/eLife.89679.2.sa1

Reviewer #2 (Public Review):

The authors presented a well-written manuscript describing the comparison of active-learning methods with state-of-art methods for several datasets of pharmaceutical interest. This is a very important topic since active learning is similar to a cyclic drug design campaign such as testing compounds followed by designing new ones which could be used to further tests and a new design cycle and so on. The experimental design is comprehensive and adequate for proposed comparisons.

Text in figures still very small and difficult to read. Please redraw the figures increasing the font size: 10-12pt is ideal in comparison with the main text. In my opinion, it seems like the authors drew the Figure properly but there is an automatic resizing by inserting it in the document. Please consider ensuring that the font size will remain legible in the final document.
I think this work will benefit from a comparison of obtained models to other models reported in the literature and the interpretability of models (e.g. contribution of molecule groups to the modeled activity) as it is required by OECD guide for QSAR purposes.

https://doi.org/10.7554/eLife.89679.2.sa0

Author Response

The following is the authors’ response to the original reviews.

Reviewer No.1 (public)

The authors present a study focused on addressing the key challenge in drug discovery, which is the optimization of absorption and affinity properties of small molecules through in silico methods. They propose active learning as a strategy for optimizing these properties and describe the development of two novel active learning batch selection methods. The methods are tested on various public datasets with different optimization goals and sizes, and new affinity datasets are curated to provide up-todate experimental information. The authors claim that their active learning methods outperform existing batch selection methods, potentially reducing the number of experiments required to achieve the same model performance. They also emphasize the general applicability of their methods, including compatibility with popular packages like DeepChem.

Strengths:

Relevance and Importance: The study addresses a significant challenge in the field of drug discovery, highlighting the importance of optimizing the absorption and affinity properties of small molecules through in silico methods. This topic is of great interest to researchers and pharmaceutical industries.

Novelty: The development of two novel active learning batch selection methods is a commendable contribution. The study also adds value by curating new affinity datasets that provide chronological information on state-of-the-art experimental strategies.

Comprehensive Evaluation: Testing the proposed methods on multiple public datasets with varying optimization goals and sizes enhances the credibility and generalizability of the findings. The focus on comparing the performance of the new methods against existing batch selection methods further strengthens the evaluation.

Weaknesses:

Lack of Technical Details: The feedback lacks specific technical details regarding the developed active learning batch selection methods. Information such as the underlying algorithms, implementation specifics, and key design choices should be provided to enable readers to understand and evaluate the methods thoroughly.

Evaluation Metrics: The feedback does not mention the specific evaluation metrics used to assess the performance of the proposed methods. The authors should clarify the criteria employed to compare their methods against existing batch selection methods and demonstrate the statistical significance of the observed improvements.

Reproducibility: While the authors claim that their methods can be used with any package, including DeepChem, no mention is made of providing the necessary code or resources to reproduce the experiments. Including code repositories or detailed instructions would enhance the reproducibility and practical utility of the study.

Suggestion 1:

Elaborate on the Methodology: Provide an in-depth explanation of the two active learning batch selection methods, including algorithmic details, implementation considerations, and any specific assumptions made. This will enable readers to better comprehend and evaluate the proposed techniques.

Answer: We thank the reviewer for this suggestion. Following this comments we have extended the text in Methods (in Section: Batch selection via determinant maximization and Section: Approximation of the posterior distribution) and in Supporting Methods (Section: Toy example). We have also included the pseudo code for the Batch optimization method.

Suggestion 2:

Clarify Evaluation Metrics: Clearly specify the evaluation metrics employed in the study to measure the performance of the active learning methods. Additionally, conduct statistical tests to establish the significance of the improvements observed over existing batch selection methods.

Answer: Following this comment we added to Table 1 details about the way we computed the cutoff times for the different methods. We also provide more details on the statistics we performed to determine the significance of these differences.

Suggestion 3:

Enhance Reproducibility: To facilitate the reproducibility of the study, consider sharing the code, data, and resources necessary for readers to replicate the experiments. This will allow researchers in the field to validate and build upon your work more effectively.

Answer: This is something we already included with the original submission. The code is publicly available. In fact, we provide a phyton library, ALIEN (Active Learning in data Exploration) which is published on the Sanofi Github(https://github.com/ Sanofi-Public/Alien). We also provide details on the public data used and expect to provide the internal data as well. We included a small paragraph on code and data availability.

Reviewer No.2 (public)

Suggestion 1:

The authors presented a well-written manuscript describing the comparison of activelearning methods with state-of-art methods for several datasets of pharmaceutical interest. This is a very important topic since active learning is similar to a cyclic drug design campaign such as testing compounds followed by designing new ones which could be used to further tests and a new design cycle and so on. The experimental design is comprehensive and adequate for proposed comparisons. However, I would expect to see a comparison regarding other regression metrics and considering the applicability domain of models which are two essential topics for the drug design modelers community.

Answer: We want to thank the reviewer for these comments. We provide a detailed response to the specific comments below.

Reviewer No.1 (Recommendations For The Authors)

Recommendation 1:

The description provided regarding the data collection process and the benchmark datasets used in the study raises some concerns. The comment specifically addresses the use of both private (Sanofi-owned) and public datasets to benchmark the various batch selection methods. Lack of Transparency: The comment lacks transparency regarding the specific sources and origins of the private datasets. It would be crucial to disclose whether these datasets were obtained from external sources or if they were generated internally within Sanofi. Without this information, it becomes difficult to assess the potential biases or conflicts of interest associated with the data.

Answer: We would like to thank the reviewer for this comment. As mentioned in the paper, the public github page contains links to all the public data and we expect also to the internal Sanofi data. We also now provide more information on the specific experiments that were internally done by Sanofi to collect that data.

Potential Data Accessibility Issues: The utilization of private datasets, particularly those owned by Sanofi, may raise concerns about data accessibility. The lack of availability of these datasets to the wider scientific community may limit the ability of other researchers to replicate and validate the study’s findings. It is essential to ensure that the data used in research is openly accessible to foster transparency and encourage collaboration.

Answer: Again, as stated above we expect to release the data collected internally on the github page.

Limited Information on Dataset Properties: The comment briefly mentions that the benchmark datasets cover properties related to absorption, distribution, pharmacokinetic processes, and affinity of small drug molecules to target proteins. However, it does not provide any specific details about the properties included in the datasets or how they were curated. Providing more comprehensive information about the properties covered and the methods used for curation would enhance the transparency and reliability of the study.

To address these concerns, it is crucial for the authors to provide more detailed information about the data sources, dataset composition, representativeness, and curation methods employed. Transparency and accessibility of data are fundamental principles in scientific research, and addressing these issues will strengthen the credibility and impact of the study.

Answer: We agree with this comment and believe that it is important to be explicit about each of the datasets and to provide information on the new data. We note that we already discuss the details of each of the experiments in Methods and, of course, provide links to the original papers for the public data. We have now added text to Supporting Methods that describes the experiments in more details as well as providing literature references for the experimental protocols used. As noted above, we expect to provide our new internal data on the public git page.

Recommendation 2:

Some comments on the modeling example Approximation of the posterior distribution. Lack of Methodological Transparency: The comment fails to provide any information regarding the specific method or approach used for approximating the posterior distribution. Without understanding the methodology employed, it is impossible to evaluate the quality or rigor of the approximation. This lack of transparency undermines the credibility of the study.

Answer: We want to thank the reviewer for pointing this out. Based on this comment we added more information to Section: Approximation of the posterior distribution. Moreover, we now provide details on the posterior approximation in Section: Two approximations for computing the epistemic covariance.

Questionable Assumptions: The comment does not mention any of the assumptions made during the approximation process. The validity of any approximation heavily depends on the underlying assumptions, and their omission suggests a lack of thorough analysis. Failing to acknowledge these assumptions leaves room for doubt regarding the accuracy and relevance of the approximation.

Answer: We are not entirely sure which assumptions the reviewer is referring to here. The main assumption we can think of that we have used is the fact that getting within X% of the optimal model is a good enough approximation. We have specifically discussed this assumption and tested multiple values of X. While it would have been great to have X = 0 this is unrealistic for retrospective studies. For Active Learning the main question is how many experiments can be saved to obtain similar results and the assumptions we used are basically ’what is the definition of similar’. We now added this to Discussion.

Inadequate Validation: There is no mention of any validation measures or techniques used to assess the accuracy and reliability of the approximated posterior distribution. Without proper validation, it is impossible to determine whether the approximation provides a reasonable representation of the true posterior. The absence of validation raises concerns about the potential biases or errors introduced by the approximation process.

Answer: We sincerely appreciate your concern regarding the validation of the approximated posterior distribution. We acknowledge that our initial submission might not have clearly highlighted our validation strategy. It is, of course, very hard to determine the accuracy of the distribution our model learns since such distribution cannot be directly inferred using experiments (no ’ground truth’). Instead, we use an indirect method to determine the accuracy. Specifically, we conducted retrospective experiment using the learned distribution. In these experiments, we indirectly validated our approximation by measuring the error with the respective method. The results from these retrospective experiments provided evidence for the accuracy and reliability of our approximation in representing the true posterior distribution. We now emphasize this in Methods.

Uncertainty Quantification: The comment does not discuss the quantification of uncertainty associated with the approximated posterior distribution. Properly characterizing the uncertainty is crucial in statistical inference and decision-making. Neglecting this aspect undermines the usefulness and applicability of the approximation results.

Answer: Thank you for pointing out the importance of characterizing uncertainty in statistical inference and decision-making, a sentiment with which we wholeheartedly agree. In our work, we have indeed addressed the quantification of uncertainty associated with the approximated posterior distribution. Specifically, we utilized Monte Carlo Dropout (MC Dropout) as our method of choice. MC Dropout is a widely recognized and employed technique in the neural networks domain to approximate the posterior distribution, and it offers an efficient way to estimate model uncertainty without requiring any changes to the existing network architecture [1, 2]. In the revised version, we provide a more detailed discussion on the use of Monte Carlo Dropout in our methodology and its implications for characterizing uncertainty.

Comparison with Gold Standard: There is no mention of comparing the approximated posterior distribution with a gold standard or benchmark. Failing to provide such a comparison leaves doubts about the performance and accuracy of the approximation method. A lack of benchmarking makes it difficult to ascertain the superiority or inferiority of the approximation technique employed.

Answer: As noted above, it is impossible to find gold standard information for the uncertainly distribution. It is not even clear to us how such gold standard can be experimentally determined since its a function of a specific model and data. If the reviewer is aware of such gold standard we would be happy to test it. Instead, in our study, we opted to benchmark our results against state-of-the-art batch active learning methods, which also rely on uncertainty prediction (such uncertainty prediction is the heart of any active learning method as we discuss). Results clearly indicate that our method outperforms prior methods though we agree that this is only an indirect way to validate the uncertainty approximation.

Reviewer No.2 (Recommendations For The Authors)

Recommendation 1:

The text is kind of messy: there are two results sections, for example. It seems that part of the text was duplicated. Please correct it.

Answer: We want to thank the reviewer pointing this out. These were typos and we fixed them accordingly.

Recommendation 2:

Text in figures is very small and difficult to read. Please redraw the figures, increasing the font size: 10-12pt is ideal in comparison with the main text.

Answer: We want to thank the reviewer for this comment and we have made the graphics larger.

Recommendation 3: Please, include specific links to data availability instead of just stating it is available at the Sanofi-Public repository.

Answer: We want to thank the reviewer for this comment and added the links and data to the Sanofi Github page listed in the paper.

Recommendation 4:

What are the descriptors used to train the models?

Answer: We represented the molecules as molecular graphs using the MolGraphConvFeaturizer from the DeepChem library. We now explicitly mention this in Methods.

Recommendation 5:

Regarding the quality of the models, I strongly suggest two approaches instead of using only RMSE as metrics of models’ performance. I recommend using the most metrics as possible as reported by Gramatica (https://doi.org/10.1021/acs.jcim.6b00088). I also recommend somehow comparing the increment on the dataset diversity according to the employed descriptors (applicability domain) as a measurement to further applications on the unseen molecules.

Answer: We want to thank the reviewer for this great suggestions. As suggested we added new comparison metrics to the Supplement.

• Distribution plot for the range of the Y values Figure 8 • Clustering of the data sets represented as fingerprints Supplementary material Figure 5,6

• Retrospective experiments with Spearman correlation coefficient. Supplementary material Figure: 2,3,4

I suggest also a better characterization of datasets including the nature and range of the Y variable, the source of data in terms of experimentation, and chemical (structural and physicochemical) comparison of samples within each dataset.

Answer: As noted above in response to a similar comment by Reviewer 1, we have added more detailed information about the different experiments we tested to Supporting Methods.

References

[1] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1050–1059, New York, New York, USA, 20–22 Jun 2016. PMLR.

[2] N.D. Lawrence. Variational Inference in Probabilistic Models. University of Cambridge, 2001.

https://doi.org/10.7554/eLife.89679.2.sa3

Significance of findings

Strength of evidence

Abstract

Introduction

Results

Evaluating active learning method on ADMET and affinity related data

Comparison of active learning methods for solubility datasets

Performance on affinity data

Methods

Datasets

Affinity datasets

ChEMBL datasets

Sanofi datasets

Active learning for small molecule optimization

Problem Formulation: Batch Active Learning for Deep Learning Regression Models

Predictive uncertainty

Batch selection via determinant maximization

Algorithm

Approximation of the posterior distribution

Comparisons to other methods

Evaluation Experiments

Neural network architecture

Code and Data availability

References

Article and author information

Author information

Michael Bailey

Saeed Moayedpour

Ruijiang Li

Alejandro Corrochano-Navarro

Alexander Kötter

Lorenzo Kogler-Anele

Saleh Riahi

Christoph Grebner

Gerhard Hessler

Hans Matter

Marc Bianciotto

Pablo Mas

Ziv Bar-Joseph

Sven Jager

Author Notes

Version history

Copyright

Peer review process

Editors