Evolution imposes an inductive bias that alters and accelerates learning dynamics

Reviewer #1 (Public review):

In this article, the authors set out to understand how evolutionary selection could introduce structural priors into neural networks that act as an inductive bias to accelerate learning. To do so, the authors propose an evolutionary conditioning (ED) algorithm and analyse its properties. I find the conceptual framing of the paper very interesting, and it addresses an important question. Since the paper adopts a mostly theoretical approach with no comparison to empirical biological data, I do have a couple of concerns regarding the setup of the computational framework/method, which I think is incomplete and limits how much we can conclude from the current results.

Major Concerns:

(1) The evolutionary conditioning (EC) algorithm proposed works as fine-tuning training, plus propagation of the best parent network's weights to the next generation with added Gaussian noise. Conceptually, I find this to be a fairly implausible mechanism for evolution, since it requires carrying the entire set of network weights at some precision. The authors themselves point out that direct weight transfer could be problematic in the introduction.

(2) More importantly, I would like the authors to conduct a baseline / null model comparison, in which the "evolution" process consists simply of training a neural network for a small number of iterations, adding Gaussian noise, and repeating. The resulting network at each step serves as the "generations", which is then trained further. The same learning speed and dynamics analyses should be applied to this null model. What I am getting at is that I am not sure to what extent the EC algorithm can be thought of as "running a few iterations of SGD" and chaining them together; how much work is the selection process in the GA actually doing?

(3) I am also unclear on why the EC algorithm does not improve throughout learning. Is this behavior the result of applying only a small number of fine-tuning steps? Presumably, with longer fine-tuning, the individual networks in the middle generations would also improve in performance?

(4) The EC algorithm applied to a single problem seems somewhat artificial in its setup. I would conceptualize evolution as learning a prior that conditions the network for a range of survival-related tasks. A more realistic setup would apply EC to an ensemble of tasks and then examine its impact on learning a specific task afterwards.

Reviewer #2 (Public review):

Summary:

This paper studies the interplay of evolutionary and in-lifetime learning. The authors develop a neural network model in which initial weight configurations evolve under selective pressure, while fitness is determined by the network's performance after a learning period. They show that such a network displays very distinct learning dynamics from those trained by either gradient descent or genetic algorithms alone: in particular, they do not learn the task, but they show evidence of learning-to-learn and unusual representational structure.

Strengths:

(1) The writing, figures, and presentation of ideas were clear.

(2) The question of how evolution on initial weights combines with learning from within-lifetime experience to structure a learning trajectory seems interesting.

(3) The analysis of existing experiments was well-done, highlighting that though these networks did not really learn, they show latent learning structure that makes the network perform better from less data.

(4) The interplay between Baldwin & learning dynamics seemed novel and interesting, presenting many attractive puzzles.

Weaknesses:

First, the authors point to an important distinction between performing evolutionary selection on the weights pre- or post- lifetime training, the latter of which is Lamarckian. They argue, correctly, that their model is interesting because it selects on the weight initialisation, unlike, for example, Shuvaev et al. However, my understanding is that a long line of papers beginning perhaps with Hinton & Nowlan also do non-Lamarckian evolution: Hinton & Nowlan have unspecified weights (denoted '?' in the paper) that can be inherited and then learnt. Is this not exactly inheritance of initial conditions (in this case, whether learnable or not)? This novelty is a primary motivation of the paper, whereas to me it seems it was already apparent in Hinton & Nowlan, and developed further in what seems to be a long line of uncited literature (see next paragraph). As such, this paper's conclusions seem poorly positioned within the existing state of knowledge/literature.

Second, the algorithm is framed as novel, but I think it is a rediscovery. This framework is very close to MAML, in which an initial weight configuration is optimised by gradient descent to be good after a few steps of fine-tuning (Finn et al., 2017). The authors' approach differs in using a genetic algorithm to perform the training of the initial weights, avoiding some of the computational complexities of MAML, especially after long fine-tuning. In this, the authors have, I think, rediscovered ES-MAML, MAML where the inner optimisation loop is gradient descent, while the outer is genetic (Song et al., 2020). Other similar work is "Meta-Learning by the Baldwin Effect" (Fernando et al., 2018).

Further, within Fernando et al. there is a rich literature review, almost none of which are cited by the authors. I point especially to Keesing & Stork, 1990, which appears to show a strong dependence of Baldwin-like improvements on the amount of data, something this paper also shows but explores less thoroughly.

To summarise my critique thus far: I think the literature already answers the main concern of the motivation (i.e. non-Lamarckian neural network evolution and learning), I think it has already discovered this particular algorithm, and I think past work has more thoroughly analysed behaviours similar to those presented in this paper. Without positioning correctly within this literature, the more general contribution of the paper is hard to establish. The true novelty of the authors' analysis seems to be the emphasis on Saxe et al.-like learning dynamics and its interplay with the Baldwin effect, but I am not certain of this without knowing the literature better.

Regarding experiments, there was an interesting effect where the EC networks didn't learn but did show latent learning (Figure 2, Figure 3), which sped up later learning (Figure 4). There were a few details I was surprised by on which I would appreciate clarity:

(1) The main result has basically no headline learning under EC. This will clearly be very dependent on parameters (e.g. if you add or remove enough training steps, the algorithm becomes SGD/GA, which both show learning). It seems like a natural analysis would examine this (e.g. a plot of final performance of EC after 4000 generations with different per-generation learning budgets).

(2) It is then shown that after 4000 generations EC can learn very quickly to perform the semantic task perfectly, at least within 200 generations (Figure 4C, and perhaps much sooner, Figure 4D, Figure 4F last panel; it was hard to say. This and the previous point seem somewhat inconsistent; was it just that Figure 3 used only 100 fine-tuning steps while the perfect-task-performing networks in Figure 4 required somewhere between 100 and 200? This seems to point to extreme parameter dependence. More broadly, how should I square this inconsistency/near-inconsistency?

(3) Figure 3k, and especially Figure 4f bottom right panel, seem to show networks that correctly separate all stimuli but cannot classify them. Should I understand this as networks learning to just push apart all pairs of datapoints without structure?

(4) If I understood the genetic algorithm correctly, only three individuals from each population seeded the next generation. This seems another important parameter to tune, since I think it is far lower than standard evolutionary work, but I am not sure.

Finally, the paper most interested me as a neural network learning puzzle: how can the network perform so badly, yet lead to such different post-fine-tuning results? The paper pointed to these as 'distinct' learning phenomena without explaining what was causing those differences. The only way I could square these results in my head was as above: that the 4000 generations pushed the initial representation to represent all datapoints differntly, effectively changing the learning problem gradient descent faces from one with a lot of structure (the semantic task) that leads to stepwise learning, to one in which it was basically linear regression on a set of well separated stimuli without the structure necessary for stepwise learning. Since the paper focuses so much on learning dynamics, and studies a task where such things can be precisely probed, it would have been nice to pin down exactly what was happening slightly more.