Reviewer #2 (Public review):

Summary:

This paper studies the interplay of evolutionary and in-lifetime learning. The authors develop a neural network model in which initial weight configurations evolve under selective pressure, while fitness is determined by the network's performance after a learning period. They show that such a network displays very distinct learning dynamics from those trained by either gradient descent or genetic algorithms alone: in particular, they do not learn the task, but they show evidence of learning-to-learn and unusual representational structure.

Strengths:

(1) The writing, figures, and presentation of ideas were clear.

(2) The question of how evolution on initial weights combines with learning from within-lifetime experience to structure a learning trajectory seems interesting.

(3) The analysis of existing experiments was well-done, highlighting that though these networks did not really learn, they show latent learning structure that makes the network perform better from less data.

(4) The interplay between Baldwin & learning dynamics seemed novel and interesting, presenting many attractive puzzles.

Weaknesses:

First, the authors point to an important distinction between performing evolutionary selection on the weights pre- or post- lifetime training, the latter of which is Lamarckian. They argue, correctly, that their model is interesting because it selects on the weight initialisation, unlike, for example, Shuvaev et al. However, my understanding is that a long line of papers beginning perhaps with Hinton & Nowlan also do non-Lamarckian evolution: Hinton & Nowlan have unspecified weights (denoted '?' in the paper) that can be inherited and then learnt. Is this not exactly inheritance of initial conditions (in this case, whether learnable or not)? This novelty is a primary motivation of the paper, whereas to me it seems it was already apparent in Hinton & Nowlan, and developed further in what seems to be a long line of uncited literature (see next paragraph). As such, this paper's conclusions seem poorly positioned within the existing state of knowledge/literature.

Second, the algorithm is framed as novel, but I think it is a rediscovery. This framework is very close to MAML, in which an initial weight configuration is optimised by gradient descent to be good after a few steps of fine-tuning (Finn et al., 2017). The authors' approach differs in using a genetic algorithm to perform the training of the initial weights, avoiding some of the computational complexities of MAML, especially after long fine-tuning. In this, the authors have, I think, rediscovered ES-MAML, MAML where the inner optimisation loop is gradient descent, while the outer is genetic (Song et al., 2020). Other similar work is "Meta-Learning by the Baldwin Effect" (Fernando et al., 2018).

Further, within Fernando et al. there is a rich literature review, almost none of which are cited by the authors. I point especially to Keesing & Stork, 1990, which appears to show a strong dependence of Baldwin-like improvements on the amount of data, something this paper also shows but explores less thoroughly.

To summarise my critique thus far: I think the literature already answers the main concern of the motivation (i.e. non-Lamarckian neural network evolution and learning), I think it has already discovered this particular algorithm, and I think past work has more thoroughly analysed behaviours similar to those presented in this paper. Without positioning correctly within this literature, the more general contribution of the paper is hard to establish. The true novelty of the authors' analysis seems to be the emphasis on Saxe et al.-like learning dynamics and its interplay with the Baldwin effect, but I am not certain of this without knowing the literature better.

Regarding experiments, there was an interesting effect where the EC networks didn't learn but did show latent learning (Figure 2, Figure 3), which sped up later learning (Figure 4). There were a few details I was surprised by on which I would appreciate clarity:

(1) The main result has basically no headline learning under EC. This will clearly be very dependent on parameters (e.g. if you add or remove enough training steps, the algorithm becomes SGD/GA, which both show learning). It seems like a natural analysis would examine this (e.g. a plot of final performance of EC after 4000 generations with different per-generation learning budgets).

(2) It is then shown that after 4000 generations EC can learn very quickly to perform the semantic task perfectly, at least within 200 generations (Figure 4C, and perhaps much sooner, Figure 4D, Figure 4F last panel; it was hard to say. This and the previous point seem somewhat inconsistent; was it just that Figure 3 used only 100 fine-tuning steps while the perfect-task-performing networks in Figure 4 required somewhere between 100 and 200? This seems to point to extreme parameter dependence. More broadly, how should I square this inconsistency/near-inconsistency?

(3) Figure 3k, and especially Figure 4f bottom right panel, seem to show networks that correctly separate all stimuli but cannot classify them. Should I understand this as networks learning to just push apart all pairs of datapoints without structure?

(4) If I understood the genetic algorithm correctly, only three individuals from each population seeded the next generation. This seems another important parameter to tune, since I think it is far lower than standard evolutionary work, but I am not sure.

Finally, the paper most interested me as a neural network learning puzzle: how can the network perform so badly, yet lead to such different post-fine-tuning results? The paper pointed to these as 'distinct' learning phenomena without explaining what was causing those differences. The only way I could square these results in my head was as above: that the 4000 generations pushed the initial representation to represent all datapoints differntly, effectively changing the learning problem gradient descent faces from one with a lot of structure (the semantic task) that leads to stepwise learning, to one in which it was basically linear regression on a set of well separated stimuli without the structure necessary for stepwise learning. Since the paper focuses so much on learning dynamics, and studies a task where such things can be precisely probed, it would have been nice to pin down exactly what was happening slightly more.

Figures and data

Brains are optimized by natural selection and online learning.

EC fails to task-optimize but shows signs of latent learning.

EC representational dynamics are unique and do not respond to known regularities in training data.

EC speeds fine-tuning and alters gradient descent representational dynamics.