Experimental design

A) Exemplary product graphs (Cartesian product, right) selected from the vast array of possible graph products that can be constructed from two simpler graph components (left). The graph product of two or more simple graph components results in complex product state spaces, which can be decomposed (and simplified) into their constituting components.

B) Graph factors used in prior learning (left) and transfer learning (right)

C) During the sequence learning task (both prior and transfer learning), graph factors were combined into compound graphs (product graphs) by merging their images. In both prior learning and transfer learning, participants were presented with sequences of meaningful compound holistic images, such as a cat on a chair or a house with a car. Depicted is an example sequence generated by the product graph of the yellow graph factor and the gold graph factor used during prior learning (panel B, left, top). The example sequence involved a simultaneous traversal in the yellow graph factor and in the gold graph factor, as shown in panel B (left, top). After each sequence presentation (consisting of 8 compound stimuli), participants’ understanding of the temporal dynamics was assessed in two ways, as detailed in Figure 2.

D) During sequence presentations in transfer learning, participants only observed a subset of transitions from the compound graph formed by the green graph factor and the black graph factor, as shown in panel B (right), e.g., the 16 depicted transitions, forming two disjoint subgraphs. The remaining possible compound graph transitions formed a held-out set (see examples in panel F). This allowed us to test participants’ understanding of the subprocesses generating the observed sequences. The same principles applied to prior learning. Depicted are example sequences produced by the subgraphs of the product graph used in transfer learning, involving a simultaneous traversal in the green graph factor and in the black graph factor, as shown in panel B (right). Subgraph sequence 1 (left) was presented two times, after which subgraph sequence 2 (right) was presented twice.

E) For illustration of the transfer learning compound graph, we only show the parts of the compound graph for transfer learning that participants experienced during sequence presentation, created from the 4-cycle and 6-bridge graphs shown in panel B (right). Note that the 6-bridge graph did not feature a truly bi-directional edge between the central nodes (7//7’ and 10/10’), but instead had a “refractory” edge, such that it was not possible to transition back and forth within one step between central nodes of the graph. In both phases of the task (prior and transfer learning), the compound graphs consisted of 24 compound images.

F) Examples of held-out transitions used in transfer learning between compound images. Note that these transitions entailed some compounds which were observed (as part of other transitions) during sequence presentations.

Behavioral task and transfer learning performance.

A) Example sequence produced by subgraph 1 of the product graph used in transfer learning (reproduced Fig. 1D left).

B) The compound image sequence in panel A was generated from a simultaneous traversal of two graph factors, by combining their images (reproducing Fig. 1F for convenience). The example sequence was generated from a 4-state cyclic graph factor and a 6-state bridge graph factor (reproducing Fig. 1B for convenience).

C) Exemplar schematic of experience and inference probes during transfer learning, respectively. During both prior and transfer learning, following presentation of an entire compound image sequence, participants were probed to predict upcoming states that adhered to one of the 16 experienced compound image transitions (experience probes, left column). Additionally, an inference probe (right column) tested participants’ ability to infer and accurately predict held-out transitions (Fig. 1C). In both probe questions, participants saw a compound image and were asked: “Imagine you see this image. What would be the next image?”. Two compound images – correct next compound image (eligible based on the graph structure) and a lure compound image (non-eligible) – were presented as choice options. The lure compound image always matched the correct option on one of the two individual images composing the compound image (e.g., D’12’ -> A’10’ or B’10’). This allowed us to test knowledge of both graph factors separately (in the current example, knowledge that D’ allows a transition to A’ but not B’) (top row: 4-state factor; bottom row: 6-state factor) underlying the observed sequence (and the held-out transitions) separately.

D) Aggregated probabilities of correct answers during the transfer learning phase, separately for experience (left) and inference probes (right) as a function of probed size (4-state cycle (green) vs 6-state bridge (black)) and condition (4-cycle prior vs 6-bridge prior). Each dot represents the arithmetic mean of experience and inference probe performance for one participant, bars represent the arithmetic mean of the distribution, error bars depict the standard error of the mean and the dashed gray line represents chance level point estimate (probability correct = 0.5).

E) Posterior density plots for each parameter estimate (posterior mean = black line) for the best-fitting GLM. The dashed vertical line represents a zero effect. In probed size (4-cycle or 6-bridge) and probe type (experience vs inference) effects, parameter estimates below zero indicate higher accuracy in the 4-cycle probes (vs. 6-bridge probes) and experience probes (vs. inference probes), respectively. In the condition effect, parameter estimates below zero indicate higher accuracy in the 4-cycle prior condition (vs. 6-bridge prior condition). The positive interaction effect indicates selectively higher accuracy in 4-cycle probes in the 4-cycle prior condition, and higher accuracy in 6-bridge probes in the 6-cycle prior condition.

Neural dynamics of structural mapping between subprocesses.

A) Participants performed two stimulus localizer tasks, once before and one after (PRE and POST localizer) the main experimental task (prior and transfer learning). These tasks comprised multiple presentations of all features used in prior and transfer learning (PRE and POST). Additionally, the PRE localizer also contained the compound images used during prior learning (not relevant for this analysis). A feature similarity analysis (akin to RSA) on stimulus-evoked neural activity recorded during the two localizer tasks (PRE and POST localizer) for the two feature sets, comprising the compound images used in prior and transfer learning at time-points from -200 to 800 ms relative to stimulus-onset.

B) Feature sets in prior learning phase (yellow frame, left) and transfer learning phase (blue frame, right).

C) The presented theoretical representational dissimilarity matrices (RDM) were regressed onto the empirical neural similarity matrices – separately for the 4-state graph factor features and the 6-state graph factor features. Lighter colors denote higher values. Note that the hypothesized similarity changes are only computed across the two feature sets used in prior learning (yellow) and transfer learning (blue), but not within each set, avoiding confounding task-related changes with feature similarity unrelated to abstract subprocess structure.

D) Time courses of condition differences (4-cycle prior condition – 6-bridge prior condition) on 4-state graph factor (green) and 6-state graph factor (black) feature similarity changes in the weighted average PCs from PRE to POST localizer. Shaded error bars indicate standard errors of difference. Dark green line represents time points at which feature similarity changes for the condition x graph factor interaction effect were statistically significant, corrected for multiple comparisons at the cluster-level (PFWE < 0.05, non-parametric permutation test). Horizontal line indicates a null effect.

Abstract representation of subprocess dynamics.

A) Knowledge reuse of decomposed subprocesses, discovered during prior learning, predicts neural similarities when subprocess dynamics are shared across prior and transfer learning. During prior learning, we trained multivariate classifiers (L1-regularized logistic regressions) to distinguish between different transition types in the 6-state graph factor for 4-cycle prior condition (6-path graph, top) and 6-bridge prior condition (6-bridge graph, bottom). We then tested the classifiers’ generalization ability on entirely new stimuli during transfer learning. Each number denotes a different type of transition, reflecting edges connecting graph nodes, namely boundary (1), intermediate (2), and central (3). During prior learning, classifiers were trained at post-transition stimulus presentations (e.g., in the 6-bridge prior, the classifier for transition type 2 was trained on neural activity recorded during presentations of the ball if preceded by the sandwich; or on the hammer if preceded by the ball). At transfer learning we tested the classifier’s generalization performance for identical transitions, but where this now involved entirely different stimulus sets. Since training and testing were performed on independent datasets this obviates cross-validation. Note that a comparison of classification accuracy across conditions was only possible for the 6-state graph factor, as the 4-state cycle does not provide for a distinction between transition types.

B) Transfer learning time courses of 6-state graph factor transitions classification accuracy, averaged per condition based on prior experience of either 4-cycle (green) or 6-bridge condition (black). Shaded error bars indicate standard errors of the mean. Black and grey bars above indicate temporal clusters in which classification accuracy differences between conditions (6-bridge prior condition > 4-cycle prior condition) are significantly different (two-sample t-test), corrected for multiple comparisons at the cluster-level (PFWE < 0.05, non-parametric permutation test (first cluster (black): PFWE = .035, two-tailed; second cluster (grey): PFWE = .027, one-tailed). Horizontal lines indicate chance level decoding accuracy (.33).

C) Spearman correlations between average behavioral accuracy in 6-bridge experience probes and per participant averaged classification accuracy across all timepoints post-image onset where there is a significant condition difference between 4-cycle prior condition and 6-bridge prior condition on classification accuracy (see panel B). Each datapoint represents an individual participant. The 6-bridge prior condition (rho = .41, p = .041) showed a stronger positive correlation than the 4-cycle prior condition (rho = -.16, p = .440, correlation difference: Z = 1.99, p = .046, Fisher r to z).

D) Spearman correlations between average behavioral accuracy in 6-bridge inference probes and per participant averaged classification accuracy across all timepoints post-image onset where there is a significant condition difference between 4-cycle prior condition and 6-bridge prior condition on classification accuracy (see panel B). Each datapoint represents an individual participant. There was no evidence for a correlation difference between the 6-bridge prior condition (rho < .01, p = .996) and the 4- cycle prior condition (rho = .09, p = .657, correlation difference: Z = .32, p = .752, Fisher r to z).

Transfer learning performance.

Aggregated probabilities of correct answers during the transfer learning phase, separately for both experience (left) and inference probes (right) as a function of probed size (4-state cycle (green) vs 6-state bridge (black)) and condition (4-cycle prior vs 6-bridge prior). Full sample (top row), high performers (>50th percentile, middle row) and low performers (<=50th percentile, bottom row). Each dot represents one participant, bars represent the arithmetic mean of the distribution, error bars depict the standard error of the mean and the dashed gray line represents chance level point estimate (probability correct = 0.5).

Temporal dynamics of successor state predictions.

Time courses of change in predicted probability of classifiers (reactivation) relative to average pre-stimulus onset baseline during transfer learning across both conditions, related to all stimulus presentations, for the respective successor features of each compound stimulus presentation. We trained multivariate classifiers (L1-regularized logistic regressions, trained excluding the respective other feature in each compound) to distinguish between features on the 4-cycle and 6-bridge graph factor during the PRE localizer tasks and tested the compound-by-compound successor feature predictions on both graph factors separately. Depicted are average changes in predicted probability (subtracting average pre-stimulus onset predicted probability), shaded error bars indicate standard errors of the mean. Horizontal line indicates null effect (0 predicted probability change). Solid lines on top index temporal clusters wherein the predicted probability is statistically significant above 0, corrected for multiple comparisons at the cluster-level (PFWE < 0.05, non-parametric permutation test, one-tailed).