Information, certainty, and learning
- The University of Sydney, Australia
- Rutgers University, United States
Figures
Figure 1
Median reinforcements to acquisition plotted against the C/T ratio.
Note: Asterisks show the data from Gibbon and Balsam, 1981 meta-analysis of acquisition in pigeons. The two open circles are from Jenkins et al., 1981 who used even bigger C/T ratios in some groups. The log of the informativeness is the mutual information, I, between the conditioned stimulus (CS) and the expected wait for reinforcement (λR; lower x-axis). The learning rate (right axis, reversed) is the reciprocal of reinforcements to acquisition (thus the axis increases towards zero). The regression model was fit only to the asterisks, but it also predicts the Jenkins et al. data. The value k is the x-intercept, the informativeness that produces one-trial learning. The regression model curves upward to infinity as informativeness goes to 1 and mutual information to 0 because, when the CS rate does not differ from the contextual rate of reinforcement, differential responding to the CS does not emerge (Rescorla, 1967; Rescorla, 1968).
Figure 2
Number of trials for acquisition of responding to the conditioned stimulus (CS), plotted against CS informativeness (C/T ratio).
Note. A, B, and C plot the number of trials to reach three different acquisition criteria for CS response rates exceeding the contextual rate. Each plot shows the median number of trials to criterion for each group (filled coloured circles) and the number of trials for each rat (unfilled coloured diamonds) plotted against informativeness of the CS. These data are superimposed onto the data (black asterisks) from Gibbon and Balsam, 1981 and the black regression line shown in Figure 1. The dashed coloured lines show the regression model, log[RtoAcq]=log[k·(C/T-1)s], fitted to the group medians. The slope, s, and R2 of each regression line is: in A, s=−0.68 [95% confidence intervals = –0.94, –0.42], R2=0.73; in B, s=−0.66 [-0.86, –0.46], R2 = 0.81; in C, s = –0.6 [–0.76, –0.45], R2=0.86. The seven plots in D show the cumulative fraction of rats that had acquired responding to the CS after n reinforced trials (x-axis) according to the same three acquisition criteria in A to C. In each plot, the data from two groups with similar informativeness (ι, iota) are combined to increase the sample size.
Figure 3
Terminal response rates (final five sessions) of individual rats as a function of the reinforcement rate on double-logarithmic coordinates.
Note. Red circles show response rates during the conditioned stimulus (CS) (number of pokes after the first poke divided by the remaining CS time when the rat’s head was out of the magazine) plotted against the CS reinforcement rate (1 /T). Black circles show response rates during the inter-trial interval plotted against the baseline reinforcement rate (1 /C). Filled circles show data of rats given 420 reinforced trials (Groups with C/T ratios ≤72) and open circles show data of rats given 126 reinforced trials (Groups with C/T ratios ≥110). The model in A gives the equation for the solid black regression line (R2=0.81). The slope of this line is virtually indistinguishable from 1 (R2=0.80, when slope fixed at 1). The dashed red and grey lines show regression lines, with slopes fixed at 1, fitted separately to the CS response rates and inter-trial interval (ITI) response rates. In B, the blue regression line has a slope fixed at 1, and the ITI response rates (black circles) are plotted against the overall reinforcement rate (1 /C) divided by 1.9.
Figure 4
Number of trials to each decile of peak response rate.
Note: The coloured line plots show the mean number of trials to each decile for each rat in each of the 14 groups (identified by the C/T ratio). The thick black line in each plot is the average for that group. The slopes of those black lines for each group are plotted against each group’s T in the rightmost plot of the third row. The final plot in the bottom right corner of the figure plots the Bayesian Information Criterion (BIC) for an exponential function against the BIC for a straight line when each was fitted to the trials-to-criterion data of individual rats (each open black circle shows the BICs for one rat). For each point in the top left of the plot (above the solid orange line), BICline <BICexponential, meaning that the data were better accounted for by the line than the exponential function (and vice versa for points in the bottom right half of the plot). The two dashed orange lines mark where the difference in BICs equals 4.6 which equates to odds of 10:1 in favour of the function with the lower BIC.
Figure 5
Modelling response probability across trials as a function of informativeness.
Note A: The cumulative probability of a response, p(R), calculated using Equation 6, as a function of number of reinforced trials. Each curve plots p(R) for one of the 14 levels of informativeness used in the current experiment, ι=1.5–300.B: The black line plots the number of trials taken for p(R) to reach 0.5 as a function of the same 14 levels of informativeness. The plot includes the median trials to acquisition (t2acq) for the 14 groups in the current experiment, replotted from Figure 2A. DtoI: The cumulative fraction of rats that had acquired responding (orange lines) for pairs of groups with similar informativeness, replotted from Figure 2D. The black and grey curves plot the cumulative p(R) using the same two values of ι (the black line is based on the higher of the two values).
Figure 6
nDKL and parsed response rates for two rats.
Note: Panels A to D show data from Rat 3; panels E to H show the corresponding data from Rat 176. For each rat, the top two panels plot the cumulative rate of poking during the conditioned stimulus (CS), λr|CS (solid black line), during pre-CS inter-trial intervals (ITIs), λr|ITI (dashed black line), and the contextual rate, λr|C (dotted black line), as functions of the number of trials (1 reinforcement per trial). In each plot, the red curve is the signed nDKL plotted against the right axis. The black vertical line marks when the cumulative CS response rate permanently exceeded the cumulative context rate. The two red dashed vertical lines to the right of the black line mark when the nDKL reached 0.82 (Odds 4:1 that CS rate >Context rate) and 1.92 (p<0.05 that CS rate = Context rate), and the unbroken red line marks the minimum of the nDKL. In A and E, response rates were calculated conventionally, as number of responses divided by total time. In B and F, response rates were calculated as the number of responses excluding the first response in each CS divided by the remaining time (after the first response) out of the magazine. The bottom two panels for each rat show the parsed estimates of λr|CS and λr|ITI. In C and G, the x-axis has been right-cropped to better reveal early changes. In all plots, the vertical red lines mark estimates of acquisition based on the nDKL as the Earliest estimate (leftmost), and when the odds against the null hypothesis that the parsed CS and ITI rates were equal reached 4:1, 10:1, 20:1, and 100:1.
Appendix 1—figure 1
Mean response rate for each group across Sessions or across time since CS onset.
There were 10 (groups with C/T ≤ 72) or 3 (groups with C/T ≥ 110) trials per session. Plot 15 (bottom right) shows the mean response rate per second during the CS, averaged from the last 5 sessions. Each group is identified by the length of the mean CS-US interval (T).
Appendix 1—figure 2
Response metrics, over the final 5 conditioning sessions, plotted against T.
Open grey circles show data for individual rats. Black filled circles show mean data for each group. The dashed orange lines are the best-fitting regression lines.
Appendix 1—figure 3
Response rate and trials to criterion for an individual rat.
The grey line in A shows the response rate on each trial for an individual rat (Rat 30, Group C/T = 4.5). The response rate was calculated as the number of responses during the CS divided by the total time out of the magazine but excluding the latency to first response (i.e., the total time across which the animal could respond). In B, the cumulative response count across trials for the same rat is plotted against the cumulative opportunity to respond (cumulative time out of the magazine). The slope of this cumulative function was used to identify the trial on which the rat’s response rate reached each decile (from 10% to 90%) of its peak response rate, as shown in C. These trials are also marked as circles on the response plots in A and B.
Appendix 1—figure 4
The mean number of trials to criterion for each of the 14 groups of rats (with different C/T ratios) in the current experiment.
In the plot on the left, the filled black circles show the number of trials for each group to reach the criterion used by Thrailkill et al., 2020. In the plot on the right, the filled black circles show the number of trials for each group to reach the criterion used by Kirkpatrick and Church, 2000. For reference, the orange circles in both graphs show the number of trials for the difference between the cumulative CS response rate and cumulative ITI rate to become permanently positive (the criterion shown in Figure 2A of the main article).
Tables
Table 1
Summary of groups.
| Group | T | T range | C | ITI [range] | C/T |
|---|---|---|---|---|---|
| 1 | 42 | 2–82 | 63 | 21 [15-27] | 1.5 |
| 2 | 48 | 2–94 | 144 | 96 [15–177] | 3 |
| 3 | 21 | 2–40 | 94.5 | 73.5 [15-132] | 4.5 |
| 4 | 12 | 2–22 | 72 | 60 [15–105] | 6 |
| 5 | 6 | 2–10 | 54 | 48 [15–81] | 9 |
| 6 | 62 | 2–122 | 930 | 868 [15–1721] | 15 |
| 7 | 30 | 2–58 | 600 | 570 [15–1125] | 20 |
| 8 | 18 | 2–34 | 486 | 468 [15–921] | 27 |
| 9 | 36 | 2–70 | 1296 | 1260 [15–2505] | 36 |
| 10 | 26 | 2–50 | 1404 | 1378 [15–2741] | 54 |
| 11 | 8 | 2–14 | 576 | 568 [15–1121] | 72 |
| 12 | 16 | 2–30 | 1760 | 1744 [15–3473] | 110 |
| 13 | 10 | 2–18 | 1800 | 1790 [15–3565] | 180 |
| 14 | 14 | 2–26 | 4200 | 4186 [15–8357] | 300 |
-
.All values for T (CS-US interval), C (US-US interval), and the inter-trial interval (ITI) are in seconds.
Additional files
-
MDAR checklist
- https://cdn.elifesciences.org/articles/102155/elife-102155-mdarchecklist1-v1.docx
-
Supplementary file 1
Data plots for each individual rat.
- https://cdn.elifesciences.org/articles/102155/elife-102155-supp1-v1.pdf
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Information, certainty, and learning
eLife 13:RP102155.
https://doi.org/10.7554/eLife.102155.4
