Shape-invariant encoding of dynamic primate facial expressions in human perception

Figures

Figure 1

Download asset Open asset

Figure 2

Download asset Open asset

Figure 3

Download asset Open asset

Figure 4

Download asset Open asset

Figure 5

Download asset Open asset

Figure 6

Download asset Open asset

Figure 7

Download asset Open asset

Appendix 1—figure 1

Download asset Open asset

Appendix 1—figure 2

Download asset Open asset

Appendix 1—figure 3

Download asset Open asset

Tables

Key resources table

Reagent type (species) or resource	Designation	Source or reference	Identifiers	Additional information
Software, algorithm	Custom-written software written in C#	This study	https://hih-git.neurologie.uni-tuebingen.de/ntaubert/FacialExpressions (copy archived at swh:1:rev:6d041a0a0cc7055618f85891b85d76e0e7f80eed; Taubert, 2021)
Software, algorithm	C3Dserver	Website	https://www.c3dserver.com
Software, algorithm	Visual C++ Redistributable for Visual Studio 2012 Update4 × 86 and x64	Website	https://www.microsoft.com/en-US/download/details.aspx?id=30679
Software, algorithm	AssimpNet	Website	https://www.nuget.org/packages/AssimpNet
Software, algorithm	Autodesk Maya 2018	Website	https://www.autodesk.com/education/free-software/maya
Software, algorithm	MATLAB 2019b	Website	https://www.mathworks.com/products/matlab.html
Software, algorithm	Psychophysics toolbox 3.0.15	Website	http://psychtoolbox.org/
Software, algorithm	R 3.6	Website	https://www.r-project.org/
Other	Training data for interpolation algorithm	This study	https://hih-git.neurologie.uni-tuebingen.de/ntaubert/FacialExpressions/tree/master/Data/MonkeyHumanFaceExpression
Other	Stimuli for experiments	This study	https://hih-git.neurologie.uni-tuebingen.de/ntaubert/FacialExpressions/tree/master/Stimuli

Appendix 1—table 1

Model comparison
Monkey front view	Model	Accuracy [%]	Accuracy increase [%]	BIC	Parameters	df	χ2	p
	Model 1	38.29		7487	33
	Model 2	57.86	19,56 (relative to Model 1)	5076	36	3	2411	<0,0001
	Model 3	49.49	11,2 (relative to Model 1)	6125	36	3	1362	<0,0001
	Model 4	77.53	19,67 (relative to Model 2)	3586	39	3	1490	<0,0001
	Model 5	77.53	0 (relative to Model 4)	3598	42	3	11.997	<0.0074
	Model 6	77.42	−0,11 (relative to Model 4)	3580	42	3	5.675	0.129
Human front view
	Model 1	36.84		7481	33
	Model 2	54.22	17,38 (relative to Model 1)	5541	36	3	1940	<0,0001
	Model 3	53.56	16,72 (relative to Model 1)	5847	36	3	1633	<0,0001
	Model 4	81.56	27,35 (relative to Model 2)	3420	39	3	2120	<0,0001
	Model 5	81.35	−0,22 (relative to Model 4)	3309	42	3	112	<0,0001
	Model 6	81.38	−0,18 (relative to Model 4)	3389	42	3	31.66	<0,0001
Monkey 30-degree
	Model 1	35.32		6913	33
	Model 2	57.40	22,08 (relative to Model 1)	4314	36	3	2622	<0,0001
	Model 3	49.36	14,04 (relative to Model 1)	5179	36	3	1757	<0,0001
	Model 4	84.04	26,64 (relative to Model 2)	2359	39	3	1977	<0,0001
	Model 5	84.88	0,84 (relative to Model 4)	2335	42	3	48	<0,0001
	Model 6	84.08	0,04 (relative to Model 4)	2331	42	3	28	<0,0001
Human 30-degree
	Model 1	37.40		6819	33
	Model 2	55.72	18,32 (relative to Model 1)	4843	36	3	1975	<0,0001
	Model 3	54.36	16,96 (relative to Model 1)	5217	36	3	1602	<0,0001
	Model 4	81.32	25,6 (relative to Model 2)	2910	39	3	1956	<0,0001
	Model 5	82.88	1,56 (relative to Model 4)	2809	42	3	101	<0,0001
	Model 6	81.92	0,6 (relative to Model 4)	2890	42	3	19	0.0002

Appendix 1—table 2

Parameters of motion morphing algorithm
Parameters	Description	Value
D	Data dimension	208
M	First layer dimension	6
Q	Second layer dimension	2
T	Number of samples per trial	150
S	Number of species	2
E	Number of expressions	two or 3
N	Number of all samples	T * S * E
Hyper parameters (learned)		Size
${\beta }_1$	Inverse width of kernel $k_1$	1
${\beta }_2$	Inverse width of kernel $k_2$	1
${\beta }_3$	Inverse width for non-linear part one of kernel $k_3$	1
${\beta }_4$	Inverse width for non-linear part two of kernel $k_3$	1
${\gamma }_1$	Precision absorbed from noise term ${\mathbf{\epsilon }}_d$	1
${\gamma }_2$	Variance for non-linear part of $k_1$	1
${\gamma }_3$	Variance for linear part of $k_1$	1
${\gamma }_4$	Variance for non-linear part of $k_2$	1
${\gamma }_5$	Variance for linear part of $k_2$	1
${\gamma }_6$	Variance for non-linear part of $k_3$	1
${\gamma }_7$	Variance for linear part one of $k_3$	1
${\gamma }_8$	Variance for linear part two of $k_3$	1
Variables		Size
Y	Data	N x D
H	Latent variable of first layer	N x M
X	Latent variable of second layer	N x Q
${\widehat{\mathbf{s}}}_{\mathrm{M}}$	Style variable vector for monkey species	S x 1
${\widehat{\mathbf{s}}}_{\mathrm{H}}$	Style variable vector for human species	S x 1
${\widehat{\mathbf{e}}}_1$	Style variable vector for expression one	E x 1
${\widehat{\mathbf{e}}}_2$	Style variable vector for expression two	E x 1

Appendix 1—table 3

ANOVAs
Threshold	Monkey avatar	Sum of square	df	Mean square	F	p
	Stimulus type	0,00	2	0,00	0,00	0999
	Expression type	1,20	1	1,20	188,83	0000
	Stimulus * Expression	0,06	2	0,03	4,51	0015
	Error	0,42	60	0,01
	Total	1,72	65
	Human avatar
	Stimulus type	0,00	2	0,00	0,01	0993
	Expression type	0,40	1	0,40	46,37	0000
	Stimulus * Expression	0,05	2	0,03	3,15	0049
	Error	0,57	60	0,01
	Total	1,02	65
Steepness	Original motion stimulus
	Avatar type	376,68	1	376,68	6,3	0016
	Expression type	0,36	1	0,36	0,01	0939
	Avatar * Expression	0,16	1	0,16	0	0959
	Error	2391,21	40	59,78
	Total	2768,41	43
	Occluded motion stimulus
	Avatar type	286,17	1	286,17	3,33	0076
	Expression type	0,02	1	0,02	0	0988
	Avatar * Expression	0,00	1	0,00	0	0995
	Error	3094,54	36	85,96
	Total	3380,73	39
	Equilibrated motion stimulus
	Avatar type	1,57	1	1,57	0,4	0533
	Expression type	0,25	1	0,25	0,06	0803
	Avatar * Expression	0,02	1	0,02	0	0945
	Error	174,76	44	3,97
	Total	176,60	47

Additional files

Transparent reporting form

https://cdn.elifesciences.org/articles/61197/elife-61197-transrepform-v1.pdf

Download elife-61197-transrepform-v1.pdf

Download links