Creating and controlling visual environments using BonVision

Abstract

Real-time rendering of closed-loop visual environments is important for next-generation understanding of brain function and behaviour, but is often prohibitively difficult for non-experts to implement and is limited to few laboratories worldwide. We developed BonVision as an easy-to-use open-source software for the display of virtual or augmented reality, as well as standard visual stimuli. BonVision has been tested on humans and mice, and is capable of supporting new experimental designs in other animal models of vision. As the architecture is based on the open-source Bonsai graphical programming language, BonVision benefits from native integration with experimental hardware. BonVision therefore enables easy implementation of closed-loop experiments, including real-time interaction with deep neural networks, and communication with behavioural and physiological measurement and manipulation devices.

Data availability

BonVision is an open-source software package available to use under the MIT license. It can be downloaded through the Bonsai (bonsai-rx.org) package manager, and the source code is available at: github.com/bonvision/BonVision. All benchmark programs and data are available at https://github.com/bonvision/benchmarks. Installation instructions, demos and learning tools are available at: bonvision.github.io/.

Article and author information

Author details

  1. Gonçalo Lopes

    NeuroGears, London, United Kingdom
    Competing interests
    Gonçalo Lopes, Gonçalo Lopes is affiliated with NeuroGEARS Ltd. The author has no financial interests to declare..
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-0731-4945
  2. Karolina Farrell

    Experimental Psychology, University College London, London, United Kingdom
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-0707-2838
  3. Edward A B Horrocks

    Experimental Psychology, University College London, London, United Kingdom
    Competing interests
    No competing interests declared.
  4. Chi Yu Lee

    Experimental Psychology, University College London, London, United Kingdom
    Competing interests
    No competing interests declared.
  5. Mai M Morimoto

    Department of Experimental Psychology, University College London, London, United Kingdom
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-9654-3960
  6. Tomaso Muzzu

    Institute of Behavioural Neuroscience, University College London, London, United Kingdom
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-0018-8416
  7. Amalia Papanikolaou

    Institute of Behavioural Neuroscience, University College London, London, United Kingdom
    Competing interests
    No competing interests declared.
  8. Fabio R Rodrigues

    Experimental Psychology, University College London, London, United Kingdom
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-4848-7167
  9. Thomas Wheatcroft

    Experimental Psychology, University College London, London, United Kingdom
    Competing interests
    No competing interests declared.
  10. Stefano Zucca

    Experimental Psychology, University College London, London, United Kingdom
    Competing interests
    No competing interests declared.
  11. Samuel G Solomon

    Experimental Psychology, University College London, London, United Kingdom
    For correspondence
    s.solomon@ucl.ac.uk
    Competing interests
    No competing interests declared.
  12. Aman B Saleem

    Experimental Psychology, University College London, London, United Kingdom
    For correspondence
    aman.saleem@ucl.ac.uk
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7100-1954

Funding

Wellcome Trust (200501)

  • Aman B Saleem

Human Frontiers in Science Program (RGY0076/2018)

  • Aman B Saleem

Stavros Niarchos Foundation / Research to Prevent Blindness

  • Samuel G Solomon

Medical Research Council (R023808)

  • Samuel G Solomon
  • Aman B Saleem

Biotechnology and Biological Sciences Research Council (R004765)

  • Samuel G Solomon
  • Aman B Saleem

Wellcome Trust: OPen Research (200501/Z/16/A)

  • Aman B Saleem

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Animal experimentation: All experiments were performed in accordance with the Animals (Scientific Procedures) Act 1986 (United Kingdom) and Home Office (United Kingdom) approved project and personal licenses. The experiments were approved by the University College London Animal Welfare Ethical Review Board under Project License 70/8637.

Human subjects: All procedures were approved by the Experimental Psychology Ethics Committee at University College London (Ethics Application EP/2019/002). We obtained informed consent, and consent to publish from all participants.

Reviewing Editor

  1. Chris I Baker, National Institute of Mental Health, National Institutes of Health, United States

Publication history

  1. Received: December 7, 2020
  2. Accepted: April 20, 2021
  3. Accepted Manuscript published: April 21, 2021 (version 1)
  4. Version of Record published: May 7, 2021 (version 2)
  5. Version of Record updated: May 19, 2021 (version 3)

Copyright

© 2021, Lopes et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,247
    Page views
  • 216
    Downloads
  • 4
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Gonçalo Lopes
  2. Karolina Farrell
  3. Edward A B Horrocks
  4. Chi Yu Lee
  5. Mai M Morimoto
  6. Tomaso Muzzu
  7. Amalia Papanikolaou
  8. Fabio R Rodrigues
  9. Thomas Wheatcroft
  10. Stefano Zucca
  11. Samuel G Solomon
  12. Aman B Saleem
(2021)
Creating and controlling visual environments using BonVision
eLife 10:e65541.
https://doi.org/10.7554/eLife.65541

Further reading

    1. Neuroscience
    Cecilia Gallego-Carracedo et al.
    Research Article

    The spiking activity of populations of cortical neurons is well described by the dynamics of a small number of population-wide covariance patterns, the 'latent dynamics'. These latent dynamics are largely driven by the same correlated synaptic currents across the circuit that determine the generation of local field potentials (LFP). Yet, the relationship between latent dynamics and LFPs remains largely unexplored. Here, we characterised this relationship for three different regions of primate sensorimotor cortex during reaching. The correlation between latent dynamics and LFPs was frequency-dependent and varied across regions. However, for any given region, this relationship remained stable throughout the behaviour: in each of primary motor and premotor cortices, the LFP-latent dynamics correlation profile was remarkably similar between movement planning and execution. These robust associations between LFPs and neural population latent dynamics help bridge the wealth of studies reporting neural correlates of behaviour using either type of recordings.

    1. Neuroscience
    Jan Boelts et al.
    Research Advance Updated

    Inferring parameters of computational models that capture experimental data are a central task in cognitive neuroscience. Bayesian statistical inference methods usually require the ability to evaluate the likelihood of the model—however, for many models of interest in cognitive neuroscience, the associated likelihoods cannot be computed efficiently. Simulation-based inference (SBI) offers a solution to this problem by only requiring access to simulations produced by the model. Previously, Fengler et al. introduced likelihood approximation networks (LANs, Fengler et al., 2021) which make it possible to apply SBI to models of decision-making, but require billions of simulations for training. Here, we provide a new SBI method that is substantially more simulation efficient. Our approach, mixed neural likelihood estimation (MNLE), trains neural density estimators on model simulations to emulate the simulator, and is designed to capture both the continuous (e.g., reaction times) and discrete (choices) data of decision-making models. The likelihoods of the emulator can then be used to perform Bayesian parameter inference on experimental data using standard approximate inference methods like Markov Chain Monte Carlo sampling. We demonstrate MNLE on two variants of the drift-diffusion model and show that it is substantially more efficient than LANs: MNLE achieves similar likelihood accuracy with six orders of magnitude fewer training simulations, and is significantly more accurate than LANs when both are trained with the same budget. Our approach enables researchers to perform SBI on custom-tailored models of decision-making, leading to fast iteration of model design for scientific discovery.