Deep Learning: Branching into brains

What can artificial intelligence learn from neuroscience, and vice versa?.
  1. Adam Shai
  2. Matthew Evan Larkum  Is a corresponding author
  1. Stanford University, United States
  2. Humboldt University, Germany

Deep learning is a subfield of machine learning that focuses on training artificial systems to find useful representations of inputs. Recent advances in deep learning have propelled the once arcane field of artificial neural networks into mainstream technology (LeCun et al., 2015). Deep neural networks now regularly outperform humans on difficult problems like face recognition and games such as Go (He et al., 2015; Silver et al., 2017). Traditional neuroscientists have also taken an interest in deep learning because it seemed initially that there were telling analogies between deep networks and the human brain. Nevertheless, there is a growing impression that the field might be approaching a new ‘wall’ and that deep networks and the brain are intrinsically different.

Chief among these differences is the widely held belief that backpropagation, the learning algorithm at the heart of modern artificial neural networks, is biologically implausible. This issue is so central to current thinking about the relationship between artificial and real brains that it has its own name: the credit assignment problem. The error in the output of a neural network (that is, the difference between the output and the 'correct' answer) can be reported or 'backpropagated' to any connection in the network, no matter where it is, to teach the network how to refine the output. But for a biological brain, neurons only receive information from the neurons they are connected to, making credit assignment a real problem. How does the brain blindly adjust the strength of the connections between neurons that are far removed from the output of the network? In the absence of a solution, we may be forced to conclude that deep learning and brains are incompatible after all.

Now, in eLife, Jordan Guerguiev, Timothy Lillicrap and Blake Richards propose a biologically inspired solution to the credit assignment problem (Guerguiev et al., 2017). Central to their model is the structure of the pyramidal neuron, which is the most prevalent cell type in the cortex (the outer layer of the brain). Pyramidal neurons have been a source of aesthetic pleasure and interesting research questions for neuroscientists for decades. Each neuron is shaped like a tree with a trunk reaching up and dividing into branches near the surface of the brain as if extending toward a source of energy or information. Can it be that, while most cells of the body have relatively simple shapes, evolution has seen to it that cortical neurons are so intricately shaped as to be apparently impractical?

Guerguiev et al. – who are based at the University of Toronto, the Canadian Institute for Advanced Research, and DeepMind – report that this impractical shape has an advantage: the long branched structure means that error signals at one end of the neuron and sensory input at the other end are kept separate from each other. These sources of information can then be brought together at the right moment in order to find the best solution to a problem.

As Guerguiev et al. note, many facts about real neurons and the structure of the cortex turn out to be just right to find optimal solutions to problems. For instance, the bottoms of cortical neurons are located just where they need to be to receive signals about sensory input, while the tops of these neurons are well placed to receive feedback error signals (Cauller, 1995; Larkum, 2013). The key to this design principle seems to be to keep these distinct information streams largely independent. At the same time, ion channels under the control of a host of other nearby neurons process and gate the transfer of information within the neuron.

Taking inspiration from these facts Guerguiev et al. implement a deep network with units that have different compartments, just like real neurons, that can separate sensory input from feedback error signals. These units have all the information they need to know in order to nudge the network toward the desired output. Guerguiev et al. prove formally that this approach is mathematically sound. Moreover, their new, biologically plausible deep network is able to perform well on a task to identify handwritten numbers, and does so by creating what are referred to as hierarchical representations. This phenomenon refers to the increasingly complex nature of the responses of the network's layers, commonly found in more traditional deep learning models, and in the sensory cortices of biological brains.

Doubtless, there will be more twists and turns to this story as more biological details are incorporated into the model. For instance the brain also faces a time-based credit assignment problem (Friedrich et al., 2011; Gütig, 2016). Guerguiev et al. admit that this network does not outperform non-biologically derived deep networks – yet. Nevertheless, the model they present paves the way for future work that links biological networks to machine learning. The hope is that this can be a two-way process, in which insights from the brain can be used to improve artificial intelligence, and insights from artificial intelligence can be used to reveal how the brain operates.

References

  1. Conference
    1. He K
    2. Zhang X
    3. Ren S
    4. Sun J
    (2015)
    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
    Proceedings of the IEEE International Conference on Computer Vision. pp. 1026–1034.

Article and author information

Author details

  1. Adam Shai

    Adam Shai is in the Department of Biology, Stanford University, Stanford, United States

    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-1833-3906
  2. Matthew Evan Larkum

    Matthew Evan Larkum is at the Neurocure Cluster of Excellence, Humboldt University of Berlin, Germany

    For correspondence
    matthew.larkum@hu-berlin.de
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-9799-2656

Publication history

  1. Version of Record published: December 5, 2017 (version 1)

Copyright

© 2017, Shai et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 6,041
    Page views
  • 691
    Downloads
  • 3
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Adam Shai
  2. Matthew Evan Larkum
(2017)
Deep Learning: Branching into brains
eLife 6:e33066.
https://doi.org/10.7554/eLife.33066
  1. Further reading

Further reading

    1. Computational and Systems Biology
    2. Medicine
    Jie Lian, Jiajun Deng ... Varut Vardhanabhuti
    Research Article

    Background:

    We proposed a population graph with Transformer-generated and clinical features for the purpose of predicting overall survival (OS) and recurrence-free survival (RFS) for patients with early stage non-small cell lung carcinomas and to compare this model with traditional models.

    Methods:

    The study included 1705 patients with lung cancer (stages I and II), and a public data set for external validation (n=127). We proposed a graph with edges representing non-imaging patient characteristics and nodes representing imaging tumour region characteristics generated by a pretrained Vision Transformer. The model was compared with a TNM model and a ResNet-Graph model. To evaluate the models' performance, the area under the receiver operator characteristic curve (ROC-AUC) was calculated for both OS and RFS prediction. The Kaplan–Meier method was used to generate prognostic and survival estimates for low- and high-risk groups, along with net reclassification improvement (NRI), integrated discrimination improvement (IDI), and decision curve analysis. An additional subanalysis was conducted to examine the relationship between clinical data and imaging features associated with risk prediction.

    Results:

    Our model achieved AUC values of 0.785 (95% confidence interval [CI]: 0.716–0.855) and 0.695 (95% CI: 0.603–0.787) on the testing and external data sets for OS prediction, and 0.726 (95% CI: 0.653–0.800) and 0.700 (95% CI: 0.615–0.785) for RFS prediction. Additional survival analyses indicated that our model outperformed the present TNM and ResNet-Graph models in terms of net benefit for survival prediction.

    Conclusions:

    Our Transformer-Graph model was effective at predicting survival in patients with early stage lung cancer, which was constructed using both imaging and non-imaging clinical features. Some high-risk patients were distinguishable by using a similarity score function defined by non-imaging characteristics such as age, gender, histology type, and tumour location, while Transformer-generated features demonstrated additional benefits for patients whose non-imaging characteristics were non-discriminatory for survival outcomes.

    Funding:

    The study was supported by the National Natural Science Foundation of China (91959126, 8210071009), and Science and Technology Commission of Shanghai Municipality (20XD1403000, 21YF1438200).

    1. Cancer Biology
    2. Computational and Systems Biology
    Erika K Ramos, Chia-Feng Tsai ... Huiping Liu
    Research Article

    Tumor-initiating cells with reprogramming plasticity or stem-progenitor cell properties (stemness) are thought to be essential for cancer development and metastatic regeneration in many cancers; however, elucidation of the underlying molecular network and pathways remains demanding. Combining machine learning and experimental investigation, here we report CD81, a tetraspanin transmembrane protein known to be enriched in extracellular vesicles (EVs), as a newly identified driver of breast cancer stemness and metastasis. Using protein structure modeling and interface prediction-guided mutagenesis, we demonstrate that membrane CD81 interacts with CD44 through their extracellular regions in promoting tumor cell cluster formation and lung metastasis of triple negative breast cancer (TNBC) in human and mouse models. In-depth global and phosphoproteomic analyses of tumor cells deficient with CD81 or CD44 unveils endocytosis-related pathway alterations, leading to further identification of a quality-keeping role of CD44 and CD81 in EV secretion as well as in EV-associated stemness-promoting function. CD81 is co-expressed along with CD44 in human circulating tumor cells (CTCs) and enriched in clustered CTCs that promote cancer stemness and metastasis, supporting the clinical significance of CD81 in association with patient outcomes. Our study highlights machine learning as a powerful tool in facilitating the molecular understanding of new molecular targets in regulating stemness and metastasis of TNBC.