Six principles for biologically based computational models of cortical cognition

doi:10.1016/S1364-6613(98)01241-8

Trends in Cognitive Sciences

Volume 2, Issue 11, 1 November 1998, Pages 455-462

https://doi.org/10.1016/S1364-6613(98)01241-8 Get rights and content

Abstract

This review describes and motivates six principles for computational cognitive neuroscience models: biological realism, distributed representations, inhibitory competition, bidirectional activation propagation, error-driven task learning, and Hebbian model learning. Although these principles are supported by a number of cognitive, computational and biological motivations, the prototypical neural-network model (a feedforward back-propagation network) incorporates only two of them, and no widely used model incorporates all of them. It is argued here that these principles should be integrated into a coherent overall framework, and some potential synergies and conflicts in doing so are discussed.

Section snippets

The principles

The six principles can be grouped into three categories. The first principle, biological realism, is in a category by itself, providing a general overriding constraint on the framework. The next three principles, distributed representations, inhibitory competition, and bidirectional activation propagation (interactivity), are concerned with the architecture of the network and the general behavior of the neuron-like processing units within it. The final two principles, error-driven task learning

(2) Distributed representations

The cortex is widely believed to use distributed representations to encode information. A distributed representation uses multiple active neuron-like processing units to encode information (as opposed to a single unit, localist representation), and the same unit can participate in multiple representations. Each unit in a distributed representation can be thought of as representing a single feature, with information being encoded by particular combinations of such features. Electrophysiological

Learning principles

Learning is essential for shaping the representations of neural networks according to the structure of the environment. A key issue is what aspects of the environmental structure should be learned, with the understanding that not everything can or should be represented. The following two learning principles exploit two complementary aspects of environmental structure: task demands, and the extent to which different things co-occur. The first is referred to as `task learning' for obvious

Interactions among the principles

The preceding discussion provided specific and compelling motivations for each of the individual principles. In this section, three examples of interactions (synergies and conflicts) among the six principles will be discussed. The first example comes from the GRAIN framework, and deals with the consequences of interactivity and noise. The second explores the interactions between distributed representations and competition, which can be at odds with each other. The last explores the interactions

Outstanding questions

•
Are there cognitive phenomena or biological facts that appear to contradict directly the core principles outlined here?
•
Is it possible that different parts of the cortex emphasize some principles over others? How might this influence functional specialization in the cortex?
•
How many other important principles are missing from the present list?
•
Can complex, sequential cognitive processing be shown to emerge from such basic principles as those discussed here, or does this require a whole new set of

Acknowledgements

I thank Yuko Munakata and Jerry Rudy for their helpful comments. This work was supported in part by NIH program project grant MH47566.

References (65)

D.H. Ackley et al.
A learning algorithm for Boltzmann machines
Cognit. Sci.
(1985)
G.L. Collingridge et al.
NMDA receptors: their role in long-term potentiation
Trends Neurosci.
(1987)
D.W. Massaro
Testing between the TRACE model and the fuzzy logical model of speech perception
Cognit. Psychol.
(1989)
G.E. Hinton
Connectionist learning procedures
Artif. Intell.
(1989)
P. Baldi et al.
Neural networks and principal components analysis: learning from examples without local minima
Neural Netw.
(1989)
J. Kay et al.
Contextually guided unsupervised learning using local multivariate binary processors
Neural Netw.
(1998)
K. Plunkett et al.
From rote learning to system building: acquiring verb morphology in children and connectionist nets
Cognition
(1993)
McClelland, J.L. (1993) The GRAIN model: a framework for modeling the dynamics of information processing, in Attention...
Y. Munakata
Rethinking infant knowledge: toward an adaptive process account of successes and failures in object permanence tasks
Psychol. Rev.
(1997)
Desimone, R. and Ungerleider, L.G. (1989) Neural mechanisms of visual processing in monkeys, in Handbook of...

Georgopoulos, A.P. (1990) Neurophysiology and reaching, in Attention and Performance (Vol. 13) (Jeannerod, M., ed.),...

S.C. Rao et al.

Integration of what and where in the primate prefrontal cortex

Science

(1997)

Hinton, G.E., McClelland, J.L. and Rumelhart, D.E. (1986) Distributed Representations, in Parallel Distributed...

P.L.A. Gabbot et al.

Quantitative distribution of GABA-immunoreactive neurons in the visual cortex (area 17) of the cat

Exp. Brain Res.

(1986)

Kohonen, T. (1984) Self-Organization and Associative Memory,...

J.L. McClelland et al.

An interactive activation model of context effects in letter perception: 1

An account of basic findings. Psychol. Rev.

(1981)

Rumelhart, D.E. and Zipser, D. (1986) Feature discovery by competitive learning, in Parallel Distributed Processing...

S. Grossberg

Adaptive pattern classification and universal recoding I: Parallel development and coding of neural feature detectors

Biol. Cybern.

(1976)

H.B. Barlow

Unsupervised learning

Neural Comput.

(1989)

D.J. Field

What is the goal of sensory coding?

Neural Comput.

(1994)

B.A. Olshausen et al.

Emergence of simple-cell receptive-field properties by learning a sparse code for natural images

Nature

(1996)

D.J. Felleman et al.

Distributed hierarchical processing in the primate cerebral cortex

Cereb. Cortex

(1991)

J.B. Levitt

Topography of pyramidal neuron intrinsic connections in macaque monkey prefrontal cortex (areas 9 and 46)

J. Comp. Neurol.

(1993)

White, E.L. (1989) Cortical Circuits: Synaptic Organization of the Cerebral Cortex: Structure, Function, and Theory,...

Smolensky, P. (1986) Information processing in dynamical systems: foundations of harmony theory, in Parallel...

S.P. Vecera et al.

Figure–ground organization and object recognition processes: an interactive account

J. Exp. Psychol. Hum. Percept. Perform.

(1998)

M.A. Peterson

Object recognition processes can and do operate before figure ground organization

Curr. Dir. Psychol. Sci.

(1994)

Rumelhart, D.E, Hinton, G.E. and Williams, R.J. (1986) Learning internal representations by error propagation, in...

F.H.C. Crick

The recent excitement about neural networks

Nature

(1989)

D. Zipser et al.

A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons

Nature

(1988)

R.C. O'Reilly

Biologically plausible error-driven learning using local activation differences: the generalized recirculation algorithm

Neural Comput.

(1996)

Hinton, G.E. and McClelland, J.L. (1988) Learning representations by recirculation, in Neural Information Processing...

Cited by (227)

The impact of early and late blindness on language and verbal working memory: A brain-constrained neural model
2024, Neuropsychologia
Neural circuits related to language exhibit a remarkable ability to reorganize and adapt in response to visual deprivation. Particularly, early and late blindness induce distinct neuroplastic changes in the visual cortex, repurposing it for language and semantic processing. Interestingly, these functional changes provoke a unique cognitive advantage – enhanced verbal working memory, particularly in early blindness. Yet, the underlying neuromechanisms and the impact on language and memory-related circuits remain not fully understood. Here, we applied a brain-constrained neural network mimicking the structural and functional features of the frontotemporal-occipital cortices, to model conceptual acquisition in early and late blindness. The results revealed differential expansion of conceptual-related neural circuits into deprived visual areas depending on the timing of visual loss, which is most prominent in early blindness. This neural recruitment is fundamentally governed by the biological principles of neural circuit expansion and the absence of uncorrelated sensory input. Critically, the degree of these changes is constrained by the availability of neural matter previously allocated to visual experiences, as in the case of late blindness. Moreover, we shed light on the implication of visual deprivation on the neural underpinnings of verbal working memory, revealing longer reverberatory neural activity in ‘blind models’ as compared to the sighted ones. These findings provide a better understanding of the interplay between visual deprivations, neuroplasticity, language processing and verbal working memory.
Competitive learning to generate sparse representations for associative memory
2023, Neural Networks
One of the most well established brain principles, Hebbian learning, has led to the theoretical concept of neural assemblies. Based on it, many interesting brain theories have spawned. Palm’s work implements this concept through multiple binary Willshaw associative memories, in a model that not only has a wide cognitive explanatory power but also makes neuroscientific predictions. Yet, Willshaw’s associative memory can only achieve top capacity when the stored vectors are extremely sparse (number of active bits can grow logarithmically with the vector’s length). This strict requirement makes it difficult to apply any model that uses this associative memory, like Palm’s, to real data. Hence the fact that most works apply the memory to optimal randomly generated codes that do not represent any information. This issue creates the need for encoders that can take real data, and produce sparse representations - a problem which is also raised following Barlow’s efficient coding principle. In this work, we propose a biologically-constrained network that encodes images into codes that are suitable for Willshaw’s associative memory. The network is organized into groups of neurons that specialize on local receptive fields, and learn through a competitive scheme. After conducting auto- and hetero-association experiments on two visual data sets, we can conclude that our network not only beats sparse coding baselines, but also that it comes close to the performance achieved using optimal random codes.
Neurobiological mechanisms for language, symbols and concepts: Clues from brain-constrained deep neural networks
2023, Progress in Neurobiology
Neural networks are successfully used to imitate and model cognitive processes. However, to provide clues about the neurobiological mechanisms enabling human cognition, these models need to mimic the structure and function of real brains. Brain-constrained networks differ from classic neural networks by implementing brain similarities at different scales, ranging from the micro- and mesoscopic levels of neuronal function, local neuronal links and circuit interaction to large-scale anatomical structure and between-area connectivity. This review shows how brain-constrained neural networks can be applied to study in silico the formation of mechanisms for symbol and concept processing and to work towards neurobiological explanations of specifically human cognitive abilities. These include verbal working memory and learning of large vocabularies of symbols, semantic binding carried by specific areas of cortex, attention focusing and modulation driven by symbol type, and the acquisition of concrete and abstract concepts partly influenced by symbols. Neuronal assembly activity in the networks is analyzed to deliver putative mechanistic correlates of higher cognitive processes and to develop candidate explanations founded in established neurobiological principles.
A multilayered bidirectional associative memory model for learning nonlinear tasks
2023, Neural Networks
A multilayered bidirectional associative memory neural network is proposed to account for learning nonlinear types of association. The model (denoted as the MF-BAM) is composed of two modules, the Multi-Feature extracting bidirectional associative memory (MF), which contains various unsupervised network layers, and a modified Bidirectional Associative Memory (BAM), which consists of a single supervised network layer. The MF generates successive feature patterns from the original inputs. These patterns change the relationship between the inputs and targets in a way that the BAM can learn. The model was tested on different nonlinear tasks, such as the N-bit, Double Moon and its variants, and the 3-class spiral task. Behaviors were reported through learning errors, decision zones, and recall performances. Results showed that it was possible to learn all tasks consistently. By manipulating the number of units per layer and the number of unsupervised network layers in the MF, it was possible to change the level of nonlinearity observed in the decision boundaries. Furthermore, results indicated that different behaviors were achieved from the same set of inputs by using the different generated patterns. These findings are significant as they showed how a BAM-inspired model could solve nonlinear tasks in a more cognitively plausible fashion.
Spiking neural predictive coding for continually learning from data streams
2023, Neurocomputing
For energy-efficient computation in specialized neuromorphic hardware, we present spiking neural coding, an instantiation of a family of artificial neural models grounded in the theory of predictive coding. This model, the first of its kind, works by operating in a never-ending process of “guess-and-check”, where neurons predict the activity values of one another and then adjust their own activities to make better future predictions. The interactive, iterative nature of our system fits well into the continuous time formulation of sensory stream prediction and, as we show, the model’s structure yields a local synaptic update rule, which can be used to complement or as an alternative to online spike-timing dependent plasticity. In this article, we experiment with an instantiation of our model consisting of leaky integrate-and-fire units. However, the framework within which our system is situated can naturally incorporate more complex neurons such as the Hodgkin-Huxley model. Our experimental results in pattern recognition demonstrate the potential of the model when binary spike trains are the primary paradigm for inter-neuron communication. Notably, spiking neural coding is competitive in terms of classification performance and experiences less forgetting when learning from a task sequence, offering a more computationally economical, biologically-motivated alternative to popular artificial neural networks.
Neural Logic Circuits: An evolutionary neural architecture that can learn and generalize
2023, Knowledge-Based Systems
We introduce Neural Logic Circuits (NLC), an evolutionary, weightless, and learnable neural architecture loosely inspired by the neuroplasticity of the brain. This new paradigm achieves learning by evolution of its architecture through reorganization of augmenting synaptic connections and generation of artificial neurons functioning as logic gates. These neural units mimic biological nerve cells stimulated by binary input signals and emit excitatory or inhibitory pulses, thus executing the “all-or-none” character of their natural counterparts. Unlike Artificial Neural Networks (ANN), our model achieves generalization ability without intensive weight training and dedicates computational resources solely to building network architecture with optimal connectivity. We evaluated our model on well-known binary classification datasets using advanced performance metrics and compared results with modern and competitive machine learning algorithms. Extensive experimental data reveal remarkable superiority of our initial model, called NLCv1, on all test instances, achieving outstanding results for implementation of this new paradigm.

View all citing articles on Scopus

View full text

Trends in Cognitive Sciences

ReviewSix principles for biologically based computational models of cortical cognition

Abstract

Section snippets

The principles

(2) Distributed representations

Learning principles

Interactions among the principles

Outstanding questions

Acknowledgements

Cognit. Sci.

Trends Neurosci.

Cognit. Psychol.

Artif. Intell.

Neural Netw.

Neural Netw.

Cognition

Rethinking infant knowledge: toward an adaptive process account of successes and failures in object permanence tasks

Psychol. Rev.

Integration of what and where in the primate prefrontal cortex

Science

Quantitative distribution of GABA-immunoreactive neurons in the visual cortex (area 17) of the cat

Exp. Brain Res.

An interactive activation model of context effects in letter perception: 1

An account of basic findings. Psychol. Rev.

Adaptive pattern classification and universal recoding I: Parallel development and coding of neural feature detectors

Biol. Cybern.

Unsupervised learning

Neural Comput.

What is the goal of sensory coding?

Neural Comput.

Emergence of simple-cell receptive-field properties by learning a sparse code for natural images

Nature

Distributed hierarchical processing in the primate cerebral cortex

Cereb. Cortex

Topography of pyramidal neuron intrinsic connections in macaque monkey prefrontal cortex (areas 9 and 46)

J. Comp. Neurol.

Figure–ground organization and object recognition processes: an interactive account

J. Exp. Psychol. Hum. Percept. Perform.

Object recognition processes can and do operate before figure ground organization

Curr. Dir. Psychol. Sci.

The recent excitement about neural networks

Nature

A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons

Nature

Biologically plausible error-driven learning using local activation differences: the generalized recirculation algorithm

Neural Comput.

Review
Six principles for biologically based computational models of cortical cognition