Hippocampus-inspired Adaptive Neural Algorithms

Abstract:For many real-world applications, the value of neural machine learning algorithms lies in an ability to adapt to continual changes in the world.  This real-time adaptation has often been a challenge to methods such as deep learning that leverage a significant data-intensive training process prior to deployment in an application domain.  To address this, we look to the brain’s ability to adapt itself and learn continuously.  While the cortex does exhibit learning in the adult brain, most rapid learning - such as one-shot learning and episodic memory - leverages the area of the brain known as the hippocampus.  Here, I will describe two forms of hippocampus-inspired algorithmic adaptation.  First, I will illustrate the ability of a biologically-inspired form of adaptation, adult neurogenesis, to allow deep learning networks to adapt in response to changes in the world.  This algorithm, Neurogenesis Deep Learning, demonstrates that the effective lifespan of conventional deep learning networks can be extended through continual adaptation mechanisms.  Second, I will introduce our novel algorithmic implementation of the hippocampus, known as Deep One-Shot Content Addressable Memory, or DOS-CAM.  DOS-CAM reformulates the classic model of hippocampus function (DG pattern separation -> CA3 Hopfield-like auto-associations -> CA1 temporal associations) to a form more suitable for context dependent associations needed to perform rapid learning of high level representations produced by a cortical-inspired or deep learning network.