A Theory of Sensorimotor Processing in the Neocortex and the Implications for Machine Intelligence

Abstract: The brain learns about the world through movement. Since the late 1800s it has been known that movement of the body and sensors is required for the brain to learn the structure of objects and how to manipulate them. How the neural circuits of the brain integrate sensation and movement, aka “sensorimotor” processing, is a mystery. With no accepted theories of sensorimotor integration, most artificial neural networks are sensory only with no means of integrating movement. In this talk I present a theory of how neocortical circuits learn the structure of the world through movement. The theory shows how individual columns in the neocortex integrate sensory features with location information derived from movement to learn the 3D structure of the world and objects within the world. Long distance connections between columns resolve ambiguity resulting in fast inference based on partial knowledge. I propose that these capabilities are an essential component of intelligence. 

I show how the theory maps to cortical anatomy, demonstrate the theory via simulation, and I discuss the implications of the theory for creating intelligent machines.