Lessons from the Primate Visual System

Abstract

The primate visual system can perform an astonishing array of tasks as reflected by the correspondingly large portion of the cerebral cortex devoted to analyzing retinal signals. Although a potential source of inspiration for computer vision, with a few exceptions, progress has been slow in this field. Principal obstacles are the lack of any exhaustive list of what vision achieves in humans and the restricting of areas of investigation to a few topics such as motion, object categories and the control of a few actions such as reaching or saccades. Here I will review how we integrated several experimental techniques to address a question that arose from interactions with computer vision scientists more than fifteen years ago: the extraction of 3D surfaces. This goal is achieved by a new type of higher-order visual neuron: the gradient-selective neurons. Neurons selective for speed gradients were initially discovered in motion processing areas, such as MT/V5, MSTd and FST, located in the monkey superior temporal sulcus (STS). Subsequently, neurons selective for disparity gradients were discovered in shape processing areas, such as TEs and AIP. By combining these single-cell studies with fMRI in human and awake monkey, we were able to localize similar neurons to human cortical areas. In the second part I address my present interest in understanding the visual signals related to the actions of conspecifics, which is perhaps the ultimate challenge of motion processing, but which receives surprisingly little attention in vision. The understanding of observed actions exemplifies my statement that to be useful visual signals have to leave the visual system, as signals related to biological motion in the STS are indeed relayed to parietal regions involved in the control of diverse actions to be understood as actions.