Computer Graphics from a Neuroscientist's Perspective

Abstract

A hallmark of human vision is to recognize objects in a complex naturalistic scene. However, the exact mechanism behind the representations of a three-dimensional scene remains obscure. This study proposes a tool to investigate human perception by using a computer graphics approach. We use three-dimensional object meshes to render synthetic scenes and try to study how these scenes will be represented in the brain. We render a collection of datasets with different appearance and pose variations by changing exactly one property at a time. A model is trained on each of these datasets for a classification task and is then evaluated using alignment metrics; deviations in the metrics indicate the importance of a particular brain region in representing a particular property. Our results indicate that significant effects are observed by changing rotation in the dataset as well as appearance effects such as texture. In conclusion, we propose a promising method to study the brain using computer graphics. While our method is not perfect, this approach can provide valuable insights into human vision.