Exploring Topological Properties in Artificial Neural Networks

Abstract

Biological neural systems often represent information on low-dimensional manifolds that reflect the topology of their encoded variables. This suggests that neural activity can be naturally organized in geometrically meaningful ways, as seen in rodent head direction cells forming circular manifolds. This proposal examines whether artificial neural networks (ANNs) trained on tasks with well-defined topologies—such as planar or spherical coordinates from autonomous driving datasets like Apolloscape, cyclic temporal variables, or graph-structured road networks—develop similar low-dimensional representations aligned with the variables' inherent topology. We consider convolutional and vision transformer models for image data, graph neural networks for road network graphs, and 3D or point-based models for LIDAR point clouds, analyzing their internal activations with dimensionality reduction and topological data analysis. If successful, this approach not only elucidates the nature of internal representations in ANNs but also offers insights into the computational principles that bridge artificial systems and biological cognition.