Algebraic Analysis for Non-Regular Learning Machines

Abstract

Hierarchical learning machines are non-regular and non-identifiable statistical models, whose true parameter sets are analytic sets with singularities. Using algebraic analysis, we rigorously prove that the stochastic complexity of a non-identifiable learning machine (ml - 1) log log n + const., is asymptotically equal to >'1 log n - where n is the number of training samples. Moreover we show that the rational number >'1 and the integer ml can be algorithmically calculated using resolution of singularities in algebraic geometry. Also we obtain inequalities 0 < >'1 ~ d/2 and 1 ~ ml ~ d, where d is the number of parameters.