The Interplay Between Functionality and Complexity in Neural Network Efficiency

The design of efficient neural network controllers entails identifying suitable network architectures, which requires a high level of expertise, or defining methods that allow users to evolve connection weights and topology in parallel. Although both approaches are widely used in the literature, they present limitations related either to the criteria used for selecting network architectures or the complexity of co-evolving weights and topology. Moreover, the role of single neurons within neural networks is often overlooked. This work presents an analysis of functional neurons --- defined as units that cannot be deleted without producing remarkable performance differences --- within neural network controllers, with the aim to provide a general framework for explaining neural network efficiency. Specifically, we investigate a series of benchmark locomotion problems, varying the number of internal neurons to capture potential relationships between functionality and complexity. Our findings show the difficulty of identifying an optimal network configuration. However, we observe that functional neurons play a crucial role in the final performance, hence emphasizing their significance. Moreover, our experiments indicate that the percentage of functional neurons appears to depend on the features of the evolutionary algorithm used. This underscores the existence of complex, intertwined relationships between the chosen optimization method and the resulting properties of neural network controllers.