Multi-objective Evolutionary Optimization of Imbalanced Fast Feedforward Networks

Many embedded applications have strict energy, memory, and time constraints, making neural network (NN) inference particularly challenging. Recently, a novel NN architecture called Fast Feedforward Networks (FFFs) has been proposed to achieve inference with extremely lightweight computational demands and minimal latency. However, FFFs still face two key limitations: (1) their memory footprint remains relatively high despite fast inference, and (2) their architectures are restricted to balanced forms, where each branch has the same depth, and each leaf has the same width. This can limit their scalability, performance, and potential for further improvements in inference speed. In this work, we address these challenges by introducing a novel Neural Architecture Search (NAS) pipeline that explores imbalanced FFF architectures using multi-objective evolutionary optimization. Specifically, we utilize NSGA-II, setting the objectives as accuracy and total number of model parameters, along with Grammatical Evolution, a method particularly effective for tree-structured models. We conduct comprehensive experiments on three different IoT datasets, benchmarking our method against standard (i.e., balanced) FFFs and feedforward models in terms of accuracy, Multiply-Accumulate Operations, and parameter count. Furthermore, we analyze the Pareto-optimal architectures found by NAS, to gain insights into their structural properties. Our experiments (https://github.com/DIOL-UniTN/MOE-IFFF) demonstrate that the proposed approach can achieve an up to ∼13% accuracy gain compared to FFFs with the same model size, and reduce model size by up to 10x compared to FFFs with equivalent performance, all while achieving lower or comparable inference cost.