Evolutionary neural architecture search on transformers for RUL prediction

Remaining useful life (RUL) predictions are a key enabler for achieving efficient maintenance in the context of Industry 4.0. Data-driven approaches, in particular employing deep neural networks (DNNs), have shown success in the RUL prediction task. However, although their architecture considerably affects performance, DNNs are usually handcrafted by human experts via a labor-intensive design process. To overcome this issue, we propose a neural architecture search (NAS) technique that explores a search space using a genetic algorithm (GA). It automatically discovers the optimal architectures of Transformers for RUL predictions. Our GA allows an efficient search, by making use of a performance predictor, updated at every generation, that reduces the needed network training. To our knowledge, this is the first work to optimize the architecture of Transformers for RUL predictions using evolutionary computation. We evaluate the performance of the found solutions on a widely-used benchmark dataset, the CMAPSS, based on RMSE and s-score. In comparison with the state-of-the-art, the Transformer obtained by our NAS method outperforms other recent handcrafted DNNs in terms of RMSE, and is comparable regarding s-score. Our results demonstrate that the proposed method provides better prediction accuracy with less human effort compared to other data-driven approaches.