A methodological framework using statistical tests for comparing machine learning based models applied to fault diagnosis in rotating machinery

© 2016 IEEE. Selecting an adequate machine learning model, e.g. for feature selection or classification, is a very important task in developing machine learning applications. In order to perform an adequate selection, statistic tests are introduced by several approaches but some of them are hard to reproduce in different case studies due to the lack of a systematic application procedure. This work presents a methodological framework based on statistic tests, either parametric or non-parametric, to compare multiple machine learning models for solving a specific problem. The procedure first aims to detect which feature selection method is the best for each machine learning based model, and then such models are compared using the previous results. A real world problem for fault detection in rotating machinery is studied to illustrate the application of the proposed methodological framework, using the accuracy in classification as the performance measure.