Fault analysis of edge router Linux system message log files with machine learning

Abstract

[EN] In our everyday life, many things are done electronically, e-mails, buying different tickets, navigation, and so on. These devices must always work correctly. Linux-based routers are no exception. The devices create a log file about everything that happens to them, which in many cases is in an unreadable state for humans. When such a device fails, professional workers spend many hours searching for individual errors and often fail to detect them.

The main goal of this research is to create a System log files analyzer for predictive maintenance because there is currently no quick and effective program for this. Therefore, several publicly available programs were overviewed that are capable of individual log files, especially system log files, and analyzing them. It was investigated how they are structured and what types of logs are analyzed in what way.

Syslog files were examined with different labels in this dissertation, which were converted using text mining methods. Then they were analyzed with several models, including decision trees, random forests, the XGBoost model, and neural networks, and they predicted labels for each log file. To make this more successful, text-mining methods were applied, and each Syslog was sliced into sequences. These transformations gave much more transparent and accurate results. After teaching and testing the models, the results were obtained, which were evaluated with different indicators, such as accuracy and a confusion matrix.

It was also part of this thesis the writing of a program for a Raspberry Pi that can give the professional workers guidance on what kind of failures happened in the system, thus reducing their time spent on maintenance and more efficient debugging. The program was written in Python and tested on two different types of Raspberry Pi.

In conclusion, several types of errors can occur in a log file, and machine learning methods can significantly help the work of professionals by guiding analysis. With this AI technology, professionals know more precisely what error they need to look for in which sequence; they do not have to look through all the rows. This allows them to spend more time on improvements and upgrades, and not have to deal with maintenance.