Loading...
Failure Prediction using Machine Learning in a Virtualised HPC System and application
Bashir, Mohammed Awan, Irfan U. Ugail, Hassan Muhammad, Y.
Bashir, Mohammed
Awan, Irfan U.
Ugail, Hassan
Muhammad, Y.
Publication Date
2019-06
End of Embargo
Supervisor
Rights
© 2019 Springer. Reproduced in accordance with the publisher's self-archiving policy.
The final publication is available at Springer via https://doi.org/10.1007/s10586-019-02917-1
Peer-Reviewed
Yes
Open Access status
Accepted for publication
2019-02-09
Institution
Department
Awarded
Embargo end date
Additional title
Abstract
Failure is an increasingly important issue in high performance computing and cloud systems. As
large-scale systems continue to grow in scale and complexity, mitigating the impact of failure and
providing accurate predictions with sufficient lead time remains a challenging research problem. Traditional
existing fault-tolerance strategies such as regular check-pointing and replication are not adequate because of
the emerging complexities of high performance computing systems. This necessitates the importance of having
an effective as well as proactive failure management approach in place aimed at minimizing the effect of failure
within the system. With the advent of machine learning techniques, the ability to learn from past information to predict future pattern of behaviours makes it possible to predict potential system failure more accurately. Thus, in this paper, we explore the predictive abilities of machine learning by applying a number of algorithms to improve the accuracy of failure prediction. We have developed a failure prediction model using time series and machine learning, and performed comparison based tests on the prediction accuracy. The primary algorithms we considered are the Support Vector Machine (SVM), Random Forest(RF), k-Nearest Neighbors (KNN), Classi cation and Regression Trees (CART) and Linear Discriminant Analysis (LDA). Experimental results indicates that the average prediction accuracy of our model using SVM when predicting failure is 90% accurate and effective compared to other algorithms. This f inding implies that our method can effectively predict all possible future system and
application failures within the system.
Version
Accepted manuscript
Citation
Bashir M, Awan IU, Ugail H et al (2019) Failure Prediction using Machine Learning in a Virtualised HPC System and application. Cluster Computing. 22(2): 471-485.
Link to publisher’s version
Link to published version
Link to Version of Record
Type
Article