Virtual machine schedulers for communicating parallel tasks on distributed platforms

Abstract

The main goal of this project is to work on CPU scheduling algorithms to improve the performance of jobs consisting of parallel communicating tasks and analyse the results with simulation. It is known that the use of Virtual Machines for High Performance Computing (HPC) parallel applications may cause performance reduction because of a phenomenon called 'process thrashing' where processes spend their time waiting for communications from other processes that are currently sleeping. There are several approaches to mitigate this problem one of them being Gang Scheduling, which basically allows processes that communicate with each other to run at the same time in the system, and other one being Co-Scheduling, which is a less restricted version of Gang Scheduling allowing only the parts of the processes that communicate with each other to be executed in parallel instead of strictly the complete processes. In this research we will specify through simulation the characteristics of the distributed system and perform experiments with different CPU scheduling algorithms to obtain metrics such as makespan, average and maximum run time, average and maximum wait time, average and maximum stretch and derivations of these metrics to calculate the effectiveness of the algorithms. To measure the performance of the algorithms, these will be simulated for different MPI workloads, including benchmarks, and under different distributed environments and the results will be compared and analysed to determine how these algorithms can be improved, proposing new scheduling strategies to improve the quality of the results.