Utilising multiple GPU cards and multiple hosts

Abstract

The sparse Matrix-Vector multiplication is a key operation in science and engineering along with the Conjugate Gradient method. Hence both of them are currently being studied nowadays with the purpose of increasing its performance mainly on GPU devices following the GPGPU trend (General Purpose GPU computing). This thesis presents a study of the speedup gained when performing the sparse Matrix-Vector multiplication based on the ELLR-T storage format, and a Conjugate Gradient solver that makes use of this algorithm, on different computing environments including multiple GPU cards and multiple hosts. The code implemented has been specifically designed to harness the computational architecture of the GPU by using the Nvidia CUDA (Compute Unified Device Architecture) API and the bottlenecks to its performance have been carefully analysed. The analysis shows that the bottleneck of the sparse Matrix-Vector algorithm performance, and therefore the Conjugate Gradient method, is the memory bandwidth of the computing architecture where it is executed. However, when executed on multiple GPUs and/or multiple nodes, the performance is bounded by the vector transfers between cards and nodes and the synchronization time. In fact, the multi-GPU version of the Conjugate Gradient solver presents approximately the same performance as the sequential one.The sequential Conjugate Gradient solver implemented in this thesis achieves a speedup up to 26 on a Tesla C1060 over an Intel Xeon E5462 and up to 14 on a Tesla C2050 over an Intel Core i7 X980 for matrices that represent real problems obtained from the University of Florida Sparse Matrix Collection.