Presenter: Paul DELESTRAC

Abstract:
Recent advances in Machine Learning have brought human-level intelligence to many popular domains such as computer vision or natural language processing. These improvements came with a huge increase in the amount of compute required by these models. Training is becoming more and more a bottleneck for these multi-billion parameters models and GPUs are still the prevailing architecture for such a task. However, the complex execution flow of ML training on GPUs is often misunderstood and this can stand in the way of exploration for more compute-efficient solutions. In this talk, our main goal is to provide a better understanding of the execution of ML training on GPUs. We describe and analyze the execution flow from top down, starting with the ML framework, going to the execution of operations on the GPU cores. We share our experience of using GPUs for ML training. Finally, we provide some performance evaluation and results from our last findings.

Date: September 27, 2023 from 2 to 4 pm (salle de séminiaires, LIRMM*)