Implement Asynchronous GPU transfer and Training with Multithreading
🚀 Feature
Asynchronous GPU transfer can be achieved by utilizing pinned memory with multithreading
Minimal example code
https://github.com/HenryJia/Lighter/blob/master/lighter/train/loaders.py
Motivation
Parallelrising GPU transfer and training will cut down time GPU is stuck waiting for data from CPU
https://devblogs.nvidia.com/how-overlap-data-transfers-cuda-cc/
Pitch
Everyone likes faster training and maximal GPU utilisation
Alternatives
Not Applicable
Additional context
None