What's the true meaning of gpus argument in Trainer
Hello, I have a question to the "gpus" argument for the Trainer() class. I don't think the documentation is clear in my view. From the docs, I only know that "gpus: Which GPUs to train on.". However, from the link here: https://pytorch-lightning.readthedocs.io/en/latest/multi_gpu.html, I get different answer.
1.
DEFAULT (int) specifies how many GPUs to use
Trainer(gpus=k)
2.
train on 32 GPUs (4 nodes)
trainer = Trainer(gpus=8, distributed_backend='ddp', num_nodes=4)
Does the gpus mean number of gpu per node or total number of gpus in use?