DDP doesn't work properly with CUDA_VISIBLE_DEVICE  
🐛 Bug
DDP trainer doesn't work properly when CUDA_VISIBLE_DEVICE is set.
To Reproduce
Steps to reproduce the behavior:

Set CUDA_VISIBLE_DEVICE=1,2
Run DDP trainer with 2 GPUs
The main process will use available_gpus[self.trainer.local_rank] that is equal to 1
The second process will use GPU process_idx that is again equal to 1
Thus both processes will use the same single GPU, instead of both

Expected behavior
Training should use both GPUs.
Suggestion
Looks like the following if-statement causes the problem:

  
    
      pytorch-lightning/pytorch_lightning/accelerators/ddp_backend.py
    
    
         Line 204
      in
      5b4db52
    
  
  
    

        
          
           if is_master: 
        
    
  


Why do we use int(available_gpus[self.trainer.local_rank]) instead of simple process_idx?
As far as I understand, the master process should always use GPU 0, which is equal to the first GPU in the CUDA_VISIBLE_DEVICE list. Please, correct me if I am wrong.