Data DistributedSampler Error  when using Multi-GPU setting (with ddp).
🐛 Bug

Hi,
I have converted my pure PyTorch code into pytorch-lightning code, however,  the pl code would be crashed when using multi-gpus settings, while the code runs successfully when I set gpus=1.
My task is a binary classification task, and the error happens in AUC-ROC-score computing using sklearn:
File "/usr/local/lib/python3.6/dist-packages/sklearn/metrics/ranking.py", line 256, in _binary_roc_auc_score                                                    raise ValueError("Only one class present in y_true. ROC AUC score "
ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.

Code sample
the trainer:
trainer = pl.Trainer(gpus=opt.gpu_num, distributed_backend='ddp')

the core part in model.py: Note that I need to collect the outputs of all validation-steps first, and then to compute the metric in the validation_epoch_end.
    def training_step(self, batch, batch_idx):
        label_list = batch[0]
        data = batch[1:]
        score = self(data)
        loss = F.cross_entropy(score, label_list.max(1)[1])
        result = pl.TrainResult(loss)
        result.log('train_loss', loss, sync_dist=True)
        return result

    def validation_step(self, batch, batch_idx):
        label_list = batch[0]
        data = batch[1:]
        score = self(data)
        loss = F.cross_entropy(score, label_list.max(1)[1])
        result = pl.EvalResult(loss)
        result.label_list = label_list.squeeze(1).tolist()
        result.pred_list = score.squeeze(1).tolist()
        return result

    def validation_epoch_end(self, val_outputs):

        labels = [i for k in val_outputs.label_list for i in k]
        preds = [i for k in val_outputs.pred_list for i in k]
    
        res = cal_metric(all_labels, all_preds)
        result = pl.EvalResult(checkpoint_on=torch.tensor(res['group_auc']))
        result.log_dict(res, prog_bar=True, sync_dist=True)
        return result


Environment

CUDA:
- GPU:
- GeForce GTX 1080 Ti
- GeForce GTX 1080 Ti
- available:         True
- version:           10.2
Packages:
- numpy:             1.17.4
- pyTorch_debug:     False
- pyTorch_version:   1.6.0
- pytorch-lightning: 0.9.0
- tensorboard:       2.0.2
- tqdm:              4.48.2
System:
- OS:                Linux
- architecture:
- 64bit
- ELF
- processor:         x86_64
- python:            3.6.9
- version:           #110-Ubuntu SMP Tue Jun 23 02:39:32 UTC 2020


So  what might be the reason that leads to the error in validation_epoch_end in multi-gpus setting. (it goes well in gpus=1).
Thanks.