validation_epoch_end needs to return CUDA tensors
🐛 Bug

I'm not sure if this is expected behaviour or not, but upgrading to the latest version (from 0.8.1) caused my validation_epoch_end to break. It appears that a CUDA tensor is expected for the metric where before the tensor was device agnostic.
This was using sklearn's roc_auc_score. I haven't yet got around to testing PL's new metrics.
Feel free to close if this is expected behaviour.
To Reproduce
This is my validation_epoch_end. Uncommenting .to(avg_loss.device) allows this to run with the dev version of PL.
This was run with ddp, precision=16 using Apex AMP.
def validation_epoch_end(self, outputs):
        avg_loss = torch.stack([x["val_loss"] for x in outputs]).mean()
        y_pred = torch.cat([x["y_pred"].squeeze() for x in outputs]).cpu().numpy()
        y_true = torch.cat([x["y_true"].squeeze() for x in outputs]).cpu().numpy()

        metric = torch.tensor(roc_auc_score(y_true, y_pred))  # .to(avg_loss.device)

        tensorboard_logs = {
            "loss/validation": avg_loss,
            "auc": metric,
        }
        return {"val_loss": avg_loss, "log": tensorboard_logs, "auc": metric}


The error message can be seen here: #2411 (comment)
Code sample

Expected behavior

Environment
Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).
You can get the script and run it with:
wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py


PyTorch Version (e.g., 1.0): 1.5
OS (e.g., Linux): Ubuntu 20.04
How you installed PyTorch (conda, pip, source):
Build command you used (if compiling from source):
Python version: 3.8.2
CUDA/cuDNN version: 10.2
GPU models and configuration: Dual GPU (ddp)
Any other relevant information: Apex AMP

Additional context