Metrics inside a dictionary are not properly moved to GPU
🐛 Bug

The following code WORKS:
self.accuracy = Accuracy()

The following code DOES NOT WORK (throws CPU error):
self.metrics['accuracy'] = Accuracy()

Please reproduce using the BoringModel and post here

https://colab.research.google.com/drive/1oLE2Ts4AkQwd2oLSaz8qNvFdiygyvgrZ?usp=sharing
To Reproduce

Expected behavior

Move metric to proper device (GPU) whether assigned to self.some_metrics or self.all_metrics['some_metric']
Actual behavior
--> 113         self.correct += torch.sum(preds == target)

RuntimeError: Trying to pass too many CPU scalars to CUDA kernel!
Environment
COLLAB

CUDA:

GPU:

Tesla P100-PCIE-16GB


available:         True
version:           10.1


Packages:

numpy:             1.18.5
pyTorch_debug:     False
pyTorch_version:   1.6.0+cu101
pytorch-lightning: 1.0.4
tqdm:              4.41.1


System:

OS:                Linux
architecture:

64bit



processor:         x86_64
python:            3.6.9
version:           #1 SMP Thu Jul 23 08:00:38 PDT 2020



Additional context

I put the Metrics in a dict, so I can use the following code:
for metric_name, metric in self.metrics.items():
    self.log(f"{metric_name}/{phase}_step", metric(predictions, targets))