Metrics do not support multilabel tasks.
🐛 Bug
Scikit-learn metrics deal well will multilabel tasks, but this doesn't seem to be supported in Pytorch-Lightning metrics.  There is this #3350 , but it seems to confuse multiclass with multilabel (multiple values to predict).
To Reproduce
Given predictions tensor:
tensor([[0., 0.],
              [0., 0.],
              [0., 0.],
              [0., 0.],
              [0., 0.],
              [0., 0.],
              [0., 0.],
              [0., 0.]])

and labels tensor:
tensor([[1, 0],
             [1, 0],
             [1, 0],
             [1, 0],
             [1, 0],
             [1, 0],
             [1, 0],
             [1, 0]])


The call to f1_score(met_preds,labels, class_reduction='macro') yields tensor(0.3333), because it flattens the tensors and macro-averages per class.
Expected behavior
I would expect it to be consistent with the call to
sk_f1_score(labels.numpy(), met_preds.numpy(), average='macro'), which yields 0.0, because it treats each column separately and macro-averages them per task.
This discrepancy also occurs for other metrics. For example sklearn deals with multilabel accuracy by using subset accuracy (0 here), but PL produces an accuracy score of 0.5.
Environment

PyTorch Version : 1.6
OS (e.g., Linux): OSX
How you installed PyTorch (conda, pip, source): conda
Python version: 3.7.8
Pytorch-Lightning version : 1.0.2