Incorrect Precision/Recall/F1 score compared to sklearn
🐛 Bug

To Reproduce
Steps to reproduce the behavior:

Copy the code
Run the code from top to bottom
Compare print results
See Difference between sklearn and Lightning


Code

import torch
import numpy as np
import pytorch_lightning as pl
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

print(pl.__version__)


#### Generate binary data
pl.seed_everything(2020)
n = 10000  # number of samples
y = np.random.choice([0, 1], n)
y_pred = np.random.choice([0, 1], n, p=[0.1, 0.9])
y_tensor = torch.tensor(y)
y_pred_tensor = torch.tensor(y_pred)


# Accuracy appears alright
print('accuracy from sklearn', accuracy_score(y, y_pred))
print('accuracy from lightning functional', pl.metrics.functional.accuracy(y_pred_tensor, y_tensor, num_classes=2))
print('accuracy from lightning tensor', pl.metrics.Accuracy(num_classes=2)(y_pred_tensor, y_tensor))

## results
## accuracy from sklearn 0.4986
## accuracy from lightning functional tensor(0.4986)
## accuracy from lightning tensor tensor(0.4986)

# Precision appears to be off, compared to sklearn
print('precision from sklearn', precision_score(y, y_pred))
print('precision from lightning functional', pl.metrics.functional.precision(y_pred_tensor, y_tensor, num_classes=2))
print('precision from lightning tensor', pl.metrics.Precision(num_classes=2)(y_pred_tensor, y_tensor))

## precision from sklearn 0.5005544466622311
## precision from lightning functional tensor(0.4906)
## precision from lightning tensor tensor(0.4906)

#Recall appears to be off, compared to sklearn
print('recall from sklearn', recall_score(y, y_pred))
print('recall from lightning functional', pl.metrics.functional.recall(y_pred_tensor, y_tensor, num_classes=2))
print('recall from lightning tensor', pl.metrics.Recall(num_classes=2)(y_pred_tensor, y_tensor))

## recall from sklearn 0.8984872611464968
## recall from lightning functional tensor(0.4967)
## recall from lightning tensor tensor(0.4967)

#F1 appears to be off, compared to sklearn
print('F1 from sklearn', f1_score(y, y_pred))
print('F1 from lightning functional', pl.metrics.functional.f1_score(y_pred_tensor, y_tensor, num_classes=2))
print('F1 from lightning tensor', pl.metrics.F1(num_classes=2)(y_pred_tensor, y_tensor))

## F1 from sklearn 0.6429283577837915
## F1 from lightning functional tensor(0.4007)
## F1 from lightning tensor tensor(0.4007)


Expected behavior
Precision/Recall/F1 results are expected to be consistent with those from sklearn.
Environment
Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).
You can get the script and run it with:
wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py


PyTorch Version : 1.5.1
OS (e.g., Linux):  MacOS
How you installed PyTorch (conda, pip, source):  Pip
Build command you used (if compiling from source):
Python version:  3.7
CUDA/cuDNN version:   None
GPU models and configuration:    @@None
Any other relevant information:

Additional context