Loss divided by `accumulate_grad_batches` number
🐛 Bug
After the 1.1.4 with the fix 5417, logging was fixed but my loss was divided by accumulate_grad_batches.
Please reproduce using the BoringModel
Sorry, there is no BoringModel. I paste my code here
To Reproduce
    def training_step(self, batch, batch_idx, optimizer_idx):
        with autocast():
            outs = self.model(**batch)
            gen_loss = outs['gen_loss']
            dis_loss = outs['dis_loss']

        (gen_opt, dis_opt) = self.optimizers()

        # Manual backward Generator loss
        self.manual_backward(gen_loss, gen_opt)
        gen_opt.step()

        # Manual backward Discriminator loss
        self.manual_backward(dis_loss, dis_opt)
        dis_opt.step()

        # Accumulate losses
        self.loss_accumulation(gen_loss.cpu().item(), dis_loss.cpu().item())

        # Log all losses
        self.log_metrics()

        # Logging total loss to the progress bar
        total_loss = (gen_loss + dis_loss).cpu().item()
        self.log('total_loss', total_loss, prog_bar=True, logger=False, on_step=True, on_epoch=False) # THIS IS THE BUG

I use DDP with manual backward.
If I used my environment w pl=1.1.0 or accumulate_grad_batches=1 (version 1.1.6), the loss is around 11 at first:

If I used accummulate_grad_batches=3, the loss is divided by 3:

Expected behavior
Loss should not be divided.
I guess 1.1.3 and before, train_loop sums all loss then average. Now it divides by accumulate_grad_batches then sum.
Environment

PyTorch Version (e.g., 1.0): 1.7.1
OS (e.g., Linux): Ubuntu 18.04
How you installed PyTorch (conda, pip, source): conda
Build command you used (if compiling from source):
Python version: 3.8
CUDA/cuDNN version: 10.1
GPU models and configuration: Quadro RTX8000
Any other relevant information: