Learning rate scheduling interval on LambdaLR scheduler cannot be set to "step".
🐛 Bug
Hello. I am trying to use LambdaLR learning rate scheduler with lr updates at every step with my custom function.
However, I have found that even with scheduler={'interval': 'step'}, the update occurs at each epoch, not each training step.
def configure_optimizers(self):
    def func(step: int, max_steps=max_steps):
        return (1 - (step / max_steps)) ** 0.9
    scheduler = optim.lr_scheduler.LambdaLR(lr_lambda=func)
    return {'optimizer': optimizer, 'lr_scheduler': scheduler, 'interval': 'step'}

To Reproduce
Apologies for the small snippet but the project is very complex and I think this is the relevant part.
Expected behavior
Learning rate should decrease at every training step after backpropagation.
Environment
Ubuntu16.04 LTS on Docker. PyTorch Lightning v.1.0.6 and 1.0.8. Pytorch version 1.6.0
Additional context
I think the usage of step-wise iteration could be better documented as to how the arguments are parsed and utilized.