conflicts of warm-up and lr scheduler
In docs, warm-up is done as below:
# learning rate warm-up
def optimizer_step(self, current_epoch, batch_nb, optimizer, optimizer_idx, second_order_closure=None, on_tpu=False, using_native_amp=False, using_lbfgs=False):
    # warm up lr
    if self.trainer.global_step < 500:
        lr_scale = min(1., float(self.trainer.global_step + 1) / 500.)
        for pg in optimizer.param_groups:
            pg['lr'] = lr_scale * self.hparams.learning_rate

    # update params
    optimizer.step()
    optimizer.zero_grad()

This will change lr in [0, 500] steps, but lr_scheduler will also change lr in [0, 500] steps,  this warm-up method will override lr_scheduler's change.  How can I use warm-up in [0, 500] steps, then starts lr_scheduler in [500, ...] steps.  Or I use warm-up in [0, 3] epochs, then starts lr_scheduler in [3, ...] epochs