Schedulers step before optimizers
According to pytorch docs,

Learning rate scheduling should be applied after optimizer’s update

Which is also apparent in the warning
UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule.See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate

And since pytorch>1.1.0 is required, we should either call the scheduler step after __run_tng_batch() is called in __train() or in __run_tng_batch() itself