Allow the scheduler to be None
🚀 Feature
Allow a scheduler to be None. This means that one could have configure_optimizers() return [optimizer], [None].
Motivation
Allowing configure_optimizers() to return [optimizer], [None] improves how one could write clean, dynamic code. One could for instance do something like this:
def configure_optimizers(self):
    self.optim = create_optimizer(self.model, **self.optim_opts)
    self.lr_scheduler = create_lr_scheduler(self.optim, **self.sched_opts)
    return [self.optim], [self.lr_scheduler]
where create_lr_scheduler() can return None when no scheduler is needed.
Alternatives
I am aware that one could rewrite as follows, but this is a lot less clean and can easily be changed in the source code.
def configure_optimizers(self):
    self.optim = create_optimizer(self.model, **self.optim_opts)
    self.lr_scheduler = create_lr_scheduler(self.optim, **self.sched_opts)

    if self.lr_scheduler is None:
        return self.optim
    else:
        return [self.optim], [self.lr_scheduler]
Additional context
I can do a PR for this. It should be as easy as adding a condition in configure_schedulers()

  
    
      pytorch-lightning/pytorch_lightning/trainer/optimizers.py
    
    
         Line 83
      in
      e79ae18
    
  
  
    

        
          
           def configure_schedulers(self, schedulers: list): 
        
    
  


namely, allowing:
elif scheduler is None:
     continue