Possible Bug for multiple optimizers, require_grads=False
🐛 Bug
Hey first of all, I really like your repository (great work).
I am using your framework to train two models at the same time, i.e. I followed the GAN example. This means that I have also 2 optimizers. The problem that I am facing is the error:
line 99, in backward allow_unreachable=True)  # allow_unreachable flag RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
I investigated a little and have a pretty good idea what solves the problem. Namely in the file https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/training_loop.py
on line 610 we loop over our optimizers (I already saw in previous issues that there was some discussion about how to do this). When we then use one optimizer, we set the parameters of the other model to require_grad=False. However if I look at the way this was implemented, assuming we are currently in the second iteration, i.e. we are optimizing with our second optimizer the following happens:
I set all my parameters to requires_grad=False. I then go to my second optimizer and set all the parameters that are being optimized by it to requires_grad=True. I can then optimize these parameters.
However if I then do the next gradient update over my next batch, the parameters of my first model were still set to requires_grad=False.
One can easily look at this with the following code
for param in self.get_model().parameters:
      print(param.requires_grad)

I am not sure, I could be completely wrong, but i feel like after the last optimizer backpropagates, you forget to reset all the parameters to param.requires_grad.  The reason why I could be wrong, is because somehow I guess in your code exmaple with the GAN's it all worked fine (I didn't test that). But in my case it yields this error.
Another way with which I was able to fix it, is by setting my loss to loss.requires_grad(True)
Environment
Mac
torch==1.3.0
pytorch-lightning==0.8.3