AttributeError: 'BoringModel' object has no attribute 'require_backward_grad_sync' when using manual optimization with TPU
🐛 Bug
Hello!
When using manual optimization with TPU, I am getting an AttributeError: 'BoringModel' object has no attribute 'require_backward_grad_sync'. When I replace self.manual_backward(loss) with loss.backward() things seem to work, but I am not sure if this is a safe or sustainable workaround. It seems the error happens at the self.manual_backward(loss) step in training_step. Any help would be much appreciated.
Please reproduce using the BoringModel
Here is the notebook reproducing the error:
https://colab.research.google.com/drive/1LPYgtUAiHd1OXuTK6I1WkRaCUQScxEPg?usp=sharing
Environment
WARNING:root:TPU has started up successfully with version pytorch-1.8

CUDA:

GPU:
available:         False
version:           10.1


Packages:

numpy:             1.19.5
pyTorch_debug:     False
pyTorch_version:   1.8.0+cu101
pytorch-lightning: 1.3.0dev
tqdm:              4.41.1


System:

OS:                Linux
architecture:

64bit



processor:         x86_64
python:            3.7.10
version:           #1 SMP Thu Jul 23 08:00:38 PDT 2020



Installed torch-xla using:
!pip install cloud-tpu-client==0.10 https://storage.googleapis.com/tpu-pytorch/wheels/torch_xla-1.8-cp37-cp37m-linux_x86_64.whl
to match colab defaults