apex amp not working in 1.2.0
🐛 Bug
Got RuntimeError: Invoked 'with amp.scale_loss, but internal Amp state has not been initialized.  model, optimizer = amp.initialize(model, optimizer, opt_level=...) must be called before with amp.scale_loss.`
When training with apex amp
No problem in 1.1.8.
Also got amp.load_state_dict when resuming from checkpoints trained with earlier version.
Please reproduce using the BoringModel
https://colab.research.google.com/drive/1OKQgh1gDUp78HMXX-mmrkxktVC2qV5oE?usp=sharing
To Reproduce
Expected behavior
Environment
Note: Bugs with code are solved faster ! Colab Notebook should be made public !


IDE: Please, use our python bug_report_model.py template.


Colab Notebook: Please copy and paste the output from our environment collection script (or fill out the checklist below manually).


You can get the script and run it with:
wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py


PyTorch Version (e.g., 1.0):
OS (e.g., Linux):
How you installed PyTorch (conda, pip, source):
Build command you used (if compiling from source):
Python version:
CUDA/cuDNN version:
GPU models and configuration:
Any other relevant information:

Additional context