Checkpoint hparams.yaml does not save current self.hparms, but only at self.save_hyperparameters
🐛 Bug

I expect that lightning_logs/version_0/hparams.yaml matches self.hparams when Checkpoint is called, or at least matches self.hparams when calling trainer.fit(model).
Instead hparams.yaml only contains the arguments given to self.save_hyperparameters() (empty is all arguments).
Please reproduce using the BoringModel and post here

https://colab.research.google.com/drive/1dmZjS_3SETwF3hAAeCq0xdrd0NQGs9nt?usp=sharing
To Reproduce

def __init__(self, output_neurons):
    super().__init__()
    # !!!!! CHANGED !!!!!
    self.save_hyperparameters()
    self.hparams["output_neurons"] = 3
    self.layer = torch.nn.Linear(32, self.hparams["output_neurons"])
...
model = BoringModel(output_neurons=2)

Expected behavior

Inside hparams.yaml the following line:
output_neurons: 3
Actual behavior
Inside hparams.yaml the following line:
output_neurons: 2
Environment
Note: Bugs with code are solved faster ! Colab Notebook should be made public !


IDE: Please, use our python bug_report_model.py template.


Colab Notebook: Please copy and paste the output from our environment collection script (or fill out the checklist below manually).


You can get the script and run it with:
wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py

Colab

CUDA:

GPU:

Tesla T4


available:         True
version:           10.1


Packages:

numpy:             1.18.5
pyTorch_debug:     False
pyTorch_version:   1.6.0+cu101
pytorch-lightning: 1.0.3
tqdm:              4.41.1


System:

OS:                Linux
architecture:

64bit



processor:         x86_64
python:            3.6.9
version:           #1 SMP Thu Jul 23 08:00:38 PDT 2020




Any other relevant information:

Additional context

I listed this as a bug, as it would go against (my) intuition. If this is intended behavior, this should be made clear in the documentation.
How I came across this issue
I trained and model and saved it with to_torchscript(). I wanted to share with my teams the hparams of this new model.pt, but found out that hparams.yaml did not contain the hparam values I used to train the model with. I'm also adding additional information to self.hparams later in the pl.LightningModule, but this was also not included.
Imagine doing transfer learning. You set self.hparams["output_neurons"] = 10 to match the output size when loading the weights of a pre-trained model. Then you change self.hparams["output_neurons"] = 5 and do a self.output_layer = torch.nn.Linear(32, self.hparams["output_neurons"]) to train on your new task.
You would expect hparams.yaml to contain this new output size, but in the current PL version it does not.