Trainer.from_argparse_args with additional kwargs causes model to not be saved
🐛 Bug
When using Trainer.from_argparse_args() to initialize the trainer, there will be some specific arguments that we would like to keep constant and not send as part of hparams. If the extra arguments turn out to be an object, such as a TensorBoardLogger or a ModelCheckpoint object, the model will not be saved because these objects get added to hparams
To Reproduce

Code sample

import os
from argparse import ArgumentParser

import pytorch_lightning as pl
import torch
from pytorch_lightning import Trainer
from pytorch_lightning.loggers import TensorBoardLogger
from torch.nn import functional as F
from torch.utils.data import DataLoader
from torchvision import transforms
from torchvision.datasets import MNIST


class LitModel(pl.LightningModule):
    def __init__(self, hparams):
        super().__init__()
        self.hparams = hparams
        self.l1 = torch.nn.Linear(28 * 28, 10)

    def forward(self, x):
        return torch.relu(self.l1(x.view(x.size(0), -1)))

    def training_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self(x)
        loss = F.cross_entropy(y_hat, y)
        tensorboard_logs = {"train_loss": loss}
        return {"loss": loss, "log": tensorboard_logs}

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=0.001)

    def train_dataloader(self):
        dataset = MNIST(
            os.getcwd(), train=True, download=True, transform=transforms.ToTensor()
        )
        loader = DataLoader(dataset, batch_size=32, num_workers=4, shuffle=True)
        return loader


def main(hparams):
    logger = TensorBoardLogger(save_dir=os.getenv("HOME"), name="logs")
    net = LitModel(hparams)
    trainer = Trainer.from_argparse_args(
        hparams, logger=logger, checkpoint_callback=True, overfit_pct=0.01,
    )
    trainer.fit(net)


if __name__ == "__main__":
    parser = ArgumentParser()
    parser.add_argument("--sample", type=int, default=42, help="Sample Argument")
    hparams = parser.parse_args()
    main(hparams)

Error
TypeError: cannot pickle '_thread.lock' object
Expected behavior
The model to be saved without considering the extra arguments sent as hparams

Environment

CUDA:

GPU:

GeForce GTX 1050 Ti


available:         True
version:           10.1


Packages:

numpy:             1.18.1
pyTorch_debug:     False
pyTorch_version:   1.4.0
pytorch-lightning: 0.7.5
tensorboard:       2.2.1
tqdm:              4.45.0


System:

OS:                Linux
architecture:

64bit
ELF


processor:         x86_64
python:            3.8.2
version:           32-Ubuntu SMP Wed Apr 22 17:40:10 UTC 2020



Additional context
Possible Fix
I believe that a possible fix is to change the from_argparse_args method
from
    @classmethod
    def from_argparse_args(cls, args, **kwargs):

        params = vars(args)
        params.update(**kwargs)

        return cls(**params)

to
    @classmethod
    def from_argparse_args(cls, args, **kwargs):

        params = vars(args)

        return cls(**params, **kwargs)

This ensures that the **kwargs are not added to the hparams of the model and the model gets saved successfully, but I'm not sure of the impact of this change