calling Trainer.test(model) will perform test twice
🐛 Bug
When calling Trainer.test(model) after training separately, the Test routine will be called twice.
To Reproduce
Steps to reproduce the behavior:
model = my_lightning_model(some_hparams)
trainer = Trainer()
trainer.fit(model)
trainer.test(model)     #the test routine will be performed twice


Expected behavior
The test should be only performed once via trainer.test(model).

Environment
PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.1
OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.10.2
Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.1.243
GPU models and configuration:
GPU 0: GeForce GTX 1080 Ti
GPU 1: GeForce GTX 1080 Ti
Nvidia driver version: 430.14
cuDNN version: Could not collect
Versions of relevant libraries:
[pip3] numpy==1.18.1
[pip3] pytorch-lightning==0.6.1.dev0
[pip3] torch==1.4.0
[pip3] torchvision==0.5.0
[conda] Could not collect
Potential reason causing this bug
By dive into pl's source code, I found the reason might be related to pytorch_lightning/trainer/trainer.py at line 1104 & line 1177:
line 1104:
        # when testing requested only run test and return
        if self.testing:
            # only load test dataloader for testing
            self.reset_test_dataloader(ref_model)
            self.run_evaluation(test_mode=True)
            return

line 1177:
        self.testing = True
        if model is not None:
            self.fit(model)
        self.run_evaluation(test_mode=True)

When calling trainer.test(model) at line 1177, self.testing will be set to True; since model is not None, self.fit(model) will be called; then in line 1104 at fit(), self.testing is True, so the self.run_evaluation will be called first time. After that, the self.run_evaluation in test() (as in line 1180) will be called again, thus result in twice test evulation.
Potential fix
change the code snippet in test() to
        self.testing = True
        if model is not None:
            #as self.testing = True, self.fit(model) will only perform test
            self.fit(model)   
        else: 
             #if model is not None, then the test has been already performed in fit()
             self.run_evaluation(test_mode=True)

might fix the problem?
P.S. if possible, I'd be happy to submit a PR on this :)