Unable to launch multiple gpus nodes
🐛 Bug
I'm having trouble launching multiple GPU nodes with pytorch-lightning-0.8.5-dev. I'm getting the following error
Traceback (most recent call last):
  File "/home/jmorton/miniconda3/envs/alignment/bin/deepblast-train", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/home/jmorton/research/gert/deepblast/scripts/deepblast-train", line 67, in <module>
    main(hparams)
  File "/home/jmorton/research/gert/deepblast/scripts/deepblast-train", line 47, in main
    trainer.fit(model)
  File "/home/jmorton/miniconda3/envs/alignment/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 964, in fit
    self.set_random_port()
  File "/home/jmorton/miniconda3/envs/alignment/lib/python3.8/site-packages/pytorch_lightning/trainer/distrib_data_parallel.py", line 392, in set_random_port
    assert self.num_nodes == 1, 'random port can only be called from single node training'
AssertionError: random port can only be called from single node training


To Reproduce
I've setup my model similar to as follows
    parser = LightningAligner.add_model_specific_args(parser)
    args = parser.parse_args()
    model = LightningAligner(args)

    trainer = Trainer(
        max_epochs=10,
        gpus=4,
        num_nodes=2,
        accumulate_grad_batches=10,
        distributed_backend='ddp',
        precision=32,
        check_val_every_n_epoch=1,
        fast_dev_run=False
    )
Environment
Output of python collect_env_details.py
* CUDA:
        - GPU:
                - Tesla V100-PCIE-32GB
                - Tesla V100-PCIE-32GB
                - Tesla V100-PCIE-32GB
                - Tesla V100-PCIE-32GB
        - available:         True
        - version:           10.1
* Packages:
	- numpy:             1.17.5
	- pyTorch_debug:     False
	- pyTorch_version:   1.5.1
	- pytorch-lightning: 0.8.5-dev
	- tensorboard:       2.2.2
	- tqdm:              4.47.0
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                - ELF
        - processor:         x86_64
        - python:            3.8.1
        - version:           #1 SMP Wed Jun 3 14:28:03 UTC 2020


Additional context
Only 4 out of 8 GPUs are recognized.

I'm curious why the assert statement is there.