In Multi GPU DDP, pytorch-lightning creates several tfevents files
Describe the bug
Right now pytorch-lightning seems to create several tfevent files in the multi-gpu ddp way:
e.g. for 2 GPUs:
-rw-rw-r--. 1 sam sam   40 Sep 19 08:11 events.out.tfevents.1568880714.google2-compute82.3156.0
-rw-rw-r--. 1 sam sam 165K Sep 19 08:22 events.out.tfevents.1568880716.google2-compute82.3186.0
-rw-rw-r--. 1 sam sam   40 Sep 19 08:11 events.out.tfevents.1568880718.google2-compute82.3199.0

I suppose the first one is created by the main process and the next 2 are created by the 2 DDP processes (one per GPU). Unfortunately, the actual events are not logged in the last created one, and that confuses tensorboard, cf tensorflow/tensorboard#1011
I have to restart tensorboard if I want to see the new data.
A clear and concise description of what the bug is.
To Reproduce
Launch any training on multi GPU DDP.
Expected behavior
Only one tfevent file is created, from the master GPU.