Single-node multi-gpu ddp backend tries to delete model checkpoints from all processes
🐛 Bug

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Run '....'
Scroll down to '....'
See error


Code sample

Expected behavior

Environment
Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).
You can get the script and run it with:
wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py


PyTorch Version (e.g., 1.0):
OS (e.g., Linux):
How you installed PyTorch (conda, pip, source):
Build command you used (if compiling from source):
Python version:
CUDA/cuDNN version:
GPU models and configuration:
Any other relevant information:

Additional context