automatically copy state-dict when using ddp
🚀 Feature
Copy model state-dict from rank 0 process to other processes when using ddp
Motivation
This would mean that the user does not need to worry about initializing models with the same weights
Alternatives
Alternatively lightning could at least check if the weights are the same and if not warn the user / thrown an exception
Not sure if this is possible and how easy it can be accomplished, but I could imagine that this could be a source for errors.