on_epoch_* callbacks should only be run for 1 process
Bug Description:
Anything done in on_epoch_* (like on_epoch_end or on_epoch_start) callbacks happens n number of times when doing distributed training i.e. ddp . Process rank check should be added.
To Reproduce
Use ddp for training and create the on_epoch_end callback method with a print statement like: print(self.current_epoch)
Expected behavior
Each epoch is supposed to printed only once.