ModelCheckpoint doesn't delete checkpoints from s3 storage using Tensorboard Logger
🐛 Bug
When using ModelCheckpoint with TensorboardLogger with a S3 bucket url path the models checkpoints are correclty uploaded into the cloud directory set by the logger but but past epochs versions are not deleted. If, instead, I use directly the ModelCheckpoint with dirpath=<s3-url> while saving tensorboard logs locally, then the checkpoints are both uploaded and deleted correctly on my s3 bucket.
Expected behavior
ModelCheckpoint should delete past checkpoints on cloud storage also when using the TensorboardLogger
Environment

PyTorch Version: 1.7.1
OS: Ubuntu 20.04
Python version: 3.8