`save_last` should only keep the most recent checkpoint (along with the top k)
🚀 Feature
After save_last saves a checkpoint, it removes the previous "last" (i.e. latest) checkpoint (i.e. separate from top k).
Motivation
For example, for someone limited by disk space, a good strategy during training would be to always save the best checkpoint as well as the latest checkpoint to restore from in case training gets interrupted (and ideally with an option to automatically remove the latest checkpoint with training finishes, but that might belong to a separate issue). Setting save_top_k=1 and save_last=True almost achieves this, but the latter actually results in all checkpoints being saved, which is unnecessary for the purpose of restoring training. So I propose save_last to keep only one latest checkpoint at all times. This is separate from the top k checkpoints. If a user really wants to keep all checkpoints around, he/she can always do save_top_k=-1 anyway.
Alternatives
Some other config option that allows always keeping one latest checkpoint.