Load callback states while testing.
🚀 Feature
Load callback states while testing.
Motivation
#5161 (comment)
Pitch
Two possible API changes:
with an additional argument restore_states:
test(ckpt_path, restore_states=True/False)  # give an option whether to load states or not
test(model, ckpt_path, restore_states=True/False)  # same as above but will just load checkpoint states and not the model

# raise an error
test(ckpt_path=None, restore_states=True)
or without any additional argument:
test(ckpt_path)  # always load states
test(ckpt_path=None)  # don't load any states.
test(model, ckpt_path)  # reload checkpoint states only from ckpt_path
Alternatives
Alternatively, one can just reload checkpoints manually, call on_load_checkpoint for all the callbacks manually, and test.
PS: There may be a better solution. Open to suggestions :)
cc: @ananthsub
cc @Borda @awaelchli @ananthsub @ninginthecloud @rohitgr7 @tchaton @akihironitta