Model checkpoint and restore via GCS on Ai platform
❓ Questions and Help
What is your question?
I am training models on Google's ai platform and every checkpointing or restoration is done via Google Cloud Storage (not using pytorch-lightning for that yet). I noticed that pytorch-lightning uses torch.save, torch.load and io operations to work with saving and loading, which do not handle GCS by default (although I could pass file-like object to torch.save/load). Is it possible to extend the library so that is works with GCS without forking and modifying the code directly? The most important component for me is checkpoint save and restore on GCS. Is it a matter of Trainer class, Callbacks or model hooks?