`Trainer.predict` stops gradients globally until `torch.set_grad_enabled(True)` is called
🐛 Bug
After Trainer.predict is called, gradients are never turned back on. This can cryptically impact tests (see #6595).
See error in notebook: https://colab.research.google.com/drive/1vbKcVwApZEcWX_ryyx-2BJXdLD_nkFBG?usp=sharing