better checking of data returned from training_step
🚀 Feature
let's add more validation checks on what's returned from training_step and provide the user with useful error messages when they're not returning the right values.
Motivation
i feel like i've seen a lot of users confused about what they're supposed to return in training_step and validation_step. additionally, i don't think we document how we treat extra keys as "callback metrics" very well.
Pitch
what you do you think about adding some structure and validation for Trainer's process_output method?
right now, we have expectations about a set of keys {progress_bar, log, loss, hiddens} and assume everything else is a callback metric. however, this is a silent assumption.
we could instead enforce a more rigid structure:
{
  'loss': loss                   # REQUIRED
  'log': {}                         # optional dict
  'progress_bar': {}       # optional dict
  'hiddens': [h0, c0]     # optional collection of tensors
  'metrics': {}                 # optional dict
}

moreover, we can leverage pydantic to do validation automatically and provide useful error message out of the box when data validation fails.
cc @PyTorchLightning/core-contributors
Alternatives
Do nothing, keep things as they are.
Additional context
This would be a backwards incompatible change.