Adding input sanitation for distributed backend and related trainer flags
🚀 Feature
Error should be thrown or a warning raised if the passed distributed_backend flag doesn't match one of the expected types (e.g ddp, ddp-spawn, etc). This maybe applicable to other trainer flags too.
Motivation
This is really minor, but I just spent an embarrassing amount of time trying to figure out why my ddp models were not using GPUs. After stepping through the the pytorch-lightning accelerator selection logic, I eventually realized I had a typo in my distributed_backend flag. This mistake could have easily been caught by input sanitation.
For more detail, when you enter an invalid backend, like dpp instead of ddp, and select gpus = 4, the logs at the top of the program show the 4 available GPUs and lead the user to believe that nothing is wrong. However, the logic of the accelerator selection silently chooses cpu in this case, result in surprising behavior.
Pitch
For various flags, like "distributed_backend" unrecognized values, e.g "dpp" instead of "ddp", pl should raise an error or warn the
user that this isn't recognized / valid value for the flag.