Add support for training on IPU's
🚀 Feature
Graphcore IPU's are a new breed of processors for training machine learning models.
Pytorch support for training on Graphcore IPU's is available in Preview. Add support for training on IPU's.
Reference: https://docs.graphcore.ai/projects/poptorch-user-guide/en/latest/
Motivation
The IPU benchmarks seem pretty impressive, and with azure supporting them, they might soon grow in popularity. With Lightning's idea of the running the same code run on CPU, GPU, and TPU, it seems natural that it should also support IPU's.
Pitch
Add support for ipus in trainer just as gpus and tpu_cores.
Additional context
Here are the benchmarks: https://www.graphcore.ai/benchmarks
IPU's are available on Azure cloud: https://www.hpcwire.com/2019/11/15/microsoft-azure-adds-graphcores-ipu