Horovod and Native Amp not work
##🐛 Bug
I'm no sure if there is a bug. But when I was tryng to use Horovod as backend to do native amp in PyTorch 1.6, Lightning always points to the function that uses apex amp instead.
To Reproduce
Steps to reproduce the behavior:

Go to 'trainer/distrib_parts.py'
Run '....'
Scroll down to '....'
See error



Code sample

trainer = pl.Trainer(gpus=1, num_nodes = 1, distributed_backend='horovod', precision = 16)
trainer.fit(generator, train_dataloader=train_loader, val_dataloaders=val_loader)
I tried to run it like: horovodrun  --verbose -np 32 -H server1-0:8,server2-0:8,server3-0:8,server4-0:8     python file.py
Expected behavior
Use native amp when trying to launch Horovod in PyTorch 1.6.
Environment
Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).
You can get the script and run it with:
wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py

CUDA:
- GPU:
- Tesla V100-SXM2-32GB
- Tesla V100-SXM2-32GB
- Tesla V100-SXM2-32GB
- Tesla V100-SXM2-32GB
- Tesla V100-SXM2-32GB
- Tesla V100-SXM2-32GB
- Tesla V100-SXM2-32GB
- Tesla V100-SXM2-32GB
- available:         True
- version:           10.1

Packages:

numpy:             1.19.1
pyTorch_debug:     False
pyTorch_version:   1.6.0.dev20200623+cu101
pytorch-lightning: 0.8.5
tensorboard:       2.3.0
tqdm:              4.48.0


System:

OS:                Linux
architecture:

64bit



processor:         x86_64
python:            3.6.9
version:           #83~16.04.1-Ubuntu SMP Wed Dec 18 04:56:23 UTC 2019




How you installed PyTorch (conda, pip, source): pip
cuDNN version: 7.6.5

Additional context