Set precision=16 (using apex) would cause early stopping break
🐛 Bug
The current early stopping monitor initilize by comparing if the function monitor_op is equal to torch.lt.
self.best = torch_inf if self.monitor_op == torch.lt else -torch_inf

  
    
      pytorch-lightning/pytorch_lightning/callbacks/early_stopping.py
    
    
         Line 110
      in
      12138ce
    
  
  
    

        
          
           self.best = torch_inf if self.monitor_op == torch.lt else -torch_inf 
        
    
  


However when intializing with the apex, it seems that the torch.lt would change and this evaluation would be always false and thus the self.best is intialized to -inf instead of +inf.
To Reproduce
import torch
from pytorch_lightning.callbacks.early_stopping import EarlyStopping
import apex.amp as amp

es = EarlyStopping()
es.monitor_op == torch.lt
Out[6]: True

model = torch.Linear(5, 5).to('cuda')
optimizers = torch.optim.Adam(model.parameters(), lr=1e-3)
amp.initialize(model, optimizers)

es.monitor_op == torch.lt
Out[22]: False

And this bug leads to the initialization of self.best to be -inf instead of inf
Expected behavior
self.best should be initialized to inf instead of -inf.
Environment

CUDA:
- GPU:
- TITAN Xp
- Quadro P400
- available:         True
- version:           10.1
Packages:
- numpy:             1.18.1
- pyTorch_debug:     False
- pyTorch_version:   1.4.0
- pytorch-lightning: 0.7.5
- tensorboard:       2.1.1
- tqdm:              4.43.0
System:
- OS:                Linux
- architecture:
- 64bit
-
- processor:         x86_64
- python:            3.6.10
- version:           #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019

Additional context
I bump into this bug after downloading from the master branch couple days ago. I would guess the old version is fine but did not test it.