Weight Initialization blocks learning
🐛 Bug
When trying to perform a custom weight initialization, such as xavier_uniform_ or orthogonal_ from torch.nn.init, the weights are kept fixed and not updated during backpropagation.
To Reproduce
Steps to reproduce the behavior:

Create a simple model, even with just FC layers and ReLU
Initialize the weight with torch.nn.init.xavier_uniform_ or another method
Start Training
Weights are not getting updated


Expected behavior
I'd expect the initialization not to block the training process
Environment
packages:
numpy:               1.17.2
pyTorch_debug:       False
pyTorch_version:     1.3.0
pytorch-lightning:   0.7.3
tensorboard:         2.0.0
tqdm:                4.45.0
system:
OS:                  Linux
architecture:
64bit
processor:           x86_64
python:              3.6.9
version:             #163-Ubuntu SMP Mon Sep 24 13:14:43 UTC 2018