Problem with truncated_bptt_steps
🐛 Bug
When setting truncated_bptt_steps we can observed 3  bugs:
1. An exception is raised when num_sanity_val_steps=2:
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/core/lightning.py in get_progress_bar_dict(self)
   1355 
   1356         if self.trainer.truncated_bptt_steps is not None:
-> 1357             tqdm_dict["split_idx"] = self.trainer.split_idx
   1358 
   1359         if self.trainer.logger is not None and self.trainer.logger.version is not None:

AttributeError: 'Trainer' object has no attribute 'split_idx'

2. tbptt_split_batch method does not split.
To observed this behavior we need to turn off num_sanity_val_steps (num_sanity_val_steps=0) to ensure that the first error does not occur. Next we need to implement costume tbptt_split_batch method.
def tbptt_split_batch(self, batch, split_size):
    x, y = batch
    print("x", x.shape, "y", y.shape)
    print("split_size", split_size)
    print("seq_len", x.shape[1])
    splits = super().tbptt_split_batch(batch, batch_size)
    print("pl spliter:", len(splits), [s[0].shape for s in splits])

    splits = []
    for t in range(0, x.shape[1], split_size):
        batch_split = []
        for i, x in enumerate(batch):
            split_x = x[:, t: t + split_size]
            batch_split.append(split_x)
        splits.append(batch_split)
    
    print("my spliter:", len(splits), [s[0].shape for s in splits])

    return splits
output:
x torch.Size([32, 20, 100]) y torch.Size([32, 20])
split_size 10
seq_len 20
pl spliter: 1 [torch.Size([32, 20, 100])]
my spliter: 2 [torch.Size([32, 10, 100]), torch.Size([32, 10, 100])]

Each element of the batch should have structure (batch, time, ...) according to the documentation.
3. An exception is raised when training model with fixed tbptt_split_batch method.
/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
    125     Variable._execution_engine.run_backward(
    126         tensors, grad_tensors, retain_graph, create_graph,
--> 127         allow_unreachable=True)  # allow_unreachable flag
    128 
    129 

RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

4. Question about detaching hidden values.
Why do we have to detach hidden values if back propagation is computed at the end of the batch, after model used all split bptt mini batches?
From the documentation:
# Truncated back-propagation through time
def training_step(self, batch, batch_idx, hiddens):
    # hiddens are the hiddens from the previous truncated backprop step
    out, hiddens = self.lstm(data, hiddens)

    return {
        "loss": ...,
        "hiddens": hiddens  # remember to detach() this
    }
Please reproduce using the BoringModel and post here
https://colab.research.google.com/drive/1rvNzw9V4L_jyLG-cA_83jHUbm5YMlahr?usp=sharing
Expected behavior

Code should not crash
tbptt_split_batch method should work correctly.
Code should not crash
A request to explain why we need to detach hidden values.

Environment

CUDA:

GPU:

Tesla T4


available:         True
version:           10.1


Packages:

numpy:             1.18.5
pyTorch_debug:     False
pyTorch_version:   1.6.0+cu101
pytorch-lightning: 1.0.3
tqdm:              4.41.1


System:

OS:                Linux
architecture:

64bit



processor:         x86_64
python:            3.6.9
version:           #1 SMP Thu Jul 23 08:00:38 PDT 2020