trainer.tune() fails when Trainer.__init__(auto_lr_find=True, auto_scale_batch_size=True)
🐛 Bug
trainer.tune() works just fine when either Trainer.__init__(auto_lr_find=False, auto_scale_batch_size=True) or Trainer.__init__(auto_lr_find=True, auto_scale_batch_size=False)
However, trainer.tune() fails when Trainer.__init__(auto_lr_find=True, auto_scale_batch_size=True)
LR finder stopped early due to diverging loss.
INFO   lr_finder.py:186:lr_find(): LR finder stopped early due to diverging loss.
/home/vin/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/distributed.py:49: UserWarning: You're resuming from a checkpoint that ended mid-epoch. This can cause unreliable results if further training is done, consider using an end of epoch checkpoint. 
  warnings.warn(*args, **kwargs)
Failed to compute suggesting for `lr`. There might not be enough points.
Traceback (most recent call last):
  File "/home/vin/.local/lib/python3.8/site-packages/pytorch_lightning/tuner/lr_finder.py", line 353, in suggestion
    min_grad = np.gradient(loss).argmin()
  File "<__array_function__ internals>", line 5, in gradient
  File "/home/vin/.local/lib/python3.8/site-packages/numpy/lib/function_base.py", line 1052, in gradient
    raise ValueError(
ValueError: Shape of array too small to calculate a numerical gradient, at least (edge_order + 1) elements are required.
ERROR  lr_finder.py:357:suggestion(): Failed to compute suggesting for `lr`. There might not be enough points.

Please reproduce using the BoringModel

To Reproduce
Use following BoringModel and post here

In your own environment trainer.tune() should fail when Trainer.init(auto_lr_find=True, auto_scale_batch_size=True). However, if you want to reproduce the bug from my code then go to Github, and fork from https://github.com/vineetk1/conversational-transaction-bot
Then run the following on commandline:
python3 ctbMain.py input_param_files/distilgpt2_params
Expected behavior
trainer.tune() should find the Batch-Size and the initial Learning-Rate

Environment
Note: Bugs with code are solved faster ! Colab Notebook should be made public !


IDE: Please, use our python bug_report_model.py template.


Colab Notebook: Please copy and paste the output from our environment collection script (or fill out the checklist below manually).


You can get the script and run it with:
wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py


PyTorch Version (e.g., 1.0): 1.6.0
OS (e.g., Linux): Ubuntu-Linux
How you installed PyTorch (conda, pip, source): pip
Build command you used (if compiling from source):
Python version: 3.8.5
CUDA/cuDNN version: 11.0
GPU models and configuration: Nvidia GeForce GTX 1080
Any other relevant information:

Additional context