testing gets stuck when num_workers is set to value >0 in tests/base/model_utilities.py
🐛 Bug
While executing bash .run_local_tests.sh the test hangs frequently (but not always) if parallel data loading is enabled in tests/base/model_utilities.py by setting num_workers to a value larger than 0. If an manual keyboard interrupt (CTRL-c) is done the test continues with a "PASSED" message. This is an issue that does not always occur which could be an indicator that the test-frame work is waiting for the process to finish, but does not realize that the process has been completed.
The same issue occurs during CI testing with the result that time is exceeded.
The issue does not happen for num_workers=0 in tests/base/model_utilities.py; which is the reason for the change in PR #2307 to set this value to zero. This is an unsatisfying solution since it excludes the case of using multiple workers in dataloaders from testing, and risk that someone might inadvertently set the value to a different.
It is unclear of this is caused by an issue with Pythorch-Lightning, Pytorch, or the testing framework.
This issues serves to inform others what to expect when num_workers is set to a value >0, and that this needs to be investigated. It might need someone wth good understanding of the pytest framework and concurrency issues in that framework.

To Reproduce
Steps to reproduce the behavior:

set num_workers=3 in  tests/base/model_utilities.py
Run bash .run_local_tests.sh or pytest -v -x tests/trainer/test_dataloaders.py (might need to run several times)
The pytest -v -x tests/trainer/test_dataloaders.py should finish successfully in 10-15 seconds but often does not
If test gets stuck doing a CRTL-c continues the testing and ends them successfully


Code sample

Expected behavior

Environment
This issue also happens in the CI environments.
My environment:
/Users/thomas.schaaf/virtualenv/Python37/pytorch-lightning/lib/python3.7/site-packages/graphql/type/directives.py:55: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
assert isinstance(locations, collections.Iterable), 'Must provide locations for directive.'

CUDA:

GPU:
available:         False
version:           None


Packages:

numpy:             1.18.5
pyTorch_debug:     False
pyTorch_version:   1.5.1
pytorch-lightning: 0.8.2-dev
tensorboard:       2.2.2
tqdm:              4.46.1


System:

OS:                Darwin
architecture:

64bit



processor:         i386
python:            3.7.7
version:           Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64



Additional context

This issue came up during PR #2307 and was discussed with @Borda and @awaelchli.
My pytest version:
`pytest -V
This is pytest version 5.4.3, imported from /Users/thomas.schaaf/virtualenv/Python37/pytorch-lightning/lib/python3.7/site-packages/pytest/init.py
setuptools registered plugins:
pytest-flake8-1.0.6 at /Users/thomas.schaaf/virtualenv/Python37/pytorch-lightning/lib/python3.7/site-packages/pytest_flake8.py
pytest-cov-2.10.0 at /Users/thomas.schaaf/virtualenv/Python37/pytorch-lightning/lib/python3.7/site-packages/pytest_cov/plugin.py`