Packages fails to import when system site-packages contain a different version
🐛 Bug
The package mixes imports from the user's site-packages and system's site-packages, which leads to import errors or other unexpected behavior.
To Reproduce

Install older version of pytorch-lightning as system package, e.g.:

sudo pip3 install pytorch-lightning==0.8.0


Install newer version of pytorch-lightning as user package, e.g.:

pip3 install pytorch-lightning==1.0.4 --user


Try to import the package:

Python 3.8.2 (default, Feb 26 2020, 02:56:10)
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pytorch_lightning
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mybot/.local/lib/python3.8/site-packages/pytorch_lightning/__init__.py", line 79, in <module>
    __import__('pkg_resources').declare_namespace(__name__)
  File "/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py", line 2293, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py", line 2226, in _handle_ns
    loader.load_module(packageName)
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/__init__.py", line 53, in <module>
    from pytorch_lightning.core import LightningModule, data_loader
ImportError: cannot import name 'data_loader' from 'pytorch_lightning.core' (/home/mybot/.local/lib/python3.8/site-packages/pytorch_lightning/core/__init__.py)

Note that python prefers the user packages over system packages so the first import use a file in /home/mybot/.local/ while the others start with /usr/local/lib/ (system packages).
Lines from the relevant file /home/mybot/.local/lib/python3.8/site-packages/pytorch_lightning/core/__init__.py:
# for compatibility with namespace packages
__import__('pkg_resources').declare_namespace(__name__)

Expected behavior
The import use only one version of the package.
Environment
Standard Ubuntu 18.04 docker image with CUDA 10.
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.3 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Pip version:
pip 20.2.4 from /home/tclbot/.local/lib/python3.8/site-packages/pip (python 3.8)

Pytorch: 1.6.0
Additional context
This scenario occurs in practice. My docker image contains preinstalled packages from requirements.txt installed as system packages. Additionally at the start of the pod I defensively run pip install -r requirements.txt, just in case my docker image was outdated. I use system packages as a sort of cache.
In my case the workaround is quite simple, but this bug can lead to other, much more confusing issues.