Prevent deprecated documentation from showing up in search engine top results
📚 Documentation
Currently when we search keywords like "pytorch lightning trainer" we get results that point to very very very outdated docs!
It should instead point to the latest stable documentation pages.
Investigate these options here:

https://docs.readthedocs.io/en/stable/faq.html#how-can-i-avoid-search-results-having-a-deprecated-version-of-my-docs
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_extra_path

adding a robots.txt is probably enough
cc @alemkhenter