Support parallelized downloading in load_dataset with Spark
Hi @lhoestq or other maintainers, this is ready for review, could you please take a look? The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5807). All of your documentation changes will be reflected on that endpoint. Per the discussion in #5798, will implement with `joblibspark` instead.
As proposed in https://github.com/huggingface/datasets/issues/5798, this adds support to parallelized downloading in `load_dataset` with Spark, which can speed up the process by distributing the workload to worker nodes.

Parallelizing dataset processing is not supported in this PR.