Add german common crawl dataset
Hi @Phil1108 !
Have you had a chance to take a look at my suggestions ?
Feel free to ping me if you have questions or if you're ready for a review

Thanks again for adding this dataset, this one is very useful ! > 
> 
> Hi @Phil1108 !
> Have you had a chance to take a look at my suggestions ?
> Feel free to ping me if you have questions or if you're ready for a review
> 
> Thanks again for adding this dataset, this one is very useful !

Hello @lhoestq , 

okay so should we go on with only the 2 parts or wait until the hosting of all 200 GB is cleared at HF? 
If you want to merge it with the 2 parts only at first und upgrade later to all, sure I'll include the minor changes in the next few days and update it here I think we can wait a bit to find a solution for the hosting and update the PR to include all the files when it's done.
In the meantime you can update the PR to take into account the suggestions regarding the dataset card, the parallel downloads and the use of eval.

A safer alternative to eval is `ast.literal_eval`, it should do the job Thanks for your contribution, @Phil1108. Are you still interested in adding this dataset?

We are removing the dataset scripts from this GitHub repo and moving them to the Hugging Face Hub: https://huggingface.co/datasets

We would suggest you create this dataset there. Please, feel free to tell us if you need some help.
Adding a subpart of the Common Crawl which was extracted with this repo https://github.com/facebookresearch/cc_net and additionally filtered for duplicates 