Improve Wikipedia loading script
Thanks! See https://public.paws.wmcloud.org/User:Isaac_(WMF)/HuggingFace%20Wikipedia%20Processing.ipynb for more implementation details / some data around the overhead induced by adding the extra preprocessing steps (stripping link prefixes and magic words) Closed by:
- #3435
As reported by @geohci, the "wikipedia" processing/loading script could be improved by some additional small suggested processing functions:
- _extract_content(filepath):
  - Replace .startswith("#redirect") with more structured approach: if elem.find(f"./{namespace}redirect") is None: continue
- _parse_and_clean_wikicode(raw_content, parser):
  - Remove rm_template from cleaning -- this is redundant with .strip_code() from mwparserformhell
  - Build a language-specific list of namespace prefixes to filter out per below get_namespace_prefixes
  - Optional: strip prefixes like categories -- e.g., Category:Towns in Tianjin becomes Towns in Tianjin
  - Optional: strip magic words



