Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages
Tomasz Limisiewicz, Jivr'i Balhar, David Marevcek
Annual Meeting of the Association for Computational Linguistics