Beyond Text Compression: Evaluating Tokenizers Across Scales
Jonas F. Lotz, Ant'onio V. Lopes, Stephan Peitz, Hendra Setiawan, Leonardo Emili
Annual Meeting of the Association for Computational Linguistics