Transformer Language Models without Positional Encodings Still Learn Positional Information
Adi Haviv, Ori Ram, Ofir Press, Peter Izsak, Omer Levy
Conference on Empirical Methods in Natural Language Processing