Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities
Mayank Jobanputra, Yana Veitsman, Yash Sarrof, A. Bakalova, Vera Demberg, Ellie Pavlick, Michael Hahn