transformers

Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse