Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned

This is a publication. If there is no link to the publication on this page, you can try the pre-formated search via the search engines listed on this page.

Authors: Voita, Elena; Talbot, David; Moiseev, Fedor; Sennrich, Rico; Titov, Ivan

Journal title: Voita, Elena; Talbot, David; Moiseev, Fedor; Sennrich, Rico; Titov, Ivan (2019). Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, Italy, 28 July 2019 - 2 August 2019, 5797-5808.

Journal number: 3

Journal publisher: Association for Computational Linguistics

Published year: 2019

DOI identifier: 10.5167/uzh-172619