Adaptively Sparse Transformers @emnlp2019 +Gonçalo Correia, André Martins α-entmax attention α=1: softmax, α=2: sparsemax, continuous in between. twist: we learn α for each head, w gradients! Some heads become dense, some sparse. arxiv.org/abs/1909.00015 github.com/deep-spin/entmax

Towards Dynamic Computation Graphs via Sparse Latent Structure: #emnlp2018 + André @clairecardie - marginalize over structured latent vars w/ SparseMAP - CG a function of discrete structure - eg latent dependency TreeLSTM pdf arxiv.org/abs/1809.00653 code github.com/vene/sparsemap/tr… – at Lisbon, Portugal

Happy to announce that SparseMAP, our project on differentiable and sparse structured inference, is accepted at ICML 2018! Joint work with Andre Martins, Mathieu Blondel, and Claire Cardie. Stay tuned for the camera-ready update, and dynet+pytorch code! arxiv.org/abs/1802.04223