Analyzing and interpreting neural networks for NLP

Summary of the program

Time	Program item
09:00-09:10	Opening remarks
09:10-10:00	Invited talk 1: Arianna Bisazza
10:00-11:15	Poster session 1 (10:30-11 tea break)
11:15-12:30	Oral presentation session 1 (5 x 15 minutes)
12:30-14:00	Lunch
14:00-14:50	Invited talk 2: Michael F. Bonner
14:50-16:00	Poster session 2 (15:30-16 tea break)
16:00-16:45	Oral presentation session 2 (3 x 15 minutes)
16:45-17:30	Panel discussion (Marco Baroni, Arianna Bisazza, Michael F. Bonner, Ellie Pavlick, Naomi Saphra, Ethan Wilcox)
17:20-17:30	Best paper announcement and closing remarks

11:15-11:30 Character Eyes: Seeing Language through Character-Level Taggers. Yuval Pinter, Marc Marone and Jacob Eisenstein.
11:30-11:45 Faithful Multimodal Explanation for Visual Question Answering. Jialin Wu and Raymond Mooney.
11:45-12:00 Evaluating Recurrent Neural Network Explanations. Leila Arras, Ahmed Osman, Klaus-Robert Müller and Wojciech Samek.
12:00-12:15 On the Realization of Compositionality in Neural Networks. Joris Baan, Jana Leible, Mitja Nikolaus, David Rau, Dennis Ulmer, Tim Baumgärtner, Dieuwke Hupkes and Elia Bruni.
12:15-12:30 Learning the Dyck Language with Attention-based Seq2Seq Models. Xiang Yu, Ngoc Thang Vu and Jonas Kuhn.

16:00-16:15 GEval: Tool for Debugging NLP Datasets and Models. Filip Graliński, Anna Wróblewska, Tomasz Stanisławek, Kamil Grabowski and Tomasz Górecki
16:15-16:30 From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self- Attentions. David Mareček and Rudolf Rosa.
16:30-16:45 What does BERT look at? An Analysis of BERT’s Attention. Kevin Clark, Urvashi Khandelwal, Omer Levy and Christopher D. Manning.

Archival papers:

Transcoding compositionally: using attention to find more generalizable solutions. Kris Korrel, Dieuwke Hupkes, Verna Dankers and Elia Bruni
Sentiment analysis is not solved! Assessing and probing sentiment classification. Jeremy Barnes, Lilja Øvrelid and Erik Velldal
Second-order Co-occurrence Sensitivity of Skip-Gram with Negative Sampling. Dominik Schlechtweg, Cennet Oguz and Sabine Schulte im Walde
Can neural networks understand monotonicity reasoning?. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, Kentaro Inui, Satoshi Sekine, Lasha Abzianidze and Johan Bos
Multi-Granular Text Encoding for Self-Explaining Categorization. Zhiguo Wang, Yue Zhang, Mo Yu, Wei Zhang, Lin Pan, Linfeng Song, Kun Xu and Yousef El-Kurdi
The meaning of “most” for visual question answering models. Alexander Kuhnle and Ann Copestake
Do Human Rationales Improve Machine Explanations?. Julia Strout, Ye Zhang and Raymond Mooney
Analyzing the Structure of Attention in a Transformer Language Model. Jesse Vig and Yonatan Belinkov
Detecting Political Bias in News Articles Using Headline Attention. Rama Rohit Reddy Gangula, Suma Reddy Duggenpudi and Radhika Mamidi
Testing the Generalization Power of Neural Network Models Across NLI Benchmarks. Aarne Talman and Stergios Chatzikyriakidis

Extended abstracts:

(Un)natural word-order biases in deep agent architectures. Rahma Chaabouni, Evgeny Kharitonov, Emmanuel Dupoux and Marco Baroni
Why does a CNN predict this class? Interpreting Convolutional Neural Networks for Text Classification. Piyawat Lertvittayakumjorn and Francesca Toni
State-Regularized Recurrent Neural Networks. Cheng Wang and Mathias Niepert
Additional evidences that BERT learn syntactic structures. Ganesh Jawahar, Benoît Sagot and Djamé Seddah
Neural Networks as Explicit Word-Based Rules. Jindřich Libovický
Neural Pathways: A Method of Abstraction for Deep Neural Model Inspection and Comparison. James Fiacco, Samridhi Choudhary and Carolyn Rose
Can we Explain Natural Language Inference Decisions taken with Neural Networks? Inference Rules in Distributed Representations. Fabio Massimo Zanzotto and Lorenzo Ferrone
Visualizing Deep Neural Networks for Speech Recognition with Learned Topographic Filter Maps. Andreas Krug and Sebastian Stober

Archival papers:

Modeling Paths for Explainable Knowledge Base Completion. Josua Stadelmaier and Sebastian Padó
Probing word and sentence embeddings for long-distance dependencies effects in French and English. Paola Merlo
Derivational Morphological Relations in Word Embeddings. Tomáš Musil, Jonáš Vidra and David MarecˇekMareček
Hierarchical Representation in Neural Language Models: Suppression and Recovery of Expectations. Ethan Wilcox, Roger Levy and Richard Futrell
Blackbox meets blackbox: Representational Similarity & Stability Analysis of Neural Language Models and Brains. Samira Abnar, Lisa Beinborn, Rochelle Choenni and Willem Zuidema
An LSTM adaptation study of (un)grammaticality. Shammur Absar Chowdhury and Roberto Zamparelli
An Analysis of Source-Side Grammatical Errors in NMT. Antonios Anastasopoulos
Finding Hierarchical Structure in Neural Stacks Using Unsupervised Parsing. William Merrill, Lenny Khazan, Noah Amsel, Yiding Hao, Simon Mendelsohn and Robert Frank
Adversarial Attack on Sentiment Classification. Yi-Ting Tsai, Min-Chu Yang and Han-Yu Cheni
Open Sesame: Getting Inside BERT’s Linguistic Knowledge. Yongjie Lin, Yi Chern Tan and Robert Frank

Extended abstracts:

Investigating sub-word embedding strategies in the morphologically rich and free phrase-order Hungarian. Bálint Döbrössy, Márton Makrai, Balázs Tarján and György Szaszák
Inducing syntactic trees from BERT representation. Rudolf Rosa and David Mareček
Exploring Universal Sentence Encoders’ ability to learn cause-and-effects reasoning using a new precedence classification probing task. Yochay Gurman and Reut Tsarfaty
Interactive White-Box Models through Collaborative Semantic Inference. Sebastian Gehrmann, Hendrik Strobelt, Robert Krüger, Hanspeter Pfister and Alexander Rush
Do LSTMs Learn Long-Distance Dependencies From Constituents?. Naomi Saphra and Adam Lopez
Learning to make accuracy judgments from intermediate and final outputs of a neural network for a question answering task. Chad DeChant, Seungwook Han and Hod Lipson
Some linguistic correlates of gradients and attention weights in BERT. Matthijs Westera
Towards Understanding Position Embeddings. Rishi Bommasani and Claire Cardie
On the Importance of Delexicalization for Fact Verification. Sandeep Suntwal, Mithun Paul, Rebecca Sharp and Mihai Surdeanu