136041 SE Topics in Deep Learning and Natural Language Processing (2020W)

[A] Word vectors:

[A.1] Mikolov et al. "Distributed representations of words and phrases and their compositionality."

[A.2] Pennington et al. "Glove: Global vectors for word representation."

[A.3] Bojanowski et al. "Enriching word vectors with subword information."

[B] Classical Neural Network Layers:

[B.1] Goodfellow et al., Long short-term memory networks, https://www.deeplearningbook.org, Chapter 10 (skip 10.8, 10.9)

[B.2] Collobert et al., "Natural language processing (almost) from scratch."

[C] Attention:

[C.1] Hermann et al. "Teaching machines to read and comprehend."

[C.2] Bahdanau et al. "Neural machine translation by jointly learning to align and translate."

[C.3] Vaswani et al. "Attention is all you need."

[D] Pre-Trained language models:

[D.1] Devlin et al. "Bert: Pre-training of deep bidirectional transformers for language understanding."

[D.2] Yang et al. "Xlnet: Generalized autoregressive pretraining for language understanding."

[D.3] Peters et al. "Knowledge enhanced contextual word representations."

[E] Data Sets and Evaluation:

[E.1] Kwiatkowski et al. "Natural questions: a benchmark for question answering research."

[E.2] Lichtarge et al. "Corpora generation for grammatical error correction."

[E.3] Geva et al. "DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion"

[E.4] Wang et al. "Glue: A multi-task benchmark and analysis platform for natural language understanding."

[F] Probing of pre-trained language models, explainability:

[F.1] Jawahar et al. "What does BERT learn about the structure of language?"

[F.2] Petroni et al. "Language models as knowledge bases?"

[F.3] Tenney et al. "BERT Rediscovers the Classical NLP Pipeline"

[F.4] Ribeiro et al. ""Why should I trust you?" Explaining the predictions of any classifier."

[F.5] Ribeiro et al. "Beyond Accuracy: Behavioral Testing of NLP Models with CheckList."

[F.6] Bender, Koller: "Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data"

[G] Language bias and learned representations:

[G.1] Bolukbasi et al. "Man is to computer programmer as woman is to homemaker? debiasing word embeddings."

[G.2] Garg et al. "Word embeddings quantify 100 years of gender and ethnic stereotypes."

[G.3] Sap et al. "Social bias frames: Reasoning about social and power implications of language."

[G.4] Zhao et al. "Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods"

[H] Relation Extraction and Distant supervision:

[H.1] Mintz et al. "Distant supervision for relation extraction without labeled data."

[H.2] Lin et al. "Neural relation extraction with selective attention over instances."

[H.3] Keith et al. "Identifying civilians killed by police with distantly supervised entity-event extraction."

[H.4] Riedel et al. "Relation extraction with matrix factorization and universal schemas."

[H.5] Verga et al. "Multilingual relation extraction using compositional universal schema."