136041 SE Topics in Deep Learning and Natural Language Processing (2022S)

=== the information below is preliminary, to be finalized soon ===

[A] Word vectors:

[A.1] Mikolov et al. "Distributed representations of words and phrases and their compositionality."
[A.2] Pennington et al. "Glove: Global vectors for word representation."
[A.3] Bojanowski et al. "Enriching word vectors with subword information."

[B] Classical Neural Network Layers:

[B.1] Goodfellow et al., Long short-term memory networks, https://www.deeplearningbook.org, Chapter 10 (skip 10.8, 10.9)
[B.2] Collobert et al., "Natural language processing (almost) from scratch."

[C] Attention:

[C.1] Hermann et al. "Teaching machines to read and comprehend."
[C.2] Bahdanau et al. "Neural machine translation by jointly learning to align and translate."
[C.3] Vaswani et al. "Attention is all you need."

[D] Pre-Trained language models:

[D.1] Devlin et al. "Bert: Pre-training of deep bidirectional transformers for language understanding."
[D.2] Yang et al. "Xlnet: Generalized autoregressive pretraining for language understanding."
[D.3] Peters et al. "Knowledge enhanced contextual word representations."

[E] Data Sets and Evaluation:

[E.1] Kwiatkowski et al. "Natural questions: a benchmark for question answering research."
[E.2] Lichtarge et al. "Corpora generation for grammatical error correction."
[E.3] Geva et al. "DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion"
[E.4] Wang et al. "Glue: A multi-task benchmark and analysis platform for natural language understanding."

[F] Probing of pre-trained language models, explainability:

[F.1] Jawahar et al. "What does BERT learn about the structure of language?"
[F.2] Petroni et al. "Language models as knowledge bases?"
[F.3] Tenney et al. "BERT Rediscovers the Classical NLP Pipeline"
[F.4] Ribeiro et al. ""Why should I trust you?" Explaining the predictions of any classifier."
[F.5] Ribeiro et al. "Beyond Accuracy: Behavioral Testing of NLP Models with CheckList."
[F.6] Bender, Koller: "Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data"

[G] Language bias and learned representations:

[G.1] Bolukbasi et al. "Man is to computer programmer as woman is to homemaker? debiasing word embeddings."
[G.2] Garg et al. "Word embeddings quantify 100 years of gender and ethnic stereotypes."
[G.3] Sap et al. "Social bias frames: Reasoning about social and power implications of language."
[G.4] Zhao et al. "Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods"

[H] Relation Extraction and Distant supervision:

[H.1] Mintz et al. "Distant supervision for relation extraction without labeled data."
[H.2] Lin et al. "Neural relation extraction with selective attention over instances."
[H.3] Keith et al. "Identifying civilians killed by police with distantly supervised entity-event extraction."
[H.4] Riedel et al. "Relation extraction with matrix factorization and universal schemas."
[H.5] Verga et al. "Multilingual relation extraction using compositional universal schema."

[I] Weak Supervision:

[I.1] Karamanolakis et al. “Self-Training with Weak Supervision”
[I.2] Fu et al. “Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methods”
[I.3] Zhang et al. “WRENCH: A Comprehensive Benchmark for Weak Supervision”
[I.4] Li et al. “Prefix-Tuning: Optimizing Continuous Prompts for Generation.“