Universität Wien FIND

Due to the COVID-19 pandemic, changes to courses and exams may be necessary at short notice. Inform yourself about the current status on u:find and check your e-mails regularly. Registration is mandatory for courses and exams. Wearing a FFP2 face mask and a valid evidence of being tested, vaccinated or have recovered from an infection are mandatory on site.

Please read the information on studieren.univie.ac.at/en/info.

136041 SE Topics in Deep Learning and Natural Language Processing (2020W)

Continuous assessment of course work

Registration/Deregistration

Note: The time of your registration within the registration period has no effect on the allocation of places (no first come, first served).

Details

max. 25 participants
Language: English

Lecturers

Classes (iCal) - next class is marked with N

The seminar mode will be fully online using BigBlueButton, you can connect to the meeting room with the following link:

https://bbb.cs.univie.ac.at/b/ben-1st-jg5-bzm

Please make sure you have a working setup that includes camera and microphone.

Thursday 01.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 08.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 15.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 22.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 29.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 05.11. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 12.11. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 19.11. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 26.11. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 03.12. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 10.12. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 17.12. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 07.01. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 14.01. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 21.01. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Thursday 28.01. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG

Information

Aims, contents and method of the course

In this seminar, participants will read, present and discuss recent papers on deep learning for natural language processing.

Possible topics to be covered in the seminar:
- Word vectors
- Classical Neural Network Layers
- Attention
- Pre-Trained language models
- Data Sets and Evaluation
- Probing of pre-trained language models, explainability
- Language bias and learned representations
- Relation Extraction and Distant supervision

Assessment and permitted materials

=== the information below is preliminary, to be finalized soon ===

Participants will have to present one topic from the list in the seminar, the presentation should be roughly 25 minutes (hard limits: min. 20 minutes, max. 30 minutes). The presentation is followed by a QA-session and discussion. Participants will also have to submit a written report (deadline: February 28, 2021; 15-20 pages, exact requirements TBD), describing the main contents of the presented paper and putting it in a wider context.

Please send an email to nlp.datamining@univie.ac.at including a selection of *5 topics* from the list below, and indicate your *study program* (Computer Science, Digital Humanities, ...). You will be assigned one topic from your selection (for your presentation and report). For additional two topics (also from your selection, but presented by somebody else) you will have to prepare some questions that can get a discussion started.
If your selection includes topics from groups [A] or [B], please send your email until Friday, September 25 (first possible presentation will be Thursday, October 8). Otherwise, send your email until Tuesday, October 6 (presentations from October 22).

Minimum requirements and assessment criteria

=== the information below is preliminary, to be finalized soon ===

Your presentation will account for 45% of the grade, participation in discussions for 10%, and the written report for 45%.

Examination topics

.

Reading list

[A] Word vectors:

[A.1] Mikolov et al. "Distributed representations of words and phrases and their compositionality."

[A.2] Pennington et al. "Glove: Global vectors for word representation."

[A.3] Bojanowski et al. "Enriching word vectors with subword information."

[B] Classical Neural Network Layers:

[B.1] Goodfellow et al., Long short-term memory networks, https://www.deeplearningbook.org, Chapter 10 (skip 10.8, 10.9)

[B.2] Collobert et al., "Natural language processing (almost) from scratch."

[C] Attention:

[C.1] Hermann et al. "Teaching machines to read and comprehend."

[C.2] Bahdanau et al. "Neural machine translation by jointly learning to align and translate."

[C.3] Vaswani et al. "Attention is all you need."

[D] Pre-Trained language models:

[D.1] Devlin et al. "Bert: Pre-training of deep bidirectional transformers for language understanding."

[D.2] Yang et al. "Xlnet: Generalized autoregressive pretraining for language understanding."

[D.3] Peters et al. "Knowledge enhanced contextual word representations."

[E] Data Sets and Evaluation:

[E.1] Kwiatkowski et al. "Natural questions: a benchmark for question answering research."

[E.2] Lichtarge et al. "Corpora generation for grammatical error correction."

[E.3] Geva et al. "DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion"

[E.4] Wang et al. "Glue: A multi-task benchmark and analysis platform for natural language understanding."

[F] Probing of pre-trained language models, explainability:

[F.1] Jawahar et al. "What does BERT learn about the structure of language?"

[F.2] Petroni et al. "Language models as knowledge bases?"

[F.3] Tenney et al. "BERT Rediscovers the Classical NLP Pipeline"

[F.4] Ribeiro et al. ""Why should I trust you?" Explaining the predictions of any classifier."

[F.5] Ribeiro et al. "Beyond Accuracy: Behavioral Testing of NLP Models with CheckList."

[F.6] Bender, Koller: "Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data"

[G] Language bias and learned representations:

[G.1] Bolukbasi et al. "Man is to computer programmer as woman is to homemaker? debiasing word embeddings."

[G.2] Garg et al. "Word embeddings quantify 100 years of gender and ethnic stereotypes."

[G.3] Sap et al. "Social bias frames: Reasoning about social and power implications of language."

[G.4] Zhao et al. "Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods"

[H] Relation Extraction and Distant supervision:

[H.1] Mintz et al. "Distant supervision for relation extraction without labeled data."

[H.2] Lin et al. "Neural relation extraction with selective attention over instances."

[H.3] Keith et al. "Identifying civilians killed by police with distantly supervised entity-event extraction."

[H.4] Riedel et al. "Relation extraction with matrix factorization and universal schemas."

[H.5] Verga et al. "Multilingual relation extraction using compositional universal schema."

Association in the course directory

S-DH (Cluster I: Language and Literature)

Last modified: Tu 29.09.2020 11:09