Universität Wien FIND

Auf Grund der COVID-19 Pandemie kann es bei Lehrveranstaltungen und Prüfungen auch kurzfristig zu Änderungen kommen. Informieren Sie sich laufend in u:find und checken Sie regelmäßig Ihre E-Mails. Bei Lehrveranstaltungen und Prüfungen gilt Anmeldepflicht, vor Ort gelten FFP2-Pflicht und 2,5G.

Lesen Sie bitte die Informationen auf studieren.univie.ac.at/info.

136041 SE Topics in Deep Learning and Natural Language Processing (2020W)

Prüfungsimmanente Lehrveranstaltung

An/Abmeldung

Hinweis: Ihr Anmeldezeitpunkt innerhalb der Frist hat keine Auswirkungen auf die Platzvergabe (kein "first come, first served").

Details

max. 25 Teilnehmer*innen
Sprache: Englisch

Lehrende

Termine (iCal) - nächster Termin ist mit N markiert

The seminar mode will be fully online using BigBlueButton, you can connect to the meeting room with the following link:

https://bbb.cs.univie.ac.at/b/ben-1st-jg5-bzm

Please make sure you have a working setup that includes camera and microphone.

Donnerstag 01.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 08.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 15.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 22.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 29.10. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 05.11. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 12.11. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 19.11. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 26.11. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 03.12. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 10.12. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 17.12. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 07.01. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 14.01. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 21.01. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG
Donnerstag 28.01. 09:45 - 11:15 Hörsaal 2, Währinger Straße 29 2.OG

Information

Ziele, Inhalte und Methode der Lehrveranstaltung

In this seminar, participants will read, present and discuss recent papers on deep learning for natural language processing.

Possible topics to be covered in the seminar:
- Word vectors
- Classical Neural Network Layers
- Attention
- Pre-Trained language models
- Data Sets and Evaluation
- Probing of pre-trained language models, explainability
- Language bias and learned representations
- Relation Extraction and Distant supervision

Art der Leistungskontrolle und erlaubte Hilfsmittel

=== the information below is preliminary, to be finalized soon ===

Participants will have to present one topic from the list in the seminar, the presentation should be roughly 25 minutes (hard limits: min. 20 minutes, max. 30 minutes). The presentation is followed by a QA-session and discussion. Participants will also have to submit a written report (deadline: February 28, 2021; 15-20 pages, exact requirements TBD), describing the main contents of the presented paper and putting it in a wider context.

Please send an email to nlp.datamining@univie.ac.at including a selection of *5 topics* from the list below, and indicate your *study program* (Computer Science, Digital Humanities, ...). You will be assigned one topic from your selection (for your presentation and report). For additional two topics (also from your selection, but presented by somebody else) you will have to prepare some questions that can get a discussion started.
If your selection includes topics from groups [A] or [B], please send your email until Friday, September 25 (first possible presentation will be Thursday, October 8). Otherwise, send your email until Tuesday, October 6 (presentations from October 22).

Mindestanforderungen und Beurteilungsmaßstab

=== the information below is preliminary, to be finalized soon ===

Your presentation will account for 45% of the grade, participation in discussions for 10%, and the written report for 45%.

Prüfungsstoff

.

Literatur

[A] Word vectors:

[A.1] Mikolov et al. "Distributed representations of words and phrases and their compositionality."

[A.2] Pennington et al. "Glove: Global vectors for word representation."

[A.3] Bojanowski et al. "Enriching word vectors with subword information."

[B] Classical Neural Network Layers:

[B.1] Goodfellow et al., Long short-term memory networks, https://www.deeplearningbook.org, Chapter 10 (skip 10.8, 10.9)

[B.2] Collobert et al., "Natural language processing (almost) from scratch."

[C] Attention:

[C.1] Hermann et al. "Teaching machines to read and comprehend."

[C.2] Bahdanau et al. "Neural machine translation by jointly learning to align and translate."

[C.3] Vaswani et al. "Attention is all you need."

[D] Pre-Trained language models:

[D.1] Devlin et al. "Bert: Pre-training of deep bidirectional transformers for language understanding."

[D.2] Yang et al. "Xlnet: Generalized autoregressive pretraining for language understanding."

[D.3] Peters et al. "Knowledge enhanced contextual word representations."

[E] Data Sets and Evaluation:

[E.1] Kwiatkowski et al. "Natural questions: a benchmark for question answering research."

[E.2] Lichtarge et al. "Corpora generation for grammatical error correction."

[E.3] Geva et al. "DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion"

[E.4] Wang et al. "Glue: A multi-task benchmark and analysis platform for natural language understanding."

[F] Probing of pre-trained language models, explainability:

[F.1] Jawahar et al. "What does BERT learn about the structure of language?"

[F.2] Petroni et al. "Language models as knowledge bases?"

[F.3] Tenney et al. "BERT Rediscovers the Classical NLP Pipeline"

[F.4] Ribeiro et al. ""Why should I trust you?" Explaining the predictions of any classifier."

[F.5] Ribeiro et al. "Beyond Accuracy: Behavioral Testing of NLP Models with CheckList."

[F.6] Bender, Koller: "Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data"

[G] Language bias and learned representations:

[G.1] Bolukbasi et al. "Man is to computer programmer as woman is to homemaker? debiasing word embeddings."

[G.2] Garg et al. "Word embeddings quantify 100 years of gender and ethnic stereotypes."

[G.3] Sap et al. "Social bias frames: Reasoning about social and power implications of language."

[G.4] Zhao et al. "Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods"

[H] Relation Extraction and Distant supervision:

[H.1] Mintz et al. "Distant supervision for relation extraction without labeled data."

[H.2] Lin et al. "Neural relation extraction with selective attention over instances."

[H.3] Keith et al. "Identifying civilians killed by police with distantly supervised entity-event extraction."

[H.4] Riedel et al. "Relation extraction with matrix factorization and universal schemas."

[H.5] Verga et al. "Multilingual relation extraction using compositional universal schema."

Zuordnung im Vorlesungsverzeichnis

S-DH (Cluster I: Language and Literature)

Letzte Änderung: Di 29.09.2020 11:09