NLP Seminar Autumn 2024
Time: every Friday from 10:15h until 12:00h
Location: Room 105 at the University Bern main building, Hochschulstrasse 4, Bern
ILIAS Course: https://ilias.unibe.ch/goto_ilias3_unibe_crs_3102245.html
KSL Entry: https://ksl.unibe.ch/KSL/kurzansicht?28&stammNr=471397&semester=HS2024&lfdNr=0
Responsible for the seminar: PD Dr. Matthias Stürmer
Main lecturer of the seminar: Luca Rolshoven
This seminar offers a conceptual and practical introduction to modern-day Natural Language Processing (NLP). The covered NLP techniques include the latest advancements such as transformer architectures, BERT, Large Language Models (LLMs), and Retrieval-Augmented Generation (RAG), alongside foundational methods like Bag-of-Words (BoW), TF-IDF, and word2vec. Some lectures are featured by guest speakers from academia or industry, giving additional perspectives to the students.
Before each session, students have to study the reading material and prepare questions for discussion. This engagement will deepen the understanding and foster analytical skills. Additionally, participants will undertake a project in the realm of NLP, which they will present at the end of the seminar.
This seminar is mandatory for all students conducting a bachelor's or master's thesis at the Research Center for Digital Sustainability.
Upon successful completion of this course, you will …
- know the most important methods of NLP
- be able to understand papers in the field of NLP
- have planned, executed and presented a project in the field of NLP
- have developed your presentation skills
- be able to understand and critically comment on the presentations of your fellow students
Schedule 2024
Date |
Topic |
Reading Material |
Guest Speaker |
---|---|---|---|
20 September 2024 |
Introduction to NLP |
None |
|
27 September 2024 |
Text Preprocessing and Language Basics |
NLP Course - Tokenizers (Hugging Face NLP Course) |
Veton Matoshi, Researcher at Bern University of Applied Sciences |
4 October 2024 |
Classical Machine Learning for NLP |
Analyzing Documents with TF-IDF (Tutorial) |
|
11 October 2024 |
Word Embeddings and Vector Space Models |
Efficient Estimation of Word Representations in Vector Space (word2vec Paper) |
Prof. Dr. Gerold Schneider, Professor of Computational Linguistics and Head of LiRI NLP group at University of Zürich |
18 October 2024 |
Transformer Architecture |
Attention is All You Need (Transformer Paper) |
|
25 October 2024 |
Introduction to Student Project |
None |
|
1 November 2024 |
Pre-Training |
Improving Language Understanding by Generative Pre-Training (GPT Paper) |
Leandro von Werra, Chief Loss Officer at Hugging Face |
8 November 2024 |
LLM Inference |
Flash Attention 3 (Blog Post) |
|
15 November 2024 |
Fine-Tuning |
Understanding Parameter-Efficient Finetuning of Large Language Models: From Prefix Tuning to LLaMA-Adapters (Blog Post) |
|
22 November 2024 |
Alignment |
The Alignment Handbook (Alignment Training Recipes) |
Lewis Tunstall, LLM Engineering & Research at Hugging Face |
29 November 2024 |
NLP in Industry |
(Will be updated soon) |
Flurin Gishamer, Senior Data Scientist at Open Systems |
6 December 2024 |
Large Language Models and Applications |
Prof. Dr. Marcel Gygli, Professor for AI in the Public Sector at Bern University of Applied Sciences |
|
13 December 2024 |
Advanced Topics and Emerging Trends |
(Will be updated soon) |
|
20 December 2024 |
Student Project Presentations |
None |