B.Sc. (Computational) Linguistics - LIN-BS-070 - Winter 21/22

Seminar: Mathematical Foundations of Language Modelling

	Brielen Madureira
	Online, Wednesdays at 12:00
	27.10.2021 - 16.02.2022
	Moodle page, course MathLM id 29573
	madureiralasota at uni-potsdam dot de

The information here may still be changed or updated until Oct 27 without notice. After that, any modification will be discussed/announced in the Moodle forum.

Description

Language models are a component of several natural language processing systems and applications such as machine translation and automatic speech recognition. They have been in the spotlight recently with the advent of gigantic, pre-trained models being released in the industry (e.g. BERT and GPT3). In this course, you will become familiar with the development of language models and with the mathematical foundations that are needed to understand, evaluate, implement and improve them, as well as to critically assess their usage and limitations.

Goals

• know the uses and limitations of language models
• comprehend the mathematical concepts behind language models
• be able to implement, review and improve Python code for training and evaluating language models
• be acquainted with the currently common language model architectures
• understand the ethical implications of widespread use of language models

Format

We’ll mostly follow a flipped classroom concept. You’ll work through the reading/watch list on your own during the week and join the meetings to work in groups on worksheets and assignments.

As long as the regulations allow, those who can may attend in person and those who cannot / do not wish to come in person can attend online, although the online experience may be limited.

The course will be taught in English, but you can discuss, present and submit your assignments and the project in German if you prefer.

Please, check the guidelines and the schedule for more details. Note that this page is meant only to present the course, but we’ll use Moodle for everything else (submissions, announcements, question forums etc), so please enrol in the Moodle course page.

Background

Everyone is welcome to take this class. Good knowledge of Python is required for the assignments. If you are in doubt, contact me :)

Requirements and grading policy

To pass the course:

• active participation (taking part on group discussions and handing in worksheets, documentation and assignments)

To pass the module:

• final project

To pass the course/module, you must achieve a performance of at least 60%, which includes abiding by the guidelines listed here.

Contact

Feel free to contact me if you have any question or suggestion. Please let me know if you have any needs (e.g. accessibility or name preference) or if you are facing any issue that demands adjustments (e.g. caring for family members or health).

I’m always willing to help if you start facing extra difficulties with the content as well.

I hope to see you soon!