Computational Semantics with Pictures - Einzelansicht

In this project class, we will look at current models that connect  language with vision, for example in tasks like resolving expressions  referring to objects in the given image, or generating natural language  descriptions of the content of an image. 

 

Language & vision is a very hot research topic at the moment, bringing together as it does two areas of AI that have seen dramatic progress in recent years, computer vision and natural language processing. In this class, we will be taking a more linguistic perspective on the topic, and investigate the kinds of language used in the available language & vision corpora, and will ask what we can learn about natural language semantics from them.

 

In the first few meetings, I will introduce some available datasets, tasks that can be defined on  them, and some common language / vision models. After that, you will  work in small groups on relevant projects within this topic area, and  the class meetings will be devoted to discussing progress. All groups will be expected to present their work to the class towards the end of the semester.

 

 

If you want to take part in the class / the first meeting, please sign up on moodle (sempix-20, passwd rpP%-2.15). There you will find, at the latest one hour before the meeting, a link to a zoom meeting room.