Computational Linguistics and Generative AI

Teacher(s)

Dupont Pierre;

Language

English
> French-friendly

Prerequisites

Main themes

• Different levels of linguistic analysis
• Probabilistic modeling of language (N-grams and Hidden Markov Models)
• Part-of-speech tagging
• Vector semantics and methods for vector word embeddings
• Deep learning applied to natural language processing
• Attention mechanisms and transformers
• Question-answering systems and conversational agents
• Applications in language engineering such as autocomplete software, automatic tagging, summarization, machine translation, information retrieval, and chatbots

Learning outcomes

At the end of this learning unit, the student is able to :

At the end of this learning unit, the student is able to : Given the learning outcomes of the "Master in Computer Science and Engineering" program, this course contributes to the development, acquisition and evaluation of the following learning outcomes:     INFO1.1-3     INFO2.3-4     INFO5.4, INFO5.5     INFO6.1, INFO6.5 Given the learning outcomes of the "Master [120] in Computer Science" program, this course contributes to the development, acquisition and evaluation of the following learning outcomes:     SINF1.M4     SINF2.3-4     SINF5.3, SINF5.4     SINF6.1, SINF6.5 Students completing successfully this course should be able to:     describe the fundamental concepts of natural language modeling;     master the methodology of using linguistic resources, in particular large scale corpora, possibly annotated or structured;     apply in a relevant way statistical language modeling techniques;     implement recent machine learning methods applied to language processing;     develop linguistic engineering applications. Students will have developed skills and operational methodology. In particular, they have developed their ability to:     integrate a multidisciplinary approach between computer science and linguistics, using wisely the terminology, tools and existing methods;     manage the time available to complete projects of medium size;     manipulate and exploit large amounts of data.

Content

• Various levels of linguistic analysis
• Probabilistic language modeling
• Part-of-speech tagging
• Vector semantics and word embeddings
• Deep Learning applied to natural language processing
• Question answering and conversational agents
• Linguistic engineering applications such as automatic completion software, POS tagging, text summarization, machine translation, information retrieval and chatbots

Teaching methods

• Lectures
• Practical projects implemented in Python on the Inginious platform.

Evaluation methods

Computation of the overall course grade

The projects are worth 30 % of the overall score for the course, 70 % for the final exam (closed-book).
The projects cannot be implemented again in second session. The global project grade is fixed at the end of the semester and included as such in the overall score for the course in the second session.
The final exam is, by default, a written exam (on paper or, when appropriate, on a computer).

Rules for student collaboration and use of external resources, including generative AIs

Collaborative studying among students is encouraged during project follow-up sessions and via an exchange forum on Moodle.
Each student is expected to submit a personal solution to each project. The use of public resources (e.g. stackoverflow.com), including generative AIs (e.g. chatGPT) is permitted, as long as each (fragment of) code submitted by the student mentions specifically all the resources used.
The distribution or exchange between students of (fragments of) code is not authorized by any means (GitHub, Facebook, Discord, etc.), even after the project deadlines.
Failure to comply with these rules for any project will result in an overall grade of 0 for all projects.
These rules are explained in detail during the first class (see course Moodle site).

Other information

Strong prerequisites 

• Good knowledge of python programming
• Introductory knowledge in data strucures and algorithms

      (see LINFO1101 and LINFO1103, or equivalent courses)

Additional prerequisites

• Basic knowledge in probability theory (e.g. conditional probability, probabilistic independence, Bayes rule, ...

Previous know-how

• Existing knowledge in machine learning is a plus but not a prerequisite

Online resources

moodle.uclouvain.be/course/view.php?id=1182

Bibliography

One recommended textbook - un ouvrage conseillé :

• Speech and Language Processing, D. Jurafsky and J.H. Martin, Prentice Hall.

Teaching materials

• Les supports obligatoires sont constitués de l'ensemble des documents (transparents des cours magistraux, énoncés des travaux pratiques, compléments, ...) disponibles depuis le site Moodle du cours.
• Required teaching material include all documents (lecture slides, project assignments, complements, ...) available from the Moodle website for this course.

Faculty or entity

> INFO