Stochastic modelling [ LINMA2470 ]

> https://icampus.uclouvain.be/claroline/course/index.php?cid=INMA2470

A probability course,  basic mathematical modeling skills, an introduction to matrix theory.

• Poisson processes
• Renewal processes
• Markov processes
• Markov decision processes.
• Queueing theory
• Simulation of discrete event systems

AA1.1
AA1.2
AA1.3
AA5.6

At the end of the course, the student will be able to :

• Know the properties of discrete stochastic processes, including renewal processes, markov processes and markov decision processes.
• Understand the impact of randomness and variability on the behavior of a system in steady state or transient regime.
• Analyse and compute the properties of different queueing systems (stationary and non-stationary).
• Use different types of stochastic processes to model a system with randomness.
• Optimize non-deterministic systems using Markov decision processes.
• Model different kind of systems with congestion with the use of queueing models.

Moreover, the student will have learnt to :

• Build a model and analyze congested systems
• Handle situations where decisions have to be taken under uncertainty.
• Build a simulation model for simple discrete event stochastic systems.

The contribution of this Teaching Unit to the development and command of the skills and learning outcomes of the programme(s) can be accessed at the end of this sheet, in the section entitled “Programmes/courses offering this Teaching Unit”.

Students will be evaluated through a written exam based on the objectives of the course. The exam consists in exercices applying the concepts viewed in the course. Many examples of questions of previous exams are solved during the exercice sessions.

The students will have to build a simulation model in order to analyse and understand the behavior of a congested stochastic system. This assignment is done in groups.

The course consists in weekly lectures and 11 exercice sessions. One of the courses will be devoted to the student presentations of their simulation projects and another session will host a practioner to present a real world application of the course contents.

• Poisson processes and their properties
• Markov chains with a finite number of states
• Renewal processes and stopping rules
• Markov chains with an infinite number of states
• The notion of reveribility
• Markov processes
• Birth and death processes
• Queueing theory and networks of queues
• Fluid models for queues
• Various applications, such as inventory management, replacement, reliability and job shop modeling.

Advised reading : book "Stochastic Processes: Theory for applications" by R. Gallagher, 2013, available on-line : http://www.rle.mit.edu/rgallager/notes.htm