PhD Position F/M [ALRC 2023] Synchronization of uncertain and different-type systems via reinforcement learning

2023-05754 - PhD Position F/M ALRC 2023 Synchronization of uncertain and different-type systems via reinforcement learning

Contract type : Fixed-term contract

Level of qualifications required : Graduate degree or equivalent

Fonction : PhD Position

About the research centre or Inria department

The Inria University of Lille centre, created in 2008, employs 360 people including 305 scientists in 15 research teams. Recognised for its strong involvement in the socio-economic development of the Hauts-De-France region, the Inria University of Lille centre pursues a close relationship with large companies and SMEs. By promoting synergies between researchers and industrialists, Inria participates in the transfer of skills and expertise in digital technologies and provides access to the best European and international research for the benefit of innovation and companies, particularly in the region.

For more than 10 years, the Inria University of Lille centre has been located at the heart of Lille's university and scientific ecosystem, as well as at the heart of Frenchtech, with a technology showroom based on Avenue de Bretagne in Lille, on the EuraTechnologies site of economic excellence dedicated to information and communication technologies (ICT).

Context

The problem of synchronizing different and uncertain dynamical systems (agents) appears in multiple contexts: electrical, mechanical or biological systems, network communications, etc. The difficulty comes from the need to force distinct dynamics (e.g., of mass, size, or even physical nature) to make similar movements by rejecting external disturbances. The uncertainty of the models for the agents introduces another technical obstacle for the realization of synchronous movements. The connection topology (leader- follower, or mutual), and the associated communication delays, represent other complications to synchronization, which must also be neglected by agents. Finally, constraints on allowable movements for each participant (e.g., avoiding collisions) turn this problem into a highly nonlinear and nonconvex question, where conventional control theory fails to provide a solution. The existing methods, mainly related to the model predictive control approach, deal with simple linear models of the systems, with a limited uncertainty, while assuming a convexity of the optimized goal functional, which is rarely observed in practice. The objective of this thesis project is to design synchronization control algorithms under severe uncertainty conditions, with the presence of other uncertain agents, which penalize the risk of error and its cost. The complexity of the posed problem requires an interdisciplinary approach for its solution, as envisaged in this project, which belongs to an intersection of the fields of machine learning and control theory. It is intended to combine reinforcement learning and model predictive control approaches for the realization of synchronization strategies for different and uncertain dynamic agents.

Assignment

For references on the subject and previous works, please, check:

Luo, A.C.J. Dynamical system synchronization. Springer, New York, 2013.

Izhikevich, E.M. Dynamical systems in neuroscience. MIT Pres, Boston, 2006.

Osipov, G.V., Kurths, J., Zhou, C. Synchronization in Oscillatory Networks. Springer, New York, 2007.

Leurent, E., Maillard, O.-A., Efimov, D. Robust-Adaptive Control of Linear Systems: beyond Quadratic Costs. Proc. NeurIPS, Vancouver, 2020.

Leurent, E., Efimov, D., Maillard, O.-A. Robust-Adaptive Interval Predictive Control for Linear Uncertain Systems. Proc. 59th IEEE CDC, Republic of Korea, 2020.

Leurent, E., Blanco, Y., Efimov, D., Maillard, O.-A. Approximate Robust Control of Uncertain Dynamical Systems. 32nd NeurIPS, Montréal, 2018.

dos Reis de Souza, A., Efimov, D., Raïssi, T. Robust output feedback MPC for LPV systems using interval observers. IEEE Trans. Automatic Control, 67(07), 2022.

Bellemare, M.G., Candido, S., Castro, P.S. et al. Autonomous navigation of stratospheric balloons using reinforcement learning. Nature 588, 77–82, 2020.

Mnih, V., Kavukcuoglu, K., Silver, D. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533, 2015.

Lillicrap, T., Hunt, J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015.

Main activities

The suggested plan of this academic thesis consists in execution of three tasks or work packages, mainly oriented on development of fundamental approaches. Each will be accompanied by publications in international top rated journals and conferences, in the spirit of open and reproducible science, open-source code, etc.

Task 1: Modeling of the systems to be synchronized.

It is planned to search for admissible models to write the dynamics of agents (e.g., Lur'e or Persidskii models, homogeneous systems), and analyze uncertainty classes. Then the methods of their identification should be studied. Particular attention should be paid to the simplicity of implementation, the accuracy and the robustness of the available estimation approaches.

Task 2: Estimation and prediction of agent behavior.

For the chosen models, robust predictors and observers must be designed for behavior recognition of other participants and the controlled system. In the case of presentation of an agent by several models, it is necessary to discriminate online the current mode compared to the other variants.

Task 3: Robust synchronization.

Finally, synchronization control algorithms should be synthesized (the main goal), which should navigate systems towards cooperative movements in selected scenarios (assuming there is no common center for agent coordination). The intelligent regulator must take into account the uncertainty linked to the environment, the constraints imposed, possibly taking into account the rational behavior of other agents. Communication strategies must be analyzed, which can help in the synchronization of uncertain and different dynamic systems, in the presence of the resulting delays.

Skills

The candidate should be familiar with basic methods in the AI theory and in the control theory (Lyapunov stability, observer design, model predictive control), as well as reinforcement learning (Markov decision processes, bandits).

Benefits package

Subsidized meals

Partial reimbursement of public transport costs

Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)

Possibility of teleworking and flexible organization of working hours

Professional equipment available (videoconferencing, loan of computer equipment, etc.)

Social, cultural and sports events and activities

Access to vocational training

Social security coverage

Remuneration

1st and 2nd year: 2 051€ gross per month

3rd year: 2 158€ gross per month

General Information

Theme/Domain : Robotics and Smart environments System & Networks (BAP E)

Town/city : Villeneuve d'Ascq

Inria Center : Centre Inria de l'Université de Lille

Starting date : 2023-10-01

Duration of contract : 3 years

Deadline to apply : 2023-04-18

Contacts

Inria Team : VALSE

PhD Supervisor : Efimov Denis / [email protected]

About Inria

Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.

Instruction to apply

CV + Cover Letter

Defence Security : This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.

Recruitment Policy : As part of its diversity policy, all Inria positions are accessible to people with disabilities.

Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.