2023-05674 - Deep Learning application for style control of virtual characters motions
Level of qualifications required : Graduate degree or equivalent
Fonction : Internship Research
About the research centre or Inria departmentThe Inria Rennes - Bretagne Atlantique Centre is one of Inria's eight centres and has more than thirty research teams. The Inria Center is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc.
ContextSubject
In recent years, we have seen a significant development in creating digital doubles and characters who closely resemble real humans 1. Creating believable behaviour and animation for these virtual humans has remained challenging, especially when applied to photorealistic characters 2. In this research project, we will be exploring the animation qualities of photorealistic virtual humans. It has been shown that character movement can be perceived as attractive, significantly affecting users' behaviour towards them 3. Creating appalling characters and their impact on the users will be the central subject of our study.
The objective of this research is to identify which aspects of the character animation affect the user's perception and interaction with the character (agent). First, the intern will explore the techniques to edit the animations to manipulate the style of motion45. The team will focus on representation of motion data in sparse components which can be combined to produce movements of various styles (happy, depressed, calm, etc.). To do so we will train a Deep Learning model on large motion capture datasets78.
The design will be validated by conducting perceptual studies, particularly by harnessing the power of VR which allows the measure of behavioural responses of the users to virtual people, while our findings will be used to advance further research and provide design guidelines for a wider spread application for entertainment, training and rehabilitation.
AssignmentTasks
The intern will first familiarise with the equipment for animating a virtual human: state-of-the-art facial and body capture procedures and technology to create the virtual characters (pipeline shown in Figure 2). Based on the research interest of the intern, different tasks will be available during this internship:
The intern will work closely with the lead researchers to complete the selected tasks.
Requirements
The candidate should have:
Contact:
Katja Zibrek ([email protected]), Yuliya Patotskaya ([email protected]) and Ludovic Hoyet ([email protected]), Inria, VirtUs team.
Literature:
1 Metahumans, Unreal Engine: https: // www. unrealengine.com/en-US/metahuman
2 McDonnell, R., Breidt, M., & Bülthoff, H. H. (2012). Render me real?: investigating the effect of render style on the perception of animated virtual humans. ACM Transactions on Graphics (TOG), 31(4), 91
3 Zibrek, K., Niay, B., Olivier, A. H., Hoyet, L., Pettre, J., & McDonnell, R. (2020). The effect of gender and attractiveness of motion on proximity in virtual reality. ACM Transactions on Applied Perception (TAP), 17(4), 1-15.
4 I. Mason, S. Starke, H. Zhang, H. Bilen and T. Komura(2018), Few-shot Learning of Homogeneous Human Locomotion Styles
5 Daniel Holden and Jun Saito and Taku Komura (2016) A deep learning framework for character motion synthesis and editing, ACM Transactions on Graphics (TOG)
6 Kfir Aberman, Yijia Weng, Dani Lischinski, Daniel Cohen-Or, Baoquan Chen.
Motion Style Transfer from Video to Animation, ACM Transactions on Graphics (SIGGRAPH 2020).
7 Learned Motion Matching, DANIEL HOLDEN, Ubisoft La Forge, Ubisoft, OUSSAMA KANOUN, Ubisoft La Forge, Ubisoft, MAKSYM PEREPICHKA, Concordia University, TIBERIU POPA, Concordia University, Canada
8 A Deep Learning Framework for Character Motion Synthesis and Editing, Daniel Holden, Jun Saito, Taku Komura
General InformationTheme/Domain : Data and Knowledge Representation and Processing Data production, processing, analysis (BAP D)
Town/city : Rennes
Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.
Instruction to applyPlease submit online : your resume, cover letter and letters of recommendation eventually
For more information, please contact [email protected]
Defence Security : This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy : As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.