Deep Learning application for style control of virtual characters motions

Inria
February 28, 2023
Contact:N/A
Offerd Salary:Negotiation
Location:N/A
Working address:N/A
Contract Type:Other
Working Time:Negotigation
Working type:N/A
Ref info:N/A

2023-05674 - Deep Learning application for style control of virtual characters motions

Level of qualifications required : Graduate degree or equivalent

Fonction : Internship Research

About the research centre or Inria department

The Inria Rennes - Bretagne Atlantique Centre is one of Inria's eight centres and has more than thirty research teams. The Inria Center is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc.

Context

Subject

In recent years, we have seen a significant development in creating digital doubles and characters who closely resemble real humans 1. Creating believable behaviour and animation for these virtual humans has remained challenging, especially when applied to photorealistic characters 2. In this research project, we will be exploring the animation qualities of photorealistic virtual humans. It has been shown that character movement can be perceived as attractive, significantly affecting users' behaviour towards them 3. Creating appalling characters and their impact on the users will be the central subject of our study.

The objective of this research is to identify which aspects of the character animation affect the user's perception and interaction with the character (agent). First, the intern will explore the techniques to edit the animations to manipulate the style of motion45. The team will focus on representation of motion data in sparse components which can be combined to produce movements of various styles (happy, depressed, calm, etc.). To do so we will train a Deep Learning model on large motion capture datasets78.

The design will be validated by conducting perceptual studies, particularly by harnessing the power of VR which allows the measure of behavioural responses of the users to virtual people, while our findings will be used to advance further research and provide design guidelines for a wider spread application for entertainment, training and rehabilitation.

Assignment

Tasks

The intern will first familiarise with the equipment for animating a virtual human: state-of-the-art facial and body capture procedures and technology to create the virtual characters (pipeline shown in Figure 2). Based on the research interest of the intern, different tasks will be available during this internship:

  • Deep Learning / Machine Learning approaches
  • Environment and agent creation in VR
  • Animation techniques
  • The intern will work closely with the lead researchers to complete the selected tasks.

    Requirements

    The candidate should have:

  • Sufficient programming background Python
  • Acquaintance with Deep / Machine Learning is plus
  • Acquaintance with Deep Learning tools, as Theano, TensorFlow, PyTorch etc. is plus
  • Interest in character animation and game (virtual reality) development
  • Previous knowledge of Unreal or Unity is a plus
  • Good level of English is required.
  • Contact:

    Katja Zibrek (katja.zibrek@inria.fr), Yuliya Patotskaya (yuliya.patotskaya@inria.fr) and Ludovic Hoyet (ludovic.hoyet@inria.fr), Inria, VirtUs team.

    Literature:

    1 Metahumans, Unreal Engine: https: // www. unrealengine.com/en-US/metahuman

    2 McDonnell, R., Breidt, M., & Bülthoff, H. H. (2012). Render me real?: investigating the effect of render style on the perception of animated virtual humans. ACM Transactions on Graphics (TOG), 31(4), 91

    3 Zibrek, K., Niay, B., Olivier, A. H., Hoyet, L., Pettre, J., & McDonnell, R. (2020). The effect of gender and attractiveness of motion on proximity in virtual reality. ACM Transactions on Applied Perception (TAP), 17(4), 1-15.

    4 I. Mason, S. Starke, H. Zhang, H. Bilen and T. Komura(2018), Few-shot Learning of Homogeneous Human Locomotion Styles

    5 Daniel Holden and Jun Saito and Taku Komura (2016) A deep learning framework for character motion synthesis and editing, ACM Transactions on Graphics (TOG)

    6 Kfir Aberman, Yijia Weng, Dani Lischinski, Daniel Cohen-Or, Baoquan Chen.

    Motion Style Transfer from Video to Animation, ACM Transactions on Graphics (SIGGRAPH 2020).

    7 Learned Motion Matching, DANIEL HOLDEN, Ubisoft La Forge, Ubisoft, OUSSAMA KANOUN, Ubisoft La Forge, Ubisoft, MAKSYM PEREPICHKA, Concordia University, TIBERIU POPA, Concordia University, Canada

    8 A Deep Learning Framework for Character Motion Synthesis and Editing, Daniel Holden, Jun Saito, Taku Komura

    General Information
  • Theme/Domain : Data and Knowledge Representation and Processing Data production, processing, analysis (BAP D)

  • Town/city : Rennes

  • Inria Center : Centre Inria de l'Université de Rennes
  • Starting date : 2023-03-01
  • Duration of contract : 7 months
  • Deadline to apply : 2023-02-28
  • Contacts
  • Inria Team : VIRTUS
  • Recruiter : Patotskaya Yuliya / yuliya.patotskaya@inria.fr
  • About Inria

    Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.

    Instruction to apply

    Please submit online : your resume, cover letter and letters of recommendation eventually

    For more information, please contact yuliya.patotskaya@inria.fr

    Defence Security : This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.

    Recruitment Policy : As part of its diversity policy, all Inria positions are accessible to people with disabilities.

    Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.

    From this employer

    Recent blogs

    Recent news