2023-06366 - PhD Position F/M Interaction for AR-Based Tangible Control of Molecular Visualization
Contract type : Fixed-term contract
Level of qualifications required : Graduate degree or equivalent
Fonction : PhD Position
About the research centre or Inria departmentThe Inria Saclay-Île-de-France Research Centre was established in 2008. It has developed as part of the Saclay site in partnership with Paris-Saclay University and with the Institut Polytechnique de Paris .
The centre has 39 project teams , 27 of which operate jointly with Paris- Saclay University and the Institut Polytechnique de Paris; Its activities occupy over 600 people, scientists and research and innovation support staff, including 44 different nationalities.
ContextThe project will be hosted by Inria's AVIZ team and will be led by AVIZ' Tobias Isenberg (GS ISN). We will collaborate with Bastien Vincke from the SATIE lab (GS SIS) and Nicolas Férey from LISN's VENISE team (GS ISN).
AVIZ has a strong expertise in all facets of visualization and the interaction with visual representations. Tobias Isenberg focuses on interaction with 3D data visualization and immersive analytics 6, 17, specifically w.r.t. AR- based data exploration but with a focus on combinations of mobile devices with virtually displayed data representations. He also has worked on multiple molecular visualization and abstraction techniques in the past.
Bastien Vincke at SATIE lab is interested in human-machine interaction based on the use of deformable, reconfigurable, and instrumented tangible interfaces as shown in https: // youtu.be/VoGPPbrwy68. He designs embedded systems that are energy-autonomous, communicate wirelessly, and allow the reconstruction of a digital twin of the physical interface.
The VENISE team at LISN lab leads research on Virtual and Augmented Reality. Nicolas Férey focuses on designing mixed reality workspace dedicated to molecular science, especially to provide interactive molecular simulation features allowing user to steer in real time components of a simulation in progress.
To effectively conduct this project we plan to also include end users during at each step of the research project, through a collaboration with Marc Baaden's team working on interactive molecular visualization and simulation, putting together UnityMol as a prototyping platform, with the particular aim to enable sharing such experiences. As a leading molecular modeling research team with ample experience in creating virtual digital twins, they will provide realistic domain applications.
In addition we will collaborate with colleagues from Linköping University (Anders Ynnerman, molecular visualization and educational application scenarios) in Sweden. We also have access to domain experts in structural biology at the Institute of Physical and Chemical Biology, Paris (Marc Baaden's host institution).
AssignmentWe will focus on the context of the hybrid interaction scenario and, specifically, how to design, use, and share workspaces that support a range of tasks that include actual data analysis, the sharing of research results, teaching about molecular structures, and research outreach. In all these contexts we need to be able to merge the tangible control for the manipulation of the target molecular structure with virtual 3D elements (e.g., solvent- accessible surfaces, electric fields, derived measures, other interacting molecules with potential bonds that are forming, etc.), interaction controls for the running simulation and visualization, and analysis tools such as plots of abstract data values.
In the different target scenarios, however, these have to be combined in different ways to support the different exploration goals. For example, researchers that explore specific molecular configurations may need to have a richer control of simulation and visualization parameters and may need to have access to multiple different abstract data plots, while for sharing the results of some investigation (e.g., in the form of material that is shared together with a scientific publication) less control and only specific plots are needed. In a teaching or outreach scenario, in contrast, the abstract plots may be of less importance and the 3D visualization elements would likely be the primary focus.
Within our envisioned hybrid context, however, we have a clear focus of the interaction in form of the tangible molecular assembly, so that we need to build the workspace around this focus point. Yet we need to place other visualization and control elements such that they are easy to access by the different users. Here we need to investigate which type of input works best—likely the tangible input will be restricted to change the configuration of the molecular structure in focus and other forms of input are needed to control visualization, simulation, and access to additional information. AR environments typically make use of gestural input for this purpose, yet this type is know for its imprecision. We thus want to explore other input, including tablets and controllers that are typically used in combination with VR headsets.
In conclusion, we aim to address the following research questions in this project: • How can we support the creation of a variety of workspaces that merge tangible control of molecular simulations with AR-based overlays and virtual AR dashboards, depending on the application case? • What type of interactive input is needed and ideal for controlling and parameterizing the different workspaces? Does it differ depending on the specific type of workspace? • How can we access, use, and control different information layers, both 3D virtual overlays for the tangible input and also abstract visual plots of data derived from the molecular simulation?
More detailed description: https: // www. aviz.fr/wiki/uploads/Research/2023AVIZPhDProject-TACTUAL.pdf
Main activitiesOur goal is to design hybrid data exploration and presentation environments that embrace both tangible and other controls and AR-based visualization. We will thus work on establishing two or three dedicated exploration scenarios, to be selected from the previously mentioned scientific exploration, scientific results presentation, teaching, and outreach goals. We will then study these in more detail and together with domain experts work toward designs for the interactive interfaces. Later, we will aim to generalize our results to application domains beyond structural biology as we believe similar designs may apply to other sciences as well.
1. Overview We will begin by studying past published work in the field, including work on (a) scientific visualization methods for 3D data, including those that focus on molecular data, (b) AR-compatible input and output devices or techniques (including gestures), mainly by characteristics of interaction, and (c) relevant general AR/VR interaction systems, with a particular focus on existing scientific information dashboards. We will specifically study the capabilities of the tangible molecule input device that are being developed by Bastien Vincke and Nicolas Férey's groups 39, 40 (see https: // youtu.be/VoGPPbrwy68).
Based on this overview of the work to date, we will design a strategy to extend the existing code base (see below for details) with the specific functionalities that are currently missing. We will formalize the workspace scenarios envisioned for the research and outreach domains so that they can serve as characteristic templates for similar use cases. Based on this overall plan, we will select the most appropriate specific examples to illustrate our hypotheses from the many modeling projects being conducted in Marc Baaden's lab, in order to address the research questions mentioned above. Focus groups with scientists who are the stakeholders in these use cases will be put together to regularly follow up on and assess the implementation prototypes in an iterative co-design process.
2. Interaction Design For the specific interaction design we will engage with domain experts from the field of structural biology to gather their needs, and use these as the basis for our designs. We will then create prototypical implementations and use user-centered design strategies and experimental validations to get feedback. Based on this feedback we will iterate over our designs, with the ultimate goal of making our results available to the domain in form of implementations within established tools (see following point).
3. Implementation We will implement our experiments by combining several well established and mature tools as the starting point. For molecular visualization and connection to interactive simulations we can build on UnityMol (unitymol.sourceforge.net), developed at one of the partner laboratories. The tangible prototype is available to us via the ongoing funded research project Pirate (anr.fr/Project-ANR-21-CE45-0014) and has a basic implementation within the UnityMol framework already. Complementary visual data analysis may be based on either VIA-MD 36 or Inviwo (inviwo.org) 16 from collaborators in Sweden at Link¨oping University.
4. Research plan for the student months 1–6: literature, device, user surveys months 3–12: interaction design, iteration, and gathering of feedback months 12–24: system design and implementation months 24–36: validate the system through several tasks and through user experiments months 18–36: gather feedback and optimize the system months 24–36: PhD thesis
5. General Methods: Open Science In general, we will follow modern open science research methods. This means we will use open-source implementations, submit all our work for publication in international journals or conferences, and use methods that facilitate the replicability of our results. For example, we will pre-register any experimental validations (on OSF), provide open repositories of our publications, implementations, and data (on services such as GitLab, GitHub, and/or OSF; as we did with past work: tobias.isenberg.cc/Publications), and apply for validations such as those from the Graphics Replicability Stamp Initiative (replicabilitystamp.org; also as with our past work: tobias.isenberg.cc/replicability).
SkillsWe are looking for a student with this profile: • highly motivated with an interest in the hardware aspects of tangible interfaces, in immersive analytics, and in scientific visualization, • fluent written and spoken communication in English to interact within the research team, to collaborate with domain scientists, and to disseminate the results in scientific publications, • excellent skills in computer graphics programming, specifically modern graphics programming and VR and/or AR headset programming, • experience in scientific visualization, • students with experience in HCI and empirical evaluation are preferred, and • students with past scientific publications in computer graphics, visualization, and/or HCI are preferred.
Benefits packageTheme/Domain : Interaction and visualization Information system (BAP E)
Town/city : Palaiseau
Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.
Instruction to applyDefence Security : This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy : As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.