Extreme conditions of spatial missions (Moon or Mars) will lead to a number of remotely controlled operations of rovers, especially for exploratory and construction activities.

In these conditions, environmental sensing will likely be achieved through on board cameras and positioning systems. Inside a rover, interfaces based on an XR environment could be implemented to ensure consistent situational awareness for the human operator.

Similar issues can be found in the context of scuba diving, providing an easier experimental background.

The following scientific challenges need to be addressed for the development of this XR environment:

• In natural viewing conditions, head turns provide valuable proprioceptive information about perception of space. In an XR environment, how does a limited field of view and non-naturalistic implementations of head position impact global situational awareness?
• The latency due to distance (1.25s between the Earth and the Moon at light speed) will most probably also impact the human operator's situational awareness but also induce cybersickness. How can it be mitigated?
• The human operator will necessarily be protected from the zero gravity external environment. This will mean that they will not have access to the perceptual and proprioceptive feedback they would ordinarily use on Earth. We will investigate if combining sensory feedback across multiple modalities can support situational awareness, and if an AI agent could be used to assess the most effective combination of sensory and proprioceptive feedback in a given scenario.

The impact of the technical constraints of XR teleoperation on user perception will first be studied by means of user experiences. This is to elicit the key elements specific to this type of environment. These user studies in simulated virtual environments will highlight and precisely assess the impact of these limitations on perception.

The second part of the program will then focus on proposing technical aids to compensate for these biases. It will include simulation experiments, based on sensory compensations but also AI-based, using agents or state prediction.

Finally, the results of these studies can be tested under real-life conditions in an underwater environment, where technical constraints are similar to experimental conditions in space (limited sensors, weak effect of gravity).

These results will contribute to the improvement of teleoperations devices in deteriorated telecommunication conditions (space, submarines, poor network coverage) and with specific physical constraints (zero or low gravity, high drag, limited sensors).

This project will leverage Prof Moreau and Dr Peillard’s expertise in XR and human behaviour, Coppin’s experience in human-autonomy teaming and human factors, and Ma-Wyatt’s experience in human factors, sensorimotor control and eye movements. In this thesis topic, the candidate will have the opportunity to work with all of these experts to devise new forms of experiments and XR that enable quantification and prediction of human performance in extreme environments off earth (or undersea).

• Benaoumeur, Ibari, Ahmed-foitih Zoubir, et Hanifi Elhachimi Amar Reda. « Remote Control of Mobile Robot Using the Virtual Reality » 5, no 5 (2015).
• Brunnström, Kjell, Elijs Dima, Tahir Qureshi, Mathias Johanson, Mattias Andersson, et Mårten Sjöström. « Latency impact on Quality of Experience in a virtual reality simulator for remote control of machines ». Signal Processing: Image Communication 89 (1 novembre 2020): 116005. https://doi.org/10.1016/j.image.2020.116005.
• Nguyen, Laurent A., Maria Bualat, Laurence J. Edwards, Lorenzo Flueckiger, Charles Neveu, Kurt Schwehr, Michael D. Wagner, et Eric Zbinden. « Virtual Reality Interfaces for Visualization and Control of Remote Vehicles ». Autonomous Robots 11, no 1 (Jul. 2001): 59‐68. https://doi.org/10.1023/A:1011208212722.
• Richardson, Barry, Mark Symmons, et Dianne Wuillemin. « The Contribution of Virtual Reality to Research on Sensory Feedback in Remote Control ». Virtual Reality 9, no 4 (1 avril 2006): 234‐42. https://doi.org/10.1007/s10055-006-0020-z.
• Tittle, James S., Axel Roesler, et David D. Woods. « The Remote Perception Problem ». Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46, no 3 (Sep. 2002): 260‐64. https://doi.org/10.1177/154193120204600309.

• IMT Atlantique: Prof. Guillaume Moreau, Prof. Gilles Coppin and Assoc. Prof. Etienne Peillard, IMT Atlantique, Brest, France

  The PhD student will stay two years at IMT Atlantique.

• International partner: Prof. Anna Ma-Wyatt, University of Adelaide, Adelaide, Australia

  The PhD student will stay one year at Prof. Ma-Wyatt's lab.

• Industrial partner(s) for short visits have not yet been determined.