AI "perceptive assistance" during tele-operations of rovers in Deep Space

The SEED program (academic track)
Co-funded by the European Union under Grant Agreement no. 101126644

PhD topic open for applications until January 31st, 2024

Download PDF



1. Definition
2. Partners and study periods


1. Definition

1.1. Domain and scientific/technical context

Extreme conditions of spatial missions (Moon or Mars) will lead to a number of remotely controlled operations of rovers, especially for exploratory and construction activities.

In these conditions, environmental sensing will likely be achieved through on board cameras and positioning systems. Inside a rover, interfaces based on an XR environment could be implemented to ensure consistent situational awareness for the human operator.

Similar issues can be found in the context of scuba diving, providing an easier experimental background.

1.2. Scientific/technical challenges

The following scientific challenges need to be addressed for the development of this XR environment:

  • In natural viewing conditions, head turns provide valuable proprioceptive information about perception of space. In an XR environment, how does a limited field of view and non-naturalistic implementations of head position impact global situational awareness?
  • The latency due to distance (1.25s between the Earth and the Moon at light speed) will most probably also impact the human operator's situational awareness but also induce cybersickness. How can it be mitigated?
  • The human operator will necessarily be protected from the zero gravity external environment. This will mean that they will not have access to the perceptual and proprioceptive feedback they would ordinarily use on Earth. We will investigate if combining sensory feedback across multiple modalities can support situational awareness, and if an AI agent could be used to assess the most effective combination of sensory and proprioceptive feedback in a given scenario.

1.3. Considered methods, targeted results and impacts

The impact of the technical constraints of XR teleoperation on user perception will first be studied by means of user experiences. This is to elicit the key elements specific to this type of environment. These user studies in simulated virtual environments will highlight and precisely assess the impact of these limitations on perception.

The second part of the program will then focus on proposing technical aids to compensate for these biases. It will include simulation experiments, based on sensory compensations but also AI-based, using agents or state prediction.

Finally, the results of these studies can be tested under real-life conditions in an underwater environment, where technical constraints are similar to experimental conditions in space (limited sensors, weak effect of gravity).

These results will contribute to the improvement of teleoperations devices in deteriorated telecommunication conditions (space, submarines, poor network coverage) and with specific physical constraints (zero or low gravity, high drag, limited sensors).

1.4. Interdisciplinarity aspects

This project will leverage Prof Moreau and Dr Peillard’s expertise in XR and human behaviour, Coppin’s experience in human-autonomy teaming and human factors, and Ma-Wyatt’s experience in human factors, sensorimotor control and eye movements. In this thesis topic, the candidate will have the opportunity to work with all of these experts to devise new forms of experiments and XR that enable quantification and prediction of human performance in extreme environments off earth (or undersea).

1.5. References

  • Benaoumeur, Ibari, Ahmed-foitih Zoubir, et Hanifi Elhachimi Amar Reda. « Remote Control of Mobile Robot Using the Virtual Reality » 5, no 5 (2015).
  • Brunnström, Kjell, Elijs Dima, Tahir Qureshi, Mathias Johanson, Mattias Andersson, et Mårten Sjöström. « Latency impact on Quality of Experience in a virtual reality simulator for remote control of machines ». Signal Processing: Image Communication 89 (1 novembre 2020): 116005.
  • Nguyen, Laurent A., Maria Bualat, Laurence J. Edwards, Lorenzo Flueckiger, Charles Neveu, Kurt Schwehr, Michael D. Wagner, et Eric Zbinden. « Virtual Reality Interfaces for Visualization and Control of Remote Vehicles ». Autonomous Robots 11, no 1 (Jul. 2001): 59‐68.
  • Richardson, Barry, Mark Symmons, et Dianne Wuillemin. « The Contribution of Virtual Reality to Research on Sensory Feedback in Remote Control ». Virtual Reality 9, no 4 (1 avril 2006): 234‐42.
  • Tittle, James S., Axel Roesler, et David D. Woods. « The Remote Perception Problem ». Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46, no 3 (Sep. 2002): 260‐64.

2. Partners and study periods

2.1. Supervisors and study periods

2.2. Hosting organizations

2.2.1. IMT Atlantique

IMT Atlantique, internationally recognized for the quality of its research, is a leading French technological university under the supervision of the Ministry of Industry and Digital Technology. IMT Atlantique maintains privileged relationships with major national and international industrial partners, as well as with a dense network of SMEs, start-ups, and innovation networks. With 290 permanent staff, 2,200 students, including 300 doctoral students, IMT Atlantique produces 1,000 publications each year and raises 18€ million in research funds.

2.2.2. University of Aidelaide

At the University of Adelaide, we pursue meaningful change as we celebrate our proud history: applying proven values in the pursuit of contemporary educational and research excellence; meeting our local and global community’s evolving needs and challenges; and striving to prepare our graduates for their aspirations and the needs of the future workforce.

Our focus is informed by the manifold changes confronting today’s society, including the:

  • need for economic transition—to new industries and jobs
  • imperative of social transformation—demanding more accessible, lifelong learning
  • impact of globalisation—making global opportunities available locally
  • pervasive nature of technological disruption—redefining socio-economic constructs
  • pursuit of sustainability—socially, economically and environmentally.