Immersive robot teleoperation
Develop remote controls and immersive views around the robot’s task. A project could evaluate how movement inputs and feedback help operators guide the equipment.
EIRSAll demos · Industrial challenges
An operator can guide a mobile robot through places that are difficult or unsafe to access and view its live camera feed in virtual reality (VR). Images tied to locations in the scene support later specialist review. The framework connects live robot control, spatial context and detailed follow-up, with opportunities to adapt it to other robots and capture equipment.

Some environments are difficult or unsafe for people to access. This can include industrial spaces that are hard to reach, hazardous or accident prone areas, remote infrastructure and places beyond practical reach. Mobile robots can carry cameras into these spaces and give operators a live view while they guide movement. Captured views and marked points of interest help specialists examine details after the live inspection.
The current testbed connects a Unitree AlienGo quadruped robot and its onboard cameras to a Unity VR application. Operators use a Meta Quest Pro headset with an Omnideck VR omnidirectional treadmill or a joystick for movement. Head movement or joystick input can change the camera view. A Matterport LiDAR scan, a laser-scanned spatial model, provides the virtual scene for navigation and spatial context.
The framework links the operator interface, robot communication and inspection review. A Matterport LiDAR scan is imported into Unity to provide a spatial model. Live camera video gives the operator a view from the robot, while the reconstructed environment provides spatial context. Captured scene information can also be explored later for detailed review.
Three layers built around ROS, the Robot Operating System, connect the robot and immersive application. Robot nodes handle sensing and movement. Middleware translates device data and commands. Containerised services in a Docker-based private cloud coordinate communication between components. Movement commands travel from the VR application to the robot, while live video and scene updates return to the operator.
The operator can save geotagged screenshots, which attach an observation to a location in the scene. These points of interest can be shared through a central platform so a specialist can review selected details and recommend follow-up.
The operator guides movement with the Omnideck or a joystick and changes the camera view using head movement or joystick input. Live video supports inspection during navigation. When a detail needs further attention, the operator captures an image linked to its scene location. A specialist can then examine that observation alongside the reconstructed environment.
Teams can use this approach to assess areas that are hard or unsafe to reach, guide a robot toward points that need closer inspection and share selected images with specialists. It could support inspections in restricted process spaces, remote infrastructure, accident prone locations and emergency response. A site adaptation can align robot controls, camera views and evidence handover with the inspection workflow.
Further development could connect additional ground robots and drones, add camera and sensing options such as thermal imaging and LiDAR, and support multiple live viewpoints and shared specialist review. Site adaptations could also align the control interface, safety procedures and evidence handover with each team’s needs.
Remote inspection involves more than getting a camera into a difficult location. Operators need to understand their surroundings, direct movement, inspect details and communicate useful observations to others.
Develop remote controls and immersive views around the robot’s task. A project could evaluate how movement inputs and feedback help operators guide the equipment.
Combine live exploration with detailed review of captured environments. The approach could be adapted to locations where access is difficult, hazardous or impractical.
Connect camera viewpoints and controls with what operators need to inspect. A project could compare ways of directing the view while maintaining awareness of the surrounding space.
Associate important images and points of interest with their location in the inspected environment. This can provide a foundation for specialist review, reporting and follow-up.
A collaboration could connect the relevant robot, cameras and inspection procedures, then evaluate the complete workflow from remote operation to specialist review.
Explore research and collaboration →A VR operator guides a mobile robot and monitors its camera views.
Explore a reconstructed tunnel environment in VR.









This work is part of DDIA, Data Driven Immersive Analytics in Digital Industries.
DDIA is an FFG COMET Module (2022 - 2026) at Know Center Research GmbH.