Robot state in spatial context
Investigate an operator interface that relates robot condition, sensor measurements and camera information to the surrounding scene. This could support a more connected understanding of remote equipment and its environment.
Potential benefits
- Bring robot condition and environmental observations into a connected view.
- Reduce switching between separate state displays and camera feeds.
- Help operators recognise information that needs checking before an action.
Relevant challenges
Remote Support and Collaboration
- Experts may need access to live machine data at the same time as video or spatial context
Inspection and Monitoring
- Data from drones, robots, cameras, and sensors may exist in separate systems
How the approach works
Selected robot-state and sensor streams are associated with the robot or relevant scene locations. The interface would coordinate those displays with the live camera view and navigation context.
Research and demonstrations
The robot and cloud framework provides a communication architecture for immersive remote operation. A combined robot-state and sensor interface would extend that foundation through task-specific data integration and evaluation.
Adapting it to your workflow
A project would define the required state variables and sensors, agree their meaning and timing and develop a consistent display across data sources. Sensor and state fusion would be evaluated against the target task.
What a project could evaluate
A project could assess whether operators interpret robot condition accurately, relate measurements to the scene and recognise when information is delayed or inconsistent.

