Connected virtual machine controls
Connect a virtual control to a physical machine action and return visible state information to the user. This approach links immersive interaction with a control and feedback loop around real equipment.
Potential benefits
- Perform selected checks or adjustments without walking to the equipment.
- Bring machine actions and returned state into one interface.
- Make equipment behaviour easier to inspect during an interaction.
Relevant challenges
Operations
- Operators must physically walk to equipment for simple checks or actions
- Remote monitoring is possible, but remote action is limited
How the approach works
An immersive interface can act as a virtual human–machine interface (HMI), mapping selected buttons, rotary inputs or parameter changes to equipment functions. Commands travel through a control interface, and machine-state updates return to the digital views so the user can inspect the response.
Research and demonstrations
The bidirectional virtual reality (VR) and augmented reality (AR) control example operates the physical JuiceMixer through virtual buttons and a rotary control. The connected applications reflect returned machine state, providing a working local example of this two-way interaction.
Adapting it to your workflow
A project would connect the target machine’s control interface. Depending on the equipment, this could involve an application programming interface (API), a programmable logic controller (PLC) or an OPC UA connection for industrial data exchange. These are project-dependent integration options rather than interfaces established by the current testbed.
The work would define permitted actions and implement the required permissions, interlocks, feedback and failure handling. Integration and operating procedures would be verified on the target system.
What a project could evaluate
A project could measure command and feedback timing, correctness of the returned state, reliability and how well operators understand and manage the selected actions. A pilot could compare task time and trips to the equipment, alongside command response and operator understanding.

