Conversational AR support at the machine
Ask for assistance while working at equipment and receive a response in the augmented scene. Conversational augmented reality (AR) can connect approved procedural knowledge with relevant task locations.
Potential benefits
- Keep procedural assistance available while the operator remains at the equipment.
- Make instructions easier to relate to the task location.
- Reduce interruptions caused by searching for relevant guidance.
Relevant challenges
Maintenance and Troubleshooting
- Step-by-step maintenance procedures are difficult to follow while working physically on equipment
Knowledge Management
- Existing knowledge bases are difficult to use while performing a physical task
How the approach works
An AR interface combines spoken interaction, displayed instructions and spatial cues. Reference documents and approved expert knowledge provide the content, while equipment alignment connects guidance with selected task locations.
Research and demonstrations
The conversational assistant runs on HoloLens 2 around the JuiceMixer testbed. It provides spoken and displayed responses with spatial cues during procedural tasks. This is a foundation for further context-aware assistance using additional information sources.
Adapting it to your workflow
A project would prepare the target documentation and task knowledge, align cues with the machine and test the relevant workflows. Response timing and answer quality would be evaluated. Camera-based context recognition would be a further development where required.
What a project could evaluate
A project could assess whether guidance is relevant to the task, spatial cues are understandable and response delays fit the pace of the work. A comparison could measure information-search interruptions and the time needed to resume the task.

