EIRS
AR and context-aware AI assistanceSolution portfolio
AR and context-aware AI assistance

Conversational AR support at the machine

Ask for assistance while working at equipment and receive a response in the augmented scene. Conversational augmented reality (AR) can connect approved procedural knowledge with relevant task locations.

Potential benefits

  • Keep procedural assistance available while the operator remains at the equipment.
  • Make instructions easier to relate to the task location.
  • Reduce interruptions caused by searching for relevant guidance.

Relevant challenges

Maintenance and Troubleshooting

  • Step-by-step maintenance procedures are difficult to follow while working physically on equipment

Knowledge Management

  • Existing knowledge bases are difficult to use while performing a physical task

How the approach works

An AR interface combines spoken interaction, displayed instructions and spatial cues. Reference documents and approved expert knowledge provide the content, while equipment alignment connects guidance with selected task locations.

Research and demonstrations

The conversational assistant runs on HoloLens 2 around the JuiceMixer testbed. It provides spoken and displayed responses with spatial cues during procedural tasks. This is a foundation for further context-aware assistance using additional information sources.

Adapting it to your workflow

A project would prepare the target documentation and task knowledge, align cues with the machine and test the relevant workflows. Response timing and answer quality would be evaluated. Camera-based context recognition would be a further development where required.

What a project could evaluate

A project could assess whether guidance is relevant to the task, spatial cues are understandable and response delays fit the pace of the work. A comparison could measure information-search interruptions and the time needed to resume the task.

↑ Top