MENTOR, Wearable AR Projector
Improving usability in augmented training systems.
Politecnico di Milano Master's Thesis for 2021/2022.
Developed a new approach to wearable AR in manufacturing facilities.
( TL;DR )
In a hurry? Just read this section.
PROBLEM
Existing AR systems for on-site training are cumbersome, especially in manufacturing environments. Maintenance technicians lack effective training tools.
APPROACH
Investigated wearable solutions, focusing on comfort and integration with existing personal protective equipment (PPE). Introduced projection mapping for an innovative AR experience.
SOLUTION
Developed MENTOR, a chest-worn AR device. Integrated cameras, depth sensors and projectors for immersive training. Overcame usability challenges with a tilt feature to ensure adaptability to both vertical and horizontal surfaces.
RESULTS
Developed a new approach to wearable AR. The versatility of this solution accommodates different workplace scenarios, promising potential widespread adoption in manufacturing facilities.
Project statement.
AR systems rely on head-mounted devices or handheld screens that can be cumbersome and uncomfortable. This inefficiency and discomfort currently prevents the implementation of augmented training platforms in manufacturing plants.
Efficient workplace training for experienced problem-solvers.
This solution is aimed at maintenance technicians, whose experience on the factory floor is proving to be more of a differentiator than in other industrial roles.
They have the ability to identify equipment malfunctions and quickly develop a strategy to solve them based on prior knowledge. This is a skill that takes years to acquire and master.
Maintenance technicians' protective gear.
From insight to visual exploration.
Current systems are poorly suited to the equipment and workflow, so I visually explored potential solutions around the themes of wearability and integration with the 'above the neck' PPEs.
Wearable projection mapping: an unconventional approach to augmented reality.
The use of projection mapping as an augmented reality technique marks a significant change in direction for this project. Previous solutions have relied on variations of AR headsets, which have known problems.
To effectively introduce this technology into a high-efficiency environment, I had to think creatively. This led me to explore projection mapping, which is a technique that involves projecting light directly onto real surfaces.
Although scaling this down to a wearable version presented technical challenges, the potential benefits in terms of design, user experience, and engineering made it a compelling avenue for further exploration in this project.
MENTOR is a chest-worn device that provides an immersive experience for the wearer.
The system utilizes cameras and depth sensors to digitally recreate the surrounding environment. Then, an integrated projector is used to place augmented reality information on real-world surfaces, using the virtual replica as a canvas.
( User Flow )
CONTENT FRUITION
CONTENT CREATION
The problem of field projection on horizontal planes.
Excluding half of the tasks from the use cases would make the solution ineffective and therefore not worth implementing for a company.
Introducing a tilt function to cover all use cases.
The challenge was met by adding an extra degree of freedom. By integrating a hinge into the connection between the chassis and the chest mount, the entire structure gains the ability to tilt forward when required, aligning the projector and sensor with the horizontal working plane.
This change allows Mentor to switch easily between configurations to suit different stages of the job. When working vertically, the unit remains securely attached to the chest, facing forward. When working horizontally, however, the entire assembly rotates autonomously up to forty degrees. This allows for optimum scanning and illumination of the surface.
