Augmented Reality Glasses

John Bjerke
May 22, 2019

Augmented reality (AR) glasses are appearing everywhere. After attending various trade shows and reading articles about their use in manufacturing, we wanted to adapt this technology in a way that would work for us. Our idea was to create an AR or mixed reality device to help with manufacturing, specifically with assembly. The initial goal would be to design a device for locating components and making sure each one is in the right spot. The glasses eventually would expand to wire routing and torque to show connection points and where employees should put their tools. It would also give production employees feedback as they are working—a kind of guided assembly. Employees can do their work and check it at the same time, reducing the need for later inspection.

Our AR Journey

To merge this virtual reality with physical reality, we partnered with Twin Cities Engineering (TCE), Minnesota State University, Mankato. We presented the project idea to the class, and students voted to participate in and execute the project. Over the next 3 months, with TCE leading the way, we worked together to develop software and identify processes so we could use the glasses in assembly.
The first step was to determine project scope and investigate all its requirements. The group tested and reviewed three main types of AR products:
1. Tablets—iPad, Android
2. Projectors—Lightform LMF-1, Werklicht Pro, Light Guide Systems
3. Glasses—Google Glass, Microsoft HoloLens, DreamGlass, ODG R-9
Our requirement for 3D capabilities helped narrow our options. We chose the HoloLens for its range of features and capabilities, such as voice command, hand gestures, storage capacity, and security.

Once we selected our technology, the next steps were determining what production line should be our main focus and configuring the documents. Through weeks of trial and error, we decided to start with jobs that were more static; this would keep the first phase straightforward. Our goal was to familiarize ourselves with the technology so we could use the glasses in jobs with more variability in the future.

In the end, we will have a pair of glasses that can guide an employee through building a panel. The employee can use the technology to complete all steps necessary in the manufacturing process.

This past January (2019), the students had the opportunity to present their hard work at the Minnesota State Capitol. They showed government officials how to use the glasses to assemble a panel, and explained the technology fundamentals behind the process.

Observations
Scope

When launching a project like this, start by defining your scope: what you need and what you hope to achieve. Your available options, features and capabilities are almost endless; without a defined scope, your vision can easily stray or get lost. A clearly defined scope ensures that all team members have a common understanding of your objectives.

Communication

Having open communication with all team members will save time and help you succeed. We ran into some trouble during the design process with sheet metal and a QR code. Sheet metal is reflective, and the glasses didn’t pick up on the features unless the lighting was right. The students came up with a QR code to put on the sheet metal, but its placement was not always consistent. If this had been communicated to everyone during the process, we could have provided other options—such as building a consistent reference point into our assembly table that the lens could pick up easily. (They are working on that now.) We could also have let them know we were open to changing the build process to make it easier for the new technology to be implemented.

Considerations

There are five major things to consider before starting a project like this.

  • What software is available, and what works for what you’re trying to accomplish
  • The conversions that need to take place
  • Defining the “must haves” and the “nice to haves”
  • Specify the type of glasses or AR—for example, whether you need 2D or 3D
  • Establish whether the project will be static or dynamic (this affects complexity)
HoloLens Review

Overall, our experience has been great. Prior knowledge of HoloLens is not mandatory. The software is mostly user friendly, and you can figure things out easily enough. If you’re going to use it in a more dynamic way, you’ll need software and coding experience. The biggest drawback our team found was ergonomics: Wearing them for a full shift causes too much neck strain. As technology improves and they become more common, the glasses will probably be sleeker—making them easier to wear and more efficient to use.

My favorite part of the design process was having so many people at our demonstration in December. It’s fun being an early adopter, and I could see some of them start to think, ”Wow, we could actually use this.” In the future, I can imagine the AR glasses being used in quality control. We could reduce labor requirements and automate more processes, giving employees time to focus on other responsibilities.

For any new project, keep an open mind. Be willing to change your ideas and actions. Just because you have always done something a certain way doesn’t mean you have to keep doing it that way. After you prepare your mind to accept something new, you will be ready to find something better.