Augmented Reality for Maintenance Operator for Problem Inspections

Current production-oriented factories need maintenance operators to work in shifts monitoring and inspecting complex systems and different equipment in the situation of mechanical breakdown. Augmented reality (AR) is an emerging technology that embeds data into the environment for situation awareness to help maintenance operators make decisions and solve problems. An application was designed to identify the problem of steam generators and inspection centrifugal pumps. The objective of this research was to find the best medium of AR and type of problem solving strategies among analogy, focal object method and mean-ends analysis. Two scenarios of inspecting leakage were temperature and vibration. Two experiments were used in usability evaluation and future innovation, which included decision-making process and problem-solving strategy. This study found that maintenance operators prefer build-in magnifier to zoom the components (55.6%), 3D exploded view to track the problem parts (50%), and line chart to find the alter data or information (61.1%). There is a significant difference in the use of analogy (44.4%), focal objects (38.9%) and mean-ends strategy (16.7%). The marked differences between maintainers and operators are of the application of a problem solving strategy. However, future work should explore multimedia information retrieval which supports maintenance operators for decision-making.

Authors:

• Chong-Yang Qiao
• Teeravarunyou Sakol

Keywords:

• Augmented reality
• situation awareness
• decision-making
• problem-solving.

References:

[1] Mekni, M., and Lemieux, A., 2014, “Augmented reality: Applications, challenges and future trends”, In Proceedings of the 13th International Conference on Applied Computer and Applied Computational Science (ACA-COS ‘14), volume 20 of Recent Advances in Computer Engineering Series, pp. 205-215.
[2] S.K, Ong., J, Zhang., Y, Shen., A.Y, Nee., 2011, “Augmented reality in product development and manufacturing”, In Handbook of Augmented Reality (2011), July 13, 2011, New York, NY, USA, pp. 651–669.
[3] P, Fite-Georgel., 2011, “Is there a reality in Industrial Augmented Reality?”, In 2011 10th IEEE International Symposium on Mixed and Augmented Reality, October 26-29, 2011, Basel, Switzerland, pp. 201-210.
[4] Colley, A., Häkkilä, J., Rantakari, J., 2014, “Augmenting the Home to Remember – Initial User Perceptions”, UBICOMP '14 ADJUNCT, Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, September 13 - 17, 2014, Seattle, WA, USA, pp. 1369-1372.
[5] J, Wang., Y, Feng., C, ZENG., S, Li., 2014, “An augmented reality based system for remote collaborative maintenance instruction of complex products”, In IEEE International conference on automation science and engineering (CASE), August 18-22, 2014, Taipei, Taiwan, pp. 309-314.
[6] Newell, B. R., and Bröder, A., 2008, “Cognitive processes, models and metaphors in decision research”, Judgment and Decision Making, Vol. 3, No. 3, pp. 195-204.
[7] Winchester, W., & Ntuen, C., 2008, “Ergonomics of augmented cognition system design and application”, IIE Annual Conference. Proceedings, 1-5. Retrieved from ABI/INFORM Trade & Industry. doi: 185665504.
[8] Flatt, H., Koch, N., R ̈ocker, C., G ̈unter, A & Jasperneite, J., 2015, “A Context-Aware Assistance System for Maintenance Applications in Smart Factories based on Augmented Reality and Indoor Localization”, 2015 IEEE 20th Conference on Emerging Technologies & Factory Automation (ETFA), September 8-11, 2015, Luxembourg, pp. 1-4.
[9] Hou, L., Wang, X., and Truijens, M., 2014, “Using Augmented Reality to Facilitate Piping Assembly: An Experiment-Based Evaluation”, Journal of Computing in Civil Engineering, 2015, 29(1): 05014007.
[10] Shatte, A., Holdsworth, J., Lee, I., 2014, “Hand-held Mobile Augmented Reality for Collaborative Problem Solving: A Case Study with Sorting”, In 47th Hawaii International Conference on System Science, January 6-9, 2014, Waikoloa, HI, pp. 91-99.
[11] Alt, F., Schneegass, S., Girgis, M., Schmidt, A., 2013, “Cognitive effects of interactive public display applications”, Proceedings of the 2nd ACM International Symposium on Pervasive Display, June 04-05, 2013, Mountain View, California, USA, pp. 13-18.
[12] Bell, B., Höllerer, T., and Feiner, S., 2002, “An Annotated Situation-Awareness Aid for Augmented Reality”, In Proc. ACM UIST 2002 (Symp. on User Interface Software and Technology), October 27-30, 2002, Paris, France, pp. 213-216.
[13] Lin, T., Liu, C., Tsai, M., and Kang, S., 2015, “Using Augmented Reality in a Multiscreen Environment for Construction Discussion”, Journal of Computing in Civil Engineering, 2015, 29(6): 04014088.
[14] Jung, D., Jo, S., Myung, R., 2008, “A Study of Relationships between Situation Awareness and Presence that Affect Performance on a Handheld Game Console”, Proceedings of the International Conference on Advances in Computer Entertainment Technology, December 3-5, 2008, Yokohama, Japan, pp. 240-243.
[15] U, Neumann., and A, Majoros., 1998, “Cognitive, Performance, and Systems Issues for Augmented Reality Applications in Manufacturing and Maintenance”, Proc. IEEE Virtual Reality Ann. Int'l Symp. (VRAIS 98), IEEE CS Press, March 14-18, 1998, Atlanta, GA, USA, pp. 4-11.
[16] Song, J., Jia, Q., Sun, H., and Gao, X., 2009, “Study on the Perception Mechanism and Method of Virtual and Real Objects in Augmented Reality Assembly Environment”, in: The 5th IEEE Conference on Industrial Electronics and Applications, Beijing, China, pp. 1452-1456.