Multi-Modal Visualization of Working Instructions for Assembly Operations

Growing individualization and higher numbers of
variants in industrial assembly products raise the complexity of
manufacturing processes. Technical assistance systems considering
both procedural and human factors allow for an increase in product
quality and a decrease in required learning times by supporting
workers with precise working instructions. Due to varying needs of
workers, the presentation of working instructions leads to several
challenges. This paper presents an approach for a multi-modal
visualization application to support assembly work of complex parts.
Our approach is integrated within an interconnected assistance system
network and supports the presentation of cloud-streamed textual
instructions, images, videos, 3D animations and audio files along
with multi-modal user interaction, customizable UI, multi-platform
support (e.g. tablet-PC, TV screen, smartphone or Augmented Reality
devices), automated text translation and speech synthesis. The worker
benefits from more accessible and up-to-date instructions presented
in an easy-to-read way.

Authors:

• Josef Wolfartsberger
• Michael Heiml
• Georg Schwarz
• Sabrina Egger

Keywords:

• Assembly
• assistive technologies
• augmented reality
• manufacturing
• visualization.

References:

[1] S. B¨uttner, H. Mucha, M. Funk, T. Kosch, M. Aehnelt, S. Robert,
and C. R¨ocker, “The design space of augmented and virtual reality
applications for assistive environments in manufacturing: A visual
approach,” in Proceedings of the 10th International Conference on
PErvasive Technologies Related to Assistive Environments, ser. PETRA
’17. New York, NY, USA: ACM, 2017, pp. 433–440.
[2] M. Funk, T. Kosch, and A. Schmidt, “Interactive worker assistance:
comparing the effects of in-situ projection, head-mounted displays,
tablet, and paper instructions,” in Proceedings of the 2016 ACM
International Joint Conference on Pervasive and Ubiquitous Computing
- UbiComp16. ACM Press, 2016.
[3] J. Blattgerste, B. Strenge, P. Renner, T. Pfeiffer, and K. Essig,
“Comparing conventional and augmented reality instructions for manual
assembly tasks,” in Proceedings of the 10th International Conference on
PErvasive Technologies Related to Assistive Environments - PETRA17.
ACM Press, 2017.
[4] S. B¨uttner, M. Funk, O. Sand, and C. R¨ocker, “Using head-mounted
displays and in-situ projection for assistive systems: A comparison,”
in Proceedings of the 9th ACM International Conference on PErvasive
Technologies Related to Assistive Environments, ser. PETRA ’16. New
York, NY, USA: ACM, 2016, pp. 44:1–44:8.
[5] M. Funk, T. Kosch, S. W. Greenwald, and A. Schmidt, “A benchmark
for interactive augmented reality instructions for assembly tasks,” in
Proceedings of the 14th International Conference on Mobile and
Ubiquitous Multimedia - MUM15. ACM Press, 2015.
[6] R. Radkowski, J. Herrema, and J. Oliver, “Augmented reality-based
manual assembly support with visual features for different degrees
of difficulty,” International Journal of Human-Computer Interaction,
vol. 31, no. 5, pp. 337–349, jan 2015.
[7] A. Tang, C. Owen, F. Biocca, and W. Mou, “Comparative effectiveness
of augmented reality in object assembly,” in Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, ser. CHI ’03.
New York, NY, USA: ACM, 2003, pp. 73–80.
[8] M. Funk, A. Bchler, L. Bchler, O. Korn, C. Krieger, T. Heidenreich,
and A. Schmidt, “Comparing projected in-situ feedback at the manual
assembly workplace with impaired workers,” in Proceedings of the 8th
ACM International Conference on PErvasive Technologies Related to
Assistive Environments - PETRA15. ACM Press, 2015.
[9] D. Li, S. Mattsson, A˚ . Fast-Berglund, and M. A˚ kerman, “Testing
operator support tools for a global production strategy,” Procedia CIRP,
vol. 44, pp. 120–125, 2016.
[10] M. Sderberg, Johansson, “Development of simple guidelines to improve
assembly instructions and operator performance,” in Proceedings of the
sixth Swedish Production Symposium. Chalmers Research, 2014.
[11] S. Mattsson and A˚ . Fast-Berglund, “How to support intuition in complex
assembly?” Procedia CIRP, vol. 50, pp. 624–628, 2016.
[12] S. Mattsson, A˚ . Fast-Berglund, and D. Li, “Evaluation of guidelines
for assembly instructions,” IFAC-PapersOnLine, vol. 49, no. 12, pp.
209–214, 2016.
[13] T. Fa¨ssberg, A˚ . Fasth, S. Mattsson, and J. Stahre, “Cognitive automation
in assembly systems for mass customization,” in Proceedings of the 4th
Swedish Production Symposium (SPS), Lund, Sweden, 2011.
[14] M. Agrawala, D. Phan, J. Heiser, J. Haymaker, J. Klingner,
P. Hanrahan, and B. Tversky, “Designing effective step-by-step assembly
instructions,” ACM Transactions on Graphics, vol. 22, no. 3, p. 828, jul
2003.
[15] R. Lindorfer, R. Froschauer, and G. Schwarz, “Adapt - a
decision-model-based approach for modeling collaborative assembly and
manufacturing tasks,” in 2018 IEEE 16th International Conference on
Industrial Informatics (INDIN), 2018, pp. 559–564.
[16] S. Hu, J. Ko, L. Weyand, H. ElMaraghy, T. Lien, Y. Koren, H. Bley,
G. Chryssolouris, N. Nasr, and M. Shpitalni, “Assembly system design
and operations for product variety,” CIRP Annals, vol. 60, no. 2, pp.
715–733, 2011.
[17] L. Gong, D. Li, S. Mattsson, M. A˚ kerman, and A˚ . F. Berglund, “The
comparison study of different operator support tools for assembly task
in the era of global production,” Procedia Manufacturing, vol. 11, pp.
1271–1278, 2017.