Sustainable Engineering Paradigm Shift in Digital Architecture, Engineering and Construction Ecology within Metaverse

In the post COVID 19 pandemic, the demand for virtual world and digital economy accelerated and became more popular and the term Metaverse is now a hot topic in different sectors in the community and society. Digital technology development in augmented reality (AR), virtual reality (VR), and networks has become more mature in recent years, the racing of the application of Metaverse in different aspects is more vigorous. Metaverse in digital architectural, engineering and construction being one of the major players in future should not be overlooked. More understanding of Metaverse which includes the Architecture, Engineering and Construction (AEC) industry is crucial and this is important for stakeholders in the AEC industry to start early development to match with the quick development, expansion and global trend of Metaverse.

Palestine Smart Tourism Augmented Reality Mobile Application

Tourism is considered an important sector for most countries, while maintaining good tourism attractions can promote national economic development. The State of Palestine is historically considered a wealthy country full of many archaeological places. In the city of Bethlehem, for example, the Church of the Nativity is the most important touristic site, but it does not have enough technology development to attract tourists. In this paper, we propose a smart mobile application named “Pal-STAR” (Palestine Smart Tourist Augmented Reality) as an innovative solution which targets tourists and assists them to make a visit inside the Church of the Nativity. The application will use augmented reality and feature a virtual tourist guide showing views of the church while providing historical information in a smart, easy, effective and user-friendly way. The proposed application is compatible with multiple mobile platforms and is considered user friendly. The findings show that this application will improve the practice of the tourism sector in the Holy Land, it will also increase the number of tourists visiting the Church of the Nativity and it will facilitate access to historical data that have been difficult to obtain using traditional tourism guidance. The value that tourism adds to a country cannot be denied, and the more technological advances are incorporated in this sector, the better the country’s tourism sector can be served. Palestine’s economy is heavily dependent on tourism in many of its main cities, despite several limitations, and technological development is needed to enable this sector to flourish. The proposed mobile application would definitely have a good impact on the development of the tourism sector by creating an Augmented Reality environment for tourists inside the church, helping them to navigate and learn about holy places in a non-traditional way, using a virtual tourist guide.

Method for Auto-Calibrate Projector and Color-Depth Systems for Spatial Augmented Reality Applications

Spatial Augmented Reality is a variation of Augmented Reality where the Head-Mounted Display is not required. This variation of Augmented Reality is useful in cases where the need for a Head-Mounted Display itself is a limitation. To achieve this, Spatial Augmented Reality techniques substitute the technological elements of Augmented Reality; the virtual world is projected onto a physical surface. To create an interactive spatial augmented experience, the application must be aware of the spatial relations that exist between its core elements. In this case, the core elements are referred to as a projection system and an input system, and the process to achieve this spatial awareness is called system calibration. The Spatial Augmented Reality system is considered calibrated if the projected virtual world scale is similar to the real-world scale, meaning that a virtual object will maintain its perceived dimensions when projected to the real world. Also, the input system is calibrated if the application knows the relative position of a point in the projection plane and the RGB-depth sensor origin point. Any kind of projection technology can be used, light-based projectors, close-range projectors, and screens, as long as it complies with the defined constraints; the method was tested on different configurations. The proposed procedure does not rely on a physical marker, minimizing the human intervention on the process. The tests are made using a Kinect V2 as an input sensor and several projection devices. In order to test the method, the constraints defined were applied to a variety of physical configurations; once the method was executed, some variables were obtained to measure the method performance. It was demonstrated that the method obtained can solve different arrangements, giving the user a wide range of setup possibilities.

The OLOS® Way to Cultural Heritage: User Interface with Anthropomorphic Characteristics

Augmented Reality and Augmented Intelligence are radically changing information technology. The path that starts from the keyboard and then, passing through milestones such as Siri, Alexa and other vocal avatars, reaches a more fluid and natural communication with computers, thus converting the dichotomy between man and machine into a harmonious interaction, now heads unequivocally towards a new IT paradigm, where holographic computing will play a key role. The OLOS® platform contributes substantially to this trend in that it infuses computers with human features, by transferring the gestures and expressions of persons of flesh and bones to anthropomorphic holographic interfaces which in turn will use them to interact with real-life humans. In fact, we could say, boldly but with a solid technological background to back the statement, that OLOS® gives reality to an altogether new entity, placed at the exact boundary between nature and technology, namely the holographic human being. Holographic humans qualify as the perfect carriers for the virtual reincarnation of characters handed down from history and tradition. Thus, they provide for an innovative and highly immersive way of experiencing our cultural heritage as something alive and pulsating in the present.

Authentication of Physical Objects with Dot-Based 2D Code

Counterfeit goods and documents are a global problem, which needs more and more sophisticated methods of resolving it. Existing techniques using watermarking or embedding symbols on objects are not suitable for all use cases. To address those special needs, we created complete system allowing authentication of paper documents and physical objects with flat surface. Objects are marked using orientation independent and resistant to camera noise 2D graphic codes, named DotAuth. Based on the identifier stored in 2D code, the system is able to perform basic authentication and allows to conduct more sophisticated analysis methods, e.g., relying on augmented reality and physical properties of the object. In this paper, we present the complete architecture, algorithms and applications of the proposed system. Results of the features comparison of the proposed solution and other products are presented as well, pointing to the existence of many advantages that increase usability and efficiency in the means of protecting physical objects.

Vr-GIS and Ar-GIS In Education: A Case Study

ICT tools and platforms endorse more and more educational process. Many models and techniques for people to be educated and trained about specific topics and skills do exist, as classroom lectures with textbooks, computers, handheld devices and others. The choice to what extent ICT is applied within learning contexts is related to personal access to technologies as well as to the infrastructure surrounding environment. Among recent techniques, the adoption of Virtual Reality (VR) and Augmented Reality (AR) provides significant impulse in fully engaging users senses. In this paper, an application of AR/VR within Geographic Information Systems (GIS) context is presented. It aims to provide immersive environment experiences for educational and training purposes (e.g. for civil protection personnel), useful especially for situations where real scenarios are not easily accessible by humans. First acknowledgments are promising for building an effective tool that helps civil protection personnel training with risk reduction.

Multi-Modal Visualization of Working Instructions for Assembly Operations

Growing individualization and higher numbers of variants in industrial assembly products raise the complexity of manufacturing processes. Technical assistance systems considering both procedural and human factors allow for an increase in product quality and a decrease in required learning times by supporting workers with precise working instructions. Due to varying needs of workers, the presentation of working instructions leads to several challenges. This paper presents an approach for a multi-modal visualization application to support assembly work of complex parts. Our approach is integrated within an interconnected assistance system network and supports the presentation of cloud-streamed textual instructions, images, videos, 3D animations and audio files along with multi-modal user interaction, customizable UI, multi-platform support (e.g. tablet-PC, TV screen, smartphone or Augmented Reality devices), automated text translation and speech synthesis. The worker benefits from more accessible and up-to-date instructions presented in an easy-to-read way.

An Efficient Motion Recognition System Based on LMA Technique and a Discrete Hidden Markov Model

Human motion recognition has been extensively increased in recent years due to its importance in a wide range of applications, such as human-computer interaction, intelligent surveillance, augmented reality, content-based video compression and retrieval, etc. However, it is still regarded as a challenging task especially in realistic scenarios. It can be seen as a general machine learning problem which requires an effective human motion representation and an efficient learning method. In this work, we introduce a descriptor based on Laban Movement Analysis technique, a formal and universal language for human movement, to capture both quantitative and qualitative aspects of movement. We use Discrete Hidden Markov Model (DHMM) for training and classification motions. We improve the classification algorithm by proposing two DHMMs for each motion class to process the motion sequence in two different directions, forward and backward. Such modification allows avoiding the misclassification that can happen when recognizing similar motions. Two experiments are conducted. In the first one, we evaluate our method on a public dataset, the Microsoft Research Cambridge-12 Kinect gesture data set (MSRC-12) which is a widely used dataset for evaluating action/gesture recognition methods. In the second experiment, we build a dataset composed of 10 gestures(Introduce yourself, waving, Dance, move, turn left, turn right, stop, sit down, increase velocity, decrease velocity) performed by 20 persons. The evaluation of the system includes testing the efficiency of our descriptor vector based on LMA with basic DHMM method and comparing the recognition results of the modified DHMM with the original one. Experiment results demonstrate that our method outperforms most of existing methods that used the MSRC-12 dataset, and a near perfect classification rate in our dataset.

Cities Simulation and Representation in Locative Games from the Perspective of Cultural Studies

This work aims to analyze the locative structure used by the locative games of the company Niantic. To fulfill this objective, a literature review on the representation and simulation of cities was developed; interviews with Ingress players and playing Ingress. Relating these data, it was possible to deepen the relationship between the virtual and the real to create the simulation of cities and their cultural objects in locative games. Cities representation associates geo-location provided by the Global Positioning System (GPS), with augmented reality and digital image, and provides a new paradigm in the city interaction with its parts and real and virtual world elements, homeomorphic to real world. Bibliographic review of papers related to the representation and simulation study and their application in locative games was carried out and is presented in the present paper. The cities representation and simulation concepts in locative games, and how this setting enables the flow and immersion in urban space, are analyzed. Some examples of games are discussed for this new setting development, which is a mix of real and virtual world. Finally, it was proposed a Locative Structure for electronic games using the concepts of heterotrophic representations and isotropic representations conjoined with immediacy and hypermediacy.

Augmenting History: Case Study Measuring Motivation of Students Using Augmented Reality Apps in History Classes

Due to the rapid advances in the use of information technology and students’ familiarity with technology, learning styles in higher education are being reshaped. One of the technology developments that has gained considerable attention in recent years is Augmented Reality (AR), where technology is used to combine overlays of digital data on physical real-world settings. While AR is being heavily promoted for entertainment by mobile phone manufacturers, it has had little adoption in higher education due to the required upfront investment that an instructor needs to undertake in creating relevant AR applications. This paper discusses a case study that uses a low upfront development approach and examines the impact on generation-Z students’ motivation whilst studying design history over a four-semester period. Even though the upfront investment in creating the AR support was minimal, the results showed a noticeable increase in student motivation. The approach used in this paper can be easily transferred to other disciplines and other areas of design education.

Applying Augmented Reality Technology for an E-Learning System

Over the past 20 years, technology was rapidly developed and no one expected what will come next. Advancements in technology open new opportunities for immersive learning environments. There is a need to transmit education to a level that makes it more effective for the student. Augmented reality is one of the most popular technologies these days. This paper is an experience of applying Augmented Reality (AR) technology using a marker-based approach in E-learning system to transmitting virtual objects into the real-world scenes. We present a marker-based approach for transmitting virtual objects into real-world scenes to explain information in a better way after we developed a mobile phone application. The mobile phone application was then tested on students to determine the extent to which it encouraged them to learn and understand the subjects. In this paper, we talk about how the beginnings of AR, the fields using AR, how AR is effective in education, the spread of AR these days and the architecture of our work. Therefore, the aim of this paper is to prove how creating an interactive e-learning system using AR technology will encourage students to learn more.

Simplified Mobile AR Platform Design for Augmented Tourism

This study outlines iterations of designing mobile augmented reality (MAR) applications for tourism specific contexts. Using a design based research model, several cycles of development to implementation were analyzed and refined upon with the goal of building a MAR platform that would facilitate the creation of augmented tours and environments by non-technical users. The project took on several stages, and through the process, a simple framework was begun to be established that can inform the design and use of MAR applications for tourism contexts. As a result of these iterations of development, a platform was developed that can allow novice computer users to create augmented tourism environments. This system was able to connect existing tools in widespread use such as Google Forms and connect them to computer vision algorithms needed for more advanced augmented tourism environments. The study concludes with a discussion of this MAR platform and reveals design elements that have implications for tourism contexts. The study also points to future case uses and design approaches for augmented tourism.

An Overview of Evaluations Using Augmented Reality for Assembly Training Tasks

Augmented Reality (AR) is a strong growing research topic in different training domains such as medicine, sports, military, education and industrial use cases like assembly and maintenance tasks. AR claims to improve the efficiency and skill-transfer of training tasks. This paper gives a comprehensive overview of evaluations using AR for assembly and maintenance training tasks published between 1992 and 2017. We search in a structured way in four different online databases and get 862 results. We select 17 relevant articles focusing on evaluating AR-based training applications for assembly and maintenance tasks. This paper also indicates design guidelines which are necessary for creating a successful application for an AR-based training. We also present five scientific limitations in the field of AR-based training for assembly tasks. Finally, we show our approach to solve current research problems using Design Science Research (DSR).

Augmented Reality Sandbox and Constructivist Approach for Geoscience Teaching and Learning

Augmented reality sandbox adds new dimensions to education and learning process. It can be a core component of geoscience teaching and learning to understand the geographic contexts and landform processes. Augmented reality sandbox is a useful tool not only to create an interactive learning environment through spatial visualization but also it can provide an active learning experience to students and enhances the cognition process of learning. Augmented reality sandbox can be used as an interactive learning tool to teach geomorphic and landform processes. This article explains the augmented reality sandbox and the constructivism approach for geoscience teaching and learning, and endeavours to explore the ways to teach the geographic processes using the three-dimensional digital environment for the deep learning of the geoscience concepts interactively.

Augmenting Navigational Aids: The Development of an Assistive Maritime Navigation Application

On the bridge of a ship the officers are looking for visual aids to guide navigation in order to reconcile the outside world with the position communicated by the digital navigation system. Aids to navigation include: Lighthouses, lightships, sector lights, beacons, buoys, and others. They are designed to help navigators calculate their position, establish their course or avoid dangers. In poor visibility and dense traffic areas, it can be very difficult to identify these critical aids to guide navigation. The paper presents the usage of Augmented Reality (AR) as a means to present digital information about these aids to support navigation. To date, nautical navigation related mobile AR applications have been limited to the leisure industry. If proved viable, this prototype can facilitate the creation of other similar applications that could help commercial officers with navigation. While adopting a user centered design approach, the team has developed the prototype based on insights from initial research carried on board of several ships. The prototype, built on Nexus 9 tablet and Wikitude, features a head-up display of the navigational aids (lights) in the area, presented in AR and a bird’s eye view mode presented on a simplified map. The application employs the aids to navigation data managed by Hydrographic Offices and the tablet’s sensors: GPS, gyroscope, accelerometer, compass and camera. Sea trials on board of a Navy and a commercial ship revealed the end-users’ interest in using the application and further possibility of other data to be presented in AR. The application calculates the GPS position of the ship, the bearing and distance to the navigational aids; all within a high level of accuracy. However, during testing several issues were highlighted which need to be resolved as the prototype is developed further. The prototype stretched the capabilities of Wikitude, loading over 500 objects during tests in a major port. This overloaded the display and required over 45 seconds to load the data. Therefore, extra filters for the navigational aids are being considered in order to declutter the screen. At night, the camera is not powerful enough to distinguish all the lights in the area. Also, magnetic interference with the bridge of the ship generated a continuous compass error of the AR display that varied between 5 and 12 degrees. The deviation of the compass was consistent over the whole testing durations so the team is now looking at the possibility of allowing users to manually calibrate the compass. It is expected that for the usage of AR in professional maritime contexts, further development of existing AR tools and hardware is needed. Designers will also need to implement a user-centered design approach in order to create better interfaces and display technologies for enhanced solutions to aid navigation.

Enhanced Planar Pattern Tracking for an Outdoor Augmented Reality System

In this paper, a scalable augmented reality framework for handheld devices is presented. The presented framework is enabled by using a server-client data communication structure, in which the search for tracking targets among a database of images is performed on the server-side while pixel-wise 3D tracking is performed on the client-side, which, in this case, is a handheld mobile device. Image search on the server-side adopts a residual-enhanced image descriptors representation that gives the framework a scalability property. The tracking algorithm on the client-side is based on a gravity-aligned feature descriptor which takes the advantage of a sensor-equipped mobile device and an optimized intensity-based image alignment approach that ensures the accuracy of 3D tracking. Automatic content streaming is achieved by using a key-frame selection algorithm, client working phase monitoring and standardized rules for content communication between the server and client. The recognition accuracy test performed on a standard dataset shows that the method adopted in the presented framework outperforms the Bag-of-Words (BoW) method that has been used in some of the previous systems. Experimental test conducted on a set of video sequences indicated the real-time performance of the tracking system with a frame rate at 15-30 frames per second. The presented framework is exposed to be functional in practical situations with a demonstration application on a campus walk-around.

Augmented Reality and Storytelling in Cities: An Application to Lisbon Street Art

Cities are spaces of memory with several zones (parts of cities) with their own history and cultural events. Today, cities are also marked by a form of intangible cultural heritage like street art, which creates a visual culture based on the process of reflection about the city and the world. To link these realities and create a personal user interaction with this cultural heritage it is important to capture the story and aesthetics, and find alternatives to immerse the user in these spaces of memory. To that end, this article presents a project which combines Augmented Reality technologies and concepts of Transmedia Storytelling applied to Lisbon City, using Street Art artifacts as markers in a framework of digital media-art.

Exploring Pisa Monuments Using Mobile Augmented Reality

Augmented Reality (AR) has taken a big leap with the introduction of mobile applications which co-locate bi-dimensional (e.g. photo, video, text) and tridimensional information with the location of the user enriching his/her experience. This study presents the advantages of using Mobile Augmented Reality (MAR) technologies in traveling applications, improving cultural heritage exploration. We propose a location-based AR application which combines co-location with the augmented visual information about Pisa monuments to establish a friendly navigation in this historic city. AR was used to render contextual visual information in the outdoor environment. The developed Android-based application offers two different options: it provides the ability to identify the monuments positioned close to the user’s position and it offers location information for getting near the key touristic objectives. We present the process of creating the monuments’ 3D map database and the navigation algorithm.

Mobile Augmented Reality for Collaboration in Operation

Mobile augmented reality (MAR) tracking targets from the surroundings and aids operators for interactive data and procedures visualization, potential equipment and system understandably. Operators remotely communicate and coordinate with each other for the continuous tasks, information and data exchange between control room and work-site. In the routine work, distributed control system (DCS) monitoring and work-site manipulation require operators interact in real-time manners. The critical question is the improvement of user experience in cooperative works through applying Augmented Reality in the traditional industrial field. The purpose of this exploratory study is to find the cognitive model for the multiple task performance by MAR. In particular, the focus will be on the comparison between different tasks and environment factors which influence information processing. Three experiments use interface and interaction design, the content of start-up, maintenance and stop embedded in the mobile application. With the evaluation criteria of time demands and human errors, and analysis of the mental process and the behavior action during the multiple tasks, heuristic evaluation was used to find the operators performance with different situation factors, and record the information processing in recognition, interpretation, judgment and reasoning. The research will find the functional properties of MAR and constrain the development of the cognitive model. Conclusions can be drawn that suggest MAR is easy to use and useful for operators in the remote collaborative works.

Augmented Reality for Maintenance Operator for Problem Inspections

Current production-oriented factories need maintenance operators to work in shifts monitoring and inspecting complex systems and different equipment in the situation of mechanical breakdown. Augmented reality (AR) is an emerging technology that embeds data into the environment for situation awareness to help maintenance operators make decisions and solve problems. An application was designed to identify the problem of steam generators and inspection centrifugal pumps. The objective of this research was to find the best medium of AR and type of problem solving strategies among analogy, focal object method and mean-ends analysis. Two scenarios of inspecting leakage were temperature and vibration. Two experiments were used in usability evaluation and future innovation, which included decision-making process and problem-solving strategy. This study found that maintenance operators prefer build-in magnifier to zoom the components (55.6%), 3D exploded view to track the problem parts (50%), and line chart to find the alter data or information (61.1%). There is a significant difference in the use of analogy (44.4%), focal objects (38.9%) and mean-ends strategy (16.7%). The marked differences between maintainers and operators are of the application of a problem solving strategy. However, future work should explore multimedia information retrieval which supports maintenance operators for decision-making.