Scholarly

Sound Selection for Gesture Sonification and Manipulation of Virtual Objects

Year: 2017 Volume: 11 Issue: 1 147 - 152 Pages

Abstract: New sensors and technologies – such as microphones, touchscreens or infrared sensors – are currently making their appearance in the automotive sector, introducing new kinds of Human-Machine Interfaces (HMIs). The interactions with such tools might be cognitively expensive, thus unsuitable for driving tasks. It could for instance be dangerous to use touchscreens with a visual feedback while driving, as it distracts the driver’s visual attention away from the road. Furthermore, new technologies in car cockpits modify the interactions of the users with the central system. In particular, touchscreens are preferred to arrays of buttons for space improvement and design purposes. However, the buttons’ tactile feedback is no more available to the driver, which makes such interfaces more difficult to manipulate while driving. Gestures combined with an auditory feedback might therefore constitute an interesting alternative to interact with the HMI. Indeed, gestures can be performed without vision, which means that the driver’s visual attention can be totally dedicated to the driving task. In fact, the auditory feedback can both inform the driver with respect to the task performed on the interface and on the performed gesture, which might constitute a possible solution to the lack of tactile information. As audition is a relatively unused sense in automotive contexts, gesture sonification can contribute to reducing the cognitive load thanks to the proposed multisensory exploitation. Our approach consists in using a virtual object (VO) to sonify the consequences of the gesture rather than the gesture itself. This approach is motivated by an ecological point of view: Gestures do not make sound, but their consequences do. In this experiment, the aim was to identify efficient sound strategies, to transmit dynamic information of VOs to users through sound. The swipe gesture was chosen for this purpose, as it is commonly used in current and new interfaces. We chose two VO parameters to sonify, the hand-VO distance and the VO velocity. Two kinds of sound parameters can be chosen to sonify the VO behavior: Spectral or temporal parameters. Pitch and brightness were tested as spectral parameters, and amplitude modulation as a temporal parameter. Performances showed a positive effect of sound compared to a no-sound situation, revealing the usefulness of sounds to accomplish the task.

Hand Gesture Interpretation Using Sensing Glove Integrated with Machine Learning Algorithms

Year: 2016 Volume: 10 Issue: 11 1857 - 1861 Pages

Abstract: In this paper, we present a low cost design for a smart glove that can perform sign language recognition to assist the speech impaired people. Specifically, we have designed and developed an Assistive Hand Gesture Interpreter that recognizes hand movements relevant to the American Sign Language (ASL) and translates them into text for display on a Thin-Film-Transistor Liquid Crystal Display (TFT LCD) screen as well as synthetic speech. Linear Bayes Classifiers and Multilayer Neural Networks have been used to classify 11 feature vectors obtained from the sensors on the glove into one of the 27 ASL alphabets and a predefined gesture for space. Three types of features are used; bending using six bend sensors, orientation in three dimensions using accelerometers and contacts at vital points using contact sensors. To gauge the performance of the presented design, the training database was prepared using five volunteers. The accuracy of the current version on the prepared dataset was found to be up to 99.3% for target user. The solution combines electronics, e-textile technology, sensor technology, embedded system and machine learning techniques to build a low cost wearable glove that is scrupulous, elegant and portable.

Authoring Tactile Gestures: Case Study for Emotion Stimulation

Year: 2016 Volume: 10 Issue: 11 1964 - 1969 Pages

Abstract: The haptic modality has brought a new dimension to human computer interaction by engaging the human sense of touch. However, designing appropriate haptic stimuli, and in particular tactile stimuli, for various applications is still challenging. To tackle this issue, we present an intuitive system that facilitates the authoring of tactile gestures for various applications. The system transforms a hand gesture into a tactile gesture that can be rendering using a home-made haptic jacket. A case study is presented to demonstrate the ability of the system to develop tactile gestures that are recognizable by human subjects. Four tactile gestures are identified and tested to intensify the following four emotional responses: high valence – high arousal, high valence – low arousal, low valence – high arousal, and low valence – low arousal. A usability study with 20 participants demonstrated high correlation between the selected tactile gestures and the intended emotional reaction. Results from this study can be used in a wide spectrum of applications ranging from gaming to interpersonal communication and multimodal simulations.

The Effect of Iconic and Beat Gestures on Memory Recall in Greek’s First and Second Language

Year: 2016 Volume: 10 Issue: 8 2919 - 2926 Pages

Authors:
Eleni Ioanna Levantinou

Abstract: Gestures play a major role in comprehension and memory recall due to the fact that aid the efficient channel of the meaning and support listeners’ comprehension and memory. In the present study, the assistance of two kinds of gestures (iconic and beat gestures) is tested in regards to memory and recall. The hypothesis investigated here is whether or not iconic and beat gestures provide assistance in memory and recall in Greek and in Greek speakers’ second language. Two groups of participants were formed, one comprising Greeks that reside in Athens and one with Greeks that reside in Copenhagen. Three kinds of stimuli were used: A video with words accompanied with iconic gestures, a video with words accompanied with beat gestures and a video with words alone. The languages used are Greek and English. The words in the English videos were spoken by a native English speaker and by a Greek speaker talking English. The reason for this is that when it comes to beat gestures that serve a meta-cognitive function and are generated according to the intonation of a language, prosody plays a major role. Thus, participants that have different influences in prosody may generate different results from rhythmic gestures. Memory recall was assessed by asking the participants to try to remember as many words as they could after viewing each video. Results show that iconic gestures provide significant assistance in memory and recall in Greek and in English whether they are produced by a native or a second language speaker. In the case of beat gestures though, the findings indicate that beat gestures may not play such a significant role in Greek language. As far as intonation is concerned, a significant difference was not found in the case of beat gestures produced by a native English speaker and by a Greek speaker talking English.

Stereotypical Motor Movement Recognition Using Microsoft Kinect with Artificial Neural Network

Year: 2016 Volume: 10 Issue: 7 1270 - 1274 Pages

Abstract: Autism spectrum disorder is a complex developmental disability. It is defined by a certain set of behaviors. Persons with Autism Spectrum Disorders (ASD) frequently engage in stereotyped and repetitive motor movements. The objective of this article is to propose a method to automatically detect this unusual behavior. Our study provides a clinical tool which facilitates for doctors the diagnosis of ASD. We focus on automatic identification of five repetitive gestures among autistic children in real time: body rocking, hand flapping, fingers flapping, hand on the face and hands behind back. In this paper, we present a gesture recognition system for children with autism, which consists of three modules: model-based movement tracking, feature extraction, and gesture recognition using artificial neural network (ANN). The first one uses the Microsoft Kinect sensor, the second one chooses points of interest from the 3D skeleton to characterize the gestures, and the last one proposes a neural connectionist model to perform the supervised classification of data. The experimental results show that our system can achieve above 93.3% recognition rate.

Females’ Usage Patterns of Information and Communication Technologies (ICTs) in the Vhembe District, South Africa

Year: 2015 Volume: 9 Issue: 9 3175 - 3182 Pages

Authors:
F. O. Maphiri-Makananise

Abstract: This paper explores and provides substantiated evidence on the usage patterns of Information and Communication Technologies (ICTs) by female users at Vhembe District in Limpopo- Province, South Africa. The study presents a comprehensive picture on the usage of ICTs from female users’ perspective. The significance of this study stems from the need to assess the role, relevance and usage patterns of ICTs such as smartphones, computers, laptops, and iPods, the internet and social networking sites among females following the developments of new media technologies in society. The objective of the study is to investigate the usability and accessibility of ICTs to empower female users in South Africa. The study used quantitative and qualitative research methods to determine the major ideas, perceptions and usage patterns of ICTs by users. Data collection involved the use of structured selfadministered questionnaire from two groups of respondents who participated in this study. Thus, (n=50) female students at the University of Venda provided their ideas and perceptions about the usefulness and usage patterns of ICTs such as smartphones, the Internet and computers at the university level, whereas, the second group were (n=50) learners from Makhado Comprehensive School who provided their perceptions and ideas about the use of ICTs at the high school level. The researcher also noted that the findings of the study were useful as a guideline and model for ICT intervention that could work as an empowerment to women in South Africa. It was observed that the central purpose of ICTs among female users was to search for information regarding assignment writing, conducting research, dating, exchanging ideas and networking with friends and relatives. This was demonstrated by a high number of females who used ICTs for e-learning (62%) and social purposes (85%). Therefore, the study revealed that most females used ICTs for social purposes and accessing the internet rather than for entertainment, a gesture that provides an opportune space to empower rural women in South Africa.

Keywords:
Female users
Information and Communication Technologies
Internet
Usage patterns.

ConductHome: Gesture Interface Control of Home Automation Boxes

Year: 2015 Volume: 9 Issue: 10 3346 - 3349 Pages

Abstract: This paper presents the interface ConductHome which controls home automation systems with a Leap Motion using “invariant gesture protocols”. This interface is meant to simplify the interaction of the user with its environment. A hardware part allows the Leap Motion to be carried around the house. A software part interacts with the home automation box and displays the useful information for the user. An objective of this work is the development of a natural/invariant/simple gesture control interface to help elder people/people with disabilities.

Interactive Shadow Play Animation System

Year: 2015 Volume: 9 Issue: 1 127 - 132 Pages

Abstract: The paper describes a Chinese shadow play animation system based on Kinect. Users, without any professional training, can personally manipulate the shadow characters to finish a shadow play performance by their body actions and get a shadow play video through giving the record command to our system if they want. In our system, Kinect is responsible for capturing human movement and voice commands data. Gesture recognition module is used to control the change of the shadow play scenes. After packaging the data from Kinect and the recognition result from gesture recognition module, VRPN transmits them to the server-side. At last, the server-side uses the information to control the motion of shadow characters and video recording. This system not only achieves human-computer interaction, but also realizes the interaction between people. It brings an entertaining experience to users and easy to operate for all ages. Even more important is that the application background of Chinese shadow play embodies the protection of the art of shadow play animation.

Users’ Preferences for Map Navigation Gestures

Year: 2015 Volume: 9 Issue: 1 77 - 83 Pages

Abstract: Map is a powerful and convenient tool in helping us to navigate to different places, but the use of indirect devices often makes its usage cumbersome. This study intends to propose a new map navigation dialogue that uses hand gesture. A set of dialogue was developed from users’ perspective to provide users complete freedom for panning, zooming, rotate, tilt and find direction operations. A participatory design experiment was involved here where one hand gesture and two hand gesture dialogues had been analysed in the forms of hand gestures to develop a set of usable dialogues. The major finding was that users prefer one-hand gesture compared to two-hand gesture in map navigation.

Control of an Asymmetrical Design of a Pneumatically Actuated Ambidextrous Robot Hand

Year: 2014 Volume: 8 Issue: 5 794 - 799 Pages

Abstract: The Ambidextrous Robot Hand is a robotic device with the purpose to mimic either the gestures of a right or a left hand. The symmetrical behavior of its fingers allows them to bend in one way or another keeping a compliant and anthropomorphic shape. However, in addition to gestures they can reproduce on both sides, an asymmetrical mechanical design with a three tendons routing has been engineered to reduce the number of actuators. As a consequence, control algorithms must be adapted to drive efficiently the ambidextrous fingers from one position to another and to include grasping features. These movements are controlled by pneumatic muscles, which are nonlinear actuators. As their elasticity constantly varies when they are under actuation, the length of pneumatic muscles and the force they provide may differ for a same value of pressurized air. The control algorithms introduced in this paper take both the fingers asymmetrical design and the pneumatic muscles nonlinearity into account to permit an accurate control of the Ambidextrous Robot Hand. The finger motion is achieved by combining a classic PID controller with a phase plane switching control that turns the gain constants into dynamic values. The grasping ability is made possible because of a sliding mode control that makes the fingers adapt to the shape of an object before strengthening their positions.

Development of a Computer Vision System for the Blind and Visually Impaired Person

Year: 2013 Volume: 7 Issue: 11 1479 - 1483 Pages

Authors:
Roselyn A. Maaño

Abstract: Eyes are an essential and conspicuous organ of the human body. Human eyes are outward and inward portals of the body that allows to see the outside world and provides glimpses into ones inner thoughts and feelings. Inevitable blindness and visual impairments may results from eye-related disease, trauma, or congenital or degenerative conditions that cannot be corrected by conventional means. The study emphasizes innovative tools that will serve as an aid to the blind and visually impaired (VI) individuals. The researchers fabricated a prototype that utilizes the Microsoft Kinect for Windows and Arduino microcontroller board. The prototype facilitates advanced gesture recognition, voice recognition, obstacle detection and indoor environment navigation. Open Computer Vision (OpenCV) performs image analysis, and gesture tracking to transform Kinect data to the desired output. A computer vision technology device provides greater accessibility for those with vision impairments.

Traditions of Theatrical Art in the Space of Nomadic Culture of the Kazakhs

Year: 2013 Volume: 7 Issue: 3 822 - 827 Pages

Abstract: A number of theoretical and methodological problems connected with substantiation of a new approach and searches of a new research paradigm and the analysis of features of formation and development of the Kazakh stage are considered in the article. The wide spectrum of questions connected with genesis of the Kazakh stage art has caused necessity of consideration of world outlook and social cultural aspects which have affected formation of the given phenomenon in the Kazakh culture. But how can we define the form of expression and aesthetics of the national theatre? Probably, the answer to this question we will find if we apply to deep world view sources, and, as a consequence, it is necessary to study deeply the plot dramaturgy, which is based on myths, rites and eposes, mastering of symbolic gestures and mimics, allegory of a word, etc.

Vision Based Hand Gesture Recognition Using Generative and Discriminative Stochastic Models

Year: 2013 Volume: 7 Issue: 11 1443 - 1449 Pages

Abstract: Many approaches to pattern recognition are founded on probability theory, and can be broadly characterized as either generative or discriminative according to whether or not the distribution of the image features. Generative and discriminative models have very different characteristics, as well as complementary strengths and weaknesses. In this paper, we study these models to recognize the patterns of alphabet characters (A-Z) and numbers (0-9). To handle isolated pattern, generative model as Hidden Markov Model (HMM) and discriminative models like Conditional Random Field (CRF), Hidden Conditional Random Field (HCRF) and Latent-Dynamic Conditional Random Field (LDCRF) with different number of window size are applied on extracted pattern features. The gesture recognition rate is improved initially as the window size increase, but degrades as window size increase further. Experimental results show that the LDCRF is the best in terms of results than CRF, HCRF and HMM at window size equal 4. Additionally, our results show that; an overall recognition rates are 91.52%, 95.28%, 96.94% and 98.05% for CRF, HCRF, HMM and LDCRF respectively.

Intuitive Robot Control Using Surface EMG and Accelerometer Signals

Year: 2013 Volume: 7 Issue: 10 1376 - 1379 Pages

Abstract: This paper proposes a method of remotely controlling robots with arm gestures using surface electromyography (EMG) and accelerometer sensors attached to the operator’s wrists. The EMG and accelerometer sensors receive signals from the arm gestures of the operator and infer the corresponding movements to execute the command to control the robot. The movements of the robot include moving forward and backward and turning left and right. The accuracy is over 99% and movements can be controlled in real time.

Infrared Camera-Based Hand Gesture Space Touch System Implementation of Smart Device Environment

Year: 2013 Volume: 7 Issue: 10 1318 - 1322 Pages

Abstract: This paper proposes a method to recognize the tip of a finger and space touch hand gesture using an infrared camera in a smart device environment. The proposed method estimates the tip of a finger with a curvature-based ellipse fitting algorithm, and verifies that the estimated object is indeed a finger with an ellipse fitting rectangular area. The feature extracted from the verified finger tip is used to implement the movement of a mouse and clicking gesture. The proposed algorithm was implemented with an actual smart device to test the proposed method. Empirical parameters were obtained from the keypad software and an image analysis tool for the performance optimization, and a comparative analysis with conventional research showed improved performance with the proposed method.

A Real Time Set Up for Retrieval of Emotional States from Human Neural Responses

Year: 2014 Volume: 8 Issue: 3 129 - 134 Pages

Abstract: Real time non-invasive Brain Computer Interfaces have a significant progressive role in restoring or maintaining a quality life for medically challenged people. This manuscript provides a comprehensive review of emerging research in the field of cognitive/affective computing in context of human neural responses. The perspectives of different emotion assessment modalities like face expressions, speech, text, gestures, and human physiological responses have also been discussed. Focus has been paid to explore the ability of EEG (Electroencephalogram) signals to portray thoughts, feelings, and unspoken words. An automated workflow-based protocol to design an EEG-based real time Brain Computer Interface system for analysis and classification of human emotions elicited by external audio/visual stimuli has been proposed. The front end hardware includes a cost effective and portable Emotiv EEG Neuroheadset unit, a personal computer and a set of external stimulators. Primary signal analysis and processing of real time acquired EEG shall be performed using MATLAB based advanced brain mapping toolbox EEGLab/BCILab. This shall be followed by the development of MATLAB based self-defined algorithm to capture and characterize temporal and spectral variations in EEG under emotional stimulations. The extracted hybrid feature set shall be used to classify emotional states using artificial intelligence tools like Artificial Neural Network. The final system would result in an inexpensive, portable and more intuitive Brain Computer Interface in real time scenario to control prosthetic devices by translating different brain states into operative control signals.

On-line Recognition of Isolated Gestures of Flight Deck Officers (FDO)

Year: 2008 Volume: 2 Issue: 1 241 - 247 Pages

Abstract: The paper presents an on-line recognition machine (RM) for continuous/isolated, dynamic and static gestures that arise in Flight Deck Officer (FDO) training. RM is based on generic pattern recognition framework. Gestures are represented as templates using summary statistics. The proposed recognition algorithm exploits temporal and spatial characteristics of gestures via dynamic programming and Markovian process. The algorithm predicts corresponding index of incremental input data in the templates in an on-line mode. Accumulated consistency in the sequence of prediction provides a similarity measurement (Score) between input data and the templates. The algorithm provides an intuitive mechanism for automatic detection of start/end frames of continuous gestures. In the present paper, we consider isolated gestures. The performance of RM is evaluated using four datasets - artificial (W TTest), hand motion (Yang) and FDO (tracker, vision-based ). RM achieves comparable results which are in agreement with other on-line and off-line algorithms such as hidden Markov model (HMM) and dynamic time warping (DTW). The proposed algorithm has the additional advantage of providing timely feedback for training purposes.

Eye-Gesture Analysis for Driver Hazard Awareness

Year: 2012 Volume: 6 Issue: 5 612 - 618 Pages

Abstract: Because road traffic accidents are a major source of death worldwide, attempts have been made to create Advanced Driver Assistance Systems (ADAS) able to detect vehicle, driver and environmental conditions that are cues for possible potential accidents. This paper presents continued work on a novel Nonintrusive Intelligent Driver Assistance and Safety System (Ni-DASS) for assessing driver attention and hazard awareness. It uses two onboard CCD cameras – one observing the road and the other observing the driver-s face. The windscreen is divided into cells and analysis of the driver-s eye-gaze patterns allows Ni-DASS to determine the windscreen cell the driver is focusing on using eye-gesture templates. Intersecting the driver-s field of view through the observed windscreen cell with subsections of the camera-s field of view containing a potential hazard allows Ni-DASS to estimate the probability that the driver has actually observed the hazard. Results have shown that the proposed technique is an accurate enough measure of driver observation to be useful in ADAS systems.

Keywords:
Advanced Driver Assistance Systems (ADAS)
Driver Hazard Awareness
Driver Vigilance
Eye Tracking

Pakistan Sign Language Recognition Using Statistical Template Matching

Year: 2007 Volume: 1 Issue: 3 779 - 782 Pages

Abstract: Sign language recognition has been a topic of research since the first data glove was developed. Many researchers have attempted to recognize sign language through various techniques. However none of them have ventured into the area of Pakistan Sign Language (PSL). The Boltay Haath project aims at recognizing PSL gestures using Statistical Template Matching. The primary input device is the DataGlove5 developed by 5DT. Alternative approaches use camera-based recognition which, being sensitive to environmental changes are not always a good choice.This paper explains the use of Statistical Template Matching for gesture recognition in Boltay Haath. The system recognizes one handed alphabet signs from PSL.

Trajectory Guided Recognition of Hand Gestures having only Global Motions

Year: 2008 Volume: 2 Issue: 9 2051 - 2062 Pages

Abstract: One very interesting field of research in Pattern Recognition that has gained much attention in recent times is Gesture Recognition. In this paper, we consider a form of dynamic hand gestures that are characterized by total movement of the hand (arm) in space. For these types of gestures, the shape of the hand (palm) during gesturing does not bear any significance. In our work, we propose a model-based method for tracking hand motion in space, thereby estimating the hand motion trajectory. We employ the dynamic time warping (DTW) algorithm for time alignment and normalization of spatio-temporal variations that exist among samples belonging to the same gesture class. During training, one template trajectory and one prototype feature vector are generated for every gesture class. Features used in our work include some static and dynamic motion trajectory features. Recognition is accomplished in two stages. In the first stage, all unlikely gesture classes are eliminated by comparing the input gesture trajectory to all the template trajectories. In the next stage, feature vector extracted from the input gesture is compared to all the class prototype feature vectors using a distance classifier. Experimental results demonstrate that our proposed trajectory estimator and classifier is suitable for Human Computer Interaction (HCI) platform.

Top Journal

SUGGEST A JOURNAL