Abstract: In global navigation satellite system (GNSS) denied settings, such as indoor environments, autonomous mobile robots are often limited to dead-reckoning navigation techniques to determine their position, velocity, and attitude (PVA). Localization is typically accomplished by employing an inertial measurement unit (IMU), which, while precise in nature, accumulates errors rapidly and severely degrades the localization solution. Standard sensor fusion methods, such as Kalman filtering, aim to fuse precise IMU measurements with accurate aiding sensors to establish a precise and accurate solution. In indoor environments, where GNSS and no other a priori information is known about the environment, effective sensor fusion is difficult to achieve, as accurate aiding sensor choices are sparse. However, an opportunity arises by employing a depth camera in the indoor environment. A depth camera can capture point clouds of the surrounding floors and walls. Extracting attitude from these surfaces can serve as an accurate aiding source, which directly combats errors that arise due to gyroscope imperfections. This configuration for sensor fusion leads to a dramatic reduction of PVA error compared to traditional aiding sensor configurations. This paper provides the theoretical basis for the depth camera aiding sensor method, initial expectations of performance benefit via simulation, and hardware implementation thus verifying its veracity. Hardware implementation is performed on the Quanser Qbot 2™ mobile robot, with a Vector-Nav VN-200™ IMU and Kinect™ camera from Microsoft.
Abstract: Spatial Augmented Reality is a variation of Augmented Reality where the Head-Mounted Display is not required. This variation of Augmented Reality is useful in cases where the need for a Head-Mounted Display itself is a limitation. To achieve this, Spatial Augmented Reality techniques substitute the technological elements of Augmented Reality; the virtual world is projected onto a physical surface. To create an interactive spatial augmented experience, the application must be aware of the spatial relations that exist between its core elements. In this case, the core elements are referred to as a projection system and an input system, and the process to achieve this spatial awareness is called system calibration. The Spatial Augmented Reality system is considered calibrated if the projected virtual world scale is similar to the real-world scale, meaning that a virtual object will maintain its perceived dimensions when projected to the real world. Also, the input system is calibrated if the application knows the relative position of a point in the projection plane and the RGB-depth sensor origin point. Any kind of projection technology can be used, light-based projectors, close-range projectors, and screens, as long as it complies with the defined constraints; the method was tested on different configurations. The proposed procedure does not rely on a physical marker, minimizing the human intervention on the process. The tests are made using a Kinect V2 as an input sensor and several projection devices. In order to test the method, the constraints defined were applied to a variety of physical configurations; once the method was executed, some variables were obtained to measure the method performance. It was demonstrated that the method obtained can solve different arrangements, giving the user a wide range of setup possibilities.
Abstract: We propose to record Activities of Daily Living
(ADLs) of elderly people using a vision-based system so as to provide
better assistive and personalization technologies. Current ADL-related
research is based on data collected with help from non-elderly subjects
in laboratory environments and the activities performed are predetermined
for the sole purpose of data collection. To obtain more
realistic datasets for the application, we recorded ADLs for the elderly
with data collected from real-world environment involving real elderly
subjects. Motivated by the need to collect data for more effective
research related to elderly care, we chose to collect data in the room of
an elderly person. Specifically, we installed Kinect, a vision-based
sensor on the ceiling, to capture the activities that the elderly subject
performs in the morning every day. Based on the data, we identified
12 morning activities that the elderly person performs daily. To
recognize these activities, we created a HARELCARE framework to
investigate into the effectiveness of existing Human Activity
Recognition (HAR) algorithms and propose the use of a transfer
learning algorithm for HAR. We compared the performance, in terms
of accuracy, and training progress. Although the collected dataset is
relatively small, the proposed algorithm has a good potential to be
applied to all daily routine activities for healthcare purposes such as
evidence-based diagnosis and treatment.
Abstract: Motion recognition from videos is actually a very
complex task due to the high variability of motions. This paper
describes the challenges of human motion recognition, especially
motion representation step with relevant features. Our descriptor
vector is inspired from Laban Movement Analysis method. We
propose discriminative features using the Random Forest algorithm
in order to remove redundant features and make learning algorithms
operate faster and more effectively. We validate our method on
MSRC-12 and UTKinect datasets.
Abstract: Human motion recognition has been extensively increased in recent years due to its importance in a wide range of applications, such as human-computer interaction, intelligent surveillance, augmented reality, content-based video compression and retrieval, etc. However, it is still regarded as a challenging task especially in realistic scenarios. It can be seen as a general machine learning problem which requires an effective human motion representation and an efficient learning method. In this work, we introduce a descriptor based on Laban Movement Analysis technique, a formal and universal language for human movement, to capture both quantitative and qualitative aspects of movement. We use Discrete Hidden Markov Model (DHMM) for training and classification motions. We improve the classification algorithm by proposing two DHMMs for each motion class to process the motion sequence in two different directions, forward and backward. Such modification allows avoiding the misclassification that can happen when recognizing similar motions. Two experiments are conducted. In the first one, we evaluate our method on a public dataset, the Microsoft Research Cambridge-12 Kinect gesture data set (MSRC-12) which is a widely used dataset for evaluating action/gesture recognition methods. In the second experiment, we build a dataset composed of 10 gestures(Introduce yourself, waving, Dance, move, turn left, turn right, stop, sit down, increase velocity, decrease velocity) performed by 20 persons. The evaluation of the system includes testing the efficiency of our descriptor vector based on LMA with basic DHMM method and comparing the recognition results of the modified DHMM with the original one. Experiment results demonstrate that our method outperforms most of existing methods that used the MSRC-12 dataset, and a near perfect classification rate in our dataset.
Abstract: This paper presents a home-based robot-rehabilitation
instrument, called ”MAGNI Dynamics”, that utilized a vision-based
kinematic/dynamic module and an adaptive haptic feedback
controller. The system is expected to provide personalized
rehabilitation by adjusting its resistive and supportive behavior
according to a fuzzy intelligence controller that acts as an inference
system, which correlates the user’s performance to different stiffness
factors. The vision module uses the Kinect’s skeletal tracking to
monitor the user’s effort in an unobtrusive and safe way, by estimating
the torque that affects the user’s arm. The system’s torque estimations
are justified by capturing electromyographic data from primitive
hand motions (Shoulder Abduction and Shoulder Forward Flexion).
Moreover, we present and analyze how the Barrett WAM generates
a force-field with a haptic controller to support or challenge the
users. Experiments show that by shifting the proportional value,
that corresponds to different stiffness factors of the haptic path, can
potentially help the user to improve his/her motor skills. Finally,
potential areas for future research are discussed, that address how
a rehabilitation robotic framework may include multisensing data, to
improve the user’s recovery process.
Abstract: In designing a kinetic façade, it is hard for the designer to make digital models due to its complex geometry with motion. This paper aims to present a methodology of converting a point cloud of a physical model into a single digital model with a certain topology and motion. The method uses a Microsoft Kinect sensor, and color markers were defined and applied to three paper folding-inspired designs. Although the resulted digital model cannot represent the whole folding range of the physical model, the method supports the designer to conduct a performance-oriented design process with the rough physical model in the reduced folding range.
Abstract: Most of traditional visual indoor navigation algorithms
and methods only consider the localization in ordinary daytime, while
we focus on the indoor re-localization in low light in the paper. As
RGB images are degraded in low light, less discriminative infrared
and depth image pairs are taken, as the input, by RGB-D cameras, the
most similar candidates, as the output, are searched from databases
which is built in the bag-of-word framework. Epipolar constraints can
be used to relocalize the query infrared and depth image sequence.
We evaluate our method in two datasets captured by Kinect2. The
results demonstrate very promising re-localization results for indoor
navigation system in low light environments.
Abstract: Proprioceptive neuromuscular facilitation exercises are a series of stretching techniques that are commonly used in rehabilitation and exercise therapy. Assessment of these exercises for true maneuvering requires extensive experience in this field and could not be down with patients themselves. In this paper, we developed software that uses Microsoft Kinect sensor, a spherical color marker, and real-time image processing methods to evaluate patient’s performance in generating true patterns of movements. The software also provides the patient with a visual feedback by showing his/her avatar in a Virtual Reality environment along with the correct path of moving hand, wrist and marker. Primary results during PNF exercise therapy of a patient in a room environment shows the ability of the system to identify any deviation of maneuvering path and direction of the hand from the one that has been performed by an expert physician.
Abstract: We aim to develop a full-body interaction game that could let children cooperate and interact with other children in small groups. As the first step for our aim, the objective of the full-body interaction game developed in this study is to make interaction between children. The game requires two children to jump together with the same timing. We let children experience the game and answer the questionnaires. The children using several strategies to coordinate the timing of their jumps were observed. These included shouting time, watching each other, and jumping in a constant rhythm as if they were skipping rope. In this manner, we observed the children playing the game while cooperating with each other. The results of a questionnaire to evaluate the proposed interactive game indicate that the jumping game was a very enjoyable experience in which the participants could immerse themselves. Therefore, the game enabled children to experience cooperation with others by using body movements.
Abstract: In the field of reverse engineering and creative industries, applying 3D scanning process to obtain geometric forms of the objects is a mature and common technique. For instance, organic objects such as faces and nonorganic objects such as products could be scanned to acquire the geometric information for further application. However, although the data resolution of 3D scanning device is increasing and there are more and more abundant complementary applications, the penetration rate of 3D scanning for the public is still limited by the relative high price of the devices. On the other hand, Kinect, released by Microsoft, is known for its powerful functions, considerably low price, and complete technology and database support. Therefore, related studies can be done with the applying of Kinect under acceptable cost and data precision. Due to the fact that Kinect utilizes optical mechanism to extracting depth information, limitations are found due to the reason of the straight path of the light. Thus, various angles are required sequentially to obtain the complete 3D information of the object when applying a single Kinect for 3D scanning. The integration process which combines the 3D data from different angles by certain algorithms is also required. This sequential scanning process costs much time and the complex integration process often encounter some technical problems. Therefore, this paper aimed to apply multiple Kinects simultaneously on the field of developing a rapid 3D mannequin scan platform and proposed suggestions on the number and angles of Kinects. In the content, a method of establishing the coordination based on the relation between mannequin and the specifications of Kinect is proposed, and a suggestion of angles and number of Kinects is also described. An experiment of applying multiple Kinect on the scanning of 3D mannequin is constructed by Microsoft API, and the results show that the time required for scanning and technical threshold can be reduced in the industries of fashion and garment design.
Abstract: Autism spectrum disorder is a complex developmental disability. It is defined by a certain set of behaviors. Persons with Autism Spectrum Disorders (ASD) frequently engage in stereotyped and repetitive motor movements. The objective of this article is to propose a method to automatically detect this unusual behavior. Our study provides a clinical tool which facilitates for doctors the diagnosis of ASD. We focus on automatic identification of five repetitive gestures among autistic children in real time: body rocking, hand flapping, fingers flapping, hand on the face and hands behind back. In this paper, we present a gesture recognition system for children with autism, which consists of three modules: model-based movement tracking, feature extraction, and gesture recognition using artificial neural network (ANN). The first one uses the Microsoft Kinect sensor, the second one chooses points of interest from the 3D skeleton to characterize the gestures, and the last one proposes a neural connectionist model to perform the supervised classification of data. The experimental results show that our system can achieve above 93.3% recognition rate.
Abstract: This paper investigates the potential use of airborne ultrasonic phased arrays for imaging in outdoor environments as a means of overcoming the limitations experienced by kinect sensors, which may fail to work in the outdoor environments due to the oversaturation of the infrared photo diodes. Ultrasonic phased arrays have been well studied for static media, yet there appears to be no comparable examination in the literature of the impact of a flowing medium on the focusing behaviour of near field focused ultrasonic arrays. This paper presents a method for predicting the sound pressure fields produced by a single ultrasound element or an ultrasonic phased array influenced by airflows. The approach can be used to determine the actual focal point location of an array exposed in a known flow field. From the presented simulation results based upon this model, it can be concluded that uniform flows in the direction orthogonal to the acoustic propagation have a noticeable influence on the sound pressure field, which is reflected in the twisting of the steering angle of the array. Uniform flows in the same direction as the acoustic propagation have negligible influence on the array. For an array impacted by a turbulent flow, determining the location of the focused sound field becomes difficult due to the irregularity and continuously changing direction and the speed of the turbulent flow. In some circumstances, ultrasonic phased arrays impacted by turbulent flows may not be capable of producing a focused sound field.
Abstract: Laban Movement Analysis (LMA), developed in the
dance community over the past seventy years, is an effective method
for observing, describing, notating, and interpreting human
movement to enhance communication and expression in everyday
and professional life. Many applications that use motion capture data
might be significantly leveraged if the Laban qualities will be
recognized automatically. This paper presents an automated
recognition method of Laban qualities from motion capture skeletal
recordings and it is demonstrated on the output of Microsoft’s Kinect
V2 sensor.
Abstract: In the field of fashion design, 3D Mannequin is a kind
of assisting tool which could rapidly realize the design concepts.
While the concept of 3D Mannequin is applied to the computer added
fashion design, it will connect with the development and the
application of design platform and system. Thus, the situation
mentioned above revealed a truth that it is very critical to develop a
module of 3D Mannequin which would correspond with the necessity
of fashion design. This research proposes a concrete plan that
developing and constructing a system of 3D Mannequin with Kinect.
In the content, ergonomic measurements of objective human features
could be attained real-time through the implement with depth camera
of Kinect, and then the mesh morphing can be implemented through
transformed the locations of the control-points on the model by
inputting those ergonomic data to get an exclusive 3D mannequin
model. In the proposed methodology, after the scanned points from the
Kinect are revised for accuracy and smoothening, a complete human
feature would be reconstructed by the ICP algorithm with the method
of image processing. Also, the objective human feature could be
recognized to analyze and get real measurements. Furthermore, the
data of ergonomic measurements could be applied to shape morphing
for the division of 3D Mannequin reconstructed by feature curves. Due
to a standardized and customer-oriented 3D Mannequin would be
generated by the implement of subdivision, the research could be
applied to the fashion design or the presentation and display of 3D
virtual clothes. In order to examine the practicality of research
structure, a system of 3D Mannequin would be constructed with JAVA
program in this study. Through the revision of experiments the
practicability-contained research result would come out.
Abstract: The paper describes a Chinese shadow play animation
system based on Kinect. Users, without any professional training, can
personally manipulate the shadow characters to finish a shadow play
performance by their body actions and get a shadow play video
through giving the record command to our system if they want. In our
system, Kinect is responsible for capturing human movement and
voice commands data. Gesture recognition module is used to control
the change of the shadow play scenes. After packaging the data from
Kinect and the recognition result from gesture recognition module,
VRPN transmits them to the server-side. At last, the server-side uses
the information to control the motion of shadow characters and video
recording. This system not only achieves human-computer interaction,
but also realizes the interaction between people. It brings an
entertaining experience to users and easy to operate for all ages. Even
more important is that the application background of Chinese shadow
play embodies the protection of the art of shadow play animation.
Abstract: Eyes are an essential and conspicuous organ of the human body. Human eyes are outward and inward portals of the body that allows to see the outside world and provides glimpses into ones inner thoughts and feelings. Inevitable blindness and visual impairments may results from eye-related disease, trauma, or congenital or degenerative conditions that cannot be corrected by conventional means. The study emphasizes innovative tools that will serve as an aid to the blind and visually impaired (VI) individuals. The researchers fabricated a prototype that utilizes the Microsoft Kinect for Windows and Arduino microcontroller board. The prototype facilitates advanced gesture recognition, voice recognition, obstacle detection and indoor environment navigation. Open Computer Vision (OpenCV) performs image analysis, and gesture tracking to transform Kinect data to the desired output. A computer vision technology device provides greater accessibility for those with vision impairments.
Abstract: It is important to give input information without other device in AR system. One solution is using hand for augmented reality application. Many researchers have proposed different solutions for hand interface in augmented reality. Analyze Histogram and connecting factor is can be example for that. Various Direction searching is one of robust way to recognition hand but it takes too much calculating time. And background should be distinguished with skin color. This paper proposes a hand tracking method to control the 3D object in augmented reality using depth device and skin color. Also in this work discussed relationship between several markers, which is based on relationship between camera and marker. One marker used for displaying virtual object and three markers for detecting hand gesture and manipulating the virtual object.
Abstract: This paper describes the process used in the
automation of the Maritime UAV commands using the Kinect sensor.
The AR Drone is a Quadrocopter manufactured by Parrot [1] to be
controlled using the Apple operating systems such as iPhones and
Ipads. However, this project uses the Microsoft Kinect SDK and
Microsoft Visual Studio C# (C sharp) software, which are compatible
with Windows Operating System for the automation of the navigation
and control of the AR drone.
The navigation and control software for the Quadrocopter runs on
a windows 7 computer. The project is divided into two sections; the
Quadrocopter control system and the Kinect sensor control system.
The Kinect sensor is connected to the computer using a USB cable
from which commands can be sent to and from the Kinect sensors.
The AR drone has Wi-Fi capabilities from which it can be connected
to the computer to enable transfer of commands to and from the
Quadrocopter.
The project was implemented in C#, a programming language that
is commonly used in the automation systems. The language was
chosen because there are more libraries already established in C# for
both the AR drone and the Kinect sensor.
The study will contribute toward research in automation of
systems using the Quadrocopter and the Kinect sensor for navigation
involving a human operator in the loop. The prototype created has
numerous applications among which include the inspection of vessels
such as ship, airplanes and areas that are not accessible by human
operators.
Abstract: This paper proposes a three-dimensional motion capture and feedback system of flying disc throwing action learners with use of Kinect device. Rather than conventional 3-D motion capture system, Kinect has advantages of cost merit, easy system development and operation. A novice learner of flying disc is trained to keep arm movement in steady height, to twist the waist, and to stretch the elbow according to the waist angle. The proposing system captures learners- body movement, checks their skeleton positions in pre-motion / motion / post-motion in several ways, and displays feedback messages to refine their actions.