Abstract: Seizure is the main factor that affects the quality of life of epileptic patients. The diagnosis of epilepsy, and hence the identification of epileptogenic zone, is commonly made by using continuous Electroencephalogram (EEG) signal monitoring. Seizure identification on EEG signals is made manually by epileptologists and this process is usually very long and error prone. The aim of this paper is to describe an automated method able to detect seizures in EEG signals, using knowledge discovery in database process and data mining methods and algorithms, which can support physicians during the seizure detection process. Our detection method is based on Artificial Neural Network classifier, trained by applying the multilayer perceptron algorithm, and by using a software application, called Training Builder that has been developed for the massive extraction of features from EEG signals. This tool is able to cover all the data preparation steps ranging from signal processing to data analysis techniques, including the sliding window paradigm, the dimensionality reduction algorithms, information theory, and feature selection measures. The final model shows excellent performances, reaching an accuracy of over 99% during tests on data of a single patient retrieved from a publicly available EEG dataset.
Abstract: Machine Learning and Data Mining are the two important tools for extracting useful information and knowledge from large datasets. In machine learning, classification is a wildly used technique to predict qualitative variables and is generally preferred over regression from an operational point of view. Due to the enormous increase in air pollution in various countries especially China, Air Quality Classification has become one of the most important topics in air quality research and modelling. This study aims at introducing a hybrid classification model based on information theory and Support Vector Machine (SVM) using the air quality data of four cities in China namely Beijing, Guangzhou, Shanghai and Tianjin from Jan 1, 2014 to April 30, 2016. China's Ministry of Environmental Protection has classified the daily air quality into 6 levels namely Serious Pollution, Severe Pollution, Moderate Pollution, Light Pollution, Good and Excellent based on their respective Air Quality Index (AQI) values. Using the information theory, information gain (IG) is calculated and feature selection is done for both categorical features and continuous numeric features. Then SVM Machine Learning algorithm is implemented on the selected features with cross-validation. The final evaluation reveals that the IG and SVM hybrid model performs better than SVM (alone), Artificial Neural Network (ANN) and K-Nearest Neighbours (KNN) models in terms of accuracy as well as complexity.
Abstract: In this paper, a methodology is presented to obtain the Static Alignment Model for any transtibial amputee person. The proposed methodology starts from experimental data collected on the Hospital Militar Central, Bogotá, Colombia. The effects of transtibial prosthesis malalignment on amputees were measured in terms of joint angles, center of pressure (COP) and weight distribution. Some statistical tools are used to obtain the model parameters. Mathematical predictive models of prosthetic alignment were created. The proposed models are validated in amputees and finding promising results for the prosthesis Static Alignment. Static alignment process is unique to each subject; nevertheless the proposed methodology can be used in each transtibial amputee.
Abstract: This paper takes a philosophical view as axiom, and
reveals the relationship between information theory and Natural
Intelligence and Artificial Intelligence under real world conditions.
This paper also derives the relationship between natural intelligence
and nature. According to communication principle of information
theory, Natural Intelligence can be divided into real part and virtual
part. Based on information theory principle that Information does not
increase, the restriction mechanism of Natural Intelligence creativity is
conducted. The restriction mechanism of creativity reveals the limit of
natural intelligence and artificial intelligence. The paper provides a
new angle to observe natural intelligence and artificial intelligence.
Abstract: This paper treats different aspects of entropy measure
in classical information theory and statistical quantum mechanics, it
presents the possibility of extending the definition of Von Neumann
entropy to image and array processing. In the first part, we generalize
the quantum entropy using singular values of arbitrary rectangular
matrices to measure the randomness and the quality of denoising
operation, this new definition of entropy can be implemented to
compare the performance analysis of filtering methods. In the second
part, we apply the concept of pure state in quantum formalism
to generalize the maximum entropy method for narrowband and
farfield source localization problem. Several computer simulation
results are illustrated to demonstrate the effectiveness of the proposed
techniques.
Abstract: Particle size distribution, the most important
characteristics of aerosols, is obtained through electrical
characterization techniques. The dynamics of charged nanoparticles
under the influence of electric field in Electrical Mobility
Spectrometer (EMS) reveals the size distribution of these particles.
The accuracy of this measurement is influenced by flow conditions,
geometry, electric field and particle charging process, therefore by
the transfer function (transfer matrix) of the instrument. In this work,
a wire-cylinder corona charger was designed and the combined fielddiffusion
charging process of injected poly-disperse aerosol particles
was numerically simulated as a prerequisite for the study of a
multichannel EMS. The result, a cloud of particles with no uniform
charge distribution, was introduced to the EMS. The flow pattern and
electric field in the EMS were simulated using Computational Fluid
Dynamics (CFD) to obtain particle trajectories in the device and
therefore to calculate the reported signal by each electrometer.
According to the output signals (resulted from bombardment of
particles and transferring their charges as currents), we proposed a
modification to the size of detecting rings (which are connected to
electrometers) in order to evaluate particle size distributions more
accurately. Based on the capability of the system to transfer
information contents about size distribution of the injected particles,
we proposed a benchmark for the assessment of optimality of the
design. This method applies the concept of Von Neumann entropy
and borrows the definition of entropy from information theory
(Shannon entropy) to measure optimality. Entropy, according to the
Shannon entropy, is the ''average amount of information contained in
an event, sample or character extracted from a data stream''.
Evaluating the responses (signals) which were obtained via various
configurations of detecting rings, the best configuration which gave
the best predictions about the size distributions of injected particles,
was the modified configuration. It was also the one that had the
maximum amount of entropy. A reasonable consistency was also
observed between the accuracy of the predictions and the entropy
content of each configuration. In this method, entropy is extracted
from the transfer matrix of the instrument for each configuration.
Ultimately, various clouds of particles were introduced to the
simulations and predicted size distributions were compared to the
exact size distributions.
Abstract: In this paper we propose a novel methodology for
extracting a road network and its nodes from satellite images of
Algeria country.
This developed technique is a progress of our previous research
works. It is founded on the information theory and the mathematical
morphology; the information theory and the mathematical
morphology are combined together to extract and link the road
segments to form a road network and its nodes.
We therefore have to define objects as sets of pixels and to study
the shape of these objects and the relations that exist between them.
In this approach, geometric and radiometric features of roads are
integrated by a cost function and a set of selected points of a crossing
road. Its performances were tested on satellite images of Algeria
country.
Abstract: In this short paper, new properties of transition matrix were introduced. Eigen values for small order transition matrices are calculated in flexible method. For benefit of these properties applications of these properties were studied in the solution of Markov's chain via steady state vector, and information theory via channel entropy. The implemented test examples were promised for usages.
Abstract: Entropy is a key measure in studies related to information theory and its many applications. Campbell for the first time recognized that the exponential of the Shannon’s entropy is just the size of the sample space, when distribution is uniform. Here is the idea to study exponentials of Shannon’s and those other entropy generalizations that involve logarithmic function for a probability distribution in general. In this paper, we introduce a measure of sample space, called ‘entropic measure of a sample space’, with respect to the underlying distribution. It is shown in both discrete and continuous cases that this new measure depends on the parameters of the distribution on the sample space - same sample space having different ‘entropic measures’ depending on the distributions defined on it. It was noted that Campbell’s idea applied for R`enyi’s parametric entropy of a given order also. Knowing that parameters play a role in providing suitable choices and extended applications, paper studies parametric entropic measures of sample spaces also. Exponential entropies related to Shannon’s and those generalizations that have logarithmic functions, i.e. are additive have been studies for wider understanding and applications. We propose and study exponential entropies corresponding to non additive entropies of type (α, β), which include Havard and Charvˆat entropy as a special case.
Abstract: Logarithms reduce products to sums and powers to products; they play an important role in signal processing, communication and information theory. They are primarily
used for hardware calculations, handling multiplications, divisions,
powers, and roots effectively. There are three commonly
used bases for logarithms; the logarithm with base-10 is called
the common logarithm, the natural logarithm with base-e and
the binary logarithm with base-2. This paper demonstrates different
methods of calculation for log2 showing the complexity
of each and finds out the most accurate and efficient besides
giving insights to their hardware design. We present a new
method called Floor Shift for fast calculation of log2, and
then we combine this algorithm with Taylor series to improve
the accuracy of the output, we illustrate that by using two
examples. We finally compare the algorithms and conclude
with our remarks.
Abstract: This paper presents a design of source encoding
calculator software which applies the two famous algorithms in the
field of information theory- the Shannon-Fano and the Huffman
schemes. This design helps to easily realize the algorithms without
going into a cumbersome, tedious and prone to error manual
mechanism of encoding the signals during the transmission. The
work describes the design of the software, how it works, comparison
with related works, its efficiency, its usefulness in the field of
information technology studies and the future prospects of the
software to engineers, students, technicians and alike. The designed
“Encodia" software has been developed, tested and found to meet the
intended requirements. It is expected that this application will help
students and teaching staff in their daily doing of information theory
related tasks. The process is ongoing to modify this tool so that it can
also be more intensely useful in research activities on source coding.
Abstract: Recently many research has been conducted to
retrieve pertinent parameters and adequate models for automatic
music genre classification. In this paper, two measures based upon
information theory concepts are investigated for mapping the features
space to decision space. A Gaussian Mixture Model (GMM) is used
as a baseline and reference system. Various strategies are proposed
for training and testing sessions with matched or mismatched
conditions, long training and long testing, long training and short
testing. For all experiments, the file sections used for testing are
never been used during training. With matched conditions all
examined measures yield the best and similar scores (almost 100%).
With mismatched conditions, the proposed measures yield better
scores than the GMM baseline system, especially for the short testing
case. It is also observed that the average discrimination information
measure is most appropriate for music category classifications and on
the other hand the divergence measure is more suitable for music
subcategory classifications.
Abstract: We introduce a novel approach to measuring how
humans learn based on techniques from information theory and
apply it to the oriental game of Go. We show that the total amount
of information observable in human strategies, called the strategic
information, remains constant for populations of players of differing
skill levels for well studied patterns of play. This is despite the very
large amount of knowledge required to progress from the recreational
players at one end of our spectrum to the very best and most
experienced players in the world at the other and is in contrast to
the idea that having more knowledge might imply more 'certainty'
in what move to play next. We show this is true for very local
up to medium sized board patterns, across a variety of different
moves using 80,000 game records. Consequences for theoretical and
practical AI are outlined.
Abstract: The theory of rough sets is generalized by using a
filter. The filter is induced by binary relations and it is used to
generalize the basic rough set concepts. The knowledge
representations and processing of binary relations in the style of
rough set theory are investigated.
Abstract: A number of competing methodologies have been developed
to identify genes and classify DNA sequences into coding
and non-coding sequences. This classification process is fundamental
in gene finding and gene annotation tools and is one of the most
challenging tasks in bioinformatics and computational biology. An
information theory measure based on mutual information has shown
good accuracy in classifying DNA sequences into coding and noncoding.
In this paper we describe a species independent iterative
approach that distinguishes coding from non-coding sequences using
the mutual information measure (MIM). A set of sixty prokaryotes is
used to extract universal training data. To facilitate comparisons with
the published results of other researchers, a test set of 51 bacterial
and archaeal genomes was used to evaluate MIM. These results
demonstrate that MIM produces superior results while remaining
species independent.
Abstract: In the literature of information theory, there is
necessity for comparing the different measures of fuzzy entropy and
this consequently, gives rise to the need for normalizing measures of
fuzzy entropy. In this paper, we have discussed this need and hence
developed some normalized measures of fuzzy entropy. It is also
desirable to maximize entropy and to minimize directed divergence
or distance. Keeping in mind this idea, we have explained the method
of optimizing different measures of fuzzy entropy.
Abstract: The performance of time-reversal MUSIC algorithm will be dramatically degrades in presence of strong noise and multiple scattering (i.e. when scatterers are close to each other). This is due to error in determining the number of scatterers. The present paper provides a new approach to alleviate such a problem using an information theoretic criterion referred as minimum description length (MDL). The merits of the novel approach are confirmed by the numerical examples. The results indicate the time-reversal MUSIC yields accurate estimate of the target locations with considerable noise and multiple scattering in the received signals.
Abstract: In this paper, the problem of estimating the optimal
radio capacity of a single-cell spread spectrum (SS) multiple-inputmultiple-
output (MIMO) system operating in a Rayleigh fading environment
is examined. The optimisation between the radio capacity
and the theoretically achievable average channel capacity (in the
sense of information theory) per user of a MIMO single-cell SS system
operating in a Rayleigh fading environment is presented. Then,
the spectral efficiency is estimated in terms of the achievable average
channel capacity per user, during the operation over a broadcast
time-varying link, and leads to a simple novel-closed form expression
for the optimal radio capacity value based on the maximization
of the achieved spectral efficiency. Numerical results are presented to
illustrate the proposed analysis.
Abstract: Information theory and Statistics play an important role in Biological Sciences when we use information measures for the study of diversity and equitability. In this communication, we develop the link among the three disciplines and prove that sampling distributions can be used to develop new information measures. Our study will be an interdisciplinary and will find its applications in Biological systems.
Abstract: The decoding of Low-Density Parity-Check (LDPC) codes is operated over a redundant structure known as the bipartite graph, meaning that the full set of bit nodes is not absolutely necessary for decoder convergence. In 2008, Soyjaudah and Catherine designed a recovery algorithm for LDPC codes based on this assumption and showed that the error-correcting performance of their codes outperformed conventional LDPC Codes. In this work, the use of the recovery algorithm is further explored to test the performance of LDPC codes while the number of iterations is progressively increased. For experiments conducted with small blocklengths of up to 800 bits and number of iterations of up to 2000, the results interestingly demonstrate that contrary to conventional wisdom, the error-correcting performance keeps increasing with increasing number of iterations.