Abstract: In neural networks, when new patterns are learned by a network, the new information radically interferes with previously stored patterns. This drawback is called catastrophic forgetting or catastrophic interference. In this paper, we propose a biologically inspired neural network model which overcomes this problem. The proposed model consists of two distinct networks: one is a Hopfield type of chaotic associative memory and the other is a multilayer neural network. We consider that these networks correspond to the hippocampus and the neocortex of the brain, respectively. Information given is firstly stored in the hippocampal network with fast learning algorithm. Then the stored information is recalled by chaotic behavior of each neuron in the hippocampal network. Finally, it is consolidated in the neocortical network by using pseudopatterns. Computer simulation results show that the proposed model has much better ability to avoid catastrophic forgetting in comparison with conventional models.
Abstract: .Hardware realization of a Neural Network (NN), to a large extent depends on the efficient implementation of a single neuron. FPGA-based reconfigurable computing architectures are suitable for hardware implementation of neural networks. FPGA realization of ANNs with a large number of neurons is still a challenging task. This paper discusses the issues involved in implementation of a multi-input neuron with linear/nonlinear excitation functions using FPGA. Implementation method with resource/speed tradeoff is proposed to handle signed decimal numbers. The VHDL coding developed is tested using Xilinx XC V50hq240 Chip. To improve the speed of operation a lookup table method is used. The problems involved in using a lookup table (LUT) for a nonlinear function is discussed. The percentage saving in resource and the improvement in speed with an LUT for a neuron is reported. An attempt is also made to derive a generalized formula for a multi-input neuron that facilitates to estimate approximately the total resource requirement and speed achievable for a given multilayer neural network. This facilitates the designer to choose the FPGA capacity for a given application. Using the proposed method of implementation a neural network based application, namely, a Space vector modulator for a vector-controlled drive is presented
Abstract: The model of neural networks on the small-world
topology, with metric (local and random connectivity) is investigated.
The synaptic weights are random, driving the network towards a
chaotic state for the neural activity. An ordered macroscopic neuron
state is induced by a bias in the network connections. When the
connections are mainly local, the network emulates a block-like
structure. It is found that the topology and the bias compete to
influence the network to evolve into a global or a block activity
ordering, according to the initial conditions.
Abstract: Statistical analysis of electrophysiological recordings
obtained under, e.g. tactile, stimulation frequently suggests participation
in the network dynamics of experimentally unobserved “hidden"
neurons. Such interneurons making synapses to experimentally
recorded neurons may strongly alter their dynamical responses to
the stimuli. We propose a mathematical method that formalizes this
possibility and provides an algorithm for inferring on the presence
and dynamics of hidden neurons based on fitting of the experimental
data to spike trains generated by the network model. The model
makes use of Integrate and Fire neurons “chemically" coupled
through exponentially decaying synaptic currents. We test the method
on simulated data and also provide an example of its application to
the experimental recording from the Dorsal Column Nuclei neurons
of the rat under tactile stimulation of a hind limb.
Abstract: In this paper, an Arabic letter recognition system based on Artificial Neural Networks (ANNs) and statistical analysis for feature extraction is presented. The ANN is trained using the Least Mean Squares (LMS) algorithm. In the proposed system, each typed Arabic letter is represented by a matrix of binary numbers that are used as input to a simple feature extraction system whose output, in addition to the input matrix, are fed to an ANN. Simulation results are provided and show that the proposed system always produces a lower Mean Squared Error (MSE) and higher success rates than the current ANN solutions.
Abstract: A feature weighting and selection method is proposed
which uses the structure of a weightless neuron and exploits the
principles that govern the operation of Genetic Algorithms and
Evolution. Features are coded onto chromosomes in a novel way
which allows weighting information regarding the features to be
directly inferred from the gene values. The proposed method is
significant in that it addresses several problems concerned with
algorithms for feature selection and weighting as well as providing
significant advantages such as speed, simplicity and suitability for
real-time systems.
Abstract: Saturated hydraulic conductivity is one of the soil
hydraulic properties which is widely used in environmental studies
especially subsurface ground water. Since, its direct measurement is
time consuming and therefore costly, indirect methods such as
pedotransfer functions have been developed based on multiple linear
regression equations and neural networks model in order to estimate
saturated hydraulic conductivity from readily available soil
properties e.g. sand, silt, and clay contents, bulk density, and organic
matter. The objective of this study was to develop neural networks
(NNs) model to estimate saturated hydraulic conductivity from
available parameters such as sand and clay contents, bulk density,
van Genuchten retention model parameters (i.e. r
θ , α , and n) as well
as effective porosity. We used two methods to calculate effective
porosity: : (1) eff s FC φ =θ -θ , and (2) inf φ =θ -θ eff s , in which s
θ is
saturated water content, FC θ is water content retained at -33 kPa
matric potential, and inf θ is water content at the inflection point.
Total of 311 soil samples from the UNSODA database was divided
into three groups as 187 for the training, 62 for the validation (to
avoid over training), and 62 for the test of NNs model. A commercial
neural network toolbox of MATLAB software with a multi-layer
perceptron model and back propagation algorithm were used for the
training procedure. The statistical parameters such as correlation
coefficient (R2), and mean square error (MSE) were also used to
evaluate the developed NNs model. The best number of neurons in
the middle layer of NNs model for methods (1) and (2) were
calculated 44 and 6, respectively. The R2 and MSE values of the test
phase were determined for method (1), 0.94 and 0.0016, and for
method (2), 0.98 and 0.00065, respectively, which shows that method
(2) estimates saturated hydraulic conductivity better than method (1).
Abstract: A multilayer self organizing neural neural network
(MLSONN) architecture for binary object extraction, guided by a beta
activation function and characterized by backpropagation of errors
estimated from the linear indices of fuzziness of the network output
states, is discussed. Since the MLSONN architecture is designed to
operate in a single point fixed/uniform thresholding scenario, it does
not take into cognizance the heterogeneity of image information in
the extraction process. The performance of the MLSONN architecture
with representative values of the threshold parameters of the beta
activation function employed is also studied. A three layer bidirectional
self organizing neural network (BDSONN) architecture
comprising fully connected neurons, for the extraction of objects from
a noisy background and capable of incorporating the underlying image
context heterogeneity through variable and adaptive thresholding,
is proposed in this article. The input layer of the network architecture
represents the fuzzy membership information of the image scene to
be extracted. The second layer (the intermediate layer) and the final
layer (the output layer) of the network architecture deal with the self
supervised object extraction task by bi-directional propagation of the
network states. Each layer except the output layer is connected to the
next layer following a neighborhood based topology. The output layer
neurons are in turn, connected to the intermediate layer following
similar topology, thus forming a counter-propagating architecture
with the intermediate layer. The novelty of the proposed architecture
is that the assignment/updating of the inter-layer connection weights
are done using the relative fuzzy membership values at the constituent
neurons in the different network layers. Another interesting feature
of the network lies in the fact that the processing capabilities of
the intermediate and the output layer neurons are guided by a beta
activation function, which uses image context sensitive adaptive
thresholding arising out of the fuzzy cardinality estimates of the
different network neighborhood fuzzy subsets, rather than resorting to
fixed and single point thresholding. An application of the proposed
architecture for object extraction is demonstrated using a synthetic
and a real life image. The extraction efficiency of the proposed
network architecture is evaluated by a proposed system transfer index
characteristic of the network.
Abstract: The performance of sensor-less controlled induction
motor drive depends on the accuracy of the estimated speed.
Conventional estimation techniques being mathematically complex
require more execution time resulting in poor dynamic response. The
nonlinear mapping capability and powerful learning algorithms of
neural network provides a promising alternative for on-line speed
estimation. The on-line speed estimator requires the NN model to be
accurate, simpler in design, structurally compact and computationally
less complex to ensure faster execution and effective control in real
time implementation. This in turn to a large extent depends on the
type of Neural Architecture. This paper investigates three types of
neural architectures for on-line speed estimation and their
performance is compared in terms of accuracy, structural
compactness, computational complexity and execution time. The
suitable neural architecture for on-line speed estimation is identified
and the promising results obtained are presented.
Abstract: Investigation of soil properties like Cation Exchange
Capacity (CEC) plays important roles in study of environmental
reaserches as the spatial and temporal variability of this property
have been led to development of indirect methods in estimation of
this soil characteristic. Pedotransfer functions (PTFs) provide an
alternative by estimating soil parameters from more readily available
soil data. 70 soil samples were collected from different horizons of
15 soil profiles located in the Ziaran region, Qazvin province, Iran.
Then, multivariate regression and neural network model (feedforward
back propagation network) were employed to develop a
pedotransfer function for predicting soil parameter using easily
measurable characteristics of clay and organic carbon. The
performance of the multivariate regression and neural network model
was evaluated using a test data set. In order to evaluate the models,
root mean square error (RMSE) was used. The value of RMSE and
R2 derived by ANN model for CEC were 0.47 and 0.94 respectively,
while these parameters for multivariate regression model were 0.65
and 0.88 respectively. Results showed that artificial neural network
with seven neurons in hidden layer had better performance in
predicting soil cation exchange capacity than multivariate regression.
Abstract: In this paper, an approach to reduce the computation steps required by fast neural networksfor the searching process is presented. The principle ofdivide and conquer strategy is applied through imagedecomposition. Each image is divided into small in sizesub-images and then each one is tested separately usinga fast neural network. The operation of fast neuralnetworks based on applying cross correlation in thefrequency domain between the input image and theweights of the hidden neurons. Compared toconventional and fast neural networks, experimentalresults show that a speed up ratio is achieved whenapplying this technique to locate human facesautomatically in cluttered scenes. Furthermore, fasterface detection is obtained by using parallel processingtechniques to test the resulting sub-images at the sametime using the same number of fast neural networks. Incontrast to using only fast neural networks, the speed upratio is increased with the size of the input image whenusing fast neural networks and image decomposition.
Abstract: During signal transmission, the combined effect of the
transmitter filter, the transmission medium, and additive white
Gaussian noise (AWGN) are included in the channel which distort
and add noise to the signal. This causes the well defined signal
constellation to spread causing errors in bit detection. A compact pi
neural network with minimum number of nodes is proposed. The
replacement of summation at each node by multiplication results in
more powerful mapping. The resultant pi network is tested on six
different channels.
Abstract: Calcium [Ca2+] dynamics is studied as a potential form
of neuron excitability that can control many irregular processes like
metabolism, secretion etc. Ca2+ ion enters presynaptic terminal and
increases the synaptic strength and thus triggers the neurotransmitter
release. The modeling and analysis of calcium dynamics in neuron
cell becomes necessary for deeper understanding of the processes
involved. A mathematical model has been developed for cylindrical
shaped neuron cell by incorporating physiological parameters like
buffer, diffusion coefficient, and association rate. Appropriate initial
and boundary conditions have been framed. The closed form solution
has been developed in terms of modified Bessel function. A computer
program has been developed in MATLAB 7.11 for the whole
approach.
Abstract: In this paper, we study the application of Extreme
Learning Machine (ELM) algorithm for single layered feedforward
neural networks to non-linear chaotic time series problems. In this
algorithm the input weights and the hidden layer bias are randomly
chosen. The ELM formulation leads to solving a system of linear
equations in terms of the unknown weights connecting the hidden
layer to the output layer. The solution of this general system of
linear equations will be obtained using Moore-Penrose generalized
pseudo inverse. For the study of the application of the method we
consider the time series generated by the Mackey Glass delay
differential equation with different time delays, Santa Fe A and
UCR heart beat rate ECG time series. For the choice of sigmoid,
sin and hardlim activation functions the optimal values for the
memory order and the number of hidden neurons which give the
best prediction performance in terms of root mean square error are
determined. It is observed that the results obtained are in close
agreement with the exact solution of the problems considered
which clearly shows that ELM is a very promising alternative
method for time series prediction.
Abstract: Conceptualization strengthens intelligent systems in generalization skill, effective knowledge representation, real-time inference, and managing uncertain and indefinite situations in addition to facilitating knowledge communication for learning agents situated in real world. Concept learning introduces a way of abstraction by which the continuous state is formed as entities called concepts which are connected to the action space and thus, they illustrate somehow the complex action space. Of computational concept learning approaches, action-based conceptualization is favored because of its simplicity and mirror neuron foundations in neuroscience. In this paper, a new biologically inspired concept learning approach based on the probabilistic framework is proposed. This approach exploits and extends the mirror neuron-s role in conceptualization for a reinforcement learning agent in nondeterministic environments. In the proposed method, instead of building a huge numerical knowledge, the concepts are learnt gradually from rewards through interaction with the environment. Moreover the probabilistic formation of the concepts is employed to deal with uncertain and dynamic nature of real problems in addition to the ability of generalization. These characteristics as a whole distinguish the proposed learning algorithm from both a pure classification algorithm and typical reinforcement learning. Simulation results show advantages of the proposed framework in terms of convergence speed as well as generalization and asymptotic behavior because of utilizing both success and failures attempts through received rewards. Experimental results, on the other hand, show the applicability and effectiveness of the proposed method in continuous and noisy environments for a real robotic task such as maze as well as the benefits of implementing an incremental learning scenario in artificial agents.
Abstract: The paper presents a new hybridization methodology involving Neural, Fuzzy and Rough Computing. A Rough Sets based approximation technique has been proposed based on a certain Neuro – Fuzzy architecture. A New Rough Neuron composition consisting of a combination of a Lower Bound neuron and a Boundary neuron has also been described. The conventional convergence of error in back propagation has been given away for a new framework based on 'Output Excitation Factor' and an inverse input transfer function. The paper also presents a brief comparison of performances, of the existing Rough Neural Networks and ANFIS architecture against the proposed methodology. It can be observed that the rough approximation based neuro-fuzzy architecture is superior to its counterparts.
Abstract: Mutations of the telomeric copy of the survival motor neuron 1 (SMN1) gene cause spinal muscular atrophy. A deletion of the Eef1a2 gene leads to lower motor neuron degeneration in wasted mice. Indirect evidences have been shown that the eEF1A protein family may interact with SMN, and our previous study showed that abnormalities of neuromuscular junctions in wasted mice were similar to those of Smn mutant mice. To determine potential colocalization between SMN and tissue-specific translation elongation factor 1A2 (eEF1A2), an immunochemical analysis of HeLa cells transfected with the plasmid pcDNA3.1(+)C-hEEF1A2- myc and a new quantitative test of colocalization by intensity correlation analysis (ICA) was used to explore the association of SMN and eEF1A2. Here the results showed that eEF1A2 redistributed from the cytoplasm to the nucleus in response to serum and epidermal growth factor. In the cytoplasm, compelling evidence showed that staining for myc-tagged eEF1A2 varied in synchrony with that for SMN, consistent with the formation of a SMN-eEF1A2 complex in the cytoplasm of HeLa cells. These findings suggest that eEF1A2 may colocalize with SMN in the cytoplasm and may be a component of the SMN complex. However, the limitation of the ICA method is an inability to resolve colocalization in components of small organelles such as the nucleus.
Abstract: The development of Artificial Neural Networks
(ANNs) is usually a slow process in which the human expert has to
test several architectures until he finds the one that achieves best
results to solve a certain problem. This work presents a new
technique that uses Genetic Programming (GP) for automatically
generating ANNs. To do this, the GP algorithm had to be changed in
order to work with graph structures, so ANNs can be developed. This
technique also allows the obtaining of simplified networks that solve
the problem with a small group of neurons. In order to measure the
performance of the system and to compare the results with other
ANN development methods by means of Evolutionary Computation
(EC) techniques, several tests were performed with problems based
on some of the most used test databases. The results of those
comparisons show that the system achieves good results comparable
with the already existing techniques and, in most of the cases, they
worked better than those techniques.
Abstract: The objective of this paper is to develop a neural
network-based residual generator to detect the fault in the actuators
for a specific communication satellite in its attitude control system
(ACS). First, a dynamic multilayer perceptron network with dynamic
neurons is used, those neurons correspond a second order linear
Infinite Impulse Response (IIR) filter and a nonlinear activation
function with adjustable parameters. Second, the parameters from the
network are adjusted to minimize a performance index specified by
the output estimated error, with the given input-output data collected
from the specific ACS. Then, the proposed dynamic neural network
is trained and applied for detecting the faults injected to the wheel,
which is the main actuator in the normal mode for the communication
satellite. Then the performance and capabilities of the proposed
network were tested and compared with a conventional model-based
observer residual, showing the differences between these two
methods, and indicating the benefit of the proposed algorithm to
know the real status of the momentum wheel. Finally, the application
of the methods in a satellite ground station is discussed.
Abstract: This paper presents a general trainable framework
for fast and robust upright human face and non-human object
detection and verification in static images. To enhance the
performance of the detection process, the technique we develop is
based on the combination of fast neural network (FNN) and
classical neural network (CNN). In FNN, a useful correlation is
exploited to sustain high level of detection accuracy between input
image and the weight of the hidden neurons. This is to enable the
use of Fourier transform that significantly speed up the time
detection. The combination of CNN is responsible to verify the
face region. A bootstrap algorithm is used to collect non human
object, which adds the false detection to the training process of the
human and non-human object. Experimental results on test images
with both simple and complex background demonstrate that the
proposed method has obtained high detection rate and low false
positive rate in detecting both human face and non-human object.