Abstract: Time Series Forecasting (TSF) is used to predict the target variables at a future time point based on the learning from previous time points. To keep the problem tractable, learning methods use data from a fixed length window in the past as an explicit input. In this paper, we study how the performance of predictive models change as a function of different look-back window sizes and different amounts of time to predict into the future. We also consider the performance of the recent attention-based transformer models, which had good success in the image processing and natural language processing domains. In all, we compare four different deep learning methods (Recurrent Neural Network (RNN), Long Short-term Memory (LSTM), Gated Recurrent Units (GRU), and Transformer) along with a baseline method. The dataset (hourly) we used is the Beijing Air Quality Dataset from the website of University of California, Irvine (UCI), which includes a multivariate time series of many factors measured on an hourly basis for a period of 5 years (2010-14). For each model, we also report on the relationship between the performance and the look-back window sizes and the number of predicted time points into the future. Our experiments suggest that Transformer models have the best performance with the lowest Mean Absolute Errors (MAE = 14.599, 23.273) and Root Mean Square Errors (RSME = 23.573, 38.131) for most of our single-step and multi-steps predictions. The best size for the look-back window to predict 1 hour into the future appears to be one day, while 2 or 4 days perform the best to predict 3 hours into the future.
Abstract: The low availability of well-trained, unlimited, dynamic-access models for specific languages makes it hard for corporate users to adopt quick translation techniques and incorporate them into product solutions. As translation tasks increasingly require a dynamic sequence learning curve; stable, cost-free opensource models are scarce. We survey and compare current translation techniques and propose a modified sequence to sequence model repurposed with attention techniques. Sequence learning using an encoder-decoder model is now paving the path for higher precision levels in translation. Using a Convolutional Neural Network (CNN) encoder and a Recurrent Neural Network (RNN) decoder background, we use Fairseq tools to produce an end-to-end bilingually trained Spanish-English machine translation model including source language detection. We acquire competitive results using a duo-lingo-corpus trained model to provide for prospective, ready-made plug-in use for compound sentences and document translations. Our model serves a decent system for large, organizational data translation needs. While acknowledging its shortcomings and future scope, it also identifies itself as a well-optimized deep neural network model and solution.
Abstract: This paper presents and benchmarks a number of
end-to-end Deep Learning based models for metaphor detection in
Greek. We combine Convolutional Neural Networks and Recurrent
Neural Networks with representation learning to bear on the metaphor
detection problem for the Greek language. The models presented
achieve exceptional accuracy scores, significantly improving the
previous state-of-the-art results, which had already achieved accuracy
0.82. Furthermore, no special preprocessing, feature engineering or
linguistic knowledge is used in this work. The methods presented
achieve accuracy of 0.92 and F-score 0.92 with Convolutional
Neural Networks (CNNs) and bidirectional Long Short Term Memory
networks (LSTMs). Comparable results of 0.91 accuracy and 0.91
F-score are also achieved with bidirectional Gated Recurrent Units
(GRUs) and Convolutional Recurrent Neural Nets (CRNNs). The
models are trained and evaluated only on the basis of training tuples,
the related sentences and their labels. The outcome is a state-of-the-art
collection of metaphor detection models, trained on limited labelled
resources, which can be extended to other languages and similar
tasks.
Abstract: The aim of this study is to predict breast cancer and to construct a supportive model that will stimulate a more reliable prediction as a factor that is fundamental for public health. In this study, we utilize general regression neural networks (GRNN) to replace the normal predictions with prediction periods to achieve a reasonable percentage of confidence. The mechanism employed here utilises a machine learning system called conformal prediction (CP), in order to assign consistent confidence measures to predictions, which are combined with GRNN. We apply the resulting algorithm to the problem of breast cancer diagnosis. The results show that the prediction constructed by this method is reasonable and could be useful in practice.
Abstract: Over the past decade, there have been promising developments in Natural Language Processing (NLP) with several investigations of approaches focusing on Recognizing Textual Entailment (RTE). These models include models based on lexical similarities, models based on formal reasoning, and most recently deep neural models. In this paper, we present a sentence encoding model that exploits the sentence-to-sentence relation information for RTE. In terms of sentence modeling, Convolutional neural network (CNN) and recurrent neural networks (RNNs) adopt different approaches. RNNs are known to be well suited for sequence modeling, whilst CNN is suited for the extraction of n-gram features through the filters and can learn ranges of relations via the pooling mechanism. We combine the strength of RNN and CNN as stated above to present a unified model for the RTE task. Our model basically combines relation vectors computed from the phrasal representation of each sentence and final encoded sentence representations. Firstly, we pass each sentence through a convolutional layer to extract a sequence of higher-level phrase representation for each sentence from which the first relation vector is computed. Secondly, the phrasal representation of each sentence from the convolutional layer is fed into a Bidirectional Long Short Term Memory (Bi-LSTM) to obtain the final sentence representations from which a second relation vector is computed. The relations vectors are combined and then used in then used in the same fashion as attention mechanism over the Bi-LSTM outputs to yield the final sentence representations for the classification. Experiment on the Stanford Natural Language Inference (SNLI) corpus suggests that this is a promising technique for RTE.
Abstract: In this paper, we present an optimization technique or
a learning algorithm using the hybrid architecture by combining the
most popular sequence recognition models such as Recurrent Neural
Networks (RNNs) and Hidden Markov models (HMMs). In order to
improve the sequence/pattern recognition/classification performance
by applying a hybrid/neural symbolic approach, a gradient descent
learning algorithm is developed using the Real Time Recurrent
Learning of Recurrent Neural Network for processing the knowledge
represented in trained Hidden Markov Models. The developed hybrid
algorithm is implemented on automata theory as a sample test beds
and the performance of the designed algorithm is demonstrated and
evaluated on learning the deterministic finite state automata.
Abstract: Predicting earnings management is vital for the capital
market participants, financial analysts and managers. The aim of this
research is attempting to respond to this query: Is there a significant
difference between the regression model and neural networks’
models in predicting earnings management, and which one leads to a
superior prediction of it? In approaching this question, a Linear
Regression (LR) model was compared with two neural networks
including Multi-Layer Perceptron (MLP), and Generalized
Regression Neural Network (GRNN). The population of this study
includes 94 listed companies in Tehran Stock Exchange (TSE)
market from 2003 to 2011. After the results of all models were
acquired, ANOVA was exerted to test the hypotheses. In general, the
summary of statistical results showed that the precision of GRNN did
not exhibit a significant difference in comparison with MLP. In
addition, the mean square error of the MLP and GRNN showed a
significant difference with the multi variable LR model. These
findings support the notion of nonlinear behavior of the earnings
management. Therefore, it is more appropriate for capital market
participants to analyze earnings management based upon neural
networks techniques, and not to adopt linear regression models.
Abstract: The Cone Penetration Test (CPT) is a common in-situ
test which generally investigates a much greater volume of soil more
quickly than possible from sampling and laboratory tests. Therefore,
it has the potential to realize both cost savings and assessment of soil
properties rapidly and continuously. The principle objective of this
paper is to demonstrate the feasibility and efficiency of using
artificial neural networks (ANNs) to predict the soil angle of internal
friction (Φ) and the soil modulus of elasticity (E) from CPT results
considering the uncertainties and non-linearities of the soil. In
addition, ANNs are used to study the influence of different
parameters and recommend which parameters should be included as
input parameters to improve the prediction. Neural networks discover
relationships in the input data sets through the iterative presentation
of the data and intrinsic mapping characteristics of neural topologies.
General Regression Neural Network (GRNN) is one of the powerful
neural network architectures which is utilized in this study. A large
amount of field and experimental data including CPT results, plate
load tests, direct shear box, grain size distribution and calculated data
of overburden pressure was obtained from a large project in the
United Arab Emirates. This data was used for the training and the
validation of the neural network. A comparison was made between
the obtained results from the ANN's approach, and some common
traditional correlations that predict Φ and E from CPT results with
respect to the actual results of the collected data. The results show
that the ANN is a very powerful tool. Very good agreement was
obtained between estimated results from ANN and actual measured
results with comparison to other correlations available in the
literature. The study recommends some easily available parameters
that should be included in the estimation of the soil properties to
improve the prediction models. It is shown that the use of friction
ration in the estimation of Φ and the use of fines content in the
estimation of E considerable improve the prediction models.
Abstract: This paper presents a comparative study between two
neural network models namely General Regression Neural Network
(GRNN) and Back Propagation Neural Network (BPNN) are used
to estimate radial overcut produced during Electrical Discharge
Machining (EDM). Four input parameters have been employed:
discharge current (Ip), pulse on time (Ton), Duty fraction (Tau) and
discharge voltage (V). Recently, artificial intelligence techniques, as
it is emerged as an effective tool that could be used to replace
time consuming procedures in various scientific or engineering
applications, explicitly in prediction and estimation of the complex
and nonlinear process. The both networks are trained, and the
prediction results are tested with the unseen validation set of the
experiment and analysed. It is found that the performance of both the
networks are found to be in good agreement with average percentage
error less than 11% and the correlation coefficient obtained for the
validation data set for GRNN and BPNN is more than 91%. However,
it is much faster to train GRNN network than a BPNN and GRNN is
often more accurate than BPNN. GRNN requires more memory space
to store the model, GRNN features fast learning that does not require
an iterative procedure, and highly parallel structure. GRNN networks
are slower than multilayer perceptron networks at classifying new
cases.
Abstract: Several meteorological parameters were used for the
prediction of monthly average daily global solar radiation on
horizontal using recurrent neural networks (RNNs). Climatological
data and measures, mainly air temperature, humidity, sunshine
duration, and wind speed between 1995 and 2007 were used to design
and validate a feed forward and recurrent neural network based
prediction systems. In this paper we present our reference system
based on a feed-forward multilayer perceptron (MLP) as well as the
proposed approach based on an RNN model. The obtained results
were promising and comparable to those obtained by other existing
empirical and neural models. The experimental results showed the
advantage of RNNs over simple MLPs when we deal with time series
solar radiation predictions based on daily climatological data.
Abstract: This research presents a handwritten signature recognition based on angle feature vector using Artificial Neural Network (ANN). Each signature image will be represented by an Angle vector. The feature vector will constitute the input to the ANN. The collection of signature images will be divided into two sets. One set will be used for training the ANN in a supervised fashion. The other set which is never seen by the ANN will be used for testing. After training, the ANN will be tested for recognition of the signature. When the signature is classified correctly, it is considered correct recognition otherwise it is a failure.
Abstract: Series of experimental tests were conducted on a
section of a 660 kW wind turbine blade to measure the pressure
distribution of this model oscillating in plunging motion. In order to
minimize the amount of data required to predict aerodynamic loads
of the airfoil, a General Regression Neural Network, GRNN, was
trained using the measured experimental data. The network once
proved to be accurate enough, was used to predict the flow behavior
of the airfoil for the desired conditions.
Results showed that with using a few of the acquired data, the
trained neural network was able to predict accurate results with
minimal errors when compared with the corresponding measured
values. Therefore with employing this trained network the
aerodynamic coefficients of the plunging airfoil, are predicted
accurately at different oscillation frequencies, amplitudes, and angles
of attack; hence reducing the cost of tests while achieving acceptable
accuracy.
Abstract: In this paper, we investigate a blind channel estimation method for Multi-carrier CDMA systems that use a subspace decomposition technique. This technique exploits the orthogonality property between the noise subspace and the received user codes to obtain channel of each user. In the past we used Singular Value Decomposition (SVD) technique but SVD have most computational complexity so in this paper use a new algorithm called URV Decomposition, which serve as an intermediary between the QR decomposition and SVD, replaced in SVD technique to track the noise space of the received data. Because of the URV decomposition has almost the same estimation performance as the SVD, but has less computational complexity.
Abstract: The key to the continued success of ANN depends, considerably,
on the use of hybrid structures implemented on cooperative
frame-works. Hybrid architectures provide the ability to the ANN
to validate heterogeneous learning paradigms. This work describes
the implementation of a set of Distributed and Hybrid ANN models
for Character Recognition applied to Anglo-Assamese scripts. The
objective is to describe the effectiveness of Hybrid ANN setups as
innovative means of neural learning for an application like multilingual
handwritten character and numeral recognition.
Abstract: We present a hybrid architecture of recurrent neural
networks (RNNs) inspired by hidden Markov models (HMMs). We
train the hybrid architecture using genetic algorithms to learn and
represent dynamical systems. We train the hybrid architecture on a
set of deterministic finite-state automata strings and observe the
generalization performance of the hybrid architecture when presented
with a new set of strings which were not present in the training data
set. In this way, we show that the hybrid system of HMM and RNN
can learn and represent deterministic finite-state automata. We ran
experiments with different sets of population sizes in the genetic
algorithm; we also ran experiments to find out which weight
initializations were best for training the hybrid architecture. The
results show that the hybrid architecture of recurrent neural networks
inspired by hidden Markov models can train and represent dynamical
systems. The best training and generalization performance is
achieved when the hybrid architecture is initialized with random real
weight values of range -15 to 15.
Abstract: recurrent neural network (RNN) is an efficient tool for
modeling production control process as well as modeling services. In
this paper one RNN was combined with regression model and were
employed in order to be checked whether the obtained data by the
model in comparison with actual data, are valid for variable process
control chart. Therefore, one maintenance process in workshop of
Esfahan Oil Refining Co. (EORC) was taken for illustration of
models. First, the regression was made for predicting the response
time of process based upon determined factors, and then the error
between actual and predicted response time as output and also the
same factors as input were used in RNN. Finally, according to
predicted data from combined model, it is scrutinized for test values
in statistical process control whether forecasting efficiency is
acceptable. Meanwhile, in training process of RNN, design of
experiments was set so as to optimize the RNN.
Abstract: Cosmic showers, from their places of origin in space,
after entering earth generate secondary particles called Extensive Air
Shower (EAS). Detection and analysis of EAS and similar High
Energy Particle Showers involve a plethora of experimental setups
with certain constraints for which soft-computational tools like
Artificial Neural Network (ANN)s can be adopted. The optimality
of ANN classifiers can be enhanced further by the use of Multiple
Classifier System (MCS) and certain data - dimension reduction
techniques. This work describes the performance of certain data
dimension reduction techniques like Principal Component Analysis
(PCA), Independent Component Analysis (ICA) and Self Organizing
Map (SOM) approximators for application with an MCS formed
using Multi Layer Perceptron (MLP), Recurrent Neural Network
(RNN) and Probabilistic Neural Network (PNN). The data inputs are
obtained from an array of detectors placed in a circular arrangement
resembling a practical detector grid which have a higher dimension
and greater correlation among themselves. The PCA, ICA and SOM
blocks reduce the correlation and generate a form suitable for real
time practical applications for prediction of primary energy and
location of EAS from density values captured using detectors in a
circular grid.
Abstract: Gene, principal unit of inheritance, is an ordered
sequence of nucleotides. The genes of eukaryotic organisms include
alternating segments of exons and introns. The region of
Deoxyribonucleic acid (DNA) within a gene containing instructions
for coding a protein is called exon. On the other hand, non-coding
regions called introns are another part of DNA that regulates gene
expression by removing from the messenger Ribonucleic acid (RNA)
in a splicing process. This paper proposes to determine splice
junctions that are exon-intron boundaries by analyzing DNA
sequences. A splice junction can be either exon-intron (EI) or intron
exon (IE). Because of the popularity and compatibility of the
artificial neural network (ANN) in genetic fields; various ANN
models are applied in this research. Multi-layer Perceptron (MLP),
Radial Basis Function (RBF) and Generalized Regression Neural
Networks (GRNN) are used to analyze and detect the splice junctions
of gene sequences. 10-fold cross validation is used to demonstrate
the accuracy of networks. The real performances of these networks
are found by applying Receiver Operating Characteristic (ROC)
analysis.
Abstract: Cosmic showers, during the transit through space, produce
sub - products as a result of interactions with the intergalactic
or interstellar medium which after entering earth generate secondary
particles called Extensive Air Shower (EAS). Detection and analysis
of High Energy Particle Showers involve a plethora of theoretical and
experimental works with a host of constraints resulting in inaccuracies
in measurements. Therefore, there exist a necessity to develop a
readily available system based on soft-computational approaches
which can be used for EAS analysis. This is due to the fact that soft
computational tools such as Artificial Neural Network (ANN)s can be
trained as classifiers to adapt and learn the surrounding variations. But
single classifiers fail to reach optimality of decision making in many
situations for which Multiple Classifier System (MCS) are preferred
to enhance the ability of the system to make decisions adjusting
to finer variations. This work describes the formation of an MCS
using Multi Layer Perceptron (MLP), Recurrent Neural Network
(RNN) and Probabilistic Neural Network (PNN) with data inputs
from correlation mapping Self Organizing Map (SOM) blocks and
the output optimized by another SOM. The results show that the setup
can be adopted for real time practical applications for prediction
of primary energy and location of EAS from density values captured
using detectors in a circular grid.
Abstract: In this paper, a class of recurrent neural networks (RNNs) with variable delays are studied on almost periodic time scales, some sufficient conditions are established for the existence and global exponential stability of the almost periodic solution. These results have important leading significance in designs and applications of RNNs. Finally, two examples and numerical simulations are presented to illustrate the feasibility and effectiveness of the results.