Improving the Utilization of Telfairia occidentalis Leaf Meal with Cellulase-Glucanase-Xylanase Combination and Selected Probiotic in Broiler Diets

Telfairia occidentalis is a leafy vegetable commonly grown in the tropics for nutritional benefits. The use of enzymes and probiotics is becoming prominent due to the ban on antibiotics as growth promoters in many parts of the world. It is conceived that with enzymes and probiotics additives, fibrous leafy vegetables can be incorporated into poultry feeds as protein source. However, certain antinutrients were also found in the leaves of Telfairia occidentalis. Four broiler starter and finisher diets were formulated for the two phases of the broiler experiments. A mixture of fiber degrading enzymes, Roxazyme G2 (combination of cellulase, glucanase and xylanase) and probiotics (Turbotox), a growth promoter, were used in broiler diets at 1:1. The Roxazyme G2/Turbotox mixtures were used in diets containing four varying levels of Telfairia occidentalis leaf meal (TOLM) at 0, 10, 20 and 30%. Diets 1 were standard broiler diets without TOLM and Roxazyme G2 and Turbotox additives. Diets 2, 3 and 4 had enzymes and probiotics additives. Certain mineral elements such as Ca, P, K, Na, Mg, Fe, Mn, Cu and Zn were found in notable quantities viz. 2.6 g/100 g, 1.2 g/100 g, 6.2 g/100 g, 5.1 g/100 g, 4.7 g/100 g, 5875 ppm, 182 ppm, 136 ppm and 1036 ppm, respectively. Phytin, phytin-P, oxalate, tannin and HCN were also found in ample quantities viz. 189.2 mg/100 g, 120.1 mg/100 g, 80.7 mg/100 g, 43.1 mg/100 g and 61.2 mg/100 g, respectively. The average weight gain was highest at 46.3 g/bird/day for birds on 10% TOLM diet but similar (P > 0.05) to 46.2 g/bird/day for birds on 20% TOLM. The feed conversion ratio (FCR) of 2.27 was the lowest and optimum for birds on 10% TOLM although similar (P > 0.05) to 2.29 obtained for birds on 20% TOLM. FCR of 2.61 was the highest at 2.61 for birds on 30% TOLM diet. The lowest FCR of 2.27 was obtained for birds on 10% TOLM diet although similar (P > 0.05) to 2.29 for birds on 20% TOLM diet. Most carcass characteristics and organ weights were similar (P > 0.05) for the experimental birds on the different diets except for kidney, gizzard and intestinal length. The values for kidney, gizzard and intestinal length were significantly higher (P < 0.05) for birds on the TOLM diets. The nitrogen retention had the highest value of 72.37 ± 0.10% for birds on 10% TOLM diet although similar (P > 0.05) to 71.54 ± 1.89 obtained for birds on the control diet without TOLM and enzymes/probiotics mixture. There was evidence of a better utilization of TOLM as a plant protein source. The carcass characteristics and organ weights all showed evidence of uniform tissue buildup and muscles development particularly in diets containing 10% of TOLM level. There was also better nitrogen utilization in birds on the 10% TOLM diet. Considering the cheap cost of TOLM, it is envisaged that its introduction into poultry feeds as a plant protein source will ultimately reduce the cost of poultry feeds.

Guidelines for Developing, Supervising, Assessing and Evaluating Capstone Design Project of BSc in Electrical and Electronic Engineering Program

Inclusion of any design project in an undergraduate electrical and electronic engineering curriculum and producing creative ideas in the final year capstone design projects have received numerous comments at the Board of Accreditation for Engineering and Technical Education (BAETE) several times by the mentors and visiting program evaluator team members at different public and private universities in Bangladesh. To eradicate this deficiency which is needed for getting the program accreditation, a thorough change was required in the Department of Electrical and Electronic Engineering (EEE) for its BSc in EEE program at Southeast University, Dhaka, Bangladesh. We suggested making changes in the course curriculum titles and contents, emphasizing to include capstone design projects, question setting, examining students through other standard methods, selecting and retaining Outcome-Based Education (OBE)-oriented engineering faculty members, improving laboratories through purchasing new equipment and software as well as developing new experiments for each laboratory courses, and engaging the students to practical designs in various courses and final year projects. This paper reports on capstone design project course objectives, course outcomes, mapping with the program outcomes, cognitive domain of learning, assessment schemes, guidelines, suggestions and recommendations for supervision processes, assessment strategy, and rubric setting, etc. It is expected that this will substantially improve the capstone design projects offering, supervision, and assessment in the undergraduate EEE program to fulfill the arduous requirements of BAETE accreditation based on OBE.

Verification and Proposal of Information Processing Model Using EEG-Based Brain Activity Monitoring

Human beings perform a task by perceiving information from outside, recognizing them, and responding them. There have been various attempts to analyze and understand internal processes behind the reaction to a given stimulus by conducting psychological experiments and analysis from multiple perspectives. Among these, we focused on Model Human Processor (MHP). However, it was built based on psychological experiments and thus the relation with brain activity was unclear so far. To verify the validity of the MHP and propose our model from a viewpoint of neuroscience, EEG (Electroencephalography) measurements are performed during experiments in this study. More specifically, first, experiments were conducted where Latin alphabet characters were used as visual stimuli. In addition to response time, ERPs (event-related potentials) such as N100 and P300 were measured by using EEG. By comparing cycle time predicted by the MHP and latency of ERPs, it was found that N100, related to perception of stimuli, appeared at the end of the perceptual processor. Furthermore, by conducting an additional experiment, it was revealed that P300, related to decision making, appeared during the response decision process, not at the end. Second, by experiments using Japanese Hiragana characters, i.e. Japan's own phonetic symbols, those findings were confirmed. Finally, Japanese Kanji characters were used as more complicated visual stimuli. A Kanji character usually has several readings and several meanings. Despite the difference, a reading-related task and a meaning-related task exhibited similar results, meaning that they involved similar information processing processes of the brain. Based on those results, our model was proposed which reflects response time and ERP latency. It consists of three processors: the perception processor from an input of a stimulus to appearance of N100, the cognitive processor from N100 to P300, and the decision-action processor from P300 to response. Using our model, an application system which reflects brain activity can be established.

Electronics Thermal Management Driven Design of an IP65-Rated Motor Inverter

Thermal management of electronic components packaged inside an IP65 rated enclosure is of prime importance in industrial applications. Electrical enclosure protects the multiple board configurations such as inverter, power, controller board components, busbars, and various power dissipating components from harsh environments. Industrial environments often experience relatively warm ambient conditions, and the electronic components housed in the enclosure dissipate heat, due to which the enclosures and the components require thermal management as well as reduction of internal ambient temperatures. Design of Experiments based thermal simulation approach with MOSFET arrangement, Heat sink design, Enclosure Volume, Copper and Aluminum Spreader, Power density, and Printed Circuit Board (PCB) type were considered to optimize air temperature inside the IP65 enclosure to ensure conducive operating temperature for controller board and electronic components through the different modes of heat transfer viz. conduction, natural convection and radiation using Ansys ICEPAK. MOSFET’s with the parallel arrangement, IP65 enclosure molded heat sink with rectangular fins on both enclosures, specific enclosure volume to satisfy the power density, Copper spreader to conduct heat to the enclosure, optimized power density value and selecting Aluminum clad PCB which improves the heat transfer were the contributors towards achieving a conducive operating temperature inside the IP-65 rated Motor Inverter enclosure. A reduction of 52 ℃ was achieved in internal ambient temperature inside the IP65 enclosure between baseline and final design parameters, which met the operative temperature requirements of the electronic components inside the IP-65 rated Motor Inverter.

Greenhouse Gasses’ Effect on Atmospheric Temperature Increase and the Observable Effects on Ecosystems

Radiative forces of greenhouse gases (GHG) increase the temperature of the Earth's surface, more on land, and less in oceans, due to their thermal capacities. Given this inertia, the temperature increase is delayed over time. Air temperature, however, is not delayed as air thermal capacity is much lower. In this study, through analysis and synthesis of multidisciplinary science and data, an estimate of atmospheric temperature increase is made. Then, this estimate is used to shed light on current observations of ice and snow loss, desertification and forest fires, and increased extreme air disturbances. The reason for this inquiry is due to the author’s skepticism that current changes cannot be explained by a "~1 oC" global average surface temperature rise within the last 50-60 years. The only other plausible cause to explore for understanding is that of atmospheric temperature rise. The study utilizes an analysis of air temperature rise from three different scientific disciplines: thermodynamics, climate science experiments, and climactic historical studies. The results coming from these diverse disciplines are nearly the same, within ± 1.6%. The direct radiative force of GHGs with a high level of scientific understanding is near 4.7 W/m2 on average over the Earth’s entire surface in 2018, as compared to one in pre-Industrial time in the mid-1700s. The additional radiative force of fast feedbacks coming from various forms of water gives approximately an additional ~15 W/m2. In 2018, these radiative forces heated the atmosphere by approximately 5.1 oC, which will create a thermal equilibrium average ground surface temperature increase of 4.6 oC to 4.8 oC by the end of this century. After 2018, the temperature will continue to rise without any additional increases in the concentration of the GHGs, primarily of carbon dioxide and methane. These findings of the radiative force of GHGs in 2018 were applied to estimates of effects on major Earth ecosystems. This additional force of nearly 20 W/m2 causes an increase in ice melting by an additional rate of over 90 cm/year, green leaves temperature increase by nearly 5 oC, and a work energy increase of air by approximately 40 Joules/mole. This explains the observed high rates of ice melting at all altitudes and latitudes, the spread of deserts and increases in forest fires, as well as increased energy of tornadoes, typhoons, hurricanes, and extreme weather, much more plausibly than the 1.5 oC increase in average global surface temperature in the same time interval. Planned mitigation and adaptation measures might prove to be much more effective when directed toward the reduction of existing GHGs in the atmosphere.

Personal Information Classification Based on Deep Learning in Automatic Form Filling System

Recently, the rapid development of deep learning makes artificial intelligence (AI) penetrate into many fields, replacing manual work there. In particular, AI systems also become a research focus in the field of automatic office. To meet real needs in automatic officiating, in this paper we develop an automatic form filling system. Specifically, it uses two classical neural network models and several word embedding models to classify various relevant information elicited from the Internet. When training the neural network models, we use less noisy and balanced data for training. We conduct a series of experiments to test my systems and the results show that our system can achieve better classification results.

Simulation of Low Cycle Fatigue Behaviour of Nickel-Based Alloy at Elevated Temperatures

Thermal power machines are subjected to cyclic loading conditions under elevated temperatures. At these extreme conditions, the durability of the components has a significant influence. The material mechanical behaviour has to be known in detail for a failsafe construction. For this study a nickel-based alloy is considered, the deformation and fatigue behaviour of the material is analysed under cyclic loading. A viscoplastic model is used for calculating the deformation behaviour as well as to simulate the rate-dependent and cyclic plasticity effects. Finally, the cyclic deformation results of the finite element simulations are compared with low cycle fatigue (LCF) experiments.

Churn Prediction for Telecommunication Industry Using Artificial Neural Networks

Telecommunication service providers demand accurate and precise prediction of customer churn probabilities to increase the effectiveness of their customer relation services. The large amount of customer data owned by the service providers is suitable for analysis by machine learning methods. In this study, expenditure data of customers are analyzed by using an artificial neural network (ANN). The ANN model is applied to the data of customers with different billing duration. The proposed model successfully predicts the churn probabilities at 83% accuracy for only three months expenditure data and the prediction accuracy increases up to 89% when the nine month data is used. The experiments also show that the accuracy of ANN model increases on an extended feature set with information of the changes on the bill amounts.

Pricing European Options under Jump Diffusion Models with Fast L-stable Padé Scheme

The goal of option pricing theory is to help the investors to manage their money, enhance returns and control their financial future by theoretically valuing their options. Modeling option pricing by Black-School models with jumps guarantees to consider the market movement. However, only numerical methods can solve this model. Furthermore, not all the numerical methods are efficient to solve these models because they have nonsmoothing payoffs or discontinuous derivatives at the exercise price. In this paper, the exponential time differencing (ETD) method is applied for solving partial integrodifferential equations arising in pricing European options under Merton’s and Kou’s jump-diffusion models. Fast Fourier Transform (FFT) algorithm is used as a matrix-vector multiplication solver, which reduces the complexity from O(M2) into O(M logM). A partial fraction form of Pad`e schemes is used to overcome the complexity of inverting polynomial of matrices. These two tools guarantee to get efficient and accurate numerical solutions. We construct a parallel and easy to implement a version of the numerical scheme. Numerical experiments are given to show how fast and accurate is our scheme.

Improved Thermal Comfort and Sensation with Occupant Control of Ceiling Personalized Ventilation System: A Lab Study

This study aims at determining the extent to which occupant control of microenvironment influences, improves thermal sensation and comfort, and saves energy in spaces equipped with ceiling personalized ventilation (CPV) system assisted by chair fans (CF) and desk fans (DF) in 2 experiments in a climatic chamber equipped with two-station CPV systems, one that allows control of fan flow rate and the other is set to the fan speed of the selected participant in control. Each experiment included two participants each entering the cooled space from transitional environment at a conventional mixed ventilation (MV) at 24 °C. For CPV diffuser, fresh air was delivered at a rate of 20 Cubic feet per minute (CFM) and a temperature of 16 °C while the recirculated air was delivered at the same temperature but at a flow rate 150 CFM. The macroclimate air of the space was at 26 °C. The full speed flow rates for both the CFs and DFs were at 5 CFM and 20 CFM, respectively. Occupant 1 was allowed to operate the CFs or the DFs at (1/3 of the full speed, 2/3 of the full speed, and the full speed) while occupant 2 had no control on the fan speed and their fan speed was selected by occupant 1. Furthermore, a parametric study was conducted to study the effect of increasing the fresh air flow rate on the occupants’ thermal comfort and whole body sensations. The results showed that most occupants in the CPV+CFs, who did not control the CF flow rate, felt comfortable 6 minutes. The participants, who controlled the CF speeds, felt comfortable in around 24 minutes because they were preoccupied with the CFs. For the DF speed control experiments, most participants who did not control the DFs felt comfortable within the first 8 minutes. Similarly to the CPV+CFs, the participants who controlled the DF flow rates felt comfortable at around 26 minutes. When the CPV system was either supported by CFs or DFs, 93% of participants in both cases reached thermal comfort. Participants in the parametric study felt more comfortable when the fresh air flow rate was low, and felt cold when as the flow rate increased.

The Effect of Zeolite on Sandy-Silt Soil Mechanical Properties

It is well known that cemented sand is one of the best approaches for soil stabilization. In some cases, a blend of sand, cement and other pozzolan materials such as zeolite, nano-particles and fiber can be widely (commercially) available and be effectively used in soil stabilization, especially in road construction. In this research, we investigate the effects of CaO which is based on the geotechnical characteristics of zeolite composition with sandy silt soil. Zeolites have low amount of CaO in their structures, that is, varying from 3% to 10%, and by removing the cement paste, we want to investigate the effect of zeolite pozzolan without any activator on soil samples strength. In this research, experiments are concentrated on various weight percentages of zeolite in the soil to examine the effect of the zeolite on drainage shear strength and California Bearing Ratio (CBR) both with and without curing. The study also investigates their liquid limit and plastic limit behavior and makes a comparative result by using Feng's and Wroth-Wood's methods in fall cone (cone penetrometer) device; in the final the SEM images have been presented. The results show that by increasing the percentage of zeolite in without-curing samples, the fine zeolite particles increase some soil's strength, but in the curing-state we can see a relatively higher strength toward without-curing state, since the zeolites have no plastic behavior, the pozzolanic property of zeolites plays a much higher role than cementing properties. Indeed, it is better to combine zeolite particle with activator material such as cement or lime to gain better results.

Inferential Reasoning for Heterogeneous Multi-Agent Mission

We describe issues bedeviling the coordination of heterogeneous (different sensors carrying agents) multi-agent missions such as belief conflict, situation reasoning, etc. We applied Bayesian and agents' presumptions inferential reasoning to solve the outlined issues with the heterogeneous multi-agent belief variation and situational-base reasoning. Bayesian Belief Network (BBN) was used in modeling the agents' belief conflict due to sensor variations. Simulation experiments were designed, and cases from agents’ missions were used in training the BBN using gradient descent and expectation-maximization algorithms. The output network is a well-trained BBN for making inferences for both agents and human experts. We claim that the Bayesian learning algorithm prediction capacity improves by the number of training data and argue that it enhances multi-agents robustness and solve agents’ sensor conflicts.

Deployment of a Biocompatible International Space Station into Geostationary Orbit

This study explores the possibility of a space station that will occupy a geostationary equatorial orbit (GEO) and create artificial gravity using centripetal acceleration. The concept of the station is to create a habitable, safe environment that can increase the possibility of space tourism by reducing the wide variation of hazards associated with space exploration. The ability to control the intensity of artificial gravity through Hall-effect thrusters will allow experiments to be carried out at different levels of artificial gravity. A feasible prototype model was built to convey the concept and to enable cost estimation. The SpaceX Falcon Heavy rocket with a 26,700 kg payload to GEO was selected to take the 675 tonne spacecraft into orbit; space station construction will require up to 30 launches, this would be reduced to 5 launches when the SpaceX BFR becomes available. The estimated total cost of implementing the Sussex Biocompatible International Space Station (BISS) is approximately $47.039 billion, which is very attractive when compared to the cost of the International Space Station, which cost $150 billion.

Optical Signal-To-Noise Ratio Monitoring Based on Delay Tap Sampling Using Artificial Neural Network

With the development of optical communication, optical performance monitoring (OPM) has received more and more attentions. Since optical signal-to-noise ratio (OSNR) is directly related to bit error rate (BER), it is one of the important parameters in optical networks. Recently, artificial neural network (ANN) has been greatly developed. ANN has strong learning and generalization ability. In this paper, a method of OSNR monitoring based on delay-tap sampling (DTS) and ANN has been proposed. DTS technique is used to extract the eigenvalues of the signal. Then, the eigenvalues are input into the ANN to realize the OSNR monitoring. The experiments of 10 Gb/s non-return-to-zero (NRZ) on–off keying (OOK), 20 Gb/s pulse amplitude modulation (PAM4) and 20 Gb/s return-to-zero (RZ) differential phase-shift keying (DPSK) systems are demonstrated for the OSNR monitoring based on the proposed method. The experimental results show that the range of OSNR monitoring is from 15 to 30 dB and the root-mean-square errors (RMSEs) for 10 Gb/s NRZ-OOK, 20 Gb/s PAM4 and 20 Gb/s RZ-DPSK systems are 0.36 dB, 0.45 dB and 0.48 dB respectively. The impact of chromatic dispersion (CD) on the accuracy of OSNR monitoring is also investigated in the three experimental systems mentioned above.

Competitors’ Influence Analysis of a Retailer by Using Customer Value and Huff’s Gravity Model

Customer relationship analysis is vital for retail stores, especially for supermarkets. The point of sale (POS) systems make it possible to record the daily purchasing behaviors of customers as an identification point of sale (ID-POS) database, which can be used to analyze customer behaviors of a supermarket. The customer value is an indicator based on ID-POS database for detecting the customer loyalty of a store. In general, there are many supermarkets in a city, and other nearby competitor supermarkets significantly affect the customer value of customers of a supermarket. However, it is impossible to get detailed ID-POS databases of competitor supermarkets. This study firstly focused on the customer value and distance between a customer's home and supermarkets in a city, and then constructed the models based on logistic regression analysis to analyze correlations between distance and purchasing behaviors only from a POS database of a supermarket chain. During the modeling process, there are three primary problems existed, including the incomparable problem of customer values, the multicollinearity problem among customer value and distance data, and the number of valid partial regression coefficients. The improved customer value, Huff’s gravity model, and inverse attractiveness frequency are considered to solve these problems. This paper presents three types of models based on these three methods for loyal customer classification and competitors’ influence analysis. In numerical experiments, all types of models are useful for loyal customer classification. The type of model, including all three methods, is the most superior one for evaluating the influence of the other nearby supermarkets on customers' purchasing of a supermarket chain from the viewpoint of valid partial regression coefficients and accuracy.

Effect of Porous Multi-Layer Envelope System on Effective Wind Pressure of Building Ventilation

Building ventilation performance is an important indicator of indoor comfort. However, in addition to the geometry of the building or the proportion of the opening, the ventilation performance is also very much related to the actual wind pressure of the building. There are more and more contemporary building designs built with multi-layer exterior envelope. Due to ventilation and view observatory requirement, the porous outer layer of the building is commonly adopted and has a significant wind damping effect, causing the phenomenon of actual wind pressure loss. However, the relationship between the wind damping effect and the actual wind pressure is not linear. This effect can make the indoor ventilation of the building rationalized to reasonable range under the condition of high wind pressure, and also maintain a good amount of ventilation performance under the condition of low wind pressure. In this study, wind tunnel experiments were carried out to simulate the different wind pressures flow through the porous outer layer, and observe the actual wind pressure strength engage with the window layer to find the decreasing relationship between the damping effect of the porous shell and the wind pressure. Experiment specimen scale was designed to be 1:50 for testing real-world building conditions; the study found that the porous enclosure has protective shielding without affecting low-pressure ventilation. Current study observed the porous skin may damp more wind energy to ease the wind pressure under high-speed wind. Differential wind speed may drop the pressure into similar pressure level by using porous skin. The actual mechanism and value of this phenomenon will need further study in the future.

An Optimal Control Method for Reconstruction of Topography in Dam-Break Flows

Modeling dam-break flows over non-flat beds requires an accurate representation of the topography which is the main source of uncertainty in the model. Therefore, developing robust and accurate techniques for reconstructing topography in this class of problems would reduce the uncertainty in the flow system. In many hydraulic applications, experimental techniques have been widely used to measure the bed topography. In practice, experimental work in hydraulics may be very demanding in both time and cost. Meanwhile, computational hydraulics have served as an alternative for laboratory and field experiments. Unlike the forward problem, the inverse problem is used to identify the bed parameters from the given experimental data. In this case, the shallow water equations used for modeling the hydraulics need to be rearranged in a way that the model parameters can be evaluated from measured data. However, this approach is not always possible and it suffers from stability restrictions. In the present work, we propose an adaptive optimal control technique to numerically identify the underlying bed topography from a given set of free-surface observation data. In this approach, a minimization function is defined to iteratively determine the model parameters. The proposed technique can be interpreted as a fractional-stage scheme. In the first stage, the forward problem is solved to determine the measurable parameters from known data. In the second stage, the adaptive control Ensemble Kalman Filter is implemented to combine the optimality of observation data in order to obtain the accurate estimation of the topography. The main features of this method are on one hand, the ability to solve for different complex geometries with no need for any rearrangements in the original model to rewrite it in an explicit form. On the other hand, its achievement of strong stability for simulations of flows in different regimes containing shocks or discontinuities over any geometry. Numerical results are presented for a dam-break flow problem over non-flat bed using different solvers for the shallow water equations. The robustness of the proposed method is investigated using different numbers of loops, sensitivity parameters, initial samples and location of observations. The obtained results demonstrate high reliability and accuracy of the proposed techniques.

Efficiency Enhancement of Photovoltaic Panels Using an Optimised Air Cooled Heat Sink

Solar panels that use photovoltaic (PV) cells are popular for converting solar radiation into electricity. One of the major problems impacting the performance of PV panels is the overheating caused by excessive solar radiation and high ambient temperatures, which degrades the efficiency of the PV panels remarkably. To overcome this issue, an aluminum heat sink was used to dissipate unwanted heat from PV cells. The dimensions of the heat sink were determined considering the optimal fin spacing that fulfils hot climatic conditions. In this study, the effects of cooling on the efficiency and power output of a PV panel were studied experimentally. Two PV modules were used: one without and one with a heat sink. The experiments ran for 11 hours from 6:00 a.m. to 5:30 p.m. where temperature readings in the rear and front of both PV modules were recorded at an interval of 15 minutes using sensors and an Arduino microprocessor. Results are recorded for both panels simultaneously for analysis, temperate comparison, and for power and efficiency calculations. A maximum increase in the solar to electrical conversion efficiency of 35% and almost 55% in the power output were achieved with the use of a heat sink, while temperatures at the front and back of the panel were reduced by 9% and 11%, respectively.

GRCNN: Graph Recognition Convolutional Neural Network for Synthesizing Programs from Flow Charts

Program synthesis is the task to automatically generate programs based on user specification. In this paper, we present a framework that synthesizes programs from flow charts that serve as accurate and intuitive specification. In order doing so, we propose a deep neural network called GRCNN that recognizes graph structure from its image. GRCNN is trained end-to-end, which can predict edge and node information of the flow chart simultaneously. Experiments show that the accuracy rate to synthesize a program is 66.4%, and the accuracy rates to recognize edge and node are 94.1% and 67.9%, respectively. On average, it takes about 60 milliseconds to synthesize a program.

Convergence Analysis of Training Two-Hidden-Layer Partially Over-Parameterized ReLU Networks via Gradient Descent

Over-parameterized neural networks have attracted a great deal of attention in recent deep learning theory research, as they challenge the classic perspective of over-fitting when the model has excessive parameters and have gained empirical success in various settings. While a number of theoretical works have been presented to demystify properties of such models, the convergence properties of such models are still far from being thoroughly understood. In this work, we study the convergence properties of training two-hidden-layer partially over-parameterized fully connected networks with the Rectified Linear Unit activation via gradient descent. To our knowledge, this is the first theoretical work to understand convergence properties of deep over-parameterized networks without the equally-wide-hidden-layer assumption and other unrealistic assumptions. We provide a probabilistic lower bound of the widths of hidden layers and proved linear convergence rate of gradient descent. We also conducted experiments on synthetic and real-world datasets to validate our theory.