Solving Partially Monotone Problems with Neural Networks

In many applications, it is a priori known that the target function should satisfy certain constraints imposed by, for example, economic theory or a human-decision maker. Here we consider partially monotone problems, where the target variable depends monotonically on some of the predictor variables but not all. We propose an approach to build partially monotone models based on the convolution of monotone neural networks and kernel functions. The results from simulations and a real case study on house pricing show that our approach has significantly better performance than partially monotone linear models. Furthermore, the incorporation of partial monotonicity constraints not only leads to models that are in accordance with the decision maker's expertise, but also reduces considerably the model variance in comparison to standard neural networks with weight decay.

Authors:

• Marina Velikova
• Hennie Daniels
• Ad Feelders

Keywords:

• Mixture models
• monotone neural networks
• partially monotone models
• partially monotone problems.

References:

[1] A. Ben-David, "Monotonicity Maintenance in Information-Theoretic
Machine Learning Algorithms", Machine Learning, 19, (1995), pp. 29-
43
[2] C. Cybenko, "Approximation by Superpositions of a Sigmoidal
Function", Mathematics of Control, Signals, and Systems, 2, (1989), pp.
303-314
[3] H.A.M. Daniels, and B. Kamp, "Application of MLP Networks to Bond
Rating and House Pricing", Neural Computation and Applications, 8,
(1999) pp. 226-234
[4] O. Harrison, and D. Rubinfeld, "Hedonic Prices and The Demand for
Clean Air", Journal of Environmental Economics and Management, 53,
(1978), pp.81-102
[5] H. Kay, and L.H. Ungar,, "Estimating Monotonic Functions and Their
Bounds", AIChE Journal, 46, (2000), pp.2426-2434
[6] H. Mukarjee, and S. Stern, "Feasible Nonparametric Estimation of
Multiargument Monotone Functions", Journal of the American
Statistical Association, 89, (1994), pp. 77-80
[7] E.A. Nadaraya, "On Estimating Regression", Theory Prob. Applic., 10,
(1964), pp.186-90
[8] R. Potharst, and A. Feelders, "Classification trees for problems with
monotonicity constraints", SIGKDD Explorations Newsletter, 4, (2002),
Issue 1
[9] P.J. Rousseeuw, "Silhouettes: a graphical aid to the interpretation and
validation of cluster analysis", Journal of Computational and Applied
Mathematics, 20, (1987), pp.53-65
[10] M. Sarfraz, M. Al-Mulhem, and F. Ashraf, "Preserving Monotonic
Shape of The Data by Using Piecewise Rational Cubic Functions",
Computers and Graphics, 21, (1997), pp.5-14
[11] J. Sill, "Monotonic Networks", Advances in Neural Information
Processing Systems, 10, (1998), pp.661-667
[12] S. Wang, "A Neural Network Method of Density Estimation for
Univariate Unimodal Data", Neural Computation & Applications, 2,
(1994), pp.160-167
[13] G. S. Watson, "Smooth Regression Analysis", Sankhya, Ser. A, 26,
(1964), pp. 359-372
[14] Wu, C.F.J., and M. Hamada, Experiments: Planning, Analysis, and
Parameter Design Optimization, Wiley Series in Probability and
Statistics, John Wiley & Sons, Inc. New York, (2000).