A Simple Adaptive Algorithm for Norm-Constrained Optimization

In this paper we propose a simple adaptive algorithm iteratively solving the unit-norm constrained optimization problem. Instead of conventional parameter norm based normalization, the proposed algorithm incorporates scalar normalization which is computationally much simpler. The analysis of stationary point is presented to show that the proposed algorithm indeed solves the constrained optimization problem. The simulation results illustrate that the proposed algorithm performs as good as conventional ones while being computationally simpler.

Authors:

• Hyun-Chool Shin

Keywords:

• constrained optimization
• unit-norm
• LMS
• principle component analysis.

References:

[1] B. Widrow and S. D. Sterns, Adaptive Signal Processing, Englewood
Cliffs, NJ: Prentice Hall, 1985.
[2] A. Benveniste, M. Metivier and P. Priouret, Adapive Algorithms and
Stochstic Approximation, New York: Springer-Verlag, 1990.
[3] S. Haykin, Adaptive Filter Theory, Englewood Cliffs, NJ: Prentice Hall,
2002.
[4] A. H. Sayed, Fundamentals of Adaptive Filtering, Englewood Cliffs,
NJ: Prentice Hall, 2003.
[5] S. C. Douglas, S. Amari and S. Y. Kung, "On gradient adaptation with
uni-norm constraints," IEEE Trans. Signal Processing, vol. 48, no. 6,
pp. 1843-1847, June 2000.
[6] S. G. Sankaran and A. A. Louis Beex, "Hyperspherical parametrization
for unit-norm based adaptive IIR filtering," IEEE Signal Processing
Letters, vol. 6, no. 12, pp. 318-320, Dec. 1999.
[7] R. L'opez-Valcare, "An algorithm for unit-norm equation error system
identification based on the method of multipliers," IEEE Tran. on Signal
Processing Letters, vol. 6, no. 12, pp. 3080-3085, Dec. 2003.
[8] P. Comon, "Independent component analysis: A new concept?," Signal
Processing, vol. 36, no. 3, pp. 287-314, Apr. 1994.
[9] A. Hyvarinen and E. Oja, "Independent component analysis by general
nonlinear Hebbian-like learning rules," Signal Processing, vol. 64, no. 3,
pp. 301-313, Feb. 1998.