Minimal Residual Method for Adaptive Filtering with Finite Termination

We present a discussion of three adaptive filtering algorithms well known for their one-step termination property, in terms of their relationship with the minimal residual method. These algorithms are the normalized least mean square (NLMS), Affine Projection algorithm (APA) and the recursive least squares algorithm (RLS). The NLMS is shown to be a result of the orthogonality condition imposed on the instantaneous approximation of the Wiener equation, while APA and RLS algorithm result from orthogonality condition in multi-dimensional minimal residual formulation. Further analysis of the minimal residual formulation for the RLS leads to a triangular system which also possesses the one-step termination property (in exact arithmetic)

Authors:

• Noor Atinah Ahmad
• Shazia Javed

Keywords:

• Adaptive filtering
• minimal residual method
• projection method.

References:

[1] B. Farhang-Boroujeny, Adaptive filters: theory and applications, John
Wiley & Sons, Inc.,1998.
[2] G. K. Boray and M. D. Srinath, Conjugate gradient techniques for
adaptive filtering, IEEE Trans. Circuits Syst., 1992.
[3] P. S. Chang and A. N. Wilson, Jr., Analysis of conjugate gradient method
for adaptive filtering, IEEE Trans. Signal Processing, 2000.
[4] M. Q. Chen, A direction set based algorithm for least squares problems
in adaptive signal processing, Linear Algebra and its Application, 1998.
[5] G. F. Xu, T. Bose and J. Schroeder, The Euclidean direction search
algorithm for adaptive filtering, Proceedings of the 1999 IEEE Int.
Symp. on Circuits and Systems, 1999.
[6] N. A. Ahmad, Accelerated euclidean direction search algorithm and
related relaxation schemes for solving adaptive filtering problem, Proceedings
of IEEE Int. Conf. on Information, Communication and Signal
Processing (ICICS 2007), 2007.
[7] N. A. Ahmad, A globally convergent stochastic pairwise conjugate
gradient based adaptive filtering algorithm, IEEE Signal Process.
Letters, 2008.
[8] P. S. R. Diniz, M. L. R. de Campos and A. Antoniou, Analysis of LMSNewton
Adaptive Filtering Algorithms with Variable Convergence Factor,
IEEE Trans. Signal Processing, 1995.
[9] C. W. Lee, Y. K. Lee and S. W. Kim, An Affine Projection Algorithm
with a Data-Selective Method of using the Condition Number, Signal
Processing 88, 2008.
[10] Y. Saad, Iterative Methods for Sparse Linear Systems, Englewood
Cliffs, NJ: Prentice-Hall, 1991.
[11] S. Haykin, Adaptive Filter Theory, 2nd edition, Prentice Hall,1991.
[12] P. S. R. Diniz,Adaptive Filtering: Algorithms and Practical Implementation,
3rd Edition, Springer, 2008.