Performance Analysis of Software Reliability Models using Matrix Method
This paper presents a computational methodology
based on matrix operations for a computer based solution to the
problem of performance analysis of software reliability models
(SRMs). A set of seven comparison criteria have been formulated to
rank various non-homogenous Poisson process software reliability
models proposed during the past 30 years to estimate software
reliability measures such as the number of remaining faults, software
failure rate, and software reliability. Selection of optimal SRM for
use in a particular case has been an area of interest for researchers in
the field of software reliability. Tools and techniques for software
reliability model selection found in the literature cannot be used with
high level of confidence as they use a limited number of model
selection criteria. A real data set of middle size software project from
published papers has been used for demonstration of matrix method.
The result of this study will be a ranking of SRMs based on the
Permanent value of the criteria matrix formed for each model based
on the comparison criteria. The software reliability model with
highest value of the Permanent is ranked at number – 1 and so on.
[1] H. Pham, and X. Zhang, "A software cost model with warranty and risk
costs" IEEE Trans. on Computers, 48 (1), 1999, pp. 71-75.
[2] M. Xie, Software Reliability Modelling, World Scientific Publishing Co.
Ltd., 1991.
[3] A. Goel, and K. Okumoto, "Time dependent error-detection rate model
for software reliability and other performance measures," IEEE Trans.
on Reliability, R-28(3), 1979, pp. 206-211.
[4] M. R. Lyu, Handbook of Software Reliability Eng., McGraw-Hill, 1996.
[5] J. D. Musa, and K. Okumoto, "A logarithmic Poisson execution time
model for software reliability measurement," Conf. Proc. 7th
International Conf. on Softw. Engineering, 1983, pp. 230-237.
[6] C. Wohlin, "Software testing and reliability for telecommunication
systems," Softw. Engineering-86", ed. D. Barnes and P. Brown, Peter
Peregrinus Ltd., Stevenage, United Kingdom, 1986, pp. 27-42.
[7] J. D. Musa, A. Iannino, and K. Okumoto, Software Reliability
Measurement, Predication, Aplication, McGraw Hill, 1987.
[8] C. Y. Huang, M. R. Lyu, and S. Y. Kuo, "A unified scheme of some
non-homogenous Poisson process models for software reliability
estimation," IEEE Trans. on Softw. Engineering, vol. 29, no. 3, March
2003, pp. 261-269.
[9] C. G. Bai,"Bayesian network based software reliability prediction with
an operational profile," J. Syst. Softw., vol. 77, no. 2, 2005, pp.103-112.
[10] L. Tian, and A. Noore, "On-line prediction of software reliability using
an evolutionary connectionist model," J. Syst. Softw., vol. 77, no. 2,
2005, pp. 173-180.
[11] W. L. Wang, D. Pan, and M. H. Chen, "Architecture-based software
reliability modeling," J. Syst. Softw., vol. 79, no. 1, 2006, pp. 132-146.
[12] X. Zhang, and H. Pham, "Software field failure rate prediction before
software deployment," J. Syst. Softw., Vol.79, no. 3, 2006, pp. 291-300.
[13] C. Y. Huang, "Performance analysis of software reliability growth
models with testing-effort and change-point," J. Syst. Softw., vol. 76, no.
2, 2005, pp. 181-194.
[14] C. Y. Huang, and C. T. Lin, "Software reliability analysis by considering
fault dependency and debugging time lag," IEEE Trans. Reliability, vol.
55, no. 3, 2006, pp. 436-450.
[15] Q. P. Hu, M. Xie, S. H. Ng, and G. Levitin, "Robust recurrent neural
network modeling for software fault detection and correction
prediction," Reliability Engineering and System Safety, vol. 92 no. 3,
2007, pp. 332-340.
[16] D. R. Jeske, and X. Zhang, "Some successful approaches to software
reliability modeling in industry," J. Syst. Softw., vol. 74, no. 1, 2005, pp.
85-99.
[17] G.H. Schick, and R.W. Wolverton, "An analysis of competing software
reliability models," IEEE Trans. on Softw. Engineering, March 1978, pp.
104-120.
[18] A.N. Sukert, "Empirical validation of three software errors predictions
models," IEEE Trans. on Reliability, August 1979, pp. 199-205.
[19] S. Brocklehurst, P.Y. Chan, B. Littlewood, and J. Snell, "Recalibrating
software reliability models," IEEE Trans. on Softw. Engineering, vol.
SE-16 no. 4, April 1990, pp. 458-470.
[20] A.L Goel, "Software reliability models: assumption, limitations, and
applicability," IEEE Trans. on Softw. Engineering, December 1985, pp.
1411-1423.
[21] Abdel-Ghaly, P.Y. Chan, and B. Littlewood, "Evaluation of competing
software reliability predictions," IEEE Trans. on Softw. Engineering,
vol. SE-12 no. 12, September 1986, pp. 950-967.
[22] T.M Khoshgoftaar, and T. Woodcock, "Software reliability model
selection: A case study," Proc. of the 2nd International Symposium on
Softw. Reliability Engineering. IEEE Computer Society Press, Austin,
TX: 1991, pp. 183-191.
[23] C. Y. Huang, and S. Y. Kuo, "Analysis of incorporating logistic testing
effort function into software reliability modeling," IEEE Trans. on
Reliability, vol. 51, no. 3, 2002, pp. 261-270.
[24] K. Pillai, and V. S. S. Nair, "A model for software development effort
and cost estimation," IEEE Trans. on Softw. Engineering, vol. 23, no. 8,
1997, pp. 485-497.
[25] S. Yamada, K. Tokuno, and S. Osaki, "Imperfect debugging models
with fault introduction rate for software reliability assessment,"
International J. Syst. Science, vol. 23, no. 12, 1992.
[26] H. Pham, "Software reliability and cost models: perspectives,
comparison and practice", European J. of Operational Research, vol.
149, 2003, p. 475- 489.
[27] H. Pham, L. Nordmann, and X. Zhang, "A general imperfect software
debugging model with s-shaped fault detection rate," IEEE Trans.
Reliability, vol. 48, June 1999, pp. 169-175.
[28] H. Pham and X. Zhang, "An NHPP software reliability models and its
comparison," International J. of Reliability, Quality and Safety
Engineering, vol. 14, no. 3, 1997, pp. 269-282.
[29] H. Pham, System software reliability, Springer London, 2006.
[30] X. Zhang, X. Teng and H. Pham, "Considering Fault Removal
Efficiency in Software Reliability Assessment", IEEE Trans. on
Systems, Man, & Cybernetics- Part A, vol. 33, no.1, 2003, pp. 114-120.
[31] S. Hwang and H. Pham, "Quasi-renewal time-delay fault-removal
consideration in software reliability modelling," IEEE Trans. on
systems, man and cybernetics-Part A:Systems and humans, vol. 39, no.
1, January 2009.
[32] K. C. Chiu, Y. S. Huang, and T. Z. Lee, "A study of software reliability
growth from the perspective of learning effects,"Reliability Engineering
and System Safety, 2008, pp. 1410-1421.
[33] M. Zhao, and M. Xie, "On the log-power NHPP software reliability
model,". Proceedings of the Third IEEE International Symposium on
Softw. Reliability Engineering, Research Triangle Park, North Carolina,
1992, pp. 14-22.
[34] M. R. Lyu, and A. Nikora, "Applying software reliability models more
effectively," IEEE Softw., 1992, 43-52.
[35] H. Pham and C. Deng, "Predictive-ratio risk criterion for selecting
software reliability models," Proc. Ninth International Conf. On
Reliability and Quality in Design, August 2003.
[36] P. L. Li, J. Herbsleb, and M. Shaw, "Forecasting field defect rates using
a combined time-based and metrics-based approach: a case study of
OpenBSD," Proceedings of the 16th IEEE International Symposium on
Softw. Reliability Engineering, Chicago, IL, 2005, pp. 193-202.
[37] R. Peng, Q.P. Hu,and S.H. Ng, ,"Incorporating Fault Dependency and
Debugging Delay in Software Reliability Analysis," Proceedings of the
IEEE ICMIT, 2008, pp.641-645.
[38] S. Dahiya et al.,"Power quality evaluation in deregulated power system
using matrix method," International J. Global Energy Issues, vol. 28,
no. 1, 2007.
[39] R. K. Garg, V. K. Gupta, and V. P. Agrawal,"Quality evaluation of
thermal power plants by graph theoretical methodology," International
J. of Power and Energy Systems, 27 (1), 2007, pp. 42-48.
[40] M. Marcus, and H. Minc," Permanents," American Mathematics, 72,
1965, pp. 571-91.
[1] H. Pham, and X. Zhang, "A software cost model with warranty and risk
costs" IEEE Trans. on Computers, 48 (1), 1999, pp. 71-75.
[2] M. Xie, Software Reliability Modelling, World Scientific Publishing Co.
Ltd., 1991.
[3] A. Goel, and K. Okumoto, "Time dependent error-detection rate model
for software reliability and other performance measures," IEEE Trans.
on Reliability, R-28(3), 1979, pp. 206-211.
[4] M. R. Lyu, Handbook of Software Reliability Eng., McGraw-Hill, 1996.
[5] J. D. Musa, and K. Okumoto, "A logarithmic Poisson execution time
model for software reliability measurement," Conf. Proc. 7th
International Conf. on Softw. Engineering, 1983, pp. 230-237.
[6] C. Wohlin, "Software testing and reliability for telecommunication
systems," Softw. Engineering-86", ed. D. Barnes and P. Brown, Peter
Peregrinus Ltd., Stevenage, United Kingdom, 1986, pp. 27-42.
[7] J. D. Musa, A. Iannino, and K. Okumoto, Software Reliability
Measurement, Predication, Aplication, McGraw Hill, 1987.
[8] C. Y. Huang, M. R. Lyu, and S. Y. Kuo, "A unified scheme of some
non-homogenous Poisson process models for software reliability
estimation," IEEE Trans. on Softw. Engineering, vol. 29, no. 3, March
2003, pp. 261-269.
[9] C. G. Bai,"Bayesian network based software reliability prediction with
an operational profile," J. Syst. Softw., vol. 77, no. 2, 2005, pp.103-112.
[10] L. Tian, and A. Noore, "On-line prediction of software reliability using
an evolutionary connectionist model," J. Syst. Softw., vol. 77, no. 2,
2005, pp. 173-180.
[11] W. L. Wang, D. Pan, and M. H. Chen, "Architecture-based software
reliability modeling," J. Syst. Softw., vol. 79, no. 1, 2006, pp. 132-146.
[12] X. Zhang, and H. Pham, "Software field failure rate prediction before
software deployment," J. Syst. Softw., Vol.79, no. 3, 2006, pp. 291-300.
[13] C. Y. Huang, "Performance analysis of software reliability growth
models with testing-effort and change-point," J. Syst. Softw., vol. 76, no.
2, 2005, pp. 181-194.
[14] C. Y. Huang, and C. T. Lin, "Software reliability analysis by considering
fault dependency and debugging time lag," IEEE Trans. Reliability, vol.
55, no. 3, 2006, pp. 436-450.
[15] Q. P. Hu, M. Xie, S. H. Ng, and G. Levitin, "Robust recurrent neural
network modeling for software fault detection and correction
prediction," Reliability Engineering and System Safety, vol. 92 no. 3,
2007, pp. 332-340.
[16] D. R. Jeske, and X. Zhang, "Some successful approaches to software
reliability modeling in industry," J. Syst. Softw., vol. 74, no. 1, 2005, pp.
85-99.
[17] G.H. Schick, and R.W. Wolverton, "An analysis of competing software
reliability models," IEEE Trans. on Softw. Engineering, March 1978, pp.
104-120.
[18] A.N. Sukert, "Empirical validation of three software errors predictions
models," IEEE Trans. on Reliability, August 1979, pp. 199-205.
[19] S. Brocklehurst, P.Y. Chan, B. Littlewood, and J. Snell, "Recalibrating
software reliability models," IEEE Trans. on Softw. Engineering, vol.
SE-16 no. 4, April 1990, pp. 458-470.
[20] A.L Goel, "Software reliability models: assumption, limitations, and
applicability," IEEE Trans. on Softw. Engineering, December 1985, pp.
1411-1423.
[21] Abdel-Ghaly, P.Y. Chan, and B. Littlewood, "Evaluation of competing
software reliability predictions," IEEE Trans. on Softw. Engineering,
vol. SE-12 no. 12, September 1986, pp. 950-967.
[22] T.M Khoshgoftaar, and T. Woodcock, "Software reliability model
selection: A case study," Proc. of the 2nd International Symposium on
Softw. Reliability Engineering. IEEE Computer Society Press, Austin,
TX: 1991, pp. 183-191.
[23] C. Y. Huang, and S. Y. Kuo, "Analysis of incorporating logistic testing
effort function into software reliability modeling," IEEE Trans. on
Reliability, vol. 51, no. 3, 2002, pp. 261-270.
[24] K. Pillai, and V. S. S. Nair, "A model for software development effort
and cost estimation," IEEE Trans. on Softw. Engineering, vol. 23, no. 8,
1997, pp. 485-497.
[25] S. Yamada, K. Tokuno, and S. Osaki, "Imperfect debugging models
with fault introduction rate for software reliability assessment,"
International J. Syst. Science, vol. 23, no. 12, 1992.
[26] H. Pham, "Software reliability and cost models: perspectives,
comparison and practice", European J. of Operational Research, vol.
149, 2003, p. 475- 489.
[27] H. Pham, L. Nordmann, and X. Zhang, "A general imperfect software
debugging model with s-shaped fault detection rate," IEEE Trans.
Reliability, vol. 48, June 1999, pp. 169-175.
[28] H. Pham and X. Zhang, "An NHPP software reliability models and its
comparison," International J. of Reliability, Quality and Safety
Engineering, vol. 14, no. 3, 1997, pp. 269-282.
[29] H. Pham, System software reliability, Springer London, 2006.
[30] X. Zhang, X. Teng and H. Pham, "Considering Fault Removal
Efficiency in Software Reliability Assessment", IEEE Trans. on
Systems, Man, & Cybernetics- Part A, vol. 33, no.1, 2003, pp. 114-120.
[31] S. Hwang and H. Pham, "Quasi-renewal time-delay fault-removal
consideration in software reliability modelling," IEEE Trans. on
systems, man and cybernetics-Part A:Systems and humans, vol. 39, no.
1, January 2009.
[32] K. C. Chiu, Y. S. Huang, and T. Z. Lee, "A study of software reliability
growth from the perspective of learning effects,"Reliability Engineering
and System Safety, 2008, pp. 1410-1421.
[33] M. Zhao, and M. Xie, "On the log-power NHPP software reliability
model,". Proceedings of the Third IEEE International Symposium on
Softw. Reliability Engineering, Research Triangle Park, North Carolina,
1992, pp. 14-22.
[34] M. R. Lyu, and A. Nikora, "Applying software reliability models more
effectively," IEEE Softw., 1992, 43-52.
[35] H. Pham and C. Deng, "Predictive-ratio risk criterion for selecting
software reliability models," Proc. Ninth International Conf. On
Reliability and Quality in Design, August 2003.
[36] P. L. Li, J. Herbsleb, and M. Shaw, "Forecasting field defect rates using
a combined time-based and metrics-based approach: a case study of
OpenBSD," Proceedings of the 16th IEEE International Symposium on
Softw. Reliability Engineering, Chicago, IL, 2005, pp. 193-202.
[37] R. Peng, Q.P. Hu,and S.H. Ng, ,"Incorporating Fault Dependency and
Debugging Delay in Software Reliability Analysis," Proceedings of the
IEEE ICMIT, 2008, pp.641-645.
[38] S. Dahiya et al.,"Power quality evaluation in deregulated power system
using matrix method," International J. Global Energy Issues, vol. 28,
no. 1, 2007.
[39] R. K. Garg, V. K. Gupta, and V. P. Agrawal,"Quality evaluation of
thermal power plants by graph theoretical methodology," International
J. of Power and Energy Systems, 27 (1), 2007, pp. 42-48.
[40] M. Marcus, and H. Minc," Permanents," American Mathematics, 72,
1965, pp. 571-91.
@article{"International Journal of Information, Control and Computer Sciences:52690", author = "RajPal Garg and Kapil Sharma and Rajive Kumar and R. K. Garg", title = "Performance Analysis of Software Reliability Models using Matrix Method", abstract = "This paper presents a computational methodology
based on matrix operations for a computer based solution to the
problem of performance analysis of software reliability models
(SRMs). A set of seven comparison criteria have been formulated to
rank various non-homogenous Poisson process software reliability
models proposed during the past 30 years to estimate software
reliability measures such as the number of remaining faults, software
failure rate, and software reliability. Selection of optimal SRM for
use in a particular case has been an area of interest for researchers in
the field of software reliability. Tools and techniques for software
reliability model selection found in the literature cannot be used with
high level of confidence as they use a limited number of model
selection criteria. A real data set of middle size software project from
published papers has been used for demonstration of matrix method.
The result of this study will be a ranking of SRMs based on the
Permanent value of the criteria matrix formed for each model based
on the comparison criteria. The software reliability model with
highest value of the Permanent is ranked at number – 1 and so on.", keywords = "Matrix method, Model ranking, Model selection,
Model selection criteria, Software reliability models.", volume = "4", number = "11", pages = "1661-8", }
