Modelling the Occurrence of Defects and Change Requests during User Acceptance Testing

Software developed for a specific customer under contract typically undergoes a period of testing by the customer before acceptance. This is known as user acceptance testing and the process can reveal both defects in the system and requests for changes to the product. This paper uses nonhomogeneous Poisson processes to model a real user acceptance data set from a recently developed system. In particular a split Poisson process is shown to provide an excellent fit to the data. The paper explains how this model can be used to aid the allocation of resources through the accurate prediction of occurrences both during the acceptance testing phase and before this activity begins.

Authors:

• Kevin McDaid
• Simon P. Wilson

Keywords:

• User acceptance testing. Software reliability growth modelling. Split Poisson process. Bayesian methods.

References:

[1] B. Hetzel. The Complete Guide to Software Testing (second edition),
Wiley, 1988.
[2] G.J. Myers. Software Reliability, Principles and Practices, Wiley, 1976.
[3] J.D. Musa. Software Reliability Engineering, McGraw-Hill, New York,
1998.
[4] T.A. Chavez. Decision-Analytic Stopping Rule for Validation of Commercial
Software Systems. IEEE Transactions on Software Engineering, 26:
907-918, 2000.
[5] S.R. Dalal and C.L. Mallows. When should one stop testing? Journal of
the American Statistical Association, 83: 872-879, 1988.
[6] S.A. Hossain and R.C. Dahiya. Estimating the parameters of a nonhomogeneous
Poisson process for software reliability. IEEE Transactions
on Reliability, 42(4):604-612, 1993.
[7] L. Yin and K.S. Trivedi. Confidence Interval Estimation of NHPP-Based
Software Reliability Models. Proceedings of the International Symposium
on Software Reliability Engineering, 1999.
[8] M. Xie, G.Y. Hong and C. Wohlin. A Practical Method for the Estimation
of Software Reliability Growth in the Early Stage of Testing. Proceedings
of the International Symposium on Software Reliability Engineering,
1997.
[9] H. Hoffmann and K. McDaid An Analysis of Failure Occurrence Patterns
during Testing and Operation Supplemental Proceedings of the International
Symposium on Software Reliability Engineering, 2004.
[10] H. Hoffmann and K. McDaid Using Software Reliability Growth Models
to Predict the Occurrence of Defects during Testing Proceedings of the
International Conference on Software Development, Iceland, May 2005.
[11] S. Gokhale and K. S. Trivedi. A Time/Structure Based Software Reliability
Model. Annals of Software Engineering , vol. 8, pp. 85-121, 1999.
[12] N.D. Singpurwalla and S.P. Wilson. Statistical Methods in Software
Engineering. Springer, 1999.
[13] N.D. Singpurwalla and S.P. Wilson. Software Reliability Modeling.
International Statistical Review, 62 3:289-317, 1994.
[14] A.L. Goel and K. Okumoto. Time dependent error-detection rate models
for Software Reliability and Other performance Measures. IEEE Transactions
on Software Engineering, R-29, pp206-211, 1979.
[15] S. Yamada, M. Ohba and S. Osaki. S-shaped Reliability Growth Modeling
for Software Error Detection. IEEE Trans. on Reliability, R-32, pp.
475-485, 1983.
[16] S. Gokhale. Analysis of Software Reliability and Performance, Ph.D.
thesis, Duke University, 1998.
[17] G. Knafl and J. Morgan. Solving ML equations for 2-parameter poissonprocess
models for ungrouped software failure data. IEEE Transactions
on Reliability, 45(1):42-53, 1996.
[18] K. McDaid and S. Wilson. Deciding how long to test software. Journal
of the Royal Statistical Society, Series D, R-50. part 2: 117-134, 2001.