Manual to Automated Testing: An Effort-Based Approach for Determining the Priority of Software Test Automation

Test automation allows performing difficult and time consuming manual software testing tasks efficiently, quickly and repeatedly. However, development and maintenance of automated tests is expensive, so it needs a proper prioritization what to automate first. This paper describes a simple yet efficient approach for such prioritization of test cases based on the effort needed for both manual execution and software test automation. The suggested approach is very flexible because it allows working with a variety of assessment methods, and adding or removing new candidates at any time. The theoretical ideas presented in this article have been successfully applied in real world situations in several software companies by the authors and their colleagues including testing of real estate websites, cryptographic and authentication solutions, OSGi-based middleware framework that has been applied in various systems for smart homes, connected cars, production plants, sensors, home appliances, car head units and engine control units (ECU), vending machines, medical devices, industry equipment and other devices that either contain or are connected to an embedded service gateway.

Authors:

• Peter Sabev
• Katalina Grigorova

Keywords:

• Automated Testing
• Manual Testing
• Test Automation
• Software testing
• Test Prioritization.

References:

[1] E. Aranha, "Estimating test execution effort based on test specifications",
Centro de Informática, Universidade Federal de Pernambuco, Recife,
Brazil, pp. 29-42, January 2009.
[2] M. Cohn, "Agile estimating and planning", Pearson Education, Prentice
Hall, New York, USA, 2005.
[3] J. Fernandes and A. Di Fonzo, "When to automate your testing (and when
not to)", July 2010. [4] S. Yoo and M. Harman, "Regression testing minimization, selection and
prioritization: A survey", King’s College London, London, England, March
2012.
[5] B. Marick, "When should a test be automated", California, USA, 1998
[11th International Software Quality Week QualWeek 98 San Francisco,
1998].
[6] D. Mosley and B. Posey, "Just enough software test automation", Yourdon
Press, Prentice Hall Professional, Englewood Cliffs, NJ, USA, 2002.
[7] V. Motwani, "The when and how of test automation", Bangalore, India
(2001). [Annual International Software Testing Conference in India, 2001].
[8] G. Robert, "Automation selection criteria – picking the "right" candidates,
LogiGear Magazine, 2014.
[9] M. Rouse, "What is requirements stability index (RSI)?", TechTarget,
2005.
[10] E. Dustin, J. Rashka and J. Paul, "Automated software testing". Reading,
Addison-Wesley, 1999.
[11] S. Desikan and G. Ramesh, Software testing. Bangalore, India, Dorling
Kindersley, India, 2006.
[12] S. Covey, A. Merrill and R. Merrill, First things first. Simon & Schuster,
New York, USA, 1994.