Benchmarking of Pentesting Tools

The benchmarking of tools for dynamic analysis of
vulnerabilities in web applications is something that is done
periodically, because these tools from time to time update their
knowledge base and search algorithms, in order to improve their
accuracy. Unfortunately, the vast majority of these evaluations are
made by software enthusiasts who publish their results on blogs
or on non-academic websites and always with the same evaluation
methodology. Similarly, academics who have carried out this type of
analysis from a scientific approach, the majority, make their analysis
within the same methodology as well the empirical authors. This
paper is based on the interest of finding answers to questions that
many users of this type of tools have been asking over the years,
such as, to know if the tool truly test and evaluate every vulnerability
that it ensures do, or if the tool, really, deliver a real report of all the
vulnerabilities tested and exploited. This kind of questions have also
motivated previous work but without real answers. The aim of this
paper is to show results that truly answer, at least on the tested tools,
all those unanswered questions. All the results have been obtained
by changing the common model of benchmarking used for all those
previous works.

Authors:

• Esteban Alejandro Armas Vega
• Ana Lucila Sandoval Orozco
• Luis Javier García Villalba

Keywords:

• Cybersecurity
• IDS
• security
• web scanners
• web vulnerabilities.

References:

[1] Verizon Enterprise. 2016 Data Breach Investigations Report. Report,
Verizon Enterprise, July 2016.
[2] A. Sagala and E. Manurung. Testing and Comparing Result
Scanning Using Web Vulnerability Scanner. Advanced Science Letters,
21(11):3458–3462, November 2015.
[3] P. Baral. Web Application Scanners: A Review of Related Articles.
IEEE Potentials, 30(2):10–14, March 2011.
[4] Y. Makino and V. Klyuev. Evaluation of Web Vulnerability Scanners.
In Proceedings of the IEEE 8th International Conference on Intelligent
Data Acquisition and Advanced Computing Systems: Technology and
Applications (IDAACS), volume 1, pages 399–402, Warsaw, PL,
September 2015.
[5] The Open Web Application Security Project OWASP. OWASP Zed
Attack Proxy Project. https://www.owasp.org/index.php/OWASP Zed
Attack Proxy Project, April 2016.
[6] Google. Google Code - Skipfish. https://code.google.com/archive/p/
skipfish/, March 2016.
[7] RandomStorm. Damn Vulnerable Web Application (DVWA). http://
www.dvwa.co.uk, March 2016.
[8] Google. Google Code - WAVSEP. https://code.google.com/archive/p/
wavsep/, March 2016.
[9] F. A. Saeed. Using WASSEC to Analysis and Evaluate Open Source
Web Application Security Scanners. International Journal of Computer
Science and Network, 3(2):43–49, April 2014.
[10] Web Application Security Consortium. Web Application Security
Scanner Evaluation Criteria WASSEC. http://goo.gl/aePtyC, April 2016.
[11] W3af. W3af - Open Source Web Application Security Scanner. http:
//w3af.org, Abril 2016.
[12] N. I. Daud, K. A. A. Bakar, and M. S. Md. Hasan. A Case Study on
Web Application Vulnerability Scanning Tools. In Proceedings of the
Conference of Science and Information (SAI), pages 595–600, 2014.
[13] Snort - Network Intrusion Detection and Prevention System. https://
www.snort.org/, Abril 2016.
[14] H. Alnabulsi, Md. R. Islam, and Q. Mamun. Detecting SQL Injection
attacks using SNORT IDS. In Proceedings of the 2014 Asia-Pacific
World Congress on Computer Science and Engineering (APWC on CSE),
pages 1–7. IEEE, Nov 2014.
[15] M. Dabbour, I. Alsmadi, and E. Alsukhni. Efficient Assessment and
Evaluation for Websites Vulnerabilities using SNORT. International
Journal of Security and its Applications, 7(1), 2013.
[16] HP. HP WebInsPect. Product Manual, HP, March 2015.
[17] Arachni. ARACHNI Web Application Security Scanner Framework.
http://www.arachni-scanner.com, March 2016.
[18] F. A. Saeed. Using WASSEC to Evaluate Commercial Web Application
Security Scanners. International Journal of Soft Computing and
Engineering (IJSCE), 4(1):177–181, March 2014.
[19] A. Doup´e, M. Cova, and G. Vigna. Detection of Intrusions and Malware,
and Vulnerability Assessment. In Christian Kreibich and Marko Jahnke,
editors, Proceedings of the 7th International Conference (DIMVA 2010),
pages 111–131, Bonn, Germany, July 2010.
[20] A. Doup´e. WackoPicko Vulnerable Website. https://github.com/
adamdoupe/WackoPicko, March 2016.
[21] The Open Web Application Security Project OWASP. OWASP Top 10
- 2013 The Ten Most Critical Web Application Security Risks. Release,
The Open Web Application Security Project OWASP, June 2013.