Evaluation of Model Evaluation Criterion for Software Development Effort Estimation
Estimation of model parameters is necessary to predict
the behavior of a system. Model parameters are estimated using
optimization criteria. Most algorithms use historical data to estimate
model parameters. The known target values (actual) and the output
produced by the model are compared. The differences between the
two form the basis to estimate the parameters. In order to compare
different models developed using the same data different criteria are
used. The data obtained for short scale projects are used here. We
consider software effort estimation problem using radial basis
function network. The accuracy comparison is made using various
existing criteria for one and two predictors. Then, we propose a new
criterion based on linear least squares for evaluation and compared
the results of one and two predictors. We have considered another
data set and evaluated prediction accuracy using the new criterion.
The new criterion is easy to comprehend compared to single statistic.
Although software effort estimation is considered, this method is
applicable for any modeling and prediction.
[1] M. Jørgensen, “A Review of Studies on Expert Estimation of Software Development Effort,”The Journal of Systems and Software, vol. 70, pp.37–60, 2004.
[2] M.J. Shepperd, C. Schofield, “ Estimating Software Project Effort Using Analogies,” IEEE Trans. Software Eng., vol. 23, pp. 736-743,1997.
[3] B. Boehm, E. Horowitz, R. Madachy, D. Reifer, B.K. Clark, B.Steece, A.W. Brown, S. Chulani,C. Abts, Software Cost Estimation with COCOMO II, Prentice Hall, 2000.
[4] J. Wen, S. Li, Z. Lin, Y. Hu, C. Huang, “Systematic literature review of machine learning based software development effort estimation models,”Information & Software Technology, vol. 54,pp. 41-59, 2012.
[5] V. S Dave, K. Dutta,” Neural Network based Models for Software Effort Estimation: A Review,” Artificial Intelligence Review, Springer, online 06 May 2012.
[6] Simon Haykin, Neural Networks and Learning Machines, PHI learning Private Limited, New Delhi, 3rd edition, 2010.
[7] A. Adri, A. Zakrani, “Design of Radial Basis Function Neural Networks for Software Effort Estimation,” International Journal of Computer Science Issues, vol. 4,pp. 11-17, 2010.
[8] P.V.G.D. Prasad Reddy, K.R. Sudha, P. Rama Sree, S.N.S.V.C. Ramesh, “Software Effort Estimation using Radial Basis and Generalized Regression Neural Networks,” Journal of Computing, vol. 2,pp. 87-92, 2010.
[9] V.S.Dave, K. Dutta, “Comparison of Regression model, Feed-forwardNeural Network and Radial Basis Neural Network for Software Development Effort estimation,” ACM SIGSOFT Software Engineering Notes, vol. 36,pp. 1-5, 2011.
[10] B.A.Kitchenham, L.M.Pickard, S.G.MacDonell and M.J.Shepperd, “What accuracy statistics really measure,” IEE Proc. Software, vol. 148,pp. 81-85, 2001.
[11] C. Lopez-Martin, “A Fuzzy Logic Model for predicting the Development effort of Short Scale Programs based upon Two Independent Variables,” Applied Soft Computing,vol.11, pp.724-732, 2011.
[12] N. Garcia-Diaz, C. Lopez-Martin, A. Chavoya, “A Comparative study of Two Fuzzy Logic Models for Software Development Effort Estimation,”Procedia Technology,vol. 7,pp. 305-314, 2013.
[13] A. Arcuri, L. Briand, “A Practical Guide for Using Statistical Tests to Assess Randomized Algorithms in Software Engineering,” ICSE’11,Honolulu, USA,May 21-28, 2011, pp. 1-10.
[14] E. Praynlin, P. Latha, “Software Effort Estimation Models Using Radial Basis Function,” International Journal of Computer, Information, Systems and Control Engineering, vol. 8, no. 1, World Academy of Science, Engineering and Technology, pp. 248-253, 2014.
[15] C. Lopez-Martin, J. LeboeufPasquier, Cornelio Yanez.M, Augustin Gutierrez, T, “Software Development Effort Estimation Using Fuzzy Logic: A case study,”Proceedings of the sixth International Conference on Computer Science, (ENC’05), IEEE Computer Society, 26-30 Sep. 2005, pp.113-120.
[1] M. Jørgensen, “A Review of Studies on Expert Estimation of Software Development Effort,”The Journal of Systems and Software, vol. 70, pp.37–60, 2004.
[2] M.J. Shepperd, C. Schofield, “ Estimating Software Project Effort Using Analogies,” IEEE Trans. Software Eng., vol. 23, pp. 736-743,1997.
[3] B. Boehm, E. Horowitz, R. Madachy, D. Reifer, B.K. Clark, B.Steece, A.W. Brown, S. Chulani,C. Abts, Software Cost Estimation with COCOMO II, Prentice Hall, 2000.
[4] J. Wen, S. Li, Z. Lin, Y. Hu, C. Huang, “Systematic literature review of machine learning based software development effort estimation models,”Information & Software Technology, vol. 54,pp. 41-59, 2012.
[5] V. S Dave, K. Dutta,” Neural Network based Models for Software Effort Estimation: A Review,” Artificial Intelligence Review, Springer, online 06 May 2012.
[6] Simon Haykin, Neural Networks and Learning Machines, PHI learning Private Limited, New Delhi, 3rd edition, 2010.
[7] A. Adri, A. Zakrani, “Design of Radial Basis Function Neural Networks for Software Effort Estimation,” International Journal of Computer Science Issues, vol. 4,pp. 11-17, 2010.
[8] P.V.G.D. Prasad Reddy, K.R. Sudha, P. Rama Sree, S.N.S.V.C. Ramesh, “Software Effort Estimation using Radial Basis and Generalized Regression Neural Networks,” Journal of Computing, vol. 2,pp. 87-92, 2010.
[9] V.S.Dave, K. Dutta, “Comparison of Regression model, Feed-forwardNeural Network and Radial Basis Neural Network for Software Development Effort estimation,” ACM SIGSOFT Software Engineering Notes, vol. 36,pp. 1-5, 2011.
[10] B.A.Kitchenham, L.M.Pickard, S.G.MacDonell and M.J.Shepperd, “What accuracy statistics really measure,” IEE Proc. Software, vol. 148,pp. 81-85, 2001.
[11] C. Lopez-Martin, “A Fuzzy Logic Model for predicting the Development effort of Short Scale Programs based upon Two Independent Variables,” Applied Soft Computing,vol.11, pp.724-732, 2011.
[12] N. Garcia-Diaz, C. Lopez-Martin, A. Chavoya, “A Comparative study of Two Fuzzy Logic Models for Software Development Effort Estimation,”Procedia Technology,vol. 7,pp. 305-314, 2013.
[13] A. Arcuri, L. Briand, “A Practical Guide for Using Statistical Tests to Assess Randomized Algorithms in Software Engineering,” ICSE’11,Honolulu, USA,May 21-28, 2011, pp. 1-10.
[14] E. Praynlin, P. Latha, “Software Effort Estimation Models Using Radial Basis Function,” International Journal of Computer, Information, Systems and Control Engineering, vol. 8, no. 1, World Academy of Science, Engineering and Technology, pp. 248-253, 2014.
[15] C. Lopez-Martin, J. LeboeufPasquier, Cornelio Yanez.M, Augustin Gutierrez, T, “Software Development Effort Estimation Using Fuzzy Logic: A case study,”Proceedings of the sixth International Conference on Computer Science, (ENC’05), IEEE Computer Society, 26-30 Sep. 2005, pp.113-120.
@article{"International Journal of Electrical, Electronic and Communication Sciences:69200", author = "S. K. Pillai and M. K. Jeyakumar", title = "Evaluation of Model Evaluation Criterion for Software Development Effort Estimation", abstract = "Estimation of model parameters is necessary to predict
the behavior of a system. Model parameters are estimated using
optimization criteria. Most algorithms use historical data to estimate
model parameters. The known target values (actual) and the output
produced by the model are compared. The differences between the
two form the basis to estimate the parameters. In order to compare
different models developed using the same data different criteria are
used. The data obtained for short scale projects are used here. We
consider software effort estimation problem using radial basis
function network. The accuracy comparison is made using various
existing criteria for one and two predictors. Then, we propose a new
criterion based on linear least squares for evaluation and compared
the results of one and two predictors. We have considered another
data set and evaluated prediction accuracy using the new criterion.
The new criterion is easy to comprehend compared to single statistic.
Although software effort estimation is considered, this method is
applicable for any modeling and prediction.
", keywords = "Software effort estimation, accuracy, Radial Basis
Function, linear least squares.", volume = "9", number = "1", pages = "118-4", }
