loading...
What Should You Optimize When Building an Estimation Model?
Como, Italy September 19-September 22
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/METRICS.2005.5511th IEEE International Software Metr ...
 This Article 
 
PDF
HTML
 
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Chris Lokan, Australian Defence Force Academy
When estimation models are derived from existing data, they are commonly evaluated using statistics such as mean magnitude of relative error. But when the models are derived in the first place, it is usually by optimizing something else — typically, as in statistical regression, by minimizing the sum of squared deviations. How do estimation models for typical software engineering data fare, on various common accuracy statistics, if they are derived using other "fitness functions"? In this study, estimation models are built using a variety of fitness functions, and evaluated using a wide range of accuracy statistics. We find that models based on minimizing actual errors generally out-perform models based on minimizing relative errors. Given the nature of software engineering data sets, minimizing the sum of absolute deviations seems an effective compromise.
Index Terms:
effort estimation, genetic programming, accuracy statistics, fitness functions
Citation:
Chris Lokan, "What Should You Optimize When Building an Estimation Model?," metrics, pp.34, 11th IEEE International Software Metrics Symposium (METRICS'05), 2005
Usage of this product signifies your acceptance of the Terms of Use.