loading...
Cost Implications of Interrater Agreement for Software Process Assessments
Bethesda, Maryland March 20-March 21
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/METRIC.1998.731225Fifth International Symposium on Soft ...
 This Article 
 
PDF
HTML
 
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Khaled El Emam, Fraunhofer Institute for Experimental Software Engineering
Sonia Rousseau, Sanofi Pharma
Eric Jacquet, Sanofi Pharma
Much empirical research has been done recently on evaluating and modeling interrater agreement in software process assessments. Interrater agreement is the extent to which assessors agree in their ratings of software process capabilities when presented with the same evidence and performing their ratings independently. This line of research was based on the premise that lack of interrater agreement can lead to erroneous decisions from process assessment scores. However, thus far we do not know the impact of interrater agreement on the cost of assessments. In this paper we report on a study that evaluates the relationship between interrater agreement and the cost of the consolidation activity in assessments. The study was conducted in the context of two assessments using the emerging international standard ISO/IEC 15504. Our results indicate that for organizational processes, the relationship is strong and in the expected direction. For project level processes no relationship was found. These results indicate that for assessments that include organizational processes in their scope, ensuring high interrater agreement could lead to a reduction in their costs.
Citation:
Khaled El Emam, Jean-Martin Simon, Sonia Rousseau, Eric Jacquet, "Cost Implications of Interrater Agreement for Software Process Assessments," metrics, pp.38, Fifth International Symposium on Software Metrics (METRICS'98), 1998
Usage of this product signifies your acceptance of the Terms of Use.


Suggestions