An Empirical Investigation of the Influence of a Type of Side Effects on Program Comprehension
|
Abstract—This paper reports the results of a study on the impact of a type of side effect (SE) upon program comprehension. We applied a crossover design on different tests involving fragments of C code that include increment and decrement operators. Each test had an SE version and a side-effect-free (SEF) counterpart. The variables measured in the treatments were the number of correct answers and the time spent in answering. The results show that the side-effect operators considered significantly reduce performance in comprehension-related tasks, providing empirical justification for the belief that side effects are harmful.
[1] 665 L. Cannon, R. Elliott, L. Kirchhoff, J. Miller, J. Milner, R. Mitze, R. Schan, E. Whittington, N. Spencer, H. Keppel, D. Brader, and M. Brader, Recommended C Style and Coding Standards http://www.cs.umd.edu/users/cml/cstyleindhill-cstyle.html , 2000.
[2] J. Cohen, Statistical Power Analysis for the Behavioral Sciences, second ed. Lawrence Erlbaum Assoc., 1988.
[3] G.E. Dallal, The Computer-Aided Analysis of Crossover Studies http://www.tufts.edu/~gdallalcrossovr.htm , 2000.
[4] E.W. Dijkstra, Goto Statement Considered Harmful Comm. ACM, vol. 11, no. 3, pp. 147-148, Mar. 1968.
[5] J.E. Grizzle, The Two-Period Change-Over Design and Its Use in Clinical Trials Biometrics, vol. 21, pp. 467-480, 1965.
[6] M. Harman, L. Hu, X. Zhang, and M. Munro, Side-Effect Removal Transformation Proc. IEEE Int'l Workshop Program Comprehension (IWPC 2001), pp. 309-319, May 2001, available athttp://www.brunel.ac.uk/~csstmmh2linsert /.
[7] D. Hendrix, J.H. Cross II, and S. Maghsoodloo, The Efectiveness of Control Structure Diagrams in Source Code Comprehension Activities IEEE Trans. Software Eng., vol. 28, no. 5, pp. 463-477, May 2002.
[8] M. Hills and P. Armitage, The Two-Period Cross-Over Clinical Trial British J. Clinical Pharmacology, vol. 8, pp. 7-20, 1979.
[9] ISO, International Standards Organisation: Programming Languages C, International standard, ISO/IEC 9899: 1990 (E), Dec. 1990.
[10] B.W. Kernighan and R. Pike, The Practice of Programming. Addison-Wesley, 1999.
[11] B.A. Kitchenham, S.L. Pfleeger, L.M. Pickard, P.W. Jones, D.C. Hoaglin, K. El-Emam, and J. Rosenberg, Preliminary Guidelines for Empirical Research in Software Engineering IEEE Trans. Software Eng., vol. 28, no. 8, pp. 721-734, Aug. 2002.
[12] R.O. Kuehl, Design of Experiments: Statistical Principles of Research Design and Analysis, second ed. Duxbury Thomson Learning, 2000.
[13] R.V. Lenth, Some Practical Guidelines for Effective Sample Size Determination The Am. Statistician, vol. 55, no. 3, pp. 187-193, 2001.
[14] L.F. Marshall and J. Webber, Gotos Considered Harmful and other Programmers' Taboos Technical Report 699, Dept. of Computing Science, Univ. of Newcastle, 2000.
[15] R.J. Miara, J.A. Musselman, J.A. Navarro, and B. Shneiderman, Program Indentation and Comprehensibility Comm. ACM, vol. 1983, no. 11, pp. 861-867, Nov. 1983.
[16] S. McConnell, Keep It Simple IEEE Software, vol. 13, no. 6, pp. 127-128, Nov. 1996.
[17] J. Miller, J. Daly, M. Wood, M. Roper, and A. Brooks, Statistical Power and Its Subcomponents Missing and Misunderstood Concepts in Empirical Software Engineering Research Information and Software Technology, vol. 39, no. 4, pp. 285-295, 1997.
[18] G.A. Milliken and D.E. Johnson, Analysis of Messy Data, volume 1: Designed Experiments. Cambridge Univ. Press, 1984.
[19] P.W. Oman and C.R. Cook, Typographic Style Is More than Cosmetic Comm. ACM, vol. 33, no. 5, pp. 506-520, May 1990.
[20] J. Stevens, Applied Multivariate Statistics for the Social Sciences, third ed. Lawrence Erlbaum Assoc., 1996.
[21] C. Wohlin, P. Runeson, M. Höst, M.C. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in Software Engineering: An Introduction. Kluwer Academic, 2000.
[22] L.Q. Yue and P. Roach, A Note on the Sample Size Determination in Two-Period Repeated Measurements Crossover Design with Application to Clinical Trials J. Biopharmaceutical Statistics, vol. 8, no. 4, pp. 577-584, 1998.
Index Terms:
Side-effect-free programs, crossover designs, program comprehension.
Citation:
J.J. Dolado, M. Harman, M.C. Otero, L. Hu, "An Empirical Investigation of the Influence of a Type of Side Effects on Program Comprehension," IEEE Transactions on Software Engineering, vol. 29, no. 7, pp. 665-670, July 2003, doi:10.1109/TSE.2003.1214329
PURCHASE ARTICLE: $19
IEEE Xplore Subscribers
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb