| Michael Bolton 2006-07-18, 8:03 am |
| > I'm doing a report on TDD. Can anyone direct me to some statistics or
> imperical comparison of how much code is required to code all the tests,
> versus just doing coding without the tests?
> In all the examples I've seen - from the very trivial, to the more
> complicated, the TDD approach seems to more than double the coding
> effort. However this is just from me reading the standard books on the
> subject and making a rough comparison.
For any given study (including yours) the measurement will be based on
your notion of what coding is.
If coding means "the typing of lines into some editor", then TDD will
take more time than code written without tests.
If coding means "the completion of some programming task, such that the
program is not only written but also meets some requirement to some
degree", then TDD will likely take less time than code written without
tests.
> Now I know that some of you will want to tell me "that TDD saves you so
> much time in this way and that way", and thats all great - I'll talk
> about that too - but I need hard facts! :-)
The hardest fact of all is that programming processes are fiendishly
difficult to compare. Studies of this nature will have a hard time
achieving construct validity. If you give the same programming task to
the same programmer twice, the programmer will typically learn
something on the first run that will affect the speed of the second
run. If you give the same programmer different programming tasks, one
task might be more difficult to accomplish than the other. If you give
the same programming task to two programmers, require one to use TDD
and the other not to use TDD, and compare the results, your result will
be distorted by the varying skills and temperaments of the programmers.
In addition, a good experiment would not only compare the speed of the
completed program, but also the quality of the completed program. A
good experiment would also consider how the programmer and the team
feel at the end of the day; whether expectations had been met, missed,
or exceeded; whether the program was more or less easily maintainable;
the performance of the code; how well the programming environment
supported TDD; and so on.
> Can anyone direct me to a more scientific discussion?
I doubt it. I keep seeing these experiments from University of
Karlsruhe that try to do things like comparing XP and other programming
processes, and the studies have so far struck me as being
experimentally bogus. If you can figure out a way to constrain the
variables and the subjectivity above, it could mean a Nobel Prize in
economics for you.
> Even a guestimate based on your experiences would be a start :-)
I predict that you'll get plenty of that. In any case, the most
important thing is to focus on the amount of time it takes to complete
a functioning, well-tested program, not merely some number of lines of
code.
After all, as Jerry Weinberg says, if you don't care about quality, you
can meet any other requirement.
---Michael B.
|