Testdriven development and use cases

I need some help figuring out how/if my approach to automated testing is
flawed.

We're considering moving towards testdriven development and the current
issue is what to consider a unit and how to define tests on that. I've
suggested that we consider the requirements (given in the form of use
case steps) as "units", my reasoning being that a green bar would mean
that the use case is, well, usable (according to requirements) and ready
to hand over to the group doing validation.

I know that the standard unit in a Java project is a class, but here are
my arguments against us doing that (and note that I'm just referring to
our specific case):

We're in the middle of construction and already have a lot of code. This
code is by no means perfect, but has been through testing and defect
correction and has been ok:ed by the validation group. We will be adding
methods to those classes, but we're pretty pressed for time and I was
hoping it would be enough to just add tests for the newly implemented
requirements, i.e. creating test just for the parts of the older classes
that are actually changed. This way, we still get "green bar == all
defined units passing all their tests".

Another reason is that this will take some load of the testers. They
base their test cases on the use case text and I see my approach as a
way to quickly get regression testing for use cases. We need them to do
the testing on stuff that is difficult to test automatically (e.g.
usability).

If we keep at it and add unit tests for older requirements when we get
defect reports on them, eventually we'll end up with tests for most of
the code. The longer the approach works, the easier it will be to get
resources to write test for the remaining requirements.

Comments, please!

BR,
Mikael Lindberg

Mikael Lindberg a écrit:
> I need some help figuring out how/if my approach to automated testing is
> flawed.
>
> We're considering moving towards testdriven development and the current
> issue is what to consider a unit and how to define tests on that. I've
> suggested that we consider the requirements (given in the form of use
> case steps) as "units", my reasoning being that a green bar would mean
> that the use case is, well, usable (according to requirements) and ready
> to hand over to the group doing validation.

Hi Mikael,

In TDD, we don't talk about unit test but developer test. The unit to be
tested is too vague to mean a useful frame of work. In TDD, each test
you write should demonstrate the smallest piece of functionnality that
you think the code must display and currently doesn't display yet.
Something as big as a use case should be declared "usable" only when it
passes acceptance tests, not developer tests. A green bar applied to
developer tests -- of course you could use a green bar in acceptance
tests as well -- only means that the code still holds all the properties
you put in it during your developers tasks.


>
> I know that the standard unit in a Java project is a class, but here are
> my arguments against us doing that (and note that I'm just referring to
> our specific case):
>
> We're in the middle of construction and already have a lot of code. This
> code is by no means perfect, but has been through testing and defect
> correction and has been ok:ed by the validation group. We will be adding
> methods to those classes, but we're pretty pressed for time and I was
> hoping it would be enough to just add tests for the newly implemented
> requirements, i.e. creating test just for the parts of the older classes
> that are actually changed. This way, we still get "green bar == all
> defined units passing all their tests".
>
In TDD green bar usually == "no part of the whole code of your app has
been broken in any way", its an insurance you have so that you can
continue to add features and change the design at will if the current
design doesn't facilitate such an addition. Seems to me that you want
the greenbar to mean: "some aspects of the new feature you are adding
seem to work". Sorry if I misinterpret what you wrote, but I think
you're just trading test coverage for automation. The danger when a part
only of the code is tested with xUnit, is that you have a partial but
fast feedback device, which could give you a very false feeling of safety.

I've done that : partially TDD'ing a legacy app. The new features were a
snap to code, but each time we assembled the whole thing, we were in for
big surprises and whole days spent debugging things we had forgotten
to (manually) re-test, just because our green bar gave us so much
confidence and was so much more user friendly than manual tests.

This is how I interpret your situation :
- you don't have full regression test for the current code
- the code is not perfect so it could contain many undetected defects
- the code has been through correction (without regression test), so new
defects could have been introduced
- you're pressed for time
- you want to take some load off the testers

The undetected defects will cause failures and you'll have to locate,
isolate and correct these defects : unplanned debugging tasks, at
unpredictible cost and delay. So the time pressure will increase, you
and the testers will have much more to do. By adding feature on untested
code, although your new code is tested, you're just making your initial
problem bigger.

I would try to write tests for the existing untested code, instead of
the new code. After all :
- perfectly testing a new feature is just waste if the feature has no
solid infrastructure in the app to rely on.
- the new code, beeing more recent is much easier to review, explore and
debug than the old one so the bottleneck will be bugs on the old code.
- it might be preferable to delay the release of new feature than put
existing features at risk ?

Hope this helps.
Regards,
Christophe

Hello Christophe,

thank you for your comments!

Christophe Thibaut wrote:

[snip]

> In TDD, we don't talk about unit test but developer test. The unit to be
> tested is too vague to mean a useful frame of work. In TDD, each test
> you write should demonstrate the smallest piece of functionnality that
> you think the code must display and currently doesn't display yet.
> Something as big as a use case should be declared "usable" only when it
> passes acceptance tests, not developer tests. A green bar applied to
> developer tests -- of course you could use a green bar in acceptance
> tests as well -- only means that the code still holds all the properties
> you put in it during your developers tasks.

Perhaps I was being a bit unclear (english is not my first language), a
use case would not be considered a unit; a step in a use case would be.
The use case text is broken down into what is basically a bulleted list
of actions and responses. (e.g. * User does X, system does Y. * User
does Z, system does W etc). Couldn't one such step be considered a small
piece of functionality?

In reality, I suspect that the tests defined on such a step would be a
suite of tests on lower level pieces (classes, beans, calls to other
systems).


[snip]

> I would try to write tests for the existing untested code, instead of
> the new code. After all :
> - perfectly testing a new feature is just waste if the feature has no
> solid infrastructure in the app to rely on.
> - the new code, beeing more recent is much easier to review, explore and
>  debug than the old one so the bottleneck will be bugs on the old code.
> - it might be preferable to delay the release of new feature than put
> existing features at risk ?

I hear you. The main problem here is a political one: the project hasn't
been able to release something for acceptance tests during the last two
iterations (we're in the middle of the third out of a planned four) and
the "customer" (really an internal function to the company) is getting a
bit annoyed. If asked, I do believe they would prefer to get some buggy
but "complete" use cases rather than none, which I think would be the
result if we halted development to write tests for existing code. I
realise that they might not actually mean that, but the organization is
pretty immature and I'm just a lowly consultant :)

Another reason for not writing tests for all older requirements is that
just a few developers in the team know anything about how to write jUnit
tests. My idea here was to let two of the more experienced developers
scout ahead and write tests for the functionality they implement to
figure out the practical details. Later, perhaps between iterations, we
can introduce the rest of the team to the idea.

BR,
Mikael Lindberg

> [snip]
> 
>
>
> I hear you. The main problem here is a political one: the project hasn't
> been able to release something for acceptance tests during the last two
> iterations (we're in the middle of the third out of a planned four) and
> the "customer" (really an internal function to the company) is getting a
> bit annoyed. If asked, I do believe they would prefer to get some buggy
> but "complete" use cases rather than none, which I think would be the
> result if we halted development to write tests for existing code. I
> realise that they might not actually mean that, but the organization is
> pretty immature and I'm just a lowly consultant :)

Hi Mikael,
In my frame of experience "buggy but complete" doesn't make it as a
deliverable product work. While trying to make things *apparently*
complete (more visible features) you are leaving future problems in your
code that will come back later, only magnified. I could be wrong, from
that distance, but it seems you're just entering in the tar pit, and you
say "let's go ahead, faster".

Deciding between an unreliable product or a smaller product is always a
political problem, not a technical. It's just that when as a customer
you decide to go for an unreliable -- seemingly complete -- product you
cannot assess your risk as well as if you choose a partially done, but
quite reliable, product. It's not choosing between two evils, it's
choosing between having a known and an unknown evil. What's the point of
getting every required feature if my business is put at risk of a major
system failure in one of these untested functions ?

> If asked, I do believe they would prefer ..
I would just ask them.

>
> Another reason for not writing tests for all older requirements is that
> just a few developers in the team know anything about how to write jUnit
> tests.
Learning how to use junit takes only 1 hour, but learning how to
increase code testability takes months.

My idea here was to let two of the more experienced developers
> scout ahead and write tests for the functionality they implement to
> figure out the practical details. Later, perhaps between iterations, we
> can introduce the rest of the team to the idea.

My advice is to introduce the whole team to the idea of testing the code
they write, lest you have neatly tested new feature standing upon
"crappy" code. Then you'd have code defects *plus* team alignment
problems when the system failures rush back in your project.

Regards.
Christophe

>
> BR,
> Mikael Lindberg

On Wed, 25 Aug 2004 20:46:49 +0200, Mikael Lindberg <d94ml@efd.lth.se>
wrote:

>I need some help figuring out how/if my approach to automated testing is
>flawed.
>
>We're considering moving towards testdriven development and the current
>issue is what to consider a unit and how to define tests on that. I've
>suggested that we consider the requirements (given in the form of use
>case steps) as "units", my reasoning being that a green bar would mean
>that the use case is, well, usable (according to requirements) and ready
>to hand over to the group doing validation.

The tests you are talking about seem more like acceptance tests than
like unit tests.  Unit tests are very fine grained white box tests
written by the developers with full knowledge of the internals of the
code.  Acceptance tests verify the external behavior of a chunk of the
system (like a feature).  They are course grained black box tests
written by customers/analysist/QA without knowledge of the internals
of the code.

I realize that you probably don't have these job functions perfectly
isolated; but it may still be valuable to consider these tests as
acceptance tests rather than unit tests simply because you can ask
others to write them in advance of the actual software being written.

In any case I often recommend that developers not bother writing unit
tests for the legacy code that already "works".  Rather they should
write their unit tests for the new code they are writing, and a few
unit tests around the area of the old code that they are changing.


-----
Robert C. Martin (Uncle Bob)  | email: unclebob@objectmentor.com
Object Mentor Inc.            | blog:  www.butunclebob.com
The Agile Transition Experts  | web:   www.objectmentor.com
800-338-6716


"The aim of science is not to open the door to infinite wisdom,
but to set a limit to infinite error."
-- Bertolt Brecht, Life of Galileo