Home > Archive > Extreme Programming > May 2004 > UI testing.
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
|
|
| cyclocross 2004-05-12, 7:44 pm |
| I've been doing unit testing on a java swing app for sometime now and
i've found a couple of things that I was curious if others had run
into. First i've come to the conclusion the my unit tests are very
"shallow", meaning that they don't actaully test much of the app.
Every line of code written has a test around it, yes so that coverage
is good. The kicker i'm seeing is that the vast majority of bugs
doesn't come from written code, but from code that wasn't written.
For instance if i drop a jtable on a panel, i'm giving the user the
ability to click, resize, move columns, etc. I can wire up a button
to do something when a user selects a row in the JTable, and I can
write a test for that first. The problem is that the user can select
multiple rows in the table, and that causes a bug, because I didn't
write code to handle that. I can fire up the app, and click around and
discover lots of bugs that happen because the UI toolkit gives me lots
of behavior that I wasn't aware of. I have lots of other examples of
problems, and i'm wondering if other people have had similar
experiences.
Thanks,
Eric
| |
|
| cyclocross wrote:
> I've been doing unit testing on a java swing app for sometime now and
> i've found a couple of things that I was curious if others had run
> into. First i've come to the conclusion the my unit tests are very
> "shallow", meaning that they don't actaully test much of the app.
Good - each one should be.
But if you only write new code after tests request it, you will have very
high coverage.
> Every line of code written has a test around it, yes so that coverage
> is good. The kicker i'm seeing is that the vast majority of bugs
> doesn't come from written code, but from code that wasn't written.
> For instance if i drop a jtable on a panel, i'm giving the user the
> ability to click, resize, move columns, etc. I can wire up a button
> to do something when a user selects a row in the JTable, and I can
> write a test for that first. The problem is that the user can select
> multiple rows in the table, and that causes a bug, because I didn't
> write code to handle that. I can fire up the app, and click around and
> discover lots of bugs that happen because the UI toolkit gives me lots
> of behavior that I wasn't aware of. I have lots of other examples of
> problems, and i'm wondering if other people have had similar
> experiences.
Define "bug".
In my experience (which includes everything except Swing), I write tests
directly to the GUI controls, but I only activate the features I need.
For example, some dialog boxes have sizing borders regardless of whether
their controls can resize. The user might drag a border in and clip some
controls. But this is a very low priority bug, because users can re-expand
the window very easily. And I usually manage to turn it off (without testing
that I did).
By "bug" do you mean that multiple list box selections escape into your
Logic Layer, and it explodes?
--
Phlip
http://www.xpsd.org/cgi-bin/wiki?Te...tUserInterfaces
| |
| Robert C. Martin 2004-05-12, 7:44 pm |
| On 6 May 2004 11:37:28 -0700, cyclocrossing@hotmail.com (cyclocross)
wrote:
>I've been doing unit testing on a java swing app for sometime now and
>i've found a couple of things that I was curious if others had run
>into. First i've come to the conclusion the my unit tests are very
>"shallow", meaning that they don't actaully test much of the app.
>Every line of code written has a test around it, yes so that coverage
>is good. The kicker i'm seeing is that the vast majority of bugs
>doesn't come from written code, but from code that wasn't written.
>For instance if i drop a jtable on a panel, i'm giving the user the
>ability to click, resize, move columns, etc. I can wire up a button
>to do something when a user selects a row in the JTable, and I can
>write a test for that first. The problem is that the user can select
>multiple rows in the table, and that causes a bug, because I didn't
>write code to handle that. I can fire up the app, and click around and
>discover lots of bugs that happen because the UI toolkit gives me lots
>of behavior that I wasn't aware of. I have lots of other examples of
>problems, and i'm wondering if other people have had similar
>experiences.
>
Sure. Flexible packages provide lots of options and freedoms. Trying
to think of all the unit tests up front is bound to fail. So once you
fire up the GUI and start clicking around on it you'll notice some odd
behaviors. Then you write unit tests that fail because of those odd
behaviors, and then you make those unit tests pass.
-----
Robert C. Martin (Uncle Bob)
Object Mentor Inc.
unclebob @ objectmentor . com
800-338-6716
"The aim of science is not to open the door to infinite wisdom,
but to set a limit to infinite error."
-- Bertolt Brecht, Life of Galileo
| |
|
| Robert C. Martin wrote:
> Sure. Flexible packages provide lots of options and freedoms. Trying
> to think of all the unit tests up front is bound to fail. So once you
> fire up the GUI and start clicking around on it you'll notice some odd
> behaviors. Then you write unit tests that fail because of those odd
> behaviors, and then you make those unit tests pass.
Note this implies tests on the Representation Layer (below the GUI). Such
tests would call that layer the same way the GUI Layer accidentally did.
--
Phlip
http://www.xpsd.org/cgi-bin/wiki?Te...tUserInterfaces
| |
| Dan Knierim 2004-05-12, 7:44 pm |
| Yes this type of occurrence is familiar to me. And not only in UI
development.
It illustrates that XP-style unit testing, while necessary, is usually
not sufficient testing.
It also illustrates that each feature should be justified by a user
acceptance test. Extra bells and whistles offered (often as defaults)
by a tool or library, should be turned off (if possible) unless
required by the user.
cyclocrossing@hotmail.com (cyclocross) wrote in message news:<7b628169.0405061037.49dbc997@posting.google.com>...
> I've been doing unit testing on a java swing app for sometime now and
> i've found a couple of things that I was curious if others had run
> into. First i've come to the conclusion the my unit tests are very
> "shallow", meaning that they don't actaully test much of the app.
> Every line of code written has a test around it, yes so that coverage
> is good. The kicker i'm seeing is that the vast majority of bugs
> doesn't come from written code, but from code that wasn't written.
> For instance if i drop a jtable on a panel, i'm giving the user the
> ability to click, resize, move columns, etc. I can wire up a button
> to do something when a user selects a row in the JTable, and I can
> write a test for that first. The problem is that the user can select
> multiple rows in the table, and that causes a bug, because I didn't
> write code to handle that. I can fire up the app, and click around and
> discover lots of bugs that happen because the UI toolkit gives me lots
> of behavior that I wasn't aware of. I have lots of other examples of
> problems, and i'm wondering if other people have had similar
> experiences.
>
> Thanks,
> Eric
| |
| John Roth 2004-05-12, 7:44 pm |
|
"Dan Knierim" <dakcalouro@yahoo.com> wrote in message
news:d5c3eef6.0405091123.452de05d@posting.google.com...
> Yes this type of occurrence is familiar to me. And not only in UI
> development.
>
> It illustrates that XP-style unit testing, while necessary, is usually
> not sufficient testing.
>
> It also illustrates that each feature should be justified by a user
> acceptance test. Extra bells and whistles offered (often as defaults)
> by a tool or library, should be turned off (if possible) unless
> required by the user.
And that really is the reason why Acceptance Testing and
Unit testing are two different animals. Acceptance tests are
to check (and specify in some fairly strong sense) the requirements;
unit tests are to check and specify the design. They have different
tools, different audiences and different purposes.
John Roth
>
>
> cyclocrossing@hotmail.com (cyclocross) wrote in message
news:<7b628169.0405061037.49dbc997@posting.google.com>...[color=darkred]
| |
| Ronald E Jeffries 2004-05-12, 7:44 pm |
| On 9 May 2004 12:23:43 -0700, dakcalouro@yahoo.com (Dan Knierim)
wrote:
>Yes this type of occurrence is familiar to me. And not only in UI
>development.
>
>It illustrates that XP-style unit testing, while necessary, is usually
>not sufficient testing.
I'm interested in this comment, because the description from the OP
doesn't sound like "XP-style unit testing" to me. XP style testing
tests everything, and the tests are written before the code, not
after.
So ... please say a bit more about what you think XP-style unit
testing is, and what else you've found to be needed.
>
>It also illustrates that each feature should be justified by a user
>acceptance test. Extra bells and whistles offered (often as defaults)
>by a tool or library, should be turned off (if possible) unless
>required by the user.
I fully agree. And XP requires acceptance tests. They serve a number
of purposes, including serving as a second level of quality checking.
Regards,
Ron[color=darkred]
>
>
>cyclocrossing@hotmail.com (cyclocross) wrote in message news:<7b628169.0405061037.49dbc997@posting.google.com>...
--
Ron Jeffries
www.XProgramming.com
I'm giving the best advice I have. You get to decide if it's true for you.
| |
| cyclocross 2004-05-12, 7:44 pm |
| dakcalouro@yahoo.com (Dan Knierim) wrote in message news:<d5c3eef6.0405091123.452de05d@posting.google.com>...
> Yes this type of occurrence is familiar to me. And not only in UI
> development.
>
> It illustrates that XP-style unit testing, while necessary, is usually
> not sufficient testing.
>
> It also illustrates that each feature should be justified by a user
> acceptance test. Extra bells and whistles offered (often as defaults)
> by a tool or library, should be turned off (if possible) unless
> required by the user.
>
Yes, turning off all the extra bells and whistles would be great, but
you need to know all the bells and whistles are there, and when they
may show up first in order to turn them off. Saying this is just like
saying that you shouldn't write bugs in the first place, this just
isn't the way it works. A good example of this is focus navigation,
if I want to add UI components to a screen dynamicly, focus navigation
through those new components will probably not work as you would like.
Your going to have to write additional focus code to handle this
situation, but first you have to know this situation exists. This
seems to be the theme of our development, bugs exist mainly because of
code we *didn't* write not code we did.
We could write UI tests to cover all possible combinations of UI
interaction, but this just isn't feasible, the project would never
end.
I'm rapidly coming to the conclusion that TDD on a thick presentation
layer like a complicated swing app just doesn't work. It only seems
to multiply the development time by a factor of 10 with no difference
in the end bug count, or code quality. In fact we have no confidence
to refactor since it only exposes more bugs through combinations of UI
interaction for which there were never any tests written, and we
rarely break tests through refactoring. Testing right up to the
presentation layer though works incredibly well, and i'm completely
sold on it.
[color=darkred]
>
> cyclocrossing@hotmail.com (cyclocross) wrote in message news:<7b628169.0405061037.49dbc997@posting.google.com>...
|
|
|
|
|