Software Testing - Collaboration between dev and testing

Author

Collaboration between dev and testing

khashraf@gmail.com

2006-04-04, 8:05 am

Collaboration between Dev team and testing team is important but risky.
Developers might want to throw their burden of unit testing to testers
which would increase software costs much more. Also, the syndrome of
"why didnt you find that defect before" is the worse nightmare of a
tester and puts added pressure on the tester (who can actually perform
much better with a free mind).

I am of the opinion that there should be separate dev and test teams
with strong communication but distinct responsibilities. Configuration
Management can play a good role here in "receiving" the code from dev,
and "delivering" it to testing.

adam

2006-04-04, 10:04 pm

Unit testing is entirely the responsibility of the developer. A tester
is not in as knowledgable a position to write meaningful unit tests as
the developer who did the feature. I would push back hard against a
development team that tried to do this. A feature is not 'done' (as in
'ready for test') until
- the requirements are met (be they specifications or use case)
- the code has all be checked into revision control
- it has been verified that the newly checked in code does not break
the compile/existing tests
- a comprehensive suite of feature specific unit tests has been created
and integrated into the build process
- there are no TODO's (or similar watermarks) left in the new code
- all FIXME's (or similar watermarks) have bug numbers assigned to them
in the new code

-adam
http://www.ninjatactics.com/blog

Phlip

2006-04-04, 10:04 pm

khashraf wrote:

> Collaboration between Dev team and testing team is important but risky.

Then collocate them, and have them work on the same features in the same
iteration.

> Developers might want to throw their burden of unit testing to testers
> which would increase software costs much more.

When developers do unit tests the right way, they can avoid endless
debugging. That makes them faster. Testers' roles here are generally to make
sure unit tests are easy to write. Testers maintain the test rig.

> Also, the syndrome of
> "why didnt you find that defect before" is the worse nightmare of a
> tester and puts added pressure on the tester (who can actually perform
> much better with a free mind).

Why? Who would ask such a question? If everyone writes tests (and if
everyone is generally aware of each other's features), then the team finds
defects.

> I am of the opinion that there should be separate dev and test teams
> with strong communication but distinct responsibilities. Configuration
> Management can play a good role here in "receiving" the code from dev,
> and "delivering" it to testing.

That is a common myth of testing. It adds a lot of friction and waste.

--
Phlip
[url]http://www.greencheese.org/Z

Land[/url] <-- NOT a blog!!!

khashraf@gmail.com

2006-04-04, 10:04 pm

@phlip,

Is it so that XP is much more in your veins :)

The syndrome comes from the psyche that testers do nothing except
finding defects... and this psyche is maintained by no one other than
developers. We discussed a lot in these forums about changing that
psyche but it exists and will continue so. This psyche flourishes when
tester is dependent to the approach of the developer (sometimes dev
manager) (right now only this fix is available so please test it before
we give you more to test)... no one knows what the developer has done
till a tester tests the build... and if the build is "shaky", egos are
at stake. The solution is ofcourse to understand and respect each
others capabilities but that is missing, in most cases.

In physically teaming up the dev and testing team, I have seen much
more iterations (and time) and much more chaos than when they perform
separately. Also, if the product fails in market, the blame is more on
testers, but if it succeeds, the dev takes the cake. By separately I
would not mean that code is built over then tested, but testing goes in
parallel though in its own planned fashion (a separate team with a
separate plan under a test lead). When collaborating the dev + testing,
if communication processes and responsibilities are well defined,
violaters are easily identified and corrected.

If processes are followed to ensure what @adam said, that is more than
acceptable.

Jose Cornado

2006-04-04, 10:04 pm

IMO, I think the problem should be restated. Defining clear
responsibilities is nice but it just helps you where to point the
finger in case all helll break loose. It's reactive rather than
proactive.

This is a strict economics/productivity problem: increasing the quality
of software while decreasing the costs of development. The technology
comes afterwards not the other way around.

A lot of developers do not have enough time to write even their own
code let alone write more code for the unit tests.

For instance a couple of months ago in a open-source project were
debating developing tests for a feature. They were thinking about using
Nunit. Somebody mentioned that refactoring of the feature was pretty
close in the horizon so Nunit was useless because some methods were
likely to change their protection level. Net result, a lot of
development was done without testing.

Around the same time, they had to used two months to just testing. Does
not strike you that as inefficient?

NOTE: far be it from me to critize that project. I am just stating
facts.

A lot of QA departments do not have the political weight to push back
much.

So adding people does not really provide a solution.

The latest MS embarrasement is an example. They were pretty close or
over the 1:1 dev/QA ratio (I read that in another forum).

What was the outcome? Is it any better teaming that that? We all like
to criticize MS but not all there are dumb, so some credit is due.

If they have not managed to make this work with the vast resources
they've got, 1:1 and endless budgets, isn't it time to re-think the
problem?

So until appropiate tools address the economic problem and the real
costs, there will be a lot of wheel-spinning.

Corey Shuman

2006-04-04, 10:04 pm

--"IMO, I think the problem should be restated. Defining clear
responsibilities is nice but it just helps you where to point the
finger in case all helll break loose. It's reactive rather than
proactive."--

If you have not defined responsibilities then you cannot pinpoint where
the problem is to address it... it would seem a lot more proactive to
have responsibilities in place. Standing around and hugging eachother
when a problem arises doesnt accomplish anything...

--"A lot of developers do not have enough time to write even their own
code let alone write more code for the unit tests. "--

If your dev team cant even write their own code then I would suggest
that the problem is a lack of resources. Every shop Ive worked in has
had developers who write their own code... that logic is like saying
you have testers who dont have time to test..

--"Around the same time, they had to used two months to just testing.
Does
not strike you that as inefficient? "--

If your test team found no defects and there was no trending to show an
increase in product quality then yes, it was inefficient and you should
restructure your QA department. Other than that, a 2 month cycle is not
uncommon, at least in the planning stages... ;)

--"A lot of QA departments do not have the political weight to push
back
much.--"

QAs job is not to push back.. its to report the state of the product,
or the lack of time to test to report the state of the product. Dont
make it a battle, report and move on...

--"The latest MS embarrasement is an example. They were pretty close or

over the 1:1 dev/QA ratio (I read that in another forum). "--

Whats the latest MS embarrasment? IE7, Vista?? If thats what you are
talking about there is no embarrassment, its easy to point the finger
and say "haha!! those idiots..." and bash away but the simple fact is
this, there are very few people who have ever attempted to create, let
alone test a product the size of Vista or IE, its huge and the amount
of features that do work and work well are a testament to the massive
efforts put in. If you think you could do a lot better you are showing
a serious lack of knowledge of a large scale developement.

--"If they have not managed to make this work with the vast resources
they've got, 1:1 and endless budgets, isn't it time to re-think the
problem? --"

Newsflash.. they have made this work... what were you expecting, a
flawless application?? An entire OS devoid of any bugs when every
application all over the internet is expected to plug into it?? Again,
the fact that it works as well as it does, and in such a short amount
of time is a testament to what they are doing right...

QA isnt rocket science, its not tools, and its not re-inventing the
wheel, it really just comes down to common sense.

All this of course, as always... IMHO...

--Corey Shuman

Phlip

2006-04-04, 10:04 pm

Corey Shuman wrote:

> If you have not defined responsibilities then you cannot pinpoint where
> the problem is to address it... it would seem a lot more proactive to
> have responsibilities in place. Standing around and hugging eachother
> when a problem arises doesnt accomplish anything...

Set this goal: "We will spot errors as early as possible in the code, and as
we fix them we will investigate why they happened, to help prevent them."

If the error was my fault, I want to know about it as soon as possible,
because I'm now in the best position to help prevent the error.

When a team enforces roles, so each role-player can only generate a few
kinds of tell-tale errors, this system delays error discovery, and impairs
the investigation. Hearing about an error verbally, from a team member
working with me in the programming pit, is a lot different than being called
before some dumb committee of people I _don't_ work with, a w

later.

--
Phlip
[url]http://www.greencheese.org/Z

Land[/url] <-- NOT a blog!!!

Jose Cornado

2006-04-04, 10:04 pm

I am sorry I hurt your feelings, but:

You said hugging each other, not me.

I explicitly stated that to be where MS is, it deserves credit. That
credit depends on you taste. In fact, we are agreeing.

MS Vista is going to miss the Xmas season. Talk about making things
work for the VP who "resigned", the PC makers who were counting on the
product to be ready, etc.

That is 3-4 years after they announced that they were going to change
their testing technology which they were to make available en the
future releases of VS.

So after all the money and brain power they spent, the result is
another missed deadline.

But if you think that after:

*) all the investments (money, or otherwise),
*) having a 1:1 (when what they need is better utilization of their
hardware)
*) promising that they will never miss a deadline again to their
distrubution channel
*) and a stagnant stock price. (If I were an financial analyst I would
think: they bloated their headcount, and where is the better product
coming faster to market?)

is getting right, you are entitled to your opinion. Like everyone else.

But using your argument: if it is common sense, why does it take them
so long to get it?

NOTE: MS deserves a lot of respect. I was tryig to offer an alternative
point of view. I apologize if If I hurt your feelings.

Ruthie

2006-04-05, 7:08 pm

I was just going to say that Jose's first post describes the last 2 QA
environments I have worked in. QA is the lowest priority. I do not like
it but it is the fact.

Regarding the OP by Khash..., I have worked where QA was both under the
development manager for years, then moved to be under a Director of QA.
When the teams are separated, QA is given more priority. When QA and
dev are working together under one manager, QA is given low priority.

This is just my experience (9 yr QA, 4 of those years as SQA)

Jose Cornado

2006-04-05, 7:08 pm

Nokia just did the same thing as MS: because of problems with software
quality, they had to delay some new gadgetry.

The cascading effect of lost reveneues,

business partners, and
disappointed customers continues.

I am guessing that the money left on the table because of bad quality
and its trickle down effect for the whole software industry would make
Ford and Deming twist and turn in their graves for a long time.

If one opens the Mythical Man Month on page 20, one will run into the
following data hidden between the lines: 25 years ago 50% of a software
project was about testing and debugging. In 2004 the same stat was 80%
(according to NIST, A Better Bug Trap).

As an industry, software has become less eficient over the last 25
years. This does not make sense: financial or otherwise.

The MS and nokia delays just reinforce those figures.

Let's imagine that each time that MS issues a security warning, all
Dell customers ask for their money back because the product is not
safe. Or at least the OS portion of the price paid.

Like the head of SAP said: "software companies have to change. We need
to be like the auto industry: quicker and better at development".

Sponsored Links