Software Engineering - Re: An example of application of Six Sigma to an IT project. Please

Author

Re: An example of application of Six Sigma to an IT project. Please

H. S. Lahman

2005-09-08, 6:58 pm

Responding to NicoleB...

I believe the 6-Sigma the OP was referring to was the process
improvement framework with that name. In that sense 6-Sigma is a
methodology like TQM, Baldridge, et al rather than a metric. But, as
long as we are here... B-)

> The use of 6-Sigma has always been a chore when discussing software.
> 6-Sigma was developed to measure manufactured goods quality -
> opportunities for error vs. errors. In this, it has also had its
> failings in defining the absolute number of opportunities for failure.
> Software greatly complicates this.

The assumption of the Sigma metric is that each line of code is an
opportunity for inserting a defect. One could use other criteria, such
as each expression, but LOC (3GL or Assembly) is convenient and
reasonably consistent across languages. While this assumption is not
perfect, it is good enough for process monitoring.

The point where the shop's defect rate lies on the Sigma curve relative
to other applications the shop has done is a quite reasonable way to
measure process effectiveness vai SPC. That is, for monitoring one
doesn't really care about the absolute value; one only cares about the
value relative to other values. So the way Sigma is measured is not
very important so long as it is done consistently. [Though it can be
important if one, say, uses single-point comparisons between development
environments. But then one has a different goal than process improvement.]

>
> Example, in IT, a combination of events can cause a failure. Is each
> even then considered as an opportunity, or is any possible combination
> of events considered an opportunity, or is it the combination of both
> of these, and how the heck would you count them?

I don't think this issue matters much. Whatever combination of events
results in the error, that only happens if the program is incorrectly
coded to deal with that combination (by commission or omission).
Wherever the correct code to deal with that combination should be is
where the opportunity for inserting a defect arose from the Sigma
perspective. So the linking is really between code opportunity and the
/combination/ of events. This segues to...

> If you need to measure 6-Sigma (because someone told you that you have
> to, and you can not argue the value of it), then go ahead and normalize
> the size of the software to the operational instructions, and use that
> as a basis for measuring the failures that occur. However, let us all
> remember that software that runs once and fails is a whole lot
> different from software that runs 12,000 a day and fails once. The
> opportunity for failure would measure the same using 6-Sigma, and could
> make both software systems look the same in the 6-Sigma sense.

While I agree that the differences between your examples are important,
I think that importance is for a different context than process
monitoring using Sigma metrics. If the program only fails once in 12K
executions, all that means is that the execution context is such that
the offending line is rarely executed and/or the offending combinatorial
context rarely occurs. The opportunity for the defect and the defect
itself exist in the code even when the circumstances for manifesting it
do not exist in the execution. So manifesting defects is an orthogonal
issue to Sigma measurement.

But I agree with the last point that using Sigma metrics for released
defects needs to be measured in a predefined time box. Similarly,
certain types of load testing also need to be normalized to a predefined
time box. That's because the opportunities for inserting defects in the
source code are different than the opportunities for manifesting those
defects. So whenever manifesting defects is important, the metric must
take that into consideration.

*************
There is nothing wrong with me that could
not be cured by a capful of Drano.

H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
(888)OOA-PATH

H. S. Lahman

2005-09-09, 6:59 pm

Responding to Manager...

> Have any of you used six sigma for software projects, either
> development or maintenance?

Alas, no. We were a TQM shop, not Six Sigma. So all I know about it is
the Executive Summary. OTOH, process improvement frameworks are all
pretty much the same so techniques should be transferable. Do you have
anything specific in mind?

*************
There is nothing wrong with me that could
not be cured by a capful of Drano.

H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
(888)OOA-PATH

Sponsored Links