Home > Archive > Software Engineering > November 2005 > Reliability Requirements?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Reliability Requirements?
|
|
| kk_oop@yahoo.com 2005-11-07, 10:00 pm |
| Hi. I'm looking for a definition of the type of non-functional
requirements that fall under the area of "reliability." Can someone
point me to a URL that defines this (an online article or website would
be great!). Feel free to provide your own definition, too.
Thanks much!
Ken
| |
| H. S. Lahman 2005-11-07, 10:00 pm |
| Responding to Kk_oop...
> Hi. I'm looking for a definition of the type of non-functional
> requirements that fall under the area of "reliability." Can someone
> point me to a URL that defines this (an online article or website would
> be great!). Feel free to provide your own definition, too.
The first place I would look are the standards organization web sites
(IEEE, ASQE, etc.). For example IEEE has a standard for requirements
specification that includes definitions of things like "reliability".
The simplistic definition is that an application is reliable if it
manifests no defects during its execution. So an application can be
reliable even if it has defects but it is never used in a context where
those defects are manifested. Furthermore, as a practical matter
reliability is usually defined in terms of time and a probabilistic
definition of whether defects will be manifested. That's because no
matter how good the construction process is, there is some probability
of failure if one waits long enough. So a common measure is Mean Time
To Failure (MTTF). (Sadly, for software one usually doesn't have to
wait very long.)
*************
There is nothing wrong with me that could
not be cured by a capful of Drano.
H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
(888)OOA-PATH
| |
|
|
| Penny Liang 2005-11-07, 10:00 pm |
| the quality properties,like reliability ,sometimes have not a precise
defination. the MTTF is a standard ,but differently applied in different
projects and different companies.
the most important thing ,i think,is history data which could give a
relatively precise reference about what MTTF is OK.
"H. S. Lahman" <h.lahman@verizon.net> ????
news:DTqbf.8916$An6.2065@trnddc08...
> Responding to Kk_oop...
>
>
> The first place I would look are the standards organization web sites
> (IEEE, ASQE, etc.). For example IEEE has a standard for requirements
> specification that includes definitions of things like "reliability".
>
> The simplistic definition is that an application is reliable if it
> manifests no defects during its execution. So an application can be
> reliable even if it has defects but it is never used in a context where
> those defects are manifested. Furthermore, as a practical matter
> reliability is usually defined in terms of time and a probabilistic
> definition of whether defects will be manifested. That's because no
> matter how good the construction process is, there is some probability
> of failure if one waits long enough. So a common measure is Mean Time
> To Failure (MTTF). (Sadly, for software one usually doesn't have to
> wait very long.)
>
>
> *************
> There is nothing wrong with me that could
> not be cured by a capful of Drano.
>
> H. S. Lahman
> hsl@pathfindermda.com
> Pathfinder Solutions -- Put MDA to Work
> http://www.pathfindermda.com
> blog: http://pathfinderpeople.blogs.com/hslahman
> (888)OOA-PATH
>
>
>
| |
| H. S. Lahman 2005-11-07, 10:00 pm |
| Responding to Liang...
> the quality properties,like reliability ,sometimes have not a precise
> defination. the MTTF is a standard ,but differently applied in different
> projects and different companies.
>
> the most important thing ,i think,is history data which could give a
> relatively precise reference about what MTTF is OK.
Quite true. Reliability is about defects, so one needs a definition of
'defect' before one can even start measuring. B-) Simply counting
defects is not very useful without some way to identify context, so most
metrics concentrate on defect rates, such as MTTF. But that still
leaves a lot of latitude for the time dimension (when one starts
counting; how long one counts). In addition, there are other context
dimensions, such as code size and type of software. So the utility of
such metrics for comparing across shops is limited. However, so long as
a metric like MTTF is constructed consistently within a shop, it
produces valuable relative measurements that can be used with high
confidence for SPC.
I agree that, in principle, if one has the raw data one can construct
the metric any way one wants so that it is consistent across shops. In
practice, that can depend a lot on how much peripheral context
information is attached to the raw data. The large project databases,
such as those maintained by Capers Jones or Howard Rubin, typically have
to make assumptions about the data because the sources do not provide
enough peripheral context information.
*************
There is nothing wrong with me that could
not be cured by a capful of Drano.
H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
(888)OOA-PATH
| |
| H. S. Lahman 2005-11-07, 10:00 pm |
| Responding to Lidor...
> Well, I really don't have a better definition than the one supplied by
> H. S. Lahman. The final sentence, however, reminded me of a short
> reflection I've just wrote a couple of w s ago:
>
> http://www.qualityprogramming.org/M...eDisclaimer.htm
>
> Not much of a definition, but something to think about....
I basically agree with most of your sentiments. I happen to think that
the software industry is on the verge of the same paradigm shift that
hit Western manufacturing in the '80s -- the shift from "testing in"
reliability after construction to preventing defect insertion in the
first place. IOW, a sea change in development processes is required
where testing becomes just a process monitoring tool.
In the '80s the PacRim beat the crap out of Western manfacturers because
the consumers became aware that the PacRim was producing better products
cheaper pretty much across the board (albeit with much smaller market).
It took the PacRim nearly three decades to launch that revolution and
overcome the perception of shoddy quality from the '40s. But eventually
consumers noticed and the Western manufacturers had to desperately play
catch-up on the process front for nearly a decade while they lost gobs
of market share.
Today software customers have noticed that the only things that break
frequently in their lives are made of software. And some shops have
already adopted modern process quality techniques. As soon as those
shops reach a critical mass for comparision, reliability will become a
major competitive advantage in the markeplace, just as it was for the
PacRim in the '80s. At that point a lot of shops are going to have to
play catch-up and it will be very painful because the entire development
culture of the shop needs to change and that doesn't happen overnight.
*************
There is nothing wrong with me that could
not be cured by a capful of Drano.
H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
(888)OOA-PATH
| |
| Ron Ruble 2005-11-09, 6:59 pm |
| H. S. Lahman wrote:
<snip>
> Reliability is about defects,
Have to take exception: my dictionary defines reliability [1]
as "Capable of being relied on; dependable", which isn't about
defects but performance.
Defects are a core concern of reliability engineering, but
reliability is much more than being defect free. I can build,
for instance, a defect free oil pipeline, which is completely
unreliable due to weather conditions, or war, or politics.
> so one needs a definition of
> 'defect' before one can even start measuring. B-) Simply counting
> defects is not very useful without some way to identify context, so most
> metrics concentrate on defect rates, such as MTTF.
MTTF can take into account external factors affecting
reliability.
Good information, I'm just nit-picking.
| |
| Ron Ruble 2005-11-09, 6:59 pm |
| H. S. Lahman wrote:
<snip>
> I happen to think that the software industry is on the
> verge of the same paradigm shift that hit Western manufacturing
> in the '80s -- the shift from "testing in" reliability after
> construction to preventing defect insertion in the first place.
God, I hope so!
;>
| |
| H. S. Lahman 2005-11-09, 6:59 pm |
| Responding to Ruble...
>
>
> Have to take exception: my dictionary defines reliability [1]
> as "Capable of being relied on; dependable", which isn't about
> defects but performance.
As I said, one has to have a definition of 'defect' to talk about
reliability. B-) The literature generally distinguishes between
'failure', 'fault', and 'defect'. If a system is not dependable, that
is manifested in failures. Failures, in turn, are traceable to faults,
which identify the condition or reason for failure. Defects, in turn,
represent a feature of the system that allowed the condition to prevail
(via commission or omission). So...
> Defects are a core concern of reliability engineering, but
> reliability is much more than being defect free. I can build,
> for instance, a defect free oil pipeline, which is completely
> unreliable due to weather conditions, or war, or politics.
If the pipeline is unreliable due to, say, freezing up (failure mode)
because it was not insulated properly for the prevailing weather
conditions (fault), one has a defect somewhere. That defect may be
improper insulation (a defect in implementation due to graft or
whatever) or it may be indirectly due to poorly formed requirements that
did not properly provide for the prevailing weather conditions. But the
defect is in there somewhere. B-)
However, I agree that reliability almost always implies some notion of
acceptable failure rate at the requirements level. So we build
pipelines for the "two hundred year winter". So long as the "three
hundred year winter" doesn't show up during the pipeline life, there is
no failure and no fault. The interesting question is: if the "three
hundred year winter" does show up and the pipeline fails, was there a
defect?
That depends on which side of the 'acceptable' one is standing. The
pipeline was built to spec so there is no inherent defect in its
construction. There is no requirements defect in the sense that the
pipeline specification deliberately chose the "two hundred year"
standard of 'acceptable'. Nonetheless, the reality is that there was a
failure in operation -- regardless of acceptability -- so I argue there
must <technically> be an underlying fault and corresponding defect. So
the liability lawyers are going to go after whoever defined the standard
of reliability. IOW, the defect may be in the way 'reliability' itself
was defined. Catch-22.
Needless to say, I lived in a world (automated test equipment) where one
had a broad definition of 'defect'.
>
>
> MTTF can take into account external factors affecting
> reliability.
Sure. Just before I retired, we got an award from the USN for 600+
systems over close to a decade because the system MTTF was 2.7 years.
However, as software people we felt that was very unfair to our MLOC of
software because virtually all of the actual failures were hardware that
we had no control over! (The systems had 48 D-size cards, each with
several hundred ICs, so the hardware guys were trashed by time and sheer
combinatorial size.)
*************
There is nothing wrong with me that could
not be cured by a capful of Drano.
H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
(888)OOA-PATH
| |
| Ron Ruble 2005-11-10, 7:57 am |
| H. S. Lahman wrote:
<snip>
> ...Nonetheless, the reality is that there was a
> failure in operation -- regardless of acceptability
> -- so I argue there must <technically> be an underlying
> fault and corresponding defect. So the liability lawyers
> are going to go after whoever defined the standard
> of reliability. IOW, the defect may be in the way
> 'reliability' itself was defined. Catch-22.
Certainly true in some cases. Applying this to all cases presumes that
we don't build anything to serve in an environment we can't completely
understand or control.
I would argue that some situations are at the edges of our knowledge,
and building to our best guess is not a specification defect, but rather
a (sometimes necessary) compromise between our need to achieve and our
limited abilities.
| |
| H. S. Lahman 2005-11-10, 6:59 pm |
| Responding to Ruble...
>
> Certainly true in some cases. Applying this to all cases presumes that
> we don't build anything to serve in an environment we can't completely
> understand or control.
Not necessarily; just because a potential defect exists doesn't mean we
have to worry (or even know) about it. B-)
*************
There is nothing wrong with me that could
not be cured by a capful of Drano.
H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
(888)OOA-PATH
|
|
|
|
|