Home > Archive > Software Testing > April 2006 > Regarding Software Testing Books
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Regarding Software Testing Books
|
|
| Trupti 2006-04-06, 8:06 am |
| hi
i am a fresher in this field. i would like to know which book is
recommended for software testing manual and automated as i m aware of
all the terms like test plan, test case etc. but in actual how do u
prepare them and how does it work for automation and manual testing. it
would be a great help.
| |
| xtremetester 2006-04-06, 8:06 am |
| Read books review at http://www.geocities.com/xtremetesting/
And you can find some useful information for software testers like
testing dictionary, online quiz etc.
regards,
Trupti wrote:
> hi
> i am a fresher in this field. i would like to know which book is
> recommended for software testing manual and automated as i m aware of
> all the terms like test plan, test case etc. but in actual how do u
> prepare them and how does it work for automation and manual testing. it
> would be a great help.
| |
| James Bond 007 2006-04-06, 7:08 pm |
|
"Trupti" <truptipatel77@googlemail.com> wrote in message =
news:1144324149.022496.160430@z34g2000cwc.googlegroups.com...
> hi
> i am a fresher in this field. i would like to know which book is
> recommended for software testing manual and automated as i m aware of
> all the terms like test plan, test case etc. but in actual how do u
> prepare them and how does it work for automation and manual testing. =
it
> would be a great help.
>
I have the following books, which I recommend:
Software Testing - A Craftsman's Approach
Software Testing In The Real World
The Art Of Software Testing
Surviving The Top Ten Challenges Of Software Testing
I also recommend the following web sites:
http://www.qaforums.com/
http://www.stickyminds.com/
http://www.whatistesting.com/
http://www.qatraining.net/
| |
| Michael Bolton 2006-04-07, 7:07 pm |
| I would strongly recommend "Lessons Learned in Software Testing"
(Kaner, Bach, and Pettichord), and, if you're a fresher, "Testing
Computer Software" (Kaner, Falk, and Nguyen).
I would not recommend many of the techniques in "The Art of Software
Testing", such as cause-and-effect diagrams--they may have been more
appropriate almost 30 years ago, but software is more elaborate and
complicated than that now. I would recommend a key point of Myers'
philosophy: that we cannot take as a given that the software is
working; if we want to be effective as testers, we must work to
demonstrate that the software is broken.
I've own all four books recommended by 007, and except for The Art of
Software Testing, I don't find any of them particularly memorable.
There's nothing bad about reading them, per se, but I think their
relevance is greatly diminished if your development organization isn't
playing by the rules that these books assume--and most development
organizations don't.
---Michael B.
| |
| Steve Devonport 2006-04-10, 8:20 am |
| In reply to Michael Bolton:
Cause & Effect graphing, state transition testing - are all very
relevenat today as they were 30+ years ago. Software still requires
input (cause) to perform an action and produce a result (effect)! In
fact most of todays software is written to be event driven, eg web
pages. Without any input(cause) a web page produces nothing(effect)! Or
to put it another way, changing from inactive (starting state) to
active (state 1).
I think - and this is my own opinion - that you are incorrect to say
that these techniques are not appropriate in today's world. They are
needed more than ever. These techniques (along with other black box
techniques) provide an early view on what can/should be tested before a
single line of code has been written/delivered to the test team.
This is my professional opinion and should be taken as such.
| |
|
| Book: Effective method for software testing"
Author: William E Perry
Pub: Wiley Pubs.
Cost: Rs 380/-
Note: Best book I've come across on Software Testing....Will surely
help you clarifjy most your doubts!!
Nupul
| |
| Michael Bolton 2006-04-10, 8:20 am |
| >I think - and this is my own opinion - that you are incorrect to say
that these techniques are not appropriate in today's world. They are
needed more than ever. These techniques (along with other black box
techniques) provide an early view on what can/should be tested before a
single line of code has been written/delivered to the test team.
>This is my professional opinion and should be taken as such.
First, I didn't say that cause and effect diagrams were inappropriate;
I said that I didn't recommend them. I didn't say that they were
inappropriate; I said that they were more appropriate 30 years ago. I
didn't suggest that the concepts of cause and effect weren't useful; I
implied that drawing diagrams wasn't necessarily the most effective use
of a tester's time.
But you contend that cause and effect diagrams are needed more than
ever. Do you use them constantly? Occasionally? Do you use them /as
they are shown in Myers/? If so, please tell us about the ways in
which you use them, the bugs they've helped you to find, the amount of
time you spend on them, and so on. Success stories are great, and will
help us all. Really.
If you don't use them, why don't you use them, and why do you tout them
if you don't use them?
---Michael B.
| |
| S Perryman 2006-04-10, 8:20 am |
| "Michael Bolton" <google@michaelbolton.net> wrote in message
news:1144657543.455781.204220@e56g2000cwe.googlegroups.com...
> First, I didn't say that cause and effect diagrams were inappropriate;
> I said that I didn't recommend them. I didn't say that they were
> inappropriate; I said that they were more appropriate 30 years ago. I
> didn't suggest that the concepts of cause and effect weren't useful; I
> implied that drawing diagrams wasn't necessarily the most effective use
> of a tester's time.
> But you contend that cause and effect diagrams are needed more than
> ever. Do you use them constantly? Occasionally? Do you use them /as
> they are shown in Myers/? If so, please tell us about the ways in
> which you use them, the bugs they've helped you to find, the amount of
> time you spend on them, and so on. Success stories are great, and will
> help us all. Really.
> If you don't use them, why don't you use them, and why do you tout them
> if you don't use them?
I will second that.
I have been considering cause/effect diagrams as a potential means of
increasing the number of tests cases per line of test infrastructure (ie
test
harness code) .
My development process already uses specification-directed testing + the
'classic' approaches (equivalence class partitioning, orthogonal array tests
etc) , which have (IMHO) very ROI. If cause/effect diagrams generate the
same test cases as the other approaches but faster, or quickly identify
tests
that the other approaches don't easily produce, I would be very interested
to hear.
Regards,
Steven Perryman
| |
| Steve Devonport 2006-04-10, 8:20 am |
| OK here goes, sorry in advance for the long reply.
Background: I currently work within the (UK) Credit Card arena and have
done for the last 5 years. I have tested many different processing
systems, based on various technologies, eg mainframe, UNIX.
During these years I have identified/established a (detailed) list of
"common scenarios" that relate to those processing systems that I have
observed. In effect I have developed a list of what I should be
concentrating on within a given function/system. I use this "list" when
I'm presented with a system wide change or the introduction of a new
(sub) system.
Coupled with the list, I know - with a high degree of certainty - that
for a pre-defined set of inputs (cause) a predicted result will
occur(effect). I do not have this as a fully documented diagram, but
rather I have a fairly detailed spreadsheet of probable inputs that I
know each processing system must be capable of handling, eg message
formats/structure/specific field contents.
To answer your queries: I do use cause/effect graphing constantly
during my analysis phase - they allow me to quickly put together a
plan, a strategy, whatever you want to call it, without me having any
code delivered.
Another comment was "...the bugs they've helped you to find..." - One
of my previous clients had implemented a new authorisation system
(developed overseas - EU) and during the analysis phase I decided to
"implement" a series of exception tests that are particular to VISA. I
queried the development team about these tests. There comments were
simply, "...are you sure theses are valid tests, as we know nothing of
these exceptions" - I provided the official VISA manuals as proof. If
these "exceptions" had not been developed the consequences could have
had a financial impact on cardholders and the clients settlement
process.
I don't use cause/effect graphing in isolation - I use it as a guiding
factor, every client I'm involved with has a "different" system - but
all these "different" systems have to accept the same input, ie
structure, format, otherwise they can't trade with MasterCard, VISA,
AMEX.
I also think that in other industries that something similar must be
true, every client/system is different, that's a given - but the inputs
are basically the same and therefore the results will be basically the
same - how the inputs/results are processed will be different, but a
date of birth field is still a date of birth field no matter where it
is represented (web site/mainframe) and the technology that will
process it. The result will either be valid or invalid - according to
some specification - and cause/effect graphing can allow us testers to
produce a series of efficient tests without having a single line of
code delivered to the test team.
I hope I have answered your queries and I hope my comments are clear.
I apologise if I mis-interpreted your comments, that was not my
intention.
| |
| S Perryman 2006-04-10, 8:20 am |
| "Steve Devonport" <s.devonport@zeda.co.uk> wrote in message
news:1144666824.131973.230870@g10g2000cwb.googlegroups.com...
> OK here goes, sorry in advance for the long reply.
No, thank you for it.
[ snip, but acknowledged ]
> I don't use cause/effect graphing in isolation - I use it as a guiding
> factor, every client I'm involved with has a "different" system - but
> all these "different" systems have to accept the same input, ie
> structure, format, otherwise they can't trade with MasterCard, VISA,
> AMEX.
> I also think that in other industries that something similar must be
> true, every client/system is different, that's a given - but the inputs
> are basically the same and therefore the results will be basically the
> same - how the inputs/results are processed will be different, but a
> date of birth field is still a date of birth field no matter where it
> is represented (web site/mainframe) and the technology that will
> process it. The result will either be valid or invalid - according to
> some specification - and cause/effect graphing can allow us testers to
> produce a series of efficient tests without having a single line of
> code delivered to the test team.
> I hope I have answered your queries and I hope my comments are clear.
I have some further queries which you may have answers for :
1. Do you use methods such as equivalence partitioning, boundary value
analysis etc ??
2. If so, compared to the other methods how much time (more, less,
the same etc) do you believe that you spend on cause/effect graphs to find a
certain amount of test cases ??
3. What would you say the degree of overlap is between cause/effect graphs
and the other methods (high overlap, complementary etc) ??
Regards,
Steven Perryman
| |
| Michael Bolton 2006-04-11, 7:08 pm |
| >During these years I have identified/established a (detailed) list of "common scenarios" that relate to those processing systems that I have observed. In effect I have developed a list of what I should be concentrating on within a given function/system.
I use this "list" when I'm presented with a system wide change or the introduction of a new (sub) system.
Depending on the focus, I might call your list a bug taxonomy or a risk
catalog. I presume that, for each project, you use the list as the
basis for a new list, with product-focused ideas being added for the
context. I think these things are powerful. They're not really
covered in Meyers. The broadest one I've seen is in Testing Computer
Software. On www.testingeducation.org, there's a taxonomy for bugs in
your shopping cart. Michael Hunter, The Braidy Tester, has a really
good risk catalog (test idea catalog, effectively) on his Web site.
Then you describe decision tables, which (like you, apparently) I've
found to be more generally useful than cause-effect diagrams (CEDs).
Do your cause-effect diagrams look like the ones in Myers, or are they
more state-diagrammish? If the latter, have you checked out the (free)
CMap tools (http://cmap.ihmc.us/)--you might find them useful.
There's one place where I'd want to be careful, though.
>I also think that in other industries that something similar must be
true, every client/system is different, that's a given - but the inputs
are basically the same ...
You're describing a kind of general systems view of software
development and testing, where there are indeed general similarities
between systems, and general differences too. Cool. But I start to
disagree here:
>and therefore the results will be basically the
same - how the inputs/results are processed will be different, but a
date of birth field is still a date of birth field no matter where it
is represented (web site/mainframe) and the technology that will
process it.
I'd say it is and it isn't. For instance:
- the field could be displayed in a variety of orders and formats (e.g.
13/11/06, 13/11/2006, 2006.11.13, 13-Nov-2006, November 13, 2006, and
so forth)
- the field could be editable, but input restricted such that
validation is simple
- the field could be editable, but input could be free-form such that
validation is complex
- the field could incorporate time, which multiplies the complexity
- the field could be linked with another field, such that there are
dependencies between them (one date must be greater than the next, for
instance)
- the field could be used as a basis for the calculation of another
field (for example, the age field could be calculated based on the date
of birth field)
- the field could be input from or output to some device
- the field could be input or output from some other program that uses
a common date representation
- the field could be input or output from some other program that uses
a different kind of date representation
(insert your other suggestions here)
....but then we start to agree again...
> The result will either be valid or invalid - according to
some specification and cause/effect graphing can allow us testers to
produce a series of efficient tests without having a single line of
code delivered to the test team.
The key words for me here are "according to some specification"; I
would say according to some heuristic, since that incorporates written
specifications, domain experience, test experiences, conversations with
others in the project community, etc. I think that more recent books
than Myers (_Testing Computer Software_, Eric Evans' _Domain Driven
Design_) provide more expansive approaches towards testing; and for me,
other approaches are quicker than CEDs in most contexts that I can
think of.
Thanks for your detailed reply. Far from being asked to apologise for
it, I think you should be commended for it.
---Michael B.
| |
| Steve Devonport 2006-04-12, 4:06 am |
| Steven,
Here goes.
1. I do use EqP and BVP, along with state transition & whichever
analysis technique seems appropriate (this can depend on the culture,
environment, product, customer, etc). I have to admit I only use these
techniques "in anger" when I'm asked to look at " significantly changed
or new" functionality. (Remember I have a pre-determined list of
scenarios that I can "fall back" on to allow me to make an informed
decision about the presence of bugs). The earlier you use these
techniques the more investment you get from them - for this read the
cheaper it is to identify and fix potential bugs.
2. Generally (and I do mean generally) I spend less time on black box
analysis, because I'm ususally "asked" to prove that a card processing
system can perform a certain set of functions. For me. this is usually
carried out at the system integration level and will also involve a
high degree of "certification" and end-to-end testing. So, therefore
I'm not usually asked to perform system(functional) testing. This does
not imply that the closer you get to acceptance testing the less need
there is for balck box techniques. you could argue the other way quite
easily and I'd be inclined to agree.
3. This may sound weird but I prefer some overlap, it is my "fail safe"
that when I'm performing analysis using different techniques that I'm
achieving the same end point, i.e. a list of test cases that satisfy
the objective of the test phase. So to answer your query, for me I aim
for round about 10-20% overlap. But for my team I explain that
different (black box) techniques are to be used and that the "output"
of these techniques should be shared and compared.
| |
| Steve Devonport 2006-04-12, 4:06 am |
| Michael,
Thanks for the objective view.
The link to cmap looks something that interests me greatly. Although I
don't use the "words" on the diagram I certainly do my CED's in this
manner. I believe this answers your question? BTW: I shall be
downloading the toolkit(?) (I would actually call this a process) both
at home and work. The supporting documentation looks heavy going - but
I shall be honest.
As for my DOB field example, I think we both agree on the same
principle - but we are looking at it from different angles.
I have not read the books you mention - but I agree with you, more
"modern" books will contain more "fashionable" ideas than Myers could
ever hope to. But for me it's the fact that the core concepts presented
by Myers still hold true today as they did over 30 years ago.
2 more things:
1. I'm glad to have had the opportunity to converse with someone who
understands that testing is not merely a series of predefined steps to
reach a predefined objective :-)
2. A call for help. I'm starting to look into what's known as
"exploratory testing" and the use of "session based
testing/management". I'm not looking for a list of names, but I was
wondering if you had a view on how someone can practically apply this
approach? e.g. I've only seen documentation that uses "web based"
technologies with this approach. But I'm from a mainframe background
and can't make the connection.
As I said earlier thank you for the time and the cmap link!
| |
| S Perryman 2006-04-12, 7:08 pm |
| "Steve Devonport" <s.devonport@zeda.co.uk> wrote in message
news:1144830623.288979.217510@z34g2000cwc.googlegroups.com...
Steve, thanks for the reply.
> 1. I do use EqP and BVP, along with state transition & whichever
> analysis technique seems appropriate (this can depend on the culture,
> environment, product, customer, etc). I have to admit I only use these
> techniques "in anger" when I'm asked to look at " significantly changed
> or new" functionality. (Remember I have a pre-determined list of
> scenarios that I can "fall back" on to allow me to make an informed
> decision about the presence of bugs). The earlier you use these
> techniques the more investment you get from them - for this read the
> cheaper it is to identify and fix potential bugs.
> 2. Generally (and I do mean generally) I spend less time on black box
> analysis, because I'm ususally "asked" to prove that a card processing
> system can perform a certain set of functions. For me. this is usually
> carried out at the system integration level and will also involve a
> high degree of "certification" and end-to-end testing. So, therefore
> I'm not usually asked to perform system(functional) testing. This does
> not imply that the closer you get to acceptance testing the less need
> there is for balck box techniques. you could argue the other way quite
> easily and I'd be inclined to agree.
> 3. This may sound weird but I prefer some overlap, it is my "fail safe"
> that when I'm performing analysis using different techniques that I'm
> achieving the same end point, i.e. a list of test cases that satisfy
> the objective of the test phase. So to answer your query, for me I aim
> for round about 10-20% overlap. But for my team I explain that
> different (black box) techniques are to be used and that the "output"
> of these techniques should be shared and compared.
The issue for me would be the degree of overlap.
If the vast majority of tests that I identify with an equivalence class
partitioning
etc are also found with cause/effect graphing, then that would irritate me.
I guess the only way to find out is to work through some examples and
compare/contrast with the other techniques (amount, speed etc) .
Regards,
Steven Perryman
| |
| dumitru.corobceanu@gmail.com 2006-04-12, 7:08 pm |
| Michael Bolton
>...
>But you contend that cause and effect diagrams are needed more than
>ever. Do you use them constantly? Occasionally? Do you use them /as
>they are shown in Myers/?
>...
Steve Devonport
>...
>Background: I currently work within the (UK) Credit Card arena and have
>done for the last 5 years.
>...
>...
>I don't use cause/effect graphing in isolation - I use it as a guiding
>factor, every client I'm involved with has a "different" system - but
>all these "different" systems have to accept the same input,
>...
What I'm getting is that in 5 years you have to dial basically with
the same input.
I don't think that would be enough time for developing cause and
effect diagrams in a more dynamic and various environments.
If you work in a company that produces software that directly interact
with user, you could never count up all possible inputs, using EqP
there are still a lot of possible inputs.
>...
>But for me it's the fact that the core concepts presented
>by Myers still hold true today as they did over 30 years ago.
>...
I'm not agree on this.
What was the user-system interaction 30 years ago?
What could user do 30 years ago and what he can do now, more freedom
for user inputs more source of problems.
What was the software delivery time 30 years ago for an application
that does X and what will be the delivery time for the application that
does the same X now?
I really think that there is simply NOT ENOUGH TIME FOR THAT.
>...
>I also think that in other industries that something similar must be
>true, every client/system is different,
>...
My company is working in banking-financial field. In all project I've
worked on there was only 2 of them that had the same business-logic.
Only 2 width similar BUSINESS-LOGIC, not similar inputs (even different
types web-based and desktop-based). So there was no way for me to make
use of the previous developed cause and effect diagrams (if I did
develop some).
I'd rather go for Heuristic Risk-Based Testing, as it is described by
James Bach.
(sorry for my english, I really should take some more lessons)
| |
| Steve Devonport 2006-04-13, 4:13 am |
| Hey don't worry about the english.
>I don't think that would be enough time for developing cause and
effect diagrams in a more dynamic and various environments.
If you work in a company that produces software that directly interact
with user, you could never count up all possible inputs, using EqP
there are still a lot of possible inputs.
The software I work on is directly used by the user, eg a call centre.
It is true that you can't possibly identify all the input combinations
and therefore the possible outputs. That is why I don't use a single
technique, eg EqP, in isolation. In my opinion the techniques mentioned
in this post allow you to make an informed decision on what to test and
when to test it (ie the most important first).
>I'm not agree on this.
What was the user-system interaction 30 years ago?
What could user do 30 years ago and what he can do now, more freedom
for user inputs more source of problems.
What was the software delivery time 30 years ago for an application
that does X and what will be the delivery time for the application that
does the same X now?
I really think that there is simply NOT ENOUGH TIME FOR THAT.
OK, 30 years ago it's true that most (nearly all) users were using
"green screens" - think of these as terminals that accessed the
mainframe or the (client/)server, if working on unix. In the
banking/financial markets the same "key" fields that are used today
were used 30 years ago!
I agree the time to market has shortened that is a given. You state
that there is not enough time for you to use black box tecniques such
as BVA and EqP? Then explain what you think Heuristic Risk-Based
Testing is? I contend that this is a black box technique. RBT is
something that has been happening since software was first written.
Remember Heuristics are a set of guidelines/framework that allow the
tester to reach an informed decision about the presence of bugs within
a function and/or system.
>My company is working in banking-financial field. In all project I've
worked on there was only 2 of them that had the same business-logic.
Only 2 width similar BUSINESS-LOGIC, not similar inputs (even different
types web-based and desktop-based). So there was no way for me to make
use of the previous developed cause and effect diagrams (if I did
develop some
I agree when you are testing BUSINESS-LOGIC I would not use EqP or BVA
in the same way I would not use these techniques when I'm testing
anti-fraud rules (within the credit card market), RBT and Exploratory
testing are better techniques, assuming you understand the business
requirements for the logic. But remembe that whatever the
BUSINESS-LOGIC rule is, it still requires an input and there a result
will be generated. So I think that you are applying EqP and BVA without
realising it, otherwise you would be generating thousands of
meaningless test cases.
>I'd rather go for Heuristic Risk-Based Testing, as it is described by
James Bach.
This is but one technique and using it in isolation does not allow you,
the tester, to provide the most informed decision to your manager and
ultimately the customer as to the quality of your project - this is my
opinion of course.
| |
| Michael Bolton 2006-04-14, 7:10 pm |
| >The issue for me would be the degree of overlap. If the vast majority of tests that I identify with an equivalence class partitioning etc are also found with cause/effect graphing, then that would irritate me.
I don't know how you can separate equivalence class partitioning from
any testing task.
When we categorize things, we recognize similarities and differences on
some dimension. Equivalence class partitioning is fancy talk for
"identifying similarities and differences, and making choices based
upon them". If we use state transition tables, we're not identifying
every circumstance in which the software can be; we're typically
suggesting implicitly that certain kinds of variations in the data
don't matter but that the states do. When we're choosing one value for
a pathway in a weighted cause-effect diagram, we're choosing that value
as representative in some way. When we opt to run a particular test
based on a particular use case, we're assuming that there are going to
be cases like it in the field, but not necessarily exactly the same.
Many references to equivalence class partition seem to view it as a
technique; I view it as an axiom. I believe that we should recognize
that, for the purposes of any test, certain things matter and certain
things don't.
As for boundary values, some things have boundaries and others don't,
so boundaries are only useful sometimes; they may form the basis for an
equivalence class or they may not. It's important to remember, in my
view, that there are more boundaries than meet the eye. Most books
I've seen refer to very human-oriented boundaries (e.g. the boundary
between 9999 and 10000), but ignore boundaries that would matter to the
machine (e.g. the boundary between 65535 and 65536). There's a
boundary between April 13 and April 14, and there's a boundary between
April 15 and April 16 (this year). The question is whether that
boundary is relevant and meaningful to the given test (and I would hold
that, depending on what you're testing, both the April 13-14 and April
15-16 boundaries could be highly relevant to some observers but not to
others.
Any test technique that I can think of has overlap with other
techniques at some level. I don't worry about that too much, since
each technique affords a different focus. The question in a given
context, for me, include "What risks am I aware of that this technique
would help to expose? Does this technique afford the possibility of
telling me something I didn't expect? Are the values in this test more
likely to expose problems than other values I might reasonably choose?
Does this test help to tell a story that hasn't been told yet?"
---Michael B.
| |
| Michael Bolton 2006-04-14, 7:10 pm |
| 2. A call for help. I'm starting to look into what's known as
"exploratory testing" and the use of "session based
testing/management". I'm not looking for a list of names, but I was
wondering if you had a view on how someone can practically apply this
approach? e.g. I've only seen documentation that uses "web based"
technologies with this approach. But I'm from a mainframe background
and can't make the connection.
Here are a few ways of thinking about it that you might find helpful.
1) Exploratory testing is simultaneous test design, test execution, and
learning. Scripted testing, for the purpose of the next few points, is
whatever we have when test design, test execution, or learning are
separated in some way or by some time. Therefore, almost all testing
is therefore to some degree scripted and to some degree exploratory.
2) Exploratory testing (except perhaps when done by toddlers) is
usually motivated by some testing mission, but that mission is not
specified down to the last detail. Scripted testing (except perhaps
when done by machines) specifies the actions to be performed, but
usually welcomes some deviation from the script, inspired by a tester's
new idea, a previously unnoticed behaviour, or a result (perhaps
expected, perhaps not) from the last test. Therefore, almost all
testing is to some degree scripted and to some degree exploratory.
3) Exploratory testing requires cycles of interaction with the machine
such that the tester can observe the system and change behaviour.
Scripted testing emphasizes the completion of a set of tasks, often
without the kind of direct interaction and observation that affords
course changes.
4) Exploratory testing is focussed on things like adaptability,
creativity, and learning. Scripted testing is focussed on things like
repeatability, accountability, and confirmation.
5) There is nothing intrinsically scriptish about the mainframe world,
and nothing fundamentally exploratory about the Web world. The more
important distinction is in the approach taken by the tester: "I've
run a test, and I've observed the result; what do I do next?" In
exploratory testing, the idea comes from the tester in the moment. In
scripted testing, the idea comes from outside the tester--a piece of
paper, a bit of automation code--and from some earlier time. That
said, there are often conditions that make exploration or scripting
somewhat more feasible in a given context. On a mainframe system,
interactivity is not always part of the design of the system or of the
test environment. In other contexts, visibility and controllability
(that is, monitors, logs, and scriptable interfaces) are not built-in,
so scripted testing is harder.
I have a number of articles and presentations on exploratory testing;
I'll send some to you. I also teach James Bach's Rapid Software
Testing course, in which we outline the fundamentals of session-based
test management, and I've practiced it myself. You can read more about
the latter here: http://www.satisfice.com/sbtm. The explanations are
very good, but also download the .ZIP file and have a look at several
the completed session sheets (dig down until you get the "approved"
folder). These should give you a good impression of what the sessions
are like and how they evolve over time.
I'm aware of several organizations near Toronto, where I live, that
have implemented SBTM to wonderful success. If you'd like referrals to
them, I can connect you with them.
Hope this helps...
----Michael B.
| |
| S Perryman 2006-04-15, 8:06 am |
| "Michael Bolton" <google@michaelbolton.net> wrote in message
news:1145031940.761835.19280@i39g2000cwa.googlegroups.com...
[color=darkred]
> I don't know how you can separate equivalence class partitioning from
> any testing task.
I haven't.
[snip]
> Many references to equivalence class partition seem to view it as a
> technique; I view it as an axiom.
So do I. I consider it to be the "1 + 1 = 2" of testing.
> Any test technique that I can think of has overlap with other
> techniques at some level. I don't worry about that too much, since
> each technique affords a different focus. The question in a given
> context, for me, include "What risks am I aware of that this technique
> would help to expose? Does this technique afford the possibility of
> telling me something I didn't expect? Are the values in this test more
> likely to expose problems than other values I might reasonably choose?
> Does this test help to tell a story that hasn't been told yet?"
I am asking whether the cause/effect technique gets me same/less/more test
cases
than apply a typical equivalence class partitioning. And then whether it
takes same/
less/more time to do.
With no empirical information given, I will investigate myself with suitable
examples.
Regards,
Steven Perryman
| |
| Boris Beizer 2006-04-15, 7:06 pm |
|
"S Perryman" <a@a.net> wrote in message news:e1qren$vb7$1@emma.aioe.org...
> "Michael Bolton" <google@michaelbolton.net> wrote in message
> news:1145031940.761835.19280@i39g2000cwa.googlegroups.com...
>
>
That is correct because most test techniques, specifically all techniques
that fall under the term "partition testing" involve some kind of
equivalence class partitioning, either explicitly or implicitly. It may
pay to explain some technicalities to clear up the many misconceptions that
abound in this regard.
Almost all non-stochastic test techniques are based on the idea that
you can select from the combination of all possible inputs and all possible
initial states of a system a subset of those possibilities for testing and
furthermore you specify that this subset is equivalent to the entire input x
state space. For example, if you elect to do branch testing, you are
asserting that selecting a branch covering set of inputs partitions the
input-state space into subsets that are equivalent for a particular set of
branch possibilities. So techniques such statement coverage, branch
coverage, data-flow testing, various mutation testings, domain testing,
state-transition testing, and for that matter most of the techniques covered
in my books and in Binder's excellent book, are examples of partition
testing.
[color=darkred]
>
> So do I. I consider it to be the "1 + 1 = 2" of testing.
>
Both of the above are correct. Equivalence class partitioning, be it
explicit or implicit (as it is in most cases) is more of an axiom than a
specific technique. Actually, it is more an article of faith than of an
axiom.
Now where does the confusion appear to come from. The first good
book on testing was Myer's "The Art of Software Testing." At the time,
Myers (and most of us who were thinking seriously about testing) understood
the issues and recognized that most non-stochastic testing techniques
involved some kind of equivalence class partitioning. What Myers believed,
as did I and others who had heavy training in the appropriate mathematics ..
we believed that it was possible for the average would-be tester to identify
and formalize these different equivalence class partitionings as appropriate
for each testing situation. That is, to effectively create a new test
technique as needed. This idea pervade's Myer's little book. It is of
course, absolutely correct .. but practice proved otherwise. The typical
would-be tester did not have the mathematical training to do that sort of
thing. By contrast, some teachers of advanced graduate courses in testing
(J. Laski, for one), gave the development of a new test technique as
homework assignments. As Laski said "I can create three new [equivalence
class partitioning] test techniques every morning before breakfast!"
There are an infinite number of ways of partitioning the input-state
space into equivalence classes. There are, therefore, an infinite number of
partitition testing techniques. The real question is not the finding of
tecniques or partition methods, it isn't even the finding of effective
methods, because most partitions will not be helpful in finding bugs. It
isn't the finding of techniques but the application of effective techniques
to actual testing situtions. That means, techniques that can be easily
mastered and exploited by typical testers, techniques supported by tools,
techniques understood by testers, programmers, managers, and unfortunately,
by lawyers.
> I am asking whether the cause/effect technique gets me same/less/more test
> cases
> than apply a typical equivalence class partitioning. And then whether it
> takes same/
> less/more time to do.
Cause-effect graphing is a typical equivalence class partition technique.
So the question is meaningless.
Now as to the effectiveness of cause-effect graphing. Cause effect
graphing is an example of a general class of logic-based testing. There is
no controversy over the correctness of the technique or over its
effectiveness at catching bugs. It is over how and to whom to teach it.
For one trained in logic design who is completely at home with boolean
algebra and knows how to do boolean algebra using Karnaugh-Veitch diagrams,
this is a simple, effective, easy to learn and apply technique. Without
this logic design background, it is a confusing, sometimes horrible and
frustrating method. See my book "Software Testing Techniques" Chapter 10
for logic based testing, KV diagrams, and other prerequisites. See
Binder's, Chapter VI for a thorough review of up-to-date logic-based testing
methods including cause-effect graphing.
--
-------------------------------------
Boris Beizer Ph.D.
1232 Glenbrook Road
Huntingdon Valley, PA 19006
TEL: 215-572-5580
FAX: 215-886-0144
Email bsquare "at" earthlink.net
------------------------------------------
| |
| S Perryman 2006-04-18, 4:08 am |
| "Boris Beizer" <bsquare@sprintmail.com> wrote in message
news:Ny70g.4573$Es3.2192@newsread3.news.atl.earthlink.net...
> "S Perryman" <a@a.net> wrote in message news:e1qren$vb7$1@emma.aioe.org...
[ big snip - but acknowledged ]
[color=darkred]
> Cause-effect graphing is a typical equivalence class partition technique.
> So the question is meaningless.
> Now as to the effectiveness of cause-effect graphing. Cause effect
> graphing is an example of a general class of logic-based testing. There
> is no controversy over the correctness of the technique or over its
> effectiveness at catching bugs. It is over how and to whom to teach it.
> For one trained in logic design who is completely at home with boolean
> algebra and knows how to do boolean algebra using Karnaugh-Veitch
> diagrams, this is a simple, effective, easy to learn and apply technique.
> Without this logic design background, it is a confusing, sometimes
> horrible and frustrating method. See my book "Software Testing
> Techniques" Chapter 10 for logic based testing, KV diagrams, and other
> prerequisites. See Binder's, Chapter VI for a thorough review of
> up-to-date logic-based testing methods including cause-effect graphing.
Based on what you have stated, I see no need to use cause/effect graphs
other than perhaps as a means to better visually present the partitions (I
use
specification-directed testing which means I specify sys reqts via
predicate/propositional logic, and derive my tests from the definitions) .
Regards,
Steven Perryman
|
|
|
|
|