Home > Archive > Perl Quality Assurance > April 2008 > Is "FIT" fit for purpose?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Is "FIT" fit for purpose?
|
|
|
|
| Eric Wilhelm 2008-03-27, 7:49 pm |
| # from Ovid
# on Thursday 27 March 2008 10:52:
>but the example Ward
>Cunningham gave was of a calculator. =A0I thought the idea was neat, but
>how would I implement it?
What do you need to test that your users need to drive? How can it be=20
expressed in a non-tedious and yet understandable way that makes them=20
feel like it is a worthwhile process? Does that fit the Fit model?
I'm curious about the calculator example. Where the inputs of=20
the "1+1=3D2" sort? I can't see a user being interested in driving that.
=46or the payroll example at fit.c2.com, I would probably have implemented=
=20
the calculation as an engine driven by a declaration of the pay scheme. =20
So, the customer-driven input "standard: 40, overtime: 1.5,=20
holiday+sunday: 2.0" would accompany the example calcs. But, if we=20
abandoned those simplistic example calculations, the test coverage=20
would be more thorough in that it would push some sample time cards=20
through the engine and make sure that sundays and holidays were treated=20
correctly, etc.
At that point, you're pushing so much data into the test that it has=20
become tedious for the user to own (create, manually review) the time=20
cards, so you really only want to involve them at the configuration. =20
But you're getting much better test coverage (and presumably the engine=20
has been thoroughly flexed by feeding it various configs) and not=20
writing new code for every variation on a theme.
That is: give the user ownership of the configuration and make sure=20
they understand it without being bored to death. If they can spell-out=20
their business rules directly, that readable logic is probably more=20
comforting than a few examples requiring mental math and providing only=20
spotty coverage.
So, I can see where Fit could be useful in funnelling data from the=20
requirements gathering directly into tests, but I've yet to see an=20
example where it would both be tolerated by the users *and* provide=20
effective test coverage. Particularly, the tabular implementation is=20
so restrictive that expressing the "user model" of the software might=20
not be possible in that form.
I like the concept, but how do I apply it to interactive software (gui=20
or web) where you have an initial state, action, and expected result? =20
That seems to be a more effective place for involving the user and=20
allowing them to verify their expectations, but doesn't fit well into a=20
=46it table.
=2D-Eric
=2D-=20
So malloc calls a timeout and starts rummaging around the free chain,
sorting things out, and merging adjacent small free blocks into larger
blocks. This takes 3 1/2 days.
=2D-Joel Spolsky
=2D--------------------------------------------------
http://scratchcomputing.com
=2D--------------------------------------------------
| |
| Chromatic 2008-03-27, 7:49 pm |
| On Thursday 27 March 2008 12:42:13 Eric Wilhelm wrote:
> What do you need to test that your users need to drive?
Business rules.
> How can it be expressed in a non-tedious and yet understandable way that
> makes them feel like it is a worthwhile process?
That's the guiding design question of FIT tests.
> At that point, you're pushing so much data into the test that it has
> become tedious for the user to own (create, manually review) the time
> cards, so you really only want to involve them at the configuration.
> But you're getting much better test coverage (and presumably the engine
> has been thoroughly flexed by feeding it various configs) and not
> writing new code for every variation on a theme.
Don't mistake FIT tests for unit tests. You'll stab someone if you do.
> So, I can see where Fit could be useful in funnelling data from the
> requirements gathering directly into tests, but I've yet to see an
> example where it would both be tolerated by the users *and* provide
> effective test coverage. Particularly, the tabular implementation is
> so restrictive that expressing the "user model" of the software might
> not be possible in that form.
Of course. It's not for testing user models (if by "user model" you mean what
a user interface designer means).
> I like the concept, but how do I apply it to interactive software (gui
> or web) where you have an initial state, action, and expected result?
That's like deciding that it's impossible to test state machines because you
don't know how to test the GUI for a state machine.
-- c
| |
| Michael G Schwern 2008-03-27, 7:49 pm |
| Ovid wrote:
> Has anyone here ever succesfully used FIT testing? I was at one of the
> first presentations of FITness a long time ago, but the example Ward
> Cunningham gave was of a calculator. I thought the idea was neat, but
> how would I implement it?
When you say "implement it" do you mean the mechanics of it or in a "when can
it be of use" sense?
> We've considered FITness testing, but so far, the only person I've met
> who claims success with it is a consultant to teaches it. Everyone
> else has claimed no experience or that it's more hassle than it's
> worth.
Tony Bowden has had success with it. It's most useful when the people with
the knowledge of how the thing is supposed to work aren't programmers.
Acceptance testing. I do X, I expect Y. This works well with well defined,
deterministic behaviors. Here's an employee and their salary, what taxes and
fees do we take out of their payroll?
Or for units where the field of test data is very, very broad (yet the inputs
and outputs are simple) and you want to employ cheap labor to test it. In
Tony's case, as I remember, he employed his nephew to test a contact
information web scraper. His nephew went to a random web page and eyeballed
it for any contact information. He wrote down the URL and contact info in a
simple table. Harness slurps in that table and compares what it found with
what the human found. Fast way to get a big wad of real world test data. A
programmer would be way more expensive and probably not do nearly as good a
job as they'd get bored.
--
E: "Would you want to maintain a 5000 line Perl program?"
d: "Why would you write a 5000 line program?"
| |
| Eric Wilhelm 2008-03-27, 7:49 pm |
| # from chromatic
# on Thursday 27 March 2008 13:02:
>On Thursday 27 March 2008 12:42:13 Eric Wilhelm wrote:
>
>Business rules.
So, what is a good example of such a business rule? I posit that=20
payroll does not count because the user could more concisely write the=20
rule in a declarative form, this isn't Java, &c.
>
>That's the guiding design question of FIT tests.
That's conveniently intuitive then. So where do I get the guiding=20
design answer? Or at least, how do you decide when to be asking the=20
above question vs when to be asking "how do I set this up such that the=20
business rules are 'programmed' directly by the users?"?
>Don't mistake FIT tests for unit tests. =A0You'll stab someone if you
> do.
Okay, so how do we be sure that the business rule is fully expressed by=20
the Fit input? (That is: guarantee that there are no edge cases.) Or,=20
is this one of those complicated things where worse is better because=20
we don't like better better than worse?
=2D-Eric
=2D-=20
"Time flies like an arrow, but fruit flies like a banana."
=2D-Groucho Marx
=2D--------------------------------------------------
http://scratchcomputing.com
=2D--------------------------------------------------
| |
| Chromatic 2008-03-27, 7:49 pm |
| On Thursday 27 March 2008 16:30:47 Eric Wilhelm wrote:
> # from chromatic
[color=darkred]
[color=darkred]
> So, what is a good example of such a business rule? I posit that
> payroll does not count because the user could more concisely write the
> rule in a declarative form, this isn't Java, &c.
Any rule the system models need to encode, such as:
A payee with no deductions pays 8% federal withholding.
A payee with one deduction pays 7.75% federal withholding.
A payee with two deductions pays 7.50% federal withholding.
=2E..
A payee with the Congressional Medal of Honor pays 0% federal withholding.
>
> That's conveniently intuitive then. So where do I get the guiding
> design answer? Or at least, how do you decide when to be asking the
> above question vs when to be asking "how do I set this up such that the
> business rules are 'programmed' directly by the users?"?
I suppose you ask them "Do you want this software to work correctly, or can=
I=20
just take the money and go home now?"
[color=darkred]
> Okay, so how do we be sure that the business rule is fully expressed by
> the Fit input? (That is: guarantee that there are no edge cases.)
You ask "And then what if?" a lot. Usually no one has written down these=20
rules. In general, software maintenance means encoding these unwritten rul=
es=20
into deployed software when people realize that they're still figuring out=
=20
what the software should do and how they do what they do.
> Or, is this one of those complicated things where worse is better because
> we don't like better better than worse?
I still think you're overthinking this. Think of FIT as a way for=20
non-programmers to write executable specifications in something resembling =
a=20
spreadsheet.
=2D- c
| |
| Michael G Schwern 2008-03-28, 5:09 am |
| Eric Wilhelm wrote:
>
> So, what is a good example of such a business rule? I posit that
> payroll does not count because the user could more concisely write the
> rule in a declarative form, this isn't Java, &c.
I'm  by that response. FIT is declarative. You give the user a
table, they fill it in, it gets run through a routine that interprets the
inputs and outputs.
| gross pay | fed tax | state tax | medicare | social sec | net pay |
---------------------------------------------------------------------
| 40000 | 24% | 6% | 2% | 5% | .... |
---------------------------------------------------------------------
Categorization is a nice example.
| URL | Category |
---------------------------------------------------------
| hooters.com | Restaurant |
| whitehouse.com | Porn |
---------------------------------------------------------
Clear inputs and outputs.
Also no matter if you could write that out as a Perl hash or something you
don't want to be handing accountants a text editor and Perl program. Totally
alien. You want to give them a web page or Excel file to fill out. Part of
the point of FIT is the method of inputing and evaluating the test results is
comfortable to the user, not the programmer.
>
> That's conveniently intuitive then. So where do I get the guiding
> design answer? Or at least, how do you decide when to be asking the
> above question vs when to be asking "how do I set this up such that the
> business rules are 'programmed' directly by the users?"?
That would be highly situational, but it's the same sort of question as "when
do I put this into a config file"?
Of course, even if the rules are specified by the users they still need to be
tested.
>
> Okay, so how do we be sure that the business rule is fully expressed by
> the Fit input? (That is: guarantee that there are no edge cases.) Or,
> is this one of those complicated things where worse is better because
> we don't like better better than worse?
Same way you determine any other test. You run coverage analysis. You give
some thought to the data. If they're numbers you try 0, -1, 1, .5, 2**33,
etc... If it's strings try whitespace, special characters, nul bytes, SQL
commands...
But FIT is not really about doing full edge testing. It's about getting a
broad suite of blackbox tests that match what the users actually do, as
opposed to what the programmer things the users do. Since the users (or the
client) are writing the tests and they probably know the domain better than
you do. Also, lacking any knowledge of the internals, they're not going to
pussyfoot around known fragile spots.
Finally, as chromatic pointed out, FIT is *not* a replacement for unit tests.
It's another tool, specialized to allow the client to write their own
acceptance tests. It's as much about drawing the client into the development
process, getting them involved, getting them to use the iterations, getting
their feedback and buy-in, as it is about testing the software.
--
60. “The Giant Space Ants” are not at the top of my chain of command.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
http://skippyslist.com/list/
| |
| Eric Wilhelm 2008-03-28, 5:09 am |
| # from Michael G Schwern
# on Friday 28 March 2008 00:30:
>
>I'm by that response. =C2=A0FIT is declarative.
It declares the inputs and outputs, not the business rule. Where the=20
table full of numbers is somewhat opaque, a table of rules would more=20
compactly express the business knowledge directly and readably -- the=20
payroll example seems to assume that all of the pay schedules are=20
spelled-out in a multitude of classes with piles of if() statements. =20
I'm saying that having the users write their rules directly in a=20
configuration (or DSL) would be more effective and understandable.
That is, an engine which lets the user directly formulate the rules in a=20
declarative form does a better job of bringing the client into the=20
development process and letting them own the rules than a pile of=20
blackbox input/output data which has to be manually calculated and=20
checked (possibly by the programmer.)
So, what place does Fit have?
| employee type | max hours | overtime | holiday | holiday + |
=2D-------------------------------------------------------------
| standard | 40 | 1.5 | 2 | Sun |
| nights | 40 | 1.5 | - | - |
| temp | 40 | 1.5 | - | - |
Similarly for tax withholding, though there you might end up with=20
subrules which refer to another table.
So, instead of trying to state every input and output, we can see all of=20
the rules clearly for all of the different sorts of employees and they=20
address all of the various situations. No, this isn't a test=20
methodology.
>Part of the point of FIT is the method of inputing and evaluating the
>test results is comfortable to the user, not the programmer.
Do the users *want* to be inputing and evaluating test results?
>Categorization is a nice example.
>
>| URL=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
A0=C2=A0=C2=A0=C2=A0=C2=A0| Categor=
y=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0|[c
olor=darkred]
>---------------------------------------------------------
>| hooters.com =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=[/color]
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | Restaurant =C2=A0 =C2=A0|
>| whitehouse.com =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0| Porn =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
>---------------------------------------------------------
Now I'm really . That looks like tabulated data, so what code=20
would it be testing?
Your earlier example of "this page contains this contact info" for a web=20
scraper was probably more effective. But, that's monkey-work and not=20
business rules.
>
>same way you determine any other test. You run coverage analysis.
Let me pretend that I didn't say "edge cases". How do I know that the=20
algorithm I implemented is the one they wanted and not some other=20
member of the infinite set which can produce the given input/output=20
pairings? If we're missing a critical set of data, we end up=20
completely rewriting code. (Yes, that's true in any methodology, but a=20
table of digits seems more apt to cause such an oversight than a=20
smaller and more readable table/list of rules.)
If Fit is about having the customer drive the *tests*, I guess it does=20
that. But I thought the goal was to have them drive the development=20
and give them a way to express the business rules -- in which case I=20
still can't think of a convincing example where it would be better than=20
the alternatives.
=2D-Eric
=2D-=20
As an old bass player friend of mine used to say: throw money, don't=20
clap.
=2D-Tony Parisi
=2D--------------------------------------------------
http://scratchcomputing.com
=2D--------------------------------------------------
| |
| Eric Wilhelm 2008-03-28, 5:09 am |
| # from chromatic
# on Thursday 27 March 2008 16:53:
>
>I still think you're overthinking this. =A0Think of FIT as a way for
>non-programmers to write executable specifications in something
> resembling a spreadsheet.
Well, here's where I'm having trouble. When I think of=20
"a way for non-programmers to write executable specifications in
something resembling a spreadsheet",
I don't end up with a test framework. I end up with a rules engine.
=2D-Eric
=2D-=20
"Politics is not a bad profession. If you succeed there are many
rewards, if you disgrace yourself you can always write a book."
=2D-Ronald Reagan
=2D--------------------------------------------------
http://scratchcomputing.com
=2D--------------------------------------------------
| |
| Michael G Schwern 2008-03-28, 8:26 am |
| Ok, let's clear this all up.
FIT is not about expressing business rules.
FIT is a tool which allows the customer to add test cases in a way they're
comfortable with. A programmer still has to write the logic behind those
tests (called a Fixture), but it allows a customer to easily add more data,
inputs and outputs.
It can have the nice side effect of getting the customer more involved with
the project.
FIT is not about getting the customer to write the rules any more than unit
testing is about writing the code. It can help clarify the rules and reveal
where the code doesn't match customer expectations. That's why it's called an
acceptance test, done right the customer accepts the work when the tests pass.
Eric Wilhelm wrote:
>
> Now I'm really . That looks like tabulated data, so what code
> would it be testing?
The code which takes a URL and decides what category it's in. Code you defined.
FIT is this...
my %tests = (
# URL # Category
"hooters.com" => "Resturaunt",
"whitehouse.com" => "Porn",
);
for my $url (keys %tests) {
my $category = $tests{$arg};
is categorize_web_site($url), $category;
}
Except instead of %tests you have the user write up a table (Excel, HTML, CSV,
whatever) so they can input new test inputs and expected results without
having to touch code. YOU write the code which interprets those inputs (the
fixture). You also give them a simple way to run the tests, usually just a
submit button on a web page.
That's it. That's all FIT is.
--
191. Our Humvees cannot be assembled into a giant battle-robot.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
http://skippyslist.com/list/
| |
|
| --- Michael G Schwern <schwern@pobox.com> wrote:
> FIT is not about expressing business rules.
<snip>
> That's it. That's all FIT is.
OK, but this all sidesteps one very important question: has anyone on
this list ever *really* used FIT? I'm not talking about toy examples.
I'm not talking about "I know a guy who ..." I would love to hear
real, live stories about it and what follows is why.
Here at the BBC, we have LOTS of programmers and many of them are "test
infected". They're so test infected that they've gone out of their
way to implement FIT, hire consultants to help them improve their
FITness, have software discussion groups on company time to talk about
improving their testing, etc.
In short, these are serious developers who care about the quality of
their software.
Those of them who have worked with FIT are also those who object to it
the loudest. "Too painful to implement and maintain". "Too difficult
to train users." "Too difficult to get users to participate." "Too
little bang for the buck compared to other testing methodologies."
In other words, it seems that as a general rule, those who support FIT
the most seem to be those who haven't used it. I'm just trying to find
out if this experience matches the rest of the world.
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Perl and CGI - http://users.easystreet.com/ovid/cgi_course/
Personal blog - http://publius-ovidius.livejournal.com/
Tech blog - http://use.perl.org/~Ovid/journal/
| |
| Eric Wilhelm 2008-03-28, 7:41 pm |
| # from Ovid
# on Friday 28 March 2008 07:26:
>Those of them who have worked with FIT are also those who object to it
>the loudest. =A0"Too painful to implement and maintain". =A0"Too difficult
>to train users." =A0"Too difficult to get users to participate." =A0"Too
>little bang for the buck compared to other testing methodologies."
Well, the 2nd and 3rd sentiments stem from different places than the 1st=20
and 4th. I haven't used it, but I'll certainly think "too little bang=20
for the buck" until I see a much more convincing example of a set of=20
data/algorithm where Fit would be a good approach.
Getting users involved is important, but it needs to be at a balance of=20
complexity and tedium they can tolerate and they need to feel like what=20
they're doing is useful. I don't think any of the Fit examples I've=20
seen meet the useful criteria and they seem rather high on the tedium=20
scale.
You've told us what the developers want to do. What do the *users* want=20
to do? Is there any buy-in from the users for participating in the qa? =20
Do you *need* them participating in the qa (vs say participating in the=20
encoding of business rules)?
=46it is just a tool, and I have yet to see a task for which it is the=20
right tool. Maybe you need to talk to your hammer-wielding team about=20
spokeshaves.
>In other words, it seems that as a general rule, those who support FIT
>the most seem to be those who haven't used it.
You can mark me as an exception, but you probably already knew that :-D
=2D-Eric
=2D-=20
Don't worry about what anybody else is going to do. The best way to
predict the future is to invent it.
=2D-Alan Kay
=2D--------------------------------------------------
http://scratchcomputing.com
=2D--------------------------------------------------
| |
| Chromatic 2008-03-28, 7:41 pm |
| On Friday 28 March 2008 10:50:07 Eric Wilhelm wrote:
> You've told us what the developers want to do. =A0What do the *users* want
> to do? =A0Is there any buy-in from the users for participating in the qa?=
=A0
> Do you *need* them participating in the qa (vs say participating in the
> encoding of business rules)?
I won't offer any strong opinion whether FIT is the right approach for this=
=20
(having left the world of trying to satisfy users quite a while back), but=
=20
the question there is "How do we know we've built the right thing?"
If the users don't really care about that question, give them a copy of GNU=
=20
Hello World on a nice DVD, take the money, and run.
If they do care about that question, find some way of encoding their answer=
s=20
as executable specifications.
=2D- c
| |
| Eric Wilhelm 2008-03-28, 7:41 pm |
| # from chromatic
# on Friday 28 March 2008 11:03:
>"How do we know we've built the right thing?"
>
>If the users don't really care about that question, give them a copy
> of GNU Hello World on a nice DVD, take the money, and run.
As an internal developer, your users might *not* care but your boss
(their boss) probably charges you with the task of building the right
thing according to what the users (who don't care) want. If it is
actually that bad, I guess you sit at your desk and take the money as
it is dispensed.
But maybe there is some understanding and a reasonable layout of
responsibility/authority, so perhaps some productive work can be done.
>If they do care about that question, find some way of encoding their
> answers as executable specifications.
Yes. That might mean translating English to TAP, but there are plenty
of options there.
--Eric
--
"Beware of bugs in the above code; I have only proved it correct, not
tried it."
--Donald Knuth
---------------------------------------------------
http://scratchcomputing.com
---------------------------------------------------
| |
| Gabor Szabo 2008-03-28, 7:41 pm |
| There were already many good answers here, let me just add my perspective
probably just repeating the previous comments.
I am not a Fit expert and I have never used that with "real" customers
but I did use
something resembling it as I think all of you have.
Occasionally I organize a "QA Day" for QA managers and show them various
techniques along with Ran Eilam. He is a Fit expert and I am CC-ing him hoping
he'll be able to share his experience.
During the QA Day I first show them the "evolution of testing"[1] that
is basically an
introduction to TAP. We build a flow-based test and then re-factor it
so that we can
move the input and the expected output into external files in some user
writable format. (a CSV file).
Effectively these are already FIT tables.
Then comes Ran and shows the FIT approach from the direction of the
tables which is
how users see it.
What I am trying to say is that if you can re factor your code to have your
input and expected output in some data structure, you'll also be able to move it
out to an external file. That file can be written by the users.
I think chromatic mentioned that you should use FIT for acceptance tests and
not for unit tests.
May I disagree here.
I think every test is a unit test. Just the size of the units is different.
For programmers the size of the unit is a function.
For the integration engineer (or the programmer when she is doing
integration between parts of the code)
it might be modules. They call it integration test but that's just bigger units.
When the customer test the product we call it acceptance test. In the
end I think it does not matter.
I can wrap each use-case in a single function (even if that function
needs to call external applications)
and then we are back in "unit-test mode" where you first write a flow
then create an array with input
and expected output values and then you can move it to external files.
So I think we all use a FIT-like approach when we separate test data
from test code.
We just did not call that FIT and did not put the data in spreadsheets.
Gabor
[1] The slides are here:
http://www.szabgab.com/talks/qa_in_.../slides/tap.xul
| |
| Chromatic 2008-03-28, 7:41 pm |
| On Friday 28 March 2008 14:22:32 Gabor Szabo wrote:
> I think chromatic mentioned that you should use FIT for acceptance tests
> and not for unit tests.
> May I disagree here.
> I think every test is a unit test. Just the size of the units is different.
"Unit test" has a very precise meaning in this context. The purpose and goals
of these tests are very different than the purpose of unit tests.
It makes little sense to discuss the code coverage of FIT tests, while it
makes plenty of sense to discuss the code coverage of unit tests, for
example.
-- c
| |
| Fergal Daly 2008-03-28, 7:41 pm |
| On 28/03/2008, chromatic <chromatic@wgz.org> wrote:
> On Friday 28 March 2008 14:22:32 Gabor Szabo wrote:
>
>
>
> "Unit test" has a very precise meaning in this context. The purpose and goals
> of these tests are very different than the purpose of unit tests.
>
> It makes little sense to discuss the code coverage of FIT tests, while it
> makes plenty of sense to discuss the code coverage of unit tests, for
> example.
Why not? If the tests don't get 100% code coverage of the relevant
code then that points to gaps in the test or possibly incorrect
implementation. Of course the FIT users can't interpret the coverage
results on their own, they need to cooperate with the developers and
there's also the issue of identifying the relevant code but coverage
seems to be just as useful here as anywhere else,
F
>
> -- c
>
| |
| David Cantrell 2008-04-01, 7:54 pm |
| On Fri, Mar 28, 2008 at 07:26:39AM -0700, Ovid wrote:
> --- Michael G Schwern <schwern@pobox.com> wrote:
> OK, but this all sidesteps one very important question: has anyone on
> this list ever *really* used FIT? I'm not talking about toy examples.
> I'm not talking about "I know a guy who ..." I would love to hear
> real, live stories about it and what follows is why.
Yes.
CPU::Emulator::Z80 has a load of tests that specify inputs like:
start with 1234 in the BC register
start with 4321 in the program counter
start with 0x34 at memory location 5678
and so on
they then say "after running for four clock cycles the state should be
...." and another set of register values and the contents of various
memory locations.
> Those of them who have worked with FIT are also those who object to it
> the loudest. "Too painful to implement and maintain".
I can see that. Thankfully, I didn't have to create all the data files,
I took them from another emulator.
> "Too difficult
> to train users." "Too difficult to get users to participate." "Too
> little bang for the buck compared to other testing methodologies."
Again, I can see the first two. However, the third is, at least in my
case, dead wrong. Those data were what I needed to attain the ideal of
test-driven development - that is, run the tests, see what fails, write
some code to pass a test, repeat until they all pass, then stop.
The end result is code that works very well, took only three w s or so
of evenings to write, and has very good test coverage. There's a small
number of conditionals that aren't taken (I put the conditionals in
based on what the hardware docs said, but obviously not all possible
variations exist in the tests) so at some point I'm going to add tests
for them.
> In other words, it seems that as a general rule, those who support FIT
> the most seem to be those who haven't used it. I'm just trying to find
> out if this experience matches the rest of the world.
I would like to use it for the Z80 assembler I'm writing. Trouble is,
this time I have to write all the test case data myself instead of just
using someone elses, and so I'm procrastinating.
--
David Cantrell | top google result for "internet beard fetish club"
PLEASE NOTE: This message was meant to offend everyone equally,
regardless of race, creed, sexual orientation, politics, choice
of beer, operating system, mode of transport, or their editor.
|
|
|
|
|