Testing

Hello,
I mean it's to QA, about testing,
So far I understand, available pear's test steps are:

* PHPUnit for unit tests, what is more function oriented
I certainly miss some doc somewhere ?
Any package introducing some method should deliver unit tests too !
I mean any new public function should have its own unit test together
in package.

* pear run-test and the .phpt scripts what is more whole script oriented
I sure miss again some documentation...
That's the next step in testing and we are quite in real world.
(quite= http://marc.theaimsgroup.com/?l=pea...21429907822&w=2 )

These 2 steps should be required for any first acceptance of a release.
and augmented for any further extensions.

I hope it's no offense if I point out that:

(1) I need to edit all of it to build unit or test scripts,
I want my box to learn as I'm testing and generate some canvas scripts!
Some kind of learning break-points, at begin of scripts or methods,
what should be removed by the developer when ok (as debugger)
So I can choose afterward what and what tests where meaningfull.
(and adapt them by edit for unit or test)
I aim to be a good developer, so I must be lazy.

(2) The dynamic in the script is not included there
What is with two clients asking that stuff tested single in the .phpt
script ?
Some funny situations can occur.
I would believe it's missing some more step,
what is related to benchmark (performance oriented)
but  more to some simulation environment.

SIMULATION is the only way to tire the service or to provoke hazard.
A third step or layer is missing.

QÃ_+
bertrand Gugger (toggg)

bertrand Gugger wrote:

> Hello,
> I mean it's to QA, about testing,
> So far I understand, available pear's test steps are:
>
> * PHPUnit for unit tests, what is more function oriented
>  I certainly miss some doc somewhere ?
>  Any package introducing some method should deliver unit tests too !
>  I mean any new public function should have its own unit test together
> in package.
>
> * pear run-test and the .phpt scripts what is more whole script oriented
>  I sure miss again some documentation...
>  That's the next step in testing and we are quite in real world.
>  (quite= http://marc.theaimsgroup.com/?l=pea...21429907822&w=2 )

this division is not accurate.  I would put it

1) PHPUnit and Simpletest can only be used to test classes and functions
2) .phpt can be used to test anything that PHP can be used for.

Of course, as with any application, I'm sure PHPUnit/Simpletest could be
bent to do global testing (PEAR_ErrorStack does this, actually), but the
main point is the design must be subverted rather than used.

>
> These 2 steps should be required for any first acceptance of a release.
> and augmented for any further extensions.

Some packages simply can't be easily regression-tested by all users.
Generally, apps that absolutely require complex external interfaces like
DB or MDB will always require some external setup by the tester.
Packages like Net_Curl, for instance, require that an external server be
set up over the internet - it's literally impossible to test its
functionality without this step.

> I hope it's no offense if I point out that:
>
> (1) I need to edit all of it to build unit or test scripts,
> I want my box to learn as I'm testing and generate some canvas scripts!
> Some kind of learning break-points, at begin of scripts or methods,
> what should be removed by the developer when ok (as debugger)
> So I can choose afterward what and what tests where meaningfull.
> (and adapt them by edit for unit or test)
> I aim to be a good developer, so I must be lazy.

I agree with the "I aim" part, but have no idea what you are talking
about in the rest of the paragraph, could you rephrase?

> (2) The dynamic in the script is not included there
> What is with two clients asking that stuff tested single in the .phpt
> script ?
> Some funny situations can occur.
> I would believe it's missing some more step,
> what is related to benchmark (performance oriented)
> but  more to some simulation environment.

again, what are you trying to say?

> SIMULATION is the only way to tire the service or to provoke hazard.
> A third step or layer is missing.


Are you talking about mock objects?  If you want to use mock objects,
it's no harder to do this with PHPUnit2, Simpletest, or simply by
writing them yourself.  I wrote several mock objects for PEAR's unit
tests, including an object that literally simulates the entire PEARweb
without using the internet.  It didn't take long, but it did take work.

Greg