standards-compliance [was: Commercial development in Scheme?]

Anton van Straaten wrote:
> Per Bothner <per@bothner.com> wrote:
> 
>
> I don't understand this.  For example, it's easy to look at R5RS, note tha
t
> it requires implementations to support continuations, compare it to an
> implementation that doesn't, and say that such an implementation is not
> standards-compliant in that respect.  No agreed-on testsuite is required.

Yes.  I'm just making the point that without "validation" (which
requires a test-suite) statements about "standards-compliance"
are meaningless.  You can say program X is *not* compliant with
standard S because we can easily show that it does the wrong
thing on input P, or because we have been told that a feature F
is not implemented.  But we cannot make a positiven statement that
X *is* compliant based on a gut feeling that X can run "hello world".

Specifically, in a legal/government/procurement sense there is
currently no way to specify "standard" R5RS, since the requirement
cannot be tested.

Of course even a comprehensive testsuite for S cannot prove that X
is bug-free, or that it in all respects follows the intent of S.
But at least with a test-suite we can make meaningful statements
about the standards-compliance of X, in both the scientific
and legal (contractual) sense.  Without a testsuite such statements
have no meaning, scientifically or legally.

Ideally we'd want to be able to mathematically *prove* that X
implements S, but unfortunately we're nowehere close to being able
to do that for non-trivial X and S.