Testing C Compiler

hi all:

I've been working on a retargetable optimizing C compiler (C89/C99) in
my spare time the last year and a half or so, because that is kind of
fun.  There is a little more work to do but it is coming along fine.
I've had it generating working code for a couple of significant
programs on the x86 while I tested the parser, basic optimizations and
code gen.  My problem now is down the road I am interested in seeing
just how well it works.  There are a variety of areas I'm interested
in:

1) how conformant is the parser?
2) how conformant is the RTL? (the RTL was actually written as part of
an earlier compiler project)
3) how accurate are the optimizations?  I'd like to find the bugs.
4) how good is the performance of generated code?

This is a personal project but I'm kinda nitpicky, I like to get the
details right.  Are there test suites that would cover all these
areas?  Any one test suite that covers it all or am I looking at
various test suites for different things?  Anything open source or do
I have to go to the test-suite vendors?  If I have to go to the
vendors can anyone give me a ballpark of the costs I'm looking at?

I'm trying to find out if I can find standardized tests that will
help, or if I'm just going to have to download every program I can
find and compile it to see what happens.

Thanks,
David

> 4) how good is the performance of generated code?

Most academic papers I've seen use SPEC benchmarks
(http://www.spec.org/). Patterson and Hennessy do the same in their
computer architecture books.

> I'm trying to find out if I can find standardized tests that will
> help, or if I'm just going to have to download every program I can
> find and compile it to see what happens.

Hi David,

gcc comes with a rather large test suite. Have a look at:
http://gcc.gnu.org/viewcvs/trunk/gcc/testsuite/

that could be a good starting point. Probably you could look at other
open compiler projects as well, such as pcc or llvm:
http://en.wikipedia.org/wiki/Portable_C_Compiler
http://llvm.org/docs/TestingGuide.html#wholeprograms

this is an example of how the tinyc compiler tests himself against the
gcc testsuite:
http://cvs.savannah.nongnu.org/view...=mark
up

Bye,
Carlo

On Sun, 30 Mar 2008 18:54:38 -0400, "camille@bluegrass.net"
>I'm trying to find out if I can find standardized tests that will
>help, or if I'm just going to have to download every program I can
>find and compile it to see what happens.

About the only free ones that I have found were for COBOL &
FORTRAN-77, and they came from Nist.gov.  :-)
--
ArarghMail803 at arargh.com
BCET Basic Compiler Page: http://www.arargh.com/basic/index.html