Installing GCC: Testing

   _Please note that this is only applicable to current development
   versions of GCC and GCC 3.0 or later. GCC 2.95.x does not come with a
   testsuite._

   Before you install GCC, you might wish to run the testsuite. This step
   is optional and may require you to download additional software.

   First, you must have [1]downloaded the testsuites. The full
   distribution contains testsuites; only if you downloaded the "core"
   compiler plus any front ends, you do not have the testsuites.

   Second, you must have a current version of dejagnu installed; dejagnu
   1.3 is not sufficient. We have made dejagnu snapshots available at
   [2]ftp://gcc.gnu.org/pub/gcc/infrastructure until a new version of
   dejagnu has been released.

   A few targets require specific preparations:
     * In order to run the libio tests on targets which do not fully
       support Unix/POSIX commands (e.g. Cgwin), the references to the
       dbz directory may be deleted from libio/configure.in.
     * When there is difficulty resolving paths to the files init.tcl and
       runtest.exp, the following environment variables should be set
       appropriately, for example:

     TCL_LIBRARY = /usr/share/tcl8.0
     DEJAGNULIBS = /usr/share/dejagnu
       On systems such as Cygwin, these paths are required to be actual
       paths, not mounts or links; presumably this is due to some lack of
       portability in the dejagnu code.

   Finally, you can run the testsuite (which may take a long time):

     cd _objdir_; make -k check

   The testing process will try to test as many components in the GCC
   distribution as possible, including the C, C++ and Fortran compilers
   as well as the C++ runtime libraries.

How can I run the test suite on selected tests?

   As a first possibility to cut down the number of tests that are run it
   is possible to use `make check-gcc` or `make check-g++` in the gcc
   subdirectory of the object directory. To further cut down the tests
   the following is possible:

     make check-gcc RUNTESTFLAGS="execute.exp <other options>"

   This will run all gcc execute tests in the testsuite.

     make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <other options>"

   This will run the g++ "old-deja" tests in the testsuite where the
   filename matches 9805*.

   The *.exp files are located in the testsuite directories of the GCC
   source, the most important ones being compile.exp, execute.exp, dg.exp
   and old-deja.exp. To get a list of the possible *.exp files, pipe the
   output of `make check` into a file and look at the "Running ... .exp"
   lines.

How to interpret test results

   After the testsuite has run you'll find various *.sum and *.log files
   in the testsuite subdirectories. The *.log files contain a detailed
   log of the compiler invocations and the corresponding results, the
   *.sum files summarize the results. These summaries list all the tests
   that have been run with a corresponding status code:
     * PASS: the test passed as expected
     * FAIL: the test unexpectedly failed
     * XFAIL: the test failed as expected
     * UNSUPPORTED: the test is not supported on this platform
     * ERROR: the testsuite detected an error
     * WARNING: the testsuite detected a possible problem

   It is normal for some tests to report unexpected failures. At the
   current time our testing harness does not allow fine grained control
   over whether or not a test is expected to fail. We expect to fix this
   problem in future releases.

Submitting test results

   If you want to report the results to the GCC project, use the
   contrib/test_summary shell script. Start it in the objdir with

     _srcdir_/contrib/test_summary -p your_commentary.txt -m
     gcc-testresults@gcc.gnu.org |sh

   This script uses the Mail program to send the results, so make sure it
   is in your PATH. The file your_commentary.txt is prepended to the
   testsuite summary and should contain any special remarks you have on
   your results or your build environment. Please do not edit the
   testsuite result block or the subject line, as these messages are
   automatically parsed and presented at the [3]GCC testresults web page.
   Here you can also gather information on how specific tests behave on
   different platforms and compare them with your results. A few failing
   testcases are possible even on released versions and you should look
   here first if you think your results are unreasonable.

References

   1. http://gcc.gnu.org/install/download.html
   2. ftp://gcc.gnu.org/pub/gcc/infrastructure
   3. http://gcc.gnu.org/testresults/