1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
|
<html>
<head>
<title>Installing GCC: Testing</title>
</head>
<body>
<h1 align="center">Installing GCC: Testing</h1>
<p><strong>Please note that this is only applicable to current development
versions of GCC and GCC 3.0 or later. GCC 2.95.x does not come with a
testsuite.</strong></p>
<p>Before you install GCC, you might wish to run the testsuite. This
step is optional and may require you to download additional software.</p>
<p>First, you must have <a href="download.html">downloaded the
testsuites</a>. The full distribution contains testsuites; only if you
downloaded the "core" compiler plus any front ends, you do not have the
testsuites.</p>
<p>Second, you must have a current version of dejagnu installed;
dejagnu 1.3 is not sufficient. We have made dejagnu snapshots available at
<a href="ftp://gcc.gnu.org/pub/gcc/infrastructure">ftp://gcc.gnu.org/pub/gcc/infrastructure</a>
until a new version of dejagnu has been released.</p>
<p>A few targets require specific preparations:</p>
<ul>
<li>
In order to run the libio tests on targets which do not fully
support Unix/POSIX commands (e.g. Cgwin), the references to the dbz
directory may be deleted from libio/configure.in.
</li>
<li>
<p>When there is difficulty resolving paths to the files init.tcl and
runtest.exp, the following environment variables should be set
appropriately, for example:</p>
<blockquote><code>
TCL_LIBRARY = /usr/share/tcl8.0<br>
DEJAGNULIBS = /usr/share/dejagnu
</code></blockquote>
<p>On systems such as Cygwin, these paths are required to be actual
paths, not mounts or links; presumably this is due to some lack of
portability in the dejagnu code.</p>
</li>
</ul>
<p>Finally, you can run the testsuite (which may take a long time):</p>
<blockquote><code>
cd <i>objdir</i>; make -k check
</code></blockquote>
<p>The testing process will try to test as many components in the GCC
distribution as possible, including the C, C++ and Fortran compilers as
well as the C++ runtime libraries.</p>
<h2>How can I run the test suite on selected tests?</h2>
<p>As a first possibility to cut down the number of tests that are run it is
possible to use `<code>make check-gcc</code>` or `<code>make check-g++</code>`
in the gcc subdirectory of the object directory. To further cut down the
tests the following is possible:</p>
<blockquote><code>
make check-gcc RUNTESTFLAGS="execute.exp <other options>"
</code></blockquote>
<p>This will run all gcc execute tests in the testsuite.</p>
<blockquote><code>
make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <other options>"
</code></blockquote>
<p>This will run the g++ "old-deja" tests in the testsuite where the filename
matches 9805*.</p>
<p>The *.exp files are located in the testsuite directories of the GCC
source, the most important ones being compile.exp, execute.exp, dg.exp
and old-deja.exp. To get a list of the possible *.exp files, pipe the
output of `<code>make check</code>` into a file and look at the
"<code>Running ... .exp</code>" lines.
<h2>How to interpret test results</h2>
<p>After the testsuite has run you'll find various *.sum and *.log
files in the testsuite subdirectories. The *.log files contain a
detailed log of the compiler invocations and the corresponding
results, the *.sum files summarize the results. These summaries list
all the tests that have been run with a corresponding status code:</p>
<ul>
<li>PASS: the test passed as expected
<li>FAIL: the test unexpectedly failed
<li>XFAIL: the test failed as expected
<li>UNSUPPORTED: the test is not supported on this platform
<li>ERROR: the testsuite detected an error
<li>WARNING: the testsuite detected a possible problem
</ul>
<p> It is normal for some tests to report
unexpected failures. At the current time our testing harness does not allow
fine grained control over whether or not a test is expected to fail. We expect
to fix this problem in future releases.</p>
<h2>Submitting test results</h2>
<p>If you want to report the results to the GCC project, use the
<code>contrib/test_summary</code> shell script. Start it in the objdir with</p>
<blockquote><code>
<i>srcdir</i>/contrib/test_summary -p your_commentary.txt -m gcc-testresults@gcc.gnu.org |sh
</code></blockquote>
<p>This script uses the <code>Mail</code> program to send the results, so
make sure it is in your PATH. The file your_commentary.txt is
prepended to the testsuite summary and should contain any special
remarks you have on your results or your build environment. Please
do not edit the testsuite result block or the subject line, as these
messages are automatically parsed and presented at the <a
href="http://gcc.gnu.org/testresults/">GCC testresults</a> web
page. Here you can also gather information on how specific tests
behave on different platforms and compare them with your results. A
few failing testcases are possible even on released versions and you
should look here first if you think your results are unreasonable.</p>
</body>
</html>
|
