1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
2 <html><head><title>LLVM Test Results for $DateString</title></head>
5 <center><font size=+3 face=Verdana><b>LLVM Test Results for $DateString</b></font></center>
9 <tr><td valign=top align=center>
11 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
12 <table border="0" cellpadding="5" cellspacing="0"><tr><td bgcolor="#DDAA77">
13 <font size=+1><b>Sections:</b></font><br>
14 </td></tr><tr><td bgcolor="#FFCC99" align=center>
15 <a href="#Overview">Overview</a><br>
16 <a href="#Changes">Changes</a><br>
17 <a href="#Trends">Trends</a><br>
18 <a href="#Programs">Programs</a><br>
19 <a href="#Feature">Feature</a><br>
20 <a href="#Regression">Regression</a><br>
21 </td></tr></table></td></tr></table>
24 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
25 <table border="0" cellpadding="5" cellspacing="0"><tr><td bgcolor="#DDAA77"
26 <font size=+1><b>Previous:</b></font><br>
27 </td></tr><tr><td bgcolor="#FFCC99">
29 </td></tr></table></td></tr></table>
32 <font size=+1><b>Back to:</b></font><br>
33 <a href="http://llvm.cs.uiuc.edu/testresults/">Test Results</a><br>
34 <a href="http://llvm.cs.uiuc.edu/">LLVM Page</a><p>
39 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
40 <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
41 <font size=+2 face=Verdana><b><a name="Overview">Today's Test Results Overview</font></b>
42 </td></tr></table></td></tr></table></center><p>
44 <!-- Running LOC graph -->
47 <a href="running_loc_large.png"
48 ><img border=0 width=480 height=360 src="running_loc.png"></a>
50 <tr><td align=center>Lines Of Code over Time<br>
51 <font size=-1><a href="running_loc_large.png">Click for larger view</a></font>
55 <h2>Nightly Test Overview:</h2>
57 <li>Start: <b>$TestStartTime GMT</b></li>
58 <li>Finish: <b>$TestFinishTime GMT</b></li>
59 <li>Platform: <b>$TestPlatform</b></li>
61 <h2>CVS Tree Overview:</h2>
63 <li><a href="$DATE-CVS-Log.txt">CVS Checkout Log</a>
65 <b>$NumDirsInCVS</b> dirs, <b>$NumFilesInCVS</b> files, <b>$LOC</b>
66 lines of code, checked out in <b>$CVSCheckoutTime</b> seconds<br></ul>
67 <li><a href="$DATE-Build-Log.txt">Compilation Log</a>
69 <tr><td><b>Item</b></td><td><b>CPU Time</b></td><td><b>Wall Clock</b></td></tr>
70 <tr><td>Configure CVS Tree</td><td>$ConfigTime</td><td>$ConfigWallTime</td></tr>
71 <tr><td>Build CVS Tree</td><td>$BuildTime</td><td>$BuildWallTime</td></tr>
72 <tr><td>Run Feature Tests</td><td>$FeatureTime</td><td>$FeatureWallTime</td></tr>
73 <tr><td>Run Regression Tests</td><td>$RegressionTime</td><td>$RegressionWallTime</td></tr>
75 <li>Number of object files compiled: <b>$NumObjects</b></li>
76 <li>Number of libraries linked: <b>$NumLibraries</b></li>
77 <li>Number of executables linked:<b> $NumExecutables</b></li>
78 <li>Build Error: $BuildError</li>
81 <h2>Warnings during the build:</h2>
86 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
87 <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
88 <font size=+2 face=Verdana><b><a name="Changes">Changes from Yesterday</font></b>
89 </td></tr></table></td></tr></table></center><p>
91 <h2>Changes to CVS:</h2>
93 <li>Users who committed to CVS: <b>$UserCommitList</b>
94 <li>Users who updated from CVS: <b>$UserUpdateList</b>
95 <li>Added Files: $AddedFilesList
96 <li>Modified Files: $ModifiedFilesList
97 <li>Removed Files: $RemovedFilesList
100 <h2>Changes to Warnings:</h2>
102 <li>Warnings Added: $WarningsAdded
103 <li>Warnings Removed: $WarningsRemoved
106 <h2>Changes in the test suite:</h2>
108 <li>New Tests: $TestsAdded
109 <li>Removed Tests: $TestsRemoved
110 <li>Newly passing tests: $TestsFixed
111 <li>Newly failing tests: $TestsBroken
113 </td></tr></tbody></table>
115 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
116 <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
117 <font size=+2 face=Verdana><b><a name="Trends">Changes Over Time</font></b>
118 </td></tr></table></td></tr></table></center><p>
121 Here are some charts showing how the LLVM optimizer and code generators are
122 changing over time. For now we use the Olden benchmark suite to measure this,
123 but eventually we will switch to using SPEC CPU2000. All programs are run with
124 "LARGE_PROBLEM_SIZE" enabled. Click on any of the charts to get a larger
127 <h2>Compilation Measurements:</h2>
129 <table border="0" align=center>
131 <td width=50% align=center>
132 <a href="running_Olden_bytecode_large.png"><img width=480 height=360 border=0 src="running_Olden_bytecode.png"></a><br>
133 Size of LLVM bytecode files
135 <td width=50% align=center>
136 <a href="running_Olden_machcode_large.png"><img width=480 height=360 border=0 src="running_Olden_machcode.png"></a><br>
137 Size of native machine code for each program (generated by the JIT)
141 <a href="running_Olden_opt_time_large.png"><img width=480 height=360 border=0 src="running_Olden_opt_time.png"></a><br>
142 Time to run the LLVM optimizer on each program
147 <h2>Program Execution Measurements:</h2>
149 <table border="0" align=center>
151 <td width=50% align=center>
152 <a href="running_Olden_cbe_time_large.png"><img width=480 height=360 border=0 src="running_Olden_cbe_time.png"></a><br>
153 Execution time for CBE generated executable
155 <td width=50% align=center>
156 <a href="running_Olden_llc_time_large.png"><img width=480 height=360 border=0 src="running_Olden_llc_time.png"></a><br>
157 Execution time for the LLC generated executable
162 <a href="running_Olden_jit_time_large.png"><img width=480 height=360 border=0 src="running_Olden_jit_time.png"></a><br>
163 Execution time for program in the JIT
172 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
173 <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
174 <font size=+2 face=Verdana><b><a name="Programs">Program Tests</font></b>
175 </td></tr></table></td></tr></table></center><p>
177 This section tests LLVM on a variety of programs in the test suite. This
178 includes benchmark suites like the Olden, McCat, Ptrdist, and SPEC benchmarks as
179 well as a few random programs with test inputs. This section is meant to track
180 how stable LLVM is as a whole. A failure in the execution of any test is marked
181 with an asterisk: `*'. The columns of the tables are:<p>
184 <li><a name="Program">Program</a> - The name of the program for that row.</li>
185 <li><a name="GCCAS">GCCAS</a> - Time to run LLVM optimizers on the program.</li>
186 <li><a name="Bytecode">Bytecode</a> - The size of the bytecode for the
188 <li><a name="Instrs">Instrs</a> - The number of LLVM instructions in the
189 compiled bytecode</li>
190 <li><a name="LLC<br>compile">LLC compile</a> - The time taken compile with
191 LLC (the static backend)</li>
192 <li><a name="JIT<br>codegen">JIT codegen</a> - The amount of time spent in the
193 JIT itself, instead of executing the program.</li>
194 <li><a name="Machine<br>code">Machine code</a> - The number of bytes of machine
195 code generated by the JIT.</li>
196 <li><a name="GCC">GCC</a> - The time taken to execute the program when compiled
198 <li><a name="CBE">CBE</a> - The time taken to execute the program after
199 compilation through the C backend, compiled with -O2.</li>
200 <li><a name="LLC">LLC</a> - How long does the program generated by the static
201 backend LLC take to execute </li>
202 <li><a name="JIT">JIT</a> - The amount of time spent running the
203 program with the JIT; this includes the code generation phase (listed above)
204 and actually running the program.</li>
205 <li><a name="GCC/LLC">GCC/LLC</a> - The speed-up of the LLC output vs the native
206 GCC output: greater than 1 is a speedup, less than 1 is a slowdown.</li>
207 <li><a name="GCC/CBE">GCC/CBE</a> - The speed-up of the CBE output vs the native
208 GCC output: greater than 1 is a speedup, less than 1 is a slowdown.</li>
209 <li><a name="LLC-BETA">LLC-BETA</a> - How long does the program generated by the static
210 backend LLC take to execute the program, when compiled with new experimental
211 features. This is temporary, for tuning.</li>
214 A complete log of testing
215 <a href="$DATE-SingleSource-ProgramTest.txt.gz">SingleSource</a>,
216 <a href="$DATE-MultiSource-ProgramTest.txt.gz">MultiSource</a>, and
217 <a href="$DATE-External-ProgramTest.txt.gz">External</a> programs are
218 available for further analysis.
220 <h2>Programs/External</h2>
223 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
224 $ExternalProgramsTable
225 </td></tr></table></center>
227 <h2>Programs/MultiSource</h2>
230 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
231 $MultiSourceProgramsTable
232 </td></tr></table></center>
234 <h2>Programs/SingleSource</h2>
237 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
238 $SingleSourceProgramsTable
239 </td></tr></table></center>
244 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
245 <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
246 <font size=+2 face=Verdana><b><a name="Feature">Feature Test Results</font></b>
247 </td></tr></table></td></tr></table></center>
252 <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
253 <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
254 <font size=+2 face=Verdana><b><a name="Regression">Regression Test Results</font></b>
255 </td></tr></table></td></tr></table></center>
257 $RegressionTestResults