X-Git-Url: http://demsky.eecs.uci.edu/git/?a=blobdiff_plain;f=utils%2FNightlyTestTemplate.html;h=8e34b7931056f0d7615f6c3bf6dd4e688dce4a42;hb=ca428df792b40a73d904670e03b23080b1cabb1a;hp=2c105fd1f89071080ae6bc2f82e06b293b1c0ca3;hpb=da5a6435b22bd01a93444f61291c8c62c51c270b;p=oota-llvm.git diff --git a/utils/NightlyTestTemplate.html b/utils/NightlyTestTemplate.html index 2c105fd1f89..8e34b793105 100644 --- a/utils/NightlyTestTemplate.html +++ b/utils/NightlyTestTemplate.html @@ -1,4 +1,3 @@ - LLVM Test Results for $DateString @@ -17,6 +16,7 @@ Changes
Trends
Programs
+Feature
Regression
@@ -41,23 +41,41 @@ Today's Test Results Overview

+ + + + +
+ +
Lines Of Code over Time
+Click for larger view +
+ +

Nightly Test Overview:

+

CVS Tree Overview:
  • Compilation Log - + + + + + + +
    ItemCPU TimeWall Clock
    Configure CVS Tree$ConfigTime$ConfigWallTime
    Build CVS Tree$BuildTime$BuildWallTime
    Run Feature Tests$FeatureTime$FeatureWallTime
    Run Regression Tests$RegressionTime$RegressionWallTime
    • +
  • Number of object files compiled: $NumObjects
    • +
  • Number of libraries linked: $NumLibraries
    • +
  • Number of executables linked: $NumExecutables
    • +
  • Build Error: $BuildError

    • Warnings during the build:

    @@ -92,22 +110,63 @@
  • Newly passing tests: $TestsFixed
  • Newly failing tests: $TestsBroken - +

    Changes over Time +Changes Over Time

    -GNUplot charts showing a couple of things: -

      -
    1. LOC in CVS -
    2. Compile Time -
    3. Testing Time -
    4. Time to run some programs, say all of Olden -
    5. Time to compile something, say Olden. -
    6. Size of bytecode files for Olden -
    + +Here are some charts showing how the LLVM optimizer and code generators are +changing over time. For now we use the Olden benchmark suite to measure this, +but eventually we will switch to using SPEC CPU2000. All programs are run with +"LARGE_PROBLEM_SIZE" enabled. Click on any of the charts to get a larger +version.

    + +

    Compilation Measurements:

    + + + + + + + + +
    +
    +Size of LLVM bytecode files +    		 +
    +Size of native machine code for each program (generated by the JIT) +
    +
    +Time to run the LLVM optimizer on each program +
    + +

    Program Execution Measurements:

    + + + + + + + + + +
    +
    +Execution time for CBE generated executable +    		 +
    +Execution time for the LLC generated executable +
    +
    +Execution time for program in the JIT +
    + + +

    @@ -116,51 +175,85 @@ GNUplot charts showing a couple of things:

    This section tests LLVM on a variety of programs in the test suite. This -includes benchmark suites like the Olden, ptrdist, and SPEC benchmarks as well -as a few random programs with test inputs. This section is meant to track how -stable LLVM is as a whole. The columns of the tables are:

    +includes benchmark suites like the Olden, McCat, Ptrdist, and SPEC benchmarks as +well as a few random programs with test inputs. This section is meant to track +how stable LLVM is as a whole. A failure in the execution of any test is marked +with an asterisk: `*'. The columns of the tables are:

      -
    1. Program - The name of the program for that row -
    2. GCCAS - Time to run LLVM optimizers on the program -
    3. Bytecode - The size of the bytecode for the program -
    4. Instrs - The number of LLVM instructions in the - compiled bytecode -
    5. LLC - The time taken compile with LLC (the static X86 backend) -
    6. NAT - The time taken to execute the program when compiled - with GCC -O3. -
    7. CBE - The time taken to execute the program after compilation - through the C Backend. If the diff fails, this a *. -
    8. JIT - How long does the program take to execute in the - Just-In-Time compiler. -
    9. MachCode - The number of bytes of machine code - generated by the JIT. -
    10. CompTime - The amount of time spent in the JIT itself, - instead of executing the program. +
    11. Program - The name of the program for that row.
      12. +
    12. GCCAS - Time to run LLVM optimizers on the program.
      13. +
    13. Bytecode - The size of the bytecode for the + program
      14. +
    14. Instrs - The number of LLVM instructions in the + compiled bytecode
      15. +
    15. LLC compile - The time taken compile with + LLC (the static backend)
      16. +
    16. JIT codegen - The amount of time spent in the + JIT itself, instead of executing the program.
      17. +
    17. Machine code - The number of bytes of machine + code generated by the JIT.
      18. +
    18. GCC - The time taken to execute the program when compiled + with GCC -O2.
      19. +
    19. CBE - The time taken to execute the program after + compilation through the C backend, compiled with -O2.
      20. +
    20. LLC - How long does the program generated by the static + backend LLC take to execute
      21. +
    21. JIT - The amount of time spent running the + program with the JIT; this includes the code generation phase (listed above) + and actually running the program.
      22. +
    22. GCC/LLC - The speed-up of the LLC output vs the native + GCC output: greater than 1 is a speedup, less than 1 is a slowdown.
      23. +
    23. GCC/CBE - The speed-up of the CBE output vs the native + GCC output: greater than 1 is a speedup, less than 1 is a slowdown.
      24. +
    24. LLC-LS - How long does the program generated by the static + backend LLC take to execute the program, when compiled with the linear scan + register allocator. This is temporary, for tuning.

    -A complete log of testing is available -for further analysis. +A complete log of testing +SingleSource, +MultiSource, and +External programs are +available for further analysis. + +

    Programs/External

    + +
    +
    +$ExternalProgramsTable +

    Programs/MultiSource

    -$ProgramsTable +$MultiSourceProgramsTable

    Programs/SingleSource

    +
    +
    +$SingleSourceProgramsTable +
    + -

    +

    Regression Tests -

    +Feature Test Results +

    +
    +$FeatureTestResults -Not yet implemented

    - - +

    +
    +
    Regression Test Results +
    +
    +$RegressionTestResults - +