[LIT] Fix failing LIT tests

Summary:
I spend some time trying to get the LIT test suite passing. Here are the changes that I needed to make on my machine.

I made the following changes for the following reasons.

1. google-test.py: The Google test format now checks for "[  PASSED  ] 1 test." to check if a test passes.
2. discovery.py: The output appears in a different order on my machine than it did in the test.
3. unittest-adaptor.py: The output appears in a different order on my machine than it did in the test.
4. The classname is now formed differently in `getJUnitXML(...)`.

I'm not sure what is causing the output order to differ in discovery.py and unittest-adaptor.py. Does anybody have any thoughts?

Reviewers: ddunbar, danalbert, jroelofs

Reviewed By: jroelofs

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9864

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239663 91177308-0d34-0410-b5e6-96231b3b80d8

diff --git a/utils/lit/tests/Inputs/googletest-format/DummySubDir/OneTest b/utils/lit/tests/Inputs/googletest-format/DummySubDir/OneTest
index 9dff137f4dec60b1a11db70be5c8e018b44712c4..dd49f025b1f2e2e85de60b83251ce65e8e0ecff0 100755 (executable)
--- a/utils/lit/tests/Inputs/googletest-format/DummySubDir/OneTest
+++ b/utils/lit/tests/Inputs/googletest-format/DummySubDir/OneTest
@@ -21,6 +21,7 @@ elif not sys.argv[1].startswith("--gtest_filter="):
 test_name = sys.argv[1].split('=',1)[1]
 if test_name == 'FirstTest.subTestA':
     print('I am subTest A, I PASS')
+    print('[  PASSED  ] 1 test.')
     sys.exit(0)
 elif test_name == 'FirstTest.subTestB':
     print('I am subTest B, I FAIL')
@@ -29,6 +30,7 @@ elif test_name == 'FirstTest.subTestB':
 elif test_name in ('ParameterizedTest/0.subTest',
                    'ParameterizedTest/1.subTest'):
     print('I am a parameterized test, I also PASS')
+    print('[  PASSED  ] 1 test.')
     sys.exit(0)
 else:
     raise SystemExit("error: invalid test name: %r" % (test_name,))

diff --git a/utils/lit/tests/discovery.py b/utils/lit/tests/discovery.py
index 28010894cda629bde45a7d02f9021eec620b7ef8..60f67dcd740e3cc38900180ef3fedf787a11e83f 100644 (file)
--- a/utils/lit/tests/discovery.py
+++ b/utils/lit/tests/discovery.py
@@ -7,8 +7,8 @@
 # RUN: FileCheck --check-prefix=CHECK-BASIC-ERR < %t.err %s
 #
 # CHECK-BASIC-ERR: loading suite config '{{.*}}/discovery/lit.cfg'
-# CHECK-BASIC-ERR: loading local config '{{.*}}/discovery/subdir/lit.local.cfg'
 # CHECK-BASIC-ERR: loading suite config '{{.*}}/discovery/subsuite/lit.cfg'
+# CHECK-BASIC-ERR: loading local config '{{.*}}/discovery/subdir/lit.local.cfg'
 #
 # CHECK-BASIC-OUT: -- Test Suites --
 # CHECK-BASIC-OUT:   sub-suite - 2 tests
@@ -51,8 +51,8 @@
 # CHECK-ASEXEC-ERR: load_config from '{{.*}}/discovery/lit.cfg'
 # CHECK-ASEXEC-ERR: loaded config '{{.*}}/discovery/lit.cfg'
 # CHECK-ASEXEC-ERR: loaded config '{{.*}}/exec-discovery/lit.site.cfg'
-# CHECK-ASEXEC-ERR: loading local config '{{.*}}/discovery/subdir/lit.local.cfg'
 # CHECK-ASEXEC-ERR: loading suite config '{{.*}}/discovery/subsuite/lit.cfg'
+# CHECK-ASEXEC-ERR: loading local config '{{.*}}/discovery/subdir/lit.local.cfg'
 #
 # CHECK-ASEXEC-OUT: -- Test Suites --
 # CHECK-ASEXEC-OUT:   sub-suite - 2 tests

diff --git a/utils/lit/tests/unittest-adaptor.py b/utils/lit/tests/unittest-adaptor.py
index 7435dda41968bfc854f374a0d7a0131556401028..0848cd22baae858309cccc782b8ebe31f735b065 100644 (file)
--- a/utils/lit/tests/unittest-adaptor.py
+++ b/utils/lit/tests/unittest-adaptor.py
@@ -3,8 +3,8 @@
 # RUN: %{python} %s %{inputs}/unittest-adaptor 2> %t.err
 # RUN: FileCheck < %t.err %s
 #
-# CHECK: unittest-adaptor :: test-one.txt ... ok
-# CHECK: unittest-adaptor :: test-two.txt ... FAIL
+# CHECK-DAG: unittest-adaptor :: test-two.txt ... FAIL
+# CHECK-DAG: unittest-adaptor :: test-one.txt ... ok
 
 import unittest
 import sys

diff --git a/utils/lit/tests/xunit-output.py b/utils/lit/tests/xunit-output.py
index 87652290f47d4048f0ba145d261a4681d3e10d89..3f4939536379d57470a0c23ee4586e2b1ba6f517 100644 (file)
--- a/utils/lit/tests/xunit-output.py
+++ b/utils/lit/tests/xunit-output.py
@@ -5,6 +5,6 @@
 # CHECK: <?xml version="1.0" encoding="UTF-8" ?>
 # CHECK: <testsuites>
 # CHECK: <testsuite name='test-data' tests='1' failures='0'>
-# CHECK: <testcase classname='test-data.' name='metrics.ini' time='0.00'/>
+# CHECK: <testcase classname='test-data.test-data' name='metrics.ini' time='0.{{[0-9]+}}'/>
 # CHECK: </testsuite>
-# CHECK: </testsuites>
\ No newline at end of file
+# CHECK: </testsuites>