From: Dmitri Gribenko <gribozavr@gmail.com>
Date: Sun, 30 Dec 2012 14:51:03 +0000 (+0000)
Subject: Documentation: add a section to prevent spurious test failures like the one
X-Git-Url: http://demsky.eecs.uci.edu/git/?a=commitdiff_plain;h=0d80c9c8c78279124bb0d7e275c2b634ca640fef;p=oota-llvm.git

Documentation: add a section to prevent spurious test failures like the one
fixed in r171243.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171258 91177308-0d34-0410-b5e6-96231b3b80d8
---

diff --git a/docs/TestingGuide.rst b/docs/TestingGuide.rst
index f66cae1d148..f26d1bf60a7 100644
--- a/docs/TestingGuide.rst
+++ b/docs/TestingGuide.rst
@@ -262,6 +262,43 @@ recommended way to examine output to figure out if the test passes it using the
 :doc:`FileCheck tool <CommandGuide/FileCheck>`. The usage of ``grep`` in RUN
 lines is discouraged.
 
+Fragile tests
+-------------
+
+It is easy to write a fragile test that would fail spuriously if the tool being
+tested outputs a full path to the input file.  For example, :program:`opt` by
+default outputs a ``ModuleID``:
+
+.. code-block:: console
+
+  $ cat example.ll
+  define i32 @main() nounwind {
+      ret i32 0
+  }
+
+  $ opt -S /path/to/example.ll
+  ; ModuleID = '/path/to/example.ll'
+
+  define i32 @main() nounwind {
+      ret i32 0
+  }
+
+``ModuleID`` can unexpetedly match against ``CHECK`` lines.  For example:
+
+.. code-block:: llvm
+
+  ; RUN: opt -S %s | FileCheck
+
+  define i32 @main() nounwind {
+      ; CHECK-NOT: load
+      ret i32 0
+  }
+
+This test will fail if placed into a ``download`` directory.
+
+To make your tests robust, always use ``opt ... < %s`` in the RUN line.
+:program:`opt` does not output a ``ModuleID`` when input comes from stdin.
+
 The FileCheck utility
 ---------------------