Also on FreeBSD - e.g. http://llvm-amd64.freebsd.your.org:8010/builders/lldb-amd64-freebsd/builds/1593/steps/check-lldb.1/logs/stdio

ERROR: test_with_dwarf (TestStopHookMechanism.StopHookMechanismTestCase)
Test the stop-hook mechanism.

Traceback (most recent call last):
File "/data/buildslave/lldb-amd64-freebsd/src-llvm/tools/lldb/test/lldbtest.py", line 709, in wrapper
func(*args, **kwargs)
File "/data/buildslave/lldb-amd64-freebsd/src-llvm/tools/lldb/test/lldbtest.py", line 370, in wrapper
return func(self, *args, **kwargs)
File "/data/buildslave/lldb-amd64-freebsd/src-llvm/tools/lldb/test/functionalities/stop-hook/TestStopHookMechanism.py", line 27, in test_with_dwarf
self.stop_hook_firing()
File "/data/buildslave/lldb-amd64-freebsd/src-llvm/tools/lldb/test/functionalities/stop-hook/TestStopHookMechanism.py", line 79, in stop_hook_firing
child.expect_exact('at main.cpp:%d'%self.correct_step_line)
File "/data/buildslave/lldb-amd64-freebsd/src-llvm/tools/lldb/test/pexpect-2.4/pexpect.py", line 1348, in expect_exact
return self.expect_loop(searcher_string(pattern_list), timeout, searchwindowsize)
File "/data/buildslave/lldb-amd64-freebsd/src-llvm/tools/lldb/test/pexpect-2.4/pexpect.py", line 1414, in expect_loop
raise TIMEOUT (str(e) + '\n' + str(self))
TIMEOUT: Timeout exceeded in read_nonblocking().

Verified this still happens for us on top of tree svn r202456, on Ubuntu 12.04 LTS x86_64, with lldb built with gcc 4.8.2 and July 2013 libedit.

Marked this bug as expected failure (XFAIL) in lldb svn r202506. I ran the test 10 times in a row and it always failed.

This is part of a test code coverage analysis effort for Linux.

r202506 marked TestStopHookMultipleThreads as XFAIL on Linux.
r202507 marked TestStopHookMechanism as XFAIL on Linux.

Started consistently failing again on FreeBSD after r214319 - most likely it has just perturbed a long-standing timing issue.

http://lists.cs.uiuc.edu/pipermail/lldb-commits/Week-of-Mon-20140728/012113.html

It seems the expectedFailure* decorator does not work on this test as the pexpect timeout raises an exception that isn't caught in TestBase::tearDown, in lldbtest.py, so that the test returns ERROR instead of XFAIL.

ERROR: test_stop_hook_multiple_threads_with_dwarf (TestStopHookMultipleThreads.StopHookForMultipleThreadsTestCase)
Test that lldb stop-hook works for multiple threads.

Traceback (most recent call last):
File "/tank/emaste/src/llvm/tools/lldb/test/lldbtest.py", line 1639, in tearDown
Base.tearDown(self)
File "/tank/emaste/src/llvm/tools/lldb/test/lldbtest.py", line 1017, in tearDown
self.child.expect(pexpect.EOF)
File "/tank/emaste/src/llvm/tools/lldb/test/pexpect-2.4/pexpect.py", line 1316, in expect
return self.expect_list(compiled_pattern_list, timeout, searchwindowsize)
File "/tank/emaste/src/llvm/tools/lldb/test/pexpect-2.4/pexpect.py", line 1330, in expect_list
return self.expect_loop(searcher_re(pattern_list), timeout, searchwindowsize)
File "/tank/emaste/src/llvm/tools/lldb/test/pexpect-2.4/pexpect.py", line 1414, in expect_loop
raise TIMEOUT (str(e) + '\n' + str(self))

It seems reasonable that an XFAIL could also consume test errors (as opposed to test failures). If that is generally acceptable, we should be able to adjust the test framework to catch timeouts as (essentially) test failures for the purpose of the decorator handling of xfail.

On FreeBSD, the two tests decorated @​expectedFlakeyFreeBSD - TestStopHookMultipleThreads.py and TestStopHookMechanism.py -are passing consistently without any (FreeBSD) decorator.

*** Bug llvm/llvm-bugzilla-archive#20352 has been marked as a duplicate of this bug. ***

mentioned in issue llvm/llvm-bugzilla-archive#20352