SemaCXX/constant-expression-cxx11.cpp test fails on 32-bit x86 due to stack exhaustion #21009
Comments
|
The first thing to check is whether this is a simple stack overflow: check whether $rsp in frame 0 is at the bottom of the containing mmap'd region, and see if increasing your stack ulimit makes the problem disappear. |
|
(We have seen similar issues in template instantiation in the past; I seem to recall that x86 freebsd has particularly low default stack limits.) |
Hmm, this actually seems to be the cause: (gdb) frame 3201 The difference between the esp values is 2,092,200 bytes, which is just below 2 MiB. According to this recent discussion: https://lists.freebsd.org/pipermail/freebsd-current/2014-August/051585.html the main thread stack size is quite limited, 4MiB on amd64, and just 2MiB on i386! And indeed, the program seems to hit the stack redzone (the page at 0xbf9fe000-0xbf9ff000 with '---' permissions): $ procstat -v 48126 I'm not sure if there currently is another way to tweak the default main thread size, except modifying FreeBSD's libthr sources, and recompiling the library. :-/ I did it though, and when I doubled the size to 4MiB, the testcase succeeded! I guess I should now go and put in a FreeBSD PR for this, but in the mean time, isn't there some way to limit the stack recursion in clang, or in the testcase? I'd say more than 3000 stack frames is a bit much for such a simple expression... |
|
IIRC, the point of this test was to check for an iterator invalidation issue caused by having sufficiently many OpaqueValueExpr evaluations simultaneously in-flight, so I don't think we can reduce the recursion depth here without risking neutering the test. Do you see this for optimized builds of clang, or only for -O0 builds? At least historically, LLVM-generated -O0 code has been very stack-hungry. |
This happens with both optimized and unoptimized builds; I just tried it with a Debug+Asserts build. (Btw, in my experience, optimized builds can also consume quite a bit of stack, due to aggressive inlining. But this really depends on the code.) One thing I just noticed in the 'working' case, is that it says "constexpr evaluation exceeded maximum depth of 512 calls". If I add -fconstexpr-depth 256 to the compile options, it doesn't crash with the default main thread stack size of 2MB. The maximum I can get it to is -fconstexpr-depth 466. Maybe there are some other things that could be done to reduce the stack size of the constexpr evaluation, but it's hard to tell without more data. I'll see if I can lift something from the stack traces. |
This is really weird: this test passes for me with -fconstexpr-depth 201 (LastNonzero should recurse 200 times, because 'arr3' has 200 elements). So perhaps there are really two separate issues here:
It certainly looks like there's a bug here beyond the stack exhaustion, but without a way to reproduce this, it'll be hard for me to figure out what's causing #1. Can you do some investigation for me?
|
Actually, I already see the 'infinite' recursion with just this minimized version: template Though creduce has mangled the test so that it never halts anyway. :)
Yes, this is the part I can't explain. It seems to depend on random factors whether this occurs or not. For example, I just did a complete rebuild of 3.5.0rc2 for Release+Asserts+Debug, and then the problem does not occur with the constant-expression-cxx11.cpp testcase, only with the minimized version. I suspect there is a code fragment somewhere in there with undefined behaviour, and this could cause the actual results depend on whatever happened to be on the stack in each of the hundreds of functions called...
As shown by the minimize testcase, this limit is at a depth of roughly 450.
It's definitely arr3; when I #if 0'd out arr1 and arr2 completely, no crash was reproducible, but with the reverse it easily crashed.
I'm having trouble reproducing this with a build with debug symbols at the moment. I'm going to try a few things, and hope that it works. Otherwise, I will have to make do with the build I had without debug symbols... |
Extended Description
During clang 3.5.0-rc2 testing, I ran into the following failure, which only occurs on 32-bit x86 (in this case, i386-unknown-freebsd10):
FAIL: Clang :: SemaCXX/constant-expression-cxx11.cpp (5186 of 18744)
******************** TEST 'Clang :: SemaCXX/constant-expression-cxx11.cpp' FAILED ********************
Script:
/home/dim/llvm-3.5.0/rc2/Phase3/Release/llvmCore-3.5.0-rc2.obj/Release/bin/clang -cc1 -internal-isystem /home/dim/llvm-3.5.0/rc2/Phase3/Release/llvmCore-3.5.0-rc2.obj/Release/bin/../lib/clang/3.5.0/include -triple i686-linux -Wno-string-plus-int -Wno-pointer-arith -Wno-zero-length-array -fsyntax-only -fcxx-exceptions -verify -std=c++11 -pedantic /home/dim/llvm-3.5.0/rc2/cfe.src/test/SemaCXX/constant-expression-cxx11.cpp -Wno-comment -Wno-tautological-pointer-compare -Wno-bool-conversion
Exit Code: 132
This turns out to be a stack overflow, leading to either a segfault, or an illegal instruction:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 2ac03080 (LWP 100095)]
0x091e5e34 in clang::StmtVisitorBase<clang::make_const_ptr, (anonymous namespace)::LValueExprEvaluator, bool>::Visit (this=0xbf9ff6c8, S=0x2ac9cb3c) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/../../include/clang/AST/StmtVisitor.h:39
39 RetTy Visit(PTR(Stmt) S) {
(gdb) bt
#0 0x091e5e34 in clang::StmtVisitorBase<clang::make_const_ptr, (anonymous namespace)::LValueExprEvaluator, bool>::Visit (this=0xbf9ff6c8, S=0x2ac9cb3c) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/../../include/clang/AST/StmtVisitor.h:39
#1 0x09197564 in EvaluateLValue (E=, Result=..., Info=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:4395
#2 0x091ce138 in VisitCastExpr (E=, this=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:4130
#3 (anonymous namespace)::PointerExprEvaluator::VisitCastExpr (this=0xbf9ff6c8, E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:4818
#4 0x091cc036 in VisitExplicitCastExpr (this=0xbfa00010, S=) at /home/dim/obj/llvm-215430-1/tools/clang/lib/AST/../../include/clang/AST/StmtNodes.inc:329
#5 VisitCStyleCastExpr (this=, S=) at /home/dim/obj/llvm-215430-1/tools/clang/lib/AST/../../include/clang/AST/StmtNodes.inc:333
#6 clang::StmtVisitorBase<clang::make_const_ptr, (anonymous namespace)::PointerExprEvaluator, bool>::Visit (this=, S=0x2ac9cb3c) at /home/dim/obj/llvm-215430-1/tools/clang/lib/AST/../../include/clang/AST/StmtNodes.inc:333
#7 0x091b064a in EvaluatePointer (E=, Result=..., Info=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:4691
#8 0x0919b2c0 in Evaluate (Result=..., Info=..., E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:8002
#9 0x0919d6fb in EvaluateArgs (ArgValues=..., Info=..., Args=..., ArgValues=..., Info=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:3601
#10 HandleFunctionCall (CallLoc=..., Callee=0xbf9ff6c8, This=0x2ac9cb3c, Args=..., Body=0x2ac9cb3c, Info=..., Result=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:3618
#11 0x091aa103 in VisitCallExpr (this=, E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:4060
#12 (anonymous namespace)::IntExprEvaluator::VisitCallExpr (this=, E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:6012
#13 0x091a2421 in clang::StmtVisitorBase<clang::make_const_ptr, (anonymous namespace)::IntExprEvaluator, bool>::Visit (this=, S=) at /home/dim/obj/llvm-215430-1/tools/clang/lib/AST/../../include/clang/AST/StmtNodes.inc:293
#14 0x0919b1ea in Evaluate (Result=..., Info=..., E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:7998
#15 0x091a091d in EvaluateStmt (Result=..., Info=..., S=, Case=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:3362
#16 0x091a0652 in EvaluateStmt (Result=..., Info=..., S=, Case=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:3372
#17 0x0919d988 in HandleFunctionCall (CallLoc=..., Callee=0xbf9ff6c8, This=, Args=..., Body=0x2ac9cb3c, Info=..., Result=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:3646
#18 0x091aa103 in VisitCallExpr (this=, E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:4060
[... repeats thousands of times ...]
#3171 0x0919b1ea in Evaluate (Result=..., Info=..., E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:7998
#3172 0x091a091d in EvaluateStmt (Result=..., Info=..., S=, Case=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:3362
#3173 0x091a0652 in EvaluateStmt (Result=..., Info=..., S=, Case=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:3372
#3174 0x0919d988 in HandleFunctionCall (CallLoc=..., Callee=0xbf9ff6c8, This=, Args=..., Body=0x2ac9cb3c, Info=..., Result=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:3646
#3175 0x091aa103 in VisitCallExpr (this=, E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:4060
#3176 (anonymous namespace)::IntExprEvaluator::VisitCallExpr (this=, E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:6012
#3177 0x091a2421 in clang::StmtVisitorBase<clang::make_const_ptr, (anonymous namespace)::IntExprEvaluator, bool>::Visit (this=, S=) at /home/dim/obj/llvm-215430-1/tools/clang/lib/AST/../../include/clang/AST/StmtNodes.inc:293
#3178 0x0919b1ea in Evaluate (Result=..., Info=..., E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:7998
#3179 0x091af8f5 in EvaluateAsBooleanCondition (Result=, Info=..., E=, Result=, Info=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:1477
#3180 (anonymous namespace)::IntExprEvaluator::VisitCastExpr (this=, E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:7194
#3181 0x091a23d3 in VisitExplicitCastExpr (this=, S=) at /home/dim/obj/llvm-215430-1/tools/clang/lib/AST/../../include/clang/AST/StmtNodes.inc:329
#3182 VisitCStyleCastExpr (this=, S=) at /home/dim/obj/llvm-215430-1/tools/clang/lib/AST/../../include/clang/AST/StmtNodes.inc:333
#3183 clang::StmtVisitorBase<clang::make_const_ptr, (anonymous namespace)::IntExprEvaluator, bool>::Visit (this=, S=) at /home/dim/obj/llvm-215430-1/tools/clang/lib/AST/../../include/clang/AST/StmtNodes.inc:333
#3184 0x0919b1ea in Evaluate (Result=..., Info=..., E=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:7998
#3185 0x09196c94 in EvaluateAsRValue (Info=..., E=, Result=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:8087
#3186 0x09196a33 in clang::Expr::EvaluateAsRValue (this=, Result=..., Ctx=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/AST/ExprConstant.cpp:8142
#3187 0x0895f5ff in clang::Sema::VerifyIntegerConstantExpression (this=, E=, Result=, Diagnoser=..., AllowFold=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Sema/SemaExpr.cpp:11161
#3188 0x0895d02d in clang::Sema::VerifyIntegerConstantExpression (this=, E=, Result=, DiagID=, AllowFold=, this=, E=, Result=, DiagID=, AllowFold=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Sema/SemaExpr.cpp:11068
#3189 0x088c7563 in clang::Sema::BuildStaticAssertDeclaration (this=0x2ac98000, StaticAssertLoc=..., AssertExpr=0x2, AssertMessage=, RParenLoc=..., Failed=<error reading variable: Cannot access memory at address 0x1>) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Sema/SemaDeclCXX.cpp:11519
#3190 0x088c746e in clang::Sema::ActOnStaticAssertDeclaration (this=, StaticAssertLoc=..., AssertExpr=, AssertMessageExpr=, RParenLoc=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Sema/SemaDeclCXX.cpp:11500
#3191 0x086f4885 in clang::Parser::ParseStaticAssertDeclaration (this=0x2ac4e600, DeclEnd=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Parse/ParseDeclCXX.cpp:725
#3192 0x086d954c in clang::Parser::ParseDeclaration (this=0x2ac4e600, Stmts=..., Context=, DeclEnd=..., attrs=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Parse/ParseDecl.cpp:1338
#3193 0x086c71eb in clang::Parser::ParseExternalDeclaration (this=0x2ac4e600, attrs=..., DS=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Parse/Parser.cpp:701
#3194 0x086c6682 in clang::Parser::ParseTopLevelDecl (this=0x2ac4e600, Result=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Parse/Parser.cpp:559
#3195 0x086c3286 in clang::ParseAST (S=..., PrintStats=, SkipFunctionBodies=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Parse/ParseAST.cpp:145
#3196 0x082d54e7 in clang::ASTFrontendAction::ExecuteAction (this=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Frontend/FrontendAction.cpp:513
#3197 0x082d4dfc in clang::FrontendAction::Execute (this=0xbf9ff6c8) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Frontend/FrontendAction.cpp:415
#3198 0x082a351f in clang::CompilerInstance::ExecuteAction (this=, Act=...) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/Frontend/CompilerInstance.cpp:810
#3199 0x082851b5 in clang::ExecuteCompilerInvocation (Clang=0x2ac32080) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:222
#3200 0x0827a29d in cc1_main (ArgBegin=, ArgEnd=, Argv0=, MainAddr=) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/tools/driver/cc1_main.cpp:112
#3201 0x08282ae6 in main (argc_=-1077942272, argv_=0x827fcc0 <GetExecutablePath(char const*, bool)>) at /home/dim/llvm-3.5.0/llvm.src/tools/clang/tools/driver/driver.cpp:318
The specific test that it explodes on is this one from test/SemaCXX/constant-expression-cxx11.cpp:
namespace RecursiveOpaqueExpr {
template
constexpr auto LastNonzero(Iter p, Iter q) -> decltype(+*p) {
return p != q ? (LastNonzero(p+1, q) ?: *p) : 0; // expected-warning {{GNU}}
}
constexpr int arr1[] = { 1, 0, 0, 3, 0, 2, 0, 4, 0, 0 };
static_assert(LastNonzero(begin(arr1), end(arr1)) == 4, "");
constexpr int arr2[] = { 1, 0, 0, 3, 0, 2, 0, 4, 0, 5 };
static_assert(LastNonzero(begin(arr2), end(arr2)) == 5, "");
constexpr int arr3[] = {
1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0,
1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0,
1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0,
1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0,
1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0,
1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0,
1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
static_assert(LastNonzero(begin(arr3), end(arr3)) == 2, "");
}
I reduced this to the following minimal testcase, constant-expression-cxx11-minimal.cpp:
template
constexpr auto LastNonzero(Iter p, Iter q) -> decltype(0) {
return LastNonzero(p, q)
}
int arr3[]{};
static_assert(LastNonzero(arr3, arr3) , ""
Compiling this with "-cc1 -triple i686-linux -fsyntax-only -verify -std=c++11" leads to the crash.
I had to go back quite far in the Subversion history to find the commit that introduces this crash:
http://llvm.org/viewvc/llvm-project?view=revision&revision=187025
Before this commit, using clang trunk r187024:
clang -cc1 -triple i686-linux -fsyntax-only -verify -std=c++11 constant-expression-cxx11-minimal.cpp
error: no expected directives found: consider use of 'expected-no-diagnostics'
error: 'error' diagnostics seen but not expected:
File constant-expression-cxx11-minimal.cpp Line 3: expected ';' after return statement
File constant-expression-cxx11-minimal.cpp Line 6: expected ')'
File constant-expression-cxx11-minimal.cpp Line 6: expected ';' after static_assert
File constant-expression-cxx11-minimal.cpp Line 6: static_assert expression is not an integral constant expression
error: 'note' diagnostics seen but not expected:
File constant-expression-cxx11-minimal.cpp Line 6: to match this '('
File constant-expression-cxx11-minimal.cpp Line 3: constexpr evaluation exceeded maximum depth of 512 calls
File constant-expression-cxx11-minimal.cpp Line 3: in call to 'LastNonzero(&arr3[0], &arr3[0])'
File constant-expression-cxx11-minimal.cpp Line 3: in call to 'LastNonzero(&arr3[0], &arr3[0])'
File constant-expression-cxx11-minimal.cpp Line 3: in call to 'LastNonzero(&arr3[0], &arr3[0])'
File constant-expression-cxx11-minimal.cpp Line 3: in call to 'LastNonzero(&arr3[0], &arr3[0])'
File constant-expression-cxx11-minimal.cpp Line 3: in call to 'LastNonzero(&arr3[0], &arr3[0])'
File constant-expression-cxx11-minimal.cpp Line 3: (skipping 502 calls in backtrace; use -fconstexpr-backtrace-limit=0 to see all)
File constant-expression-cxx11-minimal.cpp Line 3: in call to 'LastNonzero(&arr3[0], &arr3[0])'
File constant-expression-cxx11-minimal.cpp Line 3: in call to 'LastNonzero(&arr3[0], &arr3[0])'
File constant-expression-cxx11-minimal.cpp Line 3: in call to 'LastNonzero(&arr3[0], &arr3[0])'
File constant-expression-cxx11-minimal.cpp Line 3: in call to 'LastNonzero(&arr3[0], &arr3[0])'
File constant-expression-cxx11-minimal.cpp Line 6: in call to 'LastNonzero(&arr3[0], &arr3[0])'
18 errors generated.
After this commit, using clang trunk r187025:
clang -cc1 -triple i686-linux -fsyntax-only -verify -std=c++11 constant-expression-cxx11-minimal.cpp
Illegal instruction (core dumped)
Note that the problem is not always repeatable: different compiliation options and/or environments seem to make this crash disappear or reappear randomly. I have encountered it a few times in the last months, but it always went away magically, only to return later.
I suspect there is some sort of undefined behaviour going on, either in the code added by r187025, or somewhere else...
The text was updated successfully, but these errors were encountered: