Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TestDAP_commands.py flakiness due to lldb-dap exit crashes #81686

Closed
rupprecht opened this issue Feb 13, 2024 · 1 comment · Fixed by #83162
Closed

TestDAP_commands.py flakiness due to lldb-dap exit crashes #81686

rupprecht opened this issue Feb 13, 2024 · 1 comment · Fixed by #83162
Labels
lldb

Comments

@rupprecht
Copy link
Collaborator

Context: https://lab.llvm.org/buildbot/#/builders/68/builds/68776

The test has some exit commands:

    "exitCommands": [
      "?settings set target.show-hex-variable-values-with-leading-zeroes false",
      "settings set target.show-hex-variable-values-with-leading-zeroes true"
    ],
    "stopCommands": [
      "?settings set target.show-hex-variable-values-with-leading-zeroes false",
      "settings set target.show-hex-variable-values-with-leading-zeroes true"
    ],
    "terminateCommands": [
      "?settings set target.show-hex-variable-values-with-leading-zeroes false",
      "settings set target.show-hex-variable-values-with-leading-zeroes true"
    ]

The test case itself passes, but tear down occasionally fails because we fail to get a disconnect response back. The DAP log shows this crash causing the dap server to shutdown before it can respond to the disconnect:

--> 
Content-Length: 88

{
  "arguments": {
    "terminateDebuggee": true
  },
  "command": "disconnect",
  "seq": 3,
  "type": "request"
}
<-- 
Content-Length: 184

{
  "body": {
    "category": "console",
    "output": "Running exitCommands:\n(lldb) settings set target.show-hex-variable-values-with-leading-zeroes true\n"
  },
  "event": "output",
  "seq": 0,
  "type": "event"
}
<-- 
Content-Length: 63

{
  "body": {
    "exitCode": 9
  },
  "event": "exited",
  "seq": 0,
  "type": "event"
}
<-- 
Content-Length: 405

{
  "body": {
    "category": "stderr",
    "output": "assert.h assertion failed at llvm-project/lldb/source/Interpreter/CommandObject.cpp:153 in bool lldb_private::CommandObject::CheckRequirements(CommandReturnObject &): !m_exe_ctx.GetTargetPtr()\n"
  },
  "event": "output",
  "seq": 0,
  "type": "event"
}
...
<-- 
Content-Length: 430

{
  "body": {
    "category": "stderr",
    "output": "PLEASE submit a bug report ....\nStack dump:\n0.\tProgram arguments: lldb-dap\n1.\tHandleCommand(command = \"settings set target.show-hex-variable-values-with-leading-zeroes false\")\u0000\n"
  },
  "event": "output",
  "seq": 0,
  "type": "event"
}

Aside: the buildbots do not enable this DAP logging, which would make debugging buildbot errors much easier.
@rupprecht rupprecht added the lldb label Feb 13, 2024
@llvmbot
Copy link
Member

llvmbot commented Feb 13, 2024

@llvm/issue-subscribers-lldb

Author: Jordan Rupprecht (rupprecht)

Context: https://lab.llvm.org/buildbot/#/builders/68/builds/68776

The test has some exit commands:

    "exitCommands": [
      "?settings set target.show-hex-variable-values-with-leading-zeroes false",
      "settings set target.show-hex-variable-values-with-leading-zeroes true"
    ],
    "stopCommands": [
      "?settings set target.show-hex-variable-values-with-leading-zeroes false",
      "settings set target.show-hex-variable-values-with-leading-zeroes true"
    ],
    "terminateCommands": [
      "?settings set target.show-hex-variable-values-with-leading-zeroes false",
      "settings set target.show-hex-variable-values-with-leading-zeroes true"
    ]

The test case itself passes, but tear down occasionally fails because we fail to get a disconnect response back. The DAP log shows this crash causing the dap server to shutdown before it can respond to the disconnect:

--&gt; 
Content-Length: 88

{
  "arguments": {
    "terminateDebuggee": true
  },
  "command": "disconnect",
  "seq": 3,
  "type": "request"
}
&lt;-- 
Content-Length: 184

{
  "body": {
    "category": "console",
    "output": "Running exitCommands:\n(lldb) settings set target.show-hex-variable-values-with-leading-zeroes true\n"
  },
  "event": "output",
  "seq": 0,
  "type": "event"
}
&lt;-- 
Content-Length: 63

{
  "body": {
    "exitCode": 9
  },
  "event": "exited",
  "seq": 0,
  "type": "event"
}
&lt;-- 
Content-Length: 405

{
  "body": {
    "category": "stderr",
    "output": "assert.h assertion failed at llvm-project/lldb/source/Interpreter/CommandObject.cpp:153 in bool lldb_private::CommandObject::CheckRequirements(CommandReturnObject &amp;): !m_exe_ctx.GetTargetPtr()\n"
  },
  "event": "output",
  "seq": 0,
  "type": "event"
}
...
&lt;-- 
Content-Length: 430

{
  "body": {
    "category": "stderr",
    "output": "PLEASE submit a bug report ....\nStack dump:\n0.\tProgram arguments: lldb-dap\n1.\tHandleCommand(command = \"settings set target.show-hex-variable-values-with-leading-zeroes false\")\u0000\n"
  },
  "event": "output",
  "seq": 0,
  "type": "event"
}

Aside: the buildbots do not enable this DAP logging, which would make debugging buildbot errors much easier.

@rupprecht rupprecht mentioned this issue Feb 15, 2024
Merged
@rupprecht rupprecht mentioned this issue Feb 27, 2024
Merged
rupprecht added a commit that referenced this issue Feb 27, 2024
@rupprecht
[lldb][dap] Avoid concurrent HandleCommand calls (#83162)

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode
Loading
Loading status checks…
e427e93 
The `EventThreadFunction` can end up calling `HandleCommand`
concurrently with the main request processing thread. The underlying API
does not appear to be thread safe, so add a narrowly scoped mutex lock
to prevent calling it in this place from more than one thread.

Fixes #81686. Prior to this, TestDAP_launch.py is 4% flaky. After, it
passes in 1000 runs.
SquallATF pushed a commit to SquallATF/llvm-project that referenced this issue Jun 30, 2024
@rupprecht @SquallATF
[lldb][dap] Avoid concurrent HandleCommand calls (llvm#83162)
8b1956b 
The `EventThreadFunction` can end up calling `HandleCommand`
concurrently with the main request processing thread. The underlying API
does not appear to be thread safe, so add a narrowly scoped mutex lock
to prevent calling it in this place from more than one thread.

Fixes llvm#81686. Prior to this, TestDAP_launch.py is 4% flaky. After, it
passes in 1000 runs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
lldb
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[lldb][dap] Avoid concurrent HandleCommand calls rupprecht/llvm-project
2 participants
@rupprecht @llvmbot