Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lldb fails to stack unwind gold linker exes that assert #19143

Closed
llvmbot opened this issue Feb 8, 2014 · 10 comments
Closed

lldb fails to stack unwind gold linker exes that assert #19143

llvmbot opened this issue Feb 8, 2014 · 10 comments
Labels
bugzilla Issues migrated from bugzilla lldb

Comments

@llvmbot
Copy link
Collaborator

llvmbot commented Feb 8, 2014

Bugzilla Link 18769
Resolution FIXED
Resolved on Feb 12, 2014 01:33
Version unspecified
OS Linux
Reporter LLVM Bugzilla Contributor

Extended Description

I'm hitting an assertion in lldb-gdbserver. On Ubuntu 12.04 x86_64, using the gold linker 'sudo apt-get install binutils-gold', with a cmake/ninja build, I am not able to get a valid backtrace out of lldb for lldb-gdbserver when it generates

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 8, 2014

Whoops - wasn't done entering info.

Here is lldb debugging this core:

tfiala@tfiala2:~/lldb/hello/linux-x86_64$ ~/lldb/git/llgs/build/bin/lldb -c core -- ~/lldb/git/llgs/build/bin/lldb-gdbserver
Core file '/usr/local/google/home/tfiala/lldb/hello/linux-x86_64/core' (x86_64) was loaded.
Process 0 stopped

  • thread #​1: tid = 0, 0x00007ffba3bb2425, name = 'lldb-gdbserver', stop reason = signal SIGABRT
    frame #​0: 0x00007ffba3bb2425
    -> 0x7ffba3bb2425: addb %al, (%rax)
    0x7ffba3bb2427: addb %al, (%rax)
    0x7ffba3bb2429: addb %al, (%rax)
    0x7ffba3bb242b: addb %al, (%rax)
    (lldb) bt
  • thread #​1: tid = 0, 0x00007ffba3bb2425, name = 'lldb-gdbserver', stop reason = signal SIGABRT
    • frame #​0: 0x00007ffba3bb2425
      (lldb)

Here is gdb debugging this core:

tfiala@tfiala2:~/lldb/hello/linux-x86_64$ gdb ~/lldb/git/llgs/build/bin/lldb-gdbserver core
GNU gdb (GDB) 7.6-gg23
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux".

<http://go/gdb-home FAQ: http://go/gdb-faq Email: gdb-team IRC: gdb>
Reading symbols from /usr/local/google/home/tfiala/lldb/git/llgs/build/bin/lldb-gdbserver...done.
[New LWP 3005]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/grte/v3/lib64/libthread_db.so.1".
Core was generated by `/usr/local/google/home/tfiala/lldb/git/llgs/build/bin/lldb-gdbserver -l /usr/lo'.
Program terminated with signal 6, Aborted.
#​0 0x00007ffba3bb2425 in __GI_raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#​0 0x00007ffba3bb2425 in __GI_raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#​1 0x00007ffba3bb5b8b in __GI_abort () at abort.c:91
#​2 0x00007ffba3bab0ee in __assert_fail_base (fmt=, assertion=0x7ffba9906478 "g_swig_init_callback != __null",
file=0x7ffba9906420 "/mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp", line=,
function=) at assert.c:94
#​3 0x00007ffba3bab192 in __GI___assert_fail (assertion=0x7ffba9906478 "g_swig_init_callback != __null",
file=0x7ffba9906420 "/mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp", line=2539,
function=0x7ffba9906880 <lldb_private::ScriptInterpreterPython::InitializePrivate()::PRETTY_FUNCTION> "static void lldb_private::ScriptInterpreterPython::InitializePrivate()") at assert.c:103
#​4 0x00007ffba74e155f in lldb_private::ScriptInterpreterPython::InitializePrivate ()
at /mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp:2539
#​5 0x00007ffba719de09 in lldb_private::Initialize () at /mnt/ssd/work/git/llgs/llvm/tools/lldb/source/lldb.cpp:124
#​6 0x00007ffba7369ba5 in lldb_private::Debugger::Initialize (load_plugin_callback=0x0)
at /mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Core/Debugger.cpp:371
#​7 0x000000000041131f in initialize_lldb_gdbserver () at /mnt/ssd/work/git/llgs/llvm/tools/lldb/tools/lldb-gdbserver/lldb-gdbserver.cpp:117
#​8 0x0000000000411456 in main (argc=9, argv=0x7fffdb6fea98) at /mnt/ssd/work/git/llgs/llvm/tools/lldb/tools/lldb-gdbserver/lldb-gdbserver.cpp:143
(gdb) quit

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 8, 2014

Here's what I've got so far.

~/lldb/git/llgs/build-gold/bin/lldb -c core -- ~/lldb/git/llgs/build-gold/bin/lldb-gdbserver

Results:

(lldb) image lookup -a $pc
(lldb)

nothing?

(lldb) step 0
error: Thread index 0 is out of range (valid values are 0 - 1).

no threads - ??

(lldb) thread list
Process 0 stopped

  • thread #​1: tid = 0, 0x00007ff3b6daf425, name = 'lldb-gdbserver', stop reason = signal SIGABRT
    (lldb)

it seems to know about a thread. The one that did an assert.

(lldb) image list
[ 0] B68F8BE0-0000-0000-0000-000000000000 /usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver
(lldb)

hey - it knows about the exe. That's a start.

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 8, 2014

Info from gdb on the same core, this time including '(gdb) info shared'. Perhaps we're failing to get all the module info loaded.

tfiala@tfiala2:~/lldb/hello/linux-x86_64$ gdb ~/lldb/git/llgs/build-gold/bin/lldb-gdbserver core
GNU gdb (GDB) 7.6-gg23
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux".

<http://go/gdb-home FAQ: http://go/gdb-faq Email: gdb-team IRC: gdb>
Reading symbols from /usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver...done.
[New LWP 28888]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/grte/v3/lib64/libthread_db.so.1".
Core was generated by `/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver -l /u'.
Program terminated with signal 6, Aborted.
#​0 0x00007ff3b6daf425 in __GI_raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#​0 0x00007ff3b6daf425 in __GI_raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#​1 0x00007ff3b6db2b8b in __GI_abort () at abort.c:91
#​2 0x00007ff3b6da80ee in __assert_fail_base (fmt=, assertion=0x7ff3bcb03478 "g_swig_init_callback != __null",
file=0x7ff3bcb03420 "/mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp", line=,
function=) at assert.c:94
#​3 0x00007ff3b6da8192 in __GI___assert_fail (assertion=0x7ff3bcb03478 "g_swig_init_callback != __null",
file=0x7ff3bcb03420 "/mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp", line=2539,
function=0x7ff3bcb03880 <lldb_private::ScriptInterpreterPython::InitializePrivate()::PRETTY_FUNCTION> "static void lldb_private::ScriptInterpreterPython::InitializePrivate()") at assert.c:103
#​4 0x00007ff3ba6de55f in lldb_private::ScriptInterpreterPython::InitializePrivate ()
at /mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp:2539
#​5 0x00007ff3ba39ae09 in lldb_private::Initialize () at /mnt/ssd/work/git/llgs/llvm/tools/lldb/source/lldb.cpp:124
#​6 0x00007ff3ba566ba5 in lldb_private::Debugger::Initialize (load_plugin_callback=0x0)
at /mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Core/Debugger.cpp:371
#​7 0x000000000041132f in initialize_lldb_gdbserver () at /mnt/ssd/work/git/llgs/llvm/tools/lldb/tools/lldb-gdbserver/lldb-gdbserver.cpp:117
#​8 0x0000000000411466 in main (argc=9, argv=0x7fff99e85fe8) at /mnt/ssd/work/git/llgs/llvm/tools/lldb/tools/lldb-gdbserver/lldb-gdbserver.cpp:143
(gdb) info shared
From To Syms Read Shared Object Library
0x00007ff3ba39a9c0 0x00007ff3bca42f80 Yes /mnt/ssd/work/git/llgs/build-gold/lib/liblldb.so.3.5
0x00007ff3b8cec9b0 0x00007ff3b8d01b88 Yes () /lib/x86_64-linux-gnu/libncurses.so.5
0x00007ff3b8acc1d0 0x00007ff3b8ad62f8 Yes (
) /lib/x86_64-linux-gnu/libtinfo.so.5
0x00007ff3b88bc990 0x00007ff3b88bdeb8 Yes () /usr/lib/x86_64-linux-gnu/libpanel.so.5
0x00007ff3b86b61e0 0x00007ff3b86b96b8 Yes /lib/x86_64-linux-gnu/librt.so.1
0x00007ff3b84b0de0 0x00007ff3b84b1918 Yes /lib/x86_64-linux-gnu/libdl.so.2
0x00007ff3b8298740 0x00007ff3b82a4358 Yes /lib/x86_64-linux-gnu/libpthread.so.0
0x00007ff3b807e050 0x00007ff3b808a528 Yes (
) /lib/x86_64-linux-gnu/libz.so.1
0x00007ff3b7e56f50 0x00007ff3b7e6c350 Yes /usr/local/google/home/tfiala/lldb/tools/libedit/linux-x86_64/lib/libedit.so.0
0x00007ff3b7996c70 0x00007ff3b7adda48 Yes () /usr/lib/libpython2.7.so.1.0
0x00007ff3b76a5bb0 0x00007ff3b77081da Yes /usr/local/gcc/gcc-current/lib64/libstdc++.so.6
0x00007ff3b7354570 0x00007ff3b73bcfe8 Yes /lib/x86_64-linux-gnu/libm.so.6
0x00007ff3b713ba10 0x00007ff3b714b0c8 Yes /usr/local/gcc/gcc-current/lib64/libgcc_s.so.1
0x00007ff3b6d97fa0 0x00007ff3b6edc5d0 Yes /lib/x86_64-linux-gnu/libc.so.6
0x00007ff3b6b59690 0x00007ff3b6b6c798 Yes (
) /usr/lib/x86_64-linux-gnu/libedit.so.2
0x00007ff3becd1af0 0x00007ff3becebeba Yes /lib64/ld-linux-x86-64.so.2
0x00007ff3b6906e10 0x00007ff3b693ac58 Yes () /lib/x86_64-linux-gnu/libssl.so.1.0.0
0x00007ff3b65790c0 0x00007ff3b666fdf8 Yes (
) /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
0x00007ff3b6316e20 0x00007ff3b63176a8 Yes /lib/x86_64-linux-gnu/libutil.so.1
0x00007ff3b610c110 0x00007ff3b6113528 Yes () /lib/x86_64-linux-gnu/libbsd.so.0
(
): Shared library is missing debugging information.
(gdb)

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 8, 2014

Full logging on the startup:

tfiala@tfiala2:~/lldb/hello/linux-x86_64$ ~/lldb/git/llgs/build-gold/bin/lldb
(lldb) log enable linux all
(lldb) log enable lldb all
(lldb) log enable dwarf all
main-thread Processing command: log enable dwarf all
main-thread HandleCommand, cmd_obj : 'log enable'
main-thread HandleCommand, revised_command_line: 'log enable dwarf all'
main-thread HandleCommand, wants_raw_input:'False'
main-thread HandleCommand, command line after removing command name(s): 'dwarf all'
main-thread HandleCommand, command succeeded
(lldb) target create -c ./core ~/lldb/git/llgs/build-gold/bin/lldb-gdbserver
main-thread Processing command: target create -c ./core ~/lldb/git/llgs/build-gold/bin/lldb-gdbserver
main-thread HandleCommand, cmd_obj : 'target create'
main-thread HandleCommand, revised_command_line: 'target create -c ./core ~/lldb/git/llgs/build-gold/bin/lldb-gdbserver'
main-thread HandleCommand, wants_raw_input:'False'
main-thread HandleCommand, command line after removing command name(s): '-c ./core ~/lldb/git/llgs/build-gold/bin/lldb-gdbserver'
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(file="/mnt/ssd/work/git/llgs/build-gold/bin/lldb-gdbserver-3.5", offset=0x0, length=0x1ecae8, writeable=0
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(fd=3, offset=0x0, length=0x1ecae8, writeable=0, fd_is_file=1)
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec() m_mmap_addr = 0x7f3b70059000, m_mmap_size = 2018024, error = (null)
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(file="/mnt/ssd/work/git/llgs/build-gold/bin/lldb-gdbserver-3.5", offset=0x0, length=0xffffffffffffffff, writeable=0
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(fd=3, offset=0x0, length=0xffffffffffffffff, writeable=0, fd_is_file=1)
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec() m_mmap_addr = 0x7f3b6be13000, m_mmap_size = 2018024, error = (null)
main-thread DataBufferMemoryMap::Clear() m_mmap_addr = 0x7f3b70059000, m_mmap_size = 2018024
main-thread DataBufferMemoryMap::Clear() m_mmap_addr = 0x7f3b6be13000, m_mmap_size = 2018024
main-thread 0x279a7e0 Module::Module((x86_64) '/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver')
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(file="/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver", offset=0x0, length=0x1ecae8, writeable=0
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(fd=3, offset=0x0, length=0x1ecae8, writeable=0, fd_is_file=1)
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec() m_mmap_addr = 0x7f3b70059000, m_mmap_size = 2018024, error = (null)
main-thread 0x279a9d0 ObjectFile::ObjectFile() module = 0x279a7e0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver), file = /usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver, file_offset = 0x00000000, size = 2018024
main-thread 0x279ad28 Broadcaster::Broadcaster("lldb.target")
main-thread 0x23f88e0 Listener::StartListeningForEvents (broadcaster = 0x279ad28, mask = 0x00000001) acquired_mask = 0x00000001 for lldb.Debugger
main-thread 0x279ad00 Target::Target()
main-thread Target::Target created with architecture x86_64 (x86_64--linux)
main-thread 0x279ad28 Broadcaster("lldb.target")::BroadcastEvent (event_sp = {0x279c690 Event: broadcaster = 0x279ad28 (lldb.target), type = 0x00000002 (modules-loaded), data = }, unique =0) hijack = (nil)
main-thread 0x27a0060 Module::Module((x86_64) 'liblldb.so.3.5')
main-thread 0x27a0060 Module::~Module((x86_64) 'liblldb.so.3.5')
main-thread 0x27a0060 Module::Module((x86_64) 'libncurses.so.5')
main-thread 0x27a0060 Module::~Module((x86_64) 'libncurses.so.5')
main-thread 0x27a0060 Module::Module((x86_64) 'libtinfo.so.5')
main-thread 0x27a0060 Module::~Module((x86_64) 'libtinfo.so.5')
main-thread 0x27a0060 Module::Module((x86_64) 'libpanel.so.5')
main-thread 0x27a0060 Module::~Module((x86_64) 'libpanel.so.5')
main-thread 0x27a0060 Module::Module((x86_64) 'librt.so.1')
main-thread 0x27a0060 Module::~Module((x86_64) 'librt.so.1')
main-thread 0x27a0060 Module::Module((x86_64) 'libdl.so.2')
main-thread 0x27a0060 Module::~Module((x86_64) 'libdl.so.2')
main-thread 0x27a0060 Module::Module((x86_64) 'libpthread.so.0')
main-thread 0x27a0060 Module::~Module((x86_64) 'libpthread.so.0')
main-thread 0x27a0060 Module::Module((x86_64) 'libz.so.1')
main-thread 0x27a0060 Module::~Module((x86_64) 'libz.so.1')
main-thread 0x27a0060 Module::Module((x86_64) 'libedit.so.0')
main-thread 0x27a0060 Module::~Module((x86_64) 'libedit.so.0')
main-thread 0x27a0060 Module::Module((x86_64) 'libpython2.7.so.1.0')
main-thread 0x27a0060 Module::~Module((x86_64) 'libpython2.7.so.1.0')
main-thread 0x27a0060 Module::Module((x86_64) 'libstdc++.so.6')
main-thread 0x27a0060 Module::~Module((x86_64) 'libstdc++.so.6')
main-thread 0x27a0060 Module::Module((x86_64) 'libm.so.6')
main-thread 0x27a0060 Module::~Module((x86_64) 'libm.so.6')
main-thread 0x27a0060 Module::Module((x86_64) 'libgcc_s.so.1')
main-thread 0x27a0060 Module::~Module((x86_64) 'libgcc_s.so.1')
main-thread 0x27a0060 Module::Module((x86_64) 'libc.so.6')
main-thread 0x27a0060 Module::~Module((x86_64) 'libc.so.6')
main-thread 0x1c8f920 Broadcaster::Broadcaster("lldb.process")
main-thread 0x1c8fa50 Broadcaster::Broadcaster("lldb.process.internal_state_broadcaster")
main-thread 0x1c8fb08 Broadcaster::Broadcaster("lldb.process.internal_state_control_broadcaster")
main-thread 0x1c8fbc0 Listener::Listener('lldb.process.internal_state_listener')
main-thread 0x1c8ffb8 Broadcaster::Broadcaster("process.stdio")
main-thread 0x1c8ffb8 Communication::Communication (name = process.stdio)
main-thread 0x23f88e0 Listener::StartListeningForEvents (broadcaster = 0x1c8f920, mask = 0x0000000d) acquired_mask = 0x0000000d for lldb.Debugger
main-thread 0x1c8f8f0 Process::Process()
main-thread 0x23f88e0 Listener::StartListeningForEvents (broadcaster = 0x1c8f920, mask = 0x0000001f) acquired_mask = 0x0000001f for lldb.Debugger
main-thread 0x1c8fbc0 Listener::StartListeningForEvents (broadcaster = 0x1c8fa50, mask = 0x00000003) acquired_mask = 0x00000003 for lldb.process.internal_state_listener
main-thread 0x1c8fbc0 Listener::StartListeningForEvents (broadcaster = 0x1c8fb08, mask = 0x00000007) acquired_mask = 0x00000007 for lldb.process.internal_state_listener
main-thread 0x1c8f8f0 Process::~Process()
main-thread Went to stop the private state thread, but it was already invalid.
main-thread 0x1c8ffb8 Communication::~Communication (name = process.stdio)
main-thread 0x1c8ffb8 Communication::Disconnect ()
main-thread 0x1c8ffb8 Broadcaster::~Broadcaster("process.stdio")
main-thread 0x1c8fbc0 Listener::~Listener('lldb.process.internal_state_listener')
main-thread 0x1c8fb08 Broadcaster::~Broadcaster("lldb.process.internal_state_control_broadcaster")
main-thread 0x1c8fa50 Broadcaster::~Broadcaster("lldb.process.internal_state_broadcaster")
main-thread 0x1c8f920 Broadcaster::~Broadcaster("lldb.process")
main-thread 0x279ff00 Broadcaster::Broadcaster("lldb.process")
main-thread 0x27a0030 Broadcaster::Broadcaster("lldb.process.internal_state_broadcaster")
main-thread 0x27a00e8 Broadcaster::Broadcaster("lldb.process.internal_state_control_broadcaster")
main-thread 0x27a01a0 Listener::Listener('lldb.process.internal_state_listener')
main-thread 0x27a0598 Broadcaster::Broadcaster("process.stdio")
main-thread 0x27a0598 Communication::Communication (name = process.stdio)
main-thread 0x23f88e0 Listener::StartListeningForEvents (broadcaster = 0x279ff00, mask = 0x0000000d) acquired_mask = 0x0000000d for lldb.Debugger
main-thread 0x279fed0 Process::Process()
main-thread 0x23f88e0 Listener::StartListeningForEvents (broadcaster = 0x279ff00, mask = 0x0000001f) acquired_mask = 0x0000001f for lldb.Debugger
main-thread 0x27a01a0 Listener::StartListeningForEvents (broadcaster = 0x27a0030, mask = 0x00000003) acquired_mask = 0x00000003 for lldb.process.internal_state_listener
main-thread 0x27a01a0 Listener::StartListeningForEvents (broadcaster = 0x27a00e8, mask = 0x00000007) acquired_mask = 0x00000007 for lldb.process.internal_state_listener
main-thread 0x1c91410 Module::Module((x86_64) '/usr/local/google/home/tfiala/lldb/hello/linux-x86_64/core')
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(file="/usr/local/google/home/tfiala/lldb/hello/linux-x86_64/core", offset=0x0, length=0x6ac000, writeable=0
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(fd=3, offset=0x0, length=0x6ac000, writeable=0, fd_is_file=1)
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec() m_mmap_addr = 0x7f3b6b954000, m_mmap_size = 6995968, error = (null)
main-thread 0x1c917d0 ObjectFile::ObjectFile() module = 0x1c91410 (/usr/local/google/home/tfiala/lldb/hello/linux-x86_64/core), file = /usr/local/google/home/tfiala/lldb/hello/linux-x86_64/core, file_offset = 0x00000000, size = 6995968
main-thread Target::SetArchitecture setting architecture to x86_64 (x86_64--linux)
main-thread Process::StartPrivateStateThread() starting private state thread
main-thread Process::ControlPrivateStateThread (signal = 4)
main-thread 0x27a00e8 Broadcaster("lldb.process.internal_state_control_broadcaster")::BroadcastEvent (event_sp = {0x1c91aa0 Event: broadcaster = 0x27a00e8 (lldb.process.internal_state_control_broadcaster), type = 0x00000004 (control-resume), data = }, unique =0) hijack = (nil)
main-thread 0x27a01a0 Listener('lldb.process.internal_state_listener')::AddEvent (event_sp = {0x1c91aa0})
main-thread Sending control event of type: 4.
main-thread thread created
main-thread Process::RunPrivateStateThread (arg = 0x279fed0, pid = 0) thread starting...
main-thread Process::WaitForEventsPrivate (timeout = (nil), event_sp)...
main-thread 0x27a01a0 Listener::WaitForEventsInternal (timeout = { (nil) }) for lldb.process.internal_state_listener
main-thread 0x27a01a0 'lldb.process.internal_state_listener' Listener::FindNextEventInternal(broadcaster=0x27a00e8, broadcaster_names=(nil)[0], event_type_mask=0x00000000, remove=1) event 0x1c91aa0
main-thread Process::RunPrivateStateThread (arg = 0x279fed0, pid = 0) got a control event: 4
main-thread Process::WaitForEventsPrivate (timeout = (nil), event_sp)...
main-thread 0x27a01a0 Listener::WaitForEventsInternal (timeout = { (nil) }) for lldb.process.internal_state_listener
main-thread AuxVector:
main-thread AT_SYSINFO_EHDR [33]: 7fff3dfc8000
main-thread AT_HWCAP [16]: bfebfbff
main-thread AT_PAGESZ [6]: 1000
main-thread AT_CLKTCK [17]: 64
main-thread AT_PHDR [3]: 400040
main-thread AT_PHENT [4]: 38
main-thread AT_PHNUM [5]: 8
main-thread AT_BASE [7]: 7f87c3250000
main-thread AT_FLAGS [8]: 0
main-thread AT_ENTRY [9]: 410930
main-thread AT_UID [11]: 346e5
main-thread AT_EUID [12]: 346e5
main-thread AT_GID [13]: 1388
main-thread AT_EGID [14]: 1388
main-thread AT_SECURE [23]: 0
main-thread AT_RANDOM [25]: 7fff3ded2449
main-thread AT_EXECFN [31]: 7fff3ded4fae
main-thread AT_PLATFORM [15]: 7fff3ded2459
main-thread 0x1c938b0 Module::Module((x86_64) '/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver')
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(file="/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver", offset=0x0, length=0x1ecae8, writeable=0
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec(fd=3, offset=0x0, length=0x1ecae8, writeable=0, fd_is_file=1)
main-thread DataBufferMemoryMap::MemoryMapFromFileSpec() m_mmap_addr = 0x7f3b6af66000, m_mmap_size = 2018024, error = (null)
main-thread 0x1c95bc0 ObjectFile::ObjectFile() module = 0x1c938b0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver), file = /usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver, file_offset = 0x00000000, size = 2018024
main-thread 0x1c938b0 Module::~Module((x86_64) '/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver')
main-thread 0x1c95bc0 ObjectFile::~ObjectFile ()

main-thread DataBufferMemoryMap::Clear() m_mmap_addr = 0x7f3b6af66000, m_mmap_size = 2018024
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279d620 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..interp), load_addr = 0x0000000000400200) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279d6e0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..note.ABI-tag), load_addr = 0x000000000040021c) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279d7d0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..dynsym), load_addr = 0x0000000000400240) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279d890 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..dynstr), load_addr = 0x0000000000403c48) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279d9e0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..hash), load_addr = 0x000000000040d310) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279daa0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..gnu.version), load_addr = 0x000000000040e4e8) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279db60 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..gnu.version_r), load_addr = 0x000000000040e9c0) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279dc20 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..rela.dyn), load_addr = 0x000000000040ea60) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279ddf0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..rela.plt), load_addr = 0x000000000040eb08) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279deb0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..init), load_addr = 0x000000000040fd08) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279df70 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..plt), load_addr = 0x000000000040fd20) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e030 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..text), load_addr = 0x0000000000410930) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e0f0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..fini), load_addr = 0x000000000041db64) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e1b0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..rodata), load_addr = 0x000000000041db80) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e270 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..gcc_except_table), load_addr = 0x000000000041eb40) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e330 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..eh_frame), load_addr = 0x000000000041f5f0) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279dce0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..eh_frame_hdr), load_addr = 0x0000000000423bd4) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e600 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..dynamic), load_addr = 0x0000000000425000) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e6c0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..got), load_addr = 0x00000000004252b0) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e780 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..got.plt), load_addr = 0x0000000000425328) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e840 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..data), load_addr = 0x0000000000425940) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e900 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..jcr), load_addr = 0x0000000000425a48) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279e9c0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..tm_clone_table), load_addr = 0x0000000000425a50) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279ea80 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..fini_array), load_addr = 0x0000000000425a50) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279eb40 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..init_array), load_addr = 0x0000000000425a58) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279ec00 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..data.rel.ro), load_addr = 0x0000000000425a60) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279ecc0 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..data.rel.ro.local), load_addr = 0x0000000000425b60) module = 0x279a7e0
main-thread SectionLoadList::SetSectionLoadAddress (section = 0x279ed80 (/usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver..bss), load_addr = 0x0000000000425b70) module = 0x279a7e0
main-thread DynamicLoaderPOSIXDYLD::LoadAllCurrentModules unable to resolve POSIX DYLD rendezvous address
main-thread 0x279ad28 Broadcaster("lldb.target")::BroadcastEvent (event_sp = {0x1c8fb30 Event: broadcaster = 0x279ad28 (lldb.target), type = 0x00000002 (modules-loaded), data = }, unique =0) hijack = (nil)
main-thread Process::SetPrivateState (stopped)
main-thread Process::SetPrivateState (stopped) stop_id = 1
main-thread 0x27a0030 Broadcaster("lldb.process.internal_state_broadcaster")::BroadcastEvent (event_sp = {0x1c91740 Event: broadcaster = 0x27a0030 (lldb.process.internal_state_broadcaster), type = 0x00000001, data = { process = 0x279fed0 (pid = 0), state = stopped}}, unique =0) hijack = (nil)
main-thread 0x27a01a0 Listener('lldb.process.internal_state_listener')::AddEvent (event_sp = {0x1c91740})
main-thread HandleCommand, command succeeded
Core file '/usr/local/google/home/tfiala/lldb/hello/linux-x86_64/core' (x86_64) was loaded.
(lldb) main-thread 0x27a01a0 'lldb.process.internal_state_listener' Listener::FindNextEventInternal(broadcaster=(nil), broadcaster_names=(nil)[0], event_type_mask=0x00000000, remove=1) event 0x1c91740
main-thread 0x7f3b640009e8 Broadcaster::Broadcaster("lldb.thread")
main-thread 0x7f3b640009b0 Thread::Thread(tid = 0x0000)
main-thread 0x23f88e0 Listener::StartListeningForEvents (broadcaster = 0x7f3b640009e8, mask = 0x00000011) acquired_mask = 0x00000011 for lldb.Debugger
main-thread Discarding thread plans for thread (tid = 0x0000, force 1)
main-thread Thread::PushPlan(0x0x7f3b640009b0): "Base thread plan.", tid = 0x0000.
main-thread
main-thread ThreadList::ShouldStop: 1 threads
main-thread 0x7f3b640009b0: tid = 0x0: stop info = signal SIGABRT (stop_id = 1)

main-thread Thread::ShouldStop(0x7f3b640009b0) for tid = 0x0000 0x0000, pc = 0x00007f87bb32d425
main-thread ^^^^^^^^ Thread::ShouldStop Begin ^^^^^^^^
main-thread Plan stack initial state:
Plan Stack for thread #​1: tid = 0x0000, stack_size = 1
Element 0: Base thread plan.

main-thread th1/fr0 using architectural default unwind method
main-thread th1/fr0 0x00007f87bb32d425: CFA=rbp+16 => rbp=[rbp] rsp=rbp+16 rip=[rbp+8]

main-thread th1/fr0 cfa_regval = 0x00007f87c1081ab0 (cfa_regval = 0x00007f87c1081aa0, cfa_offset = 16)
main-thread th1/fr0 initialized frame current pc is 0x7f87bb32d425 cfa is 0x7f87c1081ab0 using x86_64 default unwind plan UnwindPlan
main-thread Plan base plan explains stop, auto-continue 0.
main-thread Base plan discarding thread plans for thread tid = 0x0000 (signal.)
main-thread Discarding thread plans for thread (tid = 0x0000, force 0)
main-thread Base plan says should stop: 1.
main-thread Plan stack final state:
Plan Stack for thread #​1: tid = 0x0000, stack_size = 1
Element 0: Base thread plan.

main-thread vvvvvvvv Thread::ShouldStop End (returning 1) vvvvvvvv
main-thread ThreadList::ShouldStop overall should_stop = 1
main-thread Process::ShouldBroadcastEvent (0x1c91740) => new state: stopped, last broadcast state: stopped - YES
main-thread Process::HandlePrivateEvent (pid = 0) broadcasting new state stopped (old state unloaded) to public
main-thread 0x279ff00 Broadcaster("lldb.process")::BroadcastEvent (event_sp = {0x1c91740 Event: broadcaster = 0x279ff00 (lldb.process), type = 0x00000001 (state-changed), data = { process = 0x279fed0 (pid = 0), state = stopped}}, unique =0) hijack = (nil)
main-thread 0x23f88e0 Listener('lldb.Debugger')::AddEvent (event_sp = {0x1c91740})
main-thread Process::WaitForEventsPrivate (timeout = (nil), event_sp)...
main-thread 0x27a01a0 Listener::WaitForEventsInternal (timeout = { (nil) }) for lldb.process.internal_state_listener
event-handler 0x23f88e0 'lldb.Debugger' Listener::FindNextEventInternal(broadcaster=(nil), broadcaster_names=(nil)[0], event_type_mask=0x00000000, remove=1) event 0x1c91740
event-handler Process::SetPublicState (state = stopped, restarted = 0)
Process 0 stopped

  • thread #​1: tid = 0, 0x00007f87bb32d425, name = 'lldb-gdbserver', stop reason = signal SIGABRT
    frame #​0: 0x00007f87bb32d425
    -> 0x7f87bb32d425: addb %al, (%rax)
    0x7f87bb32d427: addb %al, (%rax)
    0x7f87bb32d429: addb %al, (%rax)
    0x7f87bb32d42b: addb %al, (%rax)
    (lldb) event-handler 0x23f88e0 Listener::WaitForEventsInternal (timeout = { (nil) }) for lldb.Debugger

(lldb) image list
main-thread Processing command: image list
main-thread HandleCommand, cmd_obj : 'target modules list'
main-thread HandleCommand, revised_command_line: 'target modules list'
main-thread HandleCommand, wants_raw_input:'False'
main-thread HandleCommand, command line after removing command name(s): ''
main-thread HandleCommand, command succeeded
[ 0] 6FF7055D-0000-0000-0000-000000000000 /usr/local/google/home/tfiala/lldb/git/llgs/build-gold/bin/lldb-gdbserver
(lldb)

======

I see that the general module loading seems to be finding many modules that don't later show up in the image list. I'm not sure if this is relevant, but the output for the module list building seems to be implying that it is deleting the modules shortly after constructing them. I'm going to drill in on that next.

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 10, 2014

ModuleList's call to module->GetObjectFile() appears to be failing. Looking at that next. This is causing the module to be deleted shortly after being created, for each .so.

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 11, 2014

Live debugging of lldb-gdbserver with gold linker-linked lldb-gdbserver and lldb seems to show modules just fine.

From top of tree earlier today:

gold linker, live lldb-gdbserver (via './lldb ./lldb-gdbserver', then 'run'):

./lldb ./lldb-gdbserver
(lldb) run
Process 12516 launching
Process 12516 stopped

  • thread #​1: tid = 12516, 0x00007f3099cbe6b0, name = 'lldb-gdbserver', stop reason = trace
    frame #​0: 0x00007f3099cbe6b0
    -> 0x7f3099cbe6b0: movq %rsp, %rdi
    0x7f3099cbe6b3: callq 0x7f3099cc2010
    0x7f3099cbe6b8: movq %rax, %r12
    0x7f3099cbe6bb: movl 0x2215ff(%rip), %eax
    (lldb) Process 12516 launched: './lldb-gdbserver' (x86_64)
    Process 12516 stopped and restarted: thread 1 received signal: SIGCHLD
    lldb-gdbserver: /mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp:2539: static void lldb_private::ScriptInterpreterPython::InitializePrivate(): Assertion `g_swig_init_callback != __null' failed.
    Process 12516 stopped
  • thread #​1: tid = 12516, 0x00007f3091d95425 libc.so.6__GI_raise(sig=<unavailable>) + 53 at raise.c:64, name = 'lldb-gdbserver', stop reason = signal SIGABRT frame #&#8203;0: 0x00007f3091d95425 libc.so.6__GI_raise(sig=) + 53 at raise.c:64
    (lldb) image list
    [ 0] CCA12BF3-0000-0000-0000-000000000000 ./lldb-gdbserver
    [ 1] BBB69FA5-0000-0000-0000-000000000000 /mnt/ssd/work/git/llgs/build-gold/lib/liblldb.so.3.5
    [ 2] F755E1E3-59A2-7073-C0E4-09DC598D8E01-4AD6E41D /lib/x86_64-linux-gnu/libncurses.so.5
    [ 3] B17B9A8B-40BB-0FE4-BA34-66F28F9C1F50-04C122BD /lib/x86_64-linux-gnu/libtinfo.so.5
    [ 4] 3A7B72F9-E014-B578-F4B9-C04A26FDC4C5-6BE7D79D /usr/lib/x86_64-linux-gnu/libpanel.so.5
    [ 5] 352C5B37-3A50-E6C4-AB88-1A5DB6F5766F-DF81EEE0 /lib/x86_64-linux-gnu/librt.so.1
    /usr/lib/debug/lib/x86_64-linux-gnu/librt-2.15.so
    [ 6] D181AF55-1DBB-C43E-9D55-913D532635FD-E18E7C4E /lib/x86_64-linux-gnu/libdl.so.2
    /usr/lib/debug/lib/x86_64-linux-gnu/libdl-2.15.so
    [ 7] C340AF9D-EE97-C17C-730F-7D03693286C5-194A46B8 /lib/x86_64-linux-gnu/libpthread.so.0
    /usr/lib/debug/lib/x86_64-linux-gnu/libpthread-2.15.so
    [ 8] F695ECFC-F391-8D5D-3498-9398A14B7ECD-D9F46CD0 /lib/x86_64-linux-gnu/libz.so.1
    [ 9] 4AF89E2D-0000-0000-0000-000000000000 /usr/local/google/home/tfiala/lldb/tools/libedit/linux-x86_64/lib/libedit.so.0
    [ 10] F8778485-AF29-FF04-1C2C-CF09814C57F3-C0984A54 /usr/lib/libpython2.7.so.1.0
    [ 11] 78FB53F1-0000-0000-0000-000000000000 /usr/local/gcc/gcc-current/lib64/libstdc++.so.6
    [ 12] 817AA99B-3DD0-2501-F8BC-04A3E9A9358A-08F20D7D /lib/x86_64-linux-gnu/libm.so.6
    /usr/lib/debug/lib/x86_64-linux-gnu/libm-2.15.so
    [ 13] A337537A-0000-0000-0000-000000000000 /usr/local/gcc/gcc-current/lib64/libgcc_s.so.1
    [ 14] 53514F7C-9AD6-6147-03AC-01FD7CBCC9D3-C6F621CD /lib/x86_64-linux-gnu/libc.so.6
    /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.15.so
    [ 15] 64A22928-C392-AC7B-5D87-037DADFDBC63-7E687985 /usr/lib/x86_64-linux-gnu/libedit.so.2
    [ 16] 83B40B93-BD2F-D266-F12E-008E61309FC0-3EB406A5 /lib64/ld-linux-x86-64.so.2
    /usr/lib/debug/lib/x86_64-linux-gnu/ld-2.15.so
    [ 17] 9B30CC6B-1115-2FC8-AA7B-81F2055E760C-95503208 /lib/x86_64-linux-gnu/libssl.so.1.0.0
    [ 18] 5E5A248F-00DD-1781-2C03-B225C3DB3D26-753243F0 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
    [ 19] 7786F86F-C791-4187-00B8-2302B5ED2F9F-8F5BA246 /lib/x86_64-linux-gnu/libutil.so.1
    /usr/lib/debug/lib/x86_64-linux-gnu/libutil-2.15.so
    [ 20] 90AEBBE5-F46F-341E-50F3-D32D40D8FA87-20FEBC93 /lib/x86_64-linux-gnu/libbsd.so.0
    (lldb)

====================

Running lldb-gdbserver, producing a core, and then debugging the core produces very different results (and matches what I was initially reporting):

tfiala@tfiala2:/lldb/git/llgs/build-gold/bin$ ./lldb-gdbserver
lldb-gdbserver: /mnt/ssd/work/git/llgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp:2539: static void lldb_private::ScriptInterpreterPython::InitializePrivate(): Assertion `g_swig_init_callback != __null' failed.
Aborted (core dumped)
tfiala@tfiala2:
/lldb/git/llgs/build-gold/bin$ ls -la core
-rw------- 1 tfiala eng 6995968 Feb 10 14:32 core
tfiala@tfiala2:~/lldb/git/llgs/build-gold/bin$ ./lldb -c ./core ./lldb-gdbserver
Core file '/mnt/ssd/work/git/llgs/build-gold/bin/core' (x86_64) was loaded.
Process 0 stopped

  • thread #​1: tid = 0, 0x00007fb1810eb425, name = 'lldb-gdbserver', stop reason = signal SIGABRT
    frame #​0: 0x00007fb1810eb425
    -> 0x7fb1810eb425: addb %al, (%rax)
    0x7fb1810eb427: addb %al, (%rax)
    0x7fb1810eb429: addb %al, (%rax)
    0x7fb1810eb42b: addb %al, (%rax)
    (lldb) image list
    [ 0] CCA12BF3-0000-0000-0000-000000000000 ./lldb-gdbserver
    (lldb) run
    There is a running process, kill it and restart?: [Y/n] n
    (lldb)

=====

Looking at how core files are processed now.

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 11, 2014

Showing module loading during the core files with configure/(g)make and standard linker:

tfiala@tfiala2:~/lldb/git/llgs/build-autoconf/Debug+Asserts/bin$ ./lldb
(lldb) log enable lldb module
(lldb) target create -c ./core ./lldb-gdbserver
main-thread 0xe22c00 Module::Module((x86_64) './lldb-gdbserver')
main-thread 0xde29b0 Module::Module((x86_64) 'liblldb.so')
main-thread 0xde29b0 Module::~Module((x86_64) 'liblldb.so')
main-thread 0xde29b0 Module::Module((x86_64) 'libz.so.1')
main-thread 0xde29b0 Module::~Module((x86_64) 'libz.so.1')
main-thread 0xde29b0 Module::Module((x86_64) 'libpthread.so.0')
main-thread 0xde29b0 Module::~Module((x86_64) 'libpthread.so.0')
main-thread 0xde29b0 Module::Module((x86_64) 'libedit.so.0')
main-thread 0xde29b0 Module::~Module((x86_64) 'libedit.so.0')
main-thread 0xde29b0 Module::Module((x86_64) 'libtinfo.so.5')
main-thread 0xde29b0 Module::~Module((x86_64) 'libtinfo.so.5')
main-thread 0xde29b0 Module::Module((x86_64) 'librt.so.1')
main-thread 0xde29b0 Module::~Module((x86_64) 'librt.so.1')
main-thread 0xde29b0 Module::Module((x86_64) 'libdl.so.2')
main-thread 0xde29b0 Module::~Module((x86_64) 'libdl.so.2')
main-thread 0xde29b0 Module::Module((x86_64) 'libstdc++.so.6')
main-thread 0xde29b0 Module::~Module((x86_64) 'libstdc++.so.6')
main-thread 0xde29b0 Module::Module((x86_64) 'libm.so.6')
main-thread 0xde29b0 Module::~Module((x86_64) 'libm.so.6')
main-thread 0xde29b0 Module::Module((x86_64) 'libgcc_s.so.1')
main-thread 0xde29b0 Module::~Module((x86_64) 'libgcc_s.so.1')
main-thread 0xde29b0 Module::Module((x86_64) 'libc.so.6')
main-thread 0xde29b0 Module::~Module((x86_64) 'libc.so.6')
main-thread 0xde4e20 Module::Module((x86_64) '/mnt/ssd/work/git/llgs/build-autoconf/Debug+Asserts/bin/core')
main-thread 0xde8210 Module::Module((x86_64) './lldb-gdbserver')
main-thread 0xde8210 Module::~Module((x86_64) './lldb-gdbserver')
main-thread 0xde9bb0 Module::Module((x86_64) './lldb-gdbserver')
main-thread 0xde9bb0 Module::~Module((x86_64) './lldb-gdbserver')
main-thread 0xde9bb0 Module::Module((x86_64) '/mnt/ssd/work/git/llgs/build-autoconf/Debug+Asserts/bin/../lib/liblldb.so')
main-thread 0xde8210 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libz.so.1')
main-thread 0x88abf0 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libpthread.so.0')
main-thread 0x887c10 Module::Module((x86_64) '/usr/local/google/home/tfiala/lldb/tools/libedit/linux-x86_64/lib/libedit.so.0')
main-thread 0x8931b0 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libtinfo.so.5')
main-thread 0xe3b7f0 Module::Module((x86_64) '/lib/x86_64-linux-gnu/librt.so.1')
main-thread 0xe3ef50 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libdl.so.2')
main-thread 0x88e3e0 Module::Module((x86_64) '/usr/local/gcc/gcc-current/lib64/libstdc++.so.6')
main-thread 0xe46a30 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libm.so.6')
main-thread 0xe4b410 Module::Module((x86_64) '/usr/local/gcc/gcc-current/lib64/libgcc_s.so.1')
main-thread 0xe4eaa0 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libc.so.6')
main-thread 0xe522c0 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libutil.so.1')
main-thread 0xe55210 Module::Module((x86_64) '/usr/lib/libpython2.7.so.1.0')
main-thread 0xe57d40 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libncurses.so.5')
main-thread 0xe41b80 Module::Module((x86_64) '/usr/lib/x86_64-linux-gnu/libpanel.so.5')
main-thread 0xe5aed0 Module::Module((x86_64) '/lib64/ld-linux-x86-64.so.2')
main-thread 0xdbb5c0 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libssl.so.1.0.0')
main-thread 0xdbe500 Module::Module((x86_64) '/lib/x86_64-linux-gnu/libcrypto.so.1.0.0')
Core file '/mnt/ssd/work/git/llgs/build-autoconf/Debug+Asserts/bin/core' (x86_64) was loaded.
Process 0 stopped

  • thread #​1: tid = 0, 0x00007f47c1711425 libc.so.6__GI_raise(sig=<unavailable>) + 53 at raise.c:64, name = 'lldb-gdbserver', stop reason = signal SIGABRT frame #&#8203;0: 0x00007f47c1711425 libc.so.6__GI_raise(sig=) + 53 at raise.c:64
    (lldb) exit
    Quitting LLDB will kill one or more processes. Do you really want to proceed: [Y/n] y
    tfiala@tfiala2:~/lldb/git/llgs/build-autoconf/Debug+Asserts/bin$ cd ../../..

Notice the two passes at module resolution.

Now with gold linker and cmake/ninja build;

tfiala@tfiala2:~/lldb/git/llgs/build-gold/bin$ ./lldb
(lldb) log enable lldb module
(lldb) target create -c ./core ./lldb-gdbserver
main-thread 0x1e1e3e0 Module::Module((x86_64) './lldb-gdbserver')
main-thread 0x1dde5a0 Module::Module((x86_64) 'liblldb.so.3.5')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'liblldb.so.3.5')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libncurses.so.5')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libncurses.so.5')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libtinfo.so.5')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libtinfo.so.5')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libpanel.so.5')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libpanel.so.5')
main-thread 0x1dde5a0 Module::Module((x86_64) 'librt.so.1')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'librt.so.1')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libdl.so.2')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libdl.so.2')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libpthread.so.0')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libpthread.so.0')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libz.so.1')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libz.so.1')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libedit.so.0')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libedit.so.0')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libpython2.7.so.1.0')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libpython2.7.so.1.0')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libstdc++.so.6')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libstdc++.so.6')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libm.so.6')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libm.so.6')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libgcc_s.so.1')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libgcc_s.so.1')
main-thread 0x1dde5a0 Module::Module((x86_64) 'libc.so.6')
main-thread 0x1dde5a0 Module::~Module((x86_64) 'libc.so.6')
main-thread 0x1de0b80 Module::Module((x86_64) '/mnt/ssd/work/git/llgs/build-gold/bin/core')
main-thread 0x1de3ff0 Module::Module((x86_64) './lldb-gdbserver')
main-thread 0x1de3ff0 Module::~Module((x86_64) './lldb-gdbserver')
Core file '/mnt/ssd/work/git/llgs/build-gold/bin/core' (x86_64) was loaded.
Process 0 stopped

  • thread #​1: tid = 0, 0x00007fb1810eb425, name = 'lldb-gdbserver', stop reason = signal SIGABRT
    frame #​0: 0x00007fb1810eb425
    -> 0x7fb1810eb425: addb %al, (%rax)
    0x7fb1810eb427: addb %al, (%rax)
    0x7fb1810eb429: addb %al, (%rax)
    0x7fb1810eb42b: addb %al, (%rax)
    (lldb)

Notice we only take one pass through the modules.

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 11, 2014

DynamicLoaderPOSIXDYLD::LoadAllCurrentModules() is failing in the m_rendezvous.Resolve() in the case of a core file from a gold-linked exe. This case succeeds in the case of a core file from a standard-linked exe.

These two logs are helpful from lldb:

(lldb) log enable lldb dyld
(lldb) log enable lldb module

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 11, 2014

ResolveRendezvousAddress(Process*) within DYLDRendezvous.cpp:40 appears to be failing to read the memory location for the image info.

That makes me think that perhaps the section loading from the core is failing in some subtle way that doesn't map all the memory correctly. This causes DYLDRendezvous::Resolve() to fail.

Looking at that next.

@llvmbot
Copy link
Collaborator Author

llvmbot commented Feb 12, 2014

Fixed with r201214.

Net result: elf core file segment collapsing had a bug in it. The collapse code would collapse VMA-contiguous regions when the core-file-backed contents were contiguous; however, it failed to account for the case where there was zero padding at the end of the earlier/left side of the segment merge.

This was exactly the scenario that the gold linker was producing in the core for my case. The DYLD rendezvous (the communication point between the linker and the debugger in elf land) failed to look up due to this bug, in which case all shared library info was lost. That meant lldb knew about no images (image list showed empty), and back traces that went through shared library code could not be stack unwound through because their memory segments were not present.

@llvmbot llvmbot transferred this issue from llvm/llvm-bugzilla-archive Dec 9, 2021
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugzilla Issues migrated from bugzilla lldb
Projects
None yet
Development

No branches or pull requests

1 participant