$ g++ --version g++ (GCC) 8.2.1 20181127 $ ld --version LLD 7.0.1 (compatible with GNU linkers) $ gdb --version GNU gdb (GDB) 8.2.1 I just tried to switch to ld.lld from ld.gold for performance reasons, but hit a road block: Apparently the DWARF emitted into the final executable or library is bogus and leads to issues when trying to load it then in consumers like gdb, valgrind, bloaty, ... Here are some examples from a single KDE projects (ki18n). Note that I was so far not able to reproduce this in a simplified standalone example. Reading symbols from bin/libKF5I18n.so...Dwarf Error: bad offset (0xcc08000026980004) in compilation unit header (offset 0x6fbe2 + 6) [in module /home/milian/projects/kf5/build-dbg/frameworks/ki18n/bin/libKF5I18n.so.5.54.0] Reading symbols from ktranscript.so...Dwarf Error: wrong version in compilation unit header (is 422, should be 2, 3, 4 or 5) [in module /home/milian/projects/kf5/build-dbg/frameworks/ki18n/bin/ktranscript.so] Reading symbols from ki18n-ktranscripttest...Dwarf Error: wrong version in compilation unit header (is 1024, should be 2, 3, 4 or 5) [in module /home/milian/projects/kf5/build-dbg/frameworks/ki18n/bin/ki18n-ktranscripttest] I have seen at least one more variation of the version header issue (version 514). When I instead link with ld.gold, none of these issues show up.
>Reading symbols from ktranscript.so...Dwarf Error: wrong version in compilation ?>unit header (is 422, should be 2, 3, 4 or 5) [in module >/home/milian/projects/kf5/build-dbg/frameworks/ki18n/bin/ktranscript.so] > >Reading symbols from ki18n-ktranscripttest...Dwarf Error: wrong version in >compilation unit header (is 1024, should be 2, 3, 4 or 5) [in module >/home/milian/projects/kf5/build-dbg/frameworks/ki18n/bin/ki18n-ktranscripttest] Looking at these errors seems "2, 3, 4 or 5" is a DWARF version expected and it complains about something different is written instead probably. First what I would try is to inspect the binary with an llvm-dwarfdump to see what is wrong with DWARF info. Could you attach a /home/milian/projects/kf5/build-dbg/frameworks/ki18n/bin/ktranscript.so you have? I can take a look. Also, if you can pass -reproduce=sample.tar to the linker (and provide us a reproducible sample), that would be great. One more idea: did you try to reproduce it with the latest LLD sources?
Created attachment 21410 [details] ktranscript.so with buggy dwarf
I haven't tried it with the latest LLD sources yet, I probably need to compile all of LLVM for that too, right? I might revive my build setup for that and check if you can't find anything obvious from the sample or llvm-dwarfdump yourself. the sample can be downloaded from here: https://swanson.kdab.com/owncloud/index.php/s/4tTkKnaytBGg7HZ (it's too large to attach it here directly)
(In reply to Milian Wolff from comment #3) > I haven't tried it with the latest LLD sources yet, I probably need to > compile all of LLVM for that too, right? Yes. (In reply to Milian Wolff from comment #2) > Created attachment 21410 [details] > ktranscript.so with buggy dwarf I do not see anything unusual with ktranscript.so. llvm-dwarfdump shows it has a single compile unit: .debug_info contents: 0x00000000: Compile Unit: length = 0x0004b855 version = 0x0004 abbr_offset = 0x0000 addr_size = 0x08 (next unit at 0x0004b859) ... 0x0004b858: NULL The header looks fine to me and has DWARF version 4. I guess it might be a compiler issue and not a linker one probably. Given that you're using "GNU C++11 8.2.1 20181127" and older LLD, I would suggest trying check with clang(and/or different gcc version too maybe) and the latest LLD.
Oh, and GNU dwafdump reports an error: >> dwarfdump ktranscript.so ... arange starts at 0x00022460, length of 0x0000005a, cu_die_offset = 0x0000000b arange starts at 0x000224ba, length of 0x00000022, cu_die_offset = 0x0000000b arange end dwarfdump ERROR: dwarf_offdie: DW_DLE_VERSION_STAMP_ERROR (48) I also tried the sample you provided with the latest version of LLD and the error above still present. I am afraid I have no ideas atm what might be wrong.
Hey, I just checked with ld.lld v8.0.0 and the issue persists. Using clang to compile instead, the issue disappears. Nevertheless, this isn't a real fix, is it? gcc and ld.lld should work together, no? Also, considering that the GCC produced .o files work fine with ld.bfd and ld.gold, I still believe that it's somehow ld.lld that is missing support for something that leads to this issue here?
(In reply to Milian Wolff from comment #6) > Hey, > > I just checked with ld.lld v8.0.0 and the issue persists. Using clang to > compile instead, the issue disappears. Nevertheless, this isn't a real fix, > is it? gcc and ld.lld should work together, no? Also, considering that the > GCC produced .o files work fine with ld.bfd and ld.gold, I still believe > that it's somehow ld.lld that is missing support for something that leads to > this issue here? It'd certainly be worth understanding what lld's doing differently from binutils ld on GCC's output. But without knowing the specifics it's hard to know where the bug is. Do you have/could you try to make a small reproducer?
(In reply to David Blaikie from comment #7) > Do you have/could you try to make a small reproducer? We are seeing the same or a similar issue when we enable ASan in gcc: $ cat test.c int main() { return 0; } $ gcc test.c -o test -g -fsanitize=address -fuse-ld=gold $ readelf --debug-dump=info ./test > /dev/null # no errors reported $ ln -s /usr/bin/ld.lld ld $ gcc test.c -o test -g -fsanitize=address -B. $ readelf --debug-dump=info ./test > /dev/null readelf: Warning: Invalid pointer size (157) in compunit header, using 4 instead readelf: Warning: Debug info is corrupted, .debug_info header at 0xe17 has length 4f00 $ echo q | gdb ./test > /dev/null Dwarf Error: wrong version in compilation unit header (is 1024, should be 2, 3, 4 or 5) [in module /home/max/code/lld_bug/test] Should we create a new issue for this? We are running Arch Linux with gcc and gdb 8.2.1, lld 8.0.0, and binutils 2.31.1 from the official repositories. This behavior was found in the course of the SYMBIOSYS research project at COMSYS, RWTH Aachen University. This research is supported by the European Research Council (ERC) under the EU's Horizon 2020 Research and Innovation Programme grant agreement n. 647295 (SYMBIOSYS).
Can't seem to reproduce it with this selection of versions, at least: $ ./ld --version LLD 9.0.0 (https://llvm.org/svn/llvm-project/lld/trunk 357739) (compatible with GNU linkers) $ gcc-8 test.c -o test -g -fsanitize=address -B. $ cat test.c int main() { return 0; } $ gcc-8 --version gcc-8 (Debian 8-20180218-1) 8.0.1 20180218 (experimental) [trunk revision 257787] $ readelf --debug-dump=info ./test > /dev/null $ echo $? 0 $ readelf --version GNU readelf (GNU Binutils for Debian) 2.30
(In reply to David Blaikie from comment #9) > Can't seem to reproduce it with this selection of versions, at least: [...] This may be a problem specific to Arch Linux since I can't reproduce it on Debian and Fedora either. However, on Arch Linux, I can reproduce it even when I use the exact same GCC and LLVM versions as you. I think Milian is running Arch Linux as well, because the libraries in his reproducable build match mine exactly.
(In reply to Max Sistemich from comment #10) > (In reply to David Blaikie from comment #9) > > Can't seem to reproduce it with this selection of versions, at least: [...] > > This may be a problem specific to Arch Linux since I can't reproduce it on > Debian and Fedora either. However, on Arch Linux, I can reproduce it even > when I use the exact same GCC and LLVM versions as you. > > I think Milian is running Arch Linux as well, because the libraries in his > reproducable build match mine exactly. Does the Arch version of Clang have extra patches in it? I remember a problem in years past where Arch had picked up an un-reviewed patch and it caused problems for some people.
yes, I'm using Arch - I'll try to finally get a source build of clang up and running again and then see if the issue is resolved. the patches that Arch is applying can be found here: https://git.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/clang
Created attachment 21748 [details] reproducer This is reduced from a scylla unit test. gcc is gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2) lld is eaf92acd78daa6725d1bae481799acdb64a59e89 I am happy to add the gcc produced .o files it this still doesn't reproduce for you.
(In reply to Rafael Ávila de Espíndola from comment #13) > Created attachment 21748 [details] > reproducer > > This is reduced from a scylla unit test. > > gcc is > > gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2) > > lld is eaf92acd78daa6725d1bae481799acdb64a59e89 > > I am happy to add the gcc produced .o files it this still doesn't reproduce > for you. Any chance of further reduction? (I mean, I don't tend to do LLD development anyway - so perhaps someone else will pick it up with what's in this thread already - I'm just kind of curious about what's causing this), potentially taking assembly & seeing if it's assembled differently by GCC (or if taking GCC's assembly and assembling it with Clang's integrated assembler reproduces the same odd behavior in LLD) - and reducing the assembly might make it easier to see what the interesting feature is
Created attachment 21753 [details] reproducer This now uses readelf to display the issue, which allows dropping libc from the reproducible. I left the .s files in the archive, so it should be possible to reproduce this without gcc. Note that the debug info in udp.s must be compressed for this to reproduce.
Created attachment 21787 [details] reproducer reduced it further. It now only has two small assembly files: 52 ip_checksum.s 112 udp.s 164 total % gcc -c ip_checksum.s % gcc -c udp.s -gz % ld.lld ip_checksum.o udp.o -o t ld.lld: warning: cannot find entry symbol _start; defaulting to 0x201000 % readelf --debug-dump=info t > /dev/null readelf: t: Warning: Invalid pointer size (0) in compunit header, using 4 instead readelf: t: Warning: Debug info is corrupted, .debug_info header at 0x25 has length 4b000000 Replacing lld with gold avoids the readelf warnings. Not using -gz and lld also avoids the readelf warning.
(In reply to Rafael Ávila de Espíndola from comment #16) > Created attachment 21787 [details] > reproducer > > reduced it further. It now only has two small assembly files: > > 52 ip_checksum.s > 112 udp.s > 164 total > > % gcc -c ip_checksum.s > % gcc -c udp.s -gz > % ld.lld ip_checksum.o udp.o -o t > ld.lld: warning: cannot find entry symbol _start; defaulting to 0x201000 > % readelf --debug-dump=info t > /dev/null > readelf: t: Warning: Invalid pointer size (0) in compunit header, using 4 > instead > readelf: t: Warning: Debug info is corrupted, .debug_info header at 0x25 has > length 4b000000 > > Replacing lld with gold avoids the readelf warnings. Not using -gz and lld > also avoids the readelf warning. Thanks for providing this! I wasn't able to reproduce the readelf (2.28 in my case) warning though yet with gcc version 8.1.0 (Ubuntu 8.1.0-9ubuntu1~16.04.york1). I also tried using llvm-mc with -compress-debug-sections option (for udp.s) to produce the object but also observed nothing. For me, readelf reports no warnings, but dwarfdump (-V 2018-01-29 11:08:35-08:00) reports: dwarfdump ERROR: dwarf_srclines: DW_DLE_HEADER_LEN_BIGGER_THAN_SECSIZE (342) Corrupt dwarf That happens for both lld and gold produced binaries though. My tools and environment are a bit outdated, so I plan to update everything and retry soon.
Interesting that the tools versions are important. Maybe gold had a similar bug in the past? The tools I have are GNU readelf version 2.31.1-24.fc29 GNU gold (version 2.31.1-24.fc29) 1.16
I suspect that there may be something broken in your assembly files now, because dwarfdump already complains when I call it on udp.o, even when I assemble it with clang. Neither readelf nor llvm-dwarfdump was able to detect that error though.
Had no success either on a fresh ubuntu: umb@ubuntu:~/tests/gdb-issue$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.10 Release: 18.10 Codename: cosmic My gcc -v is gcc version 8.2.0 (Ubuntu 8.2.0-7ubuntu1) as -v shows: GNU assembler version 2.31.1 (x86_64-linux-gnu) using BFD version (GNU Binutils for Ubuntu) 2.31.1 When I do: umb@ubuntu:~/tests/gdb-issue$ gcc -c ip_checksum.s umb@ubuntu:~/tests/gdb-issue$ dwarfdump ip_checksum.o I get: .debug_info dwarfdump ERROR: dwarf_srcfiles: DW_DLE_READ_LITTLEENDIAN_ERROR (331) Corrupted dwarfdata littleendian host. Attempting to continue. If I use GNU as I have the same (expected I think): umb@ubuntu:~/tests/gdb-issue$ as ip_checksum.s -o ip_checksum.o umb@ubuntu:~/tests/gdb-issue$ dwarfdump ip_checksum.o .debug_info dwarfdump ERROR: dwarf_srcfiles: DW_DLE_READ_LITTLEENDIAN_ERROR (331) Corrupted dwarfdata littleendian host. Attempting to continue. llvm-mc (latest atm) shows: umb@ubuntu:~/tests/gdb-issue$ ~/LLVM/build/bin/llvm-mc ip_checksum.s -o ip_checksum.o -filetype=obj umb@ubuntu:~/tests/gdb-issue$ dwarfdump ip_checksum.o .debug_info dwarfdump ERROR: dwarf_srcfiles: DW_DLE_DEBUG_LINE_NULL (38) .debug_line section present but elf_getdata() failed or section is zero-length. Attempting to continue. Then if I try to readelf the output produced by lld/gold: umb@ubuntu:~/tests/gdb-issue$ ld.gold -v GNU gold (GNU Binutils for Ubuntu 2.31.1) 1.16 umb@ubuntu:~/tests/gdb-issue$ ~/LLVM/build/bin/ld.lld -v LLD 9.0.0 (http://llvm.org/svn/llvm-project/lld/trunk 358661) (compatible with GNU linkers) umb@ubuntu:~/tests/gdb-issue$ readelf -v GNU readelf (GNU Binutils for Ubuntu) 2.31.1 I have: umb@ubuntu:~/tests/gdb-issue$ ~/LLVM/build/bin/ld.lld -o test.lld ip_checksum.o udp.o ld.lld: warning: cannot find entry symbol _start; defaulting to 0x201000 umb@ubuntu:~/tests/gdb-issue$ readelf --debug-dump=info test.lld Contents of the .debug_info section: Compilation Unit @ offset 0x0: Length: 0x21 (32-bit) Version: 4 Abbrev Offset: 0x0 Pointer Size: 8 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit) <c> DW_AT_producer : (indirect string, offset: 0x12): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <10> DW_AT_language : 4 (C++) <11> DW_AT_name : (indirect string, offset: 0x0): long unsigned int <15> DW_AT_comp_dir : (indirect string, offset: 0x12): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <19> DW_AT_stmt_list : 0x1a <1><1d>: Abbrev Number: 2 (DW_TAG_base_type) <1e> DW_AT_byte_size : 8 <1f> DW_AT_encoding : 7 (unsigned) <20> DW_AT_name : (indirect string, offset: 0x0): long unsigned int <1><24>: Abbrev Number: 0 Compilation Unit @ offset 0x25: Length: 0x4b (32-bit) Version: 4 Abbrev Offset: 0x1a Pointer Size: 8 <0><30>: Abbrev Number: 1 (DW_TAG_compile_unit) <31> DW_AT_producer : (indirect string, offset: 0x58): 4 -g <35> DW_AT_language : 4 (C++) <36> DW_AT_name : (indirect string, offset: 0x58): 4 -g <3a> DW_AT_comp_dir : (indirect string, offset: 0x58): 4 -g <3e> DW_AT_stmt_list : 0x58 <1><42>: Abbrev Number: 2 (DW_TAG_namespace) <43> DW_AT_name : std <47> DW_AT_decl_file : 1 <48> DW_AT_decl_line : 0 <49> DW_AT_sibling : <0x5e> <2><4d>: Abbrev Number: 3 (DW_TAG_namespace) <4e> DW_AT_name : (indirect string, offset: 0x58): 4 -g <52> DW_AT_decl_file : 2 <53> DW_AT_decl_line : 2 <54> DW_AT_decl_column : 63 <2><55>: Abbrev Number: 4 (DW_TAG_imported_module) <56> DW_AT_decl_line : 2 <57> DW_AT_decl_column : <0x283f27> <3><5b>: Abbrev Number: 0 <2><5c>: Abbrev Number: 0 <1><5d>: Abbrev Number: 0 <0><5e>: Abbrev Number: 5 (DW_TAG_base_type) <5f> DW_AT_byte_size : 8 <60> DW_AT_encoding : 7 (unsigned) <61> DW_AT_name : (indirect string, offset: 0x58): 4 -g <0><65>: Abbrev Number: 5 (DW_TAG_base_type) <66> DW_AT_byte_size : 8 <67> DW_AT_encoding : 5 (signed) <68> DW_AT_name : (indirect string, offset: 0x58): 4 -g <0><6c>: Abbrev Number: 5 (DW_TAG_base_type) <6d> DW_AT_byte_size : 4 <6e> DW_AT_encoding : 7 (unsigned) <6f> DW_AT_name : (indirect string, offset: 0x58): 4 -g umb@ubuntu:~/tests/gdb-issue$ ld.gold -o test.gold ip_checksum.o udp.o umb@ubuntu:~/tests/gdb-issue$ readelf --debug-dump=info test.gold Contents of the .debug_info section: Compilation Unit @ offset 0x0: Length: 0x21 (32-bit) Version: 4 Abbrev Offset: 0x0 Pointer Size: 8 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit) <c> DW_AT_producer : (indirect string, offset: 0x0): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <10> DW_AT_language : 4 (C++) <11> DW_AT_name : (indirect string, offset: 0x4b): long unsigned int <15> DW_AT_comp_dir : (indirect string, offset: 0x0): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <19> DW_AT_stmt_list : 0x1a <1><1d>: Abbrev Number: 2 (DW_TAG_base_type) <1e> DW_AT_byte_size : 8 <1f> DW_AT_encoding : 7 (unsigned) <20> DW_AT_name : (indirect string, offset: 0x4b): long unsigned int <1><24>: Abbrev Number: 0 Compilation Unit @ offset 0x25: Length: 0x4b (32-bit) Version: 4 Abbrev Offset: 0x1a Pointer Size: 8 <0><30>: Abbrev Number: 1 (DW_TAG_compile_unit) <31> DW_AT_producer : (indirect string, offset: 0x58): int <35> DW_AT_language : 4 (C++) <36> DW_AT_name : (indirect string, offset: 0x58): int <3a> DW_AT_comp_dir : (indirect string, offset: 0x58): int <3e> DW_AT_stmt_list : 0x58 <1><42>: Abbrev Number: 2 (DW_TAG_namespace) <43> DW_AT_name : std <47> DW_AT_decl_file : 1 <48> DW_AT_decl_line : 0 <49> DW_AT_sibling : <0x5e> <2><4d>: Abbrev Number: 3 (DW_TAG_namespace) <4e> DW_AT_name : (indirect string, offset: 0x58): int <52> DW_AT_decl_file : 2 <53> DW_AT_decl_line : 2 <54> DW_AT_decl_column : 63 <2><55>: Abbrev Number: 4 (DW_TAG_imported_module) <56> DW_AT_decl_line : 2 <57> DW_AT_decl_column : <0x283f27> <3><5b>: Abbrev Number: 0 <2><5c>: Abbrev Number: 0 <1><5d>: Abbrev Number: 0 <0><5e>: Abbrev Number: 5 (DW_TAG_base_type) <5f> DW_AT_byte_size : 8 <60> DW_AT_encoding : 7 (unsigned) <61> DW_AT_name : (indirect string, offset: 0x58): int <0><65>: Abbrev Number: 5 (DW_TAG_base_type) <66> DW_AT_byte_size : 8 <67> DW_AT_encoding : 5 (signed) <68> DW_AT_name : (indirect string, offset: 0x58): int <0><6c>: Abbrev Number: 5 (DW_TAG_base_type) <6d> DW_AT_byte_size : 4 <6e> DW_AT_encoding : 7 (unsigned) <6f> DW_AT_name : (indirect string, offset: 0x58): int i.e. no visible errors from readelf here.
(In reply to Rafael Ávila de Espíndola from comment #13) > I am happy to add the gcc produced .o files it this still doesn't reproduce > for you. Perhaps is that what we might want to try then?
Created attachment 21804 [details] testcase This now includes * The .ii files * The .s files produced by gcc * The .o files produced by gas * The output files produced by gold and lld hopefully that will let us find what is different between our system :-)
(In reply to Rafael Ávila de Espíndola from comment #22) > Created attachment 21804 [details] > testcase > > This now includes > > * The .ii files > * The .s files produced by gcc > * The .o files produced by gas > * The output files produced by gold and lld > > hopefully that will let us find what is different between our system :-) Thanks! New results are below. For start, I tried to check the objects and binaries from your archive to see what my tools say about them. 1) My GNU readelf (GNU Binutils for Ubuntu) 2.31.1 says that udp.o is broken: umb@ubuntu:~/tests/2$ readelf --debug-dump=info udp.o readelf: Warning: compressed section '.debug_info' is corrupted 2) It says your test.gold is OK, but test.lld is not: umb@ubuntu:~/tests/2$ readelf --debug-dump=info test.gold Contents of the .debug_info section: Compilation Unit @ offset 0x0: Length: 0x21 (32-bit) Version: 4 Abbrev Offset: 0x0 Pointer Size: 8 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit) <c> DW_AT_producer : (indirect string, offset: 0x21): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <10> DW_AT_language : 4 (C++) <11> DW_AT_name : (indirect string, offset: 0x6c): ip_checksum.ii <15> DW_AT_comp_dir : (indirect string, offset: 0x0): /home/espindola/scylla/gdb-issue <19> DW_AT_stmt_list : 0x0 <1><1d>: Abbrev Number: 2 (DW_TAG_base_type) <1e> DW_AT_byte_size : 8 <1f> DW_AT_encoding : 7 (unsigned) <20> DW_AT_name : (indirect string, offset: 0x7b): long unsigned int <1><24>: Abbrev Number: 0 Compilation Unit @ offset 0x25: Length: 0x4b (32-bit) Version: 4 Abbrev Offset: 0x1b Pointer Size: 8 <0><30>: Abbrev Number: 1 (DW_TAG_compile_unit) <31> DW_AT_producer : (indirect string, offset: 0x8d): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g -gz <35> DW_AT_language : 4 (C++) <36> DW_AT_name : (indirect string, offset: 0xe9): udp.ii <3a> DW_AT_comp_dir : (indirect string, offset: 0x0): /home/espindola/scylla/gdb-issue <3e> DW_AT_stmt_list : 0x1d <1><42>: Abbrev Number: 2 (DW_TAG_namespace) <43> DW_AT_name : std <47> DW_AT_decl_file : 1 <48> DW_AT_decl_line : 0 <49> DW_AT_sibling : <0x5e> <2><4d>: Abbrev Number: 3 (DW_TAG_namespace) <4e> DW_AT_name : (indirect string, offset: 0xf9): __cxx11 <52> DW_AT_decl_file : 2 <53> DW_AT_decl_line : 2 <54> DW_AT_decl_column : 63 <55> DW_AT_export_symbols: 1 <2><55>: Abbrev Number: 4 (DW_TAG_imported_module) <56> DW_AT_decl_file : 2 <57> DW_AT_decl_line : 2 <58> DW_AT_decl_column : 63 <59> DW_AT_import : <0x4d> [Abbrev Number: 3 (DW_TAG_namespace)] <2><5d>: Abbrev Number: 0 <1><5e>: Abbrev Number: 5 (DW_TAG_base_type) <5f> DW_AT_byte_size : 8 <60> DW_AT_encoding : 7 (unsigned) <61> DW_AT_name : (indirect string, offset: 0x7b): long unsigned int <1><65>: Abbrev Number: 5 (DW_TAG_base_type) <66> DW_AT_byte_size : 8 <67> DW_AT_encoding : 5 (signed) <68> DW_AT_name : (indirect string, offset: 0xf0): long int <1><6c>: Abbrev Number: 5 (DW_TAG_base_type) <6d> DW_AT_byte_size : 4 <6e> DW_AT_encoding : 7 (unsigned) <6f> DW_AT_name : (indirect string, offset: 0xdc): unsigned int <1><73>: Abbrev Number: 0 umb@ubuntu:~/tests/2$ readelf --debug-dump=info test.lld Contents of the .debug_info section: Compilation Unit @ offset 0x0: Length: 0x21 (32-bit) Version: 4 Abbrev Offset: 0x0 Pointer Size: 8 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit) <c> DW_AT_producer : (indirect string, offset: 0x12): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <10> DW_AT_language : 4 (C++) <11> DW_AT_name : (indirect string, offset: 0xe9): ip_checksum.ii <15> DW_AT_comp_dir : (indirect string, offset: 0x6a): /home/espindola/scylla/gdb-issue <19> DW_AT_stmt_list : 0x0 <1><1d>: Abbrev Number: 2 (DW_TAG_base_type) <1e> DW_AT_byte_size : 8 <1f> DW_AT_encoding : 7 (unsigned) <20> DW_AT_name : (indirect string, offset: 0x0): long unsigned int <1><24>: Abbrev Number: 0 readelf: Warning: Invalid pointer size (0) in compunit header, using 4 instead Compilation Unit @ offset 0x25: Length: 0x4b000000 (32-bit) Version: 0 Abbrev Offset: 0x1b000400 Pointer Size: 4 readelf: Warning: Debug info is corrupted, .debug_info header at 0x25 has length 4b000000 3) Then I tried to link outputs using your objects with my versions of lld and gold: umb@ubuntu:~/tests/2$ ~/LLVM/build/bin/ld.lld -v LLD 9.0.0 (http://llvm.org/svn/llvm-project/lld/trunk 358661) (compatible with GNU linkers) umb@ubuntu:~/tests/2$ ~/LLVM/build/bin/ld.lld ip_checksum.o udp.o -o test.lld ld.lld: warning: cannot find entry symbol _start; defaulting to 0x201000 umb@ubuntu:~/tests/2$ readelf --debug-dump=info test.lld Contents of the .debug_info section: Compilation Unit @ offset 0x0: Length: 0x21 (32-bit) Version: 4 Abbrev Offset: 0x0 Pointer Size: 8 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit) <c> DW_AT_producer : (indirect string, offset: 0x12): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <10> DW_AT_language : 4 (C++) <11> DW_AT_name : (indirect string, offset: 0xe9): ip_checksum.ii <15> DW_AT_comp_dir : (indirect string, offset: 0x6a): /home/espindola/scylla/gdb-issue <19> DW_AT_stmt_list : 0x0 <1><1d>: Abbrev Number: 2 (DW_TAG_base_type) <1e> DW_AT_byte_size : 8 <1f> DW_AT_encoding : 7 (unsigned) <20> DW_AT_name : (indirect string, offset: 0x0): long unsigned int <1><24>: Abbrev Number: 0 readelf: Warning: Invalid pointer size (0) in compunit header, using 4 instead Compilation Unit @ offset 0x25: Length: 0x4b000000 (32-bit) Version: 0 Abbrev Offset: 0x1b000400 Pointer Size: 4 readelf: Warning: Debug info is corrupted, .debug_info header at 0x25 has length 4b000000 umb@ubuntu:~/tests/2$ ld.gold -v GNU gold (GNU Binutils for Ubuntu 2.31.1) 1.16 umb@ubuntu:~/tests/2$ ld.gold ip_checksum.o udp.o -o test.gold umb@ubuntu:~/tests/2$ readelf --debug-dump=info test.gold Contents of the .debug_info section: Compilation Unit @ offset 0x0: Length: 0x21 (32-bit) Version: 4 Abbrev Offset: 0x0 Pointer Size: 8 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit) <c> DW_AT_producer : (indirect string, offset: 0x21): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <10> DW_AT_language : 4 (C++) <11> DW_AT_name : (indirect string, offset: 0x6c): ip_checksum.ii <15> DW_AT_comp_dir : (indirect string, offset: 0x0): /home/espindola/scylla/gdb-issue <19> DW_AT_stmt_list : 0x0 <1><1d>: Abbrev Number: 2 (DW_TAG_base_type) <1e> DW_AT_byte_size : 8 <1f> DW_AT_encoding : 7 (unsigned) <20> DW_AT_name : (indirect string, offset: 0x7b): long unsigned int <1><24>: Abbrev Number: 0 readelf: Warning: Invalid pointer size (0) in compunit header, using 4 instead Compilation Unit @ offset 0x25: Length: 0x4b000000 (32-bit) Version: 0 Abbrev Offset: 0x1b000400 Pointer Size: 4 readelf: Warning: Debug info is corrupted, .debug_info header at 0x25 has length 4b000000 i.e. both LLD and gold produce the broken output for me with your objects. Next thing I plan to do is probably to build the latest binutils from source and debug the readelf to find out why and where it reports "Warning: compressed section '.debug_info' is corrupted" for your object for me. I'll also try linking with the gold built from head sources.
OK, great! I was able to reproduce it finally. Info is below: When I use the latest Binutils, readelf does not show me error for udp.o i.e. output for readelf 2.32.51.20190420 and 2.31.1 is different! umb@ubuntu:~/tests/2/bup$ /home/umb/binutils/binutils-gdb/build/binutils/readelf -v GNU readelf (GNU Binutils) 2.32.51.20190420 Copyright (C) 2019 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms of the GNU General Public License version 3 or (at your option) any later version. This program has absolutely no warranty. umb@ubuntu:~/tests/2/bup$ /home/umb/binutils/binutils-gdb/build/binutils/readelf --debug-dump=info udp.o > /dev/null Default readelf (2.31.1) still complains: umb@ubuntu:~/tests/2/bup$ readelf -v GNU readelf (GNU Binutils for Ubuntu) 2.31.1 Copyright (C) 2018 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms of the GNU General Public License version 3 or (at your option) any later version. This program has absolutely no warranty. umb@ubuntu:~/tests/2/bup$ readelf --debug-dump=info udp.o > /dev/null readelf: Warning: compressed section '.debug_info' is corrupted Then when I link those objects with latest gold, I see the correct output: umb@ubuntu:~/tests/2$ ~/binutils/binutils-gdb/build/gold/ld-new -v GNU gold (GNU Binutils 2.32.51.20190420) 1.16 umb@ubuntu:~/tests/2$ ~/binutils/binutils-gdb/build/gold/ld-new ip_checksum.o udp.o -o test.gold umb@ubuntu:~/tests/2$ ~/binutils/binutils-gdb/build/binutils/readelf --debug-dump=info test.gold Contents of the .debug_info section: Compilation Unit @ offset 0x0: Length: 0x21 (32-bit) Version: 4 Abbrev Offset: 0x0 Pointer Size: 8 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit) <c> DW_AT_producer : (indirect string, offset: 0x21): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <10> DW_AT_language : 4 (C++) <11> DW_AT_name : (indirect string, offset: 0x6c): ip_checksum.ii <15> DW_AT_comp_dir : (indirect string, offset: 0x0): /home/espindola/scylla/gdb-issue <19> DW_AT_stmt_list : 0x0 <1><1d>: Abbrev Number: 2 (DW_TAG_base_type) <1e> DW_AT_byte_size : 8 <1f> DW_AT_encoding : 7 (unsigned) <20> DW_AT_name : (indirect string, offset: 0x7b): long unsigned int <1><24>: Abbrev Number: 0 Compilation Unit @ offset 0x25: Length: 0x4b (32-bit) Version: 4 Abbrev Offset: 0x1b Pointer Size: 8 <0><30>: Abbrev Number: 1 (DW_TAG_compile_unit) <31> DW_AT_producer : (indirect string, offset: 0x8d): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g -gz <35> DW_AT_language : 4 (C++) <36> DW_AT_name : (indirect string, offset: 0xe9): udp.ii <3a> DW_AT_comp_dir : (indirect string, offset: 0x0): /home/espindola/scylla/gdb-issue <3e> DW_AT_stmt_list : 0x1d <1><42>: Abbrev Number: 2 (DW_TAG_namespace) <43> DW_AT_name : std <47> DW_AT_decl_file : 1 <48> DW_AT_decl_line : 0 <49> DW_AT_sibling : <0x5e> <2><4d>: Abbrev Number: 3 (DW_TAG_namespace) <4e> DW_AT_name : (indirect string, offset: 0xf9): __cxx11 <52> DW_AT_decl_file : 2 <53> DW_AT_decl_line : 2 <54> DW_AT_decl_column : 63 <55> DW_AT_export_symbols: 1 <2><55>: Abbrev Number: 4 (DW_TAG_imported_module) <56> DW_AT_decl_file : 2 <57> DW_AT_decl_line : 2 <58> DW_AT_decl_column : 63 <59> DW_AT_import : <0x4d> [Abbrev Number: 3 (DW_TAG_namespace)] <2><5d>: Abbrev Number: 0 <1><5e>: Abbrev Number: 5 (DW_TAG_base_type) <5f> DW_AT_byte_size : 8 <60> DW_AT_encoding : 7 (unsigned) <61> DW_AT_name : (indirect string, offset: 0x7b): long unsigned int <1><65>: Abbrev Number: 5 (DW_TAG_base_type) <66> DW_AT_byte_size : 8 <67> DW_AT_encoding : 5 (signed) <68> DW_AT_name : (indirect string, offset: 0xf0): long int <1><6c>: Abbrev Number: 5 (DW_TAG_base_type) <6d> DW_AT_byte_size : 4 <6e> DW_AT_encoding : 7 (unsigned) <6f> DW_AT_name : (indirect string, offset: 0xdc): unsigned int <1><73>: Abbrev Number: 0 Now the same objects with almost latest LLD: umb@ubuntu:~/tests/2$ ~/LLVM/build/bin/ld.lld -V LLD 9.0.0 (http://llvm.org/svn/llvm-project/lld/trunk 358661) (compatible with GNU linkers) umb@ubuntu:~/tests/2$ ~/LLVM/build/bin/ld.lld ip_checksum.o udp.o -o test.lld ld.lld: warning: cannot find entry symbol _start; defaulting to 0x201000 umb@ubuntu:~/tests/2$ ~/binutils/binutils-gdb/build/binutils/readelf --debug-dump=info test.lld Contents of the .debug_info section: Compilation Unit @ offset 0x0: Length: 0x21 (32-bit) Version: 4 Abbrev Offset: 0x0 Pointer Size: 8 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit) <c> DW_AT_producer : (indirect string, offset: 0x12): GNU C++14 8.3.1 20190223 (Red Hat 8.3.1-2) -mtune=generic -march=x86-64 -g <10> DW_AT_language : 4 (C++) <11> DW_AT_name : (indirect string, offset: 0xe9): ip_checksum.ii <15> DW_AT_comp_dir : (indirect string, offset: 0x6a): /home/espindola/scylla/gdb-issue <19> DW_AT_stmt_list : 0x0 <1><1d>: Abbrev Number: 2 (DW_TAG_base_type) <1e> DW_AT_byte_size : 8 <1f> DW_AT_encoding : 7 (unsigned) <20> DW_AT_name : (indirect string, offset: 0x0): long unsigned int <1><24>: Abbrev Number: 0 readelf: Warning: Invalid pointer size (0) in compunit header, using 4 instead Compilation Unit @ offset 0x25: Length: 0x4b000000 (32-bit) Version: 0 Abbrev Offset: 0x1b000400 Pointer Size: 4 readelf: Warning: Debug info is corrupted, .debug_info header at 0x25 has length 4b000000 It's a good point to continue investigation I think. Thanks for the inputs, Rafael!
I found the issue finally. Here is the partial readelf -a udp.o output: Section Headers: [Nr] Name Type Address Offset Size EntSize Flags Link Info Align [ 4] .debug_info PROGBITS 0000000000000000 00000040 000000000000004c 0000000000000000 C 0 0 8 You may notice that .debug_info is Compressed and aligned to 8 bytes. Below is disasm of gold and LLD output of the beginning of the second compile unit from the executable produced: gold: 20: 7b 00 jnp 22 <__bss_start-0x400fde> 22: 00 00 add %al,(%rax) 24: 00 4b 00 add %cl,0x0(%rbx) 27: 00 00 add %al,(%rax) //length == 0x0000004b LLD: 20: 00 00 add %al,(%rax) 22: 00 00 add %al,(%rax) 24: 00 00 add %al,(%rax) 26: 00 00 add %al,(%rax) 28: 4b 00 00 rex.WXB add %al,(%r8) //length == 0x4b000000 `length` here is a CU length read by llvm-dwarfdump. Its value is corrupted because of additional zeroes added after first .debug_info (from ip_checksum.o). LLD adds zeroes to align the .debug_info section from udp.o to 8 bytes. That is not what gold does. Looks like after decompression of section we should probably reset its Align field to 1. Or at least we obviously do not want to add any zeroes to .debug_info by ourselves in LLD, because it damages its final content. Tomorrow I am going to look into gold's source code to see what it does and hopefully will prepare a patch.
The patch to readelf that avoids the warning is https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=4207142d6a5d2359170c5f9a140fc1a2351fbda9 The .debug_info section in udp.o is aligned to 8, but the Elf64_Chdr compression header says that the compressed data has an alignment of 1. It looks like lld is and readelf was using the alignment of the compressed data as the alignment of the uncompressed data.
(In reply to Rafael Ávila de Espíndola from comment #26) > It looks like lld is and readelf was using the alignment of the compressed > data as the alignment of the uncompressed data. Yes, you are right. I forgot that we also have a ch_addralign field in a GNU compressed headers. It was used for compressing but never was used for decompressing sections in LLD. The patch for this bug is: https://reviews.llvm.org/D60959 Also, that revealed one more bug: GNU assembler version 2.31.1 and the current llvm-mc (HEAD) set the Align of compressed sections to 1. While GNU assembler version 2.32.51 (x86_64-pc-linux-gnu) sets it to 8 for me now. I think llvm-mc should do the same (perhaps it should be 4 for 32 bit target), because otherwise the code like we have in LLD auto *Hdr = reinterpret_cast<const Chdr64 *>(RawData.data()); might cause a UB. (C11 standard (§ 6.3.2.3, paragraph 7): A pointer to an object type may be converted to a pointer to a different object type. If the resulting pointer is not correctly aligned for the referenced type, the behavior is undefined.)
This bug should be fixed in r358885 Milian, could you confirm it is now fixed for you? Patch for llvm-mc was posted here: https://reviews.llvm.org/D60965
(In reply to George Rimar from comment #28) > This bug should be fixed in r358885 It is fixed for me. Thank you so much! I agree that producers should set the alignment of the compressed section to 4 or 8 since the compression header requires that alignment.
(In reply to George Rimar from comment #28) > This bug should be fixed in r358885 I can confirm that this fixes the bug for me as well. Thank you!
Closing it then. Thanks, everyone for comments and help!
I also just tested this and it works like a charm! Can the fix be backported into the 8.x branch please? It applies cleanly. Thanks a lot everyone for the help
Yeah, it looks like this is a good candidate for a dot release. George, can you file a cherry-pick request?
(In reply to Rui Ueyama from comment #33) > Yeah, it looks like this is a good candidate for a dot release. George, can > you file a cherry-pick request? Done: https://bugs.llvm.org/show_bug.cgi?id=41580
*** Bug 42401 has been marked as a duplicate of this bug. ***