LLVM Bugzilla is read-only and represents the historical archive of all LLVM issues filled before November 26, 2021. Use github to submit LLVM bugs

Bug 46832 - qsort() interceptor broken on FreeBSD
Summary: qsort() interceptor broken on FreeBSD
Status: RESOLVED FIXED
Alias: None
Product: compiler-rt
Classification: Unclassified
Component: asan (show other bugs)
Version: unspecified
Hardware: PC All
: P enhancement
Assignee: Unassigned LLVM Bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-07-24 03:46 PDT by Alexander Richardson
Modified: 2020-09-01 12:21 PDT (History)
2 users (show)

See Also:
Fixed By Commit(s):


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Richardson 2020-07-24 03:46:51 PDT
The qsort interceptor results in an infinite loop on FreeBSD as can be seen from this backtrace:

0x00000000016b6d06 in wrapped_qsort_compar(void const*, void const*) ()
    at /usr/local/poudriere/ports/brooks/devel/llvm-devel/work/llvm-project-61c2a0bb823677ce0e604b92e5dae65d9bd32b6e/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:9690
9690      return qsort_compar(a, b);
(gdb) bt
#0  0x00000000016b6d06 in wrapped_qsort_compar(void const*, void const*) ()
    at /usr/local/poudriere/ports/brooks/devel/llvm-devel/work/llvm-project-61c2a0bb823677ce0e604b92e5dae65d9bd32b6e/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:9690
#1  0x0000000802efaa70 in qsort () from /lib/libc.so.7
#2  0x00000000016b6df3 in __interceptor_qsort ()
    at /usr/local/poudriere/ports/brooks/devel/llvm-devel/work/llvm-project-61c2a0bb823677ce0e604b92e5dae65d9bd32b6e/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:9710
#3  0x0000000802efa99f in qsort () from /lib/libc.so.7
#4  0x00000000016b6df3 in __interceptor_qsort ()
    at /usr/local/poudriere/ports/brooks/devel/llvm-devel/work/llvm-project-61c2a0bb823677ce0e604b92e5dae65d9bd32b6e/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:9710
#5  0x0000000001e9284f in llvm::array_pod_sort<std::__1::__wrap_iter<(anonymous namespace)::ELFWriter::ELFSymbolData*> > (Start=..., End=...)
    at /exports/users/alr48/sources/llvm-project/llvm/include/llvm/ADT/STLExtras.h:1399
#6  0x0000000001e81e83 in (anonymous namespace)::ELFWriter::computeSymbolTable (this=0x7fffffffbf20, Asm=...,
    Layout=..., SectionIndexMap=..., RevGroupMap=..., SectionOffsets=...)


The problem is that qsort() calls qsort() on individual parts of the array which ends up calling the interceptor again. However, the interceptor uses a thread local function pointer for the comparator wrapper.

The interceptor code does not handle this case:

  qsort_compar_f old_compar = qsort_compar;
  qsort_compar = compar;
  SIZE_T old_size = qsort_size;
  qsort_size = size;
  REAL(qsort)(base, nmemb, size, wrapped_qsort_compar);
  qsort_compar = old_compar;
  qsort_size = old_size;

In the first call it will assign the real comparator to qsort_compar, but in the recursive call from inside qsort qsort_compar will end up being set to wrapped_qsort_compar and it ends up being an infinite loop.
Comment 1 Alexander Richardson 2020-07-24 08:05:46 PDT
https://reviews.llvm.org/D84509
Comment 2 Brad Smith 2020-09-01 12:21:47 PDT
Fixed by 8803ebcf3b562172687321318c423f39f22b2e5b.