How to produce backtraces from an I/O hooking preload library?

I am creating a shared library that overrides the open(2) standard routine. Its purpose is to log all uses of this API in the program under test,

LD_PRELOAD=libiohook.so ./test

And produce useful reports, including for each call of the system IO function, a backtrace of how execution ended up there. The idea will be to extend it to all C I/O interfaces.

My hooking library looks like this,

// C standard declaration
typedef int (*orig_open_f_type)(const char *pathname, int flags);
...
// My hook implementation,
int open(const char *pathname, int flags, ...) {
    FILE* log = stdout;

    // Get the "original" open function
    orig_open_f_type orig_open;
    orig_open = (orig_open_f_type)dlsym(RTLD_NEXT, "open");

    print_stack_trace();

    int fd = orig_open(pathname, flags);
    if (fd == -1)
    {
        library_printf(log,"INTERCEPT open PATH=%s (FAILED)\n", pathname);
        return -1;
    }

    library_printf(log,"INTERCEPT open FD=%d PATH=%s\n", fd, pathname);
    fflush(log);
    return fd;
}

The problem I’m having is how to implement print_stack_trace() in this context of being a preloaded library.

I tried three approaches so far,

execinfo.h – This comes from the GNU C library, but I was getting incomplete symbol information for backtraces, although it did appear to work for the most part. I suspected that my new toolchain was generating metadata it couldn’t read in some cases, because I compiled everything with stack walker friendly flags, such as -fno-omit-frame-pointers and so on. Thought I’d look at alternative to compare, but …
libunwind. Unfortunatley, part of the startup of libunwind calls open(2) in /proc/pid/maps, I assume so it can do a better job than the C library was doing. That ends in a stack exhaustion crash due to calling back into my hooking library’s open(2), which calls back into libunwind, and so on… There’s no way to configure libunwind to call a different function for open(2) (the result of RTLD_NEXT) that I could figure out.
libbacktrace. This builds on libunwind and causes the same issue, with the same problem of not being configurable as to which open(2) implementation to use.

Any ideas how I can write my IO preload library to robustly collect backtraces with logging? The idea will be to extend it to all the C standard IO calls.

You need to sign in to view this answers

Related Post