There are a ton of libsoup/GLib-related "leaks" that we can't whitelist
and with leak detective active there is a delay that interestingly doesn't
happen with soup_session_sync_new(), so tests failed with a timeout (actually
they hung due to the lock in the fetcher manager).
On Travis, the curl plugin is used for the tests, so that's not an issue
there (and without LD the tests complete quickly and successfully).
Checking for whitelisted functions in every backtrace is not very
efficient. And because OpenSSL 1.1 does no proper cleanup anymore until
the process is terminated there are now a lot more "leaks" to ignore.
For instance, in the openssl-ikev2/rw-cert scenario, just starting and
stopping the daemon (test vectors are checked) now causes 3594 whitelisted
leaks compared to the 849 before. This prolonged the shutdown of the
daemon on each guest in every scenario, amounting to multiple seconds of
additional runtime for every affected scenario. But even with this
patch there is still some overhead, compared to running the scenarios on
jessie.
This also changes how unknown/corrupted memory is handled in the free()
and realloc() hooks in general.
Incorporates changes provided by Thomas Egerer who ran into a similar
issue.
Due to the mangled C++ function names it's tricky to be more specific. The
"leaked" allocations are from a static hashtable containing EC groups.
There is another leak caused by the locking allocator singleton
(triggered by the first function that uses it, usually initialization of
a cipher, but could be a hasher in other test runners), but we can avoid
that with a Botan config option.
In case an external thread calls into our code and logs messages, a thread
object is allocated that will never be released. Even if we try to clean
up the object via thread value destructor there is no guarantee that the
thread actually terminates before we check for leaks, which seems to be the
case for the Ada Tasking threads.
Some of these are pretty broad, so maybe an alternative option is to
not use the soup plugin in the openssl-ikev2/rw-suite-b* scenarios. But
the plugin is not tested anywhere else so lets go with this for now.
If a function we whitelist allocates memory while leak detective is enabled
but only frees it after LD has already been disabled, free() will get called
with invalid pointers (not pointing to the actually allocated memory by LD),
which will cause checks in the C library to fail and the program to crash.
This tries to detect such cases and calling free with the correct pointer.
Lots of static data is allocated in this function, which isn't freed until
the library is unloaded (we can't call OPENSSL_cleanup() as initialization
would fail when calling it again later). When enabling the leak
detective the test runner eventually crashes as all the data allocated during
initialization has an invalid size when freed after leak detective has been
unloaded.
Newer versions of GCC are too "smart" and replace a call to malloc(X)
followed by a call to memset(0,X) with a call co calloc(), which obviously
results in an infinite loop when it does that in our own calloc()
implementation. Using `volatile` for the variable storing the total size
prevents the optimization and we actually call malloc().
This prevented `stroke memusage` from reporting the leaks on the
console. Instead, they were sent to the callbacks set up by libstrongswan.
Fixes a426851f63 ("leak-detective: Use callback functions to report
leaks and usage information").
gcry_check_version() does not free statically allocated resources. However,
we can't whitelist it in some versions, as it is not a resolvable symbol name.
Instead, whitelist our own plugin constructor function.
If realloc return a pointer value different from the value to be
reallocated, a double free can occur in this context.
Signed-off-by: Thomas Egerer <thomas.egerer@secunet.com>
If realloc moves an allocation, the original allocation gets freed. We
therefore must remove the hdr from the list, as it is invalid. We can add it
afterwards once it has been updated, allowing us to unlock the list during
reallocation.
Some static allocations in plugins won't get freed, because in the test case
process the plugins are not destroyed. If a plugin would clean up allocations
done while just using the plugin, these show up as leak in the child process,
letting tests fail.
If uses of dlopen(), e.g. when loading plugins, produce errors an error
string could get allocated dynamically. At this point realloc() might not
yet be resolved and when dlsym() is later called by leak detective to do
so the error string might get freed while leak detective is disabled and
real_free() will be called with a pointer into one of leak detective's
memory blocks instead of a pointer to the block itself, causing a SIGSEGV.
malloc hooks have become deprecated, and their use has always been problematic,
especially in multi-threaded applications. Replace the functionality by
overriding all malloc functions and query the system allocator functions
using dlsym() with RTLD_NEXT.
Tobias Brunner