Commit Graph

185 Commits

Author SHA1 Message Date
Evan Huus 5f7187c658 Cycle the recycler in realloc() and free() as well, reducing memory usage of the
stress-test by another ~10%.

svn path=/trunk/; revision=51840
2013-09-08 15:18:55 +00:00
Evan Huus 3c1e67c22d Send all alloc and free calls within wmem through wmem_alloc and wmem_free with
a NULL allocator. This gives us a single, central place to handle out-of-memory
errors (by, for example, throwing an exception) for basically all of epan.

The only remaining glib memory that is directly allocated is for the hash tables
used by the simple and strict allocators.

svn path=/trunk/; revision=51627
2013-09-01 13:37:38 +00:00
Evan Huus d939a6dec0 Do our list manipulation entirely after the realloc, so that if the realloc
doesn't return (do to, for example, throwing an exception) the allocator is
left in a consistent state.

svn path=/trunk/; revision=51626
2013-09-01 13:07:22 +00:00
Evan Huus f875bf8de2 In the strict allocator, do canaries in units of guint32 instead of guint8. This
makes canary checking about 20% faster, which should speed up fuzz-testing now
that more and more dissectors use wmem.

svn path=/trunk/; revision=51620
2013-08-31 16:36:37 +00:00
Guy Harris e8f42c6b48 Include config.h first; it defines _FILE_OFFSET_BITS, and if some system
header later causes it to be redefined - as happens on my Solaris 11
virtual machine - we get a redefinition warning, which gets treated as
an error.

svn path=/trunk/; revision=51344
2013-08-14 00:31:14 +00:00
Evan Huus d8f00e22e1 As suggested in https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9033
Make epan_free a no-op if the pointer is NULL. This fixes 99% of the cases
causing problems for wmem_leave_file_scope() - remove that XXX comment and add
back the assertion.

Remove the cleanup_dissection call from epan_cleanup, it doesn't make sense
there. init_dissection is only called from epan_new, so cleanup_dissection
should only be called from epan_free.

Add one missing epan_free call to tshark revealed by the above changes.

svn path=/trunk/; revision=51342
2013-08-13 22:41:34 +00:00
Evan Huus 616324c21e Be sure to initialize the jumbo flag when creating a new chunk header by
splitting an existing chunk. Thanks to Jeff for catching this one.

svn path=/trunk/; revision=51325
2013-08-12 23:35:35 +00:00
Evan Huus bf0f0ec4d9 Reorder and group chunk header flags. Reduces the bit-twiddling needed by the
compiler and provides a surprising (if fairly small) performance win.

svn path=/trunk/; revision=51305
2013-08-11 23:49:56 +00:00
Evan Huus dc46cb0f29 Split jumbo_free and jumbo_realloc into their own functions for readability and
consistency (they are called just once and will be inlined by any reasonable
compiler).

Also add some comments, fix some spacing etc. No functional changes.

svn path=/trunk/; revision=51304
2013-08-11 22:32:46 +00:00
Evan Huus 6eef00ac05 Move the null check outside the function call. Simpler, and more efficient for
compilers that decide not to inline.

svn path=/trunk/; revision=51303
2013-08-11 22:20:35 +00:00
Evan Huus 497e53ee06 Little optimization: move two statements inside an if block.
svn path=/trunk/; revision=51302
2013-08-11 19:33:30 +00:00
Evan Huus 36ff6f2c25 Add support for allocating more than will fit in a single block, through the use
of a jumbo flag in the chunk header.

svn path=/trunk/; revision=51299
2013-08-11 17:18:19 +00:00
Evan Huus 959ee8f5c0 Instead of using a glib slist to track the list of blocks, embed a short
doubly-linked list at the head of each block. This was intended as a step
towards supporting allocations bigger than the usual block size, but also shows
up as a 2% performance improvement in the speed test, so win-win.

svn path=/trunk/; revision=51298
2013-08-11 16:06:07 +00:00
Evan Huus 3bac7e845b Small optimization: the master-list and recycler cases are different enough that
doing all the safety checks was unnecessarily slow. Do only the appropriate
checks for each case.

svn path=/trunk/; revision=51296
2013-08-11 14:20:00 +00:00
Alexis La Goutte d26b7fc516 Fix (-W)documentation error found by Clang
../../epan/wmem/wmem_user_cb.h:52:11: error: parameter 'allocator' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param allocator The allocator that triggered this callback.
          ^~~~~~~~~
../../epan/wmem/wmem_user_cb.h:53:11: error: parameter 'event' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param event     The event type that triggered this callback.
          ^~~~~
../../epan/wmem/wmem_user_cb.h:54:11: error: parameter 'user_data' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param user_data Whatever user_data was originally passed to the call to
          ^~~~~~~~~
../../epan/wmem/wmem_user_cb.h:63:11: error: parameter 'recurring' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param recurring If this is FALSE then the callback is called exactly once.
          ^~~~~~~~~
4 errors generated.

svn path=/trunk/; revision=51259
2013-08-10 21:28:42 +00:00
Evan Huus 1c2cb57a38 Enhance verification to assert that the recycler and master list contain all
free chunks of sufficient size.

svn path=/trunk/; revision=51234
2013-08-09 11:20:02 +00:00
Evan Huus c5319ebc93 One more chunk header alignment fix.
svn path=/trunk/; revision=51218
2013-08-08 12:23:23 +00:00
Evan Huus 0ebe18dcea Make sure our chunk headers are large enough to correctly align the actual data
pointer we return.

svn path=/trunk/; revision=51217
2013-08-08 11:27:39 +00:00
Evan Huus d8684cf32d When splitting a free chunk, if we don't have room for a free header don't even
bother splitting. This greatly simplifies the logic, trims another 4% off the
fast path, and doesn't actually affect the results at all because of the way we
pad for alignment anyways.

svn path=/trunk/; revision=51216
2013-08-08 11:16:10 +00:00
Evan Huus c4e966b880 Combine two ternary ?: into a single if. Shaves about 1.5% off the fast timing
test.

svn path=/trunk/; revision=51112
2013-08-02 01:58:52 +00:00
Evan Huus a7b30a2a2c Small simplification
svn path=/trunk/; revision=51111
2013-08-02 01:51:06 +00:00
Evan Huus c85ec68e58 Remove unnecessary null check.
svn path=/trunk/; revision=51110
2013-08-02 01:36:52 +00:00
Evan Huus 64ed83d793 Rewrite the merge_free routine to keep the merged chunk on the master free list
if the right-hand merge target was there originally. This brings memory usage
down another ~40% when running the heavy test suite.

This also lets us extract the master-list check out of unfree() since it is now
only relevant at a single caller, and turns unfree into the more understandable
remove_from_recycler().

svn path=/trunk/; revision=51104
2013-08-02 00:12:12 +00:00
Evan Huus 8fafa9b758 Add wmem_tree_is_empty(). Also permit lookups on a NULL tree for compatibility
with emem.

svn path=/trunk/; revision=51045
2013-07-31 01:48:47 +00:00
Evan Huus 44d53b4042 More little optimizations.
svn path=/trunk/; revision=51014
2013-07-29 16:55:02 +00:00
Evan Huus 0fb6d96507 Another little optimization.
Removes one branch from the hot path, deduplicates one function call in the cold
path by effectively falling through, and makes it more obvious what the code is
actually trying to do.

svn path=/trunk/; revision=51013
2013-07-29 16:47:30 +00:00
Graham Bloice 2f6ee944d8 Added more build artefacts to the clean target
svn path=/trunk/; revision=50977
2013-07-28 14:40:56 +00:00
Evan Huus a9bccc8dc8 Don't force the allocator choice in the data-structure tests, we want to be able
to override that to simple for valgrinding (we still force the allocator in the
allocator and timing tests, of course).

svn path=/trunk/; revision=50971
2013-07-28 12:55:37 +00:00
Evan Huus b43996c8ab Fix a bug in the gc routine discovered while formally working out parts of the
algorithm.

Also add a test case to excercise it for future.

svn path=/trunk/; revision=50970
2013-07-28 12:46:44 +00:00
Evan Huus b9870c0969 Little optimizations.
Move a few assignments around to avoid one extra subtraction. I suspect having
the two if statements next to each other is friendly to the compiler's optimizer
as well.

Shaves ~1.3% off my timing tests, bringing the new design *very* close to the
old one in raw allocation speed.

svn path=/trunk/; revision=50961
2013-07-28 00:00:37 +00:00
Evan Huus e7dfea82f3 Little things make big differences: somehow while I was experimenting this
unobtrusive if statement got dropped. Without it the allocator exhibits the old
bad behaviour of 3x memory usage and heavy fragmentation.

We want it back, thank you very much.

svn path=/trunk/; revision=50960
2013-07-27 23:35:59 +00:00
Evan Huus 9b9467cb79 Small optimization: conjoin two mutually exclusive ifs with an else.
svn path=/trunk/; revision=50959
2013-07-27 23:01:24 +00:00
Evan Huus 720cce4102 Simplify out an addition that happened a lot by performing it once on the
variable it affected, since that variable wasn't used for any other purpose.

svn path=/trunk/; revision=50958
2013-07-27 22:51:27 +00:00
Evan Huus 3663498a8b Redesign of the wmem block allocator.
What was becoming apparent as more dissectors started using wmem was that the
old block allocator design had issues with memory fragmentation. This keeps the
same underlying memory layout, but completely changes how free blocks are kept.
It runs about 3% slower in my tests (still an order of magnitude faster than
g_malloc) but uses about 1/3 the memory.

I suspect some simple optimizations could reclaim that 3% as well - the design
is fast, but I did not code particularly for speed.

Thoroughly tested with the existing test suite (which caught half a dozen bugs
in my first draft) so it should actually work!

svn path=/trunk/; revision=50955
2013-07-27 22:27:28 +00:00
Evan Huus a076d6332e Use a #define, tweak a comment.
svn path=/trunk/; revision=50954
2013-07-27 21:56:43 +00:00
Evan Huus c693d9c951 whitespace
svn path=/trunk/; revision=50919
2013-07-26 16:42:06 +00:00
Evan Huus 037fc022da Add wmem queue 'implementation' by wrapping wmem_list and wmem_stack.
Also a bit of misc. refactoring of the stack while I was there, and doc tweaks.

svn path=/trunk/; revision=50769
2013-07-21 21:13:30 +00:00
Evan Huus 6635f5ef67 Replace wmem slist (singly-linked) with wmem list (doubly-linked).
The overhead is not large, and it makes append much faster (O(1) vs O(n)).

It also will make a queue easy to add, which I need for a dissector I'm
writing...

svn path=/trunk/; revision=50744
2013-07-20 20:33:38 +00:00
Chris Maynard 1c19a0d61e Fix warnings: format '%blah' expects argument of type 'foo', but argument 'i' has type 'bar' [-Wformat].
svn path=/trunk/; revision=50639
2013-07-15 21:22:34 +00:00
Evan Huus e1268955c7 Add wmem_array_sort() which just hands off to the standard library's qsort().
svn path=/trunk/; revision=50411
2013-07-06 17:47:32 +00:00
Guy Harris 24142c4ee4 In some versions of some OSes, even <string.h> declares index() under
some circumstances.  Use array_index as a variable name instead, to
avoid warnings.

svn path=/trunk/; revision=50404
2013-07-06 08:33:07 +00:00
Evan Huus 777f060f01 Trivial code style: use wmem_new in a few places, and remove extraneous
whitespace.

svn path=/trunk/; revision=50401
2013-07-06 04:08:07 +00:00
Evan Huus c574f7d1ea Simple growable array implementation for wmem.
svn path=/trunk/; revision=50400
2013-07-06 04:02:08 +00:00
Evan Huus 31bd48bce3 *sigh*
From the comment above wmem_tree_insert32_array():
 * If you use ...32_array() calls you MUST make sure that every single node
 * you add to a specific tree always has a key of exactly the same number of
 * keylen words or things will most likely crash. Or at least that every single
 * item that sits behind the same top level node always have exactly the same
 * number of words.

So clearly generating thousands of keys with random lengths while testing is
going to cause problems. Generate a set of random lengths, then use those
lengths consistently (but still generating random keys of those lengths).

Should hopefully fix the intermittent build-bot failures.

(unfortunately this does not manifest nicely, and I cannot see an easy way to
assert it so that we catch other people trying to use different-length key
subtrees)

svn path=/trunk/; revision=50184
2013-06-27 03:44:48 +00:00
Evan Huus ffe6d9c4d6 Misc little cleanups to wmem tree (comments, whitespace, assertions)
svn path=/trunk/; revision=50182
2013-06-27 03:01:53 +00:00
Evan Huus 32f4010527 Remove debugging lines from wmem_test, I think I've figured out the problem.
svn path=/trunk/; revision=50181
2013-06-27 03:00:33 +00:00
Evan Huus a69be407ab Arrays seems to be where the failure is, so remove excess verbiage around other
tests, and add more details to that section.

Now we wait for the buildbots to fail again...

svn path=/trunk/; revision=50156
2013-06-25 22:32:53 +00:00
Chris Maynard ce088c334d makefile.nmake -> Makefile.nmake and minor whitespace (spaces -> tabs) changes.
svn path=/trunk/; revision=50140
2013-06-25 15:32:49 +00:00
Evan Huus 6ebdc45af1 Fix wmem_tree.h ifdef guards, hopefully fixing the check-abi buildbot.
svn path=/trunk/; revision=50132
2013-06-24 22:37:04 +00:00
Evan Huus 29c75dae91 So it comes to this...
Put in a whole bunch of stderr output in the wmem tree tests in the hopes that
the next time one of the buildbots randomly (and irreproducibly) fails on this
step we'll have at least a bit of a hint as to where it happened.

svn path=/trunk/; revision=50131
2013-06-24 21:59:02 +00:00