Commit graph

196 commits

Author SHA1 Message Date
Evan Huus
36e68f7b36 From Jim Young via https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9613
Don't use "rand" as a variable name, it shadows a global in recent MacOSX
stdlib.h

svn path=/trunk/; revision=54598
2014-01-05 00:47:24 +00:00
Evan Huus
1dfa4e2697 Add wmem_destroy_list, wmem_destroy_queue, wmem_destroy_stack so that those
data-structures can be used with manually managed memory.

svn path=/trunk/; revision=54432
2013-12-24 01:17:41 +00:00
Anders Broman
350a207a22 Try to please the buildbot.
svn path=/trunk/; revision=54281
2013-12-20 06:50:46 +00:00
Anders Broman
5fbfa6c351 Fix [-Wmissing-prototypes]
svn path=/trunk/; revision=54280
2013-12-20 06:30:13 +00:00
Jakub Zawadzki
2c015bb386 Fix some const/ not const warnings.
svn path=/trunk/; revision=54114
2013-12-15 12:35:00 +00:00
Evan Huus
d04079eedc Expose wmem_list_remove_frame to make it efficient to do pop operations as
wmem_list_remove_frame(wmem_list_tail())

svn path=/trunk/; revision=53088
2013-11-05 17:28:19 +00:00
Evan Huus
1f3549168d Don't link explicitly with libwmem, it's already in libwireshark and the symbol
doubling leads to all sorts of very subtle badness (including test failures due
to funny internal assertions because the two wmems have mismatching state).

Make wmem_init and wmem_cleanup PUBLIC instead of LOCAL so that they don't get
stripped and don't cause a link failure when trying to build oids_test (now that
it's not linking with libwmem explicitly). There is possibly a better way to fix
this, but I'm not sure what it is.

svn path=/trunk/; revision=52694
2013-10-19 19:40:02 +00:00
Evan Huus
7b3dfaa525 As suggested separately by both Jakub and Anders: only read the
WIRESHARK_DEBUG_WMEM_OVERRIDE environment variable once in wmem_init, not every
time wmem_allocator_new is called. We currently create a new pinfo pool for
every packet we dissect, so this is a small performance win, especially when
getenv is slow (which may happen if a large number of environment variables are
set, such as when fuzz-testing).

svn path=/trunk/; revision=52634
2013-10-15 23:16:35 +00:00
Evan Huus
7d66e3806b Don't do canaries in blocks of guint32, it appears to cause alignment issues
when running tests on Solaris. Revert back to byte-at-a-time, but do fewer bytes
to avoid a performance hit.

svn path=/trunk/; revision=52617
2013-10-15 13:15:38 +00:00
Evan Huus
e7a0c26bec Subsume README.binarytrees into wmem doxygen. The README was out of date
anyways, since Michael made key operations non-destructive in r44380.

svn path=/trunk/; revision=52583
2013-10-13 13:13:24 +00:00
Evan Huus
dc0b7253e4 Don't assert that the key to a wmem tree operation is short. If XMPP wants to
use a 1500-byte string as a key, so be it. It will be slow, but at least it will
work.

Fixes https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9078

svn path=/trunk/; revision=52261
2013-09-28 23:45:18 +00:00
Evan Huus
5f7187c658 Cycle the recycler in realloc() and free() as well, reducing memory usage of the
stress-test by another ~10%.

svn path=/trunk/; revision=51840
2013-09-08 15:18:55 +00:00
Evan Huus
3c1e67c22d Send all alloc and free calls within wmem through wmem_alloc and wmem_free with
a NULL allocator. This gives us a single, central place to handle out-of-memory
errors (by, for example, throwing an exception) for basically all of epan.

The only remaining glib memory that is directly allocated is for the hash tables
used by the simple and strict allocators.

svn path=/trunk/; revision=51627
2013-09-01 13:37:38 +00:00
Evan Huus
d939a6dec0 Do our list manipulation entirely after the realloc, so that if the realloc
doesn't return (do to, for example, throwing an exception) the allocator is
left in a consistent state.

svn path=/trunk/; revision=51626
2013-09-01 13:07:22 +00:00
Evan Huus
f875bf8de2 In the strict allocator, do canaries in units of guint32 instead of guint8. This
makes canary checking about 20% faster, which should speed up fuzz-testing now
that more and more dissectors use wmem.

svn path=/trunk/; revision=51620
2013-08-31 16:36:37 +00:00
Guy Harris
e8f42c6b48 Include config.h first; it defines _FILE_OFFSET_BITS, and if some system
header later causes it to be redefined - as happens on my Solaris 11
virtual machine - we get a redefinition warning, which gets treated as
an error.

svn path=/trunk/; revision=51344
2013-08-14 00:31:14 +00:00
Evan Huus
d8f00e22e1 As suggested in https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9033
Make epan_free a no-op if the pointer is NULL. This fixes 99% of the cases
causing problems for wmem_leave_file_scope() - remove that XXX comment and add
back the assertion.

Remove the cleanup_dissection call from epan_cleanup, it doesn't make sense
there. init_dissection is only called from epan_new, so cleanup_dissection
should only be called from epan_free.

Add one missing epan_free call to tshark revealed by the above changes.

svn path=/trunk/; revision=51342
2013-08-13 22:41:34 +00:00
Evan Huus
616324c21e Be sure to initialize the jumbo flag when creating a new chunk header by
splitting an existing chunk. Thanks to Jeff for catching this one.

svn path=/trunk/; revision=51325
2013-08-12 23:35:35 +00:00
Evan Huus
bf0f0ec4d9 Reorder and group chunk header flags. Reduces the bit-twiddling needed by the
compiler and provides a surprising (if fairly small) performance win.

svn path=/trunk/; revision=51305
2013-08-11 23:49:56 +00:00
Evan Huus
dc46cb0f29 Split jumbo_free and jumbo_realloc into their own functions for readability and
consistency (they are called just once and will be inlined by any reasonable
compiler).

Also add some comments, fix some spacing etc. No functional changes.

svn path=/trunk/; revision=51304
2013-08-11 22:32:46 +00:00
Evan Huus
6eef00ac05 Move the null check outside the function call. Simpler, and more efficient for
compilers that decide not to inline.

svn path=/trunk/; revision=51303
2013-08-11 22:20:35 +00:00
Evan Huus
497e53ee06 Little optimization: move two statements inside an if block.
svn path=/trunk/; revision=51302
2013-08-11 19:33:30 +00:00
Evan Huus
36ff6f2c25 Add support for allocating more than will fit in a single block, through the use
of a jumbo flag in the chunk header.

svn path=/trunk/; revision=51299
2013-08-11 17:18:19 +00:00
Evan Huus
959ee8f5c0 Instead of using a glib slist to track the list of blocks, embed a short
doubly-linked list at the head of each block. This was intended as a step
towards supporting allocations bigger than the usual block size, but also shows
up as a 2% performance improvement in the speed test, so win-win.

svn path=/trunk/; revision=51298
2013-08-11 16:06:07 +00:00
Evan Huus
3bac7e845b Small optimization: the master-list and recycler cases are different enough that
doing all the safety checks was unnecessarily slow. Do only the appropriate
checks for each case.

svn path=/trunk/; revision=51296
2013-08-11 14:20:00 +00:00
Alexis La Goutte
d26b7fc516 Fix (-W)documentation error found by Clang
../../epan/wmem/wmem_user_cb.h:52:11: error: parameter 'allocator' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param allocator The allocator that triggered this callback.
          ^~~~~~~~~
../../epan/wmem/wmem_user_cb.h:53:11: error: parameter 'event' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param event     The event type that triggered this callback.
          ^~~~~
../../epan/wmem/wmem_user_cb.h:54:11: error: parameter 'user_data' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param user_data Whatever user_data was originally passed to the call to
          ^~~~~~~~~
../../epan/wmem/wmem_user_cb.h:63:11: error: parameter 'recurring' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param recurring If this is FALSE then the callback is called exactly once.
          ^~~~~~~~~
4 errors generated.

svn path=/trunk/; revision=51259
2013-08-10 21:28:42 +00:00
Evan Huus
1c2cb57a38 Enhance verification to assert that the recycler and master list contain all
free chunks of sufficient size.

svn path=/trunk/; revision=51234
2013-08-09 11:20:02 +00:00
Evan Huus
c5319ebc93 One more chunk header alignment fix.
svn path=/trunk/; revision=51218
2013-08-08 12:23:23 +00:00
Evan Huus
0ebe18dcea Make sure our chunk headers are large enough to correctly align the actual data
pointer we return.

svn path=/trunk/; revision=51217
2013-08-08 11:27:39 +00:00
Evan Huus
d8684cf32d When splitting a free chunk, if we don't have room for a free header don't even
bother splitting. This greatly simplifies the logic, trims another 4% off the
fast path, and doesn't actually affect the results at all because of the way we
pad for alignment anyways.

svn path=/trunk/; revision=51216
2013-08-08 11:16:10 +00:00
Evan Huus
c4e966b880 Combine two ternary ?: into a single if. Shaves about 1.5% off the fast timing
test.

svn path=/trunk/; revision=51112
2013-08-02 01:58:52 +00:00
Evan Huus
a7b30a2a2c Small simplification
svn path=/trunk/; revision=51111
2013-08-02 01:51:06 +00:00
Evan Huus
c85ec68e58 Remove unnecessary null check.
svn path=/trunk/; revision=51110
2013-08-02 01:36:52 +00:00
Evan Huus
64ed83d793 Rewrite the merge_free routine to keep the merged chunk on the master free list
if the right-hand merge target was there originally. This brings memory usage
down another ~40% when running the heavy test suite.

This also lets us extract the master-list check out of unfree() since it is now
only relevant at a single caller, and turns unfree into the more understandable
remove_from_recycler().

svn path=/trunk/; revision=51104
2013-08-02 00:12:12 +00:00
Evan Huus
8fafa9b758 Add wmem_tree_is_empty(). Also permit lookups on a NULL tree for compatibility
with emem.

svn path=/trunk/; revision=51045
2013-07-31 01:48:47 +00:00
Evan Huus
44d53b4042 More little optimizations.
svn path=/trunk/; revision=51014
2013-07-29 16:55:02 +00:00
Evan Huus
0fb6d96507 Another little optimization.
Removes one branch from the hot path, deduplicates one function call in the cold
path by effectively falling through, and makes it more obvious what the code is
actually trying to do.

svn path=/trunk/; revision=51013
2013-07-29 16:47:30 +00:00
Graham Bloice
2f6ee944d8 Added more build artefacts to the clean target
svn path=/trunk/; revision=50977
2013-07-28 14:40:56 +00:00
Evan Huus
a9bccc8dc8 Don't force the allocator choice in the data-structure tests, we want to be able
to override that to simple for valgrinding (we still force the allocator in the
allocator and timing tests, of course).

svn path=/trunk/; revision=50971
2013-07-28 12:55:37 +00:00
Evan Huus
b43996c8ab Fix a bug in the gc routine discovered while formally working out parts of the
algorithm.

Also add a test case to excercise it for future.

svn path=/trunk/; revision=50970
2013-07-28 12:46:44 +00:00
Evan Huus
b9870c0969 Little optimizations.
Move a few assignments around to avoid one extra subtraction. I suspect having
the two if statements next to each other is friendly to the compiler's optimizer
as well.

Shaves ~1.3% off my timing tests, bringing the new design *very* close to the
old one in raw allocation speed.

svn path=/trunk/; revision=50961
2013-07-28 00:00:37 +00:00
Evan Huus
e7dfea82f3 Little things make big differences: somehow while I was experimenting this
unobtrusive if statement got dropped. Without it the allocator exhibits the old
bad behaviour of 3x memory usage and heavy fragmentation.

We want it back, thank you very much.

svn path=/trunk/; revision=50960
2013-07-27 23:35:59 +00:00
Evan Huus
9b9467cb79 Small optimization: conjoin two mutually exclusive ifs with an else.
svn path=/trunk/; revision=50959
2013-07-27 23:01:24 +00:00
Evan Huus
720cce4102 Simplify out an addition that happened a lot by performing it once on the
variable it affected, since that variable wasn't used for any other purpose.

svn path=/trunk/; revision=50958
2013-07-27 22:51:27 +00:00
Evan Huus
3663498a8b Redesign of the wmem block allocator.
What was becoming apparent as more dissectors started using wmem was that the
old block allocator design had issues with memory fragmentation. This keeps the
same underlying memory layout, but completely changes how free blocks are kept.
It runs about 3% slower in my tests (still an order of magnitude faster than
g_malloc) but uses about 1/3 the memory.

I suspect some simple optimizations could reclaim that 3% as well - the design
is fast, but I did not code particularly for speed.

Thoroughly tested with the existing test suite (which caught half a dozen bugs
in my first draft) so it should actually work!

svn path=/trunk/; revision=50955
2013-07-27 22:27:28 +00:00
Evan Huus
a076d6332e Use a #define, tweak a comment.
svn path=/trunk/; revision=50954
2013-07-27 21:56:43 +00:00
Evan Huus
c693d9c951 whitespace
svn path=/trunk/; revision=50919
2013-07-26 16:42:06 +00:00
Evan Huus
037fc022da Add wmem queue 'implementation' by wrapping wmem_list and wmem_stack.
Also a bit of misc. refactoring of the stack while I was there, and doc tweaks.

svn path=/trunk/; revision=50769
2013-07-21 21:13:30 +00:00
Evan Huus
6635f5ef67 Replace wmem slist (singly-linked) with wmem list (doubly-linked).
The overhead is not large, and it makes append much faster (O(1) vs O(n)).

It also will make a queue easy to add, which I need for a dissector I'm
writing...

svn path=/trunk/; revision=50744
2013-07-20 20:33:38 +00:00
Chris Maynard
1c19a0d61e Fix warnings: format '%blah' expects argument of type 'foo', but argument 'i' has type 'bar' [-Wformat].
svn path=/trunk/; revision=50639
2013-07-15 21:22:34 +00:00