Commit Graph

206 Commits

Author SHA1 Message Date
Evan Huus b238cd1541 Scrap wmem splay trees for now.
There is confusion about API usage, and problems on my part concerning whether
keys should be compared signed or unsigned, and how to do that efficiently.
Unsigned keys in particular were behaving oddly.

Change-Id: I075693bbd04c15f79f24f9a24006003a914cc572
Reviewed-on: https://code.wireshark.org/review/924
Reviewed-by: Evan Huus <eapache@gmail.com>
2014-04-02 17:14:16 +00:00
Alexis La Goutte 757aa33220 Continue to remove $Id$ from top of file
(Using sed : sed -i '/^\# \$Id\$/,+1 d') (start with dash)

Change-Id: Ia4b5a6c2302f6a531f6a86c1ec3a2f8205c8c2dd
Reviewed-on: https://code.wireshark.org/review/881
Reviewed-by: Anders Broman <a.broman58@gmail.com>
2014-03-31 18:48:06 +00:00
Evan Huus a8f1e349c3 Doc tweaks and macro parameterization
Change-Id: I9898bedf05a785683e79866a149336cbbf402d27
Reviewed-on: https://code.wireshark.org/review/892
Reviewed-by: Evan Huus <eapache@gmail.com>
2014-03-31 18:44:26 +00:00
AndersBroman 6a3394eefa Inlining wmem_strbuf_grow seems to give better performance.
Change-Id: I80f3d42fb01461af97984b463a6e8b8217eed552
Reviewed-on: https://code.wireshark.org/review/888
Reviewed-by: Evan Huus <eapache@gmail.com>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
2014-03-31 15:20:08 +00:00
Evan Huus 7df883954e Splay tree implementation for wmem
This is a tree implementation intended to replace the current red-black tree in
wmem_tree (which was inherited from emem), assuming there are no regressions.
Splay trees bubble recently accessed keys to the top, and as such have a number
of very nice properties: https://en.wikipedia.org/wiki/Splay_tree

This implementation is a variant known as "independent semi-splaying", which has
better practical performance. It should do about as well as the red-black tree
for random insertions and accesses, but somewhat better for patterned accesses
(such as accessing each key in order, or accessing certain keys very
frequently).

There are a few other changes relative to the red-black tree implementation that
are worth mentioning:
 - Instead of requiring complex keys to be split into guint32 chunks and doing
   this weird trick with sub-trees, I let the keys be arbitrary pointers and
   allowed the user to specify an arbitrary comparison function. If the function
   is NULL then the pointers are compared directly for the simple integer-key
   case.
 - Splay trees do not need to store a red-black colour flag for each node. It is
   also much easier to do without the parent pointer in each node. And due to
   the simpler system for complex keys, I was able to remove the "is_subtree"
   boolean. As such, splay nodes are 12 bytes smaller on 32-bit platforms, and
   16 bytes smaller on a 64-bit platform.

All done in about half the lines of code.

Change-Id: I89fb57e07d2bb7e3197190c7c2597b0c5adcc03b
Reviewed-on: https://code.wireshark.org/review/758
Reviewed-by: Evan Huus <eapache@gmail.com>
2014-03-29 18:01:55 +00:00
Evan Huus 48fc43baad Implement finalize() method for wmem strbuf.
Change-Id: Ib7a2b0d348b3624f41253e2d0995a4a38a9fe45d
Reviewed-on: https://code.wireshark.org/review/859
Reviewed-by: Evan Huus <eapache@gmail.com>
2014-03-29 12:11:21 +00:00
AndersBroman 86d65a0758 Introduce wmem_ascii_strdown()
Change-Id: Icdc5a0d5033f3ab709fbf19a33ab26f609d4b1f0
Reviewed-on: https://code.wireshark.org/review/824
Reviewed-by: Evan Huus <eapache@gmail.com>
2014-03-25 16:10:43 +00:00
Evan Huus 35098d20a2 Fix memory-alignment in the block allocator.
The previous macro gave the correct alignment, but there was one case where it
would add a whole block of unnecessary ALIGN_SIZE bytes. The new one is also
slightly faster to compute.

Benchmark win of about 3%.

Change-Id: I5d8bad0f78dc0e383e14c2c7a951328a06400020
Reviewed-on: https://code.wireshark.org/review/492
Reviewed-by: Evan Huus <eapache@gmail.com>
2014-03-04 17:09:40 +00:00
Alexis La Goutte 296591399f Remove all $Id$ from top of file
(Using sed : sed -i '/^ \* \$Id\$/,+1 d')

Fix manually some typo (in export_object_dicom.c and crc16-plain.c)

Change-Id: I4c1ae68d1c4afeace8cb195b53c715cf9e1227a8
Reviewed-on: https://code.wireshark.org/review/497
Reviewed-by: Anders Broman <a.broman58@gmail.com>
2014-03-04 14:27:33 +00:00
Evan Huus 01467c294a Remove assertions from block allocator.
It has been extremely well-tested at this point, and is a very hot code path so
the performance gain is measurable (~1-2% on most captures I tried).

Change-Id: I2f5e03d2f348f56e740bf0dfbc83a4fd9cc8c5a9
Reviewed-on: https://code.wireshark.org/review/499
Reviewed-by: Anders Broman <a.broman58@gmail.com>
2014-03-04 14:18:51 +00:00
Evan Huus 36e68f7b36 From Jim Young via https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9613
Don't use "rand" as a variable name, it shadows a global in recent MacOSX
stdlib.h

svn path=/trunk/; revision=54598
2014-01-05 00:47:24 +00:00
Evan Huus 1dfa4e2697 Add wmem_destroy_list, wmem_destroy_queue, wmem_destroy_stack so that those
data-structures can be used with manually managed memory.

svn path=/trunk/; revision=54432
2013-12-24 01:17:41 +00:00
Anders Broman 350a207a22 Try to please the buildbot.
svn path=/trunk/; revision=54281
2013-12-20 06:50:46 +00:00
Anders Broman 5fbfa6c351 Fix [-Wmissing-prototypes]
svn path=/trunk/; revision=54280
2013-12-20 06:30:13 +00:00
Jakub Zawadzki 2c015bb386 Fix some const/ not const warnings.
svn path=/trunk/; revision=54114
2013-12-15 12:35:00 +00:00
Evan Huus d04079eedc Expose wmem_list_remove_frame to make it efficient to do pop operations as
wmem_list_remove_frame(wmem_list_tail())

svn path=/trunk/; revision=53088
2013-11-05 17:28:19 +00:00
Evan Huus 1f3549168d Don't link explicitly with libwmem, it's already in libwireshark and the symbol
doubling leads to all sorts of very subtle badness (including test failures due
to funny internal assertions because the two wmems have mismatching state).

Make wmem_init and wmem_cleanup PUBLIC instead of LOCAL so that they don't get
stripped and don't cause a link failure when trying to build oids_test (now that
it's not linking with libwmem explicitly). There is possibly a better way to fix
this, but I'm not sure what it is.

svn path=/trunk/; revision=52694
2013-10-19 19:40:02 +00:00
Evan Huus 7b3dfaa525 As suggested separately by both Jakub and Anders: only read the
WIRESHARK_DEBUG_WMEM_OVERRIDE environment variable once in wmem_init, not every
time wmem_allocator_new is called. We currently create a new pinfo pool for
every packet we dissect, so this is a small performance win, especially when
getenv is slow (which may happen if a large number of environment variables are
set, such as when fuzz-testing).

svn path=/trunk/; revision=52634
2013-10-15 23:16:35 +00:00
Evan Huus 7d66e3806b Don't do canaries in blocks of guint32, it appears to cause alignment issues
when running tests on Solaris. Revert back to byte-at-a-time, but do fewer bytes
to avoid a performance hit.

svn path=/trunk/; revision=52617
2013-10-15 13:15:38 +00:00
Evan Huus e7a0c26bec Subsume README.binarytrees into wmem doxygen. The README was out of date
anyways, since Michael made key operations non-destructive in r44380.

svn path=/trunk/; revision=52583
2013-10-13 13:13:24 +00:00
Evan Huus dc0b7253e4 Don't assert that the key to a wmem tree operation is short. If XMPP wants to
use a 1500-byte string as a key, so be it. It will be slow, but at least it will
work.

Fixes https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9078

svn path=/trunk/; revision=52261
2013-09-28 23:45:18 +00:00
Evan Huus 5f7187c658 Cycle the recycler in realloc() and free() as well, reducing memory usage of the
stress-test by another ~10%.

svn path=/trunk/; revision=51840
2013-09-08 15:18:55 +00:00
Evan Huus 3c1e67c22d Send all alloc and free calls within wmem through wmem_alloc and wmem_free with
a NULL allocator. This gives us a single, central place to handle out-of-memory
errors (by, for example, throwing an exception) for basically all of epan.

The only remaining glib memory that is directly allocated is for the hash tables
used by the simple and strict allocators.

svn path=/trunk/; revision=51627
2013-09-01 13:37:38 +00:00
Evan Huus d939a6dec0 Do our list manipulation entirely after the realloc, so that if the realloc
doesn't return (do to, for example, throwing an exception) the allocator is
left in a consistent state.

svn path=/trunk/; revision=51626
2013-09-01 13:07:22 +00:00
Evan Huus f875bf8de2 In the strict allocator, do canaries in units of guint32 instead of guint8. This
makes canary checking about 20% faster, which should speed up fuzz-testing now
that more and more dissectors use wmem.

svn path=/trunk/; revision=51620
2013-08-31 16:36:37 +00:00
Guy Harris e8f42c6b48 Include config.h first; it defines _FILE_OFFSET_BITS, and if some system
header later causes it to be redefined - as happens on my Solaris 11
virtual machine - we get a redefinition warning, which gets treated as
an error.

svn path=/trunk/; revision=51344
2013-08-14 00:31:14 +00:00
Evan Huus d8f00e22e1 As suggested in https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9033
Make epan_free a no-op if the pointer is NULL. This fixes 99% of the cases
causing problems for wmem_leave_file_scope() - remove that XXX comment and add
back the assertion.

Remove the cleanup_dissection call from epan_cleanup, it doesn't make sense
there. init_dissection is only called from epan_new, so cleanup_dissection
should only be called from epan_free.

Add one missing epan_free call to tshark revealed by the above changes.

svn path=/trunk/; revision=51342
2013-08-13 22:41:34 +00:00
Evan Huus 616324c21e Be sure to initialize the jumbo flag when creating a new chunk header by
splitting an existing chunk. Thanks to Jeff for catching this one.

svn path=/trunk/; revision=51325
2013-08-12 23:35:35 +00:00
Evan Huus bf0f0ec4d9 Reorder and group chunk header flags. Reduces the bit-twiddling needed by the
compiler and provides a surprising (if fairly small) performance win.

svn path=/trunk/; revision=51305
2013-08-11 23:49:56 +00:00
Evan Huus dc46cb0f29 Split jumbo_free and jumbo_realloc into their own functions for readability and
consistency (they are called just once and will be inlined by any reasonable
compiler).

Also add some comments, fix some spacing etc. No functional changes.

svn path=/trunk/; revision=51304
2013-08-11 22:32:46 +00:00
Evan Huus 6eef00ac05 Move the null check outside the function call. Simpler, and more efficient for
compilers that decide not to inline.

svn path=/trunk/; revision=51303
2013-08-11 22:20:35 +00:00
Evan Huus 497e53ee06 Little optimization: move two statements inside an if block.
svn path=/trunk/; revision=51302
2013-08-11 19:33:30 +00:00
Evan Huus 36ff6f2c25 Add support for allocating more than will fit in a single block, through the use
of a jumbo flag in the chunk header.

svn path=/trunk/; revision=51299
2013-08-11 17:18:19 +00:00
Evan Huus 959ee8f5c0 Instead of using a glib slist to track the list of blocks, embed a short
doubly-linked list at the head of each block. This was intended as a step
towards supporting allocations bigger than the usual block size, but also shows
up as a 2% performance improvement in the speed test, so win-win.

svn path=/trunk/; revision=51298
2013-08-11 16:06:07 +00:00
Evan Huus 3bac7e845b Small optimization: the master-list and recycler cases are different enough that
doing all the safety checks was unnecessarily slow. Do only the appropriate
checks for each case.

svn path=/trunk/; revision=51296
2013-08-11 14:20:00 +00:00
Alexis La Goutte d26b7fc516 Fix (-W)documentation error found by Clang
../../epan/wmem/wmem_user_cb.h:52:11: error: parameter 'allocator' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param allocator The allocator that triggered this callback.
          ^~~~~~~~~
../../epan/wmem/wmem_user_cb.h:53:11: error: parameter 'event' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param event     The event type that triggered this callback.
          ^~~~~
../../epan/wmem/wmem_user_cb.h:54:11: error: parameter 'user_data' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param user_data Whatever user_data was originally passed to the call to
          ^~~~~~~~~
../../epan/wmem/wmem_user_cb.h:63:11: error: parameter 'recurring' not found in the function declaration
      [-Werror,-Wdocumentation]
 * @param recurring If this is FALSE then the callback is called exactly once.
          ^~~~~~~~~
4 errors generated.

svn path=/trunk/; revision=51259
2013-08-10 21:28:42 +00:00
Evan Huus 1c2cb57a38 Enhance verification to assert that the recycler and master list contain all
free chunks of sufficient size.

svn path=/trunk/; revision=51234
2013-08-09 11:20:02 +00:00
Evan Huus c5319ebc93 One more chunk header alignment fix.
svn path=/trunk/; revision=51218
2013-08-08 12:23:23 +00:00
Evan Huus 0ebe18dcea Make sure our chunk headers are large enough to correctly align the actual data
pointer we return.

svn path=/trunk/; revision=51217
2013-08-08 11:27:39 +00:00
Evan Huus d8684cf32d When splitting a free chunk, if we don't have room for a free header don't even
bother splitting. This greatly simplifies the logic, trims another 4% off the
fast path, and doesn't actually affect the results at all because of the way we
pad for alignment anyways.

svn path=/trunk/; revision=51216
2013-08-08 11:16:10 +00:00
Evan Huus c4e966b880 Combine two ternary ?: into a single if. Shaves about 1.5% off the fast timing
test.

svn path=/trunk/; revision=51112
2013-08-02 01:58:52 +00:00
Evan Huus a7b30a2a2c Small simplification
svn path=/trunk/; revision=51111
2013-08-02 01:51:06 +00:00
Evan Huus c85ec68e58 Remove unnecessary null check.
svn path=/trunk/; revision=51110
2013-08-02 01:36:52 +00:00
Evan Huus 64ed83d793 Rewrite the merge_free routine to keep the merged chunk on the master free list
if the right-hand merge target was there originally. This brings memory usage
down another ~40% when running the heavy test suite.

This also lets us extract the master-list check out of unfree() since it is now
only relevant at a single caller, and turns unfree into the more understandable
remove_from_recycler().

svn path=/trunk/; revision=51104
2013-08-02 00:12:12 +00:00
Evan Huus 8fafa9b758 Add wmem_tree_is_empty(). Also permit lookups on a NULL tree for compatibility
with emem.

svn path=/trunk/; revision=51045
2013-07-31 01:48:47 +00:00
Evan Huus 44d53b4042 More little optimizations.
svn path=/trunk/; revision=51014
2013-07-29 16:55:02 +00:00
Evan Huus 0fb6d96507 Another little optimization.
Removes one branch from the hot path, deduplicates one function call in the cold
path by effectively falling through, and makes it more obvious what the code is
actually trying to do.

svn path=/trunk/; revision=51013
2013-07-29 16:47:30 +00:00
Graham Bloice 2f6ee944d8 Added more build artefacts to the clean target
svn path=/trunk/; revision=50977
2013-07-28 14:40:56 +00:00
Evan Huus a9bccc8dc8 Don't force the allocator choice in the data-structure tests, we want to be able
to override that to simple for valgrinding (we still force the allocator in the
allocator and timing tests, of course).

svn path=/trunk/; revision=50971
2013-07-28 12:55:37 +00:00
Evan Huus b43996c8ab Fix a bug in the gc routine discovered while formally working out parts of the
algorithm.

Also add a test case to excercise it for future.

svn path=/trunk/; revision=50970
2013-07-28 12:46:44 +00:00