Some of the allocators are close enough that a little bit of system load at the
time of the test was enough to throw them off, and it was causing unnecessary
buildbot failures. Benchmarking on real-world cases at the time of
implementation (as well as reliable "average" results on the buildbots) has left
me confident that block_fast is, in fact, faster than block, which is faster
than simple/strict.
Additionally, the allocators are effectively "done" and unlikely to change, so
the possibility of uncaught performance regressions is low.
Change-Id: I144ea4cd9d6000edc4075ed682ce6a4aeaf5e28a
Reviewed-on: https://code.wireshark.org/review/3356
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Apparently the new win8 vm has a *very* low-resolution timer, and the allocators
are indistinguishable at the previous amount of work.
Change-Id: If4e5bb8f85b1f0d39658f54001c88f42ffddfc47
Reviewed-on: https://code.wireshark.org/review/2768
Reviewed-by: Evan Huus <eapache@gmail.com>
As it turns out, we do occasionally need it.
Bug:10115
Change-Id: Ifec79e4d2470bbc09f15674534d01418a6571a0d
Reviewed-on: https://code.wireshark.org/review/1688
Reviewed-by: Evan Huus <eapache@gmail.com>
As Jakub pointed out, our +1/-1 logic for null terminators wasn't quite right.
Also be sure to re-copy the va_list parameter if we need to re-use it, as
otherwise things break oddly.
Change-Id: Ibeaa95af602f565791e9378a6cfce434f13025eb
Reviewed-on: https://code.wireshark.org/review/1670
Reviewed-by: Evan Huus <eapache@gmail.com>
It appears that on some platforms the timer underlying g_test_timer() is
relatively low-precision, so the small amount of work that we were previously
doing was not enough for the timer to provide a useful value.
Bumping the amount of work should provide more meaningful values, and the whole
timing step still takes only a few seconds.
Change-Id: Idce1386eaa33add845d9a2758b0beb72bbf370d6
Reviewed-on: https://code.wireshark.org/review/1523
Reviewed-by: Evan Huus <eapache@gmail.com>
Follow-up to g19e650f43adf23b0aa92821bff0cc389643944f6 since the fuzz-bot is
still failing.
Change-Id: I2d05a861b0edf50ce734b682e6e3e33729bf6771
Reviewed-on: https://code.wireshark.org/review/1503
Reviewed-by: Evan Huus <eapache@gmail.com>
I believe the XP buildbot failures are simply running out of memory, so
hopefully reducing the number of iterations should be enough to let it finish
reliably.
Change-Id: I40af789d528c2623e95f153529205b5e617c6e02
Reviewed-on: https://code.wireshark.org/review/1496
Reviewed-by: Evan Huus <eapache@gmail.com>
For packet-scope allocation, there's no need to support realloc(), free()
cause memory will be garbage collected after packet dissection.
(and this allocator is much faster than full block allocator).
Change-Id: I73fdf708c3077f48f55bdcc71f4fa859e4ac2335
Reviewed-on: https://code.wireshark.org/review/1428
Reviewed-by: Anders Broman <a.broman58@gmail.com>
We now have to call wmem_init in order to randomly seed the values for wmem_map.
This means we can no longer rely on the lack of override, so we have to force
the right allocator type when testing/timing the allocators themselves.
Change-Id: I005034465b0a98f19876899b96ef65b3e4b1d759
Reviewed-on: https://code.wireshark.org/review/1468
Reviewed-by: Anders Broman <a.broman58@gmail.com>
This has two expected uses:
- Many current users of wmem_tree don't actually need the predecessor lookup
it provides (the lookup_le function family). A hash map provides straight
insertion and lookup much more efficiently than a wmem_tree when predecessor
lookup isn't needed.
- Many current users of glib's hash table and hash functions use untrusted data
for keys, making them vulnerable to algorithmic complexity attacks. Care has
been taken to make this implementation secure against such attacks, so it
should be used whenever data is untrusted.
In my benchmarks it is measurably slower than GHashTable, but not excessively
so. Given the additional security it provides this seems like a reasonable
trade-off (and it is still faster than a wmem_tree).
Change-Id: I2d67a0d06029f14c153eaa42d5cfc774aefd9918
Reviewed-on: https://code.wireshark.org/review/1272
Reviewed-by: Evan Huus <eapache@gmail.com>
C90 permits plain %f for doubles. Thanks to Alexis for pointing this out.
Change-Id: I3f927b0a50e5764c2297c7f9d680c6363ba9fd0e
Reviewed-on: https://code.wireshark.org/review/1073
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Reviewed-by: Michael Mann <mmann78@netscape.net>
There is confusion about API usage, and problems on my part concerning whether
keys should be compared signed or unsigned, and how to do that efficiently.
Unsigned keys in particular were behaving oddly.
Change-Id: I075693bbd04c15f79f24f9a24006003a914cc572
Reviewed-on: https://code.wireshark.org/review/924
Reviewed-by: Evan Huus <eapache@gmail.com>
This is a tree implementation intended to replace the current red-black tree in
wmem_tree (which was inherited from emem), assuming there are no regressions.
Splay trees bubble recently accessed keys to the top, and as such have a number
of very nice properties: https://en.wikipedia.org/wiki/Splay_tree
This implementation is a variant known as "independent semi-splaying", which has
better practical performance. It should do about as well as the red-black tree
for random insertions and accesses, but somewhat better for patterned accesses
(such as accessing each key in order, or accessing certain keys very
frequently).
There are a few other changes relative to the red-black tree implementation that
are worth mentioning:
- Instead of requiring complex keys to be split into guint32 chunks and doing
this weird trick with sub-trees, I let the keys be arbitrary pointers and
allowed the user to specify an arbitrary comparison function. If the function
is NULL then the pointers are compared directly for the simple integer-key
case.
- Splay trees do not need to store a red-black colour flag for each node. It is
also much easier to do without the parent pointer in each node. And due to
the simpler system for complex keys, I was able to remove the "is_subtree"
boolean. As such, splay nodes are 12 bytes smaller on 32-bit platforms, and
16 bytes smaller on a 64-bit platform.
All done in about half the lines of code.
Change-Id: I89fb57e07d2bb7e3197190c7c2597b0c5adcc03b
Reviewed-on: https://code.wireshark.org/review/758
Reviewed-by: Evan Huus <eapache@gmail.com>
(Using sed : sed -i '/^ \* \$Id\$/,+1 d')
Fix manually some typo (in export_object_dicom.c and crc16-plain.c)
Change-Id: I4c1ae68d1c4afeace8cb195b53c715cf9e1227a8
Reviewed-on: https://code.wireshark.org/review/497
Reviewed-by: Anders Broman <a.broman58@gmail.com>
WIRESHARK_DEBUG_WMEM_OVERRIDE environment variable once in wmem_init, not every
time wmem_allocator_new is called. We currently create a new pinfo pool for
every packet we dissect, so this is a small performance win, especially when
getenv is slow (which may happen if a large number of environment variables are
set, such as when fuzz-testing).
svn path=/trunk/; revision=52634
a NULL allocator. This gives us a single, central place to handle out-of-memory
errors (by, for example, throwing an exception) for basically all of epan.
The only remaining glib memory that is directly allocated is for the hash tables
used by the simple and strict allocators.
svn path=/trunk/; revision=51627
header later causes it to be redefined - as happens on my Solaris 11
virtual machine - we get a redefinition warning, which gets treated as
an error.
svn path=/trunk/; revision=51344
to override that to simple for valgrinding (we still force the allocator in the
allocator and timing tests, of course).
svn path=/trunk/; revision=50971
The overhead is not large, and it makes append much faster (O(1) vs O(n)).
It also will make a queue easy to add, which I need for a dissector I'm
writing...
svn path=/trunk/; revision=50744
From the comment above wmem_tree_insert32_array():
* If you use ...32_array() calls you MUST make sure that every single node
* you add to a specific tree always has a key of exactly the same number of
* keylen words or things will most likely crash. Or at least that every single
* item that sits behind the same top level node always have exactly the same
* number of words.
So clearly generating thousands of keys with random lengths while testing is
going to cause problems. Generate a set of random lengths, then use those
lengths consistently (but still generating random keys of those lengths).
Should hopefully fix the intermittent build-bot failures.
(unfortunately this does not manifest nicely, and I cannot see an easy way to
assert it so that we catch other people trying to use different-length key
subtrees)
svn path=/trunk/; revision=50184
Put in a whole bunch of stderr output in the wmem tree tests in the hopes that
the next time one of the buildbots randomly (and irreproducibly) fails on this
step we'll have at least a bit of a hint as to where it happened.
svn path=/trunk/; revision=50131
it is out of scope, they just can't *allocate* in the pool. This is necessary
because file-scope trees (migrating from emem) are set up on program
initialization when there is no file in scope - they need to initialize with the
handle, they just won't use it until a file is actually in scope.
svn path=/trunk/; revision=50046