wireshark/epan/wmem/wmem_stack.h:39: Warning: group wmem-stack: ignoring title "Stack-bbab" that does not match old title "Queue"
Change-Id: I9376651d89d9df21aeca82ddb952c230d546169a
Reviewed-on: https://code.wireshark.org/review/1130
Reviewed-by: Anders Broman <a.broman58@gmail.com>
C90 permits plain %f for doubles. Thanks to Alexis for pointing this out.
Change-Id: I3f927b0a50e5764c2297c7f9d680c6363ba9fd0e
Reviewed-on: https://code.wireshark.org/review/1073
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Reviewed-by: Michael Mann <mmann78@netscape.net>
There is confusion about API usage, and problems on my part concerning whether
keys should be compared signed or unsigned, and how to do that efficiently.
Unsigned keys in particular were behaving oddly.
Change-Id: I075693bbd04c15f79f24f9a24006003a914cc572
Reviewed-on: https://code.wireshark.org/review/924
Reviewed-by: Evan Huus <eapache@gmail.com>
(Using sed : sed -i '/^\# \$Id\$/,+1 d') (start with dash)
Change-Id: Ia4b5a6c2302f6a531f6a86c1ec3a2f8205c8c2dd
Reviewed-on: https://code.wireshark.org/review/881
Reviewed-by: Anders Broman <a.broman58@gmail.com>
This is a tree implementation intended to replace the current red-black tree in
wmem_tree (which was inherited from emem), assuming there are no regressions.
Splay trees bubble recently accessed keys to the top, and as such have a number
of very nice properties: https://en.wikipedia.org/wiki/Splay_tree
This implementation is a variant known as "independent semi-splaying", which has
better practical performance. It should do about as well as the red-black tree
for random insertions and accesses, but somewhat better for patterned accesses
(such as accessing each key in order, or accessing certain keys very
frequently).
There are a few other changes relative to the red-black tree implementation that
are worth mentioning:
- Instead of requiring complex keys to be split into guint32 chunks and doing
this weird trick with sub-trees, I let the keys be arbitrary pointers and
allowed the user to specify an arbitrary comparison function. If the function
is NULL then the pointers are compared directly for the simple integer-key
case.
- Splay trees do not need to store a red-black colour flag for each node. It is
also much easier to do without the parent pointer in each node. And due to
the simpler system for complex keys, I was able to remove the "is_subtree"
boolean. As such, splay nodes are 12 bytes smaller on 32-bit platforms, and
16 bytes smaller on a 64-bit platform.
All done in about half the lines of code.
Change-Id: I89fb57e07d2bb7e3197190c7c2597b0c5adcc03b
Reviewed-on: https://code.wireshark.org/review/758
Reviewed-by: Evan Huus <eapache@gmail.com>
The previous macro gave the correct alignment, but there was one case where it
would add a whole block of unnecessary ALIGN_SIZE bytes. The new one is also
slightly faster to compute.
Benchmark win of about 3%.
Change-Id: I5d8bad0f78dc0e383e14c2c7a951328a06400020
Reviewed-on: https://code.wireshark.org/review/492
Reviewed-by: Evan Huus <eapache@gmail.com>
(Using sed : sed -i '/^ \* \$Id\$/,+1 d')
Fix manually some typo (in export_object_dicom.c and crc16-plain.c)
Change-Id: I4c1ae68d1c4afeace8cb195b53c715cf9e1227a8
Reviewed-on: https://code.wireshark.org/review/497
Reviewed-by: Anders Broman <a.broman58@gmail.com>
It has been extremely well-tested at this point, and is a very hot code path so
the performance gain is measurable (~1-2% on most captures I tried).
Change-Id: I2f5e03d2f348f56e740bf0dfbc83a4fd9cc8c5a9
Reviewed-on: https://code.wireshark.org/review/499
Reviewed-by: Anders Broman <a.broman58@gmail.com>
doubling leads to all sorts of very subtle badness (including test failures due
to funny internal assertions because the two wmems have mismatching state).
Make wmem_init and wmem_cleanup PUBLIC instead of LOCAL so that they don't get
stripped and don't cause a link failure when trying to build oids_test (now that
it's not linking with libwmem explicitly). There is possibly a better way to fix
this, but I'm not sure what it is.
svn path=/trunk/; revision=52694
WIRESHARK_DEBUG_WMEM_OVERRIDE environment variable once in wmem_init, not every
time wmem_allocator_new is called. We currently create a new pinfo pool for
every packet we dissect, so this is a small performance win, especially when
getenv is slow (which may happen if a large number of environment variables are
set, such as when fuzz-testing).
svn path=/trunk/; revision=52634
a NULL allocator. This gives us a single, central place to handle out-of-memory
errors (by, for example, throwing an exception) for basically all of epan.
The only remaining glib memory that is directly allocated is for the hash tables
used by the simple and strict allocators.
svn path=/trunk/; revision=51627
makes canary checking about 20% faster, which should speed up fuzz-testing now
that more and more dissectors use wmem.
svn path=/trunk/; revision=51620
header later causes it to be redefined - as happens on my Solaris 11
virtual machine - we get a redefinition warning, which gets treated as
an error.
svn path=/trunk/; revision=51344
Make epan_free a no-op if the pointer is NULL. This fixes 99% of the cases
causing problems for wmem_leave_file_scope() - remove that XXX comment and add
back the assertion.
Remove the cleanup_dissection call from epan_cleanup, it doesn't make sense
there. init_dissection is only called from epan_new, so cleanup_dissection
should only be called from epan_free.
Add one missing epan_free call to tshark revealed by the above changes.
svn path=/trunk/; revision=51342
consistency (they are called just once and will be inlined by any reasonable
compiler).
Also add some comments, fix some spacing etc. No functional changes.
svn path=/trunk/; revision=51304
doubly-linked list at the head of each block. This was intended as a step
towards supporting allocations bigger than the usual block size, but also shows
up as a 2% performance improvement in the speed test, so win-win.
svn path=/trunk/; revision=51298
../../epan/wmem/wmem_user_cb.h:52:11: error: parameter 'allocator' not found in the function declaration
[-Werror,-Wdocumentation]
* @param allocator The allocator that triggered this callback.
^~~~~~~~~
../../epan/wmem/wmem_user_cb.h:53:11: error: parameter 'event' not found in the function declaration
[-Werror,-Wdocumentation]
* @param event The event type that triggered this callback.
^~~~~
../../epan/wmem/wmem_user_cb.h:54:11: error: parameter 'user_data' not found in the function declaration
[-Werror,-Wdocumentation]
* @param user_data Whatever user_data was originally passed to the call to
^~~~~~~~~
../../epan/wmem/wmem_user_cb.h:63:11: error: parameter 'recurring' not found in the function declaration
[-Werror,-Wdocumentation]
* @param recurring If this is FALSE then the callback is called exactly once.
^~~~~~~~~
4 errors generated.
svn path=/trunk/; revision=51259
bother splitting. This greatly simplifies the logic, trims another 4% off the
fast path, and doesn't actually affect the results at all because of the way we
pad for alignment anyways.
svn path=/trunk/; revision=51216
if the right-hand merge target was there originally. This brings memory usage
down another ~40% when running the heavy test suite.
This also lets us extract the master-list check out of unfree() since it is now
only relevant at a single caller, and turns unfree into the more understandable
remove_from_recycler().
svn path=/trunk/; revision=51104
Removes one branch from the hot path, deduplicates one function call in the cold
path by effectively falling through, and makes it more obvious what the code is
actually trying to do.
svn path=/trunk/; revision=51013
to override that to simple for valgrinding (we still force the allocator in the
allocator and timing tests, of course).
svn path=/trunk/; revision=50971
Move a few assignments around to avoid one extra subtraction. I suspect having
the two if statements next to each other is friendly to the compiler's optimizer
as well.
Shaves ~1.3% off my timing tests, bringing the new design *very* close to the
old one in raw allocation speed.
svn path=/trunk/; revision=50961
unobtrusive if statement got dropped. Without it the allocator exhibits the old
bad behaviour of 3x memory usage and heavy fragmentation.
We want it back, thank you very much.
svn path=/trunk/; revision=50960
What was becoming apparent as more dissectors started using wmem was that the
old block allocator design had issues with memory fragmentation. This keeps the
same underlying memory layout, but completely changes how free blocks are kept.
It runs about 3% slower in my tests (still an order of magnitude faster than
g_malloc) but uses about 1/3 the memory.
I suspect some simple optimizations could reclaim that 3% as well - the design
is fast, but I did not code particularly for speed.
Thoroughly tested with the existing test suite (which caught half a dozen bugs
in my first draft) so it should actually work!
svn path=/trunk/; revision=50955
The overhead is not large, and it makes append much faster (O(1) vs O(n)).
It also will make a queue easy to add, which I need for a dissector I'm
writing...
svn path=/trunk/; revision=50744
From the comment above wmem_tree_insert32_array():
* If you use ...32_array() calls you MUST make sure that every single node
* you add to a specific tree always has a key of exactly the same number of
* keylen words or things will most likely crash. Or at least that every single
* item that sits behind the same top level node always have exactly the same
* number of words.
So clearly generating thousands of keys with random lengths while testing is
going to cause problems. Generate a set of random lengths, then use those
lengths consistently (but still generating random keys of those lengths).
Should hopefully fix the intermittent build-bot failures.
(unfortunately this does not manifest nicely, and I cannot see an easy way to
assert it so that we catch other people trying to use different-length key
subtrees)
svn path=/trunk/; revision=50184
Put in a whole bunch of stderr output in the wmem tree tests in the hopes that
the next time one of the buildbots randomly (and irreproducibly) fails on this
step we'll have at least a bit of a hint as to where it happened.
svn path=/trunk/; revision=50131
https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=8824
Convert bluetooth emem trees to wmem trees.
Add modelines and fix indentation.
Correct typo in wmem_tree.h that still referred to emem.
svn path=/trunk/; revision=50076
it is out of scope, they just can't *allocate* in the pool. This is necessary
because file-scope trees (migrating from emem) are set up on program
initialization when there is no file in scope - they need to initialize with the
handle, they just won't use it until a file is actually in scope.
svn path=/trunk/; revision=50046
enum for the color (red/black). Don't use bitfields since they don't save us
much (if anything) in terms of space and don't nest the fields in their own
anonymous struct.
svn path=/trunk/; revision=50011
assertions with regular glib assertions - there's no guarantee that wmem code
will always be run from within a dissector.
svn path=/trunk/; revision=49993
actual wmem_allocator_t structure. This simplifies the internal API and
deduplicates a few alloc/free calls in the individual allocator implementations.
I'd originally made the allocators responsible for this on purpose with the
idea that they'd be able to optimize something clever based on the type of
allocator, but that's clearly more work and complexity than it's worth given
the small number of allocators we create/destroy.
svn path=/trunk/; revision=49512
recurring callbacks, I suspect most other potential uses will be once-only, so
make that possible, and improve the documentation on the remaining issues.
Also separate out the code into its own files and the testing into its own
test case.
svn path=/trunk/; revision=49209
the behaviour emem has for seasonal trees, which is that the master tree
structure is not actually seasonal - it is permanent. When the seasonal memory
pool is cleared, the root node pointer in all of these permanent trees is set
to NULL, and the pool takes care of actually freeing the nodes.
Wmem can now mimic this by allocating the tree header struct in epan_scope(),
allocating any node structs in file_scope(), and registering a callback on
file_scope() that NULLs the pointer in the epan_scope() header. Yes, this is
confusing, but it seemed simpler than adding manual callback registrations to
every single dissector that currently uses seasonal trees.
The callbacks may also be useful for other things that need cleanup (I'm
thinking resource handles stored in wmem memory that need to be fclosed or
what-have-you before they the handle is lost).
As indicated by the number of caveats in README.wmem, the implementation
probably needs a bit of work to make it safer/saner/more-useful. Thoughts
(or patches!) in this direction are more than welcome.
svn path=/trunk/; revision=49205
- better tests
- fix a bug caught by the better tests
- implement append_c and append_unichar, with tests
Wmem string-buffers now have feature parity with their emem equivalents, so
remove them from the TODO list.
svn path=/trunk/; revision=49060
The wmem test suite now covers all of the existing allocators, data
structures and utility functions in at least basic cases.
svn path=/trunk/; revision=48980
simple one. At the moment it seems to be between 2x and 2.5x faster in the
common case (a simple sequence of allocations followed by free_all).
svn path=/trunk/; revision=48588
reason).
Don't use g_assert_cmpuint, since it apparently causes warnings on windows that
I don't know how to get rid of safely without breaking the conditions being
checked.
svn path=/trunk/; revision=48575
assertions in the block allocator, and fix one rare potential underflow caught
by the improved tests.
The tests now take ~200MB and 5-10 seconds to run. Hopefully this is small
enough for the build-bots to handle, if not then we can reduce the max
allocation size or max iterations to suit.
svn path=/trunk/; revision=48574
failing. I suspect it has to do with my lack of understanding of glib's unit
test framework, not the code being tested.
svn path=/trunk/; revision=48519
the unit test part of the test suite. Once I know it's building and
running properly on the buildbots then I'll actually start writing tests.
svn path=/trunk/; revision=48517
from one case I consistently forgot when typing it up originally, even though
it's clearly listed several places in my design notes.
Also include an #if0-ed out block of code to redirect emem to wmem for easy
testing (since there are very few common dissectors that use wmem right now).
svn path=/trunk/; revision=48434
Re-enable the block allocator by default in trunk since it is much better
tested now - I've spent some time with a hack redirecting all emem allocations
to wmem, so it's seen a lot of traffic. I will still likely turn it off for
1.10 whenever that branches, just to be safe.
svn path=/trunk/; revision=48416
(removed in r48218) which did nothing particularly useful. Also lets us remove
another debugging environment variable.
svn path=/trunk/; revision=48219
multiple adjacent free chunks. When splitting a used chunk, the resulting
extra unused chunk may need to be merged to its right.
svn path=/trunk/; revision=47552
Some interesting algorithmic stuff going on in here for those who are
interested.
This completes the allocator rewrites for the API additions, so those can be
exposed now.
svn path=/trunk/; revision=47547
a GSList. This permits it to implement the new realloc and free functions. Also
fill in an empty gc function, since there isn't much it can do as far as
garbage-collection goes.
svn path=/trunk/; revision=47169
Cast away some implicit 64-bit-to-32-bit conversion errors due to use of
sizeof.
Cast away some implicit 64-bit-to-32-bit conversion errors due to use of
strtol() and strtoul().
Change some data types to avoid those implicit conversion warnings.
When assigning a constant to a float, make sure the constant isn't a
double, by appending "f" to the constant.
Constify a bunch of variables, parameters, and return values to
eliminate warnings due to strings being given const qualifiers. Cast
away those warnings in some cases where an API we don't control forces
us to do so.
Enable a bunch of additional warnings by default. Note why at least
some of the other warnings aren't enabled.
randpkt.c and text2pcap.c are used to build programs, so they don't need
to be in EXTRA_DIST.
If the user specifies --enable-warnings-as-errors, add -Werror *even if
the user specified --enable-extra-gcc-flags; assume they know what
they're doing and are willing to have the compile fail due to the extra
GCC warnings being treated as errors.
svn path=/trunk/; revision=46748
determine the desired type. This has two advantages over the old way:
- just one environment variable for valgrind to override in order to guarantee
that ALL allocators use memory it can track, and just one place to check that
variable
- allocator owners no longer have to include headers specific to their
allocator, allowing them to change allocators without adjusting all their
#includes
svn path=/trunk/; revision=46604
yet initialized because I can't figure out where the enter() and leave() calls
should go - the obvious place in packet.c causes a lot of assertion errors.
svn path=/trunk/; revision=45879
Call them from epan_init() and epan_cleanup().
Expose a permanent wmem scope for allocations that should only be freed when
epan is done (which is *not* necessarily when the program finishes).
svn path=/trunk/; revision=45805
potential bugs:
- calling the wrong destroy function on an allocator
- a pool allocator forgetting to call free_all on itself in the destructor
Also, fix potential typedef redefinition warning in wmem_allocator_glib.h
svn path=/trunk/; revision=45804
Alexis La Goutte