Commit Graph

60 Commits

Author SHA1 Message Date
Evan Huus eaf1aad31e wmem: don't remove tiny blocks from the recycler
They were never put in it in the first place because they can't hold the
necessary pointers.

Bug: 11373
Change-Id: I9e2ec76850929b5ac86e6f7a344d70f56ad3911c
Reviewed-on: https://code.wireshark.org/review/9672
Reviewed-by: Evan Huus <eapache@gmail.com>
2015-07-17 02:37:13 +00:00
Evan Huus 35098d20a2 Fix memory-alignment in the block allocator.
The previous macro gave the correct alignment, but there was one case where it
would add a whole block of unnecessary ALIGN_SIZE bytes. The new one is also
slightly faster to compute.

Benchmark win of about 3%.

Change-Id: I5d8bad0f78dc0e383e14c2c7a951328a06400020
Reviewed-on: https://code.wireshark.org/review/492
Reviewed-by: Evan Huus <eapache@gmail.com>
2014-03-04 17:09:40 +00:00
Alexis La Goutte 296591399f Remove all $Id$ from top of file
(Using sed : sed -i '/^ \* \$Id\$/,+1 d')

Fix manually some typo (in export_object_dicom.c and crc16-plain.c)

Change-Id: I4c1ae68d1c4afeace8cb195b53c715cf9e1227a8
Reviewed-on: https://code.wireshark.org/review/497
Reviewed-by: Anders Broman <a.broman58@gmail.com>
2014-03-04 14:27:33 +00:00
Evan Huus 01467c294a Remove assertions from block allocator.
It has been extremely well-tested at this point, and is a very hot code path so
the performance gain is measurable (~1-2% on most captures I tried).

Change-Id: I2f5e03d2f348f56e740bf0dfbc83a4fd9cc8c5a9
Reviewed-on: https://code.wireshark.org/review/499
Reviewed-by: Anders Broman <a.broman58@gmail.com>
2014-03-04 14:18:51 +00:00
Anders Broman 5fbfa6c351 Fix [-Wmissing-prototypes]
svn path=/trunk/; revision=54280
2013-12-20 06:30:13 +00:00
Evan Huus 5f7187c658 Cycle the recycler in realloc() and free() as well, reducing memory usage of the
stress-test by another ~10%.

svn path=/trunk/; revision=51840
2013-09-08 15:18:55 +00:00
Evan Huus 3c1e67c22d Send all alloc and free calls within wmem through wmem_alloc and wmem_free with
a NULL allocator. This gives us a single, central place to handle out-of-memory
errors (by, for example, throwing an exception) for basically all of epan.

The only remaining glib memory that is directly allocated is for the hash tables
used by the simple and strict allocators.

svn path=/trunk/; revision=51627
2013-09-01 13:37:38 +00:00
Evan Huus d939a6dec0 Do our list manipulation entirely after the realloc, so that if the realloc
doesn't return (do to, for example, throwing an exception) the allocator is
left in a consistent state.

svn path=/trunk/; revision=51626
2013-09-01 13:07:22 +00:00
Evan Huus 616324c21e Be sure to initialize the jumbo flag when creating a new chunk header by
splitting an existing chunk. Thanks to Jeff for catching this one.

svn path=/trunk/; revision=51325
2013-08-12 23:35:35 +00:00
Evan Huus bf0f0ec4d9 Reorder and group chunk header flags. Reduces the bit-twiddling needed by the
compiler and provides a surprising (if fairly small) performance win.

svn path=/trunk/; revision=51305
2013-08-11 23:49:56 +00:00
Evan Huus dc46cb0f29 Split jumbo_free and jumbo_realloc into their own functions for readability and
consistency (they are called just once and will be inlined by any reasonable
compiler).

Also add some comments, fix some spacing etc. No functional changes.

svn path=/trunk/; revision=51304
2013-08-11 22:32:46 +00:00
Evan Huus 6eef00ac05 Move the null check outside the function call. Simpler, and more efficient for
compilers that decide not to inline.

svn path=/trunk/; revision=51303
2013-08-11 22:20:35 +00:00
Evan Huus 497e53ee06 Little optimization: move two statements inside an if block.
svn path=/trunk/; revision=51302
2013-08-11 19:33:30 +00:00
Evan Huus 36ff6f2c25 Add support for allocating more than will fit in a single block, through the use
of a jumbo flag in the chunk header.

svn path=/trunk/; revision=51299
2013-08-11 17:18:19 +00:00
Evan Huus 959ee8f5c0 Instead of using a glib slist to track the list of blocks, embed a short
doubly-linked list at the head of each block. This was intended as a step
towards supporting allocations bigger than the usual block size, but also shows
up as a 2% performance improvement in the speed test, so win-win.

svn path=/trunk/; revision=51298
2013-08-11 16:06:07 +00:00
Evan Huus 3bac7e845b Small optimization: the master-list and recycler cases are different enough that
doing all the safety checks was unnecessarily slow. Do only the appropriate
checks for each case.

svn path=/trunk/; revision=51296
2013-08-11 14:20:00 +00:00
Evan Huus 1c2cb57a38 Enhance verification to assert that the recycler and master list contain all
free chunks of sufficient size.

svn path=/trunk/; revision=51234
2013-08-09 11:20:02 +00:00
Evan Huus c5319ebc93 One more chunk header alignment fix.
svn path=/trunk/; revision=51218
2013-08-08 12:23:23 +00:00
Evan Huus 0ebe18dcea Make sure our chunk headers are large enough to correctly align the actual data
pointer we return.

svn path=/trunk/; revision=51217
2013-08-08 11:27:39 +00:00
Evan Huus d8684cf32d When splitting a free chunk, if we don't have room for a free header don't even
bother splitting. This greatly simplifies the logic, trims another 4% off the
fast path, and doesn't actually affect the results at all because of the way we
pad for alignment anyways.

svn path=/trunk/; revision=51216
2013-08-08 11:16:10 +00:00
Evan Huus c4e966b880 Combine two ternary ?: into a single if. Shaves about 1.5% off the fast timing
test.

svn path=/trunk/; revision=51112
2013-08-02 01:58:52 +00:00
Evan Huus a7b30a2a2c Small simplification
svn path=/trunk/; revision=51111
2013-08-02 01:51:06 +00:00
Evan Huus c85ec68e58 Remove unnecessary null check.
svn path=/trunk/; revision=51110
2013-08-02 01:36:52 +00:00
Evan Huus 64ed83d793 Rewrite the merge_free routine to keep the merged chunk on the master free list
if the right-hand merge target was there originally. This brings memory usage
down another ~40% when running the heavy test suite.

This also lets us extract the master-list check out of unfree() since it is now
only relevant at a single caller, and turns unfree into the more understandable
remove_from_recycler().

svn path=/trunk/; revision=51104
2013-08-02 00:12:12 +00:00
Evan Huus 44d53b4042 More little optimizations.
svn path=/trunk/; revision=51014
2013-07-29 16:55:02 +00:00
Evan Huus 0fb6d96507 Another little optimization.
Removes one branch from the hot path, deduplicates one function call in the cold
path by effectively falling through, and makes it more obvious what the code is
actually trying to do.

svn path=/trunk/; revision=51013
2013-07-29 16:47:30 +00:00
Evan Huus b43996c8ab Fix a bug in the gc routine discovered while formally working out parts of the
algorithm.

Also add a test case to excercise it for future.

svn path=/trunk/; revision=50970
2013-07-28 12:46:44 +00:00
Evan Huus b9870c0969 Little optimizations.
Move a few assignments around to avoid one extra subtraction. I suspect having
the two if statements next to each other is friendly to the compiler's optimizer
as well.

Shaves ~1.3% off my timing tests, bringing the new design *very* close to the
old one in raw allocation speed.

svn path=/trunk/; revision=50961
2013-07-28 00:00:37 +00:00
Evan Huus e7dfea82f3 Little things make big differences: somehow while I was experimenting this
unobtrusive if statement got dropped. Without it the allocator exhibits the old
bad behaviour of 3x memory usage and heavy fragmentation.

We want it back, thank you very much.

svn path=/trunk/; revision=50960
2013-07-27 23:35:59 +00:00
Evan Huus 9b9467cb79 Small optimization: conjoin two mutually exclusive ifs with an else.
svn path=/trunk/; revision=50959
2013-07-27 23:01:24 +00:00
Evan Huus 720cce4102 Simplify out an addition that happened a lot by performing it once on the
variable it affected, since that variable wasn't used for any other purpose.

svn path=/trunk/; revision=50958
2013-07-27 22:51:27 +00:00
Evan Huus 3663498a8b Redesign of the wmem block allocator.
What was becoming apparent as more dissectors started using wmem was that the
old block allocator design had issues with memory fragmentation. This keeps the
same underlying memory layout, but completely changes how free blocks are kept.
It runs about 3% slower in my tests (still an order of magnitude faster than
g_malloc) but uses about 1/3 the memory.

I suspect some simple optimizations could reclaim that 3% as well - the design
is fast, but I did not code particularly for speed.

Thoroughly tested with the existing test suite (which caught half a dozen bugs
in my first draft) so it should actually work!

svn path=/trunk/; revision=50955
2013-07-27 22:27:28 +00:00
Evan Huus c693d9c951 whitespace
svn path=/trunk/; revision=50919
2013-07-26 16:42:06 +00:00
Evan Huus 5426ba4e02 Minor refactor: make the framework responsible for allocating and freeing the
actual wmem_allocator_t structure. This simplifies the internal API and
deduplicates a few alloc/free calls in the individual allocator implementations.

I'd originally made the allocators responsible for this on purpose with the
idea that they'd be able to optimize something clever based on the type of
allocator, but that's clearly more work and complexity than it's worth given
the small number of allocators we create/destroy.

svn path=/trunk/; revision=49512
2013-05-22 15:42:12 +00:00
Evan Huus 7959275717 Copy over a missing flag in the block allocator to fix the build-bot failure
at http://buildbot.wireshark.org/trunk/builders/Ubuntu-12.04-x64/builds/4250

Adding --verbose to get the random seed has payed off already!

svn path=/trunk/; revision=48584
2013-03-27 12:01:36 +00:00
Evan Huus 64af18ddda Fix -Wshadow that happens with some old versions of gcc (not mine, for some
reason).

Don't use g_assert_cmpuint, since it apparently causes warnings on windows that
I don't know how to get rid of safely without breaking the conditions being
checked.

svn path=/trunk/; revision=48575
2013-03-27 02:28:45 +00:00
Evan Huus 4252a18c90 Greatly improve wmem tests, including random fuzz-style testing. Improve
assertions in the block allocator, and fix one rare potential underflow caught
by the improved tests.

The tests now take ~200MB and 5-10 seconds to run. Hopefully this is small
enough for the build-bots to handle, if not then we can reduce the max
allocation size or max iterations to suit.

svn path=/trunk/; revision=48574
2013-03-27 00:27:45 +00:00
Evan Huus d30dbcbf9e Fix a few small bugs in the block allocator and add some more assertions to it
that run during normal usage. Simplify, expand and enable the test coverage,
since what's there now passes.

svn path=/trunk/; revision=48522
2013-03-24 12:53:22 +00:00
Evan Huus 8ccea72f10 Initial simple block allocator tests, not being run by default because they're
failing. I suspect it has to do with my lack of understanding of glib's unit
test framework, not the code being tested.

svn path=/trunk/; revision=48519
2013-03-24 00:45:32 +00:00
Bill Meier 0df5a9390d From beroset:
remove C++ incompatibilities
 https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=8416 attachment #10397

svn path=/trunk/; revision=48438
2013-03-20 01:18:10 +00:00
Evan Huus 8a874b238e Trivial tweaks to clean up cppcheck warnings.
svn path=/trunk/; revision=48435
2013-03-20 00:10:07 +00:00
Evan Huus b7b30a7cdf More block allocator fixes and cleanup. Most of the issues seem to have been
from one case I consistently forgot when typing it up originally, even though
it's clearly listed several places in my design notes.

Also include an #if0-ed out block of code to redirect emem to wmem for easy
testing (since there are very few common dissectors that use wmem right now).

svn path=/trunk/; revision=48434
2013-03-20 00:04:01 +00:00
Evan Huus 9be306d162 Add more and stricter wmem block allocator debug functions.
Re-enable the block allocator by default in trunk since it is much better
tested now - I've spent some time with a hack redirecting all emem allocations
to wmem, so it's seen a lot of traffic. I will still likely turn it off for
1.10 whenever that branches, just to be safe.

svn path=/trunk/; revision=48416
2013-03-19 15:32:42 +00:00
Evan Huus e9e54d662f Minor improvements to block allocator debugging.
svn path=/trunk/; revision=48252
2013-03-11 19:24:51 +00:00
Evan Huus 7f738de7b4 Glib docs recommend using the slice API if you know you won't need to realloc.
svn path=/trunk/; revision=48228
2013-03-10 15:04:04 +00:00
Evan Huus 3e919d2b03 Add a debug function (ifdef'ed out by default) and fix another two bugs found by
reading it's output.

svn path=/trunk/; revision=48198
2013-03-08 18:50:34 +00:00
Evan Huus 680d806afc Add another pair of assertions.
svn path=/trunk/; revision=48197
2013-03-08 18:27:29 +00:00
Evan Huus 1a78a717ed Should fix assertion failure seen by Anders on -dev.
https://www.wireshark.org/lists/wireshark-dev/201303/msg00081.html

svn path=/trunk/; revision=48193
2013-03-08 17:51:45 +00:00
Bill Meier 96a24cc79f Fix spelling/typos found using a list of commonly misspelled words.
The misspellings were mostly in comments but some were
in text strings visible to the user.


svn path=/trunk/; revision=47899
2013-02-26 04:42:26 +00:00
Evan Huus 48e188b560 Remove blocks from the free list when gc'ing them in wmem's block allocator.
Thanks to Jakub Zawadzki for catching this one.

svn path=/trunk/; revision=47843
2013-02-23 22:26:18 +00:00