forked from osmocom/wireshark
253 lines
8.4 KiB
Plaintext
253 lines
8.4 KiB
Plaintext
$Id$
|
|
|
|
1. Introduction
|
|
|
|
NB: Wmem still does not provide all of the functionality of emem
|
|
(see README.malloc), although it should provide most of it. New code
|
|
may still need to use emem for the time being.
|
|
|
|
The 'emem' memory manager (described in README.malloc) has been a part of
|
|
Wireshark since 2005 and has served us well, but is starting to show its age.
|
|
The framework has become increasingly difficult to maintain, and limitations
|
|
in the API have blocked progress on other long-term goals such as multi-
|
|
threading, and opening multiple files at once.
|
|
|
|
The 'wmem' memory manager is an attempt to write a new memory management
|
|
framework to replace emem. It provides a significantly updated API, a more
|
|
modular design, and it isn't all jammed into one 2500-line file.
|
|
|
|
Wmem was originally conceived in this email to the wireshark-dev mailing list:
|
|
https://www.wireshark.org/lists/wireshark-dev/201210/msg00178.html
|
|
|
|
The wmem code can now be found in epan/wmem/ in the Wireshark source tree.
|
|
|
|
2. Usage for Consumers
|
|
|
|
If you're writing a dissector, or other "userspace" code, then using wmem
|
|
should be very similar to using emem. All you need to do is include the header
|
|
(epan/wmem/wmem.h) and get a handle to a memory pool (if you want to *create*
|
|
a memory pool, see the section "3. Usage for Producers" below).
|
|
|
|
A memory pool is an opaque pointer to an object of type wmem_allocator_t, and
|
|
it is the very first parameter passed to almost every call you make to wmem.
|
|
Other than that parameter (and the fact that functions are prefixed wmem_
|
|
instead of ep_ or se_) usage is exactly like that of emem. For example:
|
|
|
|
wmem_alloc(myPool, 20);
|
|
|
|
allocates 20 bytes in the pool pointed to by myPool.
|
|
|
|
2.1 Available Pools
|
|
|
|
2.1.1 (Sort Of) Global Pools
|
|
|
|
Dissectors that include the wmem header file will have three pools available
|
|
to them automatically: wmem_packet_scope(), wmem_file_scope() and
|
|
wmem_epan_scope();
|
|
|
|
The packet pool is scoped to the dissection of each packet, replacing
|
|
emem's ep_ allocators. The file pool is scoped to the dissection of each file,
|
|
replacing emem's se_ allocators. For example:
|
|
|
|
ep_malloc(32);
|
|
se_malloc(sizeof(guint));
|
|
|
|
could be replaced with
|
|
|
|
wmem_alloc(wmem_packet_scope(), 32);
|
|
wmem_alloc(wmem_file_scope(), sizeof(guint));
|
|
|
|
NB: Using these pools outside of the appropriate scope (eg using the packet
|
|
pool when there isn't a packet being dissected) will throw an assertion.
|
|
See the comment in epan/wmem/wmem_scopes.c for details.
|
|
|
|
The epan pool is scoped to the library's lifetime - memory allocated in it is
|
|
not freed until epan_cleanup() is called, which is typically at the end of the
|
|
program.
|
|
|
|
2.1.2 Pinfo Pool
|
|
|
|
Certain places (such as AT_STRINGZ address allocations) need their memory to
|
|
stay around a little longer than the usual packet scope - basically until the
|
|
next packet is dissected. This is effectively the scope of Wireshark's pinfo
|
|
structure, so the pinfo struct has a 'pool' member which is a wmem pool scoped
|
|
to the lifetime of the pinfo struct.
|
|
|
|
2.2 Core API
|
|
|
|
- wmem_alloc
|
|
- wmem_alloc0
|
|
|
|
2.3 String Utilities
|
|
|
|
- wmem_strdup
|
|
- wmem_strndup
|
|
- wmem_strdup_printf
|
|
- wmem_strdup_vprintf
|
|
|
|
2.4 Stack
|
|
|
|
- wmem_create_stack
|
|
- wmem_stack_push
|
|
- wmem_stack_pop
|
|
- wmem_stack_peek
|
|
- wmem_stack_count
|
|
|
|
2.5 Slab
|
|
|
|
- wmem_create_slab
|
|
- wmem_slab_alloc
|
|
- wmem_slab_free
|
|
|
|
3. Usage for Producers
|
|
|
|
NB: If you're just writing a dissector, you probably don't need to read
|
|
this section.
|
|
|
|
One of the problems with the old emem framework was that there were basically
|
|
two allocator backends (glib and mmap) that were all mixed together in a mess
|
|
of if statements, environment variables and #ifdefs. In wmem the different
|
|
allocator backends are cleanly separated out, and it's up to the owner of the
|
|
pool to pick one.
|
|
|
|
3.1 Available Allocator Back-Ends
|
|
|
|
The currently available allocators are:
|
|
- glib (wmem_allocator_glib.h)
|
|
A simple allocator that g_allocs requested memory and tracks each
|
|
allocation via a linked list.
|
|
- block (wmem_allocator_block.h)
|
|
A simple block allocator that grabs large chunks of memory at a time
|
|
(8 MB currently) and serves allocations out of those chunks.
|
|
|
|
3.2 Creating a Pool
|
|
|
|
To create a pool, include the header for the type of allocator you want to use
|
|
and call the create function available in that header. For example,
|
|
|
|
#include "wmem/wmem_allocator_glib.h"
|
|
|
|
wmem_allocator_t *myPool;
|
|
myPool = wmem_create_glib_allocator();
|
|
|
|
From here on in, you don't need to remember which type of allocator you used
|
|
(although allocator authors are welcome to expose additional allocator-specific
|
|
helper functions in their headers). The "myPool" variable can be passed around
|
|
and used as normal in allocation requests as described in section 2 of this
|
|
document.
|
|
|
|
All the other functions described in this section can be found
|
|
in epan/wmem/wmem.h.
|
|
|
|
3.3 Destroying a Pool
|
|
|
|
Regardless of which allocator you used to create a pool, it can be destroyed
|
|
with a call to the function wmem_destroy_allocator(). For example:
|
|
|
|
#include "wmem/wmem.h"
|
|
#include "wmem/wmem_allocator_glib.h"
|
|
|
|
wmem_allocator_t *myPool;
|
|
|
|
myPool = wmem_create_glib_allocator();
|
|
|
|
/* Allocate some memory in myPool ... */
|
|
|
|
wmem_destroy_allocator(myPool);
|
|
|
|
Destroying a pool will free all the memory allocated in it.
|
|
|
|
3.4 Reusing a Pool
|
|
|
|
It is possible to free all the memory in a pool without destroying it,
|
|
allowing it to be reused later. Depending on the type of allocator, doing this
|
|
(by calling wmem_free_all()) can be significantly cheaper than fully destroying
|
|
and recreating the pool. This method is therefore recommended, especially when
|
|
the pool would otherwise be scoped to a single iteration of a loop. For example:
|
|
|
|
#include "wmem/wmem.h"
|
|
#include "wmem/wmem_allocator_glib.h"
|
|
|
|
wmem_allocator_t *myPool;
|
|
|
|
myPool = wmem_create_glib_allocator();
|
|
for (...) {
|
|
|
|
/* Allocate some memory in myPool ... */
|
|
|
|
/* Free the memory, faster than destroying and recreating
|
|
the pool each time through the loop. */
|
|
wmem_free_all(myPool);
|
|
}
|
|
wmem_destroy_allocator(myPool);
|
|
|
|
4. Internal Design
|
|
|
|
Despite being written in Wireshark's standard C90, wmem follows a fairly
|
|
object-oriented design pattern. Although efficiency is always a concern, the
|
|
primary goals in writing wmem were maintainability, and preventing memory
|
|
leaks.
|
|
|
|
4.1 struct _wmem_allocator_t
|
|
|
|
The heart of wmem is the _wmem_allocator_t structure defined in the
|
|
wmem_allocator.h header file. This structure uses C function pointers to
|
|
implement a common object-oriented pattern known as an interface (AKA 'abstract
|
|
class' to those of you who are more familiar with C++).
|
|
|
|
Different allocator implementations can provide exactly the same interface by
|
|
assigning their own functions (of the appropriate signature) to the members of
|
|
an instance of the structure. The structure currently has four values:
|
|
|
|
- alloc()
|
|
- free_all()
|
|
- destroy()
|
|
- private_data
|
|
|
|
The private_data pointer is a void pointer that the implementation can use to
|
|
store whatever internal structures it needs. The other three functions should
|
|
be fairly obvious - each one is called with a pointer to the private_data in
|
|
addition to any other arguments.
|
|
|
|
4.2 Pool-Agnostic API
|
|
|
|
One of the issues with emem was that the API (including the public data
|
|
structures) required wrapper functions for each scope implemented. Even
|
|
if there was a stack implementation in emem, it wasn't necessarily available
|
|
for use with file-scope memory unless someone took the time to write se_stack_
|
|
wrapper functions for the interface.
|
|
|
|
In wmem, all public APIs take the pool as the first argument, so that they can
|
|
be written once and used with any available memory pool. Data structures like
|
|
wmem's stack implementation only take the pool when created - the provided
|
|
pointer is stored internally with the data structure, and subsequent calls
|
|
(like push and pop) will take the stack itself instead of the pool.
|
|
|
|
5. TODO List
|
|
|
|
The following is an incomplete list of things that emem provides but wmem has
|
|
not yet implemented:
|
|
|
|
- strbuf
|
|
- red-black tree
|
|
- tvb_memdup
|
|
- canaries
|
|
|
|
The following is a list of things that emem doesn't provide but that it might
|
|
be nice if wmem did provide them:
|
|
|
|
- radix tree
|
|
|
|
/*
|
|
* Editor modelines - http://www.wireshark.org/tools/modelines.html
|
|
*
|
|
* Local variables:
|
|
* c-basic-offset: 4
|
|
* tab-width: 8
|
|
* indent-tabs-mode: nil
|
|
* End:
|
|
*
|
|
* vi: set shiftwidth=4 tabstop=8 expandtab:
|
|
* :indentSize=4:tabSize=8:noTabs=true:
|
|
*/
|