wireshark/epan/tvbuff_lznt1.c

168 lines
3.7 KiB
C
Raw Normal View History

smb2: add support for decompression The latest iteration of Microsoft updates to SMB3 added compression to the protocol. This commit implements decompressing and dissecting compressed payloads. The compression algorithms that can be used are "Plain LZ77", "LZ77+Huffman" and "LZNT1" which you can read more about in the [MS-XCA] documentation. This set of algorithm is sometimes referred to as XPRESS. This commit reuses the existing uncompression API scheme already in place with zlib and brotli and adds 3 tvb_uncompress_*() function implemented in: * epan/tvbuff_lz77.c * epan/tvbuff_lz77huff.c * epan/tvbuff_lznt1.c A new function wmem_array_try_index() was added to the wmem_array API to make bound checked reads that fail gracefully. New tests for it have been added as well. Since both reads (tvb) and writes (wmem_array) are bound checked the risk for buffer overruns is drastically reduced. LZ77+Huffman has decoding tables and special care was taken to bound check these. Simplified versions of the implementations were succesfully tested against AFL (American Fuzzy Lop) for ~150 millions executions each. The SMB2/3 dissector was changed to deal with the new transform header for compressed packets (new protocol_id value) and READ request flags (COMPRESSED). Badly compressed or encrypted packets are now reported as such, and the decryption test suite was changed to reflect that. This commit also adds a test capture with 1 packet compressed with each algorithm as returned by Windows Server 2019, along with 3 matching tests in test/suite_dissection.py Change-Id: I2b84f56541f2f4ee7d886152794b993987dd10e7 Reviewed-on: https://code.wireshark.org/review/33855 Petri-Dish: Anders Broman <a.broman58@gmail.com> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2019-07-05 14:08:18 +00:00
/*
* Decompression code for LZNT1. This encoding is used by
smb2: add support for decompression The latest iteration of Microsoft updates to SMB3 added compression to the protocol. This commit implements decompressing and dissecting compressed payloads. The compression algorithms that can be used are "Plain LZ77", "LZ77+Huffman" and "LZNT1" which you can read more about in the [MS-XCA] documentation. This set of algorithm is sometimes referred to as XPRESS. This commit reuses the existing uncompression API scheme already in place with zlib and brotli and adds 3 tvb_uncompress_*() function implemented in: * epan/tvbuff_lz77.c * epan/tvbuff_lz77huff.c * epan/tvbuff_lznt1.c A new function wmem_array_try_index() was added to the wmem_array API to make bound checked reads that fail gracefully. New tests for it have been added as well. Since both reads (tvb) and writes (wmem_array) are bound checked the risk for buffer overruns is drastically reduced. LZ77+Huffman has decoding tables and special care was taken to bound check these. Simplified versions of the implementations were succesfully tested against AFL (American Fuzzy Lop) for ~150 millions executions each. The SMB2/3 dissector was changed to deal with the new transform header for compressed packets (new protocol_id value) and READ request flags (COMPRESSED). Badly compressed or encrypted packets are now reported as such, and the decryption test suite was changed to reflect that. This commit also adds a test capture with 1 packet compressed with each algorithm as returned by Windows Server 2019, along with 3 matching tests in test/suite_dissection.py Change-Id: I2b84f56541f2f4ee7d886152794b993987dd10e7 Reviewed-on: https://code.wireshark.org/review/33855 Petri-Dish: Anders Broman <a.broman58@gmail.com> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2019-07-05 14:08:18 +00:00
* Microsoft in various file formats and protocols including SMB3.
*
* See MS-XCA.
*
smb2: add support for decompression The latest iteration of Microsoft updates to SMB3 added compression to the protocol. This commit implements decompressing and dissecting compressed payloads. The compression algorithms that can be used are "Plain LZ77", "LZ77+Huffman" and "LZNT1" which you can read more about in the [MS-XCA] documentation. This set of algorithm is sometimes referred to as XPRESS. This commit reuses the existing uncompression API scheme already in place with zlib and brotli and adds 3 tvb_uncompress_*() function implemented in: * epan/tvbuff_lz77.c * epan/tvbuff_lz77huff.c * epan/tvbuff_lznt1.c A new function wmem_array_try_index() was added to the wmem_array API to make bound checked reads that fail gracefully. New tests for it have been added as well. Since both reads (tvb) and writes (wmem_array) are bound checked the risk for buffer overruns is drastically reduced. LZ77+Huffman has decoding tables and special care was taken to bound check these. Simplified versions of the implementations were succesfully tested against AFL (American Fuzzy Lop) for ~150 millions executions each. The SMB2/3 dissector was changed to deal with the new transform header for compressed packets (new protocol_id value) and READ request flags (COMPRESSED). Badly compressed or encrypted packets are now reported as such, and the decryption test suite was changed to reflect that. This commit also adds a test capture with 1 packet compressed with each algorithm as returned by Windows Server 2019, along with 3 matching tests in test/suite_dissection.py Change-Id: I2b84f56541f2f4ee7d886152794b993987dd10e7 Reviewed-on: https://code.wireshark.org/review/33855 Petri-Dish: Anders Broman <a.broman58@gmail.com> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2019-07-05 14:08:18 +00:00
* Copyright (C) 2019 Aurélien Aptel
*
* Wireshark - Network traffic analyzer
* By Gerald Combs <gerald@wireshark.org>
* Copyright 1998 Gerald Combs
*
* SPDX-License-Identifier: GPL-2.0-or-later
*/
#include <glib.h>
#include <epan/exceptions.h>
#include <epan/tvbuff.h>
#include <epan/wmem_scopes.h>
smb2: add support for decompression The latest iteration of Microsoft updates to SMB3 added compression to the protocol. This commit implements decompressing and dissecting compressed payloads. The compression algorithms that can be used are "Plain LZ77", "LZ77+Huffman" and "LZNT1" which you can read more about in the [MS-XCA] documentation. This set of algorithm is sometimes referred to as XPRESS. This commit reuses the existing uncompression API scheme already in place with zlib and brotli and adds 3 tvb_uncompress_*() function implemented in: * epan/tvbuff_lz77.c * epan/tvbuff_lz77huff.c * epan/tvbuff_lznt1.c A new function wmem_array_try_index() was added to the wmem_array API to make bound checked reads that fail gracefully. New tests for it have been added as well. Since both reads (tvb) and writes (wmem_array) are bound checked the risk for buffer overruns is drastically reduced. LZ77+Huffman has decoding tables and special care was taken to bound check these. Simplified versions of the implementations were succesfully tested against AFL (American Fuzzy Lop) for ~150 millions executions each. The SMB2/3 dissector was changed to deal with the new transform header for compressed packets (new protocol_id value) and READ request flags (COMPRESSED). Badly compressed or encrypted packets are now reported as such, and the decryption test suite was changed to reflect that. This commit also adds a test capture with 1 packet compressed with each algorithm as returned by Windows Server 2019, along with 3 matching tests in test/suite_dissection.py Change-Id: I2b84f56541f2f4ee7d886152794b993987dd10e7 Reviewed-on: https://code.wireshark.org/review/33855 Petri-Dish: Anders Broman <a.broman58@gmail.com> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2019-07-05 14:08:18 +00:00
#define MAX_INPUT_SIZE (16*1024*1024) /* 16MB */
static gboolean
uncompress_chunk(tvbuff_t *tvb, int offset, int in_size, wmem_array_t *obuf)
{
int in_off = 0, out_off = 0, out_start = 0;
guint8 flags;
guint i, j, val, pos;
out_start = wmem_array_get_count(obuf);
while (in_off < in_size) {
flags = tvb_get_guint8(tvb, offset+in_off);
in_off++;
for (i = 0; i < 8; i++) {
if (0 == ((flags>>i)&1)) {
val = tvb_get_guint8(tvb, offset+in_off);
in_off++;
wmem_array_append_one(obuf, val);
out_off++;
} else {
guint f, l_mask = 0x0FFF, o_shift = 12;
guint match_len, match_off;
f = tvb_get_letohs(tvb, offset+in_off);
in_off += 2;
pos = out_off-1;
while (pos >= 0x10) {
l_mask >>= 1;
o_shift -= 1;
pos >>= 1;
}
match_len = (f & l_mask) + 3;
match_off = (f >> o_shift) + 1;
for (j = 0; j < match_len; j++) {
guint8 byte;
if (match_off > (guint)out_off)
return FALSE;
if (wmem_array_try_index(obuf, out_start+out_off-match_off, &byte))
return FALSE;
wmem_array_append_one(obuf, byte);
out_off++;
}
}
if (in_off == in_size) {
goto out;
}
}
}
out:
return TRUE;
}
static gboolean
do_uncompress(tvbuff_t *tvb, int offset, int in_size, wmem_array_t *obuf)
{
int in_off = 0;
guint32 header, length, i;
gboolean ok;
if (!tvb)
return FALSE;
if (!in_size || in_size > MAX_INPUT_SIZE)
smb2: add support for decompression The latest iteration of Microsoft updates to SMB3 added compression to the protocol. This commit implements decompressing and dissecting compressed payloads. The compression algorithms that can be used are "Plain LZ77", "LZ77+Huffman" and "LZNT1" which you can read more about in the [MS-XCA] documentation. This set of algorithm is sometimes referred to as XPRESS. This commit reuses the existing uncompression API scheme already in place with zlib and brotli and adds 3 tvb_uncompress_*() function implemented in: * epan/tvbuff_lz77.c * epan/tvbuff_lz77huff.c * epan/tvbuff_lznt1.c A new function wmem_array_try_index() was added to the wmem_array API to make bound checked reads that fail gracefully. New tests for it have been added as well. Since both reads (tvb) and writes (wmem_array) are bound checked the risk for buffer overruns is drastically reduced. LZ77+Huffman has decoding tables and special care was taken to bound check these. Simplified versions of the implementations were succesfully tested against AFL (American Fuzzy Lop) for ~150 millions executions each. The SMB2/3 dissector was changed to deal with the new transform header for compressed packets (new protocol_id value) and READ request flags (COMPRESSED). Badly compressed or encrypted packets are now reported as such, and the decryption test suite was changed to reflect that. This commit also adds a test capture with 1 packet compressed with each algorithm as returned by Windows Server 2019, along with 3 matching tests in test/suite_dissection.py Change-Id: I2b84f56541f2f4ee7d886152794b993987dd10e7 Reviewed-on: https://code.wireshark.org/review/33855 Petri-Dish: Anders Broman <a.broman58@gmail.com> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2019-07-05 14:08:18 +00:00
return FALSE;
while (in_off < in_size) {
header = tvb_get_letohs(tvb, offset+in_off);
in_off += 2;
length = (header & 0x0FFF) + 1;
if (!(header & 0x8000)) {
for (i = 0; i < length; i++) {
guint8 v = tvb_get_guint8(tvb, offset+in_off);
wmem_array_append_one(obuf, v);
in_off++;
}
} else {
ok = uncompress_chunk(tvb, offset + in_off, length, obuf);
if (!ok)
return FALSE;
in_off += length;
}
}
return TRUE;
}
tvbuff_t *
tvb_uncompress_lznt1(tvbuff_t *tvb, const int offset, int in_size)
{
volatile gboolean ok = FALSE;
wmem_allocator_t *pool;
wmem_array_t *obuf;
tvbuff_t *out;
pool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);
obuf = wmem_array_sized_new(pool, 1, in_size*2);
TRY {
ok = do_uncompress(tvb, offset, in_size, obuf);
} CATCH_ALL {
ok = FALSE;
}
ENDTRY;
if (ok) {
/*
* Cannot pass a tvb free callback that frees the wmem
* pool, so we make an extra copy that uses bare
* pointers. This could be optimized if tvb API had a
* free pool callback of some sort.
smb2: add support for decompression The latest iteration of Microsoft updates to SMB3 added compression to the protocol. This commit implements decompressing and dissecting compressed payloads. The compression algorithms that can be used are "Plain LZ77", "LZ77+Huffman" and "LZNT1" which you can read more about in the [MS-XCA] documentation. This set of algorithm is sometimes referred to as XPRESS. This commit reuses the existing uncompression API scheme already in place with zlib and brotli and adds 3 tvb_uncompress_*() function implemented in: * epan/tvbuff_lz77.c * epan/tvbuff_lz77huff.c * epan/tvbuff_lznt1.c A new function wmem_array_try_index() was added to the wmem_array API to make bound checked reads that fail gracefully. New tests for it have been added as well. Since both reads (tvb) and writes (wmem_array) are bound checked the risk for buffer overruns is drastically reduced. LZ77+Huffman has decoding tables and special care was taken to bound check these. Simplified versions of the implementations were succesfully tested against AFL (American Fuzzy Lop) for ~150 millions executions each. The SMB2/3 dissector was changed to deal with the new transform header for compressed packets (new protocol_id value) and READ request flags (COMPRESSED). Badly compressed or encrypted packets are now reported as such, and the decryption test suite was changed to reflect that. This commit also adds a test capture with 1 packet compressed with each algorithm as returned by Windows Server 2019, along with 3 matching tests in test/suite_dissection.py Change-Id: I2b84f56541f2f4ee7d886152794b993987dd10e7 Reviewed-on: https://code.wireshark.org/review/33855 Petri-Dish: Anders Broman <a.broman58@gmail.com> Tested-by: Petri Dish Buildbot Reviewed-by: Peter Wu <peter@lekensteyn.nl>
2019-07-05 14:08:18 +00:00
*/
guint size = wmem_array_get_count(obuf);
guint8 *p = (guint8 *)g_malloc(size);
memcpy(p, wmem_array_get_raw(obuf), size);
out = tvb_new_real_data(p, size, size);
tvb_set_free_cb(out, g_free);
} else {
out = NULL;
}
wmem_destroy_allocator(pool);
return out;
}
tvbuff_t *
tvb_child_uncompress_lznt1(tvbuff_t *parent, tvbuff_t *tvb, const int offset, int in_size)
{
tvbuff_t *new_tvb = tvb_uncompress_lznt1(tvb, offset, in_size);
if (new_tvb)
tvb_set_child_real_data_tvbuff(parent, new_tvb);
return new_tvb;
}
/*
* Editor modelines - https://www.wireshark.org/tools/modelines.html
*
* Local variables:
* c-basic-offset: 8
* tab-width: 8
* indent-tabs-mode: t
* End:
*
* vi: set shiftwidth=8 tabstop=8 noexpandtab:
* :indentSize=8:tabSize=8:noTabs=false:
*/