epan: Register dynamic column fields and make them filterable

Make the text of each registered column a FT_STRING field that can be
filtered, prefixed with _ws.col - these work in display filters, filters
in taps, coloring rules, Wireshark read filters, and in the -Y, -R, -e,
and -j options to tshark. Use them as the default "Apply as Filter" value
for the columns that aren't handled by anything else currently.

Because only the columns formats that actually correspond to columns
get filled in (invisible columns work), register and deregister the
fields when the columns change.

Use the lower case version of the rest of the COL_* define for each
column as the field name.

This adds a number of conditions to "when are the columns needed",
including when the main display filter or any filter on a tap is
using one of these fields.

Custom columns are currently not implemented. For custom columns, the
tree then has to be further primed with any fields used by the custom
columns as well. (Perhaps that should happen in epan_dissect_run() -
are there any cases where we construct the columns and don't want to
prime with any field that custom columns contains? Possibly in taps
that we know only use build in columns.)

Thus, for performance reasons, you're better off matching an ordinary
field if possible; it takes extra time to generate the columns and many
of them are numeric types. (Note that you can always convert a non-string
field to a string field if you want regex matching, consult the
*wireshark-filter(4)* man page.) It does save a bit on typing (especially
for a multifield custom column) and remembering the column title might
be easier in some cases.

The columns are set before the color filters, which means that you
can have a color filter that depends on a built-in column like Info or
Protocol.

Remove the special handling for the -e option to tshark. Note that
the behavior is a little different now, because fixed field names
are used instead of the titles (using the titles allowed illegal
filter names, because it wasn't going through the filter engine.)
For default names, this means that they're no longer capitalized,
so "_ws.col.info" instead of "_ws.col.Info" - hopefully a small
price in exchange for the filters working everywhere.

The output format for -T fields remains the same; all that special
handling is removed (except for remembering if someone asked for
a column field to know that columns should be constructed.)

They're also set before the postdissectors, so postdissectors can
have access.

Anything that depends on whether a packet and previous packets are
displayed (COL_DELTA_TIME_DIS or COL_CUMULATIVE_BYTES) doesn't work
the way most people expect, so don't register fields for those.
(The same is already true of color filters that use those, along with
color filters that use the color filter fields.)

Fix #16576. Fix #17971. Fix #4684. Fix #13491. Fix #13941.
This commit is contained in:
John Thacker 2023-05-01 08:02:18 -04:00
parent 99ef0560b7
commit 1b82eda9eb
22 changed files with 472 additions and 69 deletions

View File

@ -312,9 +312,9 @@ contain a GUID.
Add a field to the list of fields to display if *-T ek|fields|json|pdml*
is selected. This option can be used multiple times on the command line.
At least one field must be provided if the *-T fields* option is
selected. Column names may be used prefixed with "_ws.col."
selected. Column types may be used prefixed with "_ws.col."
Example: *tshark -e frame.number -e ip.addr -e udp -e _ws.col.Info*
Example: *tshark -e frame.number -e ip.addr -e udp -e _ws.col.info*
Fields are separated by tab characters by default. *-E* controls the
format of the printed fields.

View File

@ -143,7 +143,24 @@ Recognizes PCAP traces with the link type LINKTYPE_FIRA_UCI=299.
the capture file via the GUI, similar to the options --inject-secrets and
--discard-all-secrets in editcap.
// === Removed Features and Support
* The text of any configured column (displayed or hidden) can be filtered
anywhere that filters are used - in display filters, filters in taps, coloring
rules, Wireshark read filters, and the -Y, -R, and -e options to tshark,
the "Apply as Filter" GUI option, etc.
** The filter field names are prefixed by "_ws.col", followed by a lowercase
version of the COL_ name found in epan/column-utils.h, e.g. "_ws.col.info"
or "_ws.col.protocol"
** Using the column names as a filter is slower than other filter types
because the columns must be constructed, so when the same filtering
can be achieved via other fields, prefer that.
=== Removed Features and Support
* With the addition of the universal and consistent filtering support for
column text, the previous support in the -e option to tshark for displaying
column text via the column title, e.g. "_ws.col.Info", has been removed.
The previous implementation allowed names that are not legal filter names
and wsbuglink:16576[silently ignored columns that didn't exist.]
// === Removed Dissectors

View File

@ -24,6 +24,10 @@ extern "C" {
* Column info.
*/
typedef struct _proto_node proto_tree;
#define COLUMN_FIELD_FILTER "_ws.col."
/** Column expression */
typedef struct {
const gchar **col_expr; /**< Filter expression */
@ -43,6 +47,7 @@ typedef struct {
gchar *col_buf; /**< Buffer into which to copy data for column */
int col_fence; /**< Stuff in column buffer before this index is immutable */
gboolean writable; /**< writable or not */
int hf_id;
} col_item_t;
/** Column info */
@ -111,6 +116,12 @@ gboolean col_has_time_fmt(column_info *cinfo, const gint col);
WS_DLL_PUBLIC
gboolean col_based_on_frame_data(column_info *cinfo, const gint col);
void
col_register_protocol(void);
extern
void col_dissect(tvbuff_t *tvb, packet_info *pinfo, proto_tree *tree);
#ifdef __cplusplus
}
#endif /* __cplusplus */

View File

@ -26,6 +26,7 @@
#include "osi-utils.h"
#include "value_string.h"
#include "column-info.h"
#include "column.h"
#include "proto.h"
#include <epan/strutil.h>
@ -49,6 +50,9 @@ static char *col_decimal_point;
/* Used to indicate updated column information, e.g. a new request/response. */
static gboolean col_data_changed_;
static int proto_cols = -1;
static gint ett_cols = -1;
/* Allocate all the data structures for constructing column data, given
the number of columns. */
void
@ -363,6 +367,34 @@ void col_custom_set_edt(epan_dissect_t *edt, column_info *cinfo)
}
}
#if 0
// Needed if we create _ws.col.custom
static void
col_custom_set(proto_tree *tree, column_info *cinfo)
{
int i;
col_item_t* col_item;
if (!HAVE_CUSTOM_COLS(cinfo))
return;
for (i = cinfo->col_first[COL_CUSTOM];
i <= cinfo->col_last[COL_CUSTOM]; i++) {
col_item = &cinfo->columns[i];
if (col_item->fmt_matx[COL_CUSTOM] &&
col_item->col_custom_fields &&
col_item->col_custom_fields_ids) {
col_item->col_data = col_item->col_buf;
cinfo->col_expr.col_expr[i] = proto_custom_set(tree, col_item->col_custom_fields_ids,
col_item->col_custom_occurrence,
col_item->col_buf,
cinfo->col_expr.col_expr_val[i],
COL_MAX_LEN);
}
}
}
#endif
void
col_custom_prime_edt(epan_dissect_t *edt, column_info *cinfo)
{
@ -2300,6 +2332,10 @@ col_fill_in(packet_info *pinfo, const gboolean fill_col_exprs, const gboolean fi
col_set_port(pinfo, i, FALSE, FALSE, fill_col_exprs);
break;
case COL_CUSTOM:
/* Formatting handled by col_custom_set_edt() / col_custom_get_filter() */
break;
case NUM_COL_FMTS: /* keep compiler happy - shouldn't get here */
ws_assert_not_reached();
break;
@ -2308,9 +2344,13 @@ col_fill_in(packet_info *pinfo, const gboolean fill_col_exprs, const gboolean fi
ws_assert_not_reached();
}
/*
* Formatting handled by col_custom_set_edt() (COL_CUSTOM), expert.c
* (COL_EXPERT), or individual dissectors.
* Formatting handled by expert.c (COL_EXPERT), or individual
* dissectors. Fill in from the text using the internal hfid.
*/
if (fill_col_exprs) {
pinfo->cinfo->col_expr.col_expr[i] = proto_registrar_get_nth(col_item->hf_id)->abbrev;
(void) g_strlcpy(pinfo->cinfo->col_expr.col_expr_val[i], pinfo->cinfo->columns[i].col_data, (col_item->col_fmt == COL_INFO) ? COL_MAX_INFO_LEN : COL_MAX_LEN);
}
break;
}
}
@ -2358,6 +2398,57 @@ gboolean col_data_changed(void) {
col_data_changed_ = FALSE;
return cur_cdc;
}
void
col_register_protocol(void)
{
/* This gets called by proto_init() before column_register_fields()
* gets called by the preference modules actually getting registered.
*/
if (proto_cols == -1) {
proto_cols = proto_get_id_by_filter_name("_ws.col");
}
if (proto_cols == -1) {
proto_cols = proto_register_protocol("Wireshark Columns", "Columns", "_ws.col");
}
static gint *ett[] = {
&ett_cols
};
proto_register_subtree_array(ett, G_N_ELEMENTS(ett));
}
void
col_dissect(tvbuff_t *tvb, packet_info *pinfo, proto_tree *tree)
{
proto_item *ti;
proto_tree *col_tree;
column_info *cinfo = pinfo->cinfo;
if (!cinfo) {
return;
}
if (proto_field_is_referenced(tree, proto_cols)) {
// XXX: Needed if we also create _ws.col.custom
//col_custom_set(tree, cinfo);
col_fill_in(pinfo, FALSE, TRUE);
ti = proto_tree_add_item(tree, proto_cols, tvb, 0, 0, ENC_NA);
proto_item_set_hidden(ti);
col_tree = proto_item_add_subtree(ti, ett_cols);
for (int i = 0; i < cinfo->num_cols; ++i) {
if (cinfo->columns[i].hf_id != -1) {
if (cinfo->columns[i].col_fmt == COL_CUSTOM) {
ti = proto_tree_add_string_format(col_tree, cinfo->columns[i].hf_id, tvb, 0, 0, get_column_text(cinfo, i), "%s: %s", get_column_title(i), get_column_text(cinfo, i));
} else {
ti = proto_tree_add_string(col_tree, cinfo->columns[i].hf_id, tvb, 0, 0, get_column_text(cinfo, i));
}
proto_item_set_hidden(ti);
}
}
}
}
/*
* Editor modelines
*

View File

@ -26,6 +26,10 @@
#include <epan/packet.h>
#include <wsutil/ws_assert.h>
static int proto_cols = -1;
static hf_register_info *hf_cols = NULL;
static unsigned int hf_cols_cleanup = 0;
/* Given a format number (as defined in column-utils.h), returns its equivalent
string */
const gchar *
@ -155,6 +159,67 @@ col_format_desc(const gint fmt_num) {
return val_str;
}
/* Given a format number (as defined in column-utils.h), returns its
filter abbreviation */
const gchar *
col_format_abbrev(const gint fmt_num) {
static const value_string alist_vals[] = {
{ COL_ABS_YMD_TIME, COLUMN_FIELD_FILTER"abs_ymd_time" },
{ COL_ABS_YDOY_TIME, COLUMN_FIELD_FILTER"abs_ydoy_time" },
{ COL_ABS_TIME, COLUMN_FIELD_FILTER"abs_time" },
{ COL_CUMULATIVE_BYTES, COLUMN_FIELD_FILTER"cumulative_bytes" },
{ COL_CUSTOM, COLUMN_FIELD_FILTER"custom" },
{ COL_DELTA_TIME_DIS, COLUMN_FIELD_FILTER"delta_time_dis" },
{ COL_DELTA_TIME, COLUMN_FIELD_FILTER"delta_time" },
{ COL_RES_DST, COLUMN_FIELD_FILTER"res_dst" },
{ COL_UNRES_DST, COLUMN_FIELD_FILTER"unres_dst" },
{ COL_RES_DST_PORT, COLUMN_FIELD_FILTER"res_dst_port" },
{ COL_UNRES_DST_PORT, COLUMN_FIELD_FILTER"unres_dst_port" },
{ COL_DEF_DST, COLUMN_FIELD_FILTER"def_dst" },
{ COL_DEF_DST_PORT, COLUMN_FIELD_FILTER"def_dst_port" },
{ COL_EXPERT, COLUMN_FIELD_FILTER"expert" },
{ COL_IF_DIR, COLUMN_FIELD_FILTER"if_dir" },
{ COL_FREQ_CHAN, COLUMN_FIELD_FILTER"freq_chan" },
{ COL_DEF_DL_DST, COLUMN_FIELD_FILTER"def_dl_dst" },
{ COL_DEF_DL_SRC, COLUMN_FIELD_FILTER"def_dl_src" },
{ COL_RES_DL_DST, COLUMN_FIELD_FILTER"res_dl_dst" },
{ COL_UNRES_DL_DST, COLUMN_FIELD_FILTER"unres_dl_dst" },
{ COL_RES_DL_SRC, COLUMN_FIELD_FILTER"res_dl_src" },
{ COL_UNRES_DL_SRC, COLUMN_FIELD_FILTER"unres_dl_src" },
{ COL_RSSI, COLUMN_FIELD_FILTER"rssi" },
{ COL_TX_RATE, COLUMN_FIELD_FILTER"tx_rate" },
{ COL_DSCP_VALUE, COLUMN_FIELD_FILTER"dscp" },
{ COL_INFO, COLUMN_FIELD_FILTER"info" },
{ COL_RES_NET_DST, COLUMN_FIELD_FILTER"res_net_dst" },
{ COL_UNRES_NET_DST, COLUMN_FIELD_FILTER"unres_net_dst" },
{ COL_RES_NET_SRC, COLUMN_FIELD_FILTER"res_net_src" },
{ COL_UNRES_NET_SRC, COLUMN_FIELD_FILTER"unres_net_src" },
{ COL_DEF_NET_DST, COLUMN_FIELD_FILTER"def_net_dst" },
{ COL_DEF_NET_SRC, COLUMN_FIELD_FILTER"def_net_src" },
{ COL_NUMBER, COLUMN_FIELD_FILTER"number" },
{ COL_PACKET_LENGTH, COLUMN_FIELD_FILTER"packet_length" },
{ COL_PROTOCOL, COLUMN_FIELD_FILTER"protocol" },
{ COL_REL_TIME, COLUMN_FIELD_FILTER"rel_time" },
{ COL_DEF_SRC, COLUMN_FIELD_FILTER"def_src" },
{ COL_DEF_SRC_PORT, COLUMN_FIELD_FILTER"def_src_port" },
{ COL_RES_SRC, COLUMN_FIELD_FILTER"res_src" },
{ COL_UNRES_SRC, COLUMN_FIELD_FILTER"unres_src" },
{ COL_RES_SRC_PORT, COLUMN_FIELD_FILTER"res_src_port" },
{ COL_UNRES_SRC_PORT, COLUMN_FIELD_FILTER"unres_src_port" },
{ COL_CLS_TIME, COLUMN_FIELD_FILTER"cls_time" },
{ COL_UTC_YMD_TIME, COLUMN_FIELD_FILTER"utc_ymc_time" },
{ COL_UTC_YDOY_TIME, COLUMN_FIELD_FILTER"utc_ydoy_time" },
{ COL_UTC_TIME, COLUMN_FIELD_FILTER"utc_time" },
{ 0, NULL }
};
const gchar *val_str = try_val_to_str(fmt_num, alist_vals);
ws_assert(val_str != NULL);
return val_str;
}
/* Array of columns that have been migrated to custom columns */
struct deprecated_columns {
const gchar *col_fmt;
@ -931,13 +996,15 @@ col_finalize(column_info *cinfo)
get_column_format_matches(col_item->fmt_matx, col_item->col_fmt);
col_item->col_data = NULL;
if (col_item->col_fmt == COL_INFO)
if (col_item->col_fmt == COL_INFO) {
col_item->col_buf = g_new(gchar, COL_MAX_INFO_LEN);
else
cinfo->col_expr.col_expr_val[i] = g_new(gchar, COL_MAX_INFO_LEN);
} else {
col_item->col_buf = g_new(gchar, COL_MAX_LEN);
cinfo->col_expr.col_expr_val[i] = g_new(gchar, COL_MAX_LEN);
}
cinfo->col_expr.col_expr[i] = "";
cinfo->col_expr.col_expr_val[i] = g_new(gchar, COL_MAX_LEN);
}
cinfo->col_expr.col_expr[i] = NULL;
@ -975,6 +1042,7 @@ build_column_format_array(column_info *cinfo, const gint num_cols, const gboolea
col_item->col_custom_fields = g_strdup(get_column_custom_fields(i));
col_item->col_custom_occurrence = get_column_custom_occurrence(i);
}
col_item->hf_id = proto_registrar_get_id_byname(col_format_abbrev(col_item->col_fmt));
if(reset_fences)
col_item->col_fence = 0;
@ -983,6 +1051,75 @@ build_column_format_array(column_info *cinfo, const gint num_cols, const gboolea
col_finalize(cinfo);
}
static void
column_deregister_fields(void)
{
if (hf_cols) {
for (unsigned int i = 0; i < hf_cols_cleanup; ++i) {
proto_deregister_field(proto_cols, *(hf_cols[i].p_id));
g_free(hf_cols[i].p_id);
}
proto_add_deregistered_data(hf_cols);
hf_cols = NULL;
hf_cols_cleanup = 0;
}
}
void
column_register_fields(void)
{
int* hf_id;
GArray *hf_col_array;
hf_register_info new_hf;
fmt_data *cfmt;
gboolean *used_fmts;
if (proto_cols == -1) {
proto_cols = proto_get_id_by_filter_name("_ws.col");
}
if (proto_cols == -1) {
proto_cols = proto_register_protocol("Wireshark Columns", "Columns", "_ws.col");
}
column_deregister_fields();
if (prefs.col_list != NULL) {
prefs.num_cols = g_list_length(prefs.col_list);
hf_col_array = g_array_new(FALSE, TRUE, sizeof(hf_register_info));
used_fmts = g_new0(gboolean, NUM_COL_FMTS);
/* Only register a field for each format type once, but don't register
* these at all. The first two behave oddly (because they depend on
* whether the current field and previous fields are displayed). We
* might want to do custom columns in the future, though.
*/
used_fmts[COL_DELTA_TIME_DIS] = 1;
used_fmts[COL_CUMULATIVE_BYTES] = 1;
used_fmts[COL_CUSTOM] = 1;
for (GList *elem = g_list_first(prefs.col_list); elem != NULL; elem = elem->next) {
cfmt = (fmt_data*)elem->data;
if (!used_fmts[cfmt->fmt]) {
used_fmts[cfmt->fmt] = TRUE;
hf_id = g_new(int, 1);
*hf_id = -1;
new_hf.p_id = hf_id;
new_hf.hfinfo.name = g_strdup(col_format_desc(cfmt->fmt));
new_hf.hfinfo.abbrev = g_strdup(col_format_abbrev(cfmt->fmt));
new_hf.hfinfo.type = FT_STRING;
new_hf.hfinfo.display = BASE_NONE;
new_hf.hfinfo.strings = NULL;
new_hf.hfinfo.bitmask = 0;
new_hf.hfinfo.blurb = NULL;
HFILL_INIT(new_hf);
g_array_append_vals(hf_col_array, &new_hf, 1);
}
}
g_free(used_fmts);
hf_cols_cleanup = hf_col_array->len;
proto_register_field_array(proto_cols, (hf_register_info*)hf_col_array->data, hf_col_array->len);
hf_cols = (hf_register_info*)g_array_free(hf_col_array, FALSE);
}
}
/*
* Editor modelines - https://www.wireshark.org/tools/modelines.html
*

View File

@ -35,6 +35,8 @@ const gchar *col_format_to_string(const gint);
WS_DLL_PUBLIC
const gchar *col_format_desc(const gint);
WS_DLL_PUBLIC
const gchar *col_format_abbrev(const gint);
WS_DLL_PUBLIC
gint get_column_format(const gint);
WS_DLL_PUBLIC
void set_column_format(const gint, const gint);
@ -117,6 +119,9 @@ gboolean parse_column_format(fmt_data *cfmt, const char *fmt);
*/
WS_DLL_PUBLIC
void try_convert_to_custom_column(char **fmt);
WS_DLL_PUBLIC
void column_register_fields(void);
#ifdef __cplusplus
}
#endif /* __cplusplus */

View File

@ -710,6 +710,23 @@ dfilter_interested_in_proto(const dfilter_t *df, int proto_id)
return FALSE;
}
gboolean
dfilter_requires_columns(const dfilter_t *df)
{
if (df == NULL) {
return FALSE;
}
/* XXX: Could cache this like packet_cache_proto_handles */
static int proto_cols = -1;
if (proto_cols == -1) {
proto_cols = proto_get_id_by_filter_name("_ws.col");
}
ws_assert(proto_cols != -1);
return dfilter_interested_in_proto(df, proto_cols);
}
GPtrArray *
dfilter_deprecated_tokens(dfilter_t *df) {
if (df->deprecated && df->deprecated->len > 0) {

View File

@ -151,6 +151,10 @@ dfilter_interested_in_field(const dfilter_t *df, int hfid);
gboolean
dfilter_interested_in_proto(const dfilter_t *df, int proto_id);
WS_DLL_PUBLIC
gboolean
dfilter_requires_columns(const dfilter_t *df);
WS_DLL_PUBLIC
GPtrArray *
dfilter_deprecated_tokens(dfilter_t *df);

View File

@ -33,6 +33,7 @@
#include <epan/proto_data.h>
#include <epan/addr_resolv.h>
#include <epan/wmem_scopes.h>
#include <epan/column-info.h>
#include "packet-frame.h"
#include "packet-bblog.h"
@ -1452,6 +1453,18 @@ dissect_frame(tvbuff_t *tvb, packet_info *pinfo, proto_tree *parent_tree, void*
proto_item_set_generated(ti);
}
/* Add the columns as fields. We have to do this here, so that
* they're available for postdissectors that want all the fields.
*
* Note the coloring rule names are set after this, which means
* that you can set a coloring rule based on the value of a column,
* like _ws.col.protocol or _ws.col.info.
* OTOH, if we created _ws.col.custom, and a custom column used
* frame.coloring_rule.name, filtering with it wouldn't work -
* but you can filter on that field directly, so that doesn't matter.
*/
col_dissect(tvb, pinfo, parent_tree);
/* Call postdissectors if we have any (while trying to avoid another
* TRY/CATCH)
*/

View File

@ -2611,6 +2611,8 @@ column_format_init_cb(pref_t* pref, GList** value)
dest_cfmt->resolved = src_cfmt->resolved;
pref->default_val.list = g_list_append(pref->default_val.list, dest_cfmt);
}
column_register_fields();
}
static void
@ -2717,6 +2719,7 @@ column_format_set_cb(pref_t* pref, const gchar* value, unsigned int* changed_fla
prefs_clear_string_list(col_l);
free_string_like_preference(hidden_pref);
column_register_fields();
return PREFS_SET_OK;
}

View File

@ -2056,8 +2056,6 @@ void output_fields_free(output_fields_t* fields)
g_free(fields);
}
#define COLUMN_FIELD_FILTER "_ws.col."
void output_fields_add(output_fields_t *fields, const gchar *field)
{
gchar *field_copy;
@ -2113,9 +2111,6 @@ output_field_check(void *data, void *user_data)
gchar *field = (gchar *)data;
GSList **invalid_fields = (GSList **)user_data;
if (!strncmp(field, COLUMN_FIELD_FILTER, strlen(COLUMN_FIELD_FILTER)))
return;
if (!proto_registrar_get_byname(field)) {
*invalid_fields = g_slist_prepend(*invalid_fields, field);
}
@ -2398,12 +2393,9 @@ static void proto_tree_get_node_field_values(proto_node *node, gpointer data)
}
}
static void write_specified_fields(fields_format format, output_fields_t *fields, epan_dissect_t *edt, column_info *cinfo, FILE *fh, json_dumper *dumper)
static void write_specified_fields(fields_format format, output_fields_t *fields, epan_dissect_t *edt, column_info *cinfo _U_, FILE *fh, json_dumper *dumper)
{
gsize i;
gint col;
gchar *col_name;
gpointer field_index;
write_field_data_t data;
@ -2447,22 +2439,6 @@ static void write_specified_fields(fields_format format, output_fields_t *fields
proto_tree_children_foreach(edt->tree, proto_tree_get_node_field_values,
&data);
/* Add columns to fields */
if (fields->includes_col_fields) {
for (col = 0; col < cinfo->num_cols; col++) {
if (!get_column_visible(col))
continue;
/* Prepend COLUMN_FIELD_FILTER as the field name */
col_name = ws_strdup_printf("%s%s", COLUMN_FIELD_FILTER, cinfo->columns[col].col_title);
field_index = g_hash_table_lookup(fields->field_indicies, col_name);
g_free(col_name);
if (NULL != field_index) {
format_field_values(fields, field_index, g_strdup(get_column_text(cinfo, col)));
}
}
}
switch (format) {
case FORMAT_CSV:
for(i = 0; i < fields->fields->len; ++i) {

View File

@ -41,7 +41,7 @@
#include "tvbuff.h"
#include <epan/wmem_scopes.h>
#include "charsets.h"
#include "column-utils.h"
#include "column-info.h"
#include "to_str.h"
#include "osi-utils.h"
#include "expert.h"
@ -585,6 +585,7 @@ proto_init(GSList *register_all_plugin_protocols_list,
register_number_string_decodinws_error();
register_string_errors();
ftypes_register_pseudofields();
col_register_protocol();
/* Have each built-in dissector register its protocols, fields,
dissector tables, and dissectors to be called through a

View File

@ -708,6 +708,29 @@ tap_listeners_require_dissection(void)
}
/*
* Return TRUE if we have one or more tap listeners that require the columns,
* FALSE otherwise.
*/
gboolean
tap_listeners_require_columns(void)
{
tap_listener_t *tap_queue = tap_listener_queue;
while(tap_queue) {
if(tap_queue->flags & TL_REQUIRES_COLUMNS)
return TRUE;
if(dfilter_requires_columns(tap_queue->code))
return TRUE;
tap_queue = tap_queue->next;
}
return FALSE;
}
/* Returns TRUE there is an active tap listener for the specified tap id. */
gboolean
have_tap_listener(int tap_id)

View File

@ -248,6 +248,12 @@ WS_DLL_PUBLIC void remove_tap_listener(void *tapdata);
*/
WS_DLL_PUBLIC gboolean tap_listeners_require_dissection(void);
/**
* Return TRUE if we have one or more tap listeners that require the columns,
* FALSE otherwise.
*/
WS_DLL_PUBLIC gboolean tap_listeners_require_columns(void);
/** Returns TRUE there is an active tap listener for the specified tap id. */
WS_DLL_PUBLIC gboolean have_tap_listener(int tap_id);

35
file.c
View File

@ -560,8 +560,10 @@ cf_read(capture_file *cf, gboolean reloading)
epan_dissect_init(&edt, cf->epan, create_proto_tree, FALSE);
/* If any tap listeners require the columns, construct them. */
cinfo = (tap_flags & TL_REQUIRES_COLUMNS) ? &cf->cinfo : NULL;
/* If the display filter or any tap listeners require the columns,
* construct them. */
cinfo = (tap_listeners_require_columns() ||
dfilter_requires_columns(dfcode)) ? &cf->cinfo : NULL;
/* Find the size of the file. */
size = wtap_file_size(cf->provider.wth, NULL);
@ -828,8 +830,10 @@ cf_continue_tail(capture_file *cf, volatile int to_read, wtap_rec *rec,
gint64 data_offset = 0;
column_info *cinfo;
/* If any tap listeners require the columns, construct them. */
cinfo = (tap_flags & TL_REQUIRES_COLUMNS) ? &cf->cinfo : NULL;
/* If the display filter or any tap listeners require the columns,
* construct them. */
cinfo = (tap_listeners_require_columns() ||
dfilter_requires_columns(dfcode)) ? &cf->cinfo : NULL;
while (to_read != 0) {
wtap_cleareof(cf->provider.wth);
@ -938,8 +942,10 @@ cf_finish_tail(capture_file *cf, wtap_rec *rec, Buffer *buf, int *err,
/* Get the union of the flags for all tap listeners. */
tap_flags = union_of_tap_listener_flags();
/* If any tap listeners require the columns, construct them. */
cinfo = (tap_flags & TL_REQUIRES_COLUMNS) ? &cf->cinfo : NULL;
/* If the display filter or any tap listeners require the columns,
* construct them. */
cinfo = (tap_listeners_require_columns() ||
dfilter_requires_columns(dfcode)) ? &cf->cinfo : NULL;
/*
* Determine whether we need to create a protocol tree.
@ -1290,12 +1296,16 @@ read_record(capture_file *cf, wtap_rec *rec, Buffer *buf, dfilter_t *dfcode,
if (cf->rfcode) {
epan_dissect_t rf_edt;
column_info *rf_cinfo = NULL;
epan_dissect_init(&rf_edt, cf->epan, TRUE, FALSE);
epan_dissect_prime_with_dfilter(&rf_edt, cf->rfcode);
if (dfilter_requires_columns(cf->rfcode)) {
rf_cinfo = &cf->cinfo;
}
epan_dissect_run(&rf_edt, cf->cd_t, rec,
frame_tvbuff_new_buffer(&cf->provider, &fdlocal, buf),
&fdlocal, NULL);
&fdlocal, rf_cinfo);
passed = dfilter_apply_edt(cf->rfcode, &rf_edt);
epan_dissect_cleanup(&rf_edt);
}
@ -1723,8 +1733,10 @@ rescan_packets(capture_file *cf, const char *action, const char *action_item, gb
/* Get the union of the flags for all tap listeners. */
tap_flags = union_of_tap_listener_flags();
/* If any tap listeners require the columns, construct them. */
cinfo = (tap_flags & TL_REQUIRES_COLUMNS) ? &cf->cinfo : NULL;
/* If the display filter or any tap listeners require the columns,
* construct them. */
cinfo = (tap_listeners_require_columns() ||
dfilter_requires_columns(dfcode)) ? &cf->cinfo : NULL;
/*
* Determine whether we need to create a protocol tree.
@ -1784,6 +1796,9 @@ rescan_packets(capture_file *cf, const char *action, const char *action_item, gb
if (!create_proto_tree && have_filtering_tap_listeners()) {
create_proto_tree = TRUE;
}
if (!cinfo && tap_listeners_require_columns()) {
cinfo = &cf->cinfo;
}
/* We need to redissect the packets so we have to discard our old
* packet list store. */
@ -2337,7 +2352,7 @@ cf_retap_packets(capture_file *cf)
tap_flags = union_of_tap_listener_flags();
/* If any tap listeners require the columns, construct them. */
callback_args.cinfo = (tap_flags & TL_REQUIRES_COLUMNS) ? &cf->cinfo : NULL;
callback_args.cinfo = (tap_listeners_require_columns()) ? &cf->cinfo : NULL;
/*
* Determine whether we need to create a protocol tree.

View File

@ -584,7 +584,7 @@ sharkd_retap(void)
tap_flags = union_of_tap_listener_flags();
/* If any tap listeners require the columns, construct them. */
cinfo = (tap_flags & TL_REQUIRES_COLUMNS) ? &cfile.cinfo : NULL;
cinfo = (tap_listeners_require_columns()) ? &cfile.cinfo : NULL;
/*
* Determine whether we need to create a protocol tree.

View File

@ -8,7 +8,7 @@ import pytest
@pytest.fixture
def dfilter_cmd(cmd_tshark, capture_file, request):
def wrapped(dfilter, frame_number=None, prefs=None):
def wrapped(dfilter, frame_number=None, prefs=None, read_filter=False):
cmd = [
cmd_tshark,
"-n", # No name resolution
@ -20,10 +20,17 @@ def dfilter_cmd(cmd_tshark, capture_file, request):
"-2", # two-pass mode
"--selected-frame={}".format(frame_number)
])
cmd.extend([
"-Y", # packet display filter (used to be -R)
dfilter
])
if read_filter:
cmd.extend([
"-2", # two-pass mode
"-R", # read filter (requires two-pass mode)
dfilter
])
else:
cmd.extend([
"-Y", # packet display filter (used to be -R)
dfilter
])
if prefs:
cmd.extend([
"-o",
@ -67,6 +74,20 @@ def checkDFilterCountWithSelectedFrame(dfilter_cmd, base_env):
assert dfp_count == expected_count, msg
return checkDFilterCount_real
@pytest.fixture
def checkDFilterCountReadFilter(dfilter_cmd, base_env):
def checkDFilterCount_real(dfilter, expected_count):
"""Run a read filter in two pass mode and expect a certain number of packets."""
output = subprocess.check_output(dfilter_cmd(dfilter, read_filter=True),
universal_newlines=True,
stderr=subprocess.STDOUT,
env=base_env)
dfp_count = output.count("\n")
msg = "Expected %d, got: %s\noutput: %r" % \
(expected_count, dfp_count, output)
assert dfp_count == expected_count, msg
return checkDFilterCount_real
@pytest.fixture
def checkDFilterFail(cmd_dftest, base_env):

View File

@ -0,0 +1,54 @@
# Copyright (c) 2013 by Gilbert Ramirez <gram@alumni.rice.edu>
#
# SPDX-License-Identifier: GPL-2.0-or-later
import pytest
from suite_dfilter.dfiltertest import *
class TestDfilterColumns:
trace_file = "http.pcap"
def test_exists_1(self, checkDFilterCount):
dfilter = "_ws.col.info"
checkDFilterCount(dfilter, 1)
def test_exists_2(self, checkDFilterFail):
# Column not in the default configuration
dfilter = "_ws.col.expert"
error = f'"{dfilter}" is not a valid protocol or protocol field'
checkDFilterFail(dfilter, error)
def test_exists_3(self, checkDFilterFail):
# Column not registered as field (it behaves unusally if filtered)
dfilter = "_ws.col.delta_time_dis"
error = f'"{dfilter}" is not a valid protocol or protocol field'
checkDFilterFail(dfilter, error)
def test_func_1(self, checkDFilterCount):
dfilter = "len(_ws.col.protocol) == 4"
checkDFilterCount(dfilter, 1)
def test_matches_1(self, checkDFilterSucceed):
dfilter = '_ws.col.info matches "^HEAD"'
checkDFilterSucceed(dfilter)
def test_equal_1(self, checkDFilterCount):
dfilter = '_ws.col.protocol == "HTTP"'
checkDFilterCount(dfilter, 1)
def test_equal_2(self, checkDFilterCount):
dfilter = '_ws.col.def_dst == "207.46.134.94"'
checkDFilterCount(dfilter, 1)
def test_not_equal_1(self, checkDFilterCount):
dfilter = '_ws.col.def_src != "10.0.0.5"'
checkDFilterCount(dfilter, 0)
def test_read_filter(self, checkDFilterCountReadFilter):
dfilter = '_ws.col.protocol == "HTTP"'
checkDFilterCountReadFilter(dfilter, 1)
def test_add_column(self, checkDFilterCount):
# Add column to configuration
dfilter = '_ws.col.expert == "Chat"'
checkDFilterCount(dfilter, 1, 'gui.column.format:"Expert","%a"')

View File

@ -628,7 +628,7 @@ class TestDissectTcp:
'-r', capture_file('http-ooo2.pcap'),
'-otcp.reassemble_out_of_order:TRUE',
'-Tfields',
'-eframe.number', '-etcp.reassembled_in', '-e_ws.col.Info',
'-eframe.number', '-etcp.reassembled_in', '-e_ws.col.info',
'-2',
), encoding='utf-8', env=test_env)
lines = stdout.split('\n')

View File

@ -116,7 +116,7 @@ static const char *separator = "";
static gboolean process_file(capture_file *, int, gint64);
static gboolean process_packet_single_pass(capture_file *cf,
epan_dissect_t *edt, gint64 offset, wtap_rec *rec,
const guchar *pd, guint tap_flags);
const guchar *pd);
static void show_print_file_io_error(int err);
static gboolean write_preamble(capture_file *cf);
static gboolean print_packet(capture_file *cf, epan_dissect_t *edt);
@ -182,7 +182,7 @@ print_usage(FILE *output)
fprintf(output, " -T pdml|ps|psml|text|fields\n");
fprintf(output, " format of text output (def: text)\n");
fprintf(output, " -e <field> field to print if -Tfields selected (e.g. tcp.port,\n");
fprintf(output, " _ws.col.Info)\n");
fprintf(output, " _ws.col.info)\n");
fprintf(output, " this option can be repeated to print multiple fields\n");
fprintf(output, " -E<fieldsoption>=<value> set options for output when -Tfields selected:\n");
fprintf(output, " header=y|n switch headers on and off\n");
@ -1091,7 +1091,7 @@ process_packet_first_pass(capture_file *cf, epan_dissect_t *edt,
static gboolean
process_packet_second_pass(capture_file *cf, epan_dissect_t *edt,
frame_data *fdata, wtap_rec *rec,
Buffer *buf, guint tap_flags)
Buffer *buf)
{
column_info *cinfo;
gboolean passed;
@ -1123,7 +1123,7 @@ process_packet_second_pass(capture_file *cf, epan_dissect_t *edt,
2) we're printing packet info but we're *not* verbose; in verbose
mode, we print the protocol tree, not the protocol summary.
*/
if ((tap_flags & TL_REQUIRES_COLUMNS) || (print_packet_info && print_summary))
if ((tap_listeners_require_columns()) || (print_packet_info && print_summary))
cinfo = &cf->cinfo;
else
cinfo = NULL;
@ -1367,8 +1367,7 @@ process_file(capture_file *cf, int max_packet_count, gint64 max_byte_count)
process_packet_second_pass(cf, edt, fdata, &cf->rec, &buf, tap_flags);
}
#else
if (!process_packet_second_pass(cf, edt, fdata, &cf->rec, &buf,
tap_flags))
if (!process_packet_second_pass(cf, edt, fdata, &cf->rec, &buf))
return FALSE;
#endif
}
@ -1424,7 +1423,7 @@ process_file(capture_file *cf, int max_packet_count, gint64 max_byte_count)
if (!process_packet_single_pass(cf, edt, data_offset,
&file_rec/*wtap_get_rec(cf->provider.wth)*/,
raw_data, tap_flags))
raw_data))
return FALSE;
/* Stop reading if we have the maximum number of packets;
@ -1531,8 +1530,7 @@ out:
static gboolean
process_packet_single_pass(capture_file *cf, epan_dissect_t *edt, gint64 offset,
wtap_rec *rec, const guchar *pd,
guint tap_flags)
wtap_rec *rec, const guchar *pd)
{
frame_data fdata;
column_info *cinfo;
@ -1566,7 +1564,7 @@ process_packet_single_pass(capture_file *cf, epan_dissect_t *edt, gint64 offset,
mode, we print the protocol tree, not the protocol summary.
or
3) there is a column mapped as an individual field */
if ((tap_flags & TL_REQUIRES_COLUMNS) || (print_packet_info && print_summary) || output_fields_has_cols(output_fields))
if ((tap_listeners_require_columns()) || (print_packet_info && print_summary) || output_fields_has_cols(output_fields))
cinfo = &cf->cinfo;
else
cinfo = NULL;

View File

@ -453,7 +453,7 @@ print_usage(FILE *output)
fprintf(output, " -J <protocolfilter> top level protocol filter if -T ek|pdml|json selected\n");
fprintf(output, " (e.g. \"http tcp\", filter which expands all child nodes)\n");
fprintf(output, " -e <field> field to print if -Tfields selected (e.g. tcp.port,\n");
fprintf(output, " _ws.col.Info)\n");
fprintf(output, " _ws.col.info)\n");
fprintf(output, " this option can be repeated to print multiple fields\n");
fprintf(output, " -E<fieldsoption>=<value> set options for output when -Tfields selected:\n");
fprintf(output, " bom=y|n print a UTF-8 BOM\n");
@ -3096,6 +3096,8 @@ process_packet_first_pass(capture_file *cf, epan_dissect_t *edt,
from the dissection or running taps on the packet; if we're doing
any of that, we'll do it in the second pass.) */
if (edt) {
column_info *cinfo = NULL;
/* If we're running a read filter, prime the epan_dissect_t with that
filter. */
if (cf->rfcode)
@ -3115,9 +3117,14 @@ process_packet_first_pass(capture_file *cf, epan_dissect_t *edt,
cf->provider.ref = &ref_frame;
}
/* If we're applying a filter that needs the columns, construct them. */
if (dfilter_requires_columns(cf->rfcode) || dfilter_requires_columns(cf->dfcode)) {
cinfo = &cf->cinfo;
}
epan_dissect_run(edt, cf->cd_t, rec,
frame_tvbuff_new_buffer(&cf->provider, &fdlocal, buf),
&fdlocal, NULL);
&fdlocal, cinfo);
/* Run the read filter if we have one. */
if (cf->rfcode)
@ -3303,7 +3310,7 @@ process_cap_file_first_pass(capture_file *cf, int max_packet_count,
static gboolean
process_packet_second_pass(capture_file *cf, epan_dissect_t *edt,
frame_data *fdata, wtap_rec *rec,
Buffer *buf, guint tap_flags)
Buffer *buf, guint tap_flags _U_)
{
column_info *cinfo;
gboolean passed;
@ -3328,12 +3335,14 @@ process_packet_second_pass(capture_file *cf, epan_dissect_t *edt,
col_custom_prime_edt(edt, &cf->cinfo);
/* We only need the columns if either
1) some tap needs the columns
1) some tap or filter needs the columns
or
2) we're printing packet info but we're *not* verbose; in verbose
mode, we print the protocol tree, not the protocol summary.
or
3) there is a column mapped to an individual field
*/
if ((tap_flags & TL_REQUIRES_COLUMNS) || (print_packet_info && print_summary) || output_fields_has_cols(output_fields))
if ((tap_listeners_require_columns()) || (print_packet_info && print_summary) || output_fields_has_cols(output_fields) || dfilter_requires_columns(cf->dfcode))
cinfo = &cf->cinfo;
else
cinfo = NULL;
@ -3959,7 +3968,7 @@ out:
static gboolean
process_packet_single_pass(capture_file *cf, epan_dissect_t *edt, gint64 offset,
wtap_rec *rec, Buffer *buf, guint tap_flags)
wtap_rec *rec, Buffer *buf, guint tap_flags _U_)
{
frame_data fdata;
column_info *cinfo;
@ -3994,13 +4003,13 @@ process_packet_single_pass(capture_file *cf, epan_dissect_t *edt, gint64 offset,
col_custom_prime_edt(edt, &cf->cinfo);
/* We only need the columns if either
1) some tap needs the columns
1) some tap or filter needs the columns
or
2) we're printing packet info but we're *not* verbose; in verbose
mode, we print the protocol tree, not the protocol summary.
or
3) there is a column mapped as an individual field */
if ((tap_flags & TL_REQUIRES_COLUMNS) || (print_packet_info && print_summary) || output_fields_has_cols(output_fields))
if ((tap_listeners_require_columns()) || (print_packet_info && print_summary) || output_fields_has_cols(output_fields) || dfilter_requires_columns(cf->dfcode))
cinfo = &cf->cinfo;
else
cinfo = NULL;

View File

@ -1106,6 +1106,8 @@ frame_data *PacketList::getFDataForRow(int row) const
void PacketList::columnsChanged()
{
columns_changed_ = true;
column_register_fields();
mainApp->emitAppSignal(MainApplication::FieldsChanged);
if (!cap_file_) {
// Keep columns_changed_ = true until we load a capture file.
return;