The case_tshark_name_resolution_clopts test doesn't need live capture,
so switch to a capture file. This should fix the current failure on the
macOS Arm builder.
The output of the "values" tshark glossary has over 1.3M lines. Writing
this to stdout with some test failures is problematic in a number of ways.
Also it's not helpful because stderr is written after stdout (not interleaved)
so there is no output context to the error message. The error/warning
message (from stderr, that triggered the test failure) needs to be
sufficient to provide a good understaning of the test failure.
The output is trimmed to first+last N lines. Some lines are kept as
informational and because it may be useful if the program aborts.
Fixes#17203.
PEP 394[1] says,
"In cases where the script is expected to be executed outside virtual
environments, developers will need to be aware of the following
discrepancies across platforms and installation methods:
* Older Linux distributions will provide a python command that refers
to Python 2, and will likely not provide a python2 command.
* Some newer Linux distributions will provide a python command that
refers to Python 3.
* Some Linux distributions will not provide a python command at all by
default, but will provide a python3 command by default."
Debian has forced the issue by choosing the third option[2]:
"NOTE: Debian testing (bullseye) has removed the "python" package and
the '/usr/bin/python' symlink due to the deprecation of Python 2."
Switch our shebang from "#!/usr/bin/env python" to "#!/usr/bin/env
python3" in some places. Remove some 2/3 version checks if we know we're
running under Python 3. Remove the "coding: utf-8" in a bunch of places
since that's the default in Python 3.
[1]https://www.python.org/dev/peps/pep-0394/#for-python-script-publishers
[2]https://wiki.debian.org/Python
Fix some issues discovered by common python linters including:
* switch `None` comparisons to use `is` rather than `==`. Identity !=
equality, and I've spent 40+ hours before tracking down a subtle bug
caused by exactly this issue. Note that this may introduce a problem if
one of the scripts is depending on this behavior, in which case the
comparison should be changed to `True`/`False` rather than `None`.
* Use `except Exception:` as bare `except:` statements have been
discouraged for years. Ideally for some of these we'd examine if there
were specific exceptions that should be caught, but for now I simply
caught all. Again, this could introduce very subtle behavioral changes
under Python 2, but IIUC, that was all fixed in Python 3, so safe to
move to `except Exception:`.
* Use more idiomatic `if not x in y`--> `if x not in y`
* Use more idiomatic 2 blank lines. I only did this at the beginning,
until I realized how overwhelming this was going to be to apply, then I
stopped.
* Add a TODO where an undefined function name is called, so will fail
whenever that code is run.
* Add more idiomatic spacing around `:`. This is also only partially
cleaned up, as I gave up when I saw how `asn2wrs.py` was clearly
infatuated with the construct.
* Various other small cleanups, removed some trailing whitespace and
improper indentation that wasn't a multiple of 4, etc.
There is still _much_ to do, but I haven't been heavily involved with
this project before, so thought this was a sufficient amount to put up
and see what the feedback is.
Linters that I have enabled which highlighted some of these issues
include:
* `pylint`
* `flake8`
* `pycodestyle`
The outputs of -T ek and -G elastic-mapping don't match. To be effective
the fields in the mapping report and the fields in the traffic output must
be the same.
2 issues have been fixed. The elastic-mapping requires the parent protocol
to be prepended to the field to match the traffic output. The field "dns.a"
has been changed to "dns_dns_a".
The traffic output prints some fields with a leading "text_". This happens
for some fields that have been created under a text only field. One example
is "dns.a", that was printed as "text_dns_a". This has been fixed by accessing
the parent hfinfo resulting in "dns_dns_a" as other fields for the dns
protocol.
Bug: 15759
Change-Id: Ibd000c865102ca49bb6a6394019a475483eae4cc
Reviewed-on: https://code.wireshark.org/review/33099
Petri-Dish: Dario Lombardo <lomato@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Eneko Gómez <eneko.gomez.tecnalia@gmail.com>
Reviewed-by: Dario Lombardo <lomato@gmail.com>
Newer versions of elastic are using 'doc' as type. Change the code
according to that.
Fix point (4) of the linked bug.
Bug: 15763
Change-Id: Ia28102a0914c6308eb3516daa57af2e49ce9a4e5
Reviewed-on: https://code.wireshark.org/review/33111
Petri-Dish: Dario Lombardo <lomato@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Eneko Gómez <eneko.gomez.tecnalia@gmail.com>
Reviewed-by: Dario Lombardo <lomato@gmail.com>
The WIRESHARK_EXTCAP_DIR environment variable is currently only used on
Windows, and on UN*X when not running from the build directory. In order
to avoid copying the sampleif.py test utility to the program directory,
let's prioritize the environment variable over the build directory.
Update the outdated comments while at it, the version directory has been
removed long time ago. (The comments are based on the one for plugins.)
This also fixes the test suite on macOS where the extcap subdirectory is
located in the appbundle directory and not the build directory.
Change-Id: I329bb233b1dd0b9c1422c2ebd60a6455347e1d62
Reviewed-on: https://code.wireshark.org/review/32890
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Dario Lombardo <lomato@gmail.com>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Check for utf8 valid line instead. Add a testcase that shows
how the former code was buggy on special characters extcap sentences.
Bug: 15668
Change-Id: Ic045c4791388af98705916e6ea84be8fc9b3c5b8
Reviewed-on: https://code.wireshark.org/review/32754
Petri-Dish: Dario Lombardo <lomato@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Check for potential Unicode-related problems on Windows.
Change-Id: I147c07749c5073a9ae00f07914dd80347d17c40f
Ping-Bug: 15118
Reviewed-on: https://code.wireshark.org/review/31154
Tested-by: Petri Dish Buildbot
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
The case_decrypt_tls.test_tls_rsa_pq test is unexpectedly passing when
GnuTLS is disabled. It checks for '/' in the output, but that also
matches an error message. Use assertRun here and pretty much everywhere
else to catch such issues. Remove a few redundant returncode checks.
Change-Id: I0f9d1dadc0ca73eef9cffb3e2f452aa7c8395c95
Reviewed-on: https://code.wireshark.org/review/30838
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
The (optional) JSON-GLib library adds dependencies on GObject, GIO. For
statically linked oss-fuzz builds it also adds libffi and more. To avoid
these dependencies, replace JSON-GLib by some custom code. This allows
`tshark -G elastic-mapping` to be enabled by default without extra deps.
API design goals of the new JSON dumper library:
- Small interface without a lot of abstraction.
- Avoid memory allocations if possible (currently none, but maybe
json_puts_string will be replaced to improve UTF-8 support).
- Do not implement parsing, this is currently handled by jsmn.
Methods to open/close array/objects and to set members are inspired by
the JsonGlib interface. The interfaces to write values is inspired by
the sharkd code (json_puts_string is also borrowed from that).
The only observed differences in the tshark output:
- JSON-GLib ignores duplicates, json_dumper does not and may produce
duplicates and currently print two "ip.opt.sec_prot_auth_unassigned".
- JSON-GLib adds a space before a colon (unimportant formatting detail).
- (Not observed, but UTF-8 strings will be wrong like bug 14948.)
A test was added to catch changes in the tshark output. I also fuzzed
json_dumper with libFuzzer + UBSAN/ASAN and fixed an off-by-one error.
Change-Id: I0c85b18777b04d1e0f613a3d59935ec59be87ff4
Link: https://www.wireshark.org/lists/wireshark-dev/201811/msg00052.html
Reviewed-on: https://code.wireshark.org/review/30732
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Add a new --capture-interface option to pytest, similar to test.py. It
will grab some Ethernet interface on Windows. An empty value overrides
this and disables capture tests. Remove the test.py --enable-capture
option since that is implied by the --capture-interface option.
Port the `test.py --program-path` option to pytest and additionally make
the pytest look in the current working directory if neither WS_BIN_PATH
nor --program-path are specified. Drop config.setProgramPath, this
allows tests to be run even if not all binaries are available.
With all capture tests converted to fixtures, it is now possible to run
tests when Wireshark is not built with libpcap as tests that depend on
cmd_dumpcap (or capture_interface) will be skipped.
Bug: 14949
Change-Id: Ie802c07904936de4cd30a4c68b6a5139e6680fbd
Reviewed-on: https://code.wireshark.org/review/30656
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Continue the conversion from use of globals (the config module) to
fixtures. If a program (like wmem_test or tshark) is unavailable, it
will be skipped now rather than failing the test.
The general conversion pattern is:
- Decorate each class with `@fixtures.uses_fixtures` and (for tests that
run tshark) `@fixtures.mark_usefixtures('test_env')`.
- Convert all `config.cmd_*` to `cmd_*` and add an argument.
- Convert all `config.*_dir` to `dirs.*_dir` and add an argument.
- Convert users of `os.path.join(dirs.capture_file, ...)` to use a new
'capture_file' fixture to reduce boilerplate code. Inline variables if
possible (this conversion was done in an automated way using regexes).
Some other changes: tests that do not require a test environment (like
wmem_test) will use 'base_env' which avoids copying config files,
`env=config.test_env` got removed since this is the default. Some test
classes in suite_clopts were combined. Removed unused imports.
Change-Id: Id5480ffaee7d8d56cf2cb3189a38ae9afa7605a1
Reviewed-on: https://code.wireshark.org/review/30591
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Previously 'tshark -z expert' was failing with abort when a packet
contains a comment
- Add a new comment parameter and update the tshark's manual page
- Add a new comment_level severity and change the default lavel to it.
- Add various 'tshark -z expert' tests
Change-Id: I188317da5e00019b8f2b725f0fe84942f774520f
Reviewed-on: https://code.wireshark.org/review/30610
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Some tests used the default home directory which can have side-effects
(such as loading plugins, loading deprecated preferences). These could
cause tests to fail. Always use a sane environment to fix this.
Change getTsharkInfo to use this clean environment as well
(WIRESHARK_CONFIG_DIR does not exist with master-2.6 and would also not
propagate things like ASAN_OPTIONS=detect_leaks=0).
Change-Id: I1674f71972d35de91d191e0c29fdb59b8a0a56ce
Reviewed-on: https://code.wireshark.org/review/30165
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Add 'v' option for VLAN ID resolving and get rid of
deprecated 'C' option.
Bug: 14826
Change-Id: I63104f4a465d251048693ad02882ea7eb2c4d926
Reviewed-on: https://code.wireshark.org/review/30029
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Move UAT file creation to config.py.
Run the text2pcap and some of the clopts tests under our default
environment.
Use "in" instead of "has_key".
Change-Id: Ie5c70fb33c29676672bed7bf8205cff0bba77f8a
Reviewed-on: https://code.wireshark.org/review/27234
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Add the fileformats and I/O suites. Move some more common code to
subprocesstest.py and add a diffOutput method.
Change-Id: I2ec34e46539022bdce78520645fdca6dfc1a8c1a
Reviewed-on: https://code.wireshark.org/review/27183
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
In util_slow_dhcp.py, open stdout as O_BINARY on Windows.
Have ctest pass --verbose to test.py.
Call config.canCapture at test time so that we don't inadvertently skip
some tests.
Stringify our dumpcap config check.
Fix our Gcrypt variable.
Change-Id: I884ec23ddfc7c28b79d4a860c6c43c308598e6db
Reviewed-on: https://code.wireshark.org/review/27182
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Create Python versions of our various test shell scripts. Add CMake
tests for each suite. Tests can now be run directly via test.py, via the
"test" target, or via ctest, e.g.
ctest --verbose --jobs 3
Add a testing chapter to the Developer's Guide.
Add a way to disable ctest in dpkg-buildpackage.
Suites completed:
- capture
- clopts
- decryption
- dissection
Remaining suites:
- fileformats
- io
- mergecap
- nameres
- text2pcap
- unittests
- wslua
Change-Id: I8936e05edefc76a86b6a7a5da302e7461bbdda0f
Reviewed-on: https://code.wireshark.org/review/27134
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Gerald Combs <gerald@wireshark.org>
João Valverde