The output of the "values" tshark glossary has over 1.3M lines. Writing
this to stdout with some test failures is problematic in a number of ways.
Also it's not helpful because stderr is written after stdout (not interleaved)
so there is no output context to the error message. The error/warning
message (from stderr, that triggered the test failure) needs to be
sufficient to provide a good understaning of the test failure.
The output is trimmed to first+last N lines. Some lines are kept as
informational and because it may be useful if the program aborts.
Fixes#17203.
PEP 394[1] says,
"In cases where the script is expected to be executed outside virtual
environments, developers will need to be aware of the following
discrepancies across platforms and installation methods:
* Older Linux distributions will provide a python command that refers
to Python 2, and will likely not provide a python2 command.
* Some newer Linux distributions will provide a python command that
refers to Python 3.
* Some Linux distributions will not provide a python command at all by
default, but will provide a python3 command by default."
Debian has forced the issue by choosing the third option[2]:
"NOTE: Debian testing (bullseye) has removed the "python" package and
the '/usr/bin/python' symlink due to the deprecation of Python 2."
Switch our shebang from "#!/usr/bin/env python" to "#!/usr/bin/env
python3" in some places. Remove some 2/3 version checks if we know we're
running under Python 3. Remove the "coding: utf-8" in a bunch of places
since that's the default in Python 3.
[1]https://www.python.org/dev/peps/pep-0394/#for-python-script-publishers
[2]https://wiki.debian.org/Python
Fix some issues discovered by common python linters including:
* switch `None` comparisons to use `is` rather than `==`. Identity !=
equality, and I've spent 40+ hours before tracking down a subtle bug
caused by exactly this issue. Note that this may introduce a problem if
one of the scripts is depending on this behavior, in which case the
comparison should be changed to `True`/`False` rather than `None`.
* Use `except Exception:` as bare `except:` statements have been
discouraged for years. Ideally for some of these we'd examine if there
were specific exceptions that should be caught, but for now I simply
caught all. Again, this could introduce very subtle behavioral changes
under Python 2, but IIUC, that was all fixed in Python 3, so safe to
move to `except Exception:`.
* Use more idiomatic `if not x in y`--> `if x not in y`
* Use more idiomatic 2 blank lines. I only did this at the beginning,
until I realized how overwhelming this was going to be to apply, then I
stopped.
* Add a TODO where an undefined function name is called, so will fail
whenever that code is run.
* Add more idiomatic spacing around `:`. This is also only partially
cleaned up, as I gave up when I saw how `asn2wrs.py` was clearly
infatuated with the construct.
* Various other small cleanups, removed some trailing whitespace and
improper indentation that wasn't a multiple of 4, etc.
There is still _much_ to do, but I haven't been heavily involved with
this project before, so thought this was a sufficient amount to put up
and see what the feedback is.
Linters that I have enabled which highlighted some of these issues
include:
* `pylint`
* `flake8`
* `pycodestyle`
The run method is not invoked when running in debug mode (for example,
with `pytest --pdb`) and would result in spurious errors due to "log_fd"
being None. Split the method to resolve this incompatibility.
Note that with `pytest --pdb`, the tearDown method is not called when
exceptions occur, see https://docs.pytest.org/en/latest/unittest.html
Change-Id: I4b66c03d5b050b53311ec64021fe17dc91bb48dd
Reviewed-on: https://code.wireshark.org/review/31339
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
If we have a single capture source and that capture source is pcapng and
we're writing a pcapng file, do the following:
- Pass its SHB and IDBs through unmodified. Don't save or write command
line interface IDBs.
- Save the most recent SHB and IDBs so that we can write them when we're
writing multiple output files.
If we have multiple capture sources, do the following:
- Write Dumpcap's SHB.
- Keep a global list of IDBs, consisting of both command line interfaces
and IDBs read from pcapng sources.
- When reading an EPB or ISB, remap its local interface number to its
corresponding global number.
Add Dumpcap pcapng section tests. Make the application IDs in the
"many_interfaces" captures unique.
Change-Id: I2005934c1f83d839727421960005f106d6c682dd
Reviewed-on: https://code.wireshark.org/review/30085
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Convert the old start_pinging routine to use pytest fixtures, rewriting
it to enable a different generator that uses (for example) UDP.
Remove the config module since it is no longer neded.
Change-Id: Ic4727157faab084b41144e8f16ea44f59c9037d8
Reviewed-on: https://code.wireshark.org/review/30659
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Add a new --capture-interface option to pytest, similar to test.py. It
will grab some Ethernet interface on Windows. An empty value overrides
this and disables capture tests. Remove the test.py --enable-capture
option since that is implied by the --capture-interface option.
Port the `test.py --program-path` option to pytest and additionally make
the pytest look in the current working directory if neither WS_BIN_PATH
nor --program-path are specified. Drop config.setProgramPath, this
allows tests to be run even if not all binaries are available.
With all capture tests converted to fixtures, it is now possible to run
tests when Wireshark is not built with libpcap as tests that depend on
cmd_dumpcap (or capture_interface) will be skipped.
Bug: 14949
Change-Id: Ie802c07904936de4cd30a4c68b6a5139e6680fbd
Reviewed-on: https://code.wireshark.org/review/30656
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Currently all binaries must be available or no tests will be executed.
This is inconvenient if you just want to test a single binary (e.g.
text2pcap) without having to build epan. The problem is essentially that
tests lack dependency annotations.
To solve this problem, add the required dependencies as parameters to
each test (so-called 'fixtures' in pytest). Skip a test if a binary
(such as tshark) is unavailable. As a demonstration, suite_dissection.py
is converted. Over time, tests should no longer depend on config.py due
to explicit dependencies fixtures (listed in fixtures_ws.py).
Since the unittest module does not support such dependency injections,
create a small glue for use with pytest and an (incomplete) emulation
layer for use with test.py.
Tested with pytest 3.8.2 + Python 3.7.0 and pytest 3.0.3 + Python 3.4.3.
Python 2.7 is not supported and will fail. Test commands:
~/wireshark/test/test.py -p ~/build/run
WS_BIN_PATH=~/build/run pytest ~/wireshark/test -ra
Change-Id: I6dc8c28f5c8b7bbc8f4c04838e9bf085cd22eb0b
Ping-Bug: 14949
Reviewed-on: https://code.wireshark.org/review/30220
Tested-by: Petri Dish Buildbot
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Reduces maintenance costs and makes it possible to simplify code.
pytest supports Python 2.7 and Python 3.4 (or newer), so that is more or
less the minimum target for now.
Change-Id: I0347b6c334bf2fc6c9480ff56e9ccfcd48886dde
Reviewed-on: https://code.wireshark.org/review/30193
Tested-by: Petri Dish Buildbot
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
pytest is a powerful test framework. The initial motivation is to have
much better detailed reporting when a test fails, see for example
https://docs.pytest.org/en/latest/
Additionally, it has full parallelization support at the test level (via
the pytest-xdist plugin) instead of being limited to the suite level
(which currently has to be hard-coded via CMakeLists.txt).
Usage with the build dir in /tmp/wsbuild and src dir in /tmp/wireshark:
export WS_BIN_PATH=/tmp/wsbuild/run
pytest /tmp/wireshark/tests
For parallelization support and verbose printing:
pip install pytest-xdist
pytest -nauto -v /tmp/wireshark/tests
To limit yourself to a case based on a pattern:
pytest -nauto -v /tmp/wireshark/tests -k test_unit_ctest_coverage
Tested on Arch Linux with Python 3.6.5, pytest-3.6.2, xdist-1.22.2.
pytest -n8 finished in 82 seconds while ctest -j8 required 87 seconds.
Change-Id: I832f4dd9f988d6656df795327e81610accf54b9f
Reviewed-on: https://code.wireshark.org/review/28651
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Run our CTest tests with PYTHONIOENCODING=UTF-8.
If someone runs our tests manually and their output encoding isn't
UTF-8, print replacement characters instead of failing with an error.
Open our log files with "errors='backslashreplace'" in case their
contents aren't UTF-8.
Change-Id: Ifa4d12c2b5e272cf3903f3e0c6102e4d961562f1
Reviewed-on: https://code.wireshark.org/review/27686
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Convert capinfos output to UTF-8 in getCaptureInfo.
Normalize our command paths, otherwise "./run/RelWithDebInfo/..." might
be interpreted as the command "." with flags "/run", "/RelWithDebInfo",
etc. on Windows.
Change-Id: Ib7336a016db3ee0805739fc44913cb9c6895aaad
Reviewed-on: https://code.wireshark.org/review/27239
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Add the fileformats and I/O suites. Move some more common code to
subprocesstest.py and add a diffOutput method.
Change-Id: I2ec34e46539022bdce78520645fdca6dfc1a8c1a
Reviewed-on: https://code.wireshark.org/review/27183
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Create Python versions of our various test shell scripts. Add CMake
tests for each suite. Tests can now be run directly via test.py, via the
"test" target, or via ctest, e.g.
ctest --verbose --jobs 3
Add a testing chapter to the Developer's Guide.
Add a way to disable ctest in dpkg-buildpackage.
Suites completed:
- capture
- clopts
- decryption
- dissection
Remaining suites:
- fileformats
- io
- mergecap
- nameres
- text2pcap
- unittests
- wslua
Change-Id: I8936e05edefc76a86b6a7a5da302e7461bbdda0f
Reviewed-on: https://code.wireshark.org/review/27134
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Gerald Combs <gerald@wireshark.org>
João Valverde