WSDG: update testing section to cover use of pytest

Remove traces of the "config" module, it was removed. Add a new section
on using pytest.

Change-Id: I763fc53359157f5fcb04198ed98e2d7f7a2c7220
Reviewed-on: https://code.wireshark.org/review/31372
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
This commit is contained in:
Peter Wu 2019-01-04 00:36:41 +01:00 committed by Anders Broman
parent 0a25bfe647
commit 40b2ba8e7b
1 changed files with 70 additions and 11 deletions

View File

@ -75,8 +75,7 @@ A test has typically additional dependencies, like the path to an
executable, the path to a capture file, a configuration directory, the
availability of an optional library, and so on. The Python unittest
library is quite limited in expressing test dependencies, these are
typically specified on the class instance itself (`self`) or via globals
(like a `config` module).
typically specified on the class instance itself (`self`) or via globals.
https://pytest.org/[pytest] is a better test framework which has full
parallelization support (test-level instead of just suite-level),
@ -147,6 +146,71 @@ $ python3 test.py -l dumpcap
$ python3 test.py suite_clopts
----
=== Listing And Running Tests (pytest)
Tests can also be run with https://pytest.org/[pytest]. Advantages include finer
test selection, full parallelism, nicer test execution summaries, better output
in case of failures (containing the contents of variables) and the ability to
open the PDB debugger on failing tests.
To get started, install pytest 3.0 or newer and
https://pypi.org/project/pytest-xdist/[pytest-xdist]:
[source,sh]
----
# Install required packages on Ubuntu 18.04 or Debian jessie-backports
$ sudo apt install python3-pytest python3-pytest-xdist
# Install required packages on other systems
$ pip install pytest pytest-xdist
----
Run `pytest` in the Wireshark build directory, Wireshark binaries are assumed to
be present in the `run` subdirectory (or `run\RelWithDebInfo` on Windows).
[source,sh]
----
# Run all tests
$ cd /path/to/wireshark/build
$ pytest
# Run all tests including capture tests
$ pytest --capture-interface lo
# Run all except capture tests (which are enabled by default on Windows)
$ pytest --disable-capture
# Run all tests with "decryption" in its name
$ pytest -k decryption
# Run all tests with an explicit path to the Wireshark executables
$ pytest --program-path /path/to/wireshark/build/run
----
To list tests instead of running them, use the `--collect-only` option:
[source,sh]
----
# List all tests
$ pytest --collect-only
# List only tests containing both "dfilter" and "tvb"
$ pytest --collect-only -k "dfilter and tvb"
----
To open a Python debugger (PDB) on failing tests, use the `--pdb` option and
disable parallelism with the `-n0` option:
[source,sh]
----
# Run decryption tests sequentially and open a debugger on failing tests
$ pytest -n0 --pdb -k decryption
----
Note that with the option `--pdb`, stray processes are not killed for failing
tests since the `SubprocessTestCase.tearDown` method is not executed. This
limitation might be addressed in the future.
=== Adding Or Modifying Tests
Tests must be in a Python module whose name matches “suite_*.py”. The
@ -164,11 +228,6 @@ variables) are injected through method parameters. Commonly used
fixtures include `cmd_tshark` and `capture_file`. See also
<<ChTestsPytest>>.
The “config” module contains common configuration information which has
been derived from the current environment or specified on the command
line. This mechanism is deprecated in favor of fixtures and might be
removed in the future.
The “subprocesstest” class contains the following methods for running
processes. Stdout and stderr is written to “<test id>.log”:
@ -193,8 +252,8 @@ class case_basic_clopts(subprocesstest.SubprocessTestCase):
self.assertRun((cmd_tshark, '-r', capture_file('dhcp.pcap')))
----
Program output can be checked using “subprocesstest.grepOutput”
or “subprocesstest.countOutput”:
Program output can be checked using `SubprocessTestCase.grepOutput`,
`SubprocessTestCase.countOutput` or other `unittest.assert*` methods:
[source,python]
----
@ -204,14 +263,14 @@ import subprocesstest
@fixtures.uses_fixtures
class case_decrypt_80211(subprocesstest.SubprocessTestCase):
def test_80211_wpa_psk(self, cmd_tshark, capture_file):
self.runProcess((cmd_tshark,
tshark_proc = self.assertRun((cmd_tshark,
'-o', 'wlan.enable_decryption: TRUE',
'-Tfields',
'-e', 'http.request.uri',
'-r', capture_file('wpa-Induction.pcap.gz'),
'-Y', 'http',
))
self.assertTrue(self.grepOutput('favicon.ico'))
self.assertIn('favicon.ico', tshark_proc.stdout_str)
----
Tests can be run in parallel. This means that any files you create must