2017-04-23 13:04:19 +00:00
|
|
|
== Test API
|
|
|
|
|
2020-06-04 17:45:13 +00:00
|
|
|
All tests run by {app-name} are python script files. On top of usual python
|
|
|
|
standard features, {app-name} provides a set of public APIs and tools that
|
|
|
|
these tests can use in order to interact with the core of {app-name}, like
|
|
|
|
creating object classes, run processes synchronously or asynchronously, wait for
|
|
|
|
events, retrieve specific configuration, etc. This section aims at documenting
|
|
|
|
the most relevant tools available for tests.
|
|
|
|
|
|
|
|
First of all, it is important to avoid blocking out of the core's main event loop in
|
|
|
|
the test, since doing that will prevent {app-name} core functionalities to work
|
|
|
|
properly, such as control of asynchronous processes.
|
|
|
|
|
|
|
|
To get access to those functionalities, a test must import a test environment
|
|
|
|
previously prepared by {app-name} before the test was started:
|
|
|
|
[source,python]
|
|
|
|
----
|
|
|
|
#!/usr/bin/env python3
|
|
|
|
from osmo_gsm_tester.testenv import *
|
|
|
|
----
|
|
|
|
|
|
|
|
After the test environment is imported, aome usual functionalities are available
|
|
|
|
directly under the global scope. Specially noticeable is the existence of object
|
|
|
|
_tenv_, which provides access to most of the functionalities.
|
|
|
|
|
|
|
|
The test can simply ask {app-name} to sleep some time, while giving control back
|
|
|
|
to {app-name} core's mainloop:
|
|
|
|
[source,python]
|
|
|
|
----
|
|
|
|
sleep(3) # sleep for 3 seconds
|
|
|
|
----
|
|
|
|
|
|
|
|
One can also wait for events in the background, for instance launch a child
|
|
|
|
process locally in the same host and wait for its termination:
|
|
|
|
[source,python]
|
|
|
|
----
|
2020-06-16 09:39:19 +00:00
|
|
|
proc = process.Process('process_description_name', working_dir_to_store_logs, 'sleep 4') # <1>
|
|
|
|
tenv.remember_to_stop(proc) # <2>
|
|
|
|
proc.launch() # <3>
|
|
|
|
proc.wait() # <4>
|
2020-06-04 17:45:13 +00:00
|
|
|
----
|
|
|
|
<1> Create process object. This line doesn't yet runs it.
|
|
|
|
<2> Make sure the core will kill the process if this test fails
|
|
|
|
<3> Start process asynchronously
|
|
|
|
<4> wait until process is done. One could waiting generically here too: _wait(proc.terminated)_
|
|
|
|
|
|
|
|
If running asynchronously is not needed, one can run synchronously in an easy
|
|
|
|
way:
|
|
|
|
[source,python]
|
|
|
|
----
|
|
|
|
proc = process.Process('process_description_name', working_dir_to_store_logs, 'sleep 4')
|
|
|
|
proc.launch_sync()
|
|
|
|
----
|
|
|
|
|
|
|
|
One can also log output using either the regular _print_ function from python,
|
|
|
|
or using {app-name} specific functions available:
|
|
|
|
[source,python]
|
|
|
|
----
|
|
|
|
log('this is a regular log message')
|
|
|
|
dbg('this is a dbg message, only printed on outputs where dbg is enabled')
|
|
|
|
err('outputs log message for non-expected events')
|
|
|
|
print('this is the same as log()')
|
|
|
|
----
|
|
|
|
|
|
|
|
The test also gains access to suite and/or test specific configuration through
|
|
|
|
different APIs:
|
|
|
|
[source,python]
|
|
|
|
----
|
|
|
|
test_config = tenv.config_test_specific()
|
|
|
|
threshold = int(test_config.get('threshold', 2))
|
|
|
|
suite_config = tenv.config_suite_specific()
|
|
|
|
foobar = suite_config['foobar']
|
|
|
|
----
|
|
|
|
|
|
|
|
A test requiring a really specific config file for an object class it is going
|
|
|
|
to run can provide its own template files by overlaying an own directory
|
|
|
|
containing them on top of the usual default directory where object class
|
|
|
|
templates are (_osmo-gsm-tester.git/src/osmo_gsm_tester/obj/templates/_):
|
|
|
|
[source,python]
|
|
|
|
----
|
|
|
|
tenv.set_overlay_template_dir(os.path.join(os.path.dirname(__file__), 'mytemplatedir'))
|
|
|
|
----
|
|
|
|
|
|
|
|
Several tests in a suite can also share code by using some APIs provided by
|
|
|
|
{app-names}. The shared python code must be placed in files under the 'lib/'
|
|
|
|
subdirectory in the suite directory where the test belongs to.
|
|
|
|
[source,python]
|
|
|
|
----
|
|
|
|
# File containing function foobar() available under ${suite_dir}/lib/testlib.py:
|
|
|
|
import testlib
|
|
|
|
tenv.test_import_modules_register_for_cleanup(testlib)
|
|
|
|
from testlib import foobar
|
|
|
|
----
|
|
|
|
|
|
|
|
For a complete set of features and how to use them, one can have a look at real
|
|
|
|
examples present in {app-name} git repository under the _sysmocom/_ directory.
|
|
|
|
Besides those, have a look too a _testenv.py_ file, which implements the 'tenv'
|
|
|
|
object available to tests.
|
|
|
|
|
|
|
|
=== Test verdict
|
|
|
|
|
|
|
|
In general, a test reaching the end of the file and returning control to
|
|
|
|
{app-name} core will be flagged as a successful test (PASS).
|
|
|
|
|
|
|
|
If an exception is thrown from within the test file and propagated to
|
|
|
|
{app-name}, the test will be considered as failed and {app-name} will store all
|
|
|
|
failure related information from the caught exception.
|