Cmdline arg -c sets main configuration file (old paths.conf) instead of dir containing it

It has been notified that current configuration system is difficult to
understand and to use, so it has been envisioned to refactor it a bit.
The idea is that the user passes a -c path/to/main.conf file, which in
turn contains whatever osmo-gsm-tester main settings supports (basically
what old paths.conf used to be, plus some files harcoded to the same -c
directory are now configurable through the main configuration file).

Change-Id: Ieca65b71b543c44cfcec8e83efd0fe053c432e55
This commit is contained in:
Pau Espin 2020-05-11 18:30:58 +02:00
parent 0433c9b671
commit 6c6c0e8599
24 changed files with 290 additions and 212 deletions

View File

@ -2,7 +2,7 @@
set -e -x
base="$PWD"
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd -P) # this file's directory
OSMO_GSM_TESTER_CONF=${OSMO_GSM_TESTER_CONF:-${SCRIPT_DIR}/../sysmocom}
OSMO_GSM_TESTER_CONF=${OSMO_GSM_TESTER_CONF:-${SCRIPT_DIR}/../sysmocom/main.conf}
time_start="$(date '+%F %T')"

View File

@ -1,13 +1,14 @@
This a sample 2G test suite configured and ready to use.
The only thing missing is a trial dir containing binaries.
You can point osmo-gsm-tester.py at this config using the '-c $DIR' command line
argument, where DIR is the directory path where this README file resides.
You can point osmo-gsm-tester.py at this config using the '-c $DIR/main.conf'
command line argument, where DIR is the directory path where this README file
resides.
If you have your trial with binary tar archives in ~/my_trial
you can run the suite for example like this:
```
osmo-gsm-tester.py -c $DIR ~/my_trial
osmo-gsm-tester.py -c $DIR/main.conf ~/my_trial
```
Alternatively you can setup this example as default config for your user by
@ -19,4 +20,4 @@ ln -s "$DIR" ~/.config/osmo-gsm-tester
A ./state dir will be created to store the current osmo-gsm-tester state. If
you prefer not to write to $DIR, set up an own configuration pointing at a
different path (see paths.conf: 'state_dir').
different path (see main.conf: 'state_dir').

View File

@ -0,0 +1,6 @@
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: './suites'
scenarios_dir: './scenarios'
default_suites_conf_path: './default-suites.conf'
defaults_conf_path: './defaults.conf'
resource_conf_path: './resources.conf'

View File

@ -1,3 +0,0 @@
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: './suites'
scenarios_dir: './scenarios'

View File

@ -1,13 +1,14 @@
This a sample 4G test suite configured and ready to use srsLTE stack.
The only thing missing is a trial dir containing binaries.
You can point osmo-gsm-tester.py at this config using the '-c $DIR' command line
argument, where DIR is the directory path where this README file resides.
You can point osmo-gsm-tester.py at this config using the '-c $DIR/main.conf'
command line argument, where DIR is the directory path where this README file
resides.
If you have your trial with binary tar archives in ~/my_trial
you can run the suite for example like this:
```
osmo-gsm-tester.py -c $DIR ~/my_trial
osmo-gsm-tester.py -c $DIR/main.conf ~/my_trial
```
Alternatively you can setup this example as default config for your user by
@ -19,4 +20,4 @@ ln -s "$DIR" ~/.config/osmo-gsm-tester
A ./state dir will be created to store the current osmo-gsm-tester state. If
you prefer not to write to $DIR, set up an own configuration pointing at a
different path (see paths.conf: 'state_dir').
different path (see main.conf: 'state_dir').

View File

@ -0,0 +1,6 @@
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: './suites'
scenarios_dir: './scenarios'
default_suites_conf_path: './default-suites.conf'
defaults_conf_path: './defaults.conf'
resource_conf_path: './resources.conf'

View File

@ -1,3 +0,0 @@
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: './suites'
scenarios_dir: './scenarios'

View File

@ -4,22 +4,29 @@
All configuration attributes in {app-name} are stored and provided as YAML
files, which are handled internally mostly as sets of dictionaries, lists and
scalars. Each of these configurations have a known format, which is called
'schema'. Each provided configuration is validated against its 'schema' at parse
time. Hence, 'schemas' can be seen as a namespace containing a structured tree
of configuration attributes. Each attribute has a schema type assigned which
constrains the type of value it can hold.
scalars. Each of these configurations have a known format (set of keys and
values), which is called 'schema'. Each provided configuration is validated
against its 'schema' at parse time. Hence, 'schemas' can be seen as a namespace
containing a structured tree of configuration attributes. Each attribute has a
schema type assigned which constrains the type of value it can hold.
There are several well-known schemas used across {app-name}, and they are
described in following sub-sections.
[[schema_main_cfg]]
==== Schema 'main config'
This schema defines all the attributes available in {app-name} the main
configuration file <<main_conf,main.conf>>, and it is used to validate it.
[[schema_resources]]
==== Schema 'resources'
This schema defines all the attributes which can be assigned to
a _resource_, and it is used to validate the <<resources_conf,resources.conf>>
file. Hence, the <<resources_conf,resources.conf>> contains a list of elements
for each resource type.
for each resource type. This schema is also used and extended by the
<<schema_want,'want' schema>>.
It is important to understand that the content in this schema refers to a list of
resources for each resource class. Since a list is ordered by definition, it
@ -34,9 +41,10 @@ On the other hand, it's also important to note that lists for simple or scalar
types are currently being treated as unordered sets, which mean combination of
filters or modifiers apply differently. In the future, it may be possible to
have both behaviors for scalar/simple types by using also the YAML 'set' type in
{app-handle}.
{app-name}.
//TODO: update this list and use a table for each resource type
//TODO: update this list and use a table for each resource type in its own object section
////
These kinds of resources and their attributes are known:
'ip_address'::
@ -127,6 +135,7 @@ These kinds of resources and their attributes are known:
- 'gprs'
- 'voice'
- 'ussd'
////
[[schema_want]]
==== Schema 'want'
@ -152,78 +161,109 @@ resources:
type: osmo-bts-sysmo
----
[[schema_conf]]
==== Schema 'conf'
[[schema_config]]
==== Schema 'config'
This schema is used by <<suite_conf,suite.conf>> and <<scenario_conf,scenario.conf>> files. It contains 3 main element sections:::
[[schema_conf_sec_resources]]
- Section 'resources': Contains a set of elements validated with <<schema_resources,resources>>
This schema defines all the attributes which can be used by object classes or
tests during test execution. The main difference between this schema and the
<<schema_resources,resources>> schema is that the former contains configuration
to be applied globally for all objects being used, while the later applies
attributes to a specific object in the list of allocated resources. This schema
hence allows setting attributes for objects which are not allocated as resources
and hence not directly accessible through scenarios, like a BSC or an iperf3
client.
This schema is built dynamically at runtime from content registered by:
- object classes registering their own attributes
- test suite registering their own attributes through <<suite_conf,suite.conf>>
and tests being able to later retrieve them through 'testenv' API.
[[schema_all]]
==== Schema 'all'
This schema is basically an aggregated namespace for <<schema_want,want>> schema
and <<schema_config,config>> schema, and is the one used by
<<suite_conf,suite.conf>> and <<scenario_conf,scenario.conf>> files. It contains
these main element sections:::
[[schema_all_sec_resources]]
- Section 'resources': Contains a set of elements validated with <<schema_want,want>>
schema. In <<suite_conf,suite.conf>> it is used to construct the list of
requested resources. In <<scenario_conf,scenario.conf>>, it is used to inject
attributes to the initial <<suite_conf,suite.conf>> _resources_ section and
hence further restrain it.
[[schema_conf_sec_modifiers]]
[[schema_all_sec_modifiers]]
- Section 'modifiers': Both in <<suite_conf,suite.conf>> and
<<scenario_conf,scenario.conf>>, values presented in here are injected into
the content of the <<schema_conf_sec_resources,resources section>> after
the content of the <<schema_all_sec_resources,resources section>> after
_resource_ allocation, hereby overwriting attributes passed to the object
class instance managing the specific _resource_ (matches by resource type and
list position). Since it is combined with the content of
<<schema_conf_sec_resources,resources section>>, it is clear that the
<<schema_resources,resources schema>> is used to validate this content.
[[schema_conf_sec_config]]
- Section 'config': Contains configuration attributes for {app-name} classes which are
not _resources_, and hence cannot be configured with <<schema_modifiers,modifiers>>.
They can overwrite values provided in the <<defaults_conf,defaults.conf>> file.
<<schema_all_sec_resources,resources section>>, it is clear that the
<<schema_want,want schema>> is used to validate this content.
[[schema_all_sec_config]]
- Section 'config': Contains configuration attributes for {app-name} object
classes which are not _resources_, and hence cannot be configured with
<<schema_all_sec_modifiers,modifiers>>. They can overwrite values provided in the
<<defaults_conf,defaults.conf>> file. Content in this section follows the
<<schema_config,config>> schema.
//TODO: defaults.timeout should be change in code to be config.test_timeout or similar
//TODO: 'config' should be split into its own schema and validate defaults.conf
[[config_paths]]
=== Config Paths
[[config]]
=== Configuration files and directories
The osmo-gsm-tester looks for configuration files in various standard
directories in this order:
Find in below sub-sections all user-defined files and directories used by
{app-name} to run tests on a given setup.
- '.' (Current Working Directory)
- '$HOME/.config/osmo-gsm-tester/'
- '/usr/local/etc/osmo-gsm-tester/'
- '/etc/osmo-gsm-tester/'
[[config_main]]
==== 'main.conf'
The config location can also be set through '-c' command line argument, which
The main configuration file is basically a placeholder for {app-name} to find
paths to all other files and directories used to operate and run tests.
{app-name} looks for the main configuration file in various standard paths in
this order:
- './main.conf' (Current Working Directory)
- '$HOME/.config/osmo-gsm-tester/main.conf'
- '/usr/local/etc/osmo-gsm-tester/main.conf'
- '/etc/osmo-gsm-tester/main.conf'
The config file location can also be set through '-c' command line argument, which
then overrides the above locations.
The osmo-gsm-tester expects to find the following configuration files in a
configuration directory:
{app-name} expects to find the following configuration settings in 'main.conf':
- <<paths_conf,paths.conf>>
- <<resource_conf,resources.conf>>
- <<default_suites_conf,default-suites.conf>> (optional)
- <<defaults_conf,defaults.conf>> (optional)
- 'state_dir': Path to <<state_dir,state_dir>> directory
- 'suites_dir': Path to <<suites_dir,suites_dir>> directory
- 'scenarios_dir': Path to <<scenarios_dir,scenarios_dir>> directory (optional)
- 'default_suites_conf_path': Path to <<default_suites_conf,default-suites.conf>> file (optional)
- 'defaults_conf_path': Path to <<defaults_conf,defaults.conf>> file (optional)
- 'resource_conf_path': Path to <<resource_conf,resources.conf>> file (optional)
These are described in detail in the following sections.
These are described in detail in the following sections. If no value is provided
for a given setting, sane default paths are used: For 'state_dir',
'/var/tmp/osmo-gsm-tester/state/' is used. All other files and directories are
expected, by default, to be in the same directory as <<config_main,main.conf>>
[[paths_conf]]
==== 'paths.conf'
IMPORTANT: Relative paths provided in 'main.conf' are parsed as being relative
to the directory of that 'main.conf' file itself, and not relative to the CWD
of the {app-name} process parsing it.
The 'paths.conf' file defines where to store the global state (of reserved
resources) and where to find suite and scenario definitions.
Any relative paths found in a 'paths.conf' file are interpreted as relative to
the directory of that 'paths.conf' file.
There's not yet any well-known schema to validate this file contents since it
has only 3 attributes.
.Sample paths.conf file:
.Sample main.conf file:
----
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: '/usr/local/src/osmo-gsm-tester/suites'
scenarios_dir: './scenarios'
default_suites_conf_path: './default-suites.conf'
defaults_conf_path: './defaults.conf'
resource_conf_path: './resources.conf'
----
[[state_dir]]
===== 'state_dir'
==== 'state_dir'
It contains global or system-wide state for osmo-gsm-tester. In a typical state
dir you can find the following files:
@ -256,7 +296,7 @@ same physical device is described differently in several
resource allocation may not work as expected.
[[suites_dir]]
===== 'suites_dir'
==== 'suites_dir'
Suites contain a set of tests which are designed to be run together to test a
set of features given a specific set of resources. As a result, resources are
@ -359,7 +399,7 @@ defaults:
----
[[scenarios_dir]]
===== 'scenarios_dir'
==== 'scenarios_dir'
This dir contains scenario configuration files.
@ -503,7 +543,7 @@ that is repeated numerous times. No special notation for these cases is
available (yet).
[[default_suites_conf]]
==== 'default-suites.conf' (optional)
==== 'default-suites.conf'
The 'default-suites.conf' file contains a YAML list of 'suite:scenario+scenario+...'
combination strings as defined by the 'osmo-gsm-tester.py -s' commandline
@ -512,10 +552,10 @@ option. If invoking the 'osmo-gsm-tester.py' without any suite definitions, the
combinations is run in sequence.
A suite name must match the name of a directory in the
<<suites_dir,suites_dir/>> as defined by <<paths_conf,paths.conf>>.
<<suites_dir,suites_dir/>> as defined by <<main_conf,main.conf>>.
A scenario name must match the name of a configuration file in the
<<scenarios_dir,scnearios_dir/>> as defined by <<paths_conf,paths.conf>>
<<scenarios_dir,scnearios_dir/>> as defined by <<main_conf,main.conf>>
(optionally without the '.conf' suffix).
.Sample 'default-suites.conf' file:
@ -530,7 +570,7 @@ A scenario name must match the name of a configuration file in the
- voice:trx+dyn_ts
----
==== 'defaults.conf' (optional)
==== 'defaults.conf'
In {app-name} object instances requested by the test and created by the suite
relate to a specific allocated resource. That's not always the case, and even if
@ -613,16 +653,17 @@ ups share mostly all configuration, main difference being the
All {app-name} related configuration for that environment is publicly available
in 'osmo-gsm-tester.git' itself:
- <<paths_conf,paths.conf>>: Available Available under 'example/', with its paths
already configured to take required bits from inside the git repository.
- <<suite_dir,suites_dir>>: Available under 'example/suites/'
- <<scenarios_dir,scenarios_dir>>: Available under 'example/scenarios/'
- <<resource_conf,resources.conf>>: Available under 'example/' as
- <<main_conf,main.conf>>: Available Available under 'sysmocom/', with its paths
already configured to take required bits from inside the git repository directory.
- <<suite_dir,suites_dir>>: Available under 'sysmocom/suites/'
- <<scenarios_dir,scenarios_dir>>: Available under 'sysmocom/scenarios/'
- <<resource_conf,resources.conf>>: Available under 'sysmocom/' as
'resources.conf.prod' for Production setup and as 'resources.conf.rnd' for the
RnD setup. One must use a symbolic link to have it available as
'resources.conf'.
//TODO: resources.conf file path should be modifiable through paths.conf!
There are also small sample setups under the 'doc/examples/' directory to
showcase how to set up different types of networks.
==== Typical Invocations
@ -633,25 +674,34 @@ such a set of binaries, see <<trials>>.
Examples for launching test trials:
- Run the default suites (see <<default_suites_conf,default_suites.conf>>) on a
given set of binaries:
given set of binaries from 'path/to/my-trial' with <<main_conf,main.conf>>
available under a standard path:
----
osmo-gsm-tester.py path/to/my-trial
----
- Run an explicit choice of 'suite:scenario' combinations:
- Same as above, but run an explicit choice of 'suite:scenario' combinations:
----
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts
----
- Run one 'suite:scenario1+scenario2' combination, setting log level to 'debug'
and enabling logging of full python tracebacks, and also only run just the
'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure:
- Same as above, but run one 'suite:scenario1+scenario2' combination, setting
log level to 'debug' and enabling logging of full python tracebacks, and also
only run just the 'mo_mt_sms.py' test from the suite, e.g. to investigate a
test failure:
----
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo+foobar -l dbg -T -t mo_mt
----
- Same as above, but tell {app-name} to read the 'main.conf' in specific
directory 'path/to/my/main.conf':
----
osmo-gsm-tester.py -c path/to/my/main.conf path/to/my-trial -s sms:sysmo+foobar -l dbg -T -t mo_mt
----
A test script may also be run step-by-step in a python debugger, see
<<debugging>>.

View File

@ -9,12 +9,14 @@ The requested resource requirements are not solvable [[0, 2], [2], [0, 2]]
ok, caused exception RuntimeError: Refusing to drop a list of resources from itself. This is probably a bug where a list of Resources() should have been copied but is passed as-is. use Resources.clear() instead.
- test removing a Resources list from one with the same list in it
- test resources config and state dir:
cnf -: DBG: Found config file paths.conf as [PATH]/selftest/resource_test/conf/paths.conf in [PATH]/selftest/resource_test/conf which is [PATH]/selftest/resource_test/conf
cnf -: DBG: [PATH]/selftest/resource_test/conf/paths.conf: relative path ./test_work/state_dir is [PATH]/selftest/resource_test/conf/test_work/state_dir
cnf -: DBG: [PATH]/selftest/resource_test/conf/paths.conf: relative path ./suite_test is [PATH]/selftest/resource_test/conf/suite_test
cnf -: DBG: Found path state_dir as [PATH]/selftest/resource_test/conf/test_work/state_dir
cnf ResourcesPool: DBG: Found config file resources.conf as [PATH]/selftest/resource_test/conf/resources.conf in [PATH]/selftest/resource_test/conf which is [PATH]/selftest/resource_test/conf
cnf ResourcesPool: DBG: Found path state_dir as [PATH]/selftest/resource_test/conf/test_work/state_dir
cnf -: DBG: Found main configuration file in [PATH]/selftest/resource_test/conf/paths.conf which is [PATH]/selftest/resource_test/conf/paths.conf
cnf -: DBG: MAIN CONFIG:
{'default_suites_conf_path': '[PATH]/selftest/resource_test/conf/default-suites.conf',
'defaults_conf_path': '[PATH]/selftest/resource_test/conf/defaults.conf',
'resource_conf_path': '[PATH]/selftest/resource_test/conf/resources.conf',
'scenarios_dir': '[PATH]/selftest/resource_test/conf/scenarios',
'state_dir': '[PATH]/selftest/resource_test/conf/test_work/state_dir',
'suites_dir': '[PATH]/selftest/resource_test/conf/suite_test'}
*** all resources:
{'arfcn': [{'_hash': 'e620569450f8259b3f0212ec19c285dd07df063c',
'arfcn': '512',

View File

@ -13,7 +13,7 @@ from osmo_gsm_tester.core.schema import generate_schemas
workdir = util.get_tempdir()
# override config locations to make sure we use only the test conf
config.override_conf = os.path.join(os.path.dirname(sys.argv[0]), 'conf')
config.override_conf = os.path.join(os.path.dirname(sys.argv[0]), 'conf', 'paths.conf')
log.get_process_id = lambda: '123-1490837279'

View File

@ -1,32 +1,28 @@
cnf -: DBG: Found config file paths.conf as [PATH]/selftest/scenario_test/paths.conf in [PATH]/selftest/scenario_test which is [PATH]/selftest/scenario_test
cnf -: DBG: [PATH]/selftest/scenario_test/paths.conf: relative path . is [PATH]/selftest/scenario_test
cnf -: DBG: [PATH]/selftest/scenario_test/paths.conf: relative path ./test_work/state_dir is [PATH]/selftest/scenario_test/test_work/state_dir
cnf -: DBG: [PATH]/selftest/scenario_test/paths.conf: relative path . is [PATH]/selftest/scenario_test
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
cnf -: DBG: Found main configuration file in [PATH]/selftest/scenario_test/paths.conf which is [PATH]/selftest/scenario_test/paths.conf
cnf -: DBG: MAIN CONFIG:
{'default_suites_conf_path': '[PATH]/selftest/scenario_test/default-suites.conf',
'defaults_conf_path': '[PATH]/selftest/scenario_test/defaults.conf',
'resource_conf_path': '[PATH]/selftest/scenario_test/resources.conf',
'scenarios_dir': '[PATH]/selftest/scenario_test',
'state_dir': '[PATH]/selftest/scenario_test/test_work/state_dir',
'suites_dir': '[PATH]/selftest/scenario_test'}
scenario_case_01.conf
{'anotherlist': ['4', '0'],
'foobar': 'True',
'somelist': [{'somelistitem': 'firststring'},
{'somelistitem': 'secondstring'},
{'somelistitem': 'thirdstring'}]}
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
scenario_case_01.conf
{'anotherlist': ['4', '0'],
'foobar': 'True',
'somelist': [{'somelistitem': 'firststring'},
{'somelistitem': 'secondstring'},
{'somelistitem': 'thirdstring'}]}
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
OK: expected RuntimeError: No such scenario file: '[PATH]/selftest/scenario_test/scenario_case_01@.conf' (nor scenario_case_01@.conf)
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
OK: expected ValueError
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
OK: expected ValueError
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
OK: expected RuntimeError: No such scenario file: '[PATH]/selftest/scenario_test/scenario_case_03.conf'
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
OK: expected RuntimeError: No such scenario file: '[PATH]/selftest/scenario_test/scenario_case_03.conf'
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
tst scenario_case_03@heyho,1,yes.conf: DBG: {param_dict={param1='heyho', param2='1', param3='yes'}}
scenario_case_03@heyho,1,yes.conf
{'anotherlist': ['1', '0'],
@ -34,7 +30,6 @@ scenario_case_03@heyho,1,yes.conf
'somelist': [{'somelistitem': 'firststring'},
{'somelistitem': 'heyho'},
{'somelistitem': 'thirdstring'}]}
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
tst scenario_case_03@heyho,1,yes.conf: DBG: {param_dict={param1='heyho', param2='1', param3='yes'}}
scenario_case_03@heyho,1,yes.conf
{'anotherlist': ['1', '0'],
@ -42,17 +37,13 @@ scenario_case_03@heyho,1,yes.conf
'somelist': [{'somelistitem': 'firststring'},
{'somelistitem': 'heyho'},
{'somelistitem': 'thirdstring'}]}
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
tst scenario_case_03@heyho,1.conf: DBG: {param_dict={param1='heyho', param2='1'}}
OK: expected NameError: Undefined
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
tst scenario_case_03@heyho,1.conf: DBG: {param_dict={param1='heyho', param2='1'}}
OK: expected NameError: Undefined
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
tst scenario_case_03@specific.conf: DBG: {param_dict={param1='specific'}}
scenario_case_03@specific.conf
{'somelist': [{'somelistitem': 'specific'}]}
cnf -: DBG: Found path scenarios_dir as [PATH]/selftest/scenario_test
tst scenario_case_03@specific.conf: DBG: {param_dict={param1='specific'}}
scenario_case_03@specific.conf
{'somelist': [{'somelistitem': 'specific'}]}

View File

@ -18,7 +18,7 @@ test_schema = {
'foobar' : schema.BOOL_STR,
}
config.override_conf = os.path.join(os.path.dirname(sys.argv[0]))
config.override_conf = os.path.join(os.path.dirname(sys.argv[0]), 'paths.conf')
def print_scenario(sc):
# we use copy() to be able to get the dictionary in super class of Scenario:

View File

@ -1,15 +1,17 @@
- non-existing suite dir
cnf -: DBG: Found config file paths.conf as [PATH]/selftest/suite_test/paths.conf in [PATH]/selftest/suite_test which is [PATH]/selftest/suite_test
cnf -: DBG: [PATH]/selftest/suite_test/paths.conf: relative path ./test_work/state_dir is [PATH]/selftest/suite_test/test_work/state_dir
cnf -: DBG: [PATH]/selftest/suite_test/paths.conf: relative path . is [PATH]/selftest/suite_test
cnf -: DBG: Found path suites_dir as [PATH]/selftest/suite_test
cnf -: DBG: Found main configuration file in [PATH]/selftest/suite_test/paths.conf which is [PATH]/selftest/suite_test/paths.conf
cnf -: DBG: MAIN CONFIG:
{'default_suites_conf_path': '[PATH]/selftest/suite_test/default-suites.conf',
'defaults_conf_path': '[PATH]/selftest/suite_test/defaults.conf',
'resource_conf_path': '[PATH]/selftest/suite_test/resources.conf',
'scenarios_dir': '[PATH]/selftest/suite_test/scenarios',
'state_dir': '[PATH]/selftest/suite_test/test_work/state_dir',
'suites_dir': '[PATH]/selftest/suite_test'}
--- -: ERR: RuntimeError: Suite not found: 'does_not_exist' in [PATH]/selftest/suite_test
- no suite.conf
cnf -: DBG: Found path suites_dir as [PATH]/selftest/suite_test
cnf empty_dir: DBG: reading suite.conf
cnf [PATH]/selftest/suite_test/empty_dir/suite.conf: ERR: FileNotFoundError: [Errno 2] No such file or directory: '[PATH]/selftest/suite_test/empty_dir/suite.conf' [empty_dir↪[PATH]/selftest/suite_test/empty_dir/suite.conf]
- valid suite dir
cnf -: DBG: Found path suites_dir as [PATH]/selftest/suite_test
cnf test_suite: DBG: reading suite.conf
defaults:
timeout: 60s
@ -25,8 +27,6 @@ resources:
- times: '2'
- run hello world test
cnf ResourcesPool: DBG: Found config file resources.conf as [PATH]/selftest/suite_test/resources.conf in [PATH]/selftest/suite_test which is [PATH]/selftest/suite_test
cnf ResourcesPool: DBG: Found path state_dir as [PATH]/selftest/suite_test/test_work/state_dir
---------------------------------------------------------------------
trial test_suite
@ -179,8 +179,6 @@ FAIL: test_suite (fail: 1, skip: 6)
FAIL: test_fail_raise.py (N.N sec) ExpectedFail: This failure is expected
skip: test_suite_params.py
- test with half empty scenario
cnf ResourcesPool: DBG: Found config file resources.conf as [PATH]/selftest/suite_test/resources.conf in [PATH]/selftest/suite_test which is [PATH]/selftest/suite_test [config.py:[LINENR]]
cnf ResourcesPool: DBG: Found path state_dir as [PATH]/selftest/suite_test/test_work/state_dir [config.py:[LINENR]]
---------------------------------------------------------------------
trial test_suite
@ -267,8 +265,6 @@ PASS: test_suite (pass: 1, skip: 6)
skip: test_fail_raise.py
skip: test_suite_params.py
- test with scenario
cnf ResourcesPool: DBG: Found config file resources.conf as [PATH]/selftest/suite_test/resources.conf in [PATH]/selftest/suite_test which is [PATH]/selftest/suite_test [config.py:[LINENR]]
cnf ResourcesPool: DBG: Found path state_dir as [PATH]/selftest/suite_test/test_work/state_dir [config.py:[LINENR]]
---------------------------------------------------------------------
trial test_suite
@ -355,8 +351,6 @@ PASS: test_suite (pass: 1, skip: 6)
skip: test_fail_raise.py
skip: test_suite_params.py
- test with scenario and modifiers
cnf ResourcesPool: DBG: Found config file resources.conf as [PATH]/selftest/suite_test/resources.conf in [PATH]/selftest/suite_test which is [PATH]/selftest/suite_test [config.py:[LINENR]]
cnf ResourcesPool: DBG: Found path state_dir as [PATH]/selftest/suite_test/test_work/state_dir [config.py:[LINENR]]
tst test_suite: reserving resources in [PATH]/selftest/suite_test/test_work/state_dir ... [suite.py:[LINENR]]
tst test_suite: DBG: {combining='resources'} [suite.py:[LINENR]]
tst {combining_scenarios='resources'}: DBG: {definition_conf={bts=[{'label': 'sysmoCell 5000'}, {'label': 'sysmoCell 5000'}, {'type': 'sysmo'}], ip_address=[{}], modem=[{}, {}]}} [test_suite↪{combining_scenarios='resources'}] [suite.py:[LINENR]]
@ -489,8 +483,6 @@ PASS: test_suite (pass: 1, skip: 6)
skip: test_fail_raise.py
skip: test_suite_params.py
- test with suite-specific config
cnf ResourcesPool: DBG: Found config file resources.conf as [PATH]/selftest/suite_test/resources.conf in [PATH]/selftest/suite_test which is [PATH]/selftest/suite_test [config.py:[LINENR]]
cnf ResourcesPool: DBG: Found path state_dir as [PATH]/selftest/suite_test/test_work/state_dir [config.py:[LINENR]]
tst test_suite: reserving resources in [PATH]/selftest/suite_test/test_work/state_dir ... [suite.py:[LINENR]]
tst test_suite: DBG: {combining='resources'} [suite.py:[LINENR]]
tst {combining_scenarios='resources'}: DBG: {definition_conf={bts=[{'label': 'sysmoCell 5000'}, {'label': 'sysmoCell 5000'}, {'type': 'sysmo'}], ip_address=[{}], modem=[{}, {}]}} [test_suite↪{combining_scenarios='resources'}] [suite.py:[LINENR]]

View File

@ -11,7 +11,7 @@ from osmo_gsm_tester.core import scenario
from osmo_gsm_tester.core import suite
from osmo_gsm_tester.core.schema import generate_schemas, get_all_schema
config.override_conf = os.path.join(os.path.dirname(sys.argv[0]))
config.override_conf = os.path.join(os.path.dirname(sys.argv[0]), 'paths.conf')
example_trial_dir = os.path.join('test_trial_tmp')

View File

@ -24,9 +24,9 @@
Examples:
./osmo-gsm-tester.py -c doc/examples/2g_osmocom/ ~/my_trial_package/ -s osmo_trx
./osmo-gsm-tester.py -c doc/examples/2g_osmocom/ ~/my_trial_package/ -s sms_tests:dyn_ts+eu_band+bts_sysmo
./osmo-gsm-tester.py -c sysmocom/ ~/my_trial_package/ -s sms_tests/mo_mt_sms:bts_trx
./osmo-gsm-tester.py -c doc/examples/2g_osmocom/main.conf ~/my_trial_package/ -s osmo_trx
./osmo-gsm-tester.py -c doc/examples/2g_osmocom/main.conf ~/my_trial_package/ -s sms_tests:dyn_ts+eu_band+bts_sysmo
./osmo-gsm-tester.py -c sysmocom/main.conf ~/my_trial_package/ -s sms_tests/mo_mt_sms:bts_trx
(The names for test suites and scenarios used in these examples must be defined
by the osmo-gsm-tester configuration.)
@ -54,8 +54,9 @@ choosing the first available item that matches the other constraints.
A test run thus needs to define:
* A trial package containing built binaries
* A set of test suites, each with its combinations of scenarios
* A configuration directory specifying sets of resources, default configurations
and paths on where to find suites, scenarios, etc.
* A main configuration file specifying paths to other files containing sets of
resources, default configurations and paths on where to find suites,
scenarios, etc.
If no combination of suites and scenarios is provided, the default list of
suites will be run as defined in the osmo-gsm-tester configuration.
@ -101,8 +102,8 @@ def main():
# is easiest to maintain.
parser.add_argument('-V', '--version', action='store_true',
help='Show version')
parser.add_argument('-c', '--conf-dir', dest='conf_dir',
help='''Specify configuration directory path (containing paths.conf)''')
parser.add_argument('-c', '--conf-path', dest='conf_path',
help='''Specify main configuration file path''')
parser.add_argument('trial_package',
help='Directory containing binaries to test')
parser.add_argument('-s', '--suite-scenario', dest='suite_scenario', action='append',
@ -140,18 +141,18 @@ optional.''')
log.style_change(trace=True)
if args.source:
log.style_change(src=True)
if args.conf_dir:
config.override_conf = args.conf_dir
if args.conf_path:
config.override_conf = args.conf_path
combination_strs = list(args.suite_scenario or [])
if not combination_strs:
combination_strs = config.read_config_file(config.DEFAULT_SUITES_CONF, if_missing_return=[])
combination_strs = config.read_config_file(config.CFG_DEFAULT_SUITES_CONF, if_missing_return=[])
if combination_strs:
print('Running default suites:\n ' + ('\n '.join(combination_strs)))
else:
print('No default suites configured (%r)' % config.DEFAULT_SUITES_CONF)
print('Failed to load default suites (%r)' % config.get_main_config_value(config.DEFAULT_SUITES_CONF, fail_if_missing=False))
if not combination_strs:

View File

@ -1,8 +1,9 @@
# osmo_gsm_tester: read and manage config files and global config
#
# Copyright (C) 2016-2017 by sysmocom - s.f.m.c. GmbH
# Copyright (C) 2016-2020 by sysmocom - s.f.m.c. GmbH
#
# Author: Neels Hofmeyr <neels@hofmeyr.de>
# Author: Pau Espin Pedrol <pespin@sysmocom.de>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as
@ -53,6 +54,7 @@
import yaml
import os
import copy
import pprint
from . import log, util, template
from . import schema
@ -60,94 +62,122 @@ from .util import is_dict, is_list, Dir, get_tempdir
override_conf = None
DEFAULT_CONFIG_LOCATIONS = [
'.',
os.path.join(os.getenv('HOME'), '.config', 'osmo-gsm-tester'),
'/usr/local/etc/osmo-gsm-tester',
'/etc/osmo-gsm-tester'
]
PATHS_CONF = 'paths.conf'
DEFAULT_SUITES_CONF = 'default-suites.conf'
DEFAULTS_CONF = 'defaults.conf'
RESOURCES_CONF = 'resources.conf'
PATH_STATE_DIR = 'state_dir'
PATH_SUITES_DIR = 'suites_dir'
PATH_SCENARIOS_DIR = 'scenarios_dir'
PATHS_SCHEMA = {
PATH_STATE_DIR: schema.STR,
PATH_SUITES_DIR: schema.STR,
PATH_SCENARIOS_DIR: schema.STR,
CFG_STATE_DIR = 'state_dir'
CFG_SUITES_DIR = 'suites_dir'
CFG_SCENARIOS_DIR = 'scenarios_dir'
CFG_DEFAULT_SUITES_CONF = 'default_suites_conf_path'
CFG_DEFAULTS_CONF = 'defaults_conf_path'
CFG_RESOURCES_CONF = 'resource_conf_path'
MAIN_CONFIG_SCHEMA = {
CFG_STATE_DIR: schema.STR,
CFG_SUITES_DIR: schema.STR,
CFG_SCENARIOS_DIR: schema.STR,
CFG_DEFAULT_SUITES_CONF: schema.STR,
CFG_DEFAULTS_CONF: schema.STR,
CFG_RESOURCES_CONF: schema.STR,
}
PATHS_TEMPDIR_STR = '$TEMPDIR'
DF_CFG_STATE_DIR = '/var/tmp/osmo-gsm-tester/state/'
DF_CFG_SUITES_DIR = './suites'
DF_CFG_SCENARIOS_DIR = './scenarios'
DF_CFG_DEFAULT_SUITES_CONF = './default-suites.conf'
DF_CFG_DEFAULTS_CONF = './defaults.conf'
DF_CFG_RESOURCES_CONF = './resources.conf'
PATHS = None
DEFAULT_CONFIG_FILENAME = 'main.conf'
def _get_config_file(basename, fail_if_missing=True):
DEFAULT_CONFIG_LOCATIONS = [
'.',
os.path.join(os.getenv('HOME'), '.config', 'osmo-gsm-tester', DEFAULT_CONFIG_FILENAME),
os.path.join('/usr/local/etc/osmo-gsm-tester', DEFAULT_CONFIG_FILENAME),
os.path.join('/etc/osmo-gsm-tester', DEFAULT_CONFIG_FILENAME)
]
MAIN_CONFIG = None
MAIN_CONFIG_PATH = None
def _find_main_config_path():
if override_conf:
locations = [ override_conf ]
elif os.getenv('OSMO_GSM_TESTER_CONF'):
ENV_CONF = os.getenv('OSMO_GSM_TESTER_CONF')
log.err('Using environment variable OSMO_GSM_TESTER_CONF=%s is deprecated. Rather use -c command line argument!' % ENV_CONF)
locations = [ ENV_CONF ]
log.err('Using environment variable OSMO_GSM_TESTER_CONF=%s(/paths.conf) is deprecated. Rather use -c command line argument!' % ENV_CONF)
locations = [ ENV_CONF + 'paths.conf' ] # directory is expected in OSMO_GSM_TESTER_CONF, bakcward compatibility
else:
locations = DEFAULT_CONFIG_LOCATIONS
for l in locations:
real_l = os.path.realpath(l)
p = os.path.realpath(os.path.join(real_l, basename))
if os.path.isfile(p):
log.dbg('Found config file', basename, 'as', p, 'in', l, 'which is', real_l, _category=log.C_CNF)
return (p, real_l)
if not fail_if_missing:
return None, None
raise RuntimeError('configuration file not found: %r in %r' % (basename,
[os.path.abspath(p) for p in locations]))
if os.path.isfile(real_l):
log.dbg('Found main configuration file in ', l, 'which is', real_l, _category=log.C_CNF)
return real_l
raise RuntimeError('Main configuration file not found in %r' % ([l for l in locations]))
def get_config_file(basename, fail_if_missing=True):
path, found_in = _get_config_file(basename, fail_if_missing)
def _get_main_config_path():
global MAIN_CONFIG_PATH
if MAIN_CONFIG_PATH is None:
MAIN_CONFIG_PATH = _find_main_config_path()
return MAIN_CONFIG_PATH
def main_config_path_to_abspath(path):
'Relative files in main config are relative towards the config file, not towards $CWD'
if not path.startswith(os.pathsep):
return os.path.realpath(os.path.join(os.path.dirname(_get_main_config_path()), path))
return path
def read_config_file(basename, validation_schema=None, if_missing_return=False):
def _get_main_config():
global MAIN_CONFIG
if MAIN_CONFIG is None:
cfg = read(_get_main_config_path(), MAIN_CONFIG_SCHEMA)
MAIN_CONFIG = {
CFG_STATE_DIR: DF_CFG_STATE_DIR,
CFG_SUITES_DIR: DF_CFG_SUITES_DIR,
CFG_SCENARIOS_DIR: DF_CFG_SCENARIOS_DIR,
CFG_DEFAULT_SUITES_CONF: DF_CFG_DEFAULT_SUITES_CONF,
CFG_DEFAULTS_CONF: DF_CFG_DEFAULTS_CONF,
CFG_RESOURCES_CONF: DF_CFG_RESOURCES_CONF,
}
overlay(MAIN_CONFIG, cfg)
for key, path in sorted(MAIN_CONFIG.items()):
MAIN_CONFIG[key] = main_config_path_to_abspath(path)
log.dbg('MAIN CONFIG:\n' + pprint.pformat(MAIN_CONFIG), _category=log.C_CNF)
return MAIN_CONFIG
def get_main_config_value(cfg_name, fail_if_missing=True):
cfg = _get_main_config()
f = cfg.get(cfg_name, None)
if f is None and fail_if_missing:
raise RuntimeError('Missing configuration %s' % (cfg_name))
return f
def read_config_file(cfg_name, validation_schema=None, if_missing_return=False):
'''Read content of config file cfg_name (referring to key in main config).
If "if_missing_return" is different than False, then instead of failing it will return whatever it is stored in that arg
'''
fail_if_missing = True
if if_missing_return is not False:
fail_if_missing = False
path = get_config_file(basename, fail_if_missing=fail_if_missing)
path = get_main_config_value(cfg_name, fail_if_missing=fail_if_missing)
if path is None:
return if_missing_return
return read(path, validation_schema=validation_schema, if_missing_return=if_missing_return)
def get_configured_path(label, allow_unset=False):
global PATHS
if PATHS is None:
paths_file, found_in = _get_config_file(PATHS_CONF)
PATHS = read(paths_file, PATHS_SCHEMA)
# sorted for deterministic regression test results
for key, path in sorted(PATHS.items()):
if not path.startswith(os.pathsep):
PATHS[key] = os.path.realpath(os.path.join(found_in, path))
log.dbg(paths_file + ': relative path', path, 'is', PATHS[key], _category=log.C_CNF)
p = PATHS.get(label)
if p is None and not allow_unset:
raise RuntimeError('missing configuration in %s: %r' % (PATHS_CONF, label))
log.dbg('Found path', label, 'as', p, _category=log.C_CNF)
if p.startswith(PATHS_TEMPDIR_STR):
p = os.path.join(get_tempdir(), p[len(PATHS_TEMPDIR_STR):])
log.dbg('Path', label, 'contained', PATHS_TEMPDIR_STR, 'and becomes', p, _category=log.C_CNF)
return p
def get_state_dir():
return Dir(get_configured_path(PATH_STATE_DIR))
return Dir(get_main_config_value(CFG_STATE_DIR))
def get_suites_dir():
return Dir(get_configured_path(PATH_SUITES_DIR))
return Dir(get_main_config_value(CFG_SUITES_DIR))
def get_scenarios_dir():
return Dir(get_configured_path(PATH_SCENARIOS_DIR))
return Dir(get_main_config_value(CFG_SCENARIOS_DIR))
DEFAULTS_CONF = None
def get_defaults(for_kind):
global DEFAULTS_CONF
if DEFAULTS_CONF is None:
DEFAULTS_CONF = read_config_file(CFG_DEFAULTS_CONF, if_missing_return={})
defaults = DEFAULTS_CONF.get(for_kind, {})
return copy.deepcopy(defaults)
def read(path, validation_schema=None, if_missing_return=False):
log.ctx(path)
@ -191,10 +221,6 @@ def _standardize(config):
config = yaml.safe_load(_tostr(_standardize_item(config)))
return config
def get_defaults(for_kind):
defaults = read_config_file(DEFAULTS_CONF, if_missing_return={})
return defaults.get(for_kind, {})
def overlay(dest, src):
if is_dict(dest):
if not is_dict(src):

View File

@ -48,7 +48,7 @@ class ResourcesPool(log.Origin):
_registered_exit_handler = False
def __init__(self):
self.config_path = config.get_config_file(config.RESOURCES_CONF)
self.config_path = config.get_main_config_value(config.CFG_RESOURCES_CONF)
self.state_dir = config.get_state_dir()
super().__init__(log.C_CNF, conf=self.config_path, state=self.state_dir.path)
self.read_conf()

View File

@ -1,13 +1,14 @@
This a real 2G test suite configured and ready to use.
The only thing missing is a trial dir containing binaries.
You can point osmo-gsm-tester.py at this config using the '-c $DIR' command line
argument, where DIR is the directory path where this README file resides.
You can point osmo-gsm-tester.py at this config using the '-c $DIR/main.conf'
command line argument, where DIR is the directory path where this README file
resides.
If you have your trial with binary tar archives in ~/my_trial
you can run the suite for example like this:
```
osmo-gsm-tester.py -c $DIR ~/my_trial
osmo-gsm-tester.py -c $DIR/main.conf ~/my_trial
```
Alternatively you can setup this example as default config for your user by
@ -19,4 +20,4 @@ ln -s "$DIR" ~/.config/osmo-gsm-tester
A ./state dir will be created to store the current osmo-gsm-tester state. If
you prefer not to write to $DIR, set up an own configuration pointing at a
different path (see paths.conf: 'state_dir').
different path (see main.conf: 'state_dir').

6
sysmocom/main.conf Normal file
View File

@ -0,0 +1,6 @@
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: './suites'
scenarios_dir: './scenarios'
default_suites_conf_path: './default-suites.conf'
defaults_conf_path: './defaults.conf'
resource_conf_path: './resources.conf'

View File

@ -1,3 +0,0 @@
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: './suites'
scenarios_dir: './scenarios'

View File

@ -1 +0,0 @@
../defaults.conf

View File

@ -1,6 +1,8 @@
#!/bin/sh
set -e -x
base="$PWD"
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd -P) # this file's directory
OSMO_GSM_TESTER_CONF=${OSMO_GSM_TESTER_CONF:-${SCRIPT_DIR}/main.conf}
time_start="$(date '+%F %T')"

6
sysmocom/ttcn3/main.conf Normal file
View File

@ -0,0 +1,6 @@
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: './suites'
scenarios_dir: './scenarios'
default_suites_conf_path: './default-suites.conf'
defaults_conf_path: '../defaults.conf'
resource_conf_path: './resources.conf'

View File

@ -1,3 +0,0 @@
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: './suites'
scenarios_dir: './scenarios'