== Configuration [[config]] === Configuration files and directories Find in below sub-sections all user-defined files and directories used by {app-name} to run tests on a given setup. [[config_main]] ==== 'main.conf' The main configuration file is basically a placeholder for {app-name} to find paths to all other files and directories used to operate and run tests. {app-name} looks for the main configuration file in various standard paths in this order: - './main.conf' (Current Working Directory) - '$HOME/.config/osmo-gsm-tester/main.conf' - '/usr/local/etc/osmo-gsm-tester/main.conf' - '/etc/osmo-gsm-tester/main.conf' The config file location can also be set through '-c' command line argument, which then overrides the above locations. {app-name} expects to find the following configuration settings in 'main.conf': - 'state_dir': Path to <> directory - 'trial_dir': Path to <> directory to test against (overridden by cmdline argument) - 'suites_dir': List of paths to <> directories. - 'scenarios_dir': List of paths to <> directories (optional) - 'default_suites_conf_path': Path to <> file (optional) - 'defaults_conf_path': Path to <> file (optional) - 'resource_conf_path': Path to <> file (optional) Configuration settings holding a list of paths, such as 'suites_dir' or 'scenarios_dir', are used to look up for paths in regular list of order, meaning first paths in list take preference over last ones. As a result, if a suite named 'A' is found in several paths, the one on the first path in the list will be used. These are described in detail in the following sections. If no value is provided for a given setting, sane default paths are used: For 'state_dir', '/var/tmp/osmo-gsm-tester/state/' is used. All other files and directories are expected, by default, to be in the same directory as <> IMPORTANT: Relative paths provided in 'main.conf' are parsed as being relative to the directory of that 'main.conf' file itself, and not relative to the CWD of the {app-name} process parsing it. .Sample main.conf file: ---- state_dir: '/var/tmp/osmo-gsm-tester/state' suites_dir: [ '/usr/local/src/osmo-gsm-tester/suites' ] scenarios_dir: [ './scenarios' ] trial_dir: './trial' default_suites_conf_path: './default-suites.conf' defaults_conf_path: './defaults.conf' resource_conf_path: './resources.conf' ---- [[state_dir]] ==== 'state_dir' It contains global or system-wide state for osmo-gsm-tester. In a typical state dir you can find the following files: 'last_used_*.state':: Contains stateful content spanning accross {app-name} instances and runs. For instance, 'last used msisdn number.state' is automatically (and atomically) increased every time osmo-gsm-tester needs to assign a new subscriber in a test, ensuring tests get unique msisdn numbers. 'reserved_resources.state':: File containing a set of reserved resources by any number of osmo-gsm-tester instances (aka pool of allocated resources). Each osmo-gsm-tester instance is responsible to clear its resources from the list once it is done using them and are no longer reserved. 'lock':: Lock file used to implement a mutual exclusion zone around any state files in the 'state_dir', to prevent race conditions between different {app-name} instances running in parallel. This way, several concurrent users of osmo-gsm-tester (ie. several osmo-gsm-tester processes running in parallel) can run without interfering with each other (e.g. using same ARFCN, same IP or same ofono modem path). If you would like to set up several separate configurations (not typical), note that the 'state_dir' is used to reserve resources, which only works when all configurations that share resources also use the same 'state_dir'. It's also important to notice that since resources are stored in YAML dictionary form, if same physical device is described differently in several <> files (used by different {app-name} instances), resource allocation may not work as expected. [[suites_dir]] ==== 'suites_dir' Suites contain a set of tests which are designed to be run together to test a set of features given a specific set of resources. As a result, resources are allocated per suite and not per test. Tests for a given suite are located in the form of '.py' python scripts in the same directory where the <> lays. Tests in the same testsuite willing to use some shared code can do so by putting it eg. in '$suites_dir/$suitename/lib/testlib.py': ---- #!/usr/bin/env python3 from osmo_gsm_tester.testenv import * def my_shared_code(foo): return foo.bar() ---- and then in the test itself use it this way: ---- #!/usr/bin/env python3 from osmo_gsm_tester.testenv import * import testlib suite.test_import_modules_register_for_cleanup(testlib) from testlib import my_shared_code bar = my_shared_code(foo) ---- .Sample 'suites_dir' directory tree: ---- suites_dir/ |-- suiteA | |-- suite.conf | '-- testA.py |-- suiteB | |-- testB.py | |-- testC.py | |-- lib | | '-- testlib.py | '-- suite.conf ---- [[suite_conf]] ===== 'suite.conf' This file content is parsed using the <> schema. On the <> section, it provides {app-name} with the base restrictions (later to be further filtered by <> files) to apply when allocating resources. It can also override attributes for the allocated resources through the <> section (to be further modified by <> files later on). Similarly it can do the same for general configuration options (no per-resource) through the <> section. The _schema_ section allows defining a suite's own schema used to validate parameters passed to it later on through <> files (See <>), and which can be retrieved by tests using the _tenv.config_suite_specific()_ and _tenv.config_test_specific()_ APIs. The first one will provide the whole dictionary under schema, while the later will return the dictionary immediatelly inside the former and matching the test name being run. For instance, if _tenv.config_test_specific()_ is called from test _a_suite_test_foo.py_, the method will return the contents under dictionary with key _a_suite_test_foo_. .Sample 'suite.conf' file: ---- resources: ip_address: - times: 9 # msc, bsc, hlr, stp, mgw*2, sgsn, ggsn, iperf3srv bts: - times: 1 modem: - times: 2 features: - gprs - voice - times: 2 features: - gprs config: bsc: net: codec_list: - fr1 schema: some_suite_parameter: 'uint' a_suite_test_foo: one_test_parameter_for_test_foo: 'str' another_test_parameter_for_test_foo: ['bool_str'] defaults: timeout: 50s ---- [[scenarios_dir]] ==== 'scenarios_dir' This dir contains scenario configuration files. .Sample 'scenarios_dir' directory tree: ---- scenarios_dir/ |-- scenarioA.conf '-- scenarioB.conf ---- [[scenario_conf]] ===== 'scenario conf file' Scenarios define further constraints to serve the resource requests of a <>, ie. to select specific resources from the general resource pool specified in <>. If only one resource is specified in the scenario, then the resource allocator assumes the restriction is to be applied to the first resource and that remaining resources have no restrictions to be taken into consideration. To apply restrictions only on the second resource, the first element can be left emtpy, like: ---- resources: bts: - {} - type: osmo-bts-sysmo ---- On the 'osmo_gsm_tester.py' command line and the <>, any number of such scenario configurations can be combined in the form: ---- :[+[+...]] ---- e.g. ---- my_suite:sysmo+tch_f+amr ---- *_Parametrized scenario conf files_*: Furthermore, scenario '.conf' files can be parametrized. The concept is similar to that of systemd's Template Unit Files. That is, an scenario file can be written so that some values inside it can be passed at the time of referencing the scenario name. The idea behind its existence is to re-use the same scenario file for a set of attributes which are changed and that can have a lot of different values. For instance, if a scenario is aimed at setting or filtering some specific attribute holding an integer value, without parametrized scenarios then a separate file would be needed for each value the user wanted to use. A parametrized scenario file, similar to systemd Template Unit Files, contain the character '@' in their file name, ie follow the syntax below: ---- scenario-name@param1,param2,param3,[...],paramN.conf ---- Then, its content can be written this way: ---- $ cat $scenario_dir/my-parametrized-scenario@.conf resources: enb: - type: srsenb rf_dev_type: ${param1} modifiers: enb: - num_prb: ${param2} ---- Finally, it can be referenced during {app-name} execution this way, for instance when running a suite named '4g': ---- - 4g:my-parametrized-scenario@uhd,6 ---- This way {app-name} when parsing the scenarios and combining them with the suite will:: . Find out it is parametrized (name contains '@'). . Split the name ('my-parametrized-scenario') from the parameter list (param1='uhd', param2='6') . Attempt to match a '.conf' file fully matching name and parameters (hence specific content can be set for specific values while still using parameters for general values), and otherwise match only by name. . Generate the final scenario content from the template available in the matched '.conf' file. [[scenario_suite_params]] *_Scenario to set suite/test parameters_*: First, the suite needs to define its schema in its <> file. Check <> on how to do so. For instance, for a suite named 'mysuite' containing a test 'a_suite_test_foo.py', and containing this schema in its <> file: ---- schema: some_suite_parameter: 'uint' a_suite_test_foo: one_test_parameter_for_test_foo: 'str' another_test_parameter_for_test_foo: ['bool_str'] ---- One could define a parametrized scenario 'myparamscenario@.conf' like this: ---- config: suite: mysuite: some_suite_parameter: ${param1} a_suite_test_foo: one_test_parameter_for_test_foo: ${param2} another_test_parameter_for_test_foo: ['true', 'false', 'false', 'true'] ---- And use it in {app-name} this way: ---- mysuite:myparamscenario@4,hello.conf ---- [[resources_conf]] ==== 'resources.conf' //TODO: update this section The 'resources.conf' file defines which hardware is connected to the main unit, as well as which limited configuration items (like IP addresses or ARFCNs) should be used. A 'resources.conf' is validated by the <>. That means it is structured as a list of items for each resource type, where each item has one or more attributes -- for an example, see <>. Side note: at first sight it might make sense to the reader to rather structure e.g. the 'ip_address' or 'arfcn' configuration as + '"arfcn: GSM-1800: [512, 514, ...]"', + but the more verbose format is chosen to stay consistent with the general structure of resource configurations, which the resource allocation algorithm uses to resolve required resources according to their traits. These configurations look cumbersome because they exhibit only one trait / a trait that is repeated numerous times. No special notation for these cases is available (yet). [[default_suites_conf]] ==== 'default-suites.conf' The 'default-suites.conf' file contains a YAML list of 'suite:scenario+scenario+...' combination strings as defined by the 'osmo-gsm-tester.py -s' commandline option. If invoking the 'osmo-gsm-tester.py' without any suite definitions, the '-s' arguments are taken from this file instead. Each of these suite + scenario combinations is run in sequence. A suite name must match the name of a directory in the <> as defined by <>. A scenario name must match the name of a configuration file in the <> as defined by <> (optionally without the '.conf' suffix). .Sample 'default-suites.conf' file: ---- - sms:sysmo - voice:sysmo+tch_f - voice:sysmo+tch_h - voice:sysmo+dyn_ts - sms:trx - voice:trx+tch_f - voice:trx+tch_h - voice:trx+dyn_ts ---- ==== 'defaults.conf' In {app-name} object instances requested by the test and created by the suite relate to a specific allocated resource. That's not always the case, and even if it the case the information stored in <> for that resource may not contain tons of attributes which the object class needs to manage the resource. For this exact reason, the 'defaults.conf' file exist. It contains a set of default attributes and values (in YAML format) that object classes can use to fill in the missing gaps, or to provide values which can easily be changed or overwritten by <> or <> files through modifiers. Each binary run by osmo-gsm-tester, e.g. 'osmo-nitb' or 'osmo-bts-sysmo', typically has a configuration file template that is populated with values for a trial run. Hence, a <>, <> or a <> providing a similar setting always has precedence over the values given in a 'defaults.conf' .Sample 'defaults.conf' file: ---- nitb: net: mcc: 901 mnc: 70 short_name: osmo-gsm-tester-nitb long_name: osmo-gsm-tester-nitb auth_policy: closed encryption: a5_0 bsc: net: mcc: 901 mnc: 70 short_name: osmo-gsm-tester-msc long_name: osmo-gsm-tester-msc auth_policy: closed encryption: a5_0 authentication: optional msc: net: mcc: 901 mnc: 70 short_name: osmo-gsm-tester-msc long_name: osmo-gsm-tester-msc auth_policy: closed encryption: a5_0 authentication: optional bsc_bts: location_area_code: 23 base_station_id_code: 63 stream_id: 255 osmobsc_bts_type: sysmobts trx_list: - nominal_power: 23 max_power_red: 0 arfcn: 868 timeslot_list: - phys_chan_config: CCCH+SDCCH4 - phys_chan_config: SDCCH8 - phys_chan_config: TCH/F_TCH/H_PDCH - phys_chan_config: TCH/F_TCH/H_PDCH - phys_chan_config: TCH/F_TCH/H_PDCH - phys_chan_config: TCH/F_TCH/H_PDCH - phys_chan_config: TCH/F_TCH/H_PDCH - phys_chan_config: TCH/F_TCH/H_PDCH ---- === Schemas All configuration attributes in {app-name} are stored and provided as YAML files, which are handled internally mostly as sets of dictionaries, lists and scalars. Each of these configurations have a known format (set of keys and values), which is called 'schema'. Each provided configuration is validated against its 'schema' at parse time. Hence, 'schemas' can be seen as a namespace containing a structured tree of configuration attributes. Each attribute has a schema type assigned which constrains the type of value it can hold. There are several well-known schemas used across {app-name}, and they are described in following sub-sections. [[schema_main_cfg]] ==== Schema 'main config' This schema defines all the attributes available in {app-name} the main configuration file <>, and it is used to validate it. [[schema_resources]] ==== Schema 'resources' This schema defines all the attributes which can be assigned to a _resource_, and it is used to validate the <> file. Hence, the <> contains a list of elements for each resource type. This schema is also used and extended by the <>. It is important to understand that the content in this schema refers to a list of resources for each resource class. Since a list is ordered by definition, it clearly identifies specific resources by order. This is important when applying filters or modifiers, since they are applied per-resource in the list. One can for instance apply attribute A to first resource of class C, while not applying it or applying another attribute B to second resources of the same class. As a result, complex forms can be used to filter and modify a list of resources required by a testsuite. On the other hand, it's also important to note that lists for simple or scalar types are currently being treated as unordered sets, which mean combination of filters or modifiers apply differently. In the future, it may be possible to have both behaviors for scalar/simple types by using also the YAML 'set' type in {app-name}. //TODO: update this list and use a table for each resource type in its own object section //// These kinds of resources and their attributes are known: 'ip_address':: List of IP addresses to run osmo-nitb instances on. The main unit typically has a limited number of such IP addresses configured, which the connected BTS models can see on their network. 'addr'::: IPv4 address of the local interface. 'bts':: List of available BTS hardware. 'label'::: human readable label for your own reference 'type'::: which way to launch this BTS, one of - 'osmo-bts-sysmo' - 'osmo-bts-trx' - 'osmo-bts-octphy' - 'ipa-nanobts' 'ipa_unit_id'::: ip.access unit id to be used by the BTS, written into BTS and BSC config. 'addr'::: Remote IP address of the BTS for BTS like sysmoBTS, and local IP address to bind to for locally run BTS such as osmo-bts-trx. 'band'::: GSM band that this BTS shoud use (*TODO*: allow multiple bands). One of: - 'GSM-1800' - 'GSM-1900' - (*TODO*: more bands) 'trx_list'::: Specific TRX configurations for this BTS. There should be as many of these as the BTS has TRXes. (*TODO*: a way to define >1 TRX without special configuration for them.) 'hw_addr':::: Hardware (MAC) address of the TRX in the form of '11:22:33:44:55:66', only used for osmo-bts-octphy. (*TODO*: and nanobts??) 'net_device':::: Local network device to reach the TRX's 'hw_addr' at, only used for osmo-bts-octphy. Example: 'eth0'. 'nominal_power':::: Nominal power to be used by the TRX. 'max_power_red':::: Max power reduction to apply to the nominal power of the TRX. 'arfcn':: List of ARFCNs to use for running BTSes, which defines the actual RF frequency bands used. 'arfcn'::: ARFCN number, see e.g. https://en.wikipedia.org/wiki/Absolute_radio-frequency_channel_number (note that the resource type 'arfcn' contains an item trait also named 'arfcn'). 'band'::: GSM band name to use this ARFCN for, same as for 'bts:band' above. 'modem':: List of modems reachable via ofono and information on the inserted SIM card. (Note: the MSISDN is allocated dynamically in test scripts). 'label'::: Human readable label for your own reference, which also appears in logs. 'path'::: Ofono's path for this modem, like '/modemkind_99'. 'imsi'::: IMSI of the inserted SIM card, like '"123456789012345"'. 'ki'::: 16 byte authentication/encryption KI of the inserted SIM card, in hexadecimal notation (32 characters) like + '"00112233445566778899aabbccddeeff"'. 'auth_algo'::: Authentication algorithm to be used with the SIM card. One of: - 'none' - 'xor' - 'comp128v1' 'ciphers'::: List of ciphers that this modem supports, used to match requirements in suites or scenarios. Any combination of: - 'a5_0' - 'a5_1' - 'a5_2' - 'a5_3' - 'a5_4' - 'a5_5' - 'a5_6' - 'a5_7' 'features'::: List of features that this modem supports, used to match requirements in suites or scenarios. Any combination of: - 'sms' - 'gprs' - 'voice' - 'ussd' //// [[schema_want]] ==== Schema 'want' This schema is basically the same as the <> one, but with an extra 'times' attribute for each resource item. All 'times' attributes are expanded before matching. For example, if a 'suite.conf' requests two BTS, one may enforce that both BTS should be of type 'osmo-bts-sysmo' in these ways: ---- resources: bts: - type: osmo-bts-sysmo - type: osmo-bts-sysmo ---- or alternatively, ---- resources: bts: - times: 2 type: osmo-bts-sysmo ---- [[schema_config]] ==== Schema 'config' This schema defines all the attributes which can be used by object classes or tests during test execution. The main difference between this schema and the <> schema is that the former contains configuration to be applied globally for all objects being used, while the later applies attributes to a specific object in the list of allocated resources. This schema hence allows setting attributes for objects which are not allocated as resources and hence not directly accessible through scenarios, like a BSC or an iperf3 client. This schema is built dynamically at runtime from content registered by: - object classes registering their own attributes - test suite registering their own attributes through <> and tests being able to later retrieve them through 'testenv' API. [[schema_all]] ==== Schema 'all' This schema is basically an aggregated namespace for <> schema and <> schema, and is the one used by <> and <> files. It contains these main element sections::: [[schema_all_sec_resources]] - Section 'resources': Contains a set of elements validated with <> schema. In <> it is used to construct the list of requested resources. In <>, it is used to inject attributes to the initial <> _resources_ section and hence further restrain it. [[schema_all_sec_modifiers]] - Section 'modifiers': Both in <> and <>, values presented in here are injected into the content of the <> after _resource_ allocation, hereby overwriting attributes passed to the object class instance managing the specific _resource_ (matches by resource type and list position). Since it is combined with the content of <>, it is clear that the <> is used to validate this content. [[schema_all_sec_config]] - Section 'config': Contains configuration attributes for {app-name} object classes which are not _resources_, and hence cannot be configured with <>. They can overwrite values provided in the <> file. Content in this section follows the <> schema. //TODO: defaults.timeout should be change in code to be config.test_timeout or similar //TODO: 'config' should be split into its own schema and validate defaults.conf === Example Setup {app-name} comes with an example official setup which is the one used to run Osmocom's setup. There are actually two different available setups: a production one and an RnD one, used to develop {app-name} itself. These two set ups share mostly all configuration, main difference being the <> file being used. All {app-name} related configuration for that environment is publicly available in 'osmo-gsm-tester.git' itself: - <>: Available Available under 'sysmocom/', with its paths already configured to take required bits from inside the git repository directory. - <>: Available under 'sysmocom/suites/' - <>: Available under 'sysmocom/scenarios/' - <>: Available under 'sysmocom/' as 'resources.conf.prod' for Production setup and as 'resources.conf.rnd' for the RnD setup. One must use a symbolic link to have it available as 'resources.conf'. There are also small sample setups under the 'doc/examples/' directory to showcase how to set up different types of networks. ==== Typical Invocations Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for the Osmocom core network as well as for the Osmocom based BTS models. To create such a set of binaries, see <>. Examples for launching test trials: - Run the default suites (see <>) on a given set of binaries from 'path/to/my-trial' with <> available under a standard path: ---- osmo-gsm-tester.py path/to/my-trial ---- - Same as above, but run an explicit choice of 'suite:scenario' combinations: ---- osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts ---- - Same as above, but run one 'suite:scenario1+scenario2' combination, setting log level to 'debug' and enabling logging of full python tracebacks, and also only run just the 'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure: ---- osmo-gsm-tester.py path/to/my-trial -s sms:sysmo+foobar -l dbg -T -t mo_mt ---- - Same as above, but tell {app-name} to read the 'main.conf' in specific directory 'path/to/my/main.conf': ---- osmo-gsm-tester.py -c path/to/my/main.conf path/to/my-trial -s sms:sysmo+foobar -l dbg -T -t mo_mt ---- A test script may also be run step-by-step in a python debugger, see <>.