== Introduction with Examples The osmo-gsm-tester is software to run automated tests of real GSM hardware, foremost to verify that ongoing Osmocom software development continues to work with various BTS models, while being flexibly configurable and extendable. A 'main unit' (general purpose computer) is connected via ethernet and/or USB to any number of BTS models and to any number of GSM modems via USB. The modems and BTS instances' RF transceivers are typically wired directly to each other via RF distribution chambers to bypass the air medium and avoid disturbing real production cellular networks. Furthermore, the setup may include adjustable RF attenuators to model various distances between modems and base stations. The osmo-gsm-tester software runs on the main unit to orchestrate the various GSM hardware and run predefined test scripts. It typically receives binary packages from a jenkins build service. It then automatically configures and launches an Osmocom core network on the main unit and sets up and runs BTS models as well as modems to form a complete ad-hoc GSM network. On this setup, predefined test suites, combined with various scenario definitions, are run to verify stability of the system. The osmo-gsm-tester is implemented in Python (version 3). It uses the ofono daemon to control the modems connected via USB. BTS software is either run directly on the main unit (e.g. for osmo-bts-trx, osmo-bts-octphy), run via SSH (e.g. for a sysmoBTS) or assumed to run on a connected BTS model (e.g. for ip.access nanoBTS). .Typical osmo-gsm-tester setup [graphviz] ---- digraph G { rankdir=LR; jenkins subgraph cluster_gsm_hardware { label = "GSM Hardware"; style=dotted modem0 [shape=box label=Modem] modem1 [shape=box label=Modem] modem2 [shape=box label=Modem] osmo_bts_sysmo [label="sysmocom sysmoBTS\nrunning osmo-bts-sysmo" shape=box] B200 [label="Ettus B200" shape=box] octphy [label="Octasic octphy BTS" shape=box] nanoBTS [label="ip.access nanoBTS" shape=box] rf_distribution [label="RF distribution"] {modem0 modem1 modem2 osmo_bts_sysmo B200 octphy nanoBTS}->rf_distribution [dir=both arrowhead="curve" arrowtail="curve"] } subgraph cluster_main_unit { label = "Main Unit" osmo_gsm_tester [label="Osmo-GSM-Tester\ntest suites\n& scenarios"] ofono [label="ofono daemon"] OsmoNITB osmo_bts_trx [label="osmo-bts-trx"] osmo_bts_octphy [label="osmo-bts-octphy"] } jenkins->osmo_gsm_tester [label="trial\n(binaries)"] osmo_gsm_tester->jenkins [label="results"] ofono->modem0 [label="USB"] ofono->modem1 [label="USB"] ofono->modem2 [label="USB"] osmo_gsm_tester-> {OsmoNITB osmo_bts_trx osmo_bts_octphy} osmo_gsm_tester-> osmo_bts_sysmo [taillabel="SSH"] osmo_gsm_tester-> ofono [taillabel="DBus"] osmo_bts_trx->B200 [label="USB"] osmo_bts_octphy->octphy [label="raw eth"] {osmo_bts_sysmo B200 octphy nanoBTS}->OsmoNITB [label="eth"] } ---- === Typical Test Script A typical single test script (part of a suite) may look like this: ---- #!/usr/bin/env python3 from osmo_gsm_tester.test import * print('use resources...') nitb = suite.nitb() bts = suite.bts() ms_mo = suite.modem() ms_mt = suite.modem() print('start nitb and bts...') nitb.bts_add(bts) nitb.start() sleep(1) assert nitb.running() bts.start() nitb.subscriber_add(ms_mo) nitb.subscriber_add(ms_mt) ms_mo.connect(nitb) ms_mt.connect(nitb) wait(nitb.subscriber_attached, ms_mo, ms_mt) sms = ms_mo.sms_send(ms_mt.msisdn) wait(ms_mt.sms_received, sms) ---- === Resource Resolution - A global configuration defines which hardware is connected to the osmo-gsm-tester main unit. - Each suite contains a number of test scripts. The amount of resources a test may use is defined by the test suite's 'suite.conf'. - Which specific modems, BTS models, NITB IP addresses etc. are made available to a test run is typically determined by a combination of scenario configurations -- or picked automatically if not. [[resources_conf_example]] === Typical 'resources.conf' A global configuration of hardware may look like below; for details, see <>. ---- nitb_iface: - addr: 10.42.42.1 - addr: 10.42.42.2 - addr: 10.42.42.3 bts: - label: sysmoBTS 1002 type: osmo-bts-sysmo addr: 10.42.42.114 band: GSM-1800 - label: octBTS 3000 type: osmo-bts-octphy addr: 10.42.42.115 band: GSM-1800 trx_list: - hw_addr: 00:0c:90:32:b5:8a net_device: eth0.2342 - label: Ettus B210 type: osmo-bts-trx addr: 10.42.42.116 band: GSM-1800 - label: nanoBTS 1900 type: nanobts addr: 10.42.42.190 band: GSM-1900 trx_list: - hw_addr: 00:02:95:00:41:b3 arfcn: - arfcn: 512 band: GSM-1800 - arfcn: 514 band: GSM-1800 - arfcn: 540 band: GSM-1900 - arfcn: 542 band: GSM-1900 modem: - label: m7801 path: '/wavecom_0' imsi: 901700000007801 ki: D620F48487B1B782DA55DF6717F08FF9 - label: m7802 path: '/wavecom_1' imsi: 901700000007802 ki: 47FDB2D55CE6A10A85ABDAD034A5B7B3 - label: m7803 path: '/wavecom_2' imsi: 901700000007803 ki: ABBED4C91417DF710F60675B6EE2C8D2 ---- === Typical 'suites/*/suite.conf' The configuration that reserves a number of resources for a test suite may look like this: ---- resources: nitb_iface: - times: 1 bts: - times: 1 modem: - times: 2 ---- It may also request e.g. specific BTS models, but this is typically left to scenario configurations. === Typical 'scenarios/*.conf' For a suite as above run as-is, any available resources are picked. This may be combined with any number of scenario definitions to constrain which specific resources should be used, e.g.: ---- resources: bts: - type: osmo-bts-sysmo ---- Which 'nitb_iface' or 'modem' is used in particular doesn't really matter, so it can be left up to the osmo-gsm-tester to pick these automatically. Any number of such scenario configurations can be combined in the form ':++...', e.g. 'my_suite:sysmo+tch_f+amr'. === Typical Invocations Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for the Osmocom core network as well as for the Osmocom based BTS models. To create such a set of binaries, see <>. Examples for launching test trials: - Run the default suites (see <>) on a given set of binaries: ---- osmo-gsm-tester.py path/to/my-trial ---- - Run an explicit choice of 'suite:scenario' combinations: ---- osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts ---- - Run one 'suite:scenario' combination, setting log level to 'debug' and enabling logging of full python tracebacks, and also only run just the 'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure: ---- osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -l dbg -T -t mo_mt ---- A test script may also be run step-by-step in a python debugger, see <>. === Resource Reservation for Concurrent Trials While a test suite runs, the used resources are noted in a global state directory in a reserved-resources file. This way, any number of trials may be run consecutively without resource conflicts. Any test trial will only use resources that are currently not reserved by any other test suite. The reservation state is human readable. The global state directory is protected by a file lock to allow access by separate processes. Also, the binaries from a trial are never installed system-wide, but are run with a specific 'LD_LIBRARY_PATH' pointing at the trial's 'inst', so that several trials can run consecutively without conflicting binary versions. Once a test suite run is complete, all its reserved resources are torn down (if the test scripts have not done so already), and the reservations are released automatically. If required resources are unavailable, the test trial fails. For consecutive test trials, a test run needs to either wait for resources to become available, or test suites need to be scheduled to make sense. (*<- TODO*)