401 lines
11 KiB
Plaintext
401 lines
11 KiB
Plaintext
== Introduction with Examples
|
|
|
|
The osmo-gsm-tester is software to run automated tests of real GSM hardware,
|
|
foremost to verify that ongoing Osmocom software development continues to work
|
|
with various BTS models, while being flexibly configurable and extendable.
|
|
|
|
A 'main unit' (general purpose computer) is connected via ethernet and/or USB to
|
|
any number of BTS models and to any number of GSM modems via USB. The modems
|
|
and BTS instances' RF transceivers are typically wired directly to each other
|
|
via RF distribution chambers to bypass the air medium and avoid disturbing real
|
|
production cellular networks. Furthermore, the setup may include adjustable RF
|
|
attenuators to model various distances between modems and base stations.
|
|
|
|
The osmo-gsm-tester software runs on the main unit to orchestrate the various
|
|
GSM hardware and run predefined test scripts. It typically receives binary
|
|
packages from a jenkins build service. It then automatically configures and
|
|
launches an Osmocom core network on the main unit and sets up and runs BTS
|
|
models as well as modems to form a complete ad-hoc GSM network. On this setup,
|
|
predefined test suites, combined with various scenario definitions, are run to
|
|
verify stability of the system.
|
|
|
|
The osmo-gsm-tester is implemented in Python (version 3). It uses the ofono
|
|
daemon to control the modems connected via USB. BTS software is either run
|
|
directly on the main unit (e.g. for osmo-bts-trx, osmo-bts-octphy), run via SSH
|
|
(e.g. for a sysmoBTS) or assumed to run on a connected BTS model (e.g. for
|
|
ip.access nanoBTS).
|
|
|
|
.Typical osmo-gsm-tester setup
|
|
[graphviz]
|
|
----
|
|
digraph G {
|
|
rankdir=LR;
|
|
jenkins
|
|
subgraph cluster_gsm_hardware {
|
|
label = "GSM Hardware";
|
|
style=dotted
|
|
|
|
modem0 [shape=box label="Modems..."]
|
|
modem1 [shape=box label="Modems..."]
|
|
osmo_bts_sysmo [label="sysmocom sysmoBTS\nrunning osmo-bts-sysmo" shape=box]
|
|
B200 [label="Ettus B200" shape=box]
|
|
sysmoCell5K [label="sysmocom sysmoCell5000" shape=box]
|
|
octphy [label="Octasic octphy BTS" shape=box]
|
|
nanoBTS [label="ip.access nanoBTS" shape=box]
|
|
rf_distribution [label="RF distribution"]
|
|
|
|
{modem0 modem1 osmo_bts_sysmo B200 octphy nanoBTS sysmoCell5K}->rf_distribution [dir=both arrowhead="curve" arrowtail="curve"]
|
|
}
|
|
subgraph cluster_main_unit {
|
|
label = "Main Unit"
|
|
osmo_gsm_tester [label="Osmo-GSM-Tester\ntest suites\n& scenarios"]
|
|
subgraph {
|
|
rank=same
|
|
ofono [label="ofono daemon"]
|
|
osmo_trx [label="osmo-trx"]
|
|
osmo_bts_trx [label="osmo-bts-trx"]
|
|
osmo_bts_octphy [label="osmo-bts-octphy"]
|
|
OsmoNITB [label="BSC + Core Network\n(Osmo{NITB,MSC,BSC,HLR,...})"]
|
|
}
|
|
}
|
|
|
|
|
|
jenkins->osmo_gsm_tester [label="trial\n(binaries)"]
|
|
osmo_gsm_tester->jenkins [label="results"]
|
|
ofono->{modem0 modem1} [label="USB"]
|
|
osmo_gsm_tester->{OsmoNITB osmo_bts_trx osmo_bts_octphy}
|
|
osmo_gsm_tester->osmo_bts_sysmo [taillabel="SSH"]
|
|
osmo_gsm_tester->ofono [taillabel="DBus"]
|
|
osmo_trx->B200 [label="USB"]
|
|
osmo_bts_trx->{osmo_trx sysmoCell5K} [dir=both label="UDP"]
|
|
osmo_bts_octphy->octphy [label="raw eth"]
|
|
{osmo_bts_sysmo nanoBTS}->OsmoNITB [label="IP"]
|
|
{B200 octphy}->OsmoNITB [label="eth" style=invis]
|
|
{osmo_bts_trx osmo_bts_octphy}->OsmoNITB
|
|
}
|
|
----
|
|
|
|
.Example of how to select resources and configurations: scenarios may pick specific resources (here BTS and ARFCN), remaining requirements are picked as available (here two modems and a NITB interface)
|
|
[graphviz]
|
|
----
|
|
digraph G {
|
|
rankdir=TB;
|
|
|
|
suite_scenarios [label="Suite+Scenarios selection\nsms:sysmo+band1800"]
|
|
|
|
subgraph {
|
|
rank=same;
|
|
suite
|
|
scenarios
|
|
}
|
|
|
|
subgraph cluster_scenarios {
|
|
label = "Scenarios";
|
|
u_sysmoBTS [label="Scenario: sysmo\nbts: type: osmo-bts-sysmo"]
|
|
u_trx [label="Scenario: trx\nbts: type: osmo-bts-trx"]
|
|
u_arfcn [label="Scenario: band1800\narfcn: band: GSM-1800"]
|
|
}
|
|
|
|
subgraph cluster_suite {
|
|
label = "Suite: sms";
|
|
requires [label="Requirements (suite.conf):\nmodem: times: 2\nbts\nip_address\narfcn"]
|
|
subgraph cluster_tests {
|
|
label = "Test Scripts (py)";
|
|
mo_mt_sms
|
|
etc
|
|
}
|
|
}
|
|
|
|
subgraph cluster_resources {
|
|
label = "Resources";
|
|
rankdir=TB;
|
|
nitb_addr1 [label="NITB interface addr\n10.42.42.1"]
|
|
nitb_addr2 [label="NITB interface addr\n10.42.42.2"]
|
|
Modem0
|
|
Modem1
|
|
Modem2
|
|
sysmoBTS [label="osmo-bts-sysmo"]
|
|
osmo_bts_trx [label="osmo-bts-trx"]
|
|
arfcn1 [label="arfcn: 512\nband: GSM-1800"]
|
|
arfcn2 [label="arfcn: 540\nband: GSM-1900"]
|
|
|
|
arfcn1->arfcn2 [style=invis]
|
|
nitb_addr1->nitb_addr2 [style=invis]
|
|
Modem0 -> Modem1 -> Modem2 [style=invis]
|
|
sysmoBTS -> osmo_bts_trx [style=invis]
|
|
}
|
|
|
|
suite_scenarios -> {suite scenarios}
|
|
scenarios -> { u_arfcn u_sysmoBTS }
|
|
|
|
suite -> requires
|
|
requires -> Modem0
|
|
requires -> Modem1
|
|
requires -> sysmoBTS
|
|
requires -> arfcn1
|
|
requires -> nitb_addr1
|
|
|
|
{ u_sysmoBTS u_arfcn } -> requires [label="influences\nresource\nselection"]
|
|
}
|
|
----
|
|
|
|
.Example of a "trial" containing binaries built by a jenkins
|
|
[graphviz]
|
|
----
|
|
digraph G {
|
|
subgraph cluster_trial {
|
|
label = "Trial (binaries)"
|
|
sysmo [label="osmo-bts-sysmo.build-23.tgz\n(osmo-bts-sysmo\n+ deps\ncompiled for sysmoBTS)"]
|
|
trx [label="osmo-bts.build-5.tgz\n(osmo-bts-octphy + osmo-bts-trx\n+ deps\ncompiled for main unit)"]
|
|
nitb [label="osmo-nitb.build-42.tgz\n(osmo-nitb\n+ deps\ncompiled for main unit)"]
|
|
checksums [label="checksums.md5"]
|
|
|
|
checksums -> {sysmo trx nitb}
|
|
}
|
|
}
|
|
----
|
|
|
|
=== Typical Test Script
|
|
|
|
A typical single test script (part of a suite) may look like this:
|
|
|
|
----
|
|
#!/usr/bin/env python3
|
|
from osmo_gsm_tester.test import *
|
|
|
|
hlr = suite.hlr()
|
|
bts = suite.bts()
|
|
mgcpgw = suite.mgcpgw(bts_ip=bts.remote_addr())
|
|
msc = suite.msc(hlr, mgcpgw)
|
|
bsc = suite.bsc(msc)
|
|
stp = suite.stp()
|
|
ms_mo = suite.modem()
|
|
ms_mt = suite.modem()
|
|
|
|
hlr.start()
|
|
stp.start()
|
|
msc.start()
|
|
mgcpgw.start()
|
|
|
|
bsc.bts_add(bts)
|
|
bsc.start()
|
|
|
|
bts.start()
|
|
|
|
hlr.subscriber_add(ms_mo)
|
|
hlr.subscriber_add(ms_mt)
|
|
|
|
ms_mo.connect(msc.mcc_mnc())
|
|
ms_mt.connect(msc.mcc_mnc())
|
|
|
|
ms_mo.log_info()
|
|
ms_mt.log_info()
|
|
|
|
print('waiting for modems to attach...')
|
|
wait(ms_mo.is_connected, msc.mcc_mnc())
|
|
wait(ms_mt.is_connected, msc.mcc_mnc())
|
|
wait(msc.subscriber_attached, ms_mo, ms_mt)
|
|
|
|
sms = ms_mo.sms_send(ms_mt)
|
|
wait(ms_mt.sms_was_received, sms)
|
|
----
|
|
|
|
=== Resource Resolution
|
|
|
|
- A global configuration 'resources.conf' defines which hardware is connected to the
|
|
osmo-gsm-tester main unit.
|
|
- Each suite contains a number of test scripts. The amount of resources a test
|
|
may use is defined by the test suite's 'suite.conf'.
|
|
- Which specific modems, BTS models, NITB IP addresses etc. are made available
|
|
to a test run is typically determined by 'suite.conf' and a combination of scenario
|
|
configurations -- or picked automatically if not.
|
|
|
|
[[resources_conf_example]]
|
|
=== Typical 'resources.conf'
|
|
|
|
A global configuration of hardware may look like below; for details, see
|
|
<<resources_conf>>.
|
|
|
|
----
|
|
ip_address:
|
|
- addr: 10.42.42.2
|
|
- addr: 10.42.42.3
|
|
- addr: 10.42.42.4
|
|
- addr: 10.42.42.5
|
|
- addr: 10.42.42.6
|
|
|
|
bts:
|
|
- label: sysmoBTS 1002
|
|
type: osmo-bts-sysmo
|
|
ipa_unit_id: 1
|
|
addr: 10.42.42.114
|
|
band: GSM-1800
|
|
ciphers:
|
|
- a5_0
|
|
- a5_1
|
|
- a5_3
|
|
|
|
- label: Ettus B200
|
|
type: osmo-bts-trx
|
|
ipa_unit_id: 6
|
|
addr: 10.42.42.50
|
|
band: GSM-1800
|
|
launch_trx: true
|
|
ciphers:
|
|
- a5_0
|
|
- a5_1
|
|
|
|
- label: sysmoCell 5000
|
|
type: osmo-bts-trx
|
|
ipa_unit_id: 7
|
|
addr: 10.42.42.51
|
|
band: GSM-1800
|
|
trx_remote_ip: 10.42.42.112
|
|
ciphers:
|
|
- a5_0
|
|
- a5_1
|
|
|
|
- label: OCTBTS 3500
|
|
type: osmo-bts-octphy
|
|
ipa_unit_id: 8
|
|
addr: 10.42.42.52
|
|
band: GSM-1800
|
|
trx_list:
|
|
- hw_addr: 00:0c:90:2e:80:1e
|
|
net_device: eth1
|
|
- hw_addr: 00:0c:90:2e:87:52
|
|
net_device: eth1
|
|
|
|
arfcn:
|
|
- arfcn: 512
|
|
band: GSM-1800
|
|
- arfcn: 514
|
|
band: GSM-1800
|
|
- arfcn: 516
|
|
band: GSM-1800
|
|
- arfcn: 546
|
|
band: GSM-1900
|
|
- arfcn: 548
|
|
band: GSM-1900
|
|
|
|
modem:
|
|
- label: sierra_1
|
|
path: '/sierra_1'
|
|
imsi: '901700000009031'
|
|
ki: '80A37E6FDEA931EAC92FFA5F671EFEAD'
|
|
auth_algo: 'xor'
|
|
ciphers:
|
|
- a5_0
|
|
- a5_1
|
|
features:
|
|
- 'sms'
|
|
- 'voice'
|
|
|
|
- label: gobi_0
|
|
path: '/gobi_0'
|
|
imsi: '901700000009030'
|
|
ki: 'BB70807226393CDBAC8DD3439FF54252'
|
|
auth_algo: 'xor'
|
|
ciphers:
|
|
- a5_0
|
|
- a5_1
|
|
features:
|
|
- 'sms'
|
|
----
|
|
|
|
=== Typical 'suites/*/suite.conf'
|
|
|
|
The configuration that reserves a number of resources for a test suite may look
|
|
like this:
|
|
|
|
----
|
|
resources:
|
|
ip_address:
|
|
- times: 1
|
|
bts:
|
|
- times: 1
|
|
modem:
|
|
- times: 2
|
|
features:
|
|
- sms
|
|
----
|
|
|
|
It may also request e.g. specific BTS models, but this is typically left to
|
|
scenario configurations.
|
|
|
|
=== Typical 'scenarios/*.conf'
|
|
|
|
For a suite as above run as-is, any available resources are picked. This may be
|
|
combined with any number of scenario definitions to constrain which specific
|
|
resources should be used, e.g.:
|
|
|
|
----
|
|
resources:
|
|
bts:
|
|
- type: osmo-bts-sysmo
|
|
----
|
|
|
|
Which 'ip_address' or 'modem' is used in particular doesn't really matter, so
|
|
it can be left up to the osmo-gsm-tester to pick these automatically.
|
|
|
|
Any number of such scenario configurations can be combined in the form
|
|
'<suite_name>:<scenario>+<scenario>+...', e.g. 'my_suite:sysmo+tch_f+amr'.
|
|
|
|
=== Typical Invocations
|
|
|
|
Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for
|
|
the Osmocom core network as well as for the Osmocom based BTS models. To create
|
|
such a set of binaries, see <<trials>>.
|
|
|
|
Examples for launching test trials:
|
|
|
|
- Run the default suites (see <<default_suites>>) on a given set of binaries:
|
|
|
|
----
|
|
osmo-gsm-tester.py path/to/my-trial
|
|
----
|
|
|
|
- Run an explicit choice of 'suite:scenario' combinations:
|
|
|
|
----
|
|
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts
|
|
----
|
|
|
|
- Run one 'suite:scenario' combination, setting log level to 'debug' and
|
|
enabling logging of full python tracebacks, and also only run just the
|
|
'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure:
|
|
|
|
----
|
|
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -l dbg -T -t mo_mt
|
|
----
|
|
|
|
A test script may also be run step-by-step in a python debugger, see
|
|
<<debugging>>.
|
|
|
|
=== Resource Reservation for Concurrent Trials
|
|
|
|
While a test suite runs, the used resources are noted in a global state
|
|
directory in a reserved-resources file. This way, any number of trials may be
|
|
run consecutively without resource conflicts. Any test trial will only use
|
|
resources that are currently not reserved by any other test suite. The
|
|
reservation state is human readable.
|
|
|
|
The global state directory is protected by a file lock to allow access by
|
|
separate processes.
|
|
|
|
Also, the binaries from a trial are never installed system-wide, but are run
|
|
with a specific 'LD_LIBRARY_PATH' pointing at the trial's 'inst', so that
|
|
several trials can run consecutively without conflicting binary versions. For
|
|
some specific binaries which require extra permissions (such as osmo-bts-octphy
|
|
requiring 'CAP_NET_RAW'), 'patchelf' program is used to modify the binary
|
|
'RPATH' field instead because the OS dynamic linker skips 'LD_LIBRARY_PATH' for
|
|
binaries with special permissions.
|
|
|
|
Once a test suite run is complete, all its reserved resources are torn down (if
|
|
the test scripts have not done so already), and the reservations are released
|
|
automatically.
|
|
|
|
If required resources are unavailable, the test trial fails. For consecutive
|
|
test trials, a test run needs to either wait for resources to become available,
|
|
or test suites need to be scheduled to make sense. (*<- TODO*)
|