cosmetic: OsmoGSMTester: fix dir name

All other subdirs are without dashes, but I added Osmo-GSM-Tester with dashes.
Comply with the naming scheme and remove dashes.

Change-Id: I36d0c94dde5deffba04b27436ae499a42c519bec
This commit is contained in:
Neels Hofmeyr 2017-04-23 15:04:19 +02:00 committed by Neels Hofmeyr
parent 5d666c46fb
commit c0827c4287
8 changed files with 599 additions and 0 deletions

12
doc/manuals/Makefile Normal file
View File

@ -0,0 +1,12 @@
TOPDIR := ..
ASCIIDOCS := osmo-gsm-tester-manual
include $(TOPDIR)/build/Makefile.asciidoc.inc
include $(TOPDIR)/build/Makefile.inc
osmo-gsm-tester-manual.pdf: chapters/*.adoc
clean:
-rm -rf $(cleanfiles)
-rm osmo-gsm-tester-manual__*.svg
-rm osmo-gsm-tester-manual__*.png

View File

@ -0,0 +1,225 @@
== Configuration
The osmo-gsm-tester looks for configuration files in various standard
directories in this order:
- '$HOME/.config/osmo-gsm-tester/'
- '/usr/local/etc/osmo-gsm-tester/'
- '/etc/osmo-gsm-tester/'
The config location can also be set by an environment variable
'$OSMO_GSM_TESTER_CONF', which then overrides the above locations.
The osmo-gsm-tester expects to find the following configuration files in a
configuration directory:
- 'paths.conf'
- 'resources.conf'
- 'default-suites.conf' (optional)
- 'defaults.conf' (optional)
=== Format: YAML, and its Drawbacks
The general configuration format used is YAML. The stock python YAML parser
does have several drawbacks: too many complex possibilities and alternative
ways of formatting a configuration, but at the time of writing seems to be the
only widely used configuration format that offers a simple and human readable
formatting as well as nested structuring. It is recommended to use only the
exact YAML subset seen in this manual in case the osmo-gsm-tester should move
to a less bloated parser in the future.
Careful: if a configuration item consists of digits and starts with a zero, you
need to quote it, or it may be interpreted as an octal notation integer! Please
avoid using the octal notation on purpose, it is not provided intentionally.
[[paths_conf]]
=== 'paths.conf'
The 'paths.conf' file defines where to store the global state (of reserved
resources) and where to find suite and scenario definitions.
Any relative paths found in a 'paths.conf' file are interpreted as relative to
the directory of that 'paths.conf' file.
Example:
----
state_dir: '/var/run/osmo-gsm-tester'
suites_dir: './suites'
scenarios_dir: './scenarios'
----
If you would like to set up several separate 'paths.conf' files (not typical),
note that the 'state_dir' is used to reserve resources, which only works when
all configurations that share resources also use the same 'state_dir'.
[[resources_conf]]
=== 'resources.conf'
The 'resources.conf' file defines which hardware is connected to the main unit,
as well as which limited configuration items (like IP addresses or ARFCNs)
should be used.
These resources are allocated dynamically and are not configured explicitly:
- MSISDN: phone numbers are dealt out to test scripts in sequence on request.
A 'resources.conf' is structured as a list of items for each resource type,
where each item has one or more settings -- for an example, see
<<resources_conf_example>>.
These kinds of resource are known:
'nitb_iface'::
List of IP addresses to run osmo-nitb instances on. The main unit
typically has a limited number of such IP addresses configured, which
the connected BTS models can see on their network.
'addr':::
IPv4 address of the local interface.
'bts'::
List of available BTS hardware.
'label':::
human readable label for your own reference
'type':::
which way to launch this BTS, one of
- 'osmo-bts-sysmo'
- 'osmo-bts-trx'
- 'osmo-bts-octphy'
- 'ipa-nanobts'
'addr':::
remote IP address of the BTS, used to start the BTS and tell it where
to find the OsmoNITB.
'band':::
GSM band that this BTS shoud use (*TODO*: allow multiple bands). One of:
- 'GSM-1800'
- 'GSM-1900'
- (*TODO*: more bands)
'trx_list':::
Specific TRX configurations for this BTS. There should be as many of
these as the BTS has TRXes. (*TODO*: a way to define >1 TRX without
special configuration for them.)
'hw_addr'::::
Hardware (MAC) address of the TRX in the form of '11:22:33:44:55:66',
only used for osmo-bts-octphy. (*TODO*: and nanobts??)
'net_device'::::
Local network device to reach the TRX's 'hw_addr' at, only used for
osmo-bts-octphy.
'arfcn'::
List of ARFCNs to use for running BTSes, which defines the actual RF
frequency bands used.
'arfcn':::
ARFCN number, see e.g.
https://en.wikipedia.org/wiki/Absolute_radio-frequency_channel_number
(note that the resource type 'arfcn' contains an item trait also named
'arfcn')
'band':::
GSM band name to use this ARFCN for, same as for 'bts:band' above.
'modem'::
List of modems reachable via ofono and information on the inserted SIM
card. (Note: the MSISDN is allocated dynamically in test scripts)
'label':::
human readable label for your own reference
'path':::
ofono's path for this modem, like '/modemkind_99'
'imsi':::
IMSI of the inserted SIM card, like '"123456789012345"'
'ki':::
16 byte authentication/encryption KI of the inserted SIM card, in
hexadecimal notation (32 characters) like +
'"00112233445566778899aabbccddeeff"' (*TODO*: authentication algorithm,
currently always comp128v1)
Side note: at first sight it might make sense to the reader to rather structure
e.g. the 'nitb_iface' or 'arfcn' configuration as +
'"arfcn: GSM-1800: [512, 514, ...]"', +
but the more verbose format is chosen to stay consistent with the general
structure of resource configurations, which the resource allocation algorithm
uses to resolve required resources according to their traits. These
configurations look cumbersome because they exhibit only one trait / a trait
that is repeated numerous times. No special notation for these cases is
available (yet).
[[default_suites]]
=== 'default-suites.conf' (optional)
The 'default-suites.conf' file contains a list of 'suite:scenario+scenario+...'
combination strings as defined by the 'osmo-gsm-tester.py -s' commandline
option. If invoking the 'osmo-gsm-tester.py' without any suite definitions, the
'-s' arguments are taken from this file instead. Each of these suite + scenario
combinations is run in sequence.
A suite name must match the name of a directory in the 'suites_dir' as defined
by 'paths.conf'.
A scenario name must match the name of a configuration file in the
'scenarios_dir' as defined by 'paths.conf' (optionally without the '.conf'
suffix).
For 'paths.conf', see <<paths_conf>>.
Example of a 'default-suites.conf' file:
----
- sms:sysmo
- voice:sysmo+tch_f
- voice:sysmo+tch_h
- voice:sysmo+dyn_ts
- sms:trx
- voice:trx+tch_f
- voice:trx+tch_h
- voice:trx+dyn_ts
----
=== 'defaults.conf' (optional)
Each binary run by osmo-gsm-tester, e.g. 'osmo-nitb' or 'osmo-bts-sysmo',
typically has a configuration file template that is populated with values for a
trial run.
Some of these values are provided by the 'resources.conf' from the allocated
resource(s), but not all values can be populated this way: some osmo-nitb
configuration values like the network name, encryption algorithm or timeslot
channel combinations are in fact not resources (only the nitb's interface
address is). These additional settings may be provided by the scenario
configurations, but in case the provided scenarios leave some values unset,
they are taken from this 'defaults.conf'. (A 'scenario.conf' providing a
similar setting always has precedence over the values given in a
'defaults.conf').
*TODO* better match this format with 'resources.conf'?
Example of a 'defaults.conf':
----
nitb:
net:
mcc: 1
mnc: 868
short_name: osmo-gsm-tester
long_name: osmo-gsm-tester
auth_policy: closed
encryption: a5 0
nitb_bts:
location_area_code: 23
base_station_id_code: 63
stream_id: 255
osmobsc_bts_type: sysmobts
trx_list:
- max_power_red: 22
arfcn: 868
timeslot_list:
- phys_chan_config: CCCH+SDCCH4
- phys_chan_config: SDCCH8
- phys_chan_config: TCH/F_TCH/H_PDCH
- phys_chan_config: TCH/F_TCH/H_PDCH
- phys_chan_config: TCH/F_TCH/H_PDCH
- phys_chan_config: TCH/F_TCH/H_PDCH
- phys_chan_config: TCH/F_TCH/H_PDCH
- phys_chan_config: TCH/F_TCH/H_PDCH
----
*TODO*: detailed descriptions

View File

@ -0,0 +1,4 @@
[[debugging]]
== Debugging
*TODO*: describe how to invoke 'ipdb3' and step into a suite's test script

View File

@ -0,0 +1,269 @@
== Introduction with Examples
The osmo-gsm-tester is software to run automated tests of real GSM hardware,
foremost to verify that ongoing Osmocom software development continues to work
with various BTS models, while being flexibly configurable and extendable.
A 'main unit' (general purpose computer) is connected via ethernet and/or USB to
any number of BTS models and to any number of GSM modems via USB. The modems
and BTS instances' RF transceivers are typically wired directly to each other
via RF distribution chambers to bypass the air medium and avoid disturbing real
production cellular networks. Furthermore, the setup may include adjustable RF
attenuators to model various distances between modems and base stations.
The osmo-gsm-tester software runs on the main unit to orchestrate the various
GSM hardware and run predefined test scripts. It typically receives binary
packages from a jenkins build service. It then automatically configures and
launches an Osmocom core network on the main unit and sets up and runs BTS
models as well as modems to form a complete ad-hoc GSM network. On this setup,
predefined test suites, combined with various scenario definitions, are run to
verify stability of the system.
The osmo-gsm-tester is implemented in Python (version 3). It uses the ofono
daemon to control the modems connected via USB. BTS software is either run
directly on the main unit (e.g. for osmo-bts-trx, osmo-bts-octphy), run via SSH
(e.g. for a sysmoBTS) or assumed to run on a connected BTS model (e.g. for
ip.access nanoBTS).
.Typical osmo-gsm-tester setup
[graphviz]
----
digraph G {
rankdir=LR;
jenkins
subgraph cluster_gsm_hardware {
label = "GSM Hardware";
style=dotted
modem0 [shape=box label=Modem]
modem1 [shape=box label=Modem]
modem2 [shape=box label=Modem]
osmo_bts_sysmo [label="sysmocom sysmoBTS\nrunning osmo-bts-sysmo" shape=box]
B200 [label="Ettus B200" shape=box]
octphy [label="Octasic octphy BTS" shape=box]
nanoBTS [label="ip.access nanoBTS" shape=box]
rf_distribution [label="RF distribution"]
{modem0 modem1 modem2 osmo_bts_sysmo B200 octphy nanoBTS}->rf_distribution [dir=both arrowhead="curve" arrowtail="curve"]
}
subgraph cluster_main_unit {
label = "Main Unit"
osmo_gsm_tester [label="Osmo-GSM-Tester\ntest suites\n& scenarios"]
ofono [label="ofono daemon"]
OsmoNITB
osmo_bts_trx [label="osmo-bts-trx"]
osmo_bts_octphy [label="osmo-bts-octphy"]
}
jenkins->osmo_gsm_tester [label="trial\n(binaries)"]
osmo_gsm_tester->jenkins [label="results"]
ofono->modem0 [label="USB"]
ofono->modem1 [label="USB"]
ofono->modem2 [label="USB"]
osmo_gsm_tester-> {OsmoNITB osmo_bts_trx osmo_bts_octphy}
osmo_gsm_tester-> osmo_bts_sysmo [taillabel="SSH"]
osmo_gsm_tester-> ofono [taillabel="DBus"]
osmo_bts_trx->B200 [label="USB"]
osmo_bts_octphy->octphy [label="raw eth"]
{osmo_bts_sysmo B200 octphy nanoBTS}->OsmoNITB [label="eth"]
}
----
=== Typical Test Script
A typical single test script (part of a suite) may look like this:
----
#!/usr/bin/env python3
from osmo_gsm_tester.test import *
print('use resources...')
nitb = suite.nitb()
bts = suite.bts()
ms_mo = suite.modem()
ms_mt = suite.modem()
print('start nitb and bts...')
nitb.bts_add(bts)
nitb.start()
sleep(1)
assert nitb.running()
bts.start()
nitb.subscriber_add(ms_mo)
nitb.subscriber_add(ms_mt)
ms_mo.connect(nitb)
ms_mt.connect(nitb)
wait(nitb.subscriber_attached, ms_mo, ms_mt)
sms = ms_mo.sms_send(ms_mt.msisdn)
wait(ms_mt.sms_received, sms)
----
=== Resource Resolution
- A global configuration defines which hardware is connected to the
osmo-gsm-tester main unit.
- Each suite contains a number of test scripts. The amount of resources a test
may use is defined by the test suite's 'suite.conf'.
- Which specific modems, BTS models, NITB IP addresses etc. are made available
to a test run is typically determined by a combination of scenario
configurations -- or picked automatically if not.
[[resources_conf_example]]
=== Typical 'resources.conf'
A global configuration of hardware may look like below; for details, see
<<resources_conf>>.
----
nitb_iface:
- addr: 10.42.42.1
- addr: 10.42.42.2
- addr: 10.42.42.3
bts:
- label: sysmoBTS 1002
type: osmo-bts-sysmo
addr: 10.42.42.114
band: GSM-1800
- label: octBTS 3000
type: osmo-bts-octphy
addr: 10.42.42.115
band: GSM-1800
trx_list:
- hw_addr: 00:0c:90:32:b5:8a
net_device: eth0.2342
- label: Ettus B210
type: osmo-bts-trx
addr: 10.42.42.116
band: GSM-1800
- label: nanoBTS 1900
type: nanobts
addr: 10.42.42.190
band: GSM-1900
trx_list:
- hw_addr: 00:02:95:00:41:b3
arfcn:
- arfcn: 512
band: GSM-1800
- arfcn: 514
band: GSM-1800
- arfcn: 540
band: GSM-1900
- arfcn: 542
band: GSM-1900
modem:
- label: m7801
path: '/wavecom_0'
imsi: 901700000007801
ki: D620F48487B1B782DA55DF6717F08FF9
- label: m7802
path: '/wavecom_1'
imsi: 901700000007802
ki: 47FDB2D55CE6A10A85ABDAD034A5B7B3
- label: m7803
path: '/wavecom_2'
imsi: 901700000007803
ki: ABBED4C91417DF710F60675B6EE2C8D2
----
=== Typical 'suites/*/suite.conf'
The configuration that reserves a number of resources for a test suite may look
like this:
----
resources:
nitb_iface:
- times: 1
bts:
- times: 1
modem:
- times: 2
----
It may also request e.g. specific BTS models, but this is typically left to
scenario configurations.
=== Typical 'scenarios/*.conf'
For a suite as above run as-is, any available resources are picked. This may be
combined with any number of scenario definitions to constrain which specific
resources should be used, e.g.:
----
resources:
bts:
- type: osmo-bts-sysmo
----
Which 'nitb_iface' or 'modem' is used in particular doesn't really matter, so
it can be left up to the osmo-gsm-tester to pick these automatically.
Any number of such scenario configurations can be combined in the form
'<suite_name>:<scenario>+<scenario>+...', e.g. 'my_suite:sysmo+tch_f+amr'.
=== Typical Invocations
Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for
the Osmocom core network as well as for the Osmocom based BTS models. To create
such a set of binaries, see <<trials>>.
Examples for launching test trials:
- Run the default suites (see <<default_suites>>) on a given set of binaries:
----
osmo-gsm-tester.py path/to/my-trial
----
- Run an explicit choice of 'suite:scenario' combinations:
----
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts
----
- Run one 'suite:scenario' combination, setting log level to 'debug' and
enabling logging of full python tracebacks, and also only run just the
'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure:
----
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -l dbg -T -t mo_mt
----
A test script may also be run step-by-step in a python debugger, see
<<debugging>>.
=== Resource Reservation for Concurrent Trials
While a test suite runs, the used resources are noted in a global state
directory in a reserved-resources file. This way, any number of trials may be
run consecutively without resource conflicts. Any test trial will only use
resources that are currently not reserved by any other test suite. The
reservation state is human readable.
The global state directory is protected by a file lock to allow access by
separate processes.
Also, the binaries from a trial are never installed system-wide, but are run
with a specific 'LD_LIBRARY_PATH' pointing at the trial's 'inst', so that
several trials can run consecutively without conflicting binary versions.
Once a test suite run is complete, all its reserved resources are torn down (if
the test scripts have not done so already), and the reservations are released
automatically.
If required resources are unavailable, the test trial fails. For consecutive
test trials, a test run needs to either wait for resources to become available,
or test suites need to be scheduled to make sense. (*<- TODO*)

View File

@ -0,0 +1,3 @@
== Test API
*TODO*

View File

@ -0,0 +1,21 @@
[[trials]]
== Trial: Binaries to be Tested
A trial is a set of pre-built binaries to be tested. They are typically built
by jenkins using the build scripts found in osmo-gsm-tester's source in the
'contrib/' dir.
A trial comes in the form of a directory containing a number of '*.tgz' tar
archives as well as a 'checksums.md5' file to verify the tar archives'
integrity.
When the osmo-gsm-tester is invoked to run on such a trial directory, it will
create a sub directory named 'inst' and unpack the tar archives into it.
For each test run on this trial, a new subdirectory in the trial dir is
created, named in the form of 'run.<timestamp>'. A symbolic link 'last-run'
will point at the most recently created run dir. This run dir will accumulate
the rendered configuration files used for the trial run as well as a test log
(<- *TODO*) and stdout and stderr outputs of the binaries run for the trial.
(*TODO*->) When the test is complete, jenkins parsable XML reports for the test
run will be written to the 'run.<timestamp>' subdir.

View File

@ -0,0 +1,47 @@
<revhistory>
<revision>
<revnumber>1</revnumber>
<date>April 13, 2017</date>
<authorinitials>NH</authorinitials>
<revremark>
Initial version.
</revremark>
</revision>
</revhistory>
<authorgroup>
<author>
<firstname>Neels</firstname>
<surname>Hofmeyr</surname>
<email>nhofmeyr@sysmocom.de</email>
<authorinitials>NH</authorinitials>
<affiliation>
<shortaffil>sysmocom</shortaffil>
<orgname>sysmocom - s.f.m.c. GmbH</orgname>
<jobtitle>Senior Developer</jobtitle>
</affiliation>
</author>
</authorgroup>
<copyright>
<year>2017</year>
<holder>sysmocom - s.f.m.c. GmbH</holder>
</copyright>
<legalnotice>
<para>
Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License,
Version 1.3 or any later version published by the Free Software
Foundation; with the Invariant Sections being just 'Foreword',
'Acknowledgements' and 'Preface', with no Front-Cover Texts,
and no Back-Cover Texts. A copy of the license is included in
the section entitled "GNU Free Documentation License".
</para>
<para>
The Asciidoc source code of this manual can be found at
<ulink url="http://git.osmocom.org/osmo-gsm-manuals/">
http://git.osmocom.org/osmo-gsm-manuals/
</ulink>
</para>
</legalnotice>

View File

@ -0,0 +1,18 @@
Osmo-GSM-Tester Manual
======================
Neels Hofmeyr <nhofmeyr@sysmocom.de>
== WARNING: Work in Progress
*NOTE: The osmo-gsm-tester is still in pre-alpha stage: some parts are still
incomplete, and details will still change and move around.*
include::chapters/intro.adoc[]
include::chapters/config.adoc[]
include::chapters/trial.adoc[]
include::chapters/test_api.adoc[]
include::chapters/debugging.adoc[]