throughput tests require the UE object to have the num_carriers
member. For Android UEs, set it to 1 by default.
Change-Id: I297d24bb578a7db0eb7629be4f2cc0d54624467d
we've provided only all_log_level so far but sometimes it's needed
to select the level per layer. This patch adds the ability
to do so for the NAS layer in the UE.
Change-Id: Iab2bce65e8af81f6d344849c97952e6441cb2846
srsENB now supports PDCP discard so we set it to a sane value.
Also enable status reporting for UL.
Change-Id: Ib6ad0ff57b95a1f220f87c1ede0f6eacc5fb21ff
with many parallel jobs running on one machine we should
relax the timeout a bit to avoid false positives.
Change-Id: Ib32a07559bd5a4b0165f6d3431c5ac57cc050f44
we previously mixed component specific and component agnostic APIs
(stdout vs. log file for example) for setting and retrieving KPI.
This patch propose to use a single abstract get_kpis() method for
all components that can be enriched with component-specific
stuff as desired.
In the case of srsLTE blocks, the main implementation will
remain in srslte_common() and is shared among srsENB/srsUE/srsEPC.
The KPI analyzer in srslte_common() extract and also manages
all three KPI sources (log, csv and stdout) independently.
In addition to the get_kpis() method that always returns a flat
dictionary, it also exposes get_kpi_tree() that return
a dict of KPI dicts that will be used for the Junit.xml generation.
Change-Id: I4bacc6b8a0cb92a581edfb947100b57022265265
osmo-bts.git ae09c8acb4aa93284cdb44f8bbdc14533dc4fa52 modified code to
avoid calling pcu_tx_info_ind() if pcu socket was not connected. Since
osmo-gsm-tester relied on log message sent from within that funtion to
find out whether BTS was able to handle PCUIF connections, that log line
is not printed anymore and hence the function ready_for_pcu() fails to
ever return true.
As a result, gprs tests for sysmo and oc2g bts types always fail with a
timeout.
Other BTS types are not affected (such as osmo-bts-trx) because in there
we simply check whether the osmo-bts process is running.
Change-Id: Ibde4b8b27b8300b4b9197c0f8d82a6ccbf3b0522
we've detected a possible race condition during the Msg3
transmission that caused the thread that sets the Msg3 grant
to be delayed. The PHY worker that executed TTI+2 finished
by that time already and didn't even see the pending UL grant.
This is issue is more likely to happen on loaded system,
for example when running parallel ZMQ jobs. We therefore decided
to reduce the number of parallel PHY workers to 2 so the issue
is circumvented.
Change-Id: Ibdb42a1705d87b6d343201458c8fe397398802bc
we've already incrased the t3413 value in 9c7fd618d2
but realized it may still be too low in some cases.
we set it to 12s now.
Change-Id: I9c3c59c0ff8f539a44b420f8ba31eea016c211ce
This patch enables setting cipher and integrity algorithms
in Amarisoft eNB and srsENB via scenario files. If no
settings are defined following defaults are applied:
- Cipher algorithm: EEA0, EEA2, EEA1
- Integrity algorithm: EIA2, EIA1, EIA0
Example of setting cipher algorithms:
- 4g:srsue-rftype@uhd+srsenb-rftype@uhd+mod-enb-cipher@eea1+mod-enb-cipher@eea0+mod-enb-nprb@6
Change-Id: I595206b7d49016fb6d0aec175c828d9537c53886
implement as noop for Amarisoft eNB, srsENB will send q+Enter to stdin,
which is implemented in class srslte_common()
Change-Id: Ide606e1a6b523997215aa2fa39d4d56ae1f49181
this patch adds the stdout counter to count events happening
on the stdout (known from the UE already) to the common
process class so they can also be used from the eNB (and other objects)
In addition, we add a PRACH counter to be used for tests.
Change-Id: I434f072b8aa6f4dce9f90889c6b40832f6798ff8
To expand the test capacities we would like to introduce
Android UEs as new modems. Currently the following tests
are supported:
- Ping
- iPerf3 DL/UL
- RRC Mobile MT Ping
In the following is a small description.
Prerequisites:
- Android UE
- Rooted (Ping, iPerf, RRC Idle MT Ping)
- Qualcomm baseband with working diag_mdlog (RRC Idle MT Ping)
- iPerf3
- Dropbear
- OGT Slave Unit
- Android SDK Platform-Tools
(https://developer.android.com/studio/releases/platform-tools#downloads)
- Pycrate (https://github.com/P1sec/pycrate)
- SCAT
clone https://github.com/bedrankara/scat/ & install dependencies
checkout branch ogt
symlink scat (ln -s ~/scat/scat.py /usr/local/bin/scat)
Infrastructure explaination:
The Android UEs are connected to the OGT Units via USB. We
activate tethering and set up a SSH server (with Dropbear).
We chose tethering over WiFi to have a more stable route
for the ssh connection. We forward incoming connections to
the OGT unit hosting the Android UE(s) on specific ports
to the UEs via iptables. This enables OGT to issue commands
directly to the UEs. In case of local execution we use ADB
to issue commands to the AndroidUE. The set up was tested
with 5 Android UEs connected in parallel but it should be
scalable to the number of available IPs in the respective
subnet. Furthermore, we need to cross compile Dropbear
and iPerf3 to use them on the UEs. These tools have to be
added to the $PATH variable of the UEs.
Examplary set up:
In this example we have two separate OGT units (master
and slave) and two Android UEs that are connected to the
slave unit. An illustration may be found here: https://ibb.co/6BXSP2C
On UE 1:
ip address add 192.168.42.130/24 dev rndis0
ip route add 192.168.42.0/24 dev rndis0 table local_network
dropbearmulti dropbear -F -E -p 130 -R -T /data/local/tmp/authorized_keys -U 0 -G 0 -N root -A
On UE 2:
ip address add 192.168.42.131/24 dev rndis0
ip route add 192.168.42.0/24 dev rndis0 table local_network
dropbearmulti dropbear -F -E -p 131 -R -T /data/local/tmp/authorized_keys -U 0 -G 0 -N root -A
On OGT slave unit:
sudo ip link add name ogt type bridge
sudo ip l set eth0 master ogt
sudo ip l set enp0s20f0u1 master ogt
sudo ip l set enp0s20f0u2 master ogt
sudo ip a a 192.168.42.1/24 dev ogt
sudo ip link set ogt up
Now we have to manually connect to every UE from OGT Master
to set up SSH keys and verify that the setup works.
Therefore, use:
ssh -p [UE-PORT] root@[OGT SLAVE UNIT's IP]
Finally, to finish the setup procedure create the
remote_run_dir for Android UEs on the slave unit like
following:
mkdir /osmo-gsm-tester-androidue
chown jenkins /osmo-gsm-tester-androidue
Example for modem in resource.conf:
- label: mi5g
type: androidue
imsi: '901700000034757'
ki: '85E9E9A947B9ACBB966ED7113C7E1B8A'
opc: '3E1C73A29B9C293DC5A763E42C061F15'
apn:
apn: 'srsapn'
mcc: '901'
mnc: '70'
select: 'True'
auth_algo: 'milenage'
features: ['4g', 'dl_qam256', 'qc_diag']
run_node:
run_type: ssh
run_addr: 100.113.1.170
ssh_user: jenkins
ssh_addr: 100.113.1.170
ue_ssh_port: 130
adb_serial_id: '8d3c79a7'
scat_parser:
run_type: local
run_addr: 127.0.0.1
adb_serial_id: '8d3c79a7'
Example for default-suites.conf:
- 4g:ms-label@mi5g+srsenb-rftype@uhd+mod-enb-nprb@25+mod-enb-txmode@1
Change-Id: I79a5d803e869a868d4dac5e0d4c2feb38038dc5c
this executed the PRACH search in the main thread so
PHY workers and PRACH processing don't diverge in loaded
scenarios
Change-Id: I97eddb5f9ca682251ded1f73093c2d0a49462bd6
we've seen that the default value of 2s might be a bit tight.
the value is network dependent so no real default value exists.
however, 6s seems to be a good compromise.
Change-Id: Icc75ee048c2cc605ab26c4c87a039519998f7e57
it turned out that we have to reduce the MCS when using QAM256,
especially for 6 PRB as subframe 0 and 5 contains PBCH and PSS
signals, so the available REs are reduced.
The eNB scheduler now has this limitation in mind and lowers the MCS.
Change-Id: I0e38fe28002fd68c768cc8dcffcf74f4f190df02
this was causing failed tests because to achieve QAM64 rates both
eNB and UE need to support it and have it activated.
Change-Id: I599df92d69eeb56a5d44327de08f004222cff073
since ZMQ runs are not using wall clock anyway, measuring
TTI execution isn't useful, disable it therefore to avoid
misleading warnings.
Change-Id: I5c2cb0abcfce0ee67806f6611356f4d5d180541d
the order of checks needs to go from high to low, i.e. the higher
release feature (e.g. qam256) needs to be checked and set first.
in theory it should also be possible to have a CA-capable UE
that does not support QAM256, but for srsUE we announce both anyway.
Change-Id: I2fa49f0cb5d80db412a811ceeb380359c8ad67a7
It is easier to traverse debugging logs if the CTRL request and response
IDs are globally unique across all programs and tests. Before this, we
were using 0 almost everywhere.
(This is not strictly needed for correctness, since each CTRL client has
its own request ID scope; just we open fairly many separate CTRL clients
all the time in our tests.)
Change-Id: I44c51f4fb5beb6cedf98ea0d6684a24c6aa418c7
In jenkins, I still saw incidents of the entire log becoming colored
after a colored stderr snippet was printed to the log. Make absolutely
sure that no unterminated ANSI coloring is leaked.
Change-Id: Ib9ac1eea4a12d6d43ac8614491f016bbe9ca17b1
Via VTY, handover two lchans of a voice call from bts0 to bts1 and back.
New scenarios/bts1-* allow selecting various types for bts1,
complementing the already existing files for selecting bts0.
Change-Id: I0b2671304165a1aaae2b386af46fbd8b098e3bd8
First user will be the upcoming handover_2G/handover.py test in
I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Change-Id: Id799b3bb81eb9c04d13c26ff611e40363920300e
The pattern to use 'with' to keep a CTRL connection open adds indents to
every test script that wants to avoid multiple reconnections to the
CTRL. Instead, keeping a single open connection that is cleaned up on
{bsc,msc,nitb} object cleanup ensures that a) the program started up
successfully and opened a CTRL port, b) always has a CTRL open without
having to worry about it and c) keeps test scripts less
complex/indented/crufted.
(These are all current users of the OsmoCtrl API.)
Change-Id: I53fedbe569c5ccbc4b1a17dafe1f8d1bb8200b24
CTRL interface interaction was mostly inherited from the first legacy
implementation of osmo-gsm-tester, and it was a pain to look at from the
start. Now, while I'm close to the topic, I want this to improve:
Properly match a GET_REPLY/SET_REPLY to a sent GET/SET by the message
ID.
Completely drop the do_get() and do_set(), which were not useful for
correct handling of the CTRL request and response messaging. The API to
use by callers is set_var(), get_var()/get_int_var() and get_trap().
These call the internal _sendrecv() (or for TRAP only _recv())
functions. Make it so that tese work both on an already connected
OsmoCtrl, as well as one that needs to establish a (short) connection,
so that both are trivially possible:
# one CTRL connection stays open
with OsmoCtrl(...) as ctrl:
ctrl.get_var('var1')
ctrl.get_var('var2')
ctrl.get_var('var3')
and
# get_var() opens a connection, does the GET and closes again
OsmoCtrl(...).get_var('var1')
Do away with doubling the instances OsmoCtrl and e.g. OsmoBscCtrl.
Rather make OsmoBscCtrl a child class of OsmoCtrl, which means that we
no longer have bsc.ctrl().ctrl(), just bsc.ctrl().
Have VERB_* constants instead of dup'd strings.
Apply to / simplify all callers of OsmoCtrl.
Some of these changes are similar to recently added OsmoVty.
Change-Id: Id561e5a55d8057a997a8ec9e7fa6f94840194df1
Implicitly use distinct IDs for CTRL commands. This is a prerequisite
for matching GET_REPLY IDs to GET IDs, to correctly handle CTRL
responses without counting on being lucky.
Rather use 'use_id' as var name instead of overloading the 'id' keyword.
Change-Id: I698faa013eb14f88759685d50d925f5d82d80aa0
The blocking CTRL socket must have a timeout.
So far we have always been reading from the CTRL socket in a blocking
way. So far we are always lucky that we are indeed getting a response to
receive. Should the CTRL interface ever fail to answer, the tester would
be stuck infinitely.
Change-Id: I20091daf13981dc3d0d894af5a65c23f4db0471d
Jenkins does support showing ANSI colors on the web, but apparently not
in the junit results output. Strip ansi colors from report fragment
<system-out> text, to make it less annoying to read those on jenkins.
Change-Id: I656ecc23bbfd3f25bdf012c890e0c998168844d3
Allow enriching the junit output with arbitrary subtasks within a test.
The current aim is, for handover tests, to not just show that a test
failed, but to show exactly which steps worked and which didn't, e.g.:
handover.py/01_bts0_started PASSED
handover.py/02.1_ms0_attach PASSED
handover.py/02.2_ms1_attach PASSED
handover.py/02.3_subscribed_in_msc PASSED
handover.py/03_call_established PASSED
handover.py/04.1_bts1_started FAILED
In this case it is immediately obvious from looking at the jenkins
results analyzer that bts1 is the cause of the test failure, and it is
visible which parts of the test are flaky, over time.
First user Will be the upcoming handover_2G suite, in
I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Change-Id: I4ca9100b6f8db24d1f7e0a09b3b7ba88b8ae3b59
Retrieve a test's own logging. The aim is to provide logging belonging
to a given report fragment in the junit XML output, will be used by
upcoming test.report_fragment() feature.
Change-Id: Idfa0a45f3e6a18dd4fe692e81d732c70b5cffb76
In a test, I called print() on a multi-line string and saw the log
showing each line 0.2 seconds apart. redirect.stdout seems to be pretty
inefficient.
Instead, put a print() function into the testenv, to directly call log()
on the strings passed to print().
The initial idea for redirect_stdout was that we could print() in any
deeper functions called from a test script. But we have no such nested
print() anywhere, only in test scripts themselves.
As a result of this, a multi-line print() in test scripts now no longer
puts the log prefix (timestamp, test name...) and suffix (backtrace /
source position) to each single line, but prints the multiline block
between a single log prefix and suffix -- exactly like the log()
function does everywhere else.
I actually briefly implemented adding the log prefix to each separate
line everywhere, but decided that it is not a good idea: in some places
we log config file snippets and other lists, and prepending the log
prefix to each line makes pasting such a snippet from (say) a jenkins
log super cumbersome. And the log prefix (backtrace) attached on each
separate line makes multiline blocks very noisy, unreadable.
Change-Id: I0972c66b9165bd7f2b0b387e0335172849199193
test.Test() overrides name() in order to provide source line number
information. However, overriding name() is the wrong place for that, as
name() is also often used for identifying an object - when listing the
tests of a suite, the line number should not appear in the test name.
For example, the line number sometimes ends up in the test results in
jenkins, making 'foo.py' and 'foo.py:23' two distinct report items.
Instead, add a separate function Origin.src() that defaults to name(),
but specific classes can override src() if they wish to provide more
detailed information with the object name.
Override src() in Test, not name().
Use src() in backtraces.
The suite_test.ok shows that the backtracing in the log remains
unchanged, but the place where the test name is printed is corrected:
I am 'test_suite' / 'hello_world.py:23'
becomes
I am 'test_suite' / 'hello_world.py'
(Notice that "[LINENR]" in suite_test.ok is a masking of an actual
number, done within the selftest suite)
Change-Id: I0c4698fa2b3db3de777d8b6dcdcee84e433c62b7
Allow showing log lines matching specific regexes, from a specific start
point of a log.
My use case is to echo the handover related logging after an expected
handover failed, so that the reason is visible already in the console
output of a jenkins run. So far I would need to open the endless bsc log
and look up the matching place in it to get a conclusion about why a
handover failed.
Change-Id: Ib6569f7486e9d961bd79a5f24232e58d053667a1
Remove ARFCNs as a concept from resource pool, assign a fixed ARFCN to
each BTS and TRX in the resource pools.
Using ARFCNs on specific bands as resources was an idea that is hard to
implement, because specific BTS dictate selection of bands which
influences which ARFCNs can be picked. That means reserving ARFCN
resources is only possible after reserving specific BTS resources, but
the tester is currently not capable of such two-stage resolution.
Writing handover tests, I got the problem that both BTS in a scenario
attempt to use the same ARFCN.
The by far easiest solution is to assign one fixed ARFCN to each BTS and
TRX. If ever needed, a scenario modifier can still configure different
ARFCNs.
(Due to uncertainty about OC2G operation stability, I prefer to leave
OC2G on ARFCN 50, as it happened to end up being configured before this
patch.)
Change-Id: I0a6c60544226f4261f9106013478d6a27fc39f38
On non-debug log level, show something like this at the beginning of
each suite:
03:45:49.439720 tst handover:sysmo+secondbts-trx-b200: RESERVED RESOURCES for handover:
bts
sysmoBTS 1002
Ettus B200
ip_address
10.42.42.2
10.42.42.3
10.42.42.4
10.42.42.5
10.42.42.6
10.42.42.7
modem
sierra_1st
sierra_2nd
Change-Id: Ic23556eafee654c93d13c5ef405028da09bd51d7
In the end of a test suite, do not omit the passed tests. For example,
running handover against N BTS combinations, it was hard to summarize
which BTS models actually succeeded, with only the failures listed.
Besides the "FAIL" listings, now print something like this in the end:
PASS: handover:sysmo+secondbts-trx-b200 (pass: 1)
pass: handover.py (198.8 sec)
PASS: handover:sysmo+secondbts-trx-umtrx (pass: 1)
pass: handover.py (192.7 sec)
PASS: handover:trx-b200+secondbts-trx-umtrx (pass: 1)
pass: handover.py (193.1 sec)
Change-Id: Ib85a5b90e267c2ed2f844691187ecadc8939b1bb
resources.conf is already allowed to set these, but in order to let
scenarios and suites modify these values, they need to be in the schema.
Change-Id: I8e0583e9208d563c66b0bdc9f25b1b74f92403c2
Will be used in upcoming handover_2G test suite in
I0b2671304165a1aaae2b386af46fbd8b098e3bd8, which needs to verify that a
handover actually ended up on the expected lchan.
Change-Id: I03df8f3ae2ee47930eee311c7ce104c36dbb3154
The BSC's VTY port will be used to trigger manual handover, and to
retrieve a list of active lchans from the BSC, in the upcoming
handover_2G test suite, I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Change-Id: I06652db04fc9e48748f3c2196334f5352e9cc48a
To trigger manual handovers, I need a VTY interface. The non-trivial
parts of this are copied from osmo-python-tests osmo_interact_vty.py.
Will be used in the upcoming handover_2G test suite in
I0b2671304165a1aaae2b386af46fbd8b098e3bd8.
Change-Id: I7c17b143b7c690b8c4105ee7c6272670046fa91d