Commit Graph

1205 Commits

Author SHA1 Message Date
Pau Espin 444129ee1f test: Move duration time calculation to helper function
Change-Id: Ica2e7d92f37d18a4f8832f38d9f9f3baba8be09f
2020-06-12 16:38:37 +02:00
Pau Espin 8cfb5d020f Remove all references to defaults: timeout: from suite.conf
This feature is not really implemented and maybe never was. In any case,
it makes sense to have that working per-test so we can specify different
values per test in case it's needed.

Change-Id: I3c1b95c10e974da87ec9abd25578d8bcc0bc55a3
2020-06-12 16:12:43 +02:00
Pau Espin a9bc93da99 iperf3: Fix iperf3cli.launch_sync() when configured to run more than 300 sec
process object always used timeout=300 while runnig wth launch_sync().
Let's allow replacing that value beforehand so that iperf3 can
pre-configure the process object and caller doesn't need to care about
calculating expected time.

Change-Id: I7f6c5078f648013515919aa35ebcdb3ef157b5e4
2020-06-12 15:34:32 +02:00
Andre Puschmann 7c759d9a89 srsenb_sib.conf.tmpl: add default SIB3
if SIB3 isn't transmitted, the UE won't do HO and cell reselection

Change-Id: I329a59d9d0fa3b25bc00b760a455eadbe0b7220f
2020-06-11 20:37:17 +02:00
Pau Espin ec28572e42 test: Store brief log per test and use as default report_stdout
Change-Id: I0972ba56a42c24f3d3b1bc94bcbdaca86026046a
2020-06-11 18:07:08 +02:00
Pau Espin 57e37f95e1 suites/4g: Wait for IP addr received before trying to use the link
Otherwise processes using the link like "ping" or "iperf3" may fail
because there's still no IP address assigned.

Change-Id: I28137f10a19db01fe90b24830a60342a448d1e92
2020-06-11 17:12:44 +02:00
Pau Espin a4bb6d315b testenv: Add debug log when overlaying templates dir
Change-Id: I10fdfc5b2e3bf4c96941a3fae53230131f384de9
2020-06-11 16:23:54 +02:00
Pau Espin 3b49d46464 amarisoft_enb: Properly set S1AP SCTP bind ip addr
Change-Id: I8e2b8609cb198c6d5e293cffaa0a9de99600f06d
2020-06-11 13:18:20 +02:00
Pau Espin 9e2e8d218d log: Fix OriginLoopError generated in log.Error from testenv.poll()
Change-Id: Iea80d06a30346533e4ecec338b03c72d1f1067ae
2020-06-11 12:41:35 +02:00
Pau Espin 8ea4ba2e5a ttcn3: Don't overwrite output junit file on multiple testsuites
Change-Id: I803fa8766acb4e90c4a735a420b24ef18ef6b0c9
2020-06-11 09:01:49 +00:00
Pau Espin 664e383021 EventLoop: Fix log error "Origin parent loop" during wait()
Setting the log.ctx manually is not needed anymore and it's actually
harmful since all palces where it was used, a log.Origin already in path
was being passed, causing a origin loop.

Change-Id: I0511b9f7bc59e3c7f2269ff3155d0c95db58d063
2020-06-10 19:50:12 +02:00
Pau Espin 3a81a7e4dc tests: Replace is_connected() with is_registered()
API was renamed and old name deprecated, let's use it everywhere.

Change-Id: Ic91dddcfd1a5a17c35d15a8a14628b3923229523
2020-06-10 17:57:37 +02:00
Pau Espin ffa325ede0 doc/manual: Fix links to nonexistent sections
Change-Id: I8742fa9c41d3ab95d41e299179df19c3d1e0a65b
2020-06-10 17:51:03 +02:00
Andre Puschmann 0a5011047a enb,ms: refactor base_rate selection for ZMQ
Use tailored sample rate only use for 6, 50 and 75 PRB. For all other bandwidths,
we use the full LTE rate and downsample.
With 5.76e6 as base sample rate, for example, we had some issue during the tests like:

[zmq] Error: tx time is 0.178 ms in the past (7833600 < 7835648)
[zmq] Error: tx time is 0.011 ms in the past (7835520 < 7835648)

Using a higher sample rate seems to work better.

Change-Id: I53678587b2c5e0e5dccb1b55a328ad2f1a97fc4c
2020-06-10 15:41:46 +02:00
Andre Puschmann acdf416796 enb_amarisoft: decrease max rate for TM2+
perhaps additional CRS for MIMO up to 25 PRB result in slighlt lower max rate

Change-Id: I128085e308f400622683905245d42f92a631e81d
2020-06-10 15:41:46 +02:00
Andre Puschmann 10ed8dc3a7 enb_srs: add extra head room for max rate with 6 PRB in MIMO mode
the overhead with 6 PRB and MIMO is a bit higher when compared
to other PRBs resulting in lower achievable throughput

Change-Id: I63888435553bba4f7be88cc745e24472921a7fb4
2020-06-10 15:41:46 +02:00
Andre Puschmann 07679207e2 handover.py: fix typo in test
Change-Id: I9a52a341d7575807d3e52460e1aa24732e3b3112
2020-06-10 15:41:46 +02:00
Andre Puschmann 898a5aaf68 4g_suite: add MT and MO test cases
add two new test cases to test paging (MT) and MO traffic

Change-Id: I5c7639f2c6953964a5b6979bc35f59464dbda43f
2020-06-10 15:41:42 +02:00
Andre Puschmann 419a662f76 ms_srs: refactor UE state getter functions
rename UE helper functions to query RRC connected/idle state
and adapt tests accordingly

also add helper to retrieve assinged IP address

Change-Id: I6cd057e34b4df6a1a73695355dd6406d7e039546
2020-06-10 15:40:00 +02:00
Andre Puschmann 8eeea7f991 srsenb.conf.tmpl: reduce RRC inactivity timer to 5s
the default value of 60s is too long to trigger errors in the handling
of this timer. So a 5s timer is more likely to reveal potential issues
during the test execution

Change-Id: Id7a6739487c7875c4f26a47d05978c9ea9f33873
2020-06-10 10:12:49 +02:00
Andre Puschmann f7915a8509 srsenb_sib.conf.tmpl: increase default T311 value to 10s
this matches the default of Amarisoft eNB so tests can use the same values

Change-Id: Ic81fb6428751305d8b37a25bddb0d018ca6f51f6
2020-06-10 10:12:49 +02:00
Andre Puschmann 98ad77e4a0 srsenb_sib.conf_tmpl: fix whitespaces
Change-Id: Ifbd3c8bd97d7e2d3f165e88f3358dce9e7e33067
2020-06-10 10:12:49 +02:00
Andre Puschmann 3c978fe0f2 ms_srs: add PRACH, reestablishment and paging counter
this allows to query the number of PRACH attempts and paging
messages received during a test

Change-Id: I592ace7c38d98bfbbec5001a3cdfffe21733ef9f
2020-06-10 10:12:49 +02:00
Andre Puschmann aa7b5b7751 epc: add run function to EPC
this allows, for example, to run a ping originating from the core to the UE

Change-Id: Id86c61cbccd41b77f94f125d8d127e2ab5e46cd9
2020-06-10 10:12:49 +02:00
Andre Puschmann 61d150b115 enb: refactor ue_max_rate() and move helper function back to sub classes
It turned out that the Amarisoft and SRS eNB scheduler produces
slightly different maximum data rates for both UL and DL.

Change-Id: I30fa7006906d101c53ba586fb06bced3945aa960
2020-06-10 10:12:49 +02:00
Andre Puschmann 487d1ef142 amarisoft_enb: fix RI config
this cause all MIMO modes to fail because RI reporting was disabled

Change-Id: Ic25233cf7feb4ded8b3d0f440191cc958d8bc0ec
2020-06-10 08:12:10 +00:00
Pau Espin 32f3e169b3 sysmocom/ttcn3/jenkins-run.sh: Pass confg file as cmdline
Fix after recent huge refactoring.

Change-Id: If0bcc9f7cb70e4ef324b3295d2174d11982a4bde
2020-06-09 16:22:50 +02:00
Pau Espin f2c2a34726 Fix ttcn3 testsuites after recent refactors
Change-Id: I545e3141ef56980e4ef2474b990bd4e4d4c21322
2020-06-09 14:32:14 +02:00
Pau Espin ef919c0f1c report: Add trial commit info to junit file
Change-Id: I997bbbeb3807af5cd927594a4155b824f0c6d03d
2020-06-05 14:08:28 +02:00
Pau Espin f3df1e4519 report: Add trial name to junit 'testsuites' node
Change-Id: I8cd62484bb67756e3d98d43a75731f4702acd2b7
2020-06-05 11:53:24 +02:00
Pau Espin ab78df2902 doc: manual: Write initial Test API section
Change-Id: I86bc9a8a0b5ff50e72c21a4bd8a857830bd84c4c
2020-06-04 20:03:07 +02:00
Pau Espin 166dc10769 testenv: Support test overlaying a directory to look for templates
This way tests which require a very specific config file can override
specific template files used by object classes.

Change-Id: I65d1b1e826d2d430ee83810d998b98d0ccaa07cd
2020-06-04 20:03:07 +02:00
Pau Espin 4ddcdaf62c selftest: template: Test existing overlay dir features
Change-Id: Iff1a4dafcbf2193e95b1fb6e5a6712f1bcdd6309
2020-06-04 17:39:17 +02:00
Pau Espin 315ba57651 testenv: Get rid of 'resources' global variable
Change-Id: Id7576d21d0f9de29dd9bc4211eaf9b9ec634c990
2020-06-04 17:09:16 +02:00
Pau Espin 7e30d84034 process: Improve launch_sync() exception due to process failing
After this commit, in some situations ssh related errors are printed
directly in the exception to quickly find cause of the issue.

Example:
    FAIL: ping.py (5.0 sec) Error: rm-remote-dir(pid=25913): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.110 port 22: No route to host) [trial↪4g:srsue-rftype@zmq+srsenb-rftype@zmq+mod-enb-nprb@6↪ping.py:9↪ping.py↪srsepc_10.42.42.118↪host-jenkins@10.42.42.110↪rm-remote-dir(pid=25913)]

Change-Id: Ia16c7dec96f70d761600ad6a50d9df8382d9c2c8
2020-06-04 16:46:43 +02:00
Pau Espin cf6bec7542 log: Improve log ctx information in log.Error exception
Before, it would show somethig like:
"""
osmo_gsm_tester.core.log.Error: Exited in error 255
"""

Now:
"""
osmo_gsm_tester.core.log.Error: rm-remote-dir(pid=24820): Exited in error 255 [trial↪4g:srsue-rftype@zmq+srsenb-rftype@zmq+mod-enb-nprb@6↪ping.py:9↪ping.py↪srsepc_10.42.42.118↪host-jenkins@10.42.42.110↪rm-remote-dir(pid=24820)]
"""

Change-Id: I8873f67a2f3df21c4dd552c92510535bf95e2c9d
2020-06-04 16:24:33 +02:00
Pau Espin 19fe3a1caf process: Fix wrong use of log.ctx(self)
That's not needed and will produce some parent loop detection in
log.find_on_stack() if logging is called under that stack frame.

Change-Id: I4ab7e8977fa9bad5c8956b7c1df1513b27bb5aa2
2020-06-04 16:11:39 +02:00
Andre Puschmann eb7ced1367 ms_srs: sum bitrates for all component carrier
when calculating the dl and ul bitrate, it is required to take the
sum of all active component carriers since they are reported seperately
inside the CSV file

Change-Id: If30ec3f22ce5871f911433e6a6997e9c6e7ca210
2020-05-29 09:55:59 +00:00
Andre Puschmann 346ffd9192 enb: remove freq config for zmq operation
specifiying the rx/tx_freq for the eNB is optional. But when
it's done it needs to be correct. There seems to be an issue
when mapping the freq[0-3] between srsENB and Amarisoft eNB.

So it's safe to just drop the config here altogether.

Change-Id: I70f5776173c2df5894031f68df2f6c3a900b79f8
2020-05-29 09:55:40 +00:00
Pau Espin eddff7c03c ms_{srs,amarisoft}: Support selecting tgz based on run_node's run label
Change-Id: I2c4cab02fa34c7acc7e57c3fec5a42c26d2a9208
2020-05-28 13:13:05 +00:00
Pau Espin b452ed6db1 enb_{srs,amarisoft}: Support selecting tgz based on run_node's run label
Change-Id: Ie1c8ccd2ffc9680bd72159872f3de0d59f1b9b0b
2020-05-28 13:13:05 +00:00
Pau Espin 5f826be20a ms_{srs,amarisoft}: Reuse run_node to hold run info
Change-Id: Ifdea48fed0bfc152131c53b87c34b4514c632917
2020-05-28 13:13:05 +00:00
Pau Espin 1abff4e260 enb: Reuse run_node to hold run info
Change-Id: If396940f154813d42ff3d3364c44b8979179127f
2020-05-28 13:13:05 +00:00
Pau Espin e921995ce7 epc_{amarisoft,srs}: Support selecting tgz based on run_node's run label
Change-Id: I90b8dab8bdc9980a7d2486d890cff183c07f1809
2020-05-28 13:13:05 +00:00
Pau Espin 6e0b6fb074 Support identifying different tgz files based on run_label attribute
tgz files in trials can be categorized in subdirectories, allowing to
select different bianry files at runtime based on the target run node
which is going to run them. This way for instance one can have a binary
linked against libs for eg. CentOS under run_label "centos/" or an ARM
target under "arm", and then use "run_label: arm" on the resource using
it.

Change-Id: Iaf2e97da3aff693395f44f0e93b184d4846cf6da
2020-05-28 13:13:05 +00:00
Pau Espin 2e495749d5 Fix processes run with RemoteProcessSafeExit not receiving stdin
Since the process is run in the background through the wrapper bash
script, stdin was disabled there. By explicitly redirecting the bash
process stdin we make sure it is always able to read from it.

Change-Id: I6cb7979aae0a7457919f353cbeb4c3b78cdd4919
2020-05-27 12:15:08 +02:00
Andre Puschmann 26890f3c31 ms_srs: wait 6s after stopping srsUE
we've only waited when running with ZMQ but in fact we need
to always wait because the UE may need some time to
gracefully exit, i.e. send the detach request and shut down correctly.

this is checked in cleanup() and verity_metrics(), so also before copying back
the CSV metrics. But we only wait once.

Change-Id: I47e46b8ccce41c9a62d2d6866260d22c927e710d
2020-05-26 13:55:39 +00:00
Andre Puschmann 71b430c22f enb: fix UL max rate calculation
don't double the maximum rate for UL traffic in MIMO since
we are only using MIMO in the downlink

Change-Id: I018e8ba8cb822a4bd57b8d0e420dd496172ac229
2020-05-26 13:55:39 +00:00
Andre Puschmann 1a96ecdeb6 ms_srs: allow + operator when specifying CSV columns
this allows to sum dl_brate and ul_brate and perform
the known operations, like max, average, etc. on both
UL and DL bitrate combined

For examples, with this CSV ..

time;cc;pci;earfcn;rsrp;pl;cfo;dl_mcs;dl_snr;dl_turbo;dl_brate;dl_bler;ul_ta;ul_mcs;ul_buff;ul_brate;ul_bler;rf_o;rf_u;rf_l;is_attached
1999;0;1;3400;-67;59;913;27;29;0.50;10432.000000;0;2.6;20;0.0;10848.000000;0;0.0;0.0;0.0;1.0
2998;0;1;3400;-67;59;912;28;29;0.50;2136.000000;0;2.6;20;0.0;1680.000000;0;0.0;0.0;0.0;1.0
3997;0;1;3400;-69;61;909;28;29;0.50;2136.000000;0;2.6;20;0.0;1680.000000;0;0.0;0.0;0.0;1.0

.. one could calculate the average of DL and UL bit rate with:

ue.verify_metric(1e6, operation='avg', metric='dl_brate+ul_brate', criterion='gt', window=test_duration)

and veriy that it's average is greater than 1 Mbit/s

Change-Id: I5c7c80bb107fa6b93b215176e6ebbb5dc8594860
2020-05-26 13:55:39 +00:00
Andre Puschmann 2cfa4a3d3c ms_srs: add max rolling average as metric operation
this allows to calculate the rolling average over a specific
window in time and take the maximum of that

this is useful to get average value for 30s UDP traffic for
example from the UE metrics

Change-Id: I34bbfe08dbc1f27b86c9805f54649d44d697fa18
2020-05-26 13:55:39 +00:00