Compare current test results to the expected results, and exit in error on
discrepancies.
Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.
Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".
In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.
In start-testsuite.sh, after running the tests, invoke the results comparison.
Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.
Related: OS#3136
Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
At the moment we use onle TCH/F and PDCH in our osmo-bsc config,
but the tests TC_assignment_codec_amr_h and TC_assignment_codec_h
requre a half rate channel in order to work. Configuring one TCH/F
to TCH/H seems not to hurt any tests except TC_chan_exhaustion
- also regard half rate channels in TC_chan_exhaustion
- change included sample osmo-bsc.cfg
Change-Id: Idd5a9602f1b49b5aa0fc510faa151fe16c0b404d
Related: OS#3100
Let's not only match if the BSSMAP ASSIGNMENT COMPL matches what codec
was requested in BSSMAP ASSIGNMENT CMD, but also verify that the
RSL side channel was activated with the right channel mode IE.
Change-Id: I6ef24b5b82a5c100ffffe056adadd26e69b968b9
The settings concerning the IPA ping are a leftover from the
time where osmo-bsc used an IPA multiplex to support the the
SCCP lite A interface dialect. Since the related code is now
removed and osmo-bsc has been migrated to libosmo-sccp and
real AoIP interface, the configuration is no longer needed.
Change-Id: If67cdba2a043c5dbf23f998659c5000db924a7df
Make sure that the "tag" member of the RslChanelNr sub-structure
is always initialized. This can be achieved without any extra code
by using the existing templates rather than hand-coding it.
Change-Id: I990ac8ac0ce51e11f1d683382c9fc2d4e1201aa7
The problem is that Junit-XML doesn't have a mapping for inconclusive
results, and hence they show up as 'passed'.
By introducing this change, we make sure all tests that don't pass
show up as failed.
Change-Id: Iddd13d0055c91f9bd304ce9833fba0485abf4c4e
Verify that the BSC does not page a subscriber when a cell identifier
with an unknown MCC/MNC is provided by the MSC.
This test introduces a magic value which represents an unknown MCC/MNC
combination: MCC=678 MNC=f90
Change-Id: I0b0af14a9a1cb7e5a7a4ec12cc489473fd7ead02
Related: OS#2980
Originally, this code was not yet in an official upstream git repo.
However, it has been for many months, so let's remove our local copy
and use upstream git repositories like for all the other modules.
Change-Id: I2c616fb865df32cfec323d42e5d0d06de40c497b
After all, we don't want stale queue entries especially in those where we
are not expecting any paging.
Change-Id: Id876b68087ef13d58177027b7e664404e6b7b2d9
The TTCN-3 data types are abstract data types, Encoding artefacts
like 'F' for padding shouldn't be seen by the user. Hence, let's
pass a 2-digit-long or 3-digit-long hexstring into the encoder
functions and let them determine if they should introduce any 'F'
for padding or not.
Change-Id: If4d3dfc16381493d7e710be746ed963975051fc1
If 5 seconds expire, the BSC will automatically re-fill a credit
of 20 paging slots [to work with BTSs where the paging buffer space
indications somehow are missing]. Let's make sure we don't hit
that case, even if the operating system sleeps for more than 5s
in this test .
Change-Id: I1c65096a685b70dc5183592382ec03553ba3628f
We are testing purely IPA/RSL/OML, and half-starting the BSSAP/M3UA
emulation is not a good idea, if it generatees events that we don't
catch
Change-Id: Ie90cd88e63ba6062e4ea2592045e9c97bc11887e
Add another macro ignore_pp_results to gen_links.sh.inc and call from all
gen_links.sh files, to add results of *.ttcnpp files, i.e. generated *.ttcn
files, to .gitignore.
Change-Id: Ic7fb176226771212d7700dafaf27ac71f12a4a61
In each subdir that is a target for symlinks, automatically ignore the results
of gen_links():
- At the top of gen_links.sh.inc, clear the .gitignore.
- In the loop, add each link name to the local .gitignore.
- In selected gen_links.sh, there is also a "manual" link creationg. So that
this also ends up in the local .gitignore, have the link creation as separate
gen_link() macro which at the same time adds to ./.gitignore.
- in the root .gitignore, ignore all the subdirs' generated */.gitignore files.
Change-Id: I73c11fe8362358bf7e1bdf0e1be53399b5d3351b
First of all, use one common place to define the gen_links() macro, in
gen_links.sh.inc.
In this new file, add a 'shift' to exclude the $DIR arg from also appearing in
$FILES.
This prevents the following wrong symlinks in the source dirs:
M3UA_CNL113537/src/src
MTP3asp_CNL113337/src/src
SCCP_CNL113341/src/src
Change-Id: Ia8493e77df1ba8723f2c5d2a49816247b0fb55f7
The cell identifier used by the paging tests is 001-01, i.e. uses a
2-digit MNC. With the introduction of 3-digit MNC support in osmo-bsc,
the paging tests became incompatible with a osmo-bsc config with:
network country code 1
mobile network code 1
Explicitly declare a Cell_Identity with 2-digit MNC (includes an 'f').
Also, fix f_enc_mcc_mnc to properly encode 2-digit MNC values.
Related: OS#2847
Change-Id: Ide5228b403e43de8649b6eda18749ea2a9f592a9
bsc: add TC_bssmap_clear_does_not_cause_bssmap_reset(), but the same triggered
by an MS Rel Ind and a BSSMAP Clear Request sent to the MSC first.
This test will only succeed once TC_chan_rel_rll_rel_ind() succeeds, i.e. with
below osmo-bsc fix.
Related: OS#3041
Depends: I0f8c9c4e6b6850b15c70250fd3f88bdf75f9accf (osmo-bsc)
Change-Id: Ie4aa2f01c83b40303fa40ed64dbfce372b7cd96c
This test sends a REL IND from the MS and immediately expects an lchan release.
Instead, osmo-bsc patch I0f8c9c4e6b6850b15c70250fd3f88bdf75f9accf decides to
signal full BSSMAP Clear Request to the MSC first, so expect that first.
Note that this test currently fails, and said osmo-bsc.git patch will make this
test pass.
Change-Id: I737be141b69a250eb6eb38007f8042981c1a31cf
Same as TC_bssmap_rlsd_does_not_cause_bssmap_reset(), but with a proper BSSMAP
Clear from the MSC first.
Related: OS#3041
Change-Id: If6ca85d7b80a727cbfdabbf07529ced22602734e
A test with BSSMAP Clear involved would also be a nice addition, but this so
far tests a direct RLSD from the MSC.
(One way to invoke a typical release situation would be a scenario like in
TC_chan_rel_rll_rel_ind(), but that test currently fails; another would be to
directly invoke a BSSMAP Clear from the MSC first.)
Related: OS#3041
Change-Id: I168cf240383485a5ffbbde377b4f89c5d1f5ab93
If the T_guard runs out, unless we self.stop, we might run into this
potentially confusing follow-up error:
00:23:04.206712 mtc BSC_Tests.ttcn:322 Dynamic test case error: Copying an unbound value of type @RSL_Types.RSL_Message.
00:23:04.206778 mtc BSC_Tests.ttcn:322 setverdict(error): fail -> error
Change-Id: I1d373159483bdd9f74e8944e430913e73c289e03
The altstep as_handover does not exit as each of its branches is
equipped with a repeat statement on the end. This trapps us in
an endless loop.
- remove the repeat statement at the last logical step which is
at the RSL_REL_REQ.
Change-Id: I8cb57a9fef606f459542708206f5ea4de1def7a1
Currently we use isbound() in f_start_handler() to check if the BTS
which we want to connect is indeed populated. However. isbound()
seems never become true in this particular situation.
- Use isvalue() instead of isbound()
Change-Id: I25ddd55b7626087570311999b85ec7632b162c06
When the test component ends and the underlaying
components are shut down. Messages from the system
under test may still be picked up and forwared. When
a message is handed from one component to one that
is already shut down, the testcase is flagged as
errornous setverdict(error).
An idea to fix thisis to stop all test ports using
all port.stop. However, this does not solve the
problem entirely. We still observing errors.
- add f_shutdown_helper() and call it from the
end of each testcase
- perform an all port.stop; in f_shutdown_helper()
to freeze all communications between the ports
of the different components.
Change-Id: Id3bc840c0428d08dfbeedacc408b3dd1cd0fa7ec
The testcase TC_paging_imsi_a_reset sends a paging request that
causes pagings on all cells. Then it performs a BSSMAP reset and
checks if the paging has stopped. In order to be sure that paging
requests from before the reset procedure are not mistakenly
detected as after-reset-pagings the RSL queue is cleared. However
this is only done for IPA_RSL[0], which means IPA_RSL[1] and
IPA_RSL[2] still contain old paging requests, which lets the test
fail.
- Clear IPA_RSL[1] and IPA_RSL[2] as well.
Change-Id: If0cdc0325fd0e1dcf3e4ce52e4de27adb4d9cf48
RSL_Emulation has recently gained a port for the common channel
management messages, but BSC_Tests was not updated with this port,
resulting in RSL_Emulation enqueuing messages to the port which then
creates dynamic test case failures.
Let's simply connect to the port, even though we currently are not
interested in any of the messages received there (mainly
RSL_MT_BCCH_INFO during startup).
Change-Id: Id8a3c4409599783ca4cd0b49f2570bcb3bb34952
The testcase TC_paging_imsi_nochan assumes that a paging for no
channel, with no specifc cell associated and without TMSI is
illegal. This is not correct. All these fields have legal values
and the TMSI field is optional.
- Replace the testcase implementation, use f_pageing_helper() to
create the paging.
Change-Id: I6a56fb0ee06ae7e72a7ac2b6b058ad54f94127ab
Let's use the preprocessor to avoid IPA_Emulation pulling *all*
dependencies into each and any of our projects. The code readability
suffers a bit from the many #ifdefs, but compilation speed increases
if we don't have to pull in all those (recursive) dependencies.
After all, a BTS test case will never need SCCP, GSUP or MGCP.
Change-Id: Ic0231adbd2171214de133d26b3fbf36130ee8aa0
This adds two new tests: One for RSL, and a second one which performs
the same test on the OML port. Both tests open an IPA connection and
send a unit ID which is unknown to the BSC. The tests expect the BSC
to close the connection immediately.
We need to add handling for a socket error in IPA_Emulation because
otherwise these tests do not pass reliably as some closed connections
are not properly detected.
Change-Id: I6a947d7411a016e4d7650031396cae3575756453
The provided BSC_Tests.cfg, which contains as a commented line for
local debugging, does not contain all tests from the control section.
- Update BSC_Tests.cfg so that it contains all tests from the
control section
Change-Id: I67f5906c3c67ae5793d88eee81b58ceac0bed658
The provided osmo-bsc.cfg does not contain any codec list, this
causes failure of the assignment related tests.
- add missing codec list
Change-Id: Id2509bcac4c54c910efd2db749eaa2085daa358d
The testcase TC_paging_counter is missing in the control section.
This means that if it is not started explicitly, it is not executed.
- Add TC_paging_counter to control section
Change-Id: Ie37b8cb554ea1db64a8d7927eb300d368ce67137
Let's make sure we share common configuration between the test
suites and split the config file into a "default" part which is
used (but not copied) in the Docker images, and a "local" part
which is basically those overrides that the user (or docker image)
wants to do from the default.
Change-Id: I3db452e24e5238aa05254d903739c64d202e61db
This allows each ConnHandler to issue telnet commands to the VTY
As a side-effect, it puts some more stress on the VTY interface,
as each [parallel] DchanHdlr now has its own telnet connection.
Change-Id: Ibd726af53219d829286da80b44ea4d9fb2ffdf3d
When f_expect_chan_rel() is called after receiving the BSSMAP
RESET and DISC.ind f_expect_chan_rel() is called. The flush
parameter is not set, which means the default flush = true is
valid. This leads into an early flush of the RSL Queue and
tosses the RSL RELEASE REQUEST we expect, so the test can not
pass, even when the BSC sends the RLEASE REQUEST.
Looking further up in the code. IPA_RSL[0].clear is called,
so the Queue is flushed to get rid of unwanted messages from
the IMMEDIATE ASSIGN. There is no need to flush the queue
a second time anyway.
- Do not flush the RSL queue, set flush=false when calling
f_expect_chan_rel()
Change-Id: I2962f741e0b13dec08ac6c918d326828beb65a6a
Neels Hofmeyr
Philipp Maier
Harald Welte