Commit Graph

27 Commits

Author SHA1 Message Date
Daniel Willmann 9abd2221ee Stop tests after failures
Call mtc.stop after setverdict(fail), add reasons to most failures and
fail with verdict error for internal errors.

Change-Id: I9b618235939fa41160b9be6677b121963d3ec857
2018-07-24 09:57:46 +02:00
Harald Welte ed380d135d hlr: Port HLR tests to use GSUP_Emulation
Going via GSUP_Emulation (rather than using GSUP_CodecPort directly)
adds many benefits, such as:
* ability to have multiple transactions in parallel
* no silent discard/ignore of unexpected GSUP messages, like those
  for IMSIs we don't expect.

Change-Id: Id2ddd6b81c374ad6350b62fcc5442436757d66cd
2018-06-15 17:59:40 +02:00
Pau Espin a4cecf5af7 Add README.md file for several suites
Content grabbed from [1] and converted to markdown as it's more probable
to have it displayed correctly on popular git repo hosting platforms.
Syntax for embedded graphviz taken from [2].

[1] http://git.gnumonks.org/laforge-slides/plain/2018/osmodevcon2018-ttcn3_test_suites/osmodevcon2018-ttcn3_test_suites.adoc
[2] https://github.com/stlehmann/markdown-graphviz

Change-Id: I9aad03e605569e9613665a7585d60bf308bcb54f
2018-06-14 18:55:36 +02:00
Alexander Couzens 98aa59e064 Osmocom_VTY_Functions: move f_vty_transceive_match from HLR to this library
f_vty_transceive_match will be used by future SGSN tests.

Change-Id: Ia69ab6d5639c2e10059f88c8cc97463820cb72e6
2018-06-12 13:51:14 +02:00
Harald Welte 4908982036 hlr: Update expected-results
as of Change-Id Change-Id: I4f51abdf44dfc62d7e8792341aad6dafe58923da,
osmo-hlr passes HLR_Tests.TC_gsup_sai_err_invalid_imsi

Change-Id: I72fb71806c72ce29e8c6c9b25723f02009463cec
2018-06-11 22:23:10 +02:00
Neels Hofmeyr fc0384a046 mask timestamps and source file nrs in expected-results.xml files
Prepare for upcoming updates with concise diffs.

Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 19:32:01 +02:00
Neels Hofmeyr 1fd6679d9d fix build: don't clean out expected-results.log: rename to *.xml
'make clean' as generated by ttcn3_makefilegen removes all *.log files, which
of course cleans out expected-results.log, which should not happen. Since this
is a junit XML file, rename the suffix to .xml.

Change-Id: Ic334f6b758eef865e3a497aa430691a3ae696d25
2018-04-11 19:29:18 +02:00
Stefan Sperling afab3e5a45 update expected HLR test results to prevent unexpected 'pass'
With https://gerrit.osmocom.org/#/c/7685/ the test TC_vty_msisdn_isd
is now passing. Update expected log output accordingly.

Change-Id: Ie8cfbef5a0adcb927917b0623f89309479a60001
Depends: Iffe1d7afb9fc7dbae542f70bbf5391ddc08a14b4
Related: OS#2785
2018-04-09 17:41:20 +00:00
Stefan Sperling 7c09687456 improve failure diagnostics in TC_vty_msisdn_isd HLR test
Check for reception of an Insert Subscriber Data with outdated MSISDN.
This happened to me while working on a fix for issue OS#2785, and it
seems to be an easy mistake implementations can make. Catch this
situation in the test and log an explicit message about the problem.

Related: OS#2785
Change-Id: Ib0809617cca621cc22f29b078828057fd49f27e5
2018-04-09 11:33:21 +02:00
Neels Hofmeyr 3cf797d7a5 add compare-results.sh, call from start-testsuite.sh
Compare current test results to the expected results, and exit in error on
discrepancies.

Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.

Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".

In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.

In start-testsuite.sh, after running the tests, invoke the results comparison.

Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.

Related: OS#3136
Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-06 01:07:15 +02:00
Neels Hofmeyr cd11223c0f ignore results of .ttcnpp files
Add another macro ignore_pp_results to gen_links.sh.inc and call from all
gen_links.sh files, to add results of *.ttcnpp files, i.e. generated *.ttcn
files, to .gitignore.

Change-Id: Ic7fb176226771212d7700dafaf27ac71f12a4a61
2018-03-16 18:25:58 +01:00
Neels Hofmeyr 1a4c4601d7 fix gen_links.sh: don't include source dir as link target
First of all, use one common place to define the gen_links() macro, in
gen_links.sh.inc.

In this new file, add a 'shift' to exclude the $DIR arg from also appearing in
$FILES.

This prevents the following wrong symlinks in the source dirs:

  M3UA_CNL113537/src/src
  MTP3asp_CNL113337/src/src
  SCCP_CNL113341/src/src

Change-Id: Ia8493e77df1ba8723f2c5d2a49816247b0fb55f7
2018-03-16 00:08:02 +01:00
Harald Welte 99ca30ce2e remove *.default from [EXECUTE] in *.default files
The problem is that the .cfg file cannot override the [EXECUTE]
provided list of tests

Change-Id: I7e1e0499b3b32014b25ae4b515263f57a4ea6daf
2018-03-13 14:30:55 +01:00
Harald Welte 3f66276bd4 hlr: Distinguish "invalid IMSI" from "unknown IMSI" cases
Change-Id: I6ded77b3029aae6bc3fe022190819b6a86189f5a
2018-03-02 10:48:20 +01:00
Harald Welte b3e3094884 f_rach_toffs: Print toffs256 value in verdict when failing
Change-Id: I82a8f7c8dd3ce3327aa43bfb9f03795531718320
2018-03-02 10:35:05 +01:00
Harald Welte 958f8b4fa4 hlr: Add testcases for PURGE_MS procedure
Change-Id: I9d54d5e4b6fe24d3ee710b57e0c69a30b89d3450
2018-03-02 08:56:59 +00:00
Harald Welte 8f0c933c2b hlr: Fix test of UL+ISD state machine
Don't exit too early: After sending ISD.resp we still need to wait
for the UL.res from the HLR before continuing processing.

Change-Id: Iab42a397cbca83b86fc8a6b26ae2d66abb81c187
2018-03-02 08:56:59 +00:00
Harald Welte 09b3c50e85 hlr: Add TC_vty_msisdn_isd
This tests whether the HLR is sending an InsertSubscriberData to the VLR
of an active/registered subscriber after the MSISDN is updated in the
HLR.

Change-Id: I597a3c2d49aa6fa65007304105363a3e99fa4ae9
Related: OS#2785
2018-03-01 22:42:22 +01:00
Harald Welte 7029606aa3 hlr: Reduce code duplication by using template
Change-Id: I7dcf0d27144e5dcf29babcb679ca9ef04b1f46a3
2018-03-01 22:42:03 +01:00
Harald Welte c2c52554ee hlr: More test coverage
Change-Id: Ic93a2e7498343abcb709d7018c4d49811252342c
2018-03-01 21:20:39 +01:00
Harald Welte b2db68eddb hlr: Add hlr sub-directory to master Makefile (for 'make compile' checks)
Change-Id: Idbd8241fc7d217885f65f84e11f9f1ea13adb82e
2018-03-01 17:10:12 +01:00
Harald Welte 79420ccab2 hlr: Make test run again using current OsmoHLR / TTCN-3 library
Change-Id: I4304d6f69a349845d74a9dbe5e366525d5d8956f
2018-03-01 16:49:29 +01:00
Harald Welte 39b82d344a hlr: Integrate VTY and CTRL support
Change-Id: I3fbc26a98d31df6d4753848bba655a517801d686
2018-03-01 10:21:56 +01:00
Harald Welte 78cae923fa hlr: Rename module + file from GSUP_Test to HLR_Tests
Change-Id: I906c0046a4cf6ae95e5a0aca918f4375fedfea55
2018-03-01 10:12:56 +01:00
Harald Welte df277258f1 IPA_Emulation: Make dependencies to RSL/MGCP/SCCP/GSUP conditional
Let's use the preprocessor to avoid IPA_Emulation pulling *all*
dependencies into each and any of our projects.  The code readability
suffers a bit from the many #ifdefs, but compilation speed increases
if we don't have to pull in all those (recursive) dependencies.

After all, a BTS test case will never need SCCP, GSUP or MGCP.

Change-Id: Ic0231adbd2171214de133d26b3fbf36130ee8aa0
2018-02-20 16:17:17 +01:00
Max 6367d7b807 Don't symlink non-existent file
Change-Id: I10309f07fb207c027703f0b43a478c152a029b6d
2018-02-05 16:58:27 +01:00
Harald Welte df32723446 GSUP Support and first test against OsmoHLR
Change-Id: Idd7a6aca1ab193da39294f49a3adc4c1cd7cecff
2017-12-28 23:17:24 +01:00