Commit Graph

68 Commits

Author SHA1 Message Date
Neels Hofmeyr 3cf797d7a5 add compare-results.sh, call from start-testsuite.sh
Compare current test results to the expected results, and exit in error on
discrepancies.

Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.

Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".

In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.

In start-testsuite.sh, after running the tests, invoke the results comparison.

Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.

Related: OS#3136
Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-06 01:07:15 +02:00
Neels Hofmeyr cd11223c0f ignore results of .ttcnpp files
Add another macro ignore_pp_results to gen_links.sh.inc and call from all
gen_links.sh files, to add results of *.ttcnpp files, i.e. generated *.ttcn
files, to .gitignore.

Change-Id: Ic7fb176226771212d7700dafaf27ac71f12a4a61
2018-03-16 18:25:58 +01:00
Neels Hofmeyr 1a4c4601d7 fix gen_links.sh: don't include source dir as link target
First of all, use one common place to define the gen_links() macro, in
gen_links.sh.inc.

In this new file, add a 'shift' to exclude the $DIR arg from also appearing in
$FILES.

This prevents the following wrong symlinks in the source dirs:

  M3UA_CNL113537/src/src
  MTP3asp_CNL113337/src/src
  SCCP_CNL113341/src/src

Change-Id: Ia8493e77df1ba8723f2c5d2a49816247b0fb55f7
2018-03-16 00:08:02 +01:00
Harald Welte 99ca30ce2e remove *.default from [EXECUTE] in *.default files
The problem is that the .cfg file cannot override the [EXECUTE]
provided list of tests

Change-Id: I7e1e0499b3b32014b25ae4b515263f57a4ea6daf
2018-03-13 14:30:55 +01:00
Harald Welte 3f66276bd4 hlr: Distinguish "invalid IMSI" from "unknown IMSI" cases
Change-Id: I6ded77b3029aae6bc3fe022190819b6a86189f5a
2018-03-02 10:48:20 +01:00
Harald Welte b3e3094884 f_rach_toffs: Print toffs256 value in verdict when failing
Change-Id: I82a8f7c8dd3ce3327aa43bfb9f03795531718320
2018-03-02 10:35:05 +01:00
Harald Welte 958f8b4fa4 hlr: Add testcases for PURGE_MS procedure
Change-Id: I9d54d5e4b6fe24d3ee710b57e0c69a30b89d3450
2018-03-02 08:56:59 +00:00
Harald Welte 8f0c933c2b hlr: Fix test of UL+ISD state machine
Don't exit too early: After sending ISD.resp we still need to wait
for the UL.res from the HLR before continuing processing.

Change-Id: Iab42a397cbca83b86fc8a6b26ae2d66abb81c187
2018-03-02 08:56:59 +00:00
Harald Welte 09b3c50e85 hlr: Add TC_vty_msisdn_isd
This tests whether the HLR is sending an InsertSubscriberData to the VLR
of an active/registered subscriber after the MSISDN is updated in the
HLR.

Change-Id: I597a3c2d49aa6fa65007304105363a3e99fa4ae9
Related: OS#2785
2018-03-01 22:42:22 +01:00
Harald Welte 7029606aa3 hlr: Reduce code duplication by using template
Change-Id: I7dcf0d27144e5dcf29babcb679ca9ef04b1f46a3
2018-03-01 22:42:03 +01:00
Harald Welte c2c52554ee hlr: More test coverage
Change-Id: Ic93a2e7498343abcb709d7018c4d49811252342c
2018-03-01 21:20:39 +01:00
Harald Welte b2db68eddb hlr: Add hlr sub-directory to master Makefile (for 'make compile' checks)
Change-Id: Idbd8241fc7d217885f65f84e11f9f1ea13adb82e
2018-03-01 17:10:12 +01:00
Harald Welte 79420ccab2 hlr: Make test run again using current OsmoHLR / TTCN-3 library
Change-Id: I4304d6f69a349845d74a9dbe5e366525d5d8956f
2018-03-01 16:49:29 +01:00
Harald Welte 39b82d344a hlr: Integrate VTY and CTRL support
Change-Id: I3fbc26a98d31df6d4753848bba655a517801d686
2018-03-01 10:21:56 +01:00
Harald Welte 78cae923fa hlr: Rename module + file from GSUP_Test to HLR_Tests
Change-Id: I906c0046a4cf6ae95e5a0aca918f4375fedfea55
2018-03-01 10:12:56 +01:00
Harald Welte df277258f1 IPA_Emulation: Make dependencies to RSL/MGCP/SCCP/GSUP conditional
Let's use the preprocessor to avoid IPA_Emulation pulling *all*
dependencies into each and any of our projects.  The code readability
suffers a bit from the many #ifdefs, but compilation speed increases
if we don't have to pull in all those (recursive) dependencies.

After all, a BTS test case will never need SCCP, GSUP or MGCP.

Change-Id: Ic0231adbd2171214de133d26b3fbf36130ee8aa0
2018-02-20 16:17:17 +01:00
Max 6367d7b807 Don't symlink non-existent file
Change-Id: I10309f07fb207c027703f0b43a478c152a029b6d
2018-02-05 16:58:27 +01:00
Harald Welte df32723446 GSUP Support and first test against OsmoHLR
Change-Id: Idd7a6aca1ab193da39294f49a3adc4c1cd7cecff
2017-12-28 23:17:24 +01:00