Commit Graph

86 Commits

Author SHA1 Message Date
Oliver Smith 280536b782 mgw: add module param mp_enable_conn_timeout_test
Fix ttcn3-mgw-latest by not running "conn-timeout 0" during f_init_vty
at the start of every test case. The latest osmo-mgw release does not
have that command yet.

Change-Id: I8bbf15baa45679d5812a5a9184520ef9b9e73bba
2019-06-28 11:04:48 +02:00
Oliver Smith 021141e6dc mgw: add connection timeout test
Change-Id: I020b682b347045818fd28de240daa0aa33fe43b4
2019-06-26 16:16:20 +02:00
Oliver Smith 350eb681e2 mgw: add osmo-mgw.cfg
Change-Id: I3dd498902db4ea0233f43099ec5b6b4292714908
2019-06-25 13:45:25 +02:00
Harald Welte 34b5a95d09 cosmetic: Update copyright statement, license notice and SPDX
Some of our files didn't have a copyright notice at all, let's add
it.  Also, update the notices in other files and ensure a SPDX
identifier is present in all but the most trivial files.

Change-Id: If7fa19ce484b415bc645e39b3d0d666b44b5f0fd
2019-05-27 10:00:06 +00:00
Pau Espin b2c6b38f3f Add Osmux support and tests for MGW
Depends: osmo-mgw.git Iac073f1db46569b46eddeaecc9934a2986bd50f1
Change-Id: Ibb58b2a4e08d6f30cfe347c217794d0d1310954f
2019-05-27 09:56:51 +00:00
Daniel Willmann 94e4e80ec0 mgw: Ensure TC_selftest passed if no error is encountered
Change-Id: Ie1601d57ac5bf9ddf99cbbd1cdb91160dc47c84d
2019-03-28 13:08:10 +01:00
Philipp Maier 4f764ce4f3 MGCP_Test: Test AMR octet-aligned/bandwith-efficient mode conversion
osmo-mgw now implements a conversion between the AMR octet-aligned
and banwith-efficient payload format. Lets add tests for this

Change-Id: I050bfeabfb5fdbf986d429eef3af69fe8158d56e
depends: osmo-mgw I622c01874b25f5049d4f59eb8157e0ea3cbe16ba
related: SYS#4470
2019-03-19 13:29:31 +00:00
Philipp Maier c8c0b40e1b MGCP_Test: allow setting of fmtp parameters
When creating an RTP flow, there is currently no way to set SDP fmtp
parameters. Lets add a template and a parameter in order to be able to
set those parameters.

Change-Id: Ic1840d5023cb3888a17980f4ed08c19175864896
Related: SYS#4470
2019-03-19 13:29:31 +00:00
Philipp Maier 6d4e094341 MGCP_Test: Add testcase to test ts101318/rfc5993 conversion
The MGW recentenly adds support to convert between ts101318 and rfc5993
when GSM-HR is used. Lets add a testcase for that.

depends: osmo-mgw Iceef19e5619f8c92dfa7c8cdecb2e9b15f0a11a1
Change-Id: I96df45fe45b53088e07b26f14173a75498a84143
Related: OS#3807
2019-02-26 09:10:41 +00:00
Philipp Maier 6137c32e05 MGCP_Test: Add VTY access
The MGCP_Test testsuite currenty lacks VTY access capabilities, lets
add them so that future testcases can access the VTY in order to change
options on the fly.

Depends: osmo-ttcn3-hacks Ife949c61156222de3026280071226ef6f5dbd959
Change-Id: If383f81af3306f8f5bdf50152498ae1303d390df
Related: OS#3807
2019-02-20 17:00:33 +01:00
Daniel Willmann a069d38b61 mgw: Tear down all RTP flows to avoid race condition on tear down
When stopping the test TC_two_crcx_and_unsolicited_rtp the unsolicited
RTP stream is not stopped. As a result it could happen that between
tearing down the other flows and stopping the test an unsolicited RTP
packet is sent to a closed socket.
The resulting ICMP destination unreachable packet translates to a
"Connection refused" error on the sending socket and fails the test.

Avoid this by making sure the unsolicited RTP sender is stopped before
stopping the test.

Change-Id: Ied839596589609e75fa487046a85db48991e4c73
2018-12-23 10:14:14 +00:00
Daniel Willmann d93047434b Add Misc_Helpers.ttcn to centralize TTCN3 shutdown handling
This function can now be called from anywhere to try and safely shutdown
a testcase. It is not optimal as we can't call "all component.stop" from
outside the mtc, but without any proper and orderly shutdown handling of
all our emulation components I believe this is the best we can do.

To use it:

import from Misc_Helpers all;

in your module and then call

Misc_Helpers.f_shutdown(__BFILE__, __LINE__);

You can also pass the function a verdict and a message and it will take care
of calling setverdict, but beware of the following:

While setverdict would accept any number of arguments as log message
and convert them to a log string f_shutdown expects one charstring.
It's possible to use the log2str function to use the log arguments in
setverdict for f_shutdown, for example

setverdict(fail, "Template didn't match: ", tmpl_foo);
would become
Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, log2str("Template didn't match: ", tmpl_foo));

Change-Id: I84d1aa6732f6b748d2bfdeac8f6309023717f267
2018-10-24 15:10:13 +02:00
Neels Hofmeyr 873ae201bd update expected results
Change-Id: I32c29e62ca317937db771f8fb1540bb1fe9da2ab
2018-09-06 14:13:34 +02:00
Daniel Willmann 43a1a0be9e mgw: Don't try to send packets while MDCX is underway
TC_two_crcx_and_one_mdcx_rtp_ho sometimes failed while trying to send an
RTP packet without a connected port. f_flow_modify re-bind()s the port,
sends the MDCX and then connect()s it after the MDCX ACK returned the
IP/port combination.
If the transmit timer fires off between the bind and connect the
resulting send call will fail.

Change-Id: Idf93ceb830a44dafa56430ab5178f05da6bdd6fb
2018-08-01 17:55:54 +02:00
Philipp Maier 7df85f63bf MGCP_Test: Test what happens when two ends use different PT
It is legal that two connections use the same codec but negotiate
different dynamic payload types for both connections. Then the MGW is
expected to receive packets with one PT and send them with the other PT.
This is currently not done in osmo-mgw so the two tests that this commit
adds are expected to fail for now.

- add testcase TC_two_crcx_diff_pt_and_rtp
- add testcase TC_two_crcx_diff_pt_and_rtp_bidir

Change-Id: Ib4606dfc08764410ee9e450949361544adb07cd3
Related: OS#3384
2018-07-25 18:51:06 +00:00
Philipp Maier 3629139790 MGCP_Test: add function to check for RTP err counters
At the moment we check the error counters of the RTP statistics in
the testcases. However, in most situations we will do the check to
make sure that no errors occurred (all counters == 0). Rather than
having a long tail of if statements in the testcases we should have
a function for this. This also makes it much easier in case we add
more error countes lateron.

- add and use function f_rtpem_stats_err_check()

Change-Id: I69e5f80b0765284ec99056ce62c315461967d2a1
Related: OS#3384
2018-07-25 18:44:54 +00:00
Philipp Maier c290d7292e MGCP_Test: check payload type of received RTP packets
When an RTP packet is received, the payload type is not checked,
so we will not detect if the MGW emits packets with a wrong payload
type for some reason.

- Introduce a statistics counter that counts packets with wrong PT
- Update testcases so that they check for the statistics for wrong
  PT count.

Change-Id: I83d4b04656a16ced624024245a2fcb7a0ad48a8a
Related: OS#3384
2018-07-24 19:01:27 +02:00
Daniel Willmann afce86630c Stop tests after failures
Call mtc.stop after setverdict(fail), add reasons to most failures and
fail with verdict error for internal errors.

Change-Id: I9b618235939fa41160b9be6677b121963d3ec857
2018-07-24 14:19:42 +02:00
Philipp Maier 28bb82990c MGCP_Test: Make sure PT in RTP-Stream and MGCP match up
At the moment the RTP stream emulation is left in its default
configuration, this means that the payload type that appears in the RTP
stream is always 0. This may mismatch the payload type that is
configured with MGCP.

If nothing else is set, we should make sure that whan we create and modify
flows, the RTP emulation is always reconfigured to use the payload-type that
we set in the flow parameters. The other rtp-emulation parameters should
be set to their defaults.

- Make sure f_flow_modify and f_flow_create set the rtp flow parameters
  properly.

Change-Id: Ie888424ac3e0bf0d960b6f071855b6dd43935a0e
Related:OS#3384
2018-07-21 08:20:59 +00:00
Philipp Maier 4de3e377d1 MGCP_Test: do not use constant IP addresses
The RTP stream tests TC_two_crcx_and_unsolicited_rtp and
TC_two_crcx_and_one_mdcx_rtp_ho, which were introduced recently
with Change-Id I556a6efff0e74aab897bd8165200eec36e46629f, use
hardcoded ip addresses (127.0.0.1) to establish the RTP streams
that are used to test how the MGW reacts on unsolicited packets.
This works fine when everything runs on local host but on docker
it failes since the containers there have different ip-addresses.

- replace hardcoded IP-Addresses with modulepar variable in
  TC_two_crcx_and_unsolicited_rtp and TC_two_crcx_and_one_mdcx_rtp_ho

Change-Id: I5af5186f173c2b8564e8034249c82245acdd09f6
Related: OS#2703
2018-06-29 17:23:18 +02:00
Philipp Maier 2321ef92a3 MGCP_Test: add tests to verify actual RTP flows
The test coverage of the RTP aspects of the MGW is currently very
minima. Lets add a few more testcase to verify RTP behaves as
expected in various situations.

- Add testcase TC_one_crcx_receive_only_rtp:
  Test recvonly mode of the MGW. All packets must be absorbed by
  the MGW, no packets must come back.

- Add testcase TC_one_crcx_loopback_rtp:
  Test loopback mode of the MGW. All packet sent to the MGW must
  come back.

- Add testcase TC_two_crcx_and_rtp_bidir:
  We already test unidirectional transmissions. This test does
  the same as TC_two_crcx_and_rtp but for both directions.

- Add testcase TC_two_crcx_mdcx_and_rtp:
  Simulate a typical behaviour of a normal call. First create
  two half open connections and complete the connections later
  using MDCX.

- Add testcase TC_two_crcx_and_unsolicited_rtp:
  Test what happens when a RTP packets from rogue source are mixed
  into the RTP stream.

- Add testcase TC_two_crcx_and_one_mdcx_rtp_ho:
  Test a typical handover situation. An existing connection is
  handovered to another source on one end but the old source will
  keep transmitting for a while.

Change-Id: I556a6efff0e74aab897bd8165200eec36e46629f
Closes: OS#2703
2018-06-28 14:15:10 +02:00
Philipp Maier 76a8d0160e MGCP_Test: ts_CRCX_no_lco: check media description instead of media attribute
The testcase ts_CRCX_no_lco looks at the media attributes to see
if it findes the expected default codec there. In this testcase
we expect PCMU/8000/1 as media attribute, but this is a codec from
the fixed payload type domain. The MGW may not list this info inside
the media attributes. Listing the payload type number in the media
description is sufficient. We should check this instead.

- Remove media attribute check
- Check meida description for PCMU (0)

Change-Id: I69600a1025e68011e8fc5d8bf22d842d9c63bf53
Related: OS#2658
2018-06-23 11:40:04 +00:00
Pau Espin a4cecf5af7 Add README.md file for several suites
Content grabbed from [1] and converted to markdown as it's more probable
to have it displayed correctly on popular git repo hosting platforms.
Syntax for embedded graphviz taken from [2].

[1] http://git.gnumonks.org/laforge-slides/plain/2018/osmodevcon2018-ttcn3_test_suites/osmodevcon2018-ttcn3_test_suites.adoc
[2] https://github.com/stlehmann/markdown-graphviz

Change-Id: I9aad03e605569e9613665a7585d60bf308bcb54f
2018-06-14 18:55:36 +02:00
Philipp Maier 45635f4379 MGCP_Test: Test non LCO crcx
When a CRCX without an LCO option (codec) is sent, then older versions
of osmo-mgw will omit the port number in the SDP part of the response.
Also no default codec is selected and reported back. This testcase
pinpoints the problem.

Change-Id: Ie16cdab936ce468fe378d4ec9e1c61f81c07fb4e
Related: OS#2658
2018-06-06 17:16:21 +02:00
Philipp Maier ac8fc7f859 MGCP_Test: expect TC_crcx_illegal_double_lco to pass
The osmo-mgw now rejects multiple appearances of LCO, the testcase
TC_crcx_illegal_double_lco now passes.

- update expected-results.xml

Change-Id: If4a68e9373b34696236935cce936e9d3c254511b
Related: OS#3119
2018-06-06 16:54:01 +02:00
Harald Welte 9220f6336e Print more self-explanatory error message on bind/connect failures
When sockets cannot be bound or connected, the existing TTCN-3 code prints
the following rather cryptic error messages:

"IPA-CTRL-IPA(47)@f70ff1fd5cfd: Dynamic test case error: Using the value of an optional field containing omit. (Transport endpoint is not connected)"

The "Transport endpoint is not connected" sort-of gives it away, but
let's make it more explicit by introducing explicit checks for the
res.connId and manual setverdict(fail) statements with proper error
message.

Change-Id: Id22a1b5189d81c4fca03d5e7aff60ffdd1ad56bf
2018-05-23 20:27:02 +02:00
Neels Hofmeyr fc0384a046 mask timestamps and source file nrs in expected-results.xml files
Prepare for upcoming updates with concise diffs.

Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 19:32:01 +02:00
Neels Hofmeyr 1fd6679d9d fix build: don't clean out expected-results.log: rename to *.xml
'make clean' as generated by ttcn3_makefilegen removes all *.log files, which
of course cleans out expected-results.log, which should not happen. Since this
is a junit XML file, rename the suffix to .xml.

Change-Id: Ic334f6b758eef865e3a497aa430691a3ae696d25
2018-04-11 19:29:18 +02:00
Neels Hofmeyr 3cf797d7a5 add compare-results.sh, call from start-testsuite.sh
Compare current test results to the expected results, and exit in error on
discrepancies.

Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.

Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".

In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.

In start-testsuite.sh, after running the tests, invoke the results comparison.

Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.

Related: OS#3136
Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-06 01:07:15 +02:00
Harald Welte edc887435e mgw: Fix mixed-up MGW / local IP address
Change-Id: Ie4f6da896f53e9fabd7275fe6d8f8c6309e02d14
2018-03-29 18:01:51 +02:00
Harald Welte 935cb8f6ba mgw: Don't use RTP_Endpoint anymore. We use RTP_Emulation new.
Change-Id: I6302412daaafc37c8e7740cce4806405d3086fa7
2018-03-29 18:01:51 +02:00
Harald Welte bb7523b15b mgw: Add first tests for testing RTP streams
The existing MGW tests were entirely on the MGCP side.  Let's start
some tests that exchange RTP frames with the MGW and validate that
the MGW can actually act on what is configured via MGCP.

Change-Id: If620d5f8927d0e3584e90a7a8f785c6fdd7c2d17
2018-03-29 08:52:01 +02:00
Harald Welte cb8b427db1 mgw: Add statistics to RTP_Emulation and link/build it in mgw test
We're not using it so far, this is left for follow-up patches.

Change-Id: I40c322374cd99adb75a0f09175023fc0b12291d2
2018-03-29 08:49:15 +02:00
Neels Hofmeyr cd11223c0f ignore results of .ttcnpp files
Add another macro ignore_pp_results to gen_links.sh.inc and call from all
gen_links.sh files, to add results of *.ttcnpp files, i.e. generated *.ttcn
files, to .gitignore.

Change-Id: Ic7fb176226771212d7700dafaf27ac71f12a4a61
2018-03-16 18:25:58 +01:00
Philipp Maier ade455b09c MGCP_Test: fix config file include path
The config file mistakenly references the defaults for the MSC.

- correct include path to use MGCP_Test.default

Change-Id: I12c6907717dfba1c31790a2b71f42e37743351af
2018-03-16 13:46:40 +01:00
Neels Hofmeyr 1a4c4601d7 fix gen_links.sh: don't include source dir as link target
First of all, use one common place to define the gen_links() macro, in
gen_links.sh.inc.

In this new file, add a 'shift' to exclude the $DIR arg from also appearing in
$FILES.

This prevents the following wrong symlinks in the source dirs:

  M3UA_CNL113537/src/src
  MTP3asp_CNL113337/src/src
  SCCP_CNL113341/src/src

Change-Id: Ia8493e77df1ba8723f2c5d2a49816247b0fb55f7
2018-03-16 00:08:02 +01:00
Harald Welte 99ca30ce2e remove *.default from [EXECUTE] in *.default files
The problem is that the .cfg file cannot override the [EXECUTE]
provided list of tests

Change-Id: I7e1e0499b3b32014b25ae4b515263f57a4ea6daf
2018-03-13 14:30:55 +01:00
Harald Welte e057146df7 re-organize configuration files; add tcpdump integration
Let's make sure we share common configuration between the test
suites and split the config file into a "default" part which is
used (but not copied) in the Docker images, and a "local" part
which is basically those overrides that the user (or docker image)
wants to do from the default.

Change-Id: I3db452e24e5238aa05254d903739c64d202e61db
2018-02-14 18:50:16 +01:00
Max 6367d7b807 Don't symlink non-existent file
Change-Id: I10309f07fb207c027703f0b43a478c152a029b6d
2018-02-05 16:58:27 +01:00
Philipp Maier 8a3dc925da MGCP_Test: test illegal wildcarded DLCX and MDCX
- Add tests to check the behaviour on wildcarded DLCX and MDCX,
  both commands not supported by the MGW and should fail.

Change-Id: I140c6d9e337cd16d6147bb582a9474cf76b5170b
2018-02-02 16:14:37 +01:00
Philipp Maier dffa6a46e6 MGCP_Test: Clean up after TC_crcx_sdp
The testcase TC_crcx_sdp does not clean up the connection it
created. This can cause subsequent tests to fail.

- Use f_dlcx_ok() to clean up the connection once the test
  is done.

Change-Id: I4e0f5ce218fadc5debe0bbf548b69919584ab66c
2018-02-02 16:14:37 +01:00
Philipp Maier 5e06ceefbe MCGP_Test: Add tests for wildcarded CRCX
The wildcarded CRCX is not tested yet.

- Add a test function that executes a single wildcarded CRCX
  request

- Add a test function that exchausts all MGW resources using
  wildcarded CRCX requests

Change-Id: I70cf486dc21aef19e4199289f5a5509f6927713e
2018-02-02 16:14:37 +01:00
Philipp Maier bb7a01c696 mgw: add constants for mgw endpoint and domain name
The testcase for osmo-mgw hardcodes the mgw domain and also lacks
an endpoint identifier, which was deprecated recently.

- replace the hardcoded occurrences of the mgw domain name with
  a constant

- add a constant for the rtpbridge endpoint identifier

- add a testcase that still uses the old method without prefix
  on the endpoint identifier.

Change-Id: If4455c4cb521270b2fe24881ade9b578a5132205
2018-02-02 16:14:28 +01:00
Harald Welte 818b6e4225 Add "SourceInfoFormat := Single;" to all test configs
This will make sure that all log files will contain information about
the .ttcn source file name and line number that has caused the log,
which is extremely useful during debugging.

Change-Id: Id6785757f20279ba84b34747f878baf67d065b20
2018-01-17 11:11:29 +00:00
Harald Welte f91edf3b49 MGCP_Test: be more liberal in some of the error codes we expect
Sometimes it's not entirely clear which error code should be returned,
so let's accept multiple options, depending on the situation

Change-Id: I6d6eaec4d025b9da170f59c759884afe2e768521
2017-12-28 14:21:03 +01:00
Harald Welte 33d8216ab4 MGCP: Add 30 endpoint number test to control section
Change-Id: Ifa03697531680c72e4179a4195aeac371660b722
2017-12-28 14:20:53 +01:00
Harald Welte 646ecdb8e4 MGCP_Test: Test use of 30 endpoint numbers
We used to have a big (OS#2784) where only endpoint numbers below
10 (0x0A) were working due to failed interpretation in hex/dec.

Let's use up to 30 endpoints to ensure hex as well as two-digit
digit numbers.

Change-Id: Ifb3e01b5d31decefbf0c87ea39ec5021c8795579
2017-12-28 03:28:40 +01:00
Harald Welte 152c38c765 move RTP_CodecPort.ttcn to library
Change-Id: I67343f016757c23a85db6b526b6f60476e91ffc3
2017-12-13 13:19:26 +01:00
Harald Welte 20a58c6d6f generalize shared code from regen_makefile.sh into shared helper script
Change-Id: I86a738420851a7d9e3bdb2671f6862c3d505a4ba
2017-12-13 01:04:42 +01:00
Harald Welte 9fbe93f532 regen_makefile.sh: Fix TITAN 6.1.0 support
When introducing the rewriting for 6.3.0 we broke the 6.1.0 support.
Now the result should build on Debian with both 6.1.0 and 6.3.0

Change-Id: I263a6abd2d9eb15ce5778ef8dbfcdac2a228b77d
2017-12-13 00:50:11 +01:00