ttcn3
/
osmo-ttcn3-hacks
mirror of https://gerrit.osmocom.org/osmo-ttcn3-hacks
10
0
Fork 1
Code Issues Releases Wiki Activity
osmo-ttcn3-hacks/mgw/expected-results.xml

53 lines
4.0 KiB
XML
Raw Normal View History

add compare-results.sh, call from start-testsuite.sh Compare current test results to the expected results, and exit in error on discrepancies. Add compare-result.sh: (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In start-testsuite.sh, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-05 14:56:38 +00:00
<?xml version="1.0"?>
update expected results Change-Id: I37014274ee97f09985c31966e7cc9122fe11a856
2020-05-14 19:10:28 +00:00
<testsuite name='Titan' tests='49' failures='0' errors='0' skipped='0' inconc='0' time='MASKED'>
<testcase classname='MGCP_Test' name='TC_selftest' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MGCP_Test' name='TC_crcx' time='MASKED'/>
update expected results Change-Id: I32c29e62ca317937db771f8fb1540bb1fe9da2ab
2018-09-06 12:13:34 +00:00
<testcase classname='MGCP_Test' name='TC_crcx_no_lco' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MGCP_Test' name='TC_crcx_noprefix' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_unsupp_mode' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_early_bidir_mode' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_unsupp_param' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_missing_callid' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_missing_mode' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_unsupp_packet_intv' time='MASKED'/>
MGCP_Test: expect TC_crcx_illegal_double_lco to pass The osmo-mgw now rejects multiple appearances of LCO, the testcase TC_crcx_illegal_double_lco now passes. - update expected-results.xml Change-Id: If4a68e9373b34696236935cce936e9d3c254511b Related: OS#3119
2018-06-06 11:15:03 +00:00
<testcase classname='MGCP_Test' name='TC_crcx_illegal_double_lco' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MGCP_Test' name='TC_crcx_sdp' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_wildcarded' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_wildcarded_exhaust' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_mdcx_without_crcx' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_dlcx_without_crcx' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_mdcx_wildcarded' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_dlcx_wildcarded' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_and_dlcx_ep_callid_connid' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_and_dlcx_ep_callid' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_and_dlcx_ep' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_and_dlcx_ep_callid_inval' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_and_dlcx_ep_callid_connid_inval' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_and_dlcx_retrans' time='MASKED'/>
Add Osmux support and tests for MGW Depends: osmo-mgw.git Iac073f1db46569b46eddeaecc9934a2986bd50f1 Change-Id: Ibb58b2a4e08d6f30cfe347c217794d0d1310954f
2019-05-14 11:40:49 +00:00
<testcase classname='MGCP_Test' name='TC_crcx_osmux_wildcard' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_osmux_fixed' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_crcx_osmux_fixed_twice' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_one_crcx_receive_only_osmux' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_one_crcx_loopback_osmux' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_two_crcx_and_rtp_osmux' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_two_crcx_and_rtp_osmux_bidir' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_two_crcx_mdcx_and_rtp_osmux_wildcard' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_two_crcx_mdcx_and_rtp_osmux_fixed' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MGCP_Test' name='TC_crcx_dlcx_30ep' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_rtpem_selftest' time='MASKED'/>
MGCP_Test: add tests to verify actual RTP flows The test coverage of the RTP aspects of the MGW is currently very minima. Lets add a few more testcase to verify RTP behaves as expected in various situations. - Add testcase TC_one_crcx_receive_only_rtp: Test recvonly mode of the MGW. All packets must be absorbed by the MGW, no packets must come back. - Add testcase TC_one_crcx_loopback_rtp: Test loopback mode of the MGW. All packet sent to the MGW must come back. - Add testcase TC_two_crcx_and_rtp_bidir: We already test unidirectional transmissions. This test does the same as TC_two_crcx_and_rtp but for both directions. - Add testcase TC_two_crcx_mdcx_and_rtp: Simulate a typical behaviour of a normal call. First create two half open connections and complete the connections later using MDCX. - Add testcase TC_two_crcx_and_unsolicited_rtp: Test what happens when a RTP packets from rogue source are mixed into the RTP stream. - Add testcase TC_two_crcx_and_one_mdcx_rtp_ho: Test a typical handover situation. An existing connection is handovered to another source on one end but the old source will keep transmitting for a while. Change-Id: I556a6efff0e74aab897bd8165200eec36e46629f Closes: OS#2703
2018-06-27 15:52:04 +00:00
<testcase classname='MGCP_Test' name='TC_one_crcx_receive_only_rtp' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_one_crcx_loopback_rtp' time='MASKED'/>
update expected results Change-Id: I32c29e62ca317937db771f8fb1540bb1fe9da2ab
2018-09-06 12:13:34 +00:00
<testcase classname='MGCP_Test' name='TC_two_crcx_and_rtp' time='MASKED'/>
MGCP_Test: add tests to verify actual RTP flows The test coverage of the RTP aspects of the MGW is currently very minima. Lets add a few more testcase to verify RTP behaves as expected in various situations. - Add testcase TC_one_crcx_receive_only_rtp: Test recvonly mode of the MGW. All packets must be absorbed by the MGW, no packets must come back. - Add testcase TC_one_crcx_loopback_rtp: Test loopback mode of the MGW. All packet sent to the MGW must come back. - Add testcase TC_two_crcx_and_rtp_bidir: We already test unidirectional transmissions. This test does the same as TC_two_crcx_and_rtp but for both directions. - Add testcase TC_two_crcx_mdcx_and_rtp: Simulate a typical behaviour of a normal call. First create two half open connections and complete the connections later using MDCX. - Add testcase TC_two_crcx_and_unsolicited_rtp: Test what happens when a RTP packets from rogue source are mixed into the RTP stream. - Add testcase TC_two_crcx_and_one_mdcx_rtp_ho: Test a typical handover situation. An existing connection is handovered to another source on one end but the old source will keep transmitting for a while. Change-Id: I556a6efff0e74aab897bd8165200eec36e46629f Closes: OS#2703
2018-06-27 15:52:04 +00:00
<testcase classname='MGCP_Test' name='TC_two_crcx_and_rtp_bidir' time='MASKED'/>
update expected results Change-Id: I32c29e62ca317937db771f8fb1540bb1fe9da2ab
2018-09-06 12:13:34 +00:00
<testcase classname='MGCP_Test' name='TC_two_crcx_diff_pt_and_rtp' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_two_crcx_diff_pt_and_rtp_bidir' time='MASKED'/>
MGCP_Test: add tests to verify actual RTP flows The test coverage of the RTP aspects of the MGW is currently very minima. Lets add a few more testcase to verify RTP behaves as expected in various situations. - Add testcase TC_one_crcx_receive_only_rtp: Test recvonly mode of the MGW. All packets must be absorbed by the MGW, no packets must come back. - Add testcase TC_one_crcx_loopback_rtp: Test loopback mode of the MGW. All packet sent to the MGW must come back. - Add testcase TC_two_crcx_and_rtp_bidir: We already test unidirectional transmissions. This test does the same as TC_two_crcx_and_rtp but for both directions. - Add testcase TC_two_crcx_mdcx_and_rtp: Simulate a typical behaviour of a normal call. First create two half open connections and complete the connections later using MDCX. - Add testcase TC_two_crcx_and_unsolicited_rtp: Test what happens when a RTP packets from rogue source are mixed into the RTP stream. - Add testcase TC_two_crcx_and_one_mdcx_rtp_ho: Test a typical handover situation. An existing connection is handovered to another source on one end but the old source will keep transmitting for a while. Change-Id: I556a6efff0e74aab897bd8165200eec36e46629f Closes: OS#2703
2018-06-27 15:52:04 +00:00
<testcase classname='MGCP_Test' name='TC_two_crcx_mdcx_and_rtp' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_two_crcx_and_unsolicited_rtp' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_two_crcx_and_one_mdcx_rtp_ho' time='MASKED'/>
MGCP_Test: Add testcase to test ts101318/rfc5993 conversion The MGW recentenly adds support to convert between ts101318 and rfc5993 when GSM-HR is used. Lets add a testcase for that. depends: osmo-mgw Iceef19e5619f8c92dfa7c8cdecb2e9b15f0a11a1 Change-Id: I96df45fe45b53088e07b26f14173a75498a84143 Related: OS#3807
2019-02-21 16:35:01 +00:00
<testcase classname='MGCP_Test' name='TC_ts101318_rfc5993_rtp_conversion' time='MASKED'/>
MGCP_Test: Test AMR octet-aligned/bandwith-efficient mode conversion osmo-mgw now implements a conversion between the AMR octet-aligned and banwith-efficient payload format. Lets add tests for this Change-Id: I050bfeabfb5fdbf986d429eef3af69fe8158d56e depends: osmo-mgw I622c01874b25f5049d4f59eb8157e0ea3cbe16ba related: SYS#4470
2019-03-07 09:54:10 +00:00
<testcase classname='MGCP_Test' name='TC_amr_oa_bwe_rtp_conversion' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_amr_oa_oa_rtp_conversion' time='MASKED'/>
<testcase classname='MGCP_Test' name='TC_amr_bwe_bwe_rtp_conversion' time='MASKED'/>
update expected results Change-Id: I37014274ee97f09985c31966e7cc9122fe11a856
2020-05-14 19:10:28 +00:00
<testcase classname='MGCP_Test' name='TC_conn_timeout' time='MASKED'/>
add compare-results.sh, call from start-testsuite.sh Compare current test results to the expected results, and exit in error on discrepancies. Add compare-result.sh: (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In start-testsuite.sh, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-05 14:56:38 +00:00
</testsuite>