ttcn3
/
osmo-ttcn3-hacks
mirror of https://gerrit.osmocom.org/osmo-ttcn3-hacks
10
0
Fork 1
Code Issues Releases Wiki Activity
osmo-ttcn3-hacks/msc/expected-results.xml

101 lines
7.3 KiB
XML
Raw Normal View History

add compare-results.sh, call from start-testsuite.sh Compare current test results to the expected results, and exit in error on discrepancies. Add compare-result.sh: (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In start-testsuite.sh, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-05 14:56:38 +00:00
<?xml version="1.0"?>
msc: update expected results Change-Id: I1151e9333da14bb2714712b57914e03e4d5b785b
2019-02-17 20:40:59 +00:00
<testsuite name='Titan' tests='84' failures='2' errors='1' skipped='0' inconc='0' time='MASKED'>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MSC_Tests' name='TC_cr_before_reset' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_imsi_noauth_tmsi' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_imsi_noauth_notmsi' time='MASKED'/>
update expected results Change-Id: I32c29e62ca317937db771f8fb1540bb1fe9da2ab
2018-09-06 12:13:34 +00:00
<testcase classname='MSC_Tests' name='TC_lu_imsi_reject' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_imsi_timeout_gsup' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_cmserv_imsi_unknown' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_and_mo_call' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_auth_sai_timeout' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_auth_sai_err' time='MASKED'/>
update expected results bsc: TC_assignment_sign fails with different message msc fixed: TC_lu_clear_request TC_emerg_call_imei_reject TC_cm_serv_req_vgcs_reject TC_cm_serv_req_vbs_reject TC_cm_serv_req_lcs_reject TC_cm_reest_req_reject TC_cl3_rnd_payload TC_lu_and_mt_sms new: TC_smpp_mo_sms TC_smpp_mt_sms sgsn fixed: TC_attach_umts_aka_gsm_sres Change-Id: Ie9ef25fb2081ebab7a2b08c06307fa391f8f747a
2018-05-02 09:59:18 +00:00
<testcase classname='MSC_Tests' name='TC_lu_clear_request' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MSC_Tests' name='TC_lu_disconnect' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_by_imei' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_by_tmsi_noauth_unknown' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_imsi_detach_by_imsi' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_imsi_detach_by_tmsi' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_imsi_detach_by_imei' time='MASKED'/>
update expected results bsc: TC_assignment_sign fails with different message msc fixed: TC_lu_clear_request TC_emerg_call_imei_reject TC_cm_serv_req_vgcs_reject TC_cm_serv_req_vbs_reject TC_cm_serv_req_lcs_reject TC_cm_reest_req_reject TC_cl3_rnd_payload TC_lu_and_mt_sms new: TC_smpp_mo_sms TC_smpp_mt_sms sgsn fixed: TC_attach_umts_aka_gsm_sres Change-Id: Ie9ef25fb2081ebab7a2b08c06307fa391f8f747a
2018-05-02 09:59:18 +00:00
<testcase classname='MSC_Tests' name='TC_emerg_call_imei_reject' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MSC_Tests' name='TC_emerg_call_imsi' time='MASKED'/>
update expected results bsc: TC_assignment_sign fails with different message msc fixed: TC_lu_clear_request TC_emerg_call_imei_reject TC_cm_serv_req_vgcs_reject TC_cm_serv_req_vbs_reject TC_cm_serv_req_lcs_reject TC_cm_reest_req_reject TC_cl3_rnd_payload TC_lu_and_mt_sms new: TC_smpp_mo_sms TC_smpp_mt_sms sgsn fixed: TC_attach_umts_aka_gsm_sres Change-Id: Ie9ef25fb2081ebab7a2b08c06307fa391f8f747a
2018-05-02 09:59:18 +00:00
<testcase classname='MSC_Tests' name='TC_cm_serv_req_vgcs_reject' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_cm_serv_req_vbs_reject' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_cm_serv_req_lcs_reject' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_cm_reest_req_reject' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MSC_Tests' name='TC_lu_auth_2G_fail' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_13_13' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_cl3_no_payload' time='MASKED'/>
update expected results bsc: TC_assignment_sign fails with different message msc fixed: TC_lu_clear_request TC_emerg_call_imei_reject TC_cm_serv_req_vgcs_reject TC_cm_serv_req_vbs_reject TC_cm_serv_req_lcs_reject TC_cm_reest_req_reject TC_cl3_rnd_payload TC_lu_and_mt_sms new: TC_smpp_mo_sms TC_smpp_mt_sms sgsn fixed: TC_attach_umts_aka_gsm_sres Change-Id: Ie9ef25fb2081ebab7a2b08c06307fa391f8f747a
2018-05-02 09:59:18 +00:00
<testcase classname='MSC_Tests' name='TC_cl3_rnd_payload' time='MASKED'/>
expect TC_establish_and_nothing to pass The test case TC_establish_and_nothing is now passing. Update expected results list accordingly. Change-Id: I925fa4ad2e38e189cf5dd1ae76a24cdb9011fdc8 Related: OS#2879
2018-06-01 15:30:45 +00:00
<testcase classname='MSC_Tests' name='TC_establish_and_nothing' time='MASKED'/>
update expected results Change-Id: Idacaf8343bed4a37878eacdf338c4d5eb46bf7a7
2019-01-23 11:44:09 +00:00
<testcase classname='MSC_Tests' name='TC_mo_setup_and_nothing' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MSC_Tests' name='TC_mo_crcx_ran_timeout' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_mo_crcx_ran_reject' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_mt_crcx_ran_reject' time='MASKED'>
add compare-results.sh, call from start-testsuite.sh Compare current test results to the expected results, and exit in error on discrepancies. Add compare-result.sh: (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In start-testsuite.sh, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-05 14:56:38 +00:00
<failure type='fail-verdict'>Timeout waiting for channel release
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
MSC_Tests.ttcn:MASKED MSC_Tests control part
MSC_Tests.ttcn:MASKED TC_mt_crcx_ran_reject testcase
add compare-results.sh, call from start-testsuite.sh Compare current test results to the expected results, and exit in error on discrepancies. Add compare-result.sh: (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In start-testsuite.sh, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-05 14:56:38 +00:00
</failure>
</testcase>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MSC_Tests' name='TC_mo_setup_and_dtmf_dup' time='MASKED'/>
update expected results Change-Id: Idacaf8343bed4a37878eacdf338c4d5eb46bf7a7
2019-01-23 11:44:09 +00:00
<testcase classname='MSC_Tests' name='TC_gsup_cancel' time='MASKED'/>
mask timestamps and source file nrs in expected-results.xml files Prepare for upcoming updates with concise diffs. Change-Id: Ic9f006aa8db1b477598605e0525faeb229b03641
2018-04-11 13:56:41 +00:00
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_1_13' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_3_13' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_3_1' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_3_1_no_cm' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_13_2' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_013_2' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_mo_release_timeout' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_and_mt_call_no_dlcx_resp' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_reset_two' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_lu_and_mt_call' time='MASKED'/>
bsc, msc: update expected results Mark TC_paging_imsi_a_reset fixed. Add various new tests. Change-Id: Ib3a36efeb086fd995d7dad4e040f5a46b1b1ca0a
2018-04-11 13:54:07 +00:00
<testcase classname='MSC_Tests' name='TC_lu_and_mo_sms' time='MASKED'/>
update expected results bsc: TC_assignment_sign fails with different message msc fixed: TC_lu_clear_request TC_emerg_call_imei_reject TC_cm_serv_req_vgcs_reject TC_cm_serv_req_vbs_reject TC_cm_serv_req_lcs_reject TC_cm_reest_req_reject TC_cl3_rnd_payload TC_lu_and_mt_sms new: TC_smpp_mo_sms TC_smpp_mt_sms sgsn fixed: TC_attach_umts_aka_gsm_sres Change-Id: Ie9ef25fb2081ebab7a2b08c06307fa391f8f747a
2018-05-02 09:59:18 +00:00
<testcase classname='MSC_Tests' name='TC_lu_and_mt_sms' time='MASKED'/>
MSC_Test: Test what happens when Paging for SMS is unanswered Trigger sending of an SM, but ignore any paging requests from the MSC, make sure that the MSC is not paging indefinitely Change-Id: Id645729551672026c6a96bb849ecd04f20cd0c56 Related: OS#3704
2018-11-22 18:01:33 +00:00
<testcase classname='MSC_Tests' name='TC_lu_and_mt_sms_paging_and_nothing' time='MASKED'/>
update expected results bsc: TC_assignment_sign fails with different message msc fixed: TC_lu_clear_request TC_emerg_call_imei_reject TC_cm_serv_req_vgcs_reject TC_cm_serv_req_vbs_reject TC_cm_serv_req_lcs_reject TC_cm_reest_req_reject TC_cl3_rnd_payload TC_lu_and_mt_sms new: TC_smpp_mo_sms TC_smpp_mt_sms sgsn fixed: TC_attach_umts_aka_gsm_sres Change-Id: Ie9ef25fb2081ebab7a2b08c06307fa391f8f747a
2018-05-02 09:59:18 +00:00
<testcase classname='MSC_Tests' name='TC_smpp_mo_sms' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_smpp_mt_sms' time='MASKED'/>
MSC_Tests.ttcn: introduce TC_gsup_mo_sms for MO SMS over GSUP The idea of this test case is to verify MO SMS transmission over GSUP towards HLR. Basically, the algorythm is the following: 1.0 establish a RAN connection, 2.1 send CP-DATA/RP-DATA on BSSAP/DTAP, 2.2 expect MO-ForwardSM-Req on GSUP, 3.1 send MO-ForwardSM-Res on GSUP, 3.2 expect CP-DATA/RP-ACK on BSSAP/DTAP. Change-Id: Id14bbd8bd51558cdacefea0fe042769cd69ed5c8 Related: OS#3587
2018-11-11 19:50:23 +00:00
<testcase classname='MSC_Tests' name='TC_gsup_mo_sms' time='MASKED'/>
MSC_Tests.ttcn: introduce TC_gsup_mo_smma for MO SMMA over GSUP The idea of this test case is to verify MO SMMA transmission over GSUP towards HLR. The algorythm is equivalent to MO SMS. Change-Id: I7abc95b8e416f7308d54e11be11c08586d18e6c5 Related: OS#3587
2018-11-14 19:06:07 +00:00
<testcase classname='MSC_Tests' name='TC_gsup_mo_smma' time='MASKED'/>
MSC_Tests.ttcn: introduce TC_gsup_mt_sms_{ack|err} The idea of this test case is to verify MT SMS transmission initiated by ESME over GSUP. Basically, the algorythm is the following: 1.0 send MT-ForwardSM-Req on GSUP, 1.1 expect Paging Request on RAN, 1.2 establish a RAN connection, 1.3 expect CP-DATA/RP-DATA on BSSAP/DTAP, 2.1 send CP-DATA/RP-ACK on BSSAP/DTAP, 2.2.a expect MT-ForwardSM-Res, 2.2.b expect MT-ForwardSM-Err. Change-Id: I63a25c8366cce0852df6b628365151661a22a25f Related: OS#3587
2018-11-23 20:40:20 +00:00
<testcase classname='MSC_Tests' name='TC_gsup_mt_sms_ack' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_gsup_mt_sms_err' time='MASKED'/>
MSC_Tests.ttcn: introduce TC_gsup_mt_sms_rp_mr The idea of this test case is to verify SM-RP-MR assignment for a few MT SMS being sent over GSUP. Basically, the algorythm is the following: 1.0 send the 1st SMS using MT-ForwardSM-Req on GSUP, 1.1 expect Paging Request on RAN, 1.2 establish a RAN connection, 1.3 expect CP-DATA/RP-DATA for the 1st SMS, 2.0 send the 2nd SMS using MT-ForwardSM-Req on GSUP, 2.1 expect CP-DATA/RP-DATA for the 2nd SMS, 3.0 compare both SM-RP-MR values assigned by the MSC, 3.1 send CP-DATA/RP-ACK for both 1st and 2nd SMS messages, 3.2 expect MT-ForwardSM-Res for both 1st and 2nd SMS messages. The SM-RP-MR values for both 1st and 2nd messages shall not match. Change-Id: I3a52d44f4abde9b6b471b9108c1cee905884c9bc
2019-01-18 08:04:13 +00:00
<testcase classname='MSC_Tests' name='TC_gsup_mt_sms_rp_mr' time='MASKED'/>
MSC_Tests.ttcn: introduce TC_gsup_mo_mt_sms_rp_mr The idea of this test case is to verify SM-RP-MR assignment for a few concurrent MO/MT SMS being sent over GSUP. Basically, the algorythm is the following: 1.0 establish a RAN connection, 1.1 send CM Service Request for MO SMMA indication, 1.2 submit MO SMMA indication on DTAP, 1.3 expect MO-ForwardSM-Req on GSUP, 2.0 send MT SMS using MT-ForwardSM-Req on GSUP, 2.1 expect CP-DATA/RP-DATA for MT SMS on DTAP, 3.0 compare both SM-RP-MR values (for MT, assigned by the MSC), 3.1 send MO-ForwardSM-Res for MO SMMA on GSUP, 3.1.1 expect CP-DATA/RP-ACK for MO SMMA on DTAP, 3.2 send CP-DATA/RP-ACK for MT SMS on DTAP, 3.2.1 expect MT-ForwardSM-Res for MT SMS on GSUP. Change-Id: I17cbbaa64d9bce770f985588e93cd3eecd732120
2019-01-24 09:57:31 +00:00
<testcase classname='MSC_Tests' name='TC_gsup_mo_mt_sms_rp_mr' time='MASKED'/>
msc/USSD: use more informative names for test cases Let's explicitly indicate is a SS/USSD message MO or MT. Change-Id: I87f16f935f015dbd2ac2867d8ea5e155cc365e3f
2018-06-19 10:51:20 +00:00
<testcase classname='MSC_Tests' name='TC_lu_and_mo_ussd_single_request' time='MASKED'/>
msc/USSD: add test cases with network-initiaded SS/USSD This change introduces two new test cases for network-initiaded USSD notification and network-initiaded USSD request, which are based on the existing SS/USSD related test cases. The idea of TC_lu_and_mt_ussd_notification is to verify that a network-initiaded USSD notification can arrive subscriber in IDLE mode using Paging procedure. The idea of TC_lu_and_mt_ussd_during_mt_call is to verify that a network-initiaded USSD notification can arrive subscriber in DEDICATED mode (in this case during a call) on a separate transaction. Change-Id: I073893c6e11be27e9e36f98f11c1491d0c173985
2018-06-19 11:24:31 +00:00
<testcase classname='MSC_Tests' name='TC_lu_and_mt_ussd_notification' time='MASKED'/>
msc/USSD: use more informative names for test cases Let's explicitly indicate is a SS/USSD message MO or MT. Change-Id: I87f16f935f015dbd2ac2867d8ea5e155cc365e3f
2018-06-19 10:51:20 +00:00
<testcase classname='MSC_Tests' name='TC_lu_and_mo_ussd_during_mt_call' time='MASKED'/>
msc/USSD: add test cases with network-initiaded SS/USSD This change introduces two new test cases for network-initiaded USSD notification and network-initiaded USSD request, which are based on the existing SS/USSD related test cases. The idea of TC_lu_and_mt_ussd_notification is to verify that a network-initiaded USSD notification can arrive subscriber in IDLE mode using Paging procedure. The idea of TC_lu_and_mt_ussd_during_mt_call is to verify that a network-initiaded USSD notification can arrive subscriber in DEDICATED mode (in this case during a call) on a separate transaction. Change-Id: I073893c6e11be27e9e36f98f11c1491d0c173985
2018-06-19 11:24:31 +00:00
<testcase classname='MSC_Tests' name='TC_lu_and_mt_ussd_during_mt_call' time='MASKED'/>
msc/USSD: introduce TC_lu_and_mo_ussd_mo_release The idea of this test case is to check the reaction of OsmoMSC on MS-initiated release during an active transaction. In other words, when the network is waiting for some response from a MS, subscriber can press the 'red button' in order to terminate this conversation. It is expected that the MSC would terminate the transaction as on DTAP interface, as on GSUP interface. Change-Id: I7936ed5072ed2ae02f039dc90a1fece1e7f70a70
2018-06-20 21:19:58 +00:00
<testcase classname='MSC_Tests' name='TC_lu_and_mo_ussd_mo_release' time='MASKED'/>
update expected results Change-Id: Idacaf8343bed4a37878eacdf338c4d5eb46bf7a7
2019-01-23 11:44:09 +00:00
<testcase classname='MSC_Tests' name='TC_lu_and_ss_session_timeout' time='MASKED'>
<error type='DTE'>Dynamic test case error: testcase.stop</error>
</testcase>
add MSC test for an invalid CIPHER MODE COMPLETE command Add new test TC_cipher_complete_with_invalid_cipher which verifies that the MSC will reject a CIPHER MODE COMPLETE command with a cipher which wasn't part of the preceding CIPHER MODE command. Change-Id: I4492eb7d77371aaa047abae81a2dcf26fe46eb6a Related: OS#2872
2018-12-17 14:06:20 +00:00
<testcase classname='MSC_Tests' name='TC_cipher_complete_with_invalid_cipher' time='MASKED'/>
msc: update expected results Change-Id: I1151e9333da14bb2714712b57914e03e4d5b785b
2019-02-17 20:40:59 +00:00
<testcase classname='MSC_Tests' name='TC_sgsap_reset' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_lu' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_lu_imsi_reject' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_lu_and_nothing' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_expl_imsi_det_eps' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_expl_imsi_det_noneps' time='MASKED'/>
MSC_Tests: add testcase TC_sgsap_impl_imsi_det_noneps We have a testcase that sends an explicit (UE-Initiated) imsi detach from non EPS services. Lets also cover the case for an implicit (Network-initated) detach. Change-Id: I76049e6717680c54c18f97b7cd51944901a81ae7 Related: OS#3614
2019-03-21 09:51:26 +00:00
<testcase classname='MSC_Tests' name='TC_sgsap_impl_imsi_det_noneps' time='MASKED'/>
msc: update expected results Change-Id: I1151e9333da14bb2714712b57914e03e4d5b785b
2019-02-17 20:40:59 +00:00
<testcase classname='MSC_Tests' name='TC_sgsap_paging_rej' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_paging_subscr_rej' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_paging_ue_unr' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_paging_and_nothing' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_paging_and_lu' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_mt_sms' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_mo_sms' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_mt_sms_and_nothing' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_mt_sms_and_reject' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_unexp_ud' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_unsol_ud' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_bssap_lu_sgsap_lu_and_mt_call' time='MASKED'/>
<testcase classname='MSC_Tests' name='TC_sgsap_lu_and_mt_call' time='MASKED'/>
update expected results Change-Id: I32c29e62ca317937db771f8fb1540bb1fe9da2ab
2018-09-06 12:13:34 +00:00
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_3_1_log_msc_debug' time='MASKED'/>
msc: update expected results Change-Id: I1151e9333da14bb2714712b57914e03e4d5b785b
2019-02-17 20:40:59 +00:00
<testcase classname='MSC_Tests' name='TC_gsup_mt_multi_part_sms' time='MASKED'>
update expected results Change-Id: Idacaf8343bed4a37878eacdf338c4d5eb46bf7a7
2019-01-23 11:44:09 +00:00
<failure type='fail-verdict'>Tguard timeout
MSC_Tests.ttcn:MASKED MSC_Tests control part
msc: update expected results Change-Id: I1151e9333da14bb2714712b57914e03e4d5b785b
2019-02-17 20:40:59 +00:00
MSC_Tests.ttcn:MASKED TC_gsup_mt_multi_part_sms testcase
update expected results Change-Id: Idacaf8343bed4a37878eacdf338c4d5eb46bf7a7
2019-01-23 11:44:09 +00:00
</failure>
</testcase>
msc: update expected results Change-Id: I1151e9333da14bb2714712b57914e03e4d5b785b
2019-02-17 20:40:59 +00:00
<testcase classname='MSC_Tests' name='TC_mo_cc_bssmap_clear' time='MASKED'/>
add compare-results.sh, call from start-testsuite.sh Compare current test results to the expected results, and exit in error on discrepancies. Add compare-result.sh: (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In start-testsuite.sh, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-05 14:56:38 +00:00
</testsuite>