add compare-results.sh, call from start-testsuite.sh
Compare current test results to the expected results, and exit in error on
discrepancies.
Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.
Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".
In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.
In start-testsuite.sh, after running the tests, invoke the results comparison.
Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.
Related: OS#3136
Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-05 14:56:38 +00:00
|
|
|
<?xml version="1.0"?>
|
2019-07-23 11:09:08 +00:00
|
|
|
<testsuite name='Titan' tests='154' failures='9' errors='0' skipped='0' inconc='0' time='MASKED'>
|
2018-04-11 13:56:41 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_cr_before_reset' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_noauth_tmsi' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_noauth_notmsi' time='MASKED'/>
|
2018-09-06 12:13:34 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_reject' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_timeout_gsup' time='MASKED'/>
|
2018-04-11 13:56:41 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi' time='MASKED'/>
|
2019-07-23 13:00:04 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth3g_tmsi' time='MASKED'/>
|
2018-04-11 13:56:41 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_cmserv_imsi_unknown' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mo_call' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_auth_sai_timeout' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_auth_sai_err' time='MASKED'/>
|
2018-05-02 09:59:18 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_clear_request' time='MASKED'/>
|
2018-04-11 13:56:41 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_disconnect' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_by_imei' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_by_tmsi_noauth_unknown' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_imsi_detach_by_imsi' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_imsi_detach_by_tmsi' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_imsi_detach_by_imei' time='MASKED'/>
|
2018-05-02 09:59:18 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_emerg_call_imei_reject' time='MASKED'/>
|
2018-04-11 13:56:41 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_emerg_call_imsi' time='MASKED'/>
|
2018-05-02 09:59:18 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_cm_serv_req_vgcs_reject' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_cm_serv_req_vbs_reject' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_cm_serv_req_lcs_reject' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_cm_reest_req_reject' time='MASKED'/>
|
2018-04-11 13:56:41 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_auth_2G_fail' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_13_13' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_cl3_no_payload' time='MASKED'/>
|
2018-05-02 09:59:18 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_cl3_rnd_payload' time='MASKED'/>
|
2018-06-01 15:30:45 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_establish_and_nothing' time='MASKED'/>
|
2019-01-23 11:44:09 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_mo_setup_and_nothing' time='MASKED'/>
|
2018-04-11 13:56:41 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_mo_crcx_ran_timeout' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_mo_crcx_ran_reject' time='MASKED'/>
|
2019-07-23 13:00:04 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_mt_crcx_ran_reject' time='MASKED'/>
|
2018-04-11 13:56:41 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_mo_setup_and_dtmf_dup' time='MASKED'/>
|
2019-01-23 11:44:09 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_gsup_cancel' time='MASKED'/>
|
2018-04-11 13:56:41 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_1_13' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_3_13' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_3_1' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_3_1_no_cm' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_13_2' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_013_2' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_mo_release_timeout' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mt_call_no_dlcx_resp' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_reset_two' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mt_call' time='MASKED'/>
|
2018-04-11 13:54:07 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mo_sms' time='MASKED'/>
|
2018-05-02 09:59:18 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mt_sms' time='MASKED'/>
|
2018-11-22 18:01:33 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mt_sms_paging_and_nothing' time='MASKED'/>
|
2018-05-02 09:59:18 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_smpp_mo_sms' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_smpp_mt_sms' time='MASKED'/>
|
2018-11-11 19:50:23 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_gsup_mo_sms' time='MASKED'/>
|
2018-11-14 19:06:07 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_gsup_mo_smma' time='MASKED'/>
|
2018-11-23 20:40:20 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_gsup_mt_sms_ack' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_gsup_mt_sms_err' time='MASKED'/>
|
2019-01-18 08:04:13 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_gsup_mt_sms_rp_mr' time='MASKED'/>
|
MSC_Tests.ttcn: introduce TC_gsup_mo_mt_sms_rp_mr
The idea of this test case is to verify SM-RP-MR assignment for
a few concurrent MO/MT SMS being sent over GSUP.
Basically, the algorythm is the following:
1.0 establish a RAN connection,
1.1 send CM Service Request for MO SMMA indication,
1.2 submit MO SMMA indication on DTAP,
1.3 expect MO-ForwardSM-Req on GSUP,
2.0 send MT SMS using MT-ForwardSM-Req on GSUP,
2.1 expect CP-DATA/RP-DATA for MT SMS on DTAP,
3.0 compare both SM-RP-MR values (for MT, assigned by the MSC),
3.1 send MO-ForwardSM-Res for MO SMMA on GSUP,
3.1.1 expect CP-DATA/RP-ACK for MO SMMA on DTAP,
3.2 send CP-DATA/RP-ACK for MT SMS on DTAP,
3.2.1 expect MT-ForwardSM-Res for MT SMS on GSUP.
Change-Id: I17cbbaa64d9bce770f985588e93cd3eecd732120
2019-01-24 09:57:31 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_gsup_mo_mt_sms_rp_mr' time='MASKED'/>
|
2019-07-23 13:00:04 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_gsup_mt_multi_part_sms' time='MASKED'/>
|
2018-06-19 10:51:20 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mo_ussd_single_request' time='MASKED'/>
|
2018-06-19 11:24:31 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mt_ussd_notification' time='MASKED'/>
|
2018-06-19 10:51:20 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mo_ussd_during_mt_call' time='MASKED'/>
|
2018-06-19 11:24:31 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mt_ussd_during_mt_call' time='MASKED'/>
|
2018-06-20 21:19:58 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mo_ussd_mo_release' time='MASKED'/>
|
2019-06-17 16:20:00 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_ss_session_timeout' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_mt_ussd_for_unknown_subscr' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_mo_ussd_for_unknown_trans' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_proc_ss_for_unknown_session' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_proc_ss_paging_fail' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_proc_ss_abort' time='MASKED'/>
|
2018-12-17 14:06:20 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_cipher_complete_with_invalid_cipher' time='MASKED'/>
|
2019-07-23 13:00:04 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_cipher_complete_1_without_cipher' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_cipher_complete_3_without_cipher' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_cipher_complete_13_without_cipher' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_with_invalid_mcc_mnc' time='MASKED'/>
|
2019-02-17 20:40:59 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_reset' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_lu' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_lu_imsi_reject' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_lu_and_nothing' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_expl_imsi_det_eps' time='MASKED'/>
|
2019-03-21 10:17:54 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_impl_imsi_det_eps' time='MASKED'/>
|
2019-02-17 20:40:59 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_expl_imsi_det_noneps' time='MASKED'/>
|
2019-03-21 09:51:26 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_impl_imsi_det_noneps' time='MASKED'/>
|
2019-02-17 20:40:59 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_paging_rej' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_paging_subscr_rej' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_paging_ue_unr' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_paging_and_nothing' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_paging_and_lu' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_mt_sms' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_mo_sms' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_mt_sms_and_nothing' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_mt_sms_and_reject' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_unexp_ud' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_unsol_ud' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_bssap_lu_sgsap_lu_and_mt_call' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_lu_and_mt_call' time='MASKED'/>
|
2019-04-09 15:36:57 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_sgsap_vlr_failure' time='MASKED'/>
|
2019-07-23 13:00:04 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_ho_inter_bsc_unknown_cell' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_ho_inter_bsc' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_ho_inter_msc_out' time='MASKED'/>
|
2019-07-03 08:57:35 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_check_imei' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth3g_tmsi_check_imei' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_noauth_tmsi_check_imei' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_noauth_notmsi_check_imei' time='MASKED'/>
|
2019-07-23 11:09:08 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_check_imei_nack' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_check_imei_err' time='MASKED'/>
|
2019-07-03 08:57:35 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_check_imei_early' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth3g_tmsi_check_imei_early' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_noauth_tmsi_check_imei_early' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_noauth_notmsi_check_imei_early' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_check_imei_early_nack' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_check_imei_early_err' time='MASKED'/>
|
2019-07-23 13:00:04 +00:00
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_imsi_auth_tmsi_encr_3_1_log_msc_debug' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_mo_cc_bssmap_clear' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_lu_and_mt_call_osmux' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_imsi_reject' time='MASKED'>
|
|
|
|
<failure type='fail-verdict'>Tguard timeout
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED MSC_Tests_Iu control part
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED TC_iu_lu_imsi_reject testcase
|
|
|
|
</failure>
|
|
|
|
</testcase>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_imsi_timeout_gsup' time='MASKED'>
|
|
|
|
<failure type='fail-verdict'>Tguard timeout
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED MSC_Tests_Iu control part
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED TC_iu_lu_imsi_timeout_gsup testcase
|
|
|
|
</failure>
|
|
|
|
</testcase>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_imsi_auth3g_tmsi' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_cmserv_imsi_unknown' time='MASKED'>
|
|
|
|
<failure type='fail-verdict'>Tguard timeout
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED MSC_Tests_Iu control part
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED TC_iu_cmserv_imsi_unknown testcase
|
|
|
|
</failure>
|
|
|
|
</testcase>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_mo_call' time='MASKED'>
|
|
|
|
<failure type='fail-verdict'>Tguard timeout
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED MSC_Tests_Iu control part
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED TC_iu_lu_and_mo_call testcase
|
|
|
|
</failure>
|
|
|
|
</testcase>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_auth_sai_timeout' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_auth_sai_err' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_release_request' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_disconnect' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_by_imei' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_imsi_detach_by_imsi' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_imsi_detach_by_tmsi' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_imsi_detach_by_imei' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_emerg_call_imei_reject' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_emerg_call_imsi' time='MASKED'>
|
|
|
|
<failure type='fail-verdict'>Tguard timeout
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED MSC_Tests_Iu control part
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED TC_iu_emerg_call_imsi testcase
|
|
|
|
</failure>
|
|
|
|
</testcase>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_establish_and_nothing' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_mo_setup_and_nothing' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_mo_crcx_ran_timeout' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_mo_crcx_ran_reject' time='MASKED'>
|
|
|
|
<failure type='fail-verdict'>Timeout waiting for channel release
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED MSC_Tests_Iu control part
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED TC_iu_mo_crcx_ran_reject testcase
|
|
|
|
</failure>
|
|
|
|
</testcase>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_mt_crcx_ran_reject' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_gsup_cancel' time='MASKED'>
|
|
|
|
<failure type='fail-verdict'>Tguard timeout
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED MSC_Tests_Iu control part
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED TC_iu_gsup_cancel testcase
|
|
|
|
</failure>
|
|
|
|
</testcase>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_mo_release_timeout' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests' name='TC_reset_two_1iu' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_mo_sms' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_mt_sms' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_mt_sms_paging_and_nothing' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_smpp_mo_sms' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_gsup_mo_sms' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_gsup_mo_smma' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_gsup_mt_sms_ack' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_gsup_mt_sms_err' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_gsup_mt_sms_rp_mr' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_gsup_mo_mt_sms_rp_mr' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_mo_ussd_single_request' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_mt_ussd_notification' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_mo_ussd_during_mt_call' time='MASKED'>
|
|
|
|
<failure type='fail-verdict'>Tguard timeout
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED MSC_Tests_Iu control part
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED TC_iu_lu_and_mo_ussd_during_mt_call testcase
|
|
|
|
</failure>
|
|
|
|
</testcase>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_mt_ussd_during_mt_call' time='MASKED'>
|
|
|
|
<failure type='fail-verdict'>Tguard timeout
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED MSC_Tests_Iu control part
|
|
|
|
MSC_Tests_Iu.ttcn:MASKED TC_iu_lu_and_mt_ussd_during_mt_call testcase
|
|
|
|
</failure>
|
|
|
|
</testcase>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_mo_ussd_mo_release' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_and_ss_session_timeout' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_iu_lu_with_invalid_mcc_mnc' time='MASKED'/>
|
|
|
|
<testcase classname='MSC_Tests_Iu' name='TC_mo_cc_iu_release' time='MASKED'/>
|
add compare-results.sh, call from start-testsuite.sh
Compare current test results to the expected results, and exit in error on
discrepancies.
Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.
Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".
In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.
In start-testsuite.sh, after running the tests, invoke the results comparison.
Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.
Related: OS#3136
Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
2018-04-05 14:56:38 +00:00
|
|
|
</testsuite>
|