Pass the CTRL_DETECT_CONNECTION_ESTABLISHMENT_RESULT parameter to
the TELNET port by default. This allows tests to make progress
into an error handling path if they are started while the osmo-*
program they want to connect on VTY is not running.
Observed with osmo-ggsn tests, where if the one test runs
into a VTY connection failure the subsequent test would get
stuck forever in a map() call on the VTY TELNET port.
Teach the function f_vty_wait_for_prompt() about connection
reports by the TELNET module. We may now receive an integer which
represents the socket file descriptor for the telnet connection.
This case was not handled by the previous change made in
commit cb111b21ab. As a result,
BSC tests started failing with "VTY Timeout for prompt" because
the alt-statement in f_vty_wait_for_prompt() would not progress
past the integer sitting on the VTY port's receive queue.
Change-Id: I56925f93af6c55e93f3f417099db135744da6a40
Related: OS#3149
With this patch, I see all ttcn3-bsc-tests failing with
"Verdict: fail reason: VTY Timeout for prompt"
This reverts commit cb111b21ab.
Change-Id: I215d7ab5eee75cf6d3afaac760af64356c943140
Pass the CTRL_DETECT_CONNECTION_ESTABLISHMENT_RESULT parameter to
the TELNET port by default. This allows tests to make progress
into an error handling path if they are started while the osmo-*
program they want to connect on VTY is not running.
Observed with osmo-ggsn tests, where if the one test runs
into a VTY connection failure the subsequent test would get
stuck forever in a map() call on the VTY TELNET port.
Change-Id: I9acf7793d5d68aec6d087cff254a10d8b673dab1
Related: OS#3149
Call mtc.stop after setverdict(fail), add reasons to most failures and
fail with verdict error for internal errors.
Change-Id: I9b618235939fa41160b9be6677b121963d3ec857
PARALLEL | VERDICTOP will log when the port is dying or when other
components will change to fail. This helped to find a timeout in the
SGSN tests where a function call message timed out.
Change-Id: I770ac964dc37e2752e7d35e493f707b091c739b0
'make clean' as generated by ttcn3_makefilegen removes all *.log files, which
of course cleans out expected-results.log, which should not happen. Since this
is a junit XML file, rename the suffix to .xml.
Change-Id: Ic334f6b758eef865e3a497aa430691a3ae696d25
Compare current test results to the expected results, and exit in error on
discrepancies.
Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.
Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".
In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.
In start-testsuite.sh, after running the tests, invoke the results comparison.
Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.
Related: OS#3136
Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
Stefan Sperling
Neels Hofmeyr
Daniel Willmann
Pau Espin
Harald Welte