Set the executable name in each regen_makefile.sh explicitly with -e,
instead of having it set indirectly from the first .ttcn file. Make it
consistent by placing the name on top of each of these files.
Fix for warning:
ttcn3_makefilegen: warning: File `BSC_Tests.ttcn' was given more than once for the Makefile.
Related: OS#5252
Change-Id: I5ed03f8f3ed905483620dc7bae33b617bbb8507f
Make all regen_makefile.sh more readable and diff friendly by moving
each entry in FILES and CPPFLAGS_TTCN3 into separate lines. Order
entries alphabetically.
Related: OS#5252
Change-Id: I6b6866eb9f6ec6232e4ae434517457a4c8c1c050
Otherwise the SIP message is retransmitted by osmo-sip-connector around
10-11 times, even when next test has started, making it fail since it's
not expected.
Change-Id: Icd0f7a7fac4be2aa97eb986c3d3d27dd70a49357
Default the MNCC version to the current osmo-sip-connector's master branch MNCC
version.
As soon as the new version (6) is merged, we can bump it here to make tests for
master work again.
For 'latest' builds, we can adjust osmo-ttcn3-hacks to use version 5, and also
see those tests still passing.
Change-Id: I3eb6e0132dc99ebe41a98cc5c329ed4864b770d6
Some stuff was wrong and some was missing after new features being
implemented in tests over time.
Change-Id: I7a279592a68ffc76408a8e728e76151534265cc0
Some of our files didn't have a copyright notice at all, let's add
it. Also, update the notices in other files and ensure a SPDX
identifier is present in all but the most trivial files.
Change-Id: If7fa19ce484b415bc645e39b3d0d666b44b5f0fd
This function can now be called from anywhere to try and safely shutdown
a testcase. It is not optimal as we can't call "all component.stop" from
outside the mtc, but without any proper and orderly shutdown handling of
all our emulation components I believe this is the best we can do.
To use it:
import from Misc_Helpers all;
in your module and then call
Misc_Helpers.f_shutdown(__BFILE__, __LINE__);
You can also pass the function a verdict and a message and it will take care
of calling setverdict, but beware of the following:
While setverdict would accept any number of arguments as log message
and convert them to a log string f_shutdown expects one charstring.
It's possible to use the log2str function to use the log arguments in
setverdict for f_shutdown, for example
setverdict(fail, "Template didn't match: ", tmpl_foo);
would become
Misc_Helpers.f_shutdown(__BFILE__, __LINE__, fail, log2str("Template didn't match: ", tmpl_foo));
Change-Id: I84d1aa6732f6b748d2bfdeac8f6309023717f267
Pass the CTRL_DETECT_CONNECTION_ESTABLISHMENT_RESULT parameter to
the TELNET port by default. This allows tests to make progress
into an error handling path if they are started while the osmo-*
program they want to connect on VTY is not running.
Observed with osmo-ggsn tests, where if the one test runs
into a VTY connection failure the subsequent test would get
stuck forever in a map() call on the VTY TELNET port.
Teach the function f_vty_wait_for_prompt() about connection
reports by the TELNET module. We may now receive an integer which
represents the socket file descriptor for the telnet connection.
This case was not handled by the previous change made in
commit cb111b21ab. As a result,
BSC tests started failing with "VTY Timeout for prompt" because
the alt-statement in f_vty_wait_for_prompt() would not progress
past the integer sitting on the VTY port's receive queue.
Change-Id: I56925f93af6c55e93f3f417099db135744da6a40
Related: OS#3149
With this patch, I see all ttcn3-bsc-tests failing with
"Verdict: fail reason: VTY Timeout for prompt"
This reverts commit cb111b21ab.
Change-Id: I215d7ab5eee75cf6d3afaac760af64356c943140
Pass the CTRL_DETECT_CONNECTION_ESTABLISHMENT_RESULT parameter to
the TELNET port by default. This allows tests to make progress
into an error handling path if they are started while the osmo-*
program they want to connect on VTY is not running.
Observed with osmo-ggsn tests, where if the one test runs
into a VTY connection failure the subsequent test would get
stuck forever in a map() call on the VTY TELNET port.
Change-Id: I9acf7793d5d68aec6d087cff254a10d8b673dab1
Related: OS#3149
Call mtc.stop after setverdict(fail), add reasons to most failures and
fail with verdict error for internal errors.
Change-Id: I9b618235939fa41160b9be6677b121963d3ec857
PARALLEL | VERDICTOP will log when the port is dying or when other
components will change to fail. This helped to find a timeout in the
SGSN tests where a function call message timed out.
Change-Id: I770ac964dc37e2752e7d35e493f707b091c739b0
'make clean' as generated by ttcn3_makefilegen removes all *.log files, which
of course cleans out expected-results.log, which should not happen. Since this
is a junit XML file, rename the suffix to .xml.
Change-Id: Ic334f6b758eef865e3a497aa430691a3ae696d25
Compare current test results to the expected results, and exit in error on
discrepancies.
Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.
Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".
In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.
In start-testsuite.sh, after running the tests, invoke the results comparison.
Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.
Related: OS#3136
Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
Oliver Smith
Pau Espin
Alexander Couzens
Harald Welte
Daniel Willmann
Neels Hofmeyr