Compare current test results to the expected results, and exit in error on
discrepancies.
Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.
Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".
In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.
In start-testsuite.sh, after running the tests, invoke the results comparison.
Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.
Related: OS#3136
Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
In each subdir that is a target for symlinks, automatically ignore the results
of gen_links():
- At the top of gen_links.sh.inc, clear the .gitignore.
- In the loop, add each link name to the local .gitignore.
- In selected gen_links.sh, there is also a "manual" link creationg. So that
this also ends up in the local .gitignore, have the link creation as separate
gen_link() macro which at the same time adds to ./.gitignore.
- in the root .gitignore, ignore all the subdirs' generated */.gitignore files.
Change-Id: I73c11fe8362358bf7e1bdf0e1be53399b5d3351b
Neels Hofmeyr
Pau Espin
Oliver Smith
Harald Welte