Commit Graph

44 Commits

Author SHA1 Message Date
Neels Hofmeyr 4ff41d94ce jenkins distcheck: enable external tests
It was recently discovered that logging_vty_test.vty was missing from
EXTRA_DIST. To make sure we don't forget similar files in the future, add
python tests to 'make distcheck', by means of DISTCHECK_CONFIGURE_FLAGS.

Related: I1bcedf3097f02b2adc679560d1cbceb27dbc345e
Change-Id: Id569b2a932c1428cabb4d7ff17822cff8eee02af
2019-01-29 01:03:25 +01:00
Neels Hofmeyr 7ab5fc1f3b add contrib/struct_endianess.py
In libosmocore (and likely elsewhere) we have scores of packed structs with
sub-byte integer members that lack the necessary member reversal shims to be
able to work on big endian architectures.

Instead of manually editing each one of them and probably introduce errors in
the process, this script handles the change automatically, and in the future
allows us to verify correctness in gerrit verifications.

Change-Id: I8e75b17d8071c7b3a2a171ba776fb76854b28a53
2018-11-22 13:49:24 +00:00
Neels Hofmeyr ba2bd2eeb1 jenkins: print gcc version in log
Change-Id: I8dfac78868f64213413e80cc52ac585d9d4a2c93
2018-11-06 21:43:16 +00:00
Neels Hofmeyr d79f01e0a6 jenkins: add --enable-external-tests
Run the new logging_vty_test.vty VTY transcript test in our CI.

Change-Id: Ic4ec1f6c131da50b3956f16f0b49375f387196b3
2018-09-13 15:46:55 +00:00
Harald Welte 61650d5117 jenkins_arm.sh: Use same Werror related options as on other builds
Change-Id: I45ab11b461a35923853f38b5bedd5025f53cc5b5
2018-08-01 17:42:06 +02:00
Harald Welte 79d49af93a jenkins_arch.sh: Exit with error on unknown architecture
The jenkins build job used to call this script using an "arch"
it doesn't understand.  This should have resulted in an error,
but it didn't as there was a missing "exit 1" statement :(

Related: OS#3360
Change-Id: Ib27c9ebaf2630c432b1923f8e14b36e7772a6033
2018-06-28 10:57:42 +02:00
Harald Welte cae93e51d9 jenkins_arch.sh: Accept "arm-none-eabi" as alias for "arm"
The jenkins build job is calling the script using "amd64" and
"arm-none-eabi", while the script expects "amd64" and "arm".

Let's add "arm-none-eabi" as an alias for "arm".

Closes: OS#3360
Change-Id: Idedd4778a63d67cdbf4f4d538bf4a225abb7547a
2018-06-28 10:57:42 +02:00
Harald Welte 1e78313e8e jenkins_arm.sh: Don't run 'make check' on embedded builds
If we're cross-compiling for arm-none-eabi, we cannot execute test
programs.

Change-Id: I64e88a31091b67c37c308c44013c42d4574d2312
2018-06-28 10:57:42 +02:00
Neels Hofmeyr 71f781c303 contrib/fsm-to-dot: warn about identically named FSMs
FSMs with identical names confuse the script, so at least warn about them.
IMHO we should also have different names for each FSM, so not bothering to make
the script safe against identical naming.

Change-Id: I4fe7e85d2fdf23e2ba964f3b47be1954edc63cef
2018-03-26 15:04:32 +02:00
Neels Hofmeyr 338d174399 contrib/fsm-to-dot: mark illegal states in red
If an FSM transitions to a state that actually belongs to a different FSM,
print an error and mark it red, instead of exiting with exception.

Change-Id: I73d95a0c65ca1ea586ad55234610671a53d6220f
2018-03-26 15:04:18 +02:00
Neels Hofmeyr ec0f334e99 contrib/fsm-to-dot: less false positives on event parsing
Do some more mad hacking to only detect event triggers if event names actually
appear in if() or case: statements. Also combine events in consecutive 'case'
statements.

When will this madness stop?

Change-Id: Iabaaab57f0d1687607eaaa4e09c7c2439fbd7a25
2018-03-26 13:03:20 +00:00
Neels Hofmeyr 536534a62f contrib/fsm-to-dot: mark arrows that feed events
Most arrows draw state transitions, but some want to show that an event is
delivered to a state. Mark those with a "halfopen" arrow head.

Change-Id: Ib07380894a6d513896c9135f175ecbf653a23eec
2018-03-26 13:03:20 +00:00
Neels Hofmeyr 75ee4e8aa9 contrib/fsm-to-dot: add default edge label '-'
When some edges have no label, it is sometimes hard to interpret which label
belongs to which edge. Adding a '-' default label clarifies the edge labeling.

Change-Id: I3a10b615288107e8fc12ffdbbe0099cf51abe94f
2018-03-26 13:03:20 +00:00
Neels Hofmeyr fcf79926e5 contrib/fsm-to-dot: warn and draw unallowed state transitions
Hacked as it is, fsm-to-dot is capable of detecting action functions
transitioning to states that are not allowed according to the FSM definition
struct.

Draw those in red and output a warning.

Found these osmo-bsc gscon errors with this patch:

ERROR: gscon_fsm_active() triggers a transition to ST_WAIT_HO_COMPL, but this is not allowed by the FSM definition
ERROR: gscon_fsm_wait_ho_compl() triggers a transition to ST_WAIT_MDCX_BTS_HO, but this is not allowed by the FSM definition

Related: OS#3109
Change-Id: Ic6319a958b3c7247510c1930bac8b02b95f9dcf2
2018-03-26 13:03:19 +00:00
Neels Hofmeyr bb22df3db9 contrib/fsm-to-dot: don't match on event names in comments
Strip comments from function bodies before matching on event names.

In osmo-bsc's gscon FSM, there often are event names in comments. The naive
parsing of fsm-to-dot.py mistakes these as events causing state transitions,
but the comments are just explaining how states interact.

Makes me reconsider parsing the C with clang instead, but I got away with a
dirty hack once more.

Change-Id: I56d70ae14d363f7ca655dced16d93d795b3f940d
2018-03-26 13:03:19 +00:00
Neels Hofmeyr 46145e80ec contrib/fsm-to-dot: allow transition from and to the same state
In osmo-bsc's new gscon FSM, there is an osmo_fsm_inst_state_chg() from
ST_ACTIVE to ST_ACTIVE. Avoid an exception triggering on this simple fact.

Change-Id: I420c7be84e3af555cc5e8bddbff7261013348375
2018-03-26 13:03:19 +00:00
Neels Hofmeyr 167f808d76 contrib/fsm-to-dot: add -h option to print basic instructions
Change-Id: I196033e44d50ebb73cf9b44cbdc94a2b8b4f98ce
2018-03-26 13:03:19 +00:00
Neels Hofmeyr b10ec0be5f jenkins_amd64.sh: use --enable-werror configure flag, not CFLAGS
Note that the jenkins_arm.sh passes various CFLAGS and potentially overwrites
./configure internal CFLAGS like that. I'm staying out of that for now.

Change-Id: I81b50c39cd6e908c4c95651829b679425de87a28
2018-03-13 00:00:30 +00:00
Daniel Willmann f4cd228f1e contrib/jenkins_arch.sh: Fix script comparison and arm support
Change-Id: Ic53a675963cae328b296b4abaed9524451153271
2018-02-19 08:34:30 +00:00
Max e41ad4d008 jenkins: move make invocation into shared function
Change-Id: Ied8c1085b8bee2cc4fa65592e805838b8cdae287
2018-01-24 11:33:19 +00:00
Max 8e8378a84d jenkins: remove obsolete scripts
* jenkins.sh is superseded by jenkins_amd64.sh
* jenkins-arm.sh is superseded by jenkins_arm.sh

N. B: this requires I76dfc11a05007ae5c6e0554fe8132695b67cccaa in
osmo-ci.

Change-Id: Ifbd253cff88c3ac18e469c34e79f1501501e1657
2018-01-24 11:33:10 +00:00
Max 1ac277dcef jenkins: add dispatcher script
Similar to the way we test osmo-bts, add simple dispatcher script which
calls appropriate test depending on a given parameter. This will allow
to simplify the job description. While at it, also rename jenkins-arm.sh
-> jenkins_arm.sh to match the OsmoBTS.

The older scripts are preserved for compatibility and shall be removed
once we update job description in osmo-ci.

Change-Id: I2955e866bce4f000a53369bd601a346c36c82468
2018-01-18 19:17:45 +01:00
Neels Hofmeyr 901c197881 jenkins: add 'make V=1' for more verbose build logs
In the logs, it is nice to see whether e.g. sanitize or -Werror args actually
made it to the gcc command line. With V=1 we see the complete command
invocations that would be hidden otherwise.

Change-Id: Ie89b1c39489ba80fb47716f4c747f2c85960e32e
2017-12-20 15:54:46 +00:00
Neels Hofmeyr 229fa070c4 jenkins: use osmo-clean-workspace.sh before and after build
See osmo-ci change I2409b2928b4d7ebbd6c005097d4ad7337307dd93 for rationale.

Depends: I2409b2928b4d7ebbd6c005097d4ad7337307dd93
Change-Id: I99fdd20e7b18c2f4a59a9db35e5efccca6546b77
2017-10-27 22:41:57 +02:00
Neels Hofmeyr c8ef736370 jenkins: bail early if $MAKE env var is not set
If $MAKE is unset, it produces confusing errors, rather check explicitly.

Change-Id: I59b111dfdee4e1edea04155144f01d99f5f72aa1
2017-10-24 15:31:07 +00:00
Neels Hofmeyr 754e6877cc fixups for recent "Cleanup jenkins build scripts"
Change I24e500e132f5c8e8133d35548cb7b4e4552331d0 was merged, but IMHO needs
improvement. Fix these:

- clean the git source tree before each build step, in common prep_build().
- fix indenting inside the build() macros.
- change build() arg to be build_dir, to absorb cleanup steps into build().
- in jenkins.sh, use $ENABLE_SANITIZE as global env, not passed as arg.
- in jenkins.sh, don't do 'make distcheck' twice. It is not necessary to do it
  from source tree as well as separately from source tree, since distcheck
  already moves to a different build dir.

Change-Id: I09d306350602f21943d5bd45f7388c83ede9b524
2017-10-24 15:31:07 +00:00
Neels Hofmeyr a568af25d2 contrib/fsm-to-dot.py: some tweaks that help with osmo-bsc's new FSMs
Combine the C source file name and the string name into the fsm's internal name
token, and use it in most places instead of the plain struct name: osmo-bsc's
new FSMs have identical struct names in each static c context.

Output in a file name that includes all of these more detailed name tokens.

Also parse '(1 << EVENT)' as event names.

Note that besides this patch, there are also some tweaks to the osmo-bsc patch
that improve the fsm-to-dot experience...
- call fsm-to-dot for single files to avoid name conflicts, or rename each
  struct as a unique name.
- Add comments for the event name a callback is intended for, so that not all
  transitions are interpreted as TEARDOWN (because it is invoked in common
  error handling, which causes the script to interpret it as the causing
  event). (or change the event-checking if into a switch that names the valid
  event and has a default case for all others.)

Change-Id: Ib60df7fd19efc99ba9fe797f14c0e3239c4bea20
2017-10-24 15:30:58 +00:00
Max d7ee5759cb Disable doxygen in jenkins for embedded tests
There's no point in regenerating documentation with Doxygen in embedded
environment.

Change-Id: I6504db04445f5e9c1117379c8803e10df5634324
2017-10-20 18:17:42 +00:00
Max 15b05fef76 Cleanup jenkins build scripts
* reorder builds to avoid rm -rf invocation
* avoid useless double autoreconf
* move common parts into shared helper
* move common build steps into separate function

Change-Id: I24e500e132f5c8e8133d35548cb7b4e4552331d0
2017-10-20 18:17:42 +00:00
Max 6081bd0e6f Use value string check from osmo-ci
Change-Id: I513835be2d931d0a931cdfc996f361a451bc1a15
2017-08-25 17:56:19 +02:00
Pau Espin 0d119ef8b3 contrib: Enable -Werror by default
Change-Id: I1752cad1e34cb6e5f36edf3f538d803eff29d18c
2017-06-23 15:41:49 +02:00
Pau Espin 56c5b949fa jenkins.sh: Disable sanitize for FreeBSD
Our current build server uses clang-3.4.1 and FreeBSD10.3, and
AddressSanitizer support is not good yet on those versions, triggering
some link issues when it is enabled.

I tested using newer clang 3.7.1 in the same FreeBSD version but another
compiler error related to __builtin_cpu_supports is triggered.

Conclusion: Explicitly disable AddressSanitizer until we move to
FreeBSD11 with clang 3.8. Before this commit, AddressSanitizer was
implicitly disabled due to a shell compatibility issue which is fixed
in next commit in the series, thus we need to explicitly disable it here.

Change-Id: I0db497045a2ebf7877232b28ae1fcf1fd89fb3f3
2017-06-23 15:41:49 +02:00
blobb cd8e7a46b6 Introduce build script for arm-none-eabi cross-compilations
This is a necessary step to use libosmocore-arm dep in OsmocomBB.

After this patch is submitted the "./contrib/jenkins.sh" line in
libosmocore jenkins jobs [1][2] needs to be replaced with:

  if [[ "$JOB_NAME" == *"arch=arm-none-eabi,label=linux_amd64_debian8"* ]]
  then
    ./contrib/jenkins-arm.sh
  else
    # (FreeBSD && debian8)_amd64 builds
    ./contrib/jenkins.sh
  fi

Furthermore, the "arch" matrix-build axis has to be created,
holding "arm-none-eabi" and "amd64". Following combination filter 
is necessary to skip cross-compilations on FreeBSD buildslave:

  !(arch=="arm-none-eabi" && label=="FreeBSD_amd64")

[1] https://jenkins.osmocom.org/jenkins/job/libosmocore/
[2] https://jenkins.osmocom.org/jenkins/job/libosmocore-gerrit/

Change-Id: I0db83ad2c94ada65c00fd1bcf867830dec80085d
2017-05-26 09:05:11 +00:00
blobb 2d899a0157 Revert "Introduce build script for arm-none-eabi cross-compilations"
"--enable-sanitize" does not work on jenkins build slave (debian8), although it worked out locally for me.

Following patch will introduce corrected arm build script without "--enable-sanitize" option:

https://gerrit.osmocom.org/#/c/2742/  

This reverts commit ba58167e1b.

Change-Id: I644114679db1394e75ff12b0203ae8e568aff82c
2017-05-26 08:38:27 +00:00
blobb ba58167e1b Introduce build script for arm-none-eabi cross-compilations
This is a necessary step to use libosmocore-arm dep in OsmocomBB.

After this patch is submitted the "./contrib/jenkins.sh" line in
libosmocore jenkins jobs [1][2] needs to be replaced with:

  if [[ "$JOB_NAME" == *"arch=arm-none-eabi,label=linux_amd64_debian8"* ]]
  then
    ./contrib/jenkins-arm.sh
  else
    # (FreeBSD && debian8)_amd64 builds
    ./contrib/jenkins.sh
  fi

Furthermore, the "arch" matrix-build axis has to be created,
holding "arm-none-eabi". Following combination filter is
necessary to skip cross-compilations on FreeBSD buildslave:

  !(arch=="arm-none-eabi" && label=="FreeBSD_amd64")

[1] https://jenkins.osmocom.org/jenkins/job/libosmocore/
[2] https://jenkins.osmocom.org/jenkins/job/libosmocore-gerrit/

Change-Id: I9e07fb9e60830dbd585f22d9c658c46252ce399f
2017-05-25 00:06:07 +00:00
Neels Hofmeyr d4089f4b07 jenkins: also test build in separate dir
To make sure that new patches don't break a build done in another dir than
srcdir, make jenkins verify that it still works.

Even though 'make distcheck' also tests a build from a separate dir, this does
not verify that BUILT_SOURCES are generated properly. If these already exist in
the source tree from a previous 'make' issued in the source tree, the
BUILT_SOURCES are not regenerated during 'make distcheck'. Hence a separate
test run is necessary to ensure stability of new patches.

Change-Id: I4b31538c155941fd241bcd33b0d39f2f8491ac1e
2017-03-31 12:06:13 +00:00
Neels Hofmeyr 9ab00721e5 jenkins: add value_string termination check
Change-Id: I7fe3678b524d602fc6aa14bc0ed06308df809a3e
2017-03-16 19:00:04 +00:00
Neels Hofmeyr d80a217f7f contrib: add script to find unterminated value_string arrays
Unterminated value_string arrays are dangerous since get_value_string() and
get_string_value() need to know where the struct ends. If the terminator is
missing, they might run through and return arbitrary memory locations.

Employ some regexes to find such unterminated value string arrays and return
nonzero if any are found.

This can be used in our jenkins build jobs to avoid committing unterminated
value_string arrays. In fact I've found one in current libosmocore:
gsm0808_bssap_names in gsm/gsm0808.c, fixed in a separate patch.

Change-Id: I2bc93ab4781487e7685cfb63091a489cd126b1a8
2017-03-16 12:53:50 +00:00
Neels Hofmeyr 0898a007ba contrib: add fsm-to-dot.py to draw osmo_fsm dotty graphs
Add a first version of a python script that tries to analyze .c source files to
draw graphs of osmo_fsm implementations. So far it uses quick-and-dirty
regexes.

Change-Id: I155f57a608d600f59aedfd27ef66eb9772c124e7
2016-12-09 11:29:06 +00:00
Neels Hofmeyr a23817622b configure: add --enable-sanitize, use in jenkins.sh
--enable-sanitize adds address sanitizer CFLAGS/CPPFLAGS to the entire build.
Also pass UBSAN_OPTIONS to the test suite run (only has effect during runtime).

Add this flag to jenkins.sh's configure step. To ensure that we get the
sanitize results, add 'make check' to jenkins to catch sanitize failures;

Keep 'make distcheck' without ASAN; it has its own configure which omits
--enable-sanitize. This way we test both with and without ASAN.

Change-Id: Idf7f46fa048608c2951f2473cb528f6c8dc2681d
2016-10-12 22:50:24 +00:00
Neels Hofmeyr 530736f315 jenkins.sh: drop unecessary PATH, now in slave config
Change-Id: Ifaff3e5e13c920eb716a5609bfec2f96e10f17b0
2016-10-12 11:28:33 +00:00
Neels Hofmeyr 56155c16b9 jenkins.sh: output all test logs when 'make check' failed
Assume that cat-testlogs.sh from osmo-ci is installed in $HOME/osmo-ci/scripts,
and call from jenkins.sh upon 'make check' failure.

Change-Id: I18a08e7ade1a53783d5a4171fe825f61b49457be
2016-09-30 00:58:13 +00:00
Holger Hans Peter Freyther 700297528c jenkins: Enable tracing and error handling in bash
Like jenkins we want to see what is actually being executed
and fail if a subcommand has failed.
2016-04-13 18:50:41 -04:00
Holger Hans Peter Freyther d16321801d jenkins: Add build script used by jenkins on our public CI
The actual script (removed unused and commented out handling) as
invoked by jenkins. This can be used to locally try to reproduce
the build.
2016-04-13 18:47:15 -04:00