Commit Graph

155 Commits

Author SHA1 Message Date
Your Name 3eca350f60 docker: rebuild: if building fails, try a --no-cache build
Building a docker image depending on a debian upstream has the problem that an
intermediate build result will depend on an APT package archive that is
probable to become outdated. It's necessary to do an 'apt-get update' regularly
to get the newest package archives and be able to install new packages. We
never know which 'apt-get install' steps we might be editing, so we'd have to
add an 'apt-get update' before each, or use an ADD line to find out whether the
package archive has changed, before each and every apt-get install step. We're
likely to miss those in the future, and it would be a large, complex change.

Instead, try to build the docker image with --no-cache in case a cached build
has failed. This should fetch the most recent debian upstream with a proper
archive.

Fixes the current problem that the rebuild_osmocom_jenkins_image.sh is stuck on
various build slaves, should trigger a --no-cache build on each slave.

Change-Id: I37110287dabd53d3537d94ecd74cf513396971b3
2018-01-15 14:10:21 +01:00
Neels Hofmeyr 1ad873d4d5 docker: debian: install stow IN THE RIGHT dockerfile
Recent change I8f5012419495a656912b7b71e4f76ce102c6b63a adds use of stow in
osmo-build-dep.sh, hence our jenkins build processes now require the 'stow'
dependency. Add 'stow' to our debian docker images, used for various builds
(those that invoke 'docker' in the jobs/master-builds.yml and
jobs/gerrit-verifications.yml to produce concurrent builds).

A recent commit made this change in the Dockerfile.deb8_* dockerfiles, in the
assumption that the jenkins one would depend on that, which is actually not the
case. Instead, add stow in the dockerfile that is actually used in the jenkins
jobs. (It is not harmful to still keep the added dependency in the other two
dockerfiles.)

Change-Id: If97176f4aea66c42a6820f14ceb4b91369841ca0
2018-01-15 14:00:26 +01:00
Neels Hofmeyr a17a83d656 docker: debian: install stow, now used in various builds
Recent change I8f5012419495a656912b7b71e4f76ce102c6b63a adds use of stow in
osmo-build-dep.sh, hence our jenkins build processes now require the 'stow'
dependency. Add 'stow' to our debian docker images, used for various builds
(those that invoke 'docker' in the jobs/master-builds.yml and
jobs/gerrit-verifications.yml to produce concurrent builds).

Note: the Dockerfile.deb8_i386 doesn't seem to be used, currently. Add stow
there anyway.

Change-Id: I7a44ba5ed130a3311460185507f76151c6daa7f1
2018-01-15 12:54:22 +00:00
Neels Hofmeyr 3eb4a4e4ac README: also hint at /n/home/dub8/.config/jenkins_job_builder/jenkins_jobs.ini
Change-Id: I8bb6792ab061edd3d688e387fa69cd8637617dee
2018-01-15 12:53:57 +00:00
Pau Espin 7d5bfdd722 gerrit-verifications.yml: Build osmo-trx with different optimization instruction sets enabled
Change-Id: I1d73165a397d98d8fb630f7adec45f00887c139a
2018-01-15 10:32:18 +01:00
Pau Espin 02ec80b3cd master-builds.yml: Build osmo-trx with different optimization instruction sets enabled
Change-Id: I651e182dfd62ef4b88d013d8b5960ce7e3562ecf
2018-01-15 10:31:57 +01:00
Pau Espin 7546e94852 master-builds.yml: osmo-sgsn: Remove duplicated concurrent attribute
Change-Id: I32938d4c1854ce65faa718eccaecb5e3143dbcd0
2018-01-15 09:51:12 +01:00
Alexander Huemer 69350b67e6 Use stow for dependency management
This commit introduces the usage of GNU stow[1] for dependency
management.
Stow uses symlinks to make dependencies available in a single directory
althoguh they were installed in distinct directories.
Keeping installation directories seperate has the advantage of letting the
build fail if AM_CFLAGS and LDADD do not contain all dependencies which are
actually used.
Installing multiple dependencies into a single directory causes x_CFLAGS
and x_LIBS variables to magically point where other dependencies are
found as well, therefore missing entries can be overlooked.

Stow acts as a convenience layer here, making it unnecessary to supply a
list of locations in LD_LIBRARY_PATH, PKG_CONFIG_PATH and so forth for
building when dependencies are installed in distinct directories
manually.
Stow has to be present on the jenkins build nodes for successful executing of
osmo-build-dep.sh.

[1] https://www.gnu.org/software/stow/

Change-Id: I8f5012419495a656912b7b71e4f76ce102c6b63a
2018-01-12 19:14:01 +00:00
Max 971489df4a Fix missing OsmoPCU gerrit verification
The [yes, no] tuple was automatically converted to [True, False] by the
jenkins-job-builder but the combination filter still used old literal
check. Fix this by using boolean from the very beginning.

Change-Id: Idac6e53b1e0addbcb5b6c99b8051e1e0ad26bc5b
2018-01-12 17:07:27 +00:00
Max 191a374790 Notify per email about master branch build failure
Change-Id: I0a72d36d777ba582815b2cdef8130ee88c060b54
2018-01-12 16:56:05 +00:00
Neels Hofmeyr 577f2a95e4 docker osmo jenkins: osmo-py-tests: use contrib/jenkins.sh to install
No need to duplicate the install steps we have in osmo-python-tests/contrib
now. Also runs the tests.

Change-Id: Ibb8eb04e3a5c9c5dcc9b99c83425d2604378f960
2018-01-12 16:52:25 +00:00
Max fbcc11b473 Install setuptools in docker images
Make sure that python-setuptools are installed for both python v2 and v3
to make sure osmo-python-tests can be installed.

Change-Id: I3ecd96af2e2c0c7d141977639103d71e46c0cdc5
Related: OS#2821
2018-01-12 14:03:40 +00:00
Max 66b46cd9b9 Enable osmo-iuh submission
It was disabled back in a days when it was dependent on a separate
branch. Thisis no longer the case so we can enable it back. Also remove
remaining leftovers.

Change-Id: I5fd3bdab217b61f9ee50a5f1413b60d396535395
2018-01-11 23:07:58 +01:00
Max 3527fd63ad Send email on update-* job failures
Notify intern@ if either update-osmo-ci or update-osmo-python jobs
failed. Those are essential for our CI and test infrastructure so we
should look into related failures right away.

Change-Id: I0d51e35424a26eab7d38152e13f3ac6fd3f3997c
Related: OS#2821
2018-01-08 12:36:50 +01:00
Neels Hofmeyr c528cc12d0 jobs: master: fix: libosmoabis should trigger osmo-hlr
Change-Id: I9a122cee4fa98115140e4a34f623958697affed2
2017-12-19 23:55:49 +01:00
Neels Hofmeyr 85f74b5c90 jobs: gerrit: fix osmo-python-tests: gerrit project name
Change-Id: Idb4e93d88123f0a7e403af627345300dae86aca7
2017-12-15 20:29:31 +01:00
Neels Hofmeyr 51f77030fb jobs: gerrit: fix osmo-ttcn3-hacks scm and slave
Change-Id: I252c103450ad0da59ab18c88d78f20c750b22c98
2017-12-15 19:55:38 +01:00
Neels Hofmeyr fd7b2aae89 jobs: gerrit: fix URL for osmo-python-tests
Change-Id: Icaa44207ebbdc03c3af1a32f7491d2b4c9fb982d
2017-12-15 19:55:35 +01:00
Neels Hofmeyr 27af5a2c6f jobs: gerrit-verifications: make repos_url configurable
Change-Id: I24aadc2aeeeda762df61e7c56e4159a730010c97
2017-12-15 19:55:15 +01:00
Neels Hofmeyr 7e12155042 jobs: master: osmo-gsm-manuals: use new jenkins.sh, only on OsmoBuild1
Change-Id: Ida1301a3f83caff400e80c8b46504f98cb0894ab
2017-12-15 19:31:27 +01:00
Neels Hofmeyr 163eef27de jobs: cosmetic: readability: docker one-arg-per-line
Change-Id: I8b18a5bc54a879002b40b0fc51cd295e82beba4f
2017-12-15 19:31:27 +01:00
Neels Hofmeyr b62f264eaf jobs: master: adjust triggers to what was found on old jenkins jobs
Change-Id: Ib59247978ed92b72949260a79628b28754a44484
2017-12-15 19:31:27 +01:00
Neels Hofmeyr 4711c92582 jobs: add update-osmo-python-on-slaves.yml
Change-Id: Ib3982e1f18286cc45d5c7b849abfb91cad9e1b27
2017-12-15 19:31:27 +01:00
Neels Hofmeyr 7fd10510d2 jobs: fix poll-scm on update-osmo-ci-on-slaves
poll-scm was missing by accident

Change-Id: I027c5fa12c8be9434e3efec11976004c7924c472
2017-12-15 19:31:26 +01:00
Neels Hofmeyr 520c8a58d4 jobs: add build2-deb8build to update-osmo-ci-on-slaves job
Change-Id: I48246067f05227efc18b46cb1b29d1c50f4eab84
2017-12-15 19:31:26 +01:00
Max 1e857dcf1f Add osmo-python-tests to gerrit verification
Change-Id: Ia5db32b8ffe62f9578a55c9049d0c13fc6da7e83
2017-12-15 15:43:22 +00:00
Max 3e1ca6f866 Remove OpenBSC from Coverity submission
The OpenBSC codebase only receive occasional backports hence none of the
issues uncovered by Coverity would be fixed unless it's also fixed in
corresponding Osmo* project. Let's not clutter output with the
information about issues which won't be fixed anyway.

Change-Id: Ief3dfc641684fa33407957bf7cfcb6ecf35b847a
2017-12-14 15:21:16 +01:00
Harald Welte 795c27976e add libgtpnl also to 'latest', not just to 'nightly'
Change-Id: Ifd9b3c6e98d890f135356648ef3f9a62c07dcf94
2017-12-14 14:53:21 +01:00
Alexander Couzens 61038264ed rewrite osmocom-nightly-package based on the old osmocom-nightly-split.sh
The split build script also initilize the repository if it's empty and doesn't
need any state of the osc repository.

It also downloads bumpversion and limesuite

Change-Id: I3b55e14b5b4915a3aae23ee382d65bce4ef82774
2017-12-13 10:19:30 +00:00
Neels Hofmeyr 53bb902b3f jobs: master-builds: fix git master branch polling
The 'refs/heads/origin/master' somehow caused that git polling would not
trigger builds. 'origin/master' was succesful in a manual test, so set all jobs
to that.

Change-Id: Id033d1bfce6cc9e20fbbf9be462842b9e44bde83
2017-12-13 02:32:22 +01:00
Harald Welte cb7c6ee40f Add gerrit JJB for osmo-ttcn3-hacks
we simply call 'make compile' which will compile all of the projects
from TTCN-3 to C++

Change-Id: I2bb94c08259c130558f82bb4c084484be5b81c40
2017-12-13 01:21:21 +01:00
Pau Espin 787a51f51f jobs: Fix osmo-ggsn GTP build option
As far as I understand, the variable is populated using the name field.
In openggsn build it matches, but it doesn't in osmo-ggsn.

Change-Id: Ifb1a630b77a8c2f442e26dbef8e608882e8f9a71
2017-12-09 02:45:25 +01:00
Neels Hofmeyr 50201fc9e3 jobs: master: drop legacy openggsn (now is osmo-ggsn, also present)
Change-Id: I219a988f94b465ac8bb77319735aedbe49707c16
2017-12-09 02:45:04 +01:00
Neels Hofmeyr 55ea22ede3 jobs: master: drop unused axis a4
Change-Id: I4b9db4077eacca649c679333e24b2eb7338a96e6
2017-12-09 02:18:04 +01:00
Neels Hofmeyr 3e69d3d7d4 jobs: master: fix post-build trigger of downstream jobs
Firstly, we don't have the downstream-ext plugin installed on our jenkins. We
want to use the 'trigger' publisher instead.

Secondly, since the jobs created here are called master-*, we also want to
trigger master-*.

master-libosmocore also triggers SIMtrace and xgoldmon, which aren't covered by
this jobs config, hence they don't get a master- prefix.

Change-Id: If9e8c4b02fce34fddceb4f07bf024210600f6270
2017-12-09 02:18:04 +01:00
Neels Hofmeyr 28564a6977 jobs: gerrit, master, update-osmo-ci: fix build-discarder section
The build-discarder section was silently not working. It needs to be nested
below a 'properties' node.

Also the names need to be dashed and not camelCased.

Change-Id: I9503200a8873e616f9195d4bb1d6163c464b305e
2017-12-09 01:13:18 +01:00
Alexander Couzens 6655542cec osmocom-nightly/latest: check if osc is installed
Change-Id: I10ce91eab415c19bb811fbb2fd617d5ce2b58d41
2017-12-07 22:35:33 +00:00
Alexander Couzens 8bcebb5350 osmocom-nightly/latest: document which debian packages are required
Change-Id: If127d0e852ea8e321461d6aa709621167a0a8ecb
2017-12-06 14:38:41 +00:00
Neels Hofmeyr 40c52cf1d3 jobs: master-builds: libosmo-sigtran should trigger bsc and iuh
osmo-iuh in turn triggers osmo-msc and osmo-sgsn, no explicit triggers needed
here.

Change-Id: Ifd950660cdc3417e82b0a8f4ce2b1efde880cb7a
2017-12-02 23:15:56 +01:00
Neels Hofmeyr 80d0d8cd9b jobs: master-branches: trigger sgsn from iuh
osmo-sgsn uses libosmo-ranap and hence should be triggered from osmo-iuh. This
naturally "includes" a trigger for libosmo-sigtran.

Change-Id: Ia356dc2a8d5120f9d6262bf8eb25c32fe71e76c9
2017-12-02 23:15:56 +01:00
Neels Hofmeyr 43a1c96075 jobs: master-builds: osmo-bsc is now also a downstream of osmo-mgw
Change-Id: I03c63108bd845ee723d565c3dddd4e930bb68bef
2017-12-02 23:15:56 +01:00
Neels Hofmeyr d2bd8f5095 jobs: master-builds: apply various initial fixes
libosmo-sccp, osmo-ggsn: The osmo-gsm-tester builds are downstream builds, yes,
but we configured that with the osmo-gsm-tester builds: instead of telling
libosmo-sccp to build osmo-gsm-tester_build-osmo-stp when done, we configure
the osmo-gsm-tester_osmo-stp to build after libosmo-sccp. So that the master
branch builds don't need to have any knowledge of osmo-gsm-tester.

osmo-msc build triggers should rather be post-build triggers of osmo-iuh and
osmo-mgw, like the others. Then we can also drop the pollscm here, and use the
pollscm that is common to all other builds.

Call these jobs 'master-*'. It more accurately says what they build, and also
we can install the jobs from this file next to the current, old ones, without
overwriting them and thus we'll have an easy rollback path. The new ones can
co-exist with the old ones until the new ones are verified to work, at which
point we can drop the old ones.  Line 313:

IIUC the safest git branch is 'refs/remotes/origin/master'.

This is still untested!

Change-Id: If2ad9c90a0986d1304cd53383d3df5b375f23ac8
2017-12-02 23:15:56 +01:00
Neels Hofmeyr 30e4b60a13 jobs: rename normal-builds.yml to master-builds.yml
Also apply the rename in the job names

Change-Id: If623e573e49d4f1a409139c112afbc34e4cc450e
2017-12-02 23:15:37 +01:00
Harald Welte 7535f0cd16 Describe our normal (non-gerrit) build jobs in jenkins-job-builder
after the recent successful conversion from manual job definitions
to jenkins-job-buildre of the gerrit jobs, this is an attempt to convert
also the non-gerrit jobs for the common osmocom projects.

WARNING: this file has not been tested yet, it's a WIP.

Change-Id: Ib04707393264a845876659d7bee0cdc9f8b897b6
2017-12-02 22:11:25 +01:00
Neels Hofmeyr b3fda807bd docker for jenkins: update on changes in osmo-python-tests
Download (ADD) the latest patch from git.osmocom.org so that the image gets
rebuilt when new changes were merged to master.

Change-Id: I215f5f6504018d589fa44776a332757a7b870d53
2017-11-29 18:48:02 +01:00
Neels Hofmeyr c7d6cc8638 jenkins docker: move smalltalk to separate Dockerfile
To reduce docker image rebuild time, move the Smalltalk related commands to a
separate file, which is currently not built by rebuild_osmocom_jenkins_image.sh
since there are no jenkins builds using that yet.

Change-Id: I1142f068100ef07ce7f177adaa8a0fe2fedb1b7b
2017-11-29 17:47:44 +01:00
Neels Hofmeyr 6a97f52b9a jobs: describe how to install jenkins-jobs on deb 9
Change-Id: Id3a0684e567ae26ca8158fda56898eb683374d09
2017-11-28 05:44:06 +01:00
Neels Hofmeyr f074019027 contrib/jenkins.sh: rather git checkout -f instead of pull
Change-Id: Idc672879dcb3d38978e27b90c173f0aa950c4eb1
2017-11-28 05:43:26 +01:00
Neels Hofmeyr 48f0c0a2ec jenkins docker: drop hardcoded osmo-deps.sh
The jenkins invocation of the docker image commonly includes the osmo-ci
scripts via binding ~/bin to an up-to-date checkout. We don't need another
version of osmo-deps.sh in /usr/local/bin.

Change-Id: I5ce9ab992afa3c5a7a0bb13b55cae016bc8e805f
2017-11-28 05:16:37 +01:00
Neels Hofmeyr bc07ff92a1 jenkins docker: also install py3 scripts of osmo-python-tests
Allow using the osmo_verify_transcript_{ctrl,vty}.py scripts from the docker
image.

Change-Id: I27d02d516c1cccd5a2447ee8d76ae476d0951d56
2017-11-28 05:16:37 +01:00