Commit Graph

335 Commits

Author SHA1 Message Date
Harald Welte 482c99223e ansible: Add docbook5-xml to jenkins build slaves
Change-Id: Ic0a33a87a61c9cdd1ca60a023bbf2920d04fefcd
Related: OS#3072
2018-04-02 19:35:13 +00:00
Neels Hofmeyr e30bbc5906 jobs/master-builds: move all builds to debian 9 slave
Change-Id: I6f025c642d5ae57cdda9d7b34bbaeb82ca119354
2018-03-29 16:11:33 +02:00
Neels Hofmeyr 59a15e05c3 jobs/master-builds: drop useless 'node' field
The slaves to build on are determined by the slave_axis, the node parameter has
no useful effect.

Change-Id: I029de2a46f22a1b1dc977bce71137eb78b826157
2018-03-29 16:11:33 +02:00
Alexander Couzens 43a7e83921 ansible: jenkins-slave: setup ttcn3 repository
Change-Id: I3aa1bf52f6fcae129278dab7081ea5cfc97d0759
2018-03-27 15:11:44 +02:00
Pau Espin f49376516b jobs: osmo-trx: Disable temporarily ARM builds
proot crashes with current jenkins node setup, which means we cannot use
it to run ARM based axis.

proot bug is already reported upstream in:
https://github.com/proot-me/PRoot/issues/134

Related: OS#3061

Change-Id: I9bc48349c78f395b3842bc5caaf6e948fb4c299e
2018-03-17 21:25:08 +01:00
Harald Welte e0e8ac5395 ansible/roles/osmocom-jenkins-slave/tasks/main.yml: Fix alpha sorting
Change-Id: Ifdcdcd76edf791d66cbd44daa0797c61ab5d83f0
2018-03-17 14:40:31 +01:00
Harald Welte b602e355f2 Add laforge@gnumonks.org to all build failure notifications
Change-Id: I989075eb0cd930d42daa36d9d8501592141c0310
2018-03-17 14:40:31 +01:00
Alexander Couzens 196402ac65 add ansible playbooks
Introduce playbooks to do:
- setup-jenkins-slave - setup a usualy or special jenkins-slave
- setup-gsm-tester - setup the gsm-tester

Change-Id: I7007a4e6c38f73843390ec2b3b91133aff21e36a
2018-03-16 09:18:11 +00:00
Pau Espin 0765327601 osmocom-latest-packages.sh: Catch latest tags outside master branch
Some projects, like osmo-trx, may have tags created in a non-master
branch. Let's catch those too.

Tested with several osmocom repositories. The ones showing different
behaviour are osmo-trx (expected) and other repositories which have
wrong or no tags and are anyway not used since there's no OBS build for
them.

Old cmdline is left of ==, new is right:
artwork: ( == )
asn1c: ( == )
docker-playground: (debian-jessie-buildslave-0.1.0 == )
layer1-api: (superfemto_v3.8.1 == )
libasn1c: (0.9.28 == 0.9.28)
libgtpnl: (1.1.0 == 1.1.0)
libosmo-abis: (0.4.0 == 0.4.0)
libosmocore: (0.10.2 == 0.10.2)
libosmo-netif: (0.1.1 == 0.1.1)
libosmo-sccp: (0.8.1 == 0.8.1)
libsmpp34: (1.12.0 == 1.12.0)
meta-telephony: ( == )
mncc-python: ( == )
octphy-2g-headers: (OCTSDR-OPENBSC-02.09.00-B1121 == )
openbsc: (1.0.0 == 1.0.0)
osmo-bsc: (1.1.2 == 1.1.2)
osmo-bts: (0.7.0 == 0.7.0)
osmo-ci: ( == )
osmocom-bb: (osmocon_v0.0.0 == )
osmo-ggsn: (1.1.0 == 1.1.0)
osmo-gsm-manuals: ( == v1)
osmo-gsm-tester: ( == 0.1)
osmo-gsm-tester-conf: ( == )
osmo-gsm-tester-sysmocom: ( == )
osmo-hlr: (0.1.0 == 0.1.0)
osmo-iuh: (0.2.0 == 0.2.0)
osmo-mgw: (1.2.0 == 1.2.0)
osmo-msc: (1.1.2 == 1.1.2)
osmo-pcu: (0.4.0 == 0.4.0)
osmo-python-tests: ( == )
osmo-sgsn: (1.2.0 == 1.2.0)
osmo-sip-connector: (1.1.0 == 1.1.0)
osmo-trx: (0.2.0 == 0.3.0)
pysim: ( == )
python-smpplib: ( == )

Change-Id: I3b521a194f0ae9fdfa358423e66f30c1316c23e8
2018-03-07 14:11:48 +01:00
Alexander Couzens a45a34e9d0 osmo-clean-workspace: fix force checkout of a subdirectory
Instead of the subdirectory, the main repository was
forced checked out.

Change-Id: If23e89872edf48dc5b751b3c069faf585bd6a3ef
2018-02-27 15:55:13 +01:00
Alexander Couzens 8bab2702b4 jenkins: gerrit-builds: replace the label linux_amd64_debian8/9 -> osmocom-gerrit-debian8/9
Introduce more precise labels to allow more flexibility when extending the jenkins setup.
The linux_amd64_debian8 or linux_amd64_debian9 is used across all build jobs which
make it hard to add new nodes which might only support one group of
jobs.

Change-Id: I900b7b50b33cc95e127ca78d2a47f59d32a6dfee
2018-02-25 20:38:13 +01:00
Alexander Couzens 77f5ceb8d4 jenkins: master-builds: replace the label linux_amd64_debian8/9 -> osmocom-master-debian8/9
Introduce more precise labels to allow more flexibility when extending the jenkins setup.
The linux_amd64_debian8 or linux_amd64_debian9 is used across all build jobs which
make it hard to add new nodes which might only support one group of
jobs.

Change-Id: I0fa3d3f81ab01e2488fe07601740f42eb54b6d9c
2018-02-25 19:58:30 +01:00
Alexander Couzens 45bb81c8c0 jenkins: add node admin2-deb8build to the update jobs
Change-Id: I6b703b9f27865b146f20bea953fe3d6500a88166
2018-02-25 17:15:40 +01:00
Alexander Couzens 71cb57cc88 jenkins: add node admin2-deb9build to the update jobs
Change-Id: Ie3803a46ee192dc72823291004ee20f4bdbe0126
2018-02-25 14:07:10 +01:00
Harald Welte 8efadaf793 ttcn3 gerrit verification: Use 'make deps-update' target
In osmo-ttcn3-hacks Idc165425b45872d2eb958a662d03e69aaf60669d
we introduced the new 'deps-update' Makefile target to properly
update all 'deps' repositories without removing them. Let's use it.

Change-Id: Iabc54182d1d30ef26e4f72fb9db52fd25a6c9800
2018-02-15 18:37:38 +01:00
Alexander Couzens e122fef10e jobs: add Coverity Upload Job
Change-Id: I852b42a698a4568b51e993bb5e86efeed6afdb32
2018-02-10 14:48:00 +00:00
Harald Welte 64be006cd2 master-builds: Add missing dependencies / triggers
There were a lot of downstream triggers that we were missing.

I manually reviewed the debian/control files and used that to update
the trigger lists for all jobs in master-builds.yml.

Change-Id: I12057c9bb389041ef3bcabd1c335a0fa8c358092
2018-02-09 11:25:21 +01:00
Alexander Couzens 9ac76472ba jobs/README.adoc: explain the usage of API Tokens instead of passwords
Change-Id: Ie96288913aae2fe97fa11266ffa9ed098044ccfd
2018-02-06 14:39:58 +00:00
Max 9294d2124f jenkins: fix jobs template
The sequential parameter was silently skipped because it was absent from
the project template. Fix this for both master- and gerrit- jobs.

Change-Id: I0bc28695f4f270bc7b1cc4bcd5d4d43ede6172f3
2018-01-22 16:27:38 +01:00
Max 1239cabeea cosmetic: remove wrapper script
The jenkins.sh is just a tiny wrapper around coverity_Osmocom.sh - let's
remove this unnecessary indirection and move the code directly to
jenkins.sh

Change-Id: Iead3b8f39327f1d0dd80e12a9d38563c35701993
2018-01-22 11:03:41 +01:00
Max f086d63c13 cosmetic: remove unused function
Change-Id: Ie3885237cea2afbe5e8c0f89d2280284bb4d9150
2018-01-22 11:02:14 +01:00
Max e10bf97f8f Coverity: specify sysmoBTS L1 headers explicitly
Fix upload by specifying L1 headers location properly.

Fixes: OS#2845
Change-Id: Iba55922295cb2a258f90c5345f592d7d5f34ac51
2018-01-22 11:01:49 +01:00
Max ce1d21fec1 libosmocore: use dispatcher script
Get rid of job name comparison because it depends and exact build server
name and hence is highly fragile. Use dispatcher script the same way we
do in osmo-bts.

N. B: this requires I2955e866bce4f000a53369bd601a346c36c82468 in
libosmocore.

Change-Id: I76dfc11a05007ae5c6e0554fe8132695b67cccaa
2018-01-21 21:14:41 +00:00
Harald Welte 9b23074eeb jobs: Add {master,gerrit}-libusrp
Change-Id: I2b208455b7bd008fb25322280894c100e51f2227
2018-01-21 18:34:13 +01:00
Pau Espin f98070d9d5 gerrit-verifications.yml: osmo-sgsn: Remove duplicated concurrent attribute
Change-Id: I808c46c2f4488fd577bb1c23cd13c5b73519265b
2018-01-19 15:11:18 +00:00
Neels Hofmeyr 8bd700d33d jobs: master: also trigger once per day
Recently we had changes to osmo-ci, and I noticed that although some master
builds were broken by that, the builds were still showing success -- of twenty
days ago.

Run each master build at least once a day to indicate odd side effect failure
sooner.

Change-Id: I126de2bab3db22cb693b0fa665f6579de9238fdf
2018-01-19 12:52:22 +00:00
Max 5dbe9a4617 Don't run OsmoPCU tests in parallel
The vty and ctrl tests are enabled by default and are run on hard-coded
ports. This causes some builds to fail when run in parallel.

Change-Id: I23d5b75825a667e4f043d16a12b841cd8f01af5e
2018-01-17 16:01:53 +00:00
Max a28b789022 Install pip tool into containers
The pip(2|3) tool is the officially recommended tool to deal with Python
packages [1] with much greater flexibility compared to invoking setup.py
directly. As a preparation step for using it to install
osmo-python-tests let's add it to container images.

[1] https://packaging.python.org/guides/tool-recommendations/

Change-Id: If2702c71cd268ca688e9ecc264f8cd1257c27899
2018-01-17 14:56:36 +01:00
Max 595ea2df16 Fix missing OsmoPCU master build
Similar to Idac6e53b1e0addbcb5b6c99b8051e1e0ad26bc5b fix the master
branch builds.

Change-Id: Icdd53a7cb037324a568e8eab529716e030e00583
2018-01-17 14:28:30 +01:00
Pau Espin 13738fa235 jobs: osmo-trx: Set slave_axis to use only debian9
Debian 8 contains quite old qemu and proot packages which have some
issues running the chroot infrastructure set up in osmo-trx's
jenkins.sh.

Change-Id: I24665880fff5a5b918bb6ffaf1e7bb51ae860b0b
2018-01-15 18:40:59 +01:00
Your Name 3eca350f60 docker: rebuild: if building fails, try a --no-cache build
Building a docker image depending on a debian upstream has the problem that an
intermediate build result will depend on an APT package archive that is
probable to become outdated. It's necessary to do an 'apt-get update' regularly
to get the newest package archives and be able to install new packages. We
never know which 'apt-get install' steps we might be editing, so we'd have to
add an 'apt-get update' before each, or use an ADD line to find out whether the
package archive has changed, before each and every apt-get install step. We're
likely to miss those in the future, and it would be a large, complex change.

Instead, try to build the docker image with --no-cache in case a cached build
has failed. This should fetch the most recent debian upstream with a proper
archive.

Fixes the current problem that the rebuild_osmocom_jenkins_image.sh is stuck on
various build slaves, should trigger a --no-cache build on each slave.

Change-Id: I37110287dabd53d3537d94ecd74cf513396971b3
2018-01-15 14:10:21 +01:00
Neels Hofmeyr 1ad873d4d5 docker: debian: install stow IN THE RIGHT dockerfile
Recent change I8f5012419495a656912b7b71e4f76ce102c6b63a adds use of stow in
osmo-build-dep.sh, hence our jenkins build processes now require the 'stow'
dependency. Add 'stow' to our debian docker images, used for various builds
(those that invoke 'docker' in the jobs/master-builds.yml and
jobs/gerrit-verifications.yml to produce concurrent builds).

A recent commit made this change in the Dockerfile.deb8_* dockerfiles, in the
assumption that the jenkins one would depend on that, which is actually not the
case. Instead, add stow in the dockerfile that is actually used in the jenkins
jobs. (It is not harmful to still keep the added dependency in the other two
dockerfiles.)

Change-Id: If97176f4aea66c42a6820f14ceb4b91369841ca0
2018-01-15 14:00:26 +01:00
Neels Hofmeyr a17a83d656 docker: debian: install stow, now used in various builds
Recent change I8f5012419495a656912b7b71e4f76ce102c6b63a adds use of stow in
osmo-build-dep.sh, hence our jenkins build processes now require the 'stow'
dependency. Add 'stow' to our debian docker images, used for various builds
(those that invoke 'docker' in the jobs/master-builds.yml and
jobs/gerrit-verifications.yml to produce concurrent builds).

Note: the Dockerfile.deb8_i386 doesn't seem to be used, currently. Add stow
there anyway.

Change-Id: I7a44ba5ed130a3311460185507f76151c6daa7f1
2018-01-15 12:54:22 +00:00
Neels Hofmeyr 3eb4a4e4ac README: also hint at /n/home/dub8/.config/jenkins_job_builder/jenkins_jobs.ini
Change-Id: I8bb6792ab061edd3d688e387fa69cd8637617dee
2018-01-15 12:53:57 +00:00
Pau Espin 7d5bfdd722 gerrit-verifications.yml: Build osmo-trx with different optimization instruction sets enabled
Change-Id: I1d73165a397d98d8fb630f7adec45f00887c139a
2018-01-15 10:32:18 +01:00
Pau Espin 02ec80b3cd master-builds.yml: Build osmo-trx with different optimization instruction sets enabled
Change-Id: I651e182dfd62ef4b88d013d8b5960ce7e3562ecf
2018-01-15 10:31:57 +01:00
Pau Espin 7546e94852 master-builds.yml: osmo-sgsn: Remove duplicated concurrent attribute
Change-Id: I32938d4c1854ce65faa718eccaecb5e3143dbcd0
2018-01-15 09:51:12 +01:00
Alexander Huemer 69350b67e6 Use stow for dependency management
This commit introduces the usage of GNU stow[1] for dependency
management.
Stow uses symlinks to make dependencies available in a single directory
althoguh they were installed in distinct directories.
Keeping installation directories seperate has the advantage of letting the
build fail if AM_CFLAGS and LDADD do not contain all dependencies which are
actually used.
Installing multiple dependencies into a single directory causes x_CFLAGS
and x_LIBS variables to magically point where other dependencies are
found as well, therefore missing entries can be overlooked.

Stow acts as a convenience layer here, making it unnecessary to supply a
list of locations in LD_LIBRARY_PATH, PKG_CONFIG_PATH and so forth for
building when dependencies are installed in distinct directories
manually.
Stow has to be present on the jenkins build nodes for successful executing of
osmo-build-dep.sh.

[1] https://www.gnu.org/software/stow/

Change-Id: I8f5012419495a656912b7b71e4f76ce102c6b63a
2018-01-12 19:14:01 +00:00
Max 971489df4a Fix missing OsmoPCU gerrit verification
The [yes, no] tuple was automatically converted to [True, False] by the
jenkins-job-builder but the combination filter still used old literal
check. Fix this by using boolean from the very beginning.

Change-Id: Idac6e53b1e0addbcb5b6c99b8051e1e0ad26bc5b
2018-01-12 17:07:27 +00:00
Max 191a374790 Notify per email about master branch build failure
Change-Id: I0a72d36d777ba582815b2cdef8130ee88c060b54
2018-01-12 16:56:05 +00:00
Neels Hofmeyr 577f2a95e4 docker osmo jenkins: osmo-py-tests: use contrib/jenkins.sh to install
No need to duplicate the install steps we have in osmo-python-tests/contrib
now. Also runs the tests.

Change-Id: Ibb8eb04e3a5c9c5dcc9b99c83425d2604378f960
2018-01-12 16:52:25 +00:00
Max fbcc11b473 Install setuptools in docker images
Make sure that python-setuptools are installed for both python v2 and v3
to make sure osmo-python-tests can be installed.

Change-Id: I3ecd96af2e2c0c7d141977639103d71e46c0cdc5
Related: OS#2821
2018-01-12 14:03:40 +00:00
Max 66b46cd9b9 Enable osmo-iuh submission
It was disabled back in a days when it was dependent on a separate
branch. Thisis no longer the case so we can enable it back. Also remove
remaining leftovers.

Change-Id: I5fd3bdab217b61f9ee50a5f1413b60d396535395
2018-01-11 23:07:58 +01:00
Max 3527fd63ad Send email on update-* job failures
Notify intern@ if either update-osmo-ci or update-osmo-python jobs
failed. Those are essential for our CI and test infrastructure so we
should look into related failures right away.

Change-Id: I0d51e35424a26eab7d38152e13f3ac6fd3f3997c
Related: OS#2821
2018-01-08 12:36:50 +01:00
Neels Hofmeyr c528cc12d0 jobs: master: fix: libosmoabis should trigger osmo-hlr
Change-Id: I9a122cee4fa98115140e4a34f623958697affed2
2017-12-19 23:55:49 +01:00
Neels Hofmeyr 85f74b5c90 jobs: gerrit: fix osmo-python-tests: gerrit project name
Change-Id: Idb4e93d88123f0a7e403af627345300dae86aca7
2017-12-15 20:29:31 +01:00
Neels Hofmeyr 51f77030fb jobs: gerrit: fix osmo-ttcn3-hacks scm and slave
Change-Id: I252c103450ad0da59ab18c88d78f20c750b22c98
2017-12-15 19:55:38 +01:00
Neels Hofmeyr fd7b2aae89 jobs: gerrit: fix URL for osmo-python-tests
Change-Id: Icaa44207ebbdc03c3af1a32f7491d2b4c9fb982d
2017-12-15 19:55:35 +01:00
Neels Hofmeyr 27af5a2c6f jobs: gerrit-verifications: make repos_url configurable
Change-Id: I24aadc2aeeeda762df61e7c56e4159a730010c97
2017-12-15 19:55:15 +01:00
Neels Hofmeyr 7e12155042 jobs: master: osmo-gsm-manuals: use new jenkins.sh, only on OsmoBuild1
Change-Id: Ida1301a3f83caff400e80c8b46504f98cb0894ab
2017-12-15 19:31:27 +01:00