Commit Graph

650 Commits

Author SHA1 Message Date
Oliver Smith 90544fe934 Revert "osmo-build-dep.sh: don't build osmo-gsm-manuals"
Now that we have the proper fix of making a clone of
osmo-gsm-manuals.git available before builds start, and not using
'osmo-build-dep.sh osmo-gsm-manuals' in the contrib/jenkins.sh files
anymore, we can remove the temporary solution. This reverts commit
4cbc445616.

Related: OS#4912
Depends: https://gerrit.osmocom.org/q/topic:jenkins-no-manuals-dep
Change-Id: I88d57ee04775dc75e6ca3152d7edfa7f47608c8a
2021-01-18 13:59:14 +00:00
Oliver Smith 0550cf9b92 OBS: nightly: depend on dummy pkg of current date
Make it impossible to mix nightly/next builds of different dates.

Related: OS#4733
Change-Id: Ida8db8e6c91834c9d9e43694d9b4daea55a2f8f8
2021-01-15 16:51:25 +01:00
Oliver Smith 245b68316a OBS: add %H%M to DT
Make sure that each time osmocom-nightly-packages.sh and
osmocom-next-packages.sh runs, we get a different date appended to the
versions, even if it runs twice on the same day (e.g. because the
jenkins job was triggered manually).

This is in preparation to let all packages depend on a specific version
(with that date) of the conflicting dummy package.

DT in osmocom-latest-packages.sh is adjusted for consistency (though it
is not appended to the package versions, only used in the commit message
when pushing the latest packages).

Related: OS#4733
Change-Id: I7f08c694a549f1b3dd938a68e05082f2c31fdb92
2021-01-15 16:51:24 +01:00
Oliver Smith e738afafa3 OBS: osmo_obs_add_rpm_spec: depend on conflict pkg
Related: OS#4733
Change-Id: I69bebdadaf89c2a6e5dd9a27161703a567882359
2021-01-15 16:51:22 +01:00
Oliver Smith d92c2d362d OBS: osmo_obs_add_rpm_spec: refactor
Rename $spec to $spec_in and add $spec to hold the destination path, to
avoid constructing it multiple times below.

Related: OS#4733
Change-Id: I4f3d4f8a8bc83ff22983e49f6a496dc8318b53cd
2021-01-15 16:51:18 +01:00
Oliver Smith d0ab94579b OBS: add_depend_deb: no self-dependencies
Check if we are trying to make a package depend on itself, and skip in
that case. This happens for the osmocom-nightly etc. metapackages, as
they go through the same code path as regular packages. While at it, use
proper variable names in the function.

Add the new variable as second argument and not as third, because a
fourth argument will be added with the dependency version, and because
this order will be consistent with osmo_obs_add_rpm_spec() when it gets
extended in a future commit.

Fix the following warning:
W: osmocom-nightly source: package-depends-on-itself osmocom-nightly depends

Related: OS#4733
Change-Id: I439079c00259d73a18cb8617a3e76d05df5a7a35
2021-01-15 16:51:13 +01:00
Oliver Smith b7f05a92e5 OBS: rename osmo_obs_add_debian_dependency
Change name to osmo_obs_add_depend_deb. I'll add a _rpm function in a
future patch, and so we get consistent names.

Related: OS#4733
Change-Id: Icf444b86df993184c9fe4db8d3e67ab4bb06bd47
2021-01-15 16:51:12 +01:00
Oliver Smith 12416b8f5c OBS: conflict: generate conflicting rpm too
Related: OS#4733
Change-Id: Ifac815574758668172b0a7bb089585ce5a17b9e3
2021-01-15 16:51:11 +01:00
Oliver Smith 38fccbdd18 OBS: conflict: move common code from _deb to main
Move logic to create the package directory, change into it, and to put
the directory into a git repository and tag it with the package version,
into the common function. Again, in preparation to add a _rpm function.

Related: OS#4733
Change-Id: I3066147ef5469cce9d269b119d9ffa3e53f00403
2021-01-15 16:51:09 +01:00
Oliver Smith 912a2d36df OBS: conflict: move pkgver to global variable
Prepare to move the 'put in git repository' code from _deb to the main
function by having the pkgver available outside of the _deb function.

Change the version to 1.0.0 while at it, as it looks better than 0.0.0.

Related: OS#4733
Change-Id: Ic56ff12b5f2fe596d73b341e1e7750a9e202ed6b
2021-01-15 16:51:07 +01:00
Oliver Smith 02fb9ca1e7 OBS: conflict: move _deb to separate function
Prepare to add the _rpm function by moving the debian code to its own
function and tweaking the comment above the function.

Related: OS#4733
Change-Id: Ic8d55c432c6035e7ac855cf6869d2c86ace468df
2021-01-15 16:51:02 +01:00
Oliver Smith 0771116a49 OBS: move obs_prepare_conflict to own file
Prepare to add another function that creates similar dummy packages for
rpm (current version only works for deb).

Related: OS#4733
Change-Id: I29a5f54fe3a09610cffa0eacb5f0ad99154612f7
2021-01-15 16:51:00 +01:00
Oliver Smith c32a0f60e0 contrib/jenkins: add missing cd ~/osmo-ci
Fixes: 829f8b94 ("ansible: tasks/osmo-ci: clone osmo-gsm-manuals")
Change-Id: I3ca101f73cecfc5cda447ffce53a104ecc39e1fd
2021-01-13 16:25:20 +01:00
Oliver Smith 9dadec3209 jobs: remove duplicate docker_run
docker_run was duplicated for osmo-e1-hardware and no2bootloader, so the
'-e PATH' line could be removed from the global docker_run. It was
overriding the PATH, which then did not contain the path to the compiler
toolchain anymore.

This workaround is not needed anymore since de5aa17d6 ('jobs: don't
overwrite PATH in "docker run"').

Related: OS#4911
Change-Id: I97a4add17d9fa1bd28ab0ceb054b9a8d948b6f86
2021-01-13 12:40:18 +01:00
Oliver Smith 212e43fa1c jobs: update OSMO_GSM_MANUALS_DIR
Use new location inside / outside docker, where the repository was
cloned already. So it does not need to be cloned during the job.

Roll out: make sure that the docker images are already built! (or let me
merge it)

Related: OS#4912
Depends: docker-playground Id402e0ea372b5ca8a6c8f6f5beb190544c888207
Change-Id: Ibe91a613162687589e006cba99193606fb8a3eb9
2021-01-13 12:40:18 +01:00
Oliver Smith 5150f0ec62 jobs/osmo-gsm-manuals-trigger: new job
On change, roll out osmo-gsm-manuals to relevant build environments.

Related: OS#4912
Change-Id: I104525826068b016b5e06ea52567da4d7beaf7a4
2021-01-13 12:38:12 +01:00
Oliver Smith 829f8b948d ansible: tasks/osmo-ci: clone osmo-gsm-manuals
Perform the initial clone of osmo-gsm-manuals.git, just like osmo-ci.git.

Related: OS#4912
Change-Id: I39a09a4297c7c78357b3d4db4ab89547fa08db59
2021-01-13 11:33:16 +01:00
Oliver Smith d02c86663e contrib/jenkins.sh: clone osmo-gsm-manuals too
Make osmo-gsm-manuals.git available as ~/osmo-gsm-manuals, so it can be
used by osmo-trx and osmo-bts (the only two projects building manuals,
which are not running in docker as of writing) instead of cloning the
repository in their contrib/jenkins.sh.

A similar change to clone osmo-gsm-manuals in the debian-stretch-jenkins
image used by all other jobs that build manuals, is done in
docker-playground 7e4c8c6f1f798b9b8e57af97131ce3759528e0de.

osmo-ci's contrib/jenkins.sh is called by the update-osmo-ci-on-slaves
job.

Related: OS#4912
Change-Id: I742fd929e39ca32d6034a30af75b6c8e5b47b233
2021-01-13 11:33:16 +01:00
Alexander Couzens 4c22d1ba36 scripts: repo-install-test: fix missing version of centos8 example
Change-Id: I5eee27a9ba1cca43e056ef0ac4ae4bbb4cd827d7
2021-01-12 18:02:19 +00:00
Oliver Smith 24a5609f30 jobs: add missing OSMO_GSM_MANUALS_DIR
Add the variable to projects that don't use docker at all or overwrite
docker_run instead of using the global variable that has
OSMO_GSM_MANUALS_DIR defined.

Related: OS#4912
Change-Id: I65102655b4b3e907a0b0de8026f8f9af76281f90
2021-01-12 13:54:55 +01:00
Oliver Smith de5aa17d6d jobs: don't overwrite PATH in "docker run"
Don't mount $HOME/bin to /build_bin, and don't overwrite PATH to include
/build_bin anymore. This isn't needed because the scripts are installed
to /usr/local/bin inside the docker images now.

Depends: docker-playground I0820f13cd740f7d4a8999d279cef6ae293a67d81
Related: OS#4911
Change-Id: I45bd7ea56849dab839a98cfd52806f7cd288094e
2021-01-12 09:15:45 +01:00
Oliver Smith 4cbc445616 osmo-build-dep.sh: don't build osmo-gsm-manuals
Save time by only cloning the repository, and not running autoreconf
-fi, ./configure, make and make install. Especially the tests during
make took up significant time, that slowed every project depending on
osmo-gsm-manuals down while being built through master-builds or
gerrit-verifications jobs. Set OSMO_GSM_MANUALS_DIR to the clone
location.

This is an interim measure, I'll submit more patches soon that remove
the 'osmo-build-dep.sh osmo-gsm-manuals' calls from all projects.

Change-Id: I5238cf3f93ded97ed2b44d27868123a646e122dc
Related: OS#4912
2021-01-12 09:14:38 +01:00
Harald Welte 7115f90728 nightly-packages: Add osmo-cbc
Change-Id: I73b04f46e7634d8e9bfbc58fcbf3f421db810145
2021-01-06 18:54:03 +01:00
Daniel Willmann e7b1594497 Add timeout to gerrit verifications
Gerrit verification jobs sometimes hang (usually during the VTY tests).
At some point we should just time out, default is 30 minutes

Change-Id: I6fc6ee976f2d88efdab1582a418d1adcd384c850
2021-01-04 18:40:18 +01:00
Harald Welte 7fc59fcc08 repo-install-test: Move from CentOS_8_Stream to CentOS_8
Change-Id: I233c3c944ac58823dc6ff489b5ecf1fdf56b3951
2020-12-22 07:34:24 +01:00
Pau Espin Pedrol d386689e57 jobs: ogt: Update default suite for osmo-gsm-tester_virtual job
Since osmo-nitb is deprecated, the test was moved to use the "new" split
stack.

Change-Id: I6cc47f7a748339de406c1216c718a0d0042955d4
2020-12-21 18:38:55 +01:00
Harald Welte 09891fbb95 Add master-no2firmware build job
This is the DFU bootloader we use in icE1usb and e1-tracer.

Change-Id: Iaedb4e8f487c11b0247c27cba403fe2dfcd6b708
2020-12-19 11:20:16 +01:00
Harald Welte cbce526b0f Add master + gerrit job for osmo-e1-hardware
Change-Id: I02295e157f909bca985c1919a7828da0554ee1ef
2020-12-17 18:48:28 +01:00
Harald Welte 660d7ba7c7 add registry-rebuild-upload-fpga-build.yml
This will rune once per week and make sure that registry.osmocom.org
has a fpga-build container that the buld slaves can use.

Change-Id: Ie700d7c6e3c5e794edd81c3acc4b689e05d6864d
2020-12-17 15:54:41 +01:00
Harald Welte b4e042736c update-osmo-ci-on-slaves: Add build slaves on gtplab0
Change-Id: I555cb26be138e2969d07ad561fe8eba8994f2429
2020-12-17 13:50:04 +01:00
Harald Welte bfccd70721 jobs: Add ttcn3-gbproxy-test-fr for Frame Relay testing
this job can only run on a specific type of buildslave: A system
with Linux kernel HDLC support.

Change-Id: Id440c70a8f21583bce109366f44cdc22e0ca1ae8
2020-12-17 12:32:10 +01:00
Pau Espin Pedrol 2adc80b86f repo-install-test: Fix name blacklist_centos.txt to centos8
Change-Id: I9d50def17b3645b59b71fc1f1bcf8ff2a4ac790b
2020-12-07 14:50:16 +01:00
Pau Espin Pedrol e935398ae9 jobs: repo-install-test: Use centos8 as distro name
We use centos8 instead of centos in all docker-playground.git setups and
the infrastructure there expects that kind of naming.

Related: OS#4888
Change-Id: Idfbb2c4fc1ca10741406c8ab8930dabe8ce632ee
2020-12-07 12:54:17 +01:00
Harald Welte 11f89dfc6a ansible: add new deb9build slave on gtplab0
Change-Id: Ide72397cff15c2caa785a117ff095d6f08cb411b
2020-11-28 22:52:11 +01:00
Harald Welte bf07865cb3 ansible: Fix OBS directory naming
It's really annoying that OBS uses Debian_8.0 and Debian_9.0 (for 8.x
and 9.x) and then Debian_10 for 10.x.  Let's work around this by some
conditional vars_file trickery, hopefully once and for all.

Change-Id: I45c4752774f44332b3f9aaf875f49043c03ec5ca
2020-11-28 22:33:02 +01:00
Harald Welte 8df41af010 osmocom-jenkins-slave: Re-order
Install TTCN3 before installing build-deps, as build-deps
requires libulfius, which is only provided by network:osmocom:latest
feed, which in turn is only activated as part of the TTCN3 / TITAN
installation.

Change-Id: I66f1d5f7479a6d028abdac0cc6f4000faa78b135
2020-11-28 22:02:52 +01:00
Harald Welte cf9625379a ansible: add role for 'podman'
podman has one particularly interesting feature which docker
rejected years ago: The ability to run a container in a pre-existing
namespace.  This is quite useful for some of our test setups,
particularly those that involve hdlc net-devices

Change-Id: I4b14599f39c07ec83fa4222f9d040d443caae3e6
2020-11-18 22:57:07 +01:00
Pau Espin 8edbe0f134 ansible: ogt: Install new dependency python3-watchdog
As of osmo-gsm-tester.git Change-Id
I938602ee890712fda82fd3f812d8edb1bcd05e08, python3-watchdog is required.

Change-Id: I35026e82b255cd4e06b4211c4829cd12be59976e
2020-11-17 09:52:10 +00:00
Harald Welte 1f5faf85bd remove all debian8 slaves
we cannot even create new slaves as lxc-create doesn't support debian
jessie anymore.  Let's focus on Debian 9 + Debian 10

Change-Id: I2fddd460f7d1db755cc9b732b2be80494d26e091
2020-11-15 10:01:03 +01:00
Harald Welte e6c46260da ansible/docker: Use docker.io on architectures not supported by docker-ce
The docker company official builds are only present for amd64, but not
for other architectures.  Let's use the Debian docker.io package in such
situations.

Change-Id: Id85137fc0eb7fa5e2b31aa928c78bfdaadc281ac
2020-11-15 10:01:03 +01:00
Harald Welte f8b9737728 jobs: master + gerrit: Mark them as running on debian9. not debian8
We actually do the verification inside a (debian 9) docker container anyway,
so there's no need to run those on the old debian8 slaves.

Change-Id: I2168bd2f0e6221518ece111e29dc604c33eaca6a
2020-11-14 23:24:36 +01:00
Harald Welte 285c752569 ansible: ttcn3-slave: Use libfftranscode 0.3
0.3 is the first version for which we have i386 builds

Change-Id: I8bc20e43a82ff797a4f70db476cddaab39fdc2a2
2020-11-14 22:34:18 +01:00
Harald Welte a06cc0e3d1 ansible: ttcn3-slave: Fix for Debian 10 slaves
Change-Id: I943e4a8e49e4f95abf1e44dc297d30c950580680
2020-11-14 22:34:18 +01:00
Harald Welte f1fbfc95fe ansible: roles/docker: Add ca-certificates
If that package is not installed, adding the GPG key fails with SSL
certificate verification, at least on a debian10 i386 lxc:

TASK [add docker gpg key to apt keyring] *************************************************************
fatal: [gtp0-deb10build32-ansible]: FAILED! => {"changed": false, "msg": "Failed to download key at https://download.docker.com/linux/debian/gpg: Request failed: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>"}

Change-Id: I58259a99258c9c7471e1fcf03a3a7b62fa9c6a48
2020-11-14 22:34:18 +01:00
Vadim Yanitskiy 905a1e6fc8 jobs/master-builds.yml: enable manuals for all osmo-bts variants
Since Idba84164b90e3d183a20b5eb69cbfe15745e447c, we started to
generate separate PDFs for different osmo-bts-variants, so
let's enable them all.  Otherwise only osmo-bts-{virtual,sysmo}
PDFs are generated and uploaded to the server.

Change-Id: I517be2d475f57026dab7f001d2298cf1a1aa2c7c
Related: SYS#4937
2020-11-06 21:52:38 +07:00
Daniel Willmann 5806d0a63f jjb: Add ttcn3-gbproxy-test to ttcn3-testsuites.yml
Fixes: OS#4850
Change-Id: I05d1855a6daea6a67df01a1f9fa65955cfd72661
2020-11-05 18:54:37 +01:00
Harald Welte 262f164fb2 Add jenkins job to update docker base images on registry.osmocom.org
Change-Id: Iec2d2e3a43b074cb60f75cbe852351ffa8e0009e
Related: OS#4839
2020-11-04 19:30:57 +00:00
Harald Welte 7685f9c349 ttcn3-testsuites: Set REGISTRY_HOST="registry.osmocom.org"
This will make the Dockerfiles from docker-playground.git
use registry.osmocom.org rather than the public docker.io for base
distribution images like debian:stretch or the like.

This will avoid any strange timeout/EOF problems while doing
docker build --pull in our test jobs.

Change-Id: If4275cf3c8a5b70a67d742950a35ae8ead59a52e
Closes: OS#4839
2020-11-02 16:57:06 +00:00
Harald Welte 7d83ceb25c README.md: Fixup ansible command line for docker login
We need to do a docker log in as 'osmocom-build' user, not as 'root'

Related: OS#4839
Change-Id: I8ea0338e49d332382c9d08b21ee247244f97d141
2020-11-02 17:54:05 +01:00
Harald Welte a01babb6e1 README.md: Add instructions on how to make jenkins slaves 'log in' private docker registry
Change-Id: Ibf7e8c6ef2b3efd5a0a193501de90797e2d6f080
2020-11-02 17:17:57 +01:00