Make the retry logic in rebuild_osmocom_jenkins_image.sh obsolete by
merging all apt commands for installing dependencies into one. This
makes development easier, because the docker image will not get rebuilt
again when any error happens (whether that error was related to the apt
cache or not, during development it is mostly not related to the apt
cache).
This also makes it easier to read through the list of dependencies, and
removes duplicate (or even conflicting, pkg-config vs pkgconf) entries.
While at it, also merge the pip commands into one.
Change-Id: I190e62dd39533b284c93107607dac24ece6d7be6
Prepare to run osmo-python-tests in docker. I've noticed that its
contrib/jenkins.sh tries to write to /usr/local, which makes sense for
the "update-osmo-python-on-slaves" job to actually deploy new versions.
However, the same script gets called in gerrit-verifications too, and
there we should definitively not install a version that is still in
review to whichever random buildslave that is executing the
gerrit-verification job! The follow-up patch for building most Osmocom
projects in docker will fix this bug.
Related: OS#3726
Change-Id: Ie1de8a6d436de0b286ed3372166eceb408bab084
Prepare for building almost all Osmocom repositories in docker, by
installing new required dependencies:
* libaio-dev: needed for osmo-ccid-firmware
* libasound2-dev: needed for gapk
* libboost-all-dev: needed for libusrp
* libcsv-dev: needed for osmo-remsim
* libmnl-dev: needed for libgtpnl
* libnewlib-arm-none-eabi: needed to cross compile libosmocore to arm
* liboping-dev: needed for osmo-sysmon
* libulfius: needed for osmo-remsim (libulfius is not packaged in debian
stretch, download the project's official .deb release instead)
* libzmq3-dev: needed for osmo-pcap
* sdcc: needed for libusrp
* systemd: needed for osmo-remsim (provides systemd.pc, indirect
dependency through libulfius)
Related: OS#3726
Change-Id: I5caebd26090169cc3c9fdd1f14984ea2a75347e8
Debian jessie (8) is EOL already, so switch to stretch (9). This is
required to build all Osmocom projects in docker, because at least
libosmocore fails to build with the old image (tests won't build with
GCC-4.9.x from jessie unless -Werror is disabled).
Do not build the jessie image anymore, and update the shared dockerfile.
Remove everything jessie specific, and make it work with stretch by
installing asciidoc-dblatex. Otherwise we get this error when building
manuals:
a2x: ERROR: missing configuration file: /etc/asciidoc/dblatex/asciidoc-dblatex.xsl
Add '--cap-add SYS_PTRACE' argument to the docker run command, because
otherwise the address sanitizer isn't working anymore with stretch
(which makes the OsmoPCU tests fail).
I've tried to split this patch up into multiple ones, but since there is
no asciidoc-dblatex package in jessie, we must add the dependency and
switch the image at the same time.
Related: OS#3726
Change-Id: I61fe93863d5367e3b9c01ab4726bb95bd16d891d
Exit with error if one of the docker commands is failing (i.e. don't
exit with 0 when building the jessie image failed, but the stretch
image got built properly).
Change-Id: I6faaf9303b4facb0965131615bc73541dd309c51
Prevent building docker images on top of outdated Debian images, where
the package download feeds have been disabled.
Related: Change-Id I1076bbb7d77bdc99f5d60d641c09ce965fb9dfc6
(docker-playground.git)
Related: OS#3869
Change-Id: Id840094aec51bf51d920aaa017a2f99fcb866f55
Fix docker/Dockerfile_osmocom_jenkins.amd64 by fetching jessie-backports
from debian archive.
The jessie image is still being used by various jenkins jobs generated
by master-builds.yml and gerrit-verifications.yml. We want to keep using
it for some more time, to make sure that build testing ("make check")
passes on debian8/jessie, as some users may very well be using old
distros.
See also:
* https://lists.debian.org/debian-devel-announce/2019/03/msg00006.html
* https://unix.stackexchange.com/q/508724
Related: OS#3894
Change-Id: Iba033e64876b7ed9d329c2a636a0ee5eb3af946e
The binaries built on the Jenkins to be used by the osmo-gsm-tester
require libgnutls.so.30. This is provided by the libgnutls30 package
which is not available on jessie (Debian 8) and a backport requires
some effort.
Re-use the existing Dockerfile for jessie and stretch (Debian 9) using
an argument. Refactor the shell script to pass the right argument. Add
the debian version into the tag name to make space for buster (Debian
10).
Change-Id: I44c8e49a548cb64defd2eba868ec4bd435de90ba
We have been updating some of these files besides no container being
built or used from them. Let's clean this up by removing them.
Change-Id: I7c5b559975e5f9a50368a5bb5bd961bdb9f85270
The osmo-gsm-tester requires a UTF-8 locale to generate the log
messages. Most of our users will have a UTF-8 and it seems okay
to change the default from C to en_US.UTF-8.
Change-Id: I79082a57fd5c9122dbd66997dd48e3ed3e89acb9
Add one more dependency and its dependency for the osmo-gsm-tester
(a separate patch is coming to update the docs).
Change-Id: Icd6a93a8a91c5443b310fdd2123183a8059d31d1
Install all dependencies required for building PDF manuals inside the
Docker container, so we can build projects with their manuals in the
Gerrit build verification job.
I ran the Docker container locally to confirm that this works, as
described here:
https://osmocom.org/projects/osmocom-servers/wiki/Jenkins_build_verification_jobs
Related: OS#3385
Change-Id: I379ce02857353c097f79cf91af2c4c9dc14754ec
The pip(2|3) tool is the officially recommended tool to deal with Python
packages [1] with much greater flexibility compared to invoking setup.py
directly. As a preparation step for using it to install
osmo-python-tests let's add it to container images.
[1] https://packaging.python.org/guides/tool-recommendations/
Change-Id: If2702c71cd268ca688e9ecc264f8cd1257c27899
Building a docker image depending on a debian upstream has the problem that an
intermediate build result will depend on an APT package archive that is
probable to become outdated. It's necessary to do an 'apt-get update' regularly
to get the newest package archives and be able to install new packages. We
never know which 'apt-get install' steps we might be editing, so we'd have to
add an 'apt-get update' before each, or use an ADD line to find out whether the
package archive has changed, before each and every apt-get install step. We're
likely to miss those in the future, and it would be a large, complex change.
Instead, try to build the docker image with --no-cache in case a cached build
has failed. This should fetch the most recent debian upstream with a proper
archive.
Fixes the current problem that the rebuild_osmocom_jenkins_image.sh is stuck on
various build slaves, should trigger a --no-cache build on each slave.
Change-Id: I37110287dabd53d3537d94ecd74cf513396971b3
Recent change I8f5012419495a656912b7b71e4f76ce102c6b63a adds use of stow in
osmo-build-dep.sh, hence our jenkins build processes now require the 'stow'
dependency. Add 'stow' to our debian docker images, used for various builds
(those that invoke 'docker' in the jobs/master-builds.yml and
jobs/gerrit-verifications.yml to produce concurrent builds).
A recent commit made this change in the Dockerfile.deb8_* dockerfiles, in the
assumption that the jenkins one would depend on that, which is actually not the
case. Instead, add stow in the dockerfile that is actually used in the jenkins
jobs. (It is not harmful to still keep the added dependency in the other two
dockerfiles.)
Change-Id: If97176f4aea66c42a6820f14ceb4b91369841ca0
Recent change I8f5012419495a656912b7b71e4f76ce102c6b63a adds use of stow in
osmo-build-dep.sh, hence our jenkins build processes now require the 'stow'
dependency. Add 'stow' to our debian docker images, used for various builds
(those that invoke 'docker' in the jobs/master-builds.yml and
jobs/gerrit-verifications.yml to produce concurrent builds).
Note: the Dockerfile.deb8_i386 doesn't seem to be used, currently. Add stow
there anyway.
Change-Id: I7a44ba5ed130a3311460185507f76151c6daa7f1
No need to duplicate the install steps we have in osmo-python-tests/contrib
now. Also runs the tests.
Change-Id: Ibb8eb04e3a5c9c5dcc9b99c83425d2604378f960
Make sure that python-setuptools are installed for both python v2 and v3
to make sure osmo-python-tests can be installed.
Change-Id: I3ecd96af2e2c0c7d141977639103d71e46c0cdc5
Related: OS#2821
Download (ADD) the latest patch from git.osmocom.org so that the image gets
rebuilt when new changes were merged to master.
Change-Id: I215f5f6504018d589fa44776a332757a7b870d53
To reduce docker image rebuild time, move the Smalltalk related commands to a
separate file, which is currently not built by rebuild_osmocom_jenkins_image.sh
since there are no jenkins builds using that yet.
Change-Id: I1142f068100ef07ce7f177adaa8a0fe2fedb1b7b
The jenkins invocation of the docker image commonly includes the osmo-ci
scripts via binding ~/bin to an up-to-date checkout. We don't need another
version of osmo-deps.sh in /usr/local/bin.
Change-Id: I5ce9ab992afa3c5a7a0bb13b55cae016bc8e805f
We are not using the Docker registry due difficult GPL compliance
but build the image on the build node itself. After prototyping
if containers can be used for building these files remained forked
on the node. These days it seems to be easier to complain than to
say thank you that this new approach has its merits. Copy the files
from the build slave to the directory they should have been in.
Change-Id: Icb0406f96b0c18e77be51ad8317c2668fb23a45e
Create a clean build environment for amd64/i386 based on debian
jessie. Once an official build and sadly once someones 32bit build
# Build
sudo docker build -t osmocom:ci_debian8_32bit -f Dockerfile.deb8_i386 .
sudo docker build -t osmocom:ci_debian8_64bit -f Dockerfile.deb8_amd64 .
# Run
sudo docker run -it --rm=true osmocom:ci_debian8_32bit bash
Oliver Smith
Harald Welte
Vasil Velichkov