Installing osmo-python-tests to the jenkins nodes should not be needed anymore, as builds of almost all jenkins jobs are done in docker where we install osmo-python-tests inside the containers where it is needed. We already removed the initial deployment to jenkins nodes via ansible in I7c87bd516d1bcdbcc21f65909dabbd3ccc38eede ("ansible: osmocom_build_deps: remove"), let's adjust the job that updates osmo-python-tests too. Instead of deploying it directly, only trigger a rebuild of containers. I'm doing this now because the jenkins job started failing after I42ba71e2fd0ceb8466758ea46e9ae33e3d4fcb91 ("contrib/jenkins.sh: use virtualenv and pip") on some nodes where virtualenv is not available. The contrib/jenkins.sh script was used for both CI verification and to actually deploy this python package to the jenkins nodes, which was not really obvious and also as noted above isn't needed anymore. Change-Id: I3b06a2fc2c7182d0b0a4d13b82f67a9db1be2fd0 |
||
---|---|---|
.. | ||
README.md | ||
coverity-status.yml | ||
coverity.yml | ||
gerrit-binpkgs.yml | ||
gerrit-lint.yml | ||
gerrit-verifications-comment.yml | ||
gerrit-verifications-dahdi.yml | ||
gerrit-verifications-parameters.yaml.inc | ||
gerrit-verifications.yml | ||
master-builds-dahdi.yml | ||
master-builds.yml | ||
octsim_osmo-ccid-firmware.yml | ||
osmo-gsm-manuals-trigger.yml | ||
osmo-gsm-tester-builder.yml | ||
osmo-gsm-tester-runner.yml | ||
osmo-gsm-tester-trigger.yml | ||
osmo-gsm-tester_run-gerrit.sh | ||
osmo-gsm-tester_run-prod.sh | ||
osmo-gsm-tester_run-rnd.sh | ||
osmo-gsm-tester_ttcn3.sh | ||
osmocom-api.yml | ||
osmocom-build-tags-against-master.yml | ||
osmocom-depcheck.yml | ||
osmocom-list-commits.yml | ||
osmocom-obs-check-builders.yml | ||
osmocom-obs-check-new-distros.yml | ||
osmocom-obs-ewild.yml | ||
osmocom-obs-nhofmeyr.yml | ||
osmocom-obs-nightly-asan.yml | ||
osmocom-obs-rhizomatica.yml | ||
osmocom-obs-sync.yml | ||
osmocom-obs-wireshark.yml | ||
osmocom-obs.yml | ||
osmocom-release-manuals.yml | ||
osmocom-release-tarballs.yml | ||
registry-rebuild-upload-fpga-build.yml | ||
registry-rebuild-upload-titan.yml | ||
registry-triggers.yml | ||
registry-update-base-images.yml | ||
repo-install-test.yml | ||
ttcn3-testsuites-kernel-git.yml | ||
ttcn3-testsuites.yml | ||
update-containers-osmo-python-tests.yml | ||
update-osmo-ci-on-slaves.yml |
README.md
Osmocom jenkins jobs
These jenkins.osmocom.org job definitions, managed by Jenkins Job Builder
Prepare
Install jenkins-job-builder:
# apt-get install jenkins-job-builder
Create the following file:
~/.config/jenkins_jobs/jenkins_jobs_osmo-ci.ini
Make sure the file not world readable to minimally safeguard your jenkins password. Instead of using your jenkins password, use an API Token. To retrieve your token go to Jenkins via a Webbrowser, click on your Username in the right corner, click on configure, click on Show API Toke....
jenkins_jobs_osmo-ci.ini
:
[jenkins]
user=my_user_name
password=my_api_token
url=https://jenkins.osmocom.org/jenkins
and
$ chmod go-rwx jenkins_jobs_osmo-ci.ini
Update a single job on jenkins.osmocom.org
$ cd ..
$ ./jenkins-jobs-osmo.sh update jobs/gerrit-verifications.yml gerrit-osmo-msc
NOTE: when you supply a name not defined in that yml file, you will not get an error message, just nothing will happen.
Update all jobs of one file
$ cd ..
$ ./jenkins-jobs-osmo.sh update jobs/gerrit-verifications.yml
Update all jobs in all files
$ cd ..
$ ./jenkins-jobs-osmo.sh update jobs/
Troubleshooting
jenkins.JenkinsException: create[gerrit-osmo-msc] failed
jenkins.osmocom.org is not reachable, or URL in the config file is erratic. Make sure it is exactly
url=https://jenkins.osmocom.org/jenkins
Newlines
Use 'key: |' to keep new lines in multiline values, e.g.:
- shell: |
echo hello
echo world
See also:
Jobs named on cmdline are not updated
Make sure the job name is correct, or just issue an entire yml file without individual job names.
Also be aware that jobs are only actually updated when anything changed.
Jenkins labels
Most jenkins jobs should run a docker container and install all required dependencies inside that, so we don't need to install them on the jenkins node. These jobs don't need to set a label, they can just run on any generic jenkins node that has docker available. So if you add a new job, you probably don't need a label at all.
Existing jobs typically have a label set by the topic they belong to, e.g.:
- osmocom-master
- osmocom-gerrit
- ttcn3
Other labels indicate specific software/hardware works here, e.g.:
- coverity
- hdlc
- osmo-gsm-tester
- podman
ccache
The jobs from master-builds and gerrit-verifications use ccache. View the statistics with SSH on the build nodes with:
$ CCACHE_DIR=~/ccache/gerrit-verifications ccache -s
$ CCACHE_DIR=~/ccache/master-builds ccache -s
Note that running multiple jobs in parallel influence the ccache statistics, and it's impossible to tell which job caused which change in the stats (that's why they are not printed at the end of each job, it would be confusing).
Timers
A lot of the jenkins jobs run daily with a timer:
triggers:
- timed: "H 20 * * *"
or weekly:
triggers:
- timed: "H 20 * * H"
Use H for the minute / day of week, to have it derivated as hash of the job name. Replace 20 with the hour (UTC) the job should run.
The jobs follow this timetable, to ensure we don't attempt to use binary packages before they have been built (leading to failing jobs).
18:00 - 21:00 OBS related
18:XX osmocom-obs-sync (sync Debian:12 etc. with openSUSE OBS)
19:XX osmocom-obs-check-new-distros
19:XX osmocom-obs-wireshark
20:XX osmocom-obs (new binary packages start building on OBS!)
22:00 - 03:00 Jobs that don't need binary packages
22:XX coverity
22:XX octsim_osmo-ccid-firmware
22:XX osmo-gsm-tester-runner (virtual)
23:XX master-builds-dahdi
00:XX osmocom-api
00:XX registry-update-base-images
01:XX osmocom-build-tags-against-master
01:XX osmocom-list-commits
02:XX registry-triggers
02:XX registry-rebuild-upload-fpga-build (weekly)
03:00 - 18:00 Jobs that need binary packages
ttcn3-testsuites
04:XX osmocom-release-manuals
05:XX osmocom-release-tarballs
06:XX repo-install-test
06:XX coverity-status (runs intentionally much later than the coverity job)
master-builds
: to avoid complexity, these run throughout the day (H H * * *).