Prepare to add a wireshark package. It will be in its own
osmocom:wireshark repository, and not in osmocom:nightly, :latest,
:master, so it's not added to lib/config.py.
Move lib.set_args() before parse_packages(), so the latter is able to
use lib.args.allow_unknown_package.
Related: OS#2537
Change-Id: I007f937ccb629e0593ec9253541ed05df42fac66
Prepare to use this for wireshark, to build a branch on-the-fly that
does not get pushed to origin, with Osmocom patches on top of upstream
master.
Related: OS#2537
Change-Id: Ifc963daf51ba3542f67420daaf7b29745404a92e
Prepare to add more arguments to the OBS scripts, so they can be used
for building the wireshark package in its own OBS project.
It was not so clear to me when writing this initially, but with more and
more arguments getting added: it's not very maintainable to pass each
variable from the argparse result several functions down to where it
gets used... so unfortunately this means the code needs to be refactored
to just use libs.args in the few lines of code where the arguments
actually get used.
This patch adds lib.args, and the next patches refactor existing code to
use it instead of passing parameters, where it makes sense. After that
follow the patches related to adding the wireshark package.
Related: OS#2537
Change-Id: I48853444f1386dfb842c174ebc45f9decc8bec0f
Allow running build_srcpkg.py with "osmo_dia2gsup" instead of
"erlang/osmo_dia2gsup", because that's how it gets passed along from
jobs/gerrit-verifications.yml for the binary package build
verifications.
Related: OS#2385
Change-Id: I09304b219e7002495bd3fcce61bc68c34d5ffcd8
Add a script to build deb/rpm packages, as it would be done on
obs.osmocom.org. This will be used by jenkins to verify deb and rpm
builds for each submitted gerrit patch.
I have attempted to use 'osc build' instead of directly calling
apt-get build-dep and dpkg-buildpackage (and rpm equivalents). Using
'osc build' would have the advantage that the build works as close to
the OBS build as possible. However it would try to install dependencies
with sudo, so we would need to have sudo available in the docker
container that builds the untrusted code from gerrit. Let's not do that.
Related: OS#2385
Change-Id: I4c6b5d61af35df98cbc70d9ddc8ad36d38a9ce18
Add a new master feed, where packages are updated as soon as patches get
merged to master. Upload a commit_$COMMIT.txt file for each package in
this feed and use it to determine if the package needs to be updated or
not.
In most packages the commit is already part of the version in the dsc
file, e.g. "libosmocore_1.7.0.38.c3b90.dsc", but that's harder to parse
and is more likely to have a hash collision (just 5 characters).
Related: OS#2385
Change-Id: I3b0b4f4876b8c1eeb61f20d903a6f2cac6e99638
Not all the required programs are actually required to build subset of packages.
Add option to ignore the default check.
Change-Id: I8b5eaedab87b586fb1ca4c22746be4dfbb3c2176
Don't pass stdin to the programs, as we expect them to run
non-interactively and also don't show the program's output unless -v is
used or the exit code is not 0.
Change-Id: I7e893101c2a3e7b005659ec72aa44fa932b7ccd9
When running in verbose mode, it's enough that the output is displayed
while the command is running. Don't print it again on error.
Change-Id: I4fcf2421a6f82bd94ba098b39f58e5d3f18aa8fd
Harald requested that the OBS scripts should not stop if building one
specific source package fails, instead it should keep going and report
at the end a non-success exit code.
Given that the shell script code has historically grown and became hard
to maintain, I decided to rewrite the scripts for implementing this
feature. This rewrite solves additional problems:
* No full checkout of an OBS project like network:osmocom:latest
anymore, with lots of packages that won't get updated (e.g. the uhd
package has a uhd-images_3.14.1.1.tar.xz file that is 108 MB). With
the old code, developers had to wait minutes during the checkout
before the script reaches code that is currently being developed. Now
only single packages get checked out right before they get updated.
* No need to clone git repositories over and over. With the new code,
git repos only get cloned if needed (for latest it is not needed if
the remote git tag is the same as the version in OBS). During
development, the cloned git repositories are cached.
* Output from commands like "git tag -l" is not written to the log
unless they failed. This makes the log more readable, which is
especially important when a package fails to build, we keep going and
need to spot the build error in the middle of the log later on.
* No more duplicated code for nightly and latest scripts that worked
similar but had slight differences. Also the list of packages is not
duplicated for nightly and latest anymore; nightly uses all packages
and latest uses packages that have at least one git tag.
* Building source packages is decoupled from uploading them. A separate
script build_srcpkg.py can be used to just build the deb + rpm spec
source packages, without interacting with the OBS server.
* The scripts can optionally run in docker with a command-line switch,
and this is used by jenkins. This way we don't need to install
more dependencies on the host such as rebar3 which is now needed for
erlang/osmo_dia2gsup.
* Add erlang/osmo_dia2gsup and run its generate_build_dep.sh (SYS#6006)
I have done the new implementation in python to make use of argparse
and to be able to use try/except and print a trace when building one
package fails.
Example output:
* https://jenkins.osmocom.org/jenkins/job/Osmocom_OBS_nightly_obs.osmocom.org/48/console
* https://jenkins.osmocom.org/jenkins/job/Osmocom_OBS_latest_obs.osmocom.org/46/console
Change-Id: I45a555d05a9da808c0fe0145aae665f583cb80d9
Oliver Smith
Max
Neels Hofmeyr