gen_makefile.py: drop argument for deps file

Rename 3G+2G.deps to all.deps and remove the parameter from
gen_makefile.py. Create new Makefile targets for the various use cases,
usually we build either 'cn-bsc' or 'usrp'. The next patch will add more
projects to all.deps.

This change brings the Makefile logic closer to how it is actually used,
one build directory that is building most Osmocom projects. New projects
can be added to all.deps on demand, but then we have it in the
repository and not just a local change that we add temporarily to the
3G+2G.deps file when we need to build a specific project.

Extend all.deps with projects needed by ttcn3.sh, so it doesn't need to
write its custom .deps file anymore.

Change-Id: I6394882f67eecda3d2c03a97d3119657f7d3873f

2G.deps

@@ -1,18 +0,0 @@
-# project	build these first
-libosmocore
-libosmo-abis	libosmocore
-libosmo-netif	libosmo-abis
-libosmo-sccp	libosmo-netif
-libsmpp34
-osmo-ggsn	libosmocore
-osmo-hlr	libosmo-abis
-osmo-mgw	libosmo-netif
-osmo-msc	libosmo-sccp osmo-mgw libsmpp34 osmo-hlr
-osmo-bsc	libosmo-sccp osmo-mgw
-osmo-sgsn	libosmo-sccp osmo-ggsn osmo-hlr
-osmo-sip-connector	libosmocore
-osmo-smlc	libosmo-sccp
-# osmo-trx can build with --enable-sanitize, but then won't work reliably.
-# When omitting --enable-sanitize from osmo-trx only, its 'make check' will fail.
-# So if you want osmo-trx, uncomment and use no sanitize.opts, or use LD_PRELOAD for osmo-trx binaries.
-#osmo-trx	libosmocore

README

@@ -15,7 +15,7 @@ sudo apt install \
   libdbd-sqlite3 libsqlite3-dev libpcap-dev libc-ares-dev libgnutls28-dev \
   libsctp-dev sqlite3
 
-./gen_makefile.py 3G+2G.deps default.opts iu.opts -I -m make
+./gen_makefile.py default.opts iu.opts -I -m make
 
 cd make
 make
@@ -40,29 +40,39 @@ projects, but works generically.
 The idea is to have all your Osmocom git clones in ./src, while keeping one or
 more separate build trees in ./make-*.
 
-Run ./gen_makefile.py with a choice of projects (2G only or also 3G?)
+Run ./gen_makefile.py with a choice of choice of configure options.
-and a choice of configure options.
 
 
 Examples:
 
-Full 2G and 3G support:
+CN with full 2G and 3G support:
 
-  ./gen_makefile.py 3G+2G.deps default.opts iu.opts  -m make
+  ./gen_makefile.py default.opts iu.opts -m make
   cd make
-  make
+  make cn
 
-2G only, and a custom dir name of 'make-2G':
+Other make targets exist for specific use cases:
 
-  ./gen_makefile.py 2G.deps default.opts -m make-2G
+  - 'usrp':
-  cd make-2G
+    Build the CN, OsmoBSC, OsmoBTS and OsmoTRX (default, e.g. when connecting
-  make
+    to an USRP)
+
+  - 'cn-bsc':
+    Build the CN and OsmoBSC (e.g. when connecting to an external sysmoBTS)
+
+  - 'osmo-msc':
+    Build only the OsmoMSC project and its dependencies (this can be used for
+    any project in all.deps).
+
+  - '.make.osmo-ttcn3-hacks.clone'
+    Clone the osmo-ttcn3-hacks git repository (it cannot be built by osmo-dev,
+    but cloning it is still useful.)
 
 If you make modifications in one of the source trees, this Makefile will pick
-it up, rebuild the project and also rebuild all dependencies (according to the
+it up, rebuild the project and also rebuild all dependencies (according to
-*.deps file the Makefile was generated from).
+all.deps).
 
-If you modify the *.deps or *.opts file, you can easily run 'make regen' in a
+If you modify the all.deps or *.opts file, you can easily run 'make regen' in a
 make-* subdir to regenerate the Makefile from the same files.
 
 In your make-* subdir there are empty status files that are touched for every
@@ -72,7 +82,7 @@ You can manually remove them to force a rebuild of a specific target.
 For example, if you 'rm .make.libosmocore.autoconf', libosmocore and all
 projects depending on libosmocore will be rebuilt from scratch.
 
-For more details on the *.opts and *.deps syntax, read the docs at the top of
+For more details on the *.opts and all.deps syntax, read the docs at the top of
 ./gen_makefile.py.
 
 It is also easily possible to keep sources and build trees in various

all.deps

@@ -17,5 +17,10 @@ osmo-bts	libosmo-netif
 osmo-smlc	libosmo-sccp
 # osmo-trx can build with --enable-sanitize, but then won't work reliably.
 # When omitting --enable-sanitize from osmo-trx only, its 'make check' will fail.
-# So if you want osmo-trx, uncomment and use no sanitize.opts, or use LD_PRELOAD for osmo-trx binaries.
+# So if you want osmo-trx, use no sanitize.opts, or use LD_PRELOAD for osmo-trx binaries.
-#osmo-trx	libosmocore
+osmo-trx	libosmocore
+osmo-pcu	libosmocore
+
+# can only clone these
+docker-playground
+osmo-ttcn3-hacks

gen_makefile.py

@@ -2,20 +2,19 @@
 '''
 Generate a top-level makefile that builds the Osmocom 2G + 3G network components.
 
-  ./gen_makefile.py projects.deps [configure.opts [more.opts]] [-o Makefile.output]
+  ./gen_makefile.py [configure.opts [more.opts]] [-o Makefile.output]
 
 Configured by text files:
 
-  *.deps: whitespace-separated listing of
+  all.deps: whitespace-separated listing of
     project_name depends_on_project_1 depends_on_project_2 ...
 
   *.opts: whitespace-separated listing of
     project_name --config-opt-1 --config-opt-2 ...
 
 Thus it is possible to choose between e.g.
-- 2G+3G or 2G-only by picking a different projects_and_deps.conf,
+- building each of those with or without mgcp transcoding support by adding or
-- and between building each of those with or without mgcp transcoding support
+  removing "transcoding.opts" from the command line
-  by picking a different configure_opts.conf.
 
 From the Makefile nature, the dependencies extend, no need to repeat common deps.
 
@@ -41,7 +40,7 @@ need sudo to install there, you may issue the --sudo-make-install option.
 
 EXAMPLE:
 
-  ./gen_makefile.py 3G+2G.deps default.opts iu.opts -I -m build
+  ./gen_makefile.py default.opts iu.opts -I -m build
   cd build
   make
 
@@ -51,12 +50,10 @@ import sys
 import os
 import argparse
 
+topdir = os.path.dirname(os.path.realpath(__file__))
+all_deps_file = os.path.join(topdir, "all.deps")
 parser = argparse.ArgumentParser(epilog=__doc__, formatter_class=argparse.RawTextHelpFormatter)
 
-parser.add_argument('projects_and_deps_file',
-  help='''Config file containing projects to build and
-dependencies between those''')
-
 parser.add_argument('configure_opts_files',
   help='''Config file containing project name and
 ./configure options''',
@@ -64,7 +61,7 @@ parser.add_argument('configure_opts_files',
 
 parser.add_argument('-m', '--make-dir', dest='make_dir',
   help='''Place Makefile in this dir (default: create
-a new dir named after deps and opts files).''')
+a new dir named after opts files).''')
 
 parser.add_argument('-s', '--src-dir', dest='src_dir', default='./src',
   help='Parent dir for all git clones.')
@@ -242,18 +239,20 @@ def gen_make(proj, deps, configure_opts, jobs, make_dir, src_dir, build_dir, url
     )
 
 
-projects_deps = read_projects_deps(args.projects_and_deps_file)
+projects_deps = read_projects_deps(all_deps_file)
 configure_opts = listdict()
 configure_opts_files = sorted(args.configure_opts_files or [])
 for configure_opts_file in configure_opts_files:
+  if configure_opts_file.endswith(".deps"):
+    print(f"WARNING: using {all_deps_file} instead of {configure_opts_file}")
+    continue
   r = read_configure_opts(configure_opts_file)
   configure_opts.extend_dict(read_configure_opts(configure_opts_file))
 
 make_dir = args.make_dir
 if not make_dir:
-  deps_name = args.projects_and_deps_file.replace('.deps', '')
   opts_names = '+'.join([f.replace('.opts', '') for f in configure_opts_files])
-  make_dir = 'make-%s-%s' % (deps_name, opts_names)
+  make_dir = 'make-%s' % opts_names
 
 if not os.path.isdir(make_dir):
   os.makedirs(make_dir)
@@ -268,9 +267,36 @@ print('Writing to %r' % output)
 with open(output, 'w') as out:
   out.write('# This Makefile was generated by %s\n' % os.path.basename(sys.argv[0]))
 
+  configure_opts_args = ""
+  for f in configure_opts_files:
+    if not f.endswith(".deps"):
+      configure_opts_args += f' \\\n\t\t{os.path.relpath(f, make_dir)}'
+
   # convenience: add a regen target that updates the generated makefile itself
   out.write(r'''
-default: all
+default: usrp
+
+cn: \
+	osmo-ggsn \
+	osmo-hlr \
+	osmo-iuh \
+	osmo-mgw \
+	osmo-msc \
+	osmo-sgsn \
+	osmo-sip-connector \
+	osmo-smlc \
+	$(NULL)
+
+cn-bsc: \
+	cn \
+	osmo-bsc \
+	$(NULL)
+
+usrp: \
+	cn-bsc \
+	osmo-bts \
+	osmo-trx \
+	$(NULL)
 
 .PHONY: all_debug
 all_debug:
@@ -281,7 +307,6 @@ all_debug:
 .PHONY: regen
 regen:
 	{script} \
-		{projects_and_deps} \
 		{configure_opts} \
 		-m {make_dir} \
 		-o {makefile} \
@@ -291,8 +316,7 @@ regen:
 
 '''.format(
     script=os.path.relpath(sys.argv[0], make_dir),
-    projects_and_deps=os.path.relpath(args.projects_and_deps_file, make_dir),
+    configure_opts=configure_opts_args,
-    configure_opts=' \\\n\t\t'.join([os.path.relpath(f, make_dir) for f in configure_opts_files]),
     make_dir='.',
     makefile=args.output,
     src_dir=os.path.relpath(args.src_dir, make_dir),

ttcn3/ttcn3.sh

@@ -95,20 +95,10 @@ check_ttcn3_install() {
 setup_dir_make() {
 	cd "$DIR_OSMODEV"
 
-	( echo "# Generated by ttcn3.sh, do not edit"
-	  cat ./3G+2G.deps
-	  echo
-	  echo "osmo-bts	libosmocore libosmo-abis"
-	  echo "osmo-pcu	libosmocore"
-	  # just clone these, building is handled by ttcn3.sh
-	  echo "docker-playground"
-	  echo "osmo-ttcn3-hacks" ) > ttcn3/3G+2G_ttcn3.deps
-
 	local docker_cmd="$DIR_OSMODEV/ttcn3/scripts/docker_configure_make.sh"
 	docker_cmd="$docker_cmd $USER/$DOCKER_IMG_BUILD"
 
 	./gen_makefile.py \
-		ttcn3/3G+2G_ttcn3.deps \
 		default.opts \
 		iu.opts \
 		no_systemd.opts \