Commit Graph

140 Commits

Author SHA1 Message Date
Oliver Smith a55e63b087 tests/db/db_test.err: adjust to XOR-3G rename
Adjust the test to the related libosmocore change.

Related: libosmocore I446e54d0ddf4a18c46ee022b1249af73552e3ce1
Change-Id: I68878d24340659f888e5e348b937161cffbd54e2
2023-02-22 16:58:57 +01:00
Alexander Couzens 268a33e58b Add vty `reject-cause` to set the reject cause
Allow to set the LU reject cause independently for both of the
following cases; either when an IMSI is unknown to the HLR or
when the mslookup client does not a receive a timely response
to a GSUP request for the remote home HLR.

Original patchset modified by <>

Change-Id: Icea39020c23fbbea9e92847df76af8986fdbf48a
2023-01-20 14:03:01 +00:00
Keith Whyte c27bc90ae6 Vty: Fixup config shown/written from vty
This commit fixes the following errors:
Missing timeout param for mslookup.
Fails to start reading a written config if an msc node
is configured in the mslookup server.
Places the individual msc node services in the
top-level wildcard node due to incorrect indentation.

* Add missing timeout param.
* Add missing "ipa-name" part to mslookup-server-msc section.
* Add one extra space indentation for msc services.

This commit also adds a DEFINE for the default timeout value.

Change-Id: Ibb60e74a0803f05f37ac13ca529d77b4cb736025
2022-12-27 17:03:08 +00:00
Oliver Smith f80ab768d0 osmo_mdns_rfc_record/question_encode: remove ctx
Remove the ctx argument to both functions, as it's not used anymore.

Related: OS#5821
Change-Id: I5e3dd18bf08f7aa42f49c262e13250778ea0c6a2
2022-12-23 13:51:28 +01:00
Oliver Smith 5decd49713 mslookup: use apn functions from libosmocore
Use the apn functions from libosmocore to encode and decode qnames to
avoid code duplication and to avoid unnecessary dynamic allocation.

The unit tests for encoding and decoding rfc_question / rfc_record are
still passing and have the same output.

Fixes: OS#5821
Change-Id: I09d3c617fd6eb4075084ee106d3f3c5803861d2f
2022-12-23 13:51:22 +01:00
Harald Welte 7a4765325d Support building with -Werror=strict-prototypes / -Werror=old-style-definition
Unfortunately "-std=c99" is not sufficient to make gcc ignore code that
uses constructs of earlier C standards, which were abandoned in C99.

See for
some related discussion.

Change-Id: I861be39275b40c522c02f553074c5a4489e87127
2022-11-03 12:56:24 +01:00
Pau Espin 777860ddb5 ctrl: Introduce CTRL command*.aud3g <algo[,KI,(op|opc),OP_C[,ind_bitlen]]>
This command provides getter and setter to set and retrieve the
authentication data for 3g subscribers.

Change-Id: Ibe7aeec3cabab0406eb7a84ecd24e529ef1696c2
Related: SYS#5993
2022-06-21 10:35:40 +02:00
Pau Espin 1d0a030aa4 ctrl: Introduce CTRL command*.aud2g <algo[,ki]>
This command provides getter and setter to set and retrieve the
authentication data for 2g subscribers.

Change-Id: Ibebac232fa173bce8a075cacf477214d5bdb590f
Related: SYS#5993
2022-06-21 10:35:18 +02:00
Pau Espin 3ca9a1fd4f ctrl: Introduce cmd SET subscriber.delete <imsi>
On success SET_REPLY contains the ID of the deleted subscriber.

Related: SYS#5993
Change-Id: I24c6149d7c223a12549d712a7929666220c0210f
2022-06-20 18:05:36 +02:00
Pau Espin 140dffd8f7 ctrl: Introduce CTRL command*.msisdn
This command provides getter and setter to set and retrieve the MSISDN
of a subscriber.

Related: SYS#5993
Change-Id: I5f2e807859f7e28e0984c8dc37edc69319fd8e10
2022-06-20 13:37:59 +02:00
Pau Espin d63ec88dba ctrl: Introduce cmd SET subscriber.create <imsi>
Create a new subscriber from CTRL, similar to VTY command
"imsi IDENT create".

On success SET_REPLY contains the ID of the newly created subscriber.

Related: SYS#5993
Change-Id: Id1b760cd07712245a0eeabaac7891bce93c1fe8e
2022-06-20 13:37:53 +02:00
Pau Espin d456fced21 Fix db_subscr_create() not returning -EEXIST expected by VTY subscriber create cmd
As a result, the -EEXIST code path printing a specific error for
inserting already existing subscribers was not being triggered.

Change-Id: Id24dc6e0ff5115c8c9025404dd7296250d2b03ee
2022-06-17 19:09:39 +02:00
Pau Espin c772e525ef tests/ctrl: Move ERROR test scenario to proper file
Change-Id: I26ba561c22c40877d99f2aed27be2df9faaa4bec
2022-06-17 17:07:56 +02:00
Pau Espin 83a41471da tests: Allow specyfing specific ctrl test to run
Similar to what's already available for vty tests.

Change-Id: I3b01edfb22197f4d46bd24e15bdcd37ea0382e4a
2022-06-17 17:03:25 +02:00
Vadim Yanitskiy b246580900 tests: use 'check_PROGRAMS' instead of 'noinst_PROGRAMS'
When using 'check_PROGRAMS', autoconf/automake generates smarter
Makefiles, so that the test programs are not being compiled during
the normal 'make all', but only during 'make check'.

Change-Id: I70cf75b3cb6ae59c92248cc529546750fb0da07b
2022-04-14 02:22:14 +03:00
Neels Hofmeyr 3f9d1977df db v6: determine 3G AUC IND from VLR name
Each VLR requesting auth tuples should use a distinct IND pool for 3G
auth.  So far we tied the IND to the GSUP peer connection; MSC and SGSN
were always distinct GSUP peers, they ended up using distinct INDs.

However, we have implemented a GSUP proxy, so that, in a distributed
setup, a remotely roaming subscriber has only one direct GSUP peer
proxying for both remote MSC and SGSN. That means as soon as a
subscriber roams to a different site, we would use the GSUP proxy name
to determine the IND instead of the separate MSC and SGSN. The site's
MSC and SGSN appear as the same client, get the same IND bucket, waste
SQNs rapidly and cause auth tuple generation load.

So instead of using the local client as IND, persistently keep a list of
VLR names and assign a different IND to each. Use the
gsup_req->source_name as indicator, which reflects the actual remote
VLR's name (remote MSC or SGSN).

Persist the site <-> IND assignments in the database.

Add an IND test to db_test.c

There was an earlier patch version that separated the IND pools by
cn_domain, but it turned out to add complex semantics, while only
solving one aspect of the "adjacent VLR" problem. We need a solution not
only for CS vs PS, but also for 2,3G vs 4G, and for sites that are
physically adjacent to each other. This patch version does not offer any
automatic solution for that -- as soon as more than 2^IND_bitlen
(usually 32) VLRs show up, it is the responsibility of the admin to
ensure the 'ind' table in the hlr.db does not have unfortunate IND
assignments. So far no VTY commands exist for that, they may be added in
the future.

Related: OS#4319
Change-Id: I6f0a6bbef3a27507605c3b4a0e1a89bdfd468374
2021-07-19 09:01:06 +00:00
Keith Whyte ca8e6efca6 vty: enable show subscribers filtered by IMEI
In 89fda3024a I added a
vty command to show a summary of filtered subscribers by imsi
or msisdn. In practice there is a also need to be able to
filter on IMEI.

The idea here is not to replace every operation that could be
done directly on the sql database in the vty, but this one
is useful.

Change-Id: Ic4a11d3ebcf8909c68e9f4617e94dc822491e008
2021-05-13 18:00:43 +00:00
Keith Whyte 649c335602 Correct configuration written from vty
osmo-hlr would not read back written configuration due to
bad indentation in the mslookup->server->service section.

Also ipa-name was missing.

Change-Id: Ied4b9504f06a51c3895054a8b9d07a67eda3aeeb
2021-02-26 01:21:41 +01:00
Pau Espin d6993ea4b5 tests: Replace deprecated API log_set_print_filename
log_set_print_category_hex(0) is added in some places since the old API
used to set that internally to keep backward compatiblity.

Change-Id: I3cf46b7c24452254319d5c3eacceff418211bcf7
2021-02-19 13:31:13 +01:00
Keith Whyte 89fda3024a Add vty command to show summary of all or filtered subscribers
Adds the following commands:

show subscribers all - Display summary of all entries in HLR
show subscribers (imsi|msisdn|cs|ps) ... As above but filter on search field/string
show subscribers last seen - Display only subscribers with data in
Last LU update field, and sorts by Last LU.

Change-Id: I7f0573381a6d0d13841ac6d42d50f0e8389decf4
2021-01-29 21:26:08 +00:00
Harald Welte bd94b41fa8 auc_test.c: Add some comments on what the test cases actually do
Change-Id: Id2b9cf12823e05aeadc6f15df396f8a46ae1639d
2021-01-04 14:03:25 +01:00
Harald Welte 6e237d3a90 support the XOR algorithm for UMTS AKA
Test USIMs as specified by 3GPP use the XOR algorithm not only for 2G
but also for 3G.  libosmocore includes the XOR-3G support since v1.3.0,
but osmo-hlr somehow never made use of it.

Change-Id: I3a452af9c18cd90d00ab4766d6bd1679456bc1a2
Closes: OS#4924
2020-12-28 22:40:09 +01:00
Vadim Yanitskiy dac855e5c8 USSD: add special 'idle' handler to IUSE for testing
Change-Id: I0d566a60e95ce2da951f22ad47c6155c5b6a338c
2020-11-17 18:50:27 +07:00
Pau Espin ed18fa908c Support setting rt-prio and cpu-affinity mask through VTY
Change-Id: I33101aae3e2851febc335f6fbf96228eab7cf6df
Depends: libosmocore.git Change-Id If76a4bd2cc7b3c7adf5d84790a944d78be70e10a
Depends: osmo-gsm-masnuals.git Change-Id Icd75769ef630c3fa985fc5e2154d5521689cdd3c
Related: SYS#4986
2020-08-18 13:25:34 +02:00
Philipp Maier 377fe5a645 doc: do not use loglevel info for log category ss
The log category ss uses info as loglevel vor ss, this is to verbose,
lets use notice instead.

Change-Id: I192a5f07cb7f45adb6f3af1c511b706738bdadf4
2020-05-12 13:34:13 +02:00
Neels Hofmeyr 0d82a87c0d drop error log for when a subscriber does not exist
Checking for existence of a subscriber and seeing that there is none is not
inherently an error. However, osmo-hlr currently logs on all occasions:

  DAUC ERROR Cannot read subscriber from db: MSISDN='1001': No such subscriber

This spams the ERROR log level. Particularly when a D-GSM setup does subscriber
existence checks for every incoming mslookup request, that potentially creates
constant ERROR logging.

The "No such subscriber" part comes from db_sel(), which might also return an
sqlite3_errmsg(). We still want those sqlite3_errmsg()es in the ERROR log.

Hence print an ERROR log only if db_sel() returns an rc != -ENOENT.

Change-Id: I5044e9b4519b948edc4e451cef0f7830d315619b
2020-04-30 19:22:24 +02:00
Neels Hofmeyr 76328bdc91 D-GSM 3/n: implement roaming by mslookup in osmo-hlr
Add mslookup client to find remote home HLRs of unknown IMSIs, and
proxy/forward GSUP for those to the right remote HLR instances.

Add remote_hlr.c to manage one GSUP client per remote HLR GSUP address.

Add proxy.c to keep state about remotely handled IMSIs (remote GSUP address,
MSISDN, and probably more in future patches).  The mslookup_server that
determines whether a given MSISDN is attached locally now also needs to look in
the proxy record: it is always the osmo-hlr immediately peering for the MSC
that should respond to mslookup service address queries like SIP and SMPP.
(Only gsup.hlr service is always answered by the home HLR.)

Add dgsm.c to set up an mdns mslookup client, ask for IMSI homes, and to decide
which GSUP is handled locally and which needs to go to a remote HLR.

Add full VTY config and VTY tests.

For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).

Change-Id: I2fe453553c90e6ee527ed13a13089900efd488aa
2020-04-30 19:22:24 +02:00
Neels Hofmeyr 407925dcab D-GSM 2/n: implement mDNS method of mslookup server
Implement the mslookup server's mDNS responder, to actually service remote
mslookup requests:
- VTY mslookup/server config with service names,
- the mslookup_mdns_server listening for mslookup requests,

For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).

Change-Id: I5cae6459090588b4dd292be90a5e8903432669d2
2020-04-30 19:22:24 +02:00
Neels Hofmeyr 86b507b6ea test_nodes.vty: remove cruft
This stuff is not testing osmo-hlr specific nodes, remove.

Change-Id: Ia11a209778b78ab02424e2abf3f9004fe97cf570
2020-04-30 19:22:24 +02:00
Neels Hofmeyr 04c2375b38 db v5: prep for D-GSM: add vlr_via_proxy and sgsn_via_proxy
D-GSM will store in the HLR DB whether a locally connected MSC has attached the
subscriber (last_lu_seen[_ps]), or whether the attach happened via a GSUP proxy
from a different site.

Add columns for this separately in this patch.

Change-Id: I98c7b3870559ede84adf56e4bf111f53c7487745
2020-04-30 19:22:24 +02:00
Neels Hofmeyr c79bcdedc9 2/2: wrap ipa_name in osmo_cni_peer_id with type enum and union
To be prepared for the future in public API, wrap the new osmo_ipa_name struct
in an enum-type and union called osmo_cni_peer.

During code review it was requested to insert an ability to handle different
kinds of peer id, in order to be able to add a Global Title in the future.

Use the generic osmo_cni_peer only in the publicly visible API. For osmo-hlr
internal code, I intend to postpone implementing this into the future, when a
different peer identification actually gets introduced.

This way we don't need to implement it now in all osmo-hlr code paths (save
time now), but still make all external API users aware that this type may be
extended in the future.

Change-Id: Ide9dcdca283ab989240cfc6e53e9211862a199c5
2020-04-30 19:19:17 +02:00
Neels Hofmeyr ad868e29ba 1/2: refactor: add and use lu_fsm, osmo_gsup_req, osmo_ipa_name
These are seemingly orthogonal changes in one patch, because they are in fact
sufficiently intertwined that we are not willing to spend the time to separate
them. They are also refactoring changes, unlikely to make sense on their own.

** lu_fsm:

Attempting to make luop.c keep state about incoming GSUP requests made me find
shortcomings in several places:
- since it predates osmo_fsm, it is a state machine that does not strictly
  enforce the order of state transitions or the right sequence of incoming
- several places OSMO_ASSERT() on data received from the network.
- modifies the subscriber state before a LU is accepted.
- dead code about canceling a subscriber in a previous VLR. That would be a
  good thing to actually do, which should also be trivial now that we record
  vlr_name and sgsn_name, but I decided to remove the dead code for now.

To both step up the LU game *and* make it easier for me to integrate
osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now,
ample experience of writing osmo_fsms.

** osmo_gsup_req:

Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs /
communicate with remote MSCs via a proxy:

a) It is important that a response that osmo-hlr generates and that is sent
back to a requesting MSC contains all IEs that are needed to route it back to
the requester. Particularly source_name must become destination_name in the
response to be able to even reach the requesting MSC. Other fields are also
necessary to match, which were so far taken care of in individual numerous code

b) For some operations, the response to a GSUP request is generated
asynchronously (like Update Location Request -> Response, or taking the
response from an EUSE, or the upcoming proxying to a remote HLR). To be able to
feed a request message's information back into the response, we must thus keep
the request data around. Since struct osmo_gsup_message references a lot of
external data, usually with pointers directly into the received msgb, it is not
so trivial to pass GSUP message data around asynchronously, on its own.

osmo_gsup_req is the combined solution for both a and b: it keeps all data for
a GSUP message by taking ownership of the incoming msgb, and it provides an
explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that
all code paths trivially are definitely responding with the correct IEs set to
match the request's routing (by using osmo_gsup_make_response() recently added
to libosmocore).

Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to
incoming requests received on the GSUP server (above LU code being one of

In fact, the same should be done on the client side. Hence osmo_gsup_req is
implemented in a server/client agnostic way, and is placed in
libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups,
using osmo_gsup_req in the related GSUP client is likely to resolve those
problems without much thinking required beyond making all code paths use it.

libosmo-gsupclient is hence added to osmo-hlr binary's own library
dependencies. It would have been added by the D-GSM proxy routing anyway, we
are just doing it a little sooner.

** cni_peer_id.c / osmo_ipa_name:

We so far handle an IPA unit name as pointer + size, or as just pointer with
implicit talloc size. To ease working with GSUP peer identification data, I

- a non-allocated storage of an IPA Name. It brings the drawback of being
  size limited, but our current implementation is anyway only able to handle
  MSC and SGSN names of 31 characters (see struct hlr_subscriber).
- a single-argument handle for IPA Name,
- easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying
  by simple assignment, a = b.

Hence this patch adds a osmo_ipa_name in cni_peer_id.h and cni_peer_id.c. Heavily
used in LU and osmo_gsup_req.

Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566
Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
2020-04-30 19:16:09 +02:00
Oliver Smith 5424dcb879 mslookup_client_mdns_test: no automatic skip
Exit with error code if multicast is disabled. The test is disabled by
default already, so when explicitly enabling it, we should not
automatically skip it.

Related: OS#4385
Change-Id: I82022c23fa9c40535f922b12d917efd7e229912b
2020-02-14 11:16:47 +00:00
Oliver Smith f0e90e6bd5 mslookup_client_mdns_test: disable by default
Only build and run the test, if --enable-mslookup-client-mdns-test is
passed to ./configure. Enable that option in

Related: OS#4385
Change-Id: Ie0cd4b0c55a1fbb00c215aeec7dcd0c15805add3
2020-02-14 11:16:47 +00:00
Oliver Smith 3a9f267983 add mDNS lookup method to libosmo-mslookup (#2)
Add the first actually useful lookup method to the mslookup library: multicast

The server side is added in a subsequent commit, when the mslookup server is
implemented for the osmo-hlr program.

Use custom DNS encoding instead of libc-ares (which we use in OsmoSGSN
already), because libc-ares is only a DNS client implementation and we will
need both client and server.

Resubmit of f10463c5fc after being
reverted in 110a49f69f. This new version
skips the mslookup_client_mdns test if multicast is not supported in the
build environment. I have verified that it doesn't break the build
anymore in my own OBS namespace.

Related: OS#4237, OS#4361
Patch-by: osmith, nhofmeyr
Change-Id: I3c340627181b632dd6a0d577aa2ea2a7cd035c0c
2020-01-13 14:10:50 +01:00
Harald Welte 110a49f69f Revert "add mDNS lookup method to libosmo-mslookup"
This reverts commit f10463c5fc, as it
causes all OBS osmo-hlr builds to fail in the mslookup_client_mdns test.

Change-Id: I5aec5b59f304c7f732c4a31131beedf29c966d9d
2020-01-12 16:27:42 +01:00
Oliver Smith f10463c5fc add mDNS lookup method to libosmo-mslookup
Add the first actually useful lookup method to the mslookup library: multicast

The server side is added in a subsequent commit, when the mslookup server is
implemented for the osmo-hlr program.

Use custom DNS encoding instead of libc-ares (which we use in OsmoSGSN
already), because libc-ares is only a DNS client implementation and we will
need both client and server.

Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I03a0ffa1d4dc1b24ac78a5ad0975bca90a49c728
2020-01-10 16:07:40 +00:00
Oliver Smith bf7deda0fc add libosmo-mslookup abstract client
mslookup is a key concept in Distributed GSM, which allows querying the current
location of a subscriber in a number of cooperating but independent core
network sites, by arbitrary service names and by MSISDN/IMSI.

Add the abstract mslookup client library. An actual lookup method (besides
mslookup_client_fake.c) is added in a subsequent patch.

For a detailed overview of this and upcoming patches, please see the elaborate
comment at the top of mslookup.c.

Add as separate library, libosmo-mslookup, to allow adding D-GSM capability to
arbitrary client programs.

osmo-hlr will be the only mslookup server implementation, added in a subsequent

osmo-hlr itself will also use this library and act as an mslookup client, when
requesting the home HLR for locally unknown IMSIs.

Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I83487ab8aad1611eb02e997dafbcb8344da13df1
2020-01-10 16:07:40 +00:00
Oliver Smith 276c5a7719 tests/auc: change back to python3
Change the interpreter of the python script back to python3, as it was
when the script was initially added in
Idff9d757ab956179aa41ada2a223fd9f439aafbd. In the meantime, it had been
changed to python2 to make it work with build slaves that were missing
python3, but this is not necessary anymore.

This should be merged shortly after osmo-python-tests was migrated to
python3, and the jenkins build slaves were (automatically) updated to
have the new osmo-python-tests installed.

Related: OS#2819
Depends: osmo-python-tests I3ffc3519bf6c22536a49dad7a966188ddad351a7
Change-Id: Ifbb8f8f5428657a1c2d4d6d1217f8e374be43aba
2019-12-10 14:37:25 +01:00
Harald Welte 06f5af22c8 AUC: Add support for setting the AMF separation bit to '1' for EUTRAN
Despite LTE/EUTRAN using the same authentication procedure (UMTS AKA)
as 3G, there's one difference: The "operator defined" 16bit AMF field
is reduced to 15 bits, with the first bit now being used as 'separation
bit'.  That bit should be '0' for 2G/3G (as it is right now) and '1'
for authentication vectores generated for authentication over

Depends: libosmocore I93850710ab55a605bf61b95063a69682a2899bb1 (OSMO_GSUP_RAT_TYPES_IE)
Change-Id: Ic766bc40f6126bb479bd0a05b0e96bec3e240008
2019-12-01 16:09:16 +01:00
Neels Hofmeyr 07e1602d2d db v4: add column last_lu_seen_ps
Location Updating procedures from both CS and PS overwrite the same
last_lu_seen field of a subscriber. For upcoming D-GSM it will be important to
distinguish those, because only CS attaches qualify for MSISDN lookup.

Add column last_lu_seen_ps, and upon PS LU, do not overwrite last_lu_seen, so
that last_lu_seen now only reflects CS LU.

In the VTY, dump both LU dates distinctively.

Change-Id: Id7fc50567211a0870ac0524f6dee94d4513781ba
2019-11-27 03:19:06 +01:00
Neels Hofmeyr abdfdb8a4a test_nodes.vty: tweak: add some '?' checks
I added these "by accident" when implementing D-GSM related VTY tests, now
submitting them separately.

Change-Id: I92a4245cae806270b00330403cc114017ab7af53
2019-11-25 13:58:51 +01:00
Neels Hofmeyr 7355d0ddfe remove gsup_test
The test doesn't do much: it encodes an Insert Subscriber Data request for the
sole purpose to ensure the msgb is allocated large enough. A bug like that is
easily avoided statically.

Also, the lu functions will get refactored soon, it doesn't make sense to me to
drag this test along.

Change-Id: I42e1c72bf4cce8034f968cd4392773bf2b643c1b
2019-11-25 13:58:51 +01:00
Neels Hofmeyr 6b305b4c30 Makefile convenience: add VTY_TEST var to run only one test
VTY transcript tests run all *.vty test scripts, and it is not so
trivial to figure out the test-db creation and cmdline to run only one
of them when debugging. Add VTY_TEST var, useful to pick one test on the

  cd tests
  make vty-test VTY_TEST=test_nodes.vty

Not all VTY tests leave files behind that match hlr_vty_test.db-*, so
make sure that make does not fail it they can't be deleted (rm -f).

Change-Id: I4ad7ddb31b2bfb668b3540cfef658417dd442375
2019-11-25 13:58:16 +01:00
Neels Hofmeyr 981e126686 db upgrade: remove some code dup
Instead of a switch() for each version number with identical switch cases
except for the function name, use an array of function pointers and loop.

Also print a success message after each individual version upgrade, instead of
only one in the end (see change in db_upgrade_test.ok).

Change-Id: I1736af3d9a3f02e29db836966ac15ce49f94737b
2019-11-25 13:13:18 +01:00
Neels Hofmeyr 7143f3a0cb fix upgrade test in presence of ~/.sqliterc

- If an ~/.sqliterc file exists, it causes output of '-- Loading resources from
  ~/.sqliterc'. Use -batch option to omit that.

- To make sure that column headers are off when required, add -noheaders in
  some places.

Change-Id: I279a39984563594a4a3914b2ce3d803ad9468fe8
2019-11-25 13:13:18 +01:00
Neels Hofmeyr 2f75803e5d move headers to include/osmocom/hlr
Apply the same headers structure that we keep in most Osmocom source trees:
Keep noinst_HEADERS in include/osmocom/hlr and include them using
  #include <osmocom/hlr/*.h>

The only header kept in src/ is db_bootstrap.h, because it is generated during
build time. If it was built in include/osmocom/hlr, we would need db.o to
depend on db_bootstrap.h in a different subdir, which automake can't do well.

Change-Id: Ic912fe27f545b85443c5fb713d8c3c8aac23c9ad
2019-11-20 01:25:39 +01:00
Neels Hofmeyr a8045daeef hlr db schema 3: hlr_number -> msc_number
The osmo-hlr DB schema indicates a hlr_number column and references it as 3GPP
TS 23.008 chapter 2.4.6. However, chapter 2.4.6 refers to the "MSC number",
while the "HLR number" is chapter 2.4.7.

Taking a closer look, 2.4.6 says "The MSC number is [...] stored in the HLR",
while 2.4.7 says "The HLR number may be stored in the VLR". As quite obvious,
the HLR does not store the HLR number. This was a typo from the start.

The osmo-hlr code base so far does not use the hlr_number column at all, so we
get away with renaming the column without any effects on the code base.
However, let's rather make this a new schema version to be safe.

Change-Id: I527e8627b24b79f3e9eec32675c7f5a3a6d25440
2019-11-12 16:23:48 +00:00
Oliver Smith 4359b885d4 tests/db_upgrade: disable for old sqlite versions
Skip the test if the installed sqlite version is older than 3.16.2
(current version of debian 9). This prevents test failures caused by
the way we dump tables in the test, which does not work with older

This patch is a middle ground between reverting the db upgrade patch,
and spending lots of time to replace the table dumping code with
something that works with old sqlite versions to fix current build
failures in OBS.

Usually version checking is done in, however I could not
find a good way to pass the result to So I decided to
use pkg-config to do the test there.

Fixes: 5b65461d68 ("add db_upgrade test")
Change-Id: I348c133003a95badbd6807d1519aa669115872fb
2019-11-12 13:39:00 +01:00
Neels Hofmeyr 5b65461d68 add db_upgrade test
We have a database schema upgrade path, but so far nothing that verifies that
we don't break it. It almost seems like the user data weren't important to us!?

Add a db upgrade test:
- Create a db with an .sql dump taken from a db created with an old osmo-hlr,
  producing DB schema version 0.
- Run osmo-hlr --db-upgrade --db-check
- Verify that the upgrade exited successfully.
- Verify that restarting with the upgraded DB works.

If python tests are enabled, also:
- create a new database using the new osmo-hlr (just built).
- replay a VTY transcript to create subscribers as in the old snapshot.
- replay some sql modifications as done in the old snapshot.
- Get a list of sorted table names,
- a list of their sorted columns with all their properties,
- and dump the table contents in a column- and value-sorted way.
- Compare the resulting dumps and error if there are any diffs.
(This is how I found the difference in the imei column that was fixed in

Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4
2019-10-31 21:32:58 +01:00