These are seemingly orthogonal changes in one patch, because they are in fact
sufficiently intertwined that we are not willing to spend the time to separate
them. They are also refactoring changes, unlikely to make sense on their own.
** lu_fsm:
Attempting to make luop.c keep state about incoming GSUP requests made me find
shortcomings in several places:
- since it predates osmo_fsm, it is a state machine that does not strictly
enforce the order of state transitions or the right sequence of incoming
events.
- several places OSMO_ASSERT() on data received from the network.
- modifies the subscriber state before a LU is accepted.
- dead code about canceling a subscriber in a previous VLR. That would be a
good thing to actually do, which should also be trivial now that we record
vlr_name and sgsn_name, but I decided to remove the dead code for now.
To both step up the LU game *and* make it easier for me to integrate
osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now,
ample experience of writing osmo_fsms.
** osmo_gsup_req:
Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs /
communicate with remote MSCs via a proxy:
a) It is important that a response that osmo-hlr generates and that is sent
back to a requesting MSC contains all IEs that are needed to route it back to
the requester. Particularly source_name must become destination_name in the
response to be able to even reach the requesting MSC. Other fields are also
necessary to match, which were so far taken care of in individual numerous code
paths.
b) For some operations, the response to a GSUP request is generated
asynchronously (like Update Location Request -> Response, or taking the
response from an EUSE, or the upcoming proxying to a remote HLR). To be able to
feed a request message's information back into the response, we must thus keep
the request data around. Since struct osmo_gsup_message references a lot of
external data, usually with pointers directly into the received msgb, it is not
so trivial to pass GSUP message data around asynchronously, on its own.
osmo_gsup_req is the combined solution for both a and b: it keeps all data for
a GSUP message by taking ownership of the incoming msgb, and it provides an
explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that
all code paths trivially are definitely responding with the correct IEs set to
match the request's routing (by using osmo_gsup_make_response() recently added
to libosmocore).
Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to
incoming requests received on the GSUP server (above LU code being one of
them).
In fact, the same should be done on the client side. Hence osmo_gsup_req is
implemented in a server/client agnostic way, and is placed in
libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups,
using osmo_gsup_req in the related GSUP client is likely to resolve those
problems without much thinking required beyond making all code paths use it.
libosmo-gsupclient is hence added to osmo-hlr binary's own library
dependencies. It would have been added by the D-GSM proxy routing anyway, we
are just doing it a little sooner.
** cni_peer_id.c / osmo_ipa_name:
We so far handle an IPA unit name as pointer + size, or as just pointer with
implicit talloc size. To ease working with GSUP peer identification data, I
require:
- a non-allocated storage of an IPA Name. It brings the drawback of being
size limited, but our current implementation is anyway only able to handle
MSC and SGSN names of 31 characters (see struct hlr_subscriber).
- a single-argument handle for IPA Name,
- easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying
by simple assignment, a = b.
Hence this patch adds a osmo_ipa_name in cni_peer_id.h and cni_peer_id.c. Heavily
used in LU and osmo_gsup_req.
Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566
Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
From https://sqlite.org/c3ref/exec.html:
To avoid memory leaks, the application should invoke sqlite3_free()
on error message strings returned through the 5th parameter of
sqlite3_exec() after the error message string is no longer needed.
If the 5th parameter to sqlite3_exec() is not NULL and no errors
occur, then sqlite3_exec() sets the pointer in its 5th parameter
to NULL before returning.
Change-Id: Ic9ed9bad3165bc4a637fe963f51e923f012e19ac
Fixes: CID#208182
Standalone program using libosmo-mslookup to easily integrate with programs
that want to connect services (SIP, SMS,...) to the current location of a
subscriber. Also useful for manual testing.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Resubmit of 637bbfcd92 after revert in
41fe362591.
Change-Id: Ie39d30e20461ab10ae3584863d8bfc6b76a12f37
Add the first actually useful lookup method to the mslookup library: multicast
DNS.
The server side is added in a subsequent commit, when the mslookup server is
implemented for the osmo-hlr program.
Use custom DNS encoding instead of libc-ares (which we use in OsmoSGSN
already), because libc-ares is only a DNS client implementation and we will
need both client and server.
Resubmit of f10463c5fc after being
reverted in 110a49f69f. This new version
skips the mslookup_client_mdns test if multicast is not supported in the
build environment. I have verified that it doesn't break the build
anymore in my own OBS namespace.
Related: OS#4237, OS#4361
Patch-by: osmith, nhofmeyr
Change-Id: I3c340627181b632dd6a0d577aa2ea2a7cd035c0c
This reverts commit f10463c5fc, as it
causes all OBS osmo-hlr builds to fail in the mslookup_client_mdns test.
Change-Id: I5aec5b59f304c7f732c4a31131beedf29c966d9d
This reverts commit 637bbfcd92, as it
is a follow-up commit to Change-Id I03a0ffa1d4dc1b24ac78a5ad0975bca90a49c728
which causes all OBS osmo-hlr builds to fail in the mslookup_client_mdns test.
Change-Id: I43084ac3b24684f17df43fefc82019e44baaa236
Standalone program using libosmo-mslookup to easily integrate with programs
that want to connect services (SIP, SMS,...) to the current location of a
subscriber. Also useful for manual testing.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Change-Id: Ie68a5c1db04fb4dff00dc3c774a1162f5b9fabf7
Add the first actually useful lookup method to the mslookup library: multicast
DNS.
The server side is added in a subsequent commit, when the mslookup server is
implemented for the osmo-hlr program.
Use custom DNS encoding instead of libc-ares (which we use in OsmoSGSN
already), because libc-ares is only a DNS client implementation and we will
need both client and server.
Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I03a0ffa1d4dc1b24ac78a5ad0975bca90a49c728
mslookup is a key concept in Distributed GSM, which allows querying the current
location of a subscriber in a number of cooperating but independent core
network sites, by arbitrary service names and by MSISDN/IMSI.
Add the abstract mslookup client library. An actual lookup method (besides
mslookup_client_fake.c) is added in a subsequent patch.
For a detailed overview of this and upcoming patches, please see the elaborate
comment at the top of mslookup.c.
Add as separate library, libosmo-mslookup, to allow adding D-GSM capability to
arbitrary client programs.
osmo-hlr will be the only mslookup server implementation, added in a subsequent
patch.
osmo-hlr itself will also use this library and act as an mslookup client, when
requesting the home HLR for locally unknown IMSIs.
Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I83487ab8aad1611eb02e997dafbcb8344da13df1
Previous the hlr always returned the maximum possible auth vectors (5)
to the client. Even when only asked for a single auth vector.
Change-Id: I20c2b648456bc7ba1fc1321a7d42852158a3523c
Despite LTE/EUTRAN using the same authentication procedure (UMTS AKA)
as 3G, there's one difference: The "operator defined" 16bit AMF field
is reduced to 15 bits, with the first bit now being used as 'separation
bit'. That bit should be '0' for 2G/3G (as it is right now) and '1'
for authentication vectores generated for authentication over
EUTRAN/EPS.
Depends: libosmocore I93850710ab55a605bf61b95063a69682a2899bb1 (OSMO_GSUP_RAT_TYPES_IE)
Change-Id: Ic766bc40f6126bb479bd0a05b0e96bec3e240008
Location Updating procedures from both CS and PS overwrite the same
last_lu_seen field of a subscriber. For upcoming D-GSM it will be important to
distinguish those, because only CS attaches qualify for MSISDN lookup.
Add column last_lu_seen_ps, and upon PS LU, do not overwrite last_lu_seen, so
that last_lu_seen now only reflects CS LU.
In the VTY, dump both LU dates distinctively.
Change-Id: Id7fc50567211a0870ac0524f6dee94d4513781ba
Throughout osmo-hlr's code, the GSUP msgb allocation is duplicated as:
msgb_alloc_headroom(1024+16, 16, "foo");
Instead, use one common function to keep the magic numbers in one place.
Change-Id: I40e99b5bc4fd8f750da7643c03b2119ac3bfd95e
Instead of a switch() for each version number with identical switch cases
except for the function name, use an array of function pointers and loop.
Also print a success message after each individual version upgrade, instead of
only one in the end (see change in db_upgrade_test.ok).
Change-Id: I1736af3d9a3f02e29db836966ac15ce49f94737b
Apply the same headers structure that we keep in most Osmocom source trees:
Keep noinst_HEADERS in include/osmocom/hlr and include them using
#include <osmocom/hlr/*.h>
The only header kept in src/ is db_bootstrap.h, because it is generated during
build time. If it was built in include/osmocom/hlr, we would need db.o to
depend on db_bootstrap.h in a different subdir, which automake can't do well.
Change-Id: Ic912fe27f545b85443c5fb713d8c3c8aac23c9ad
The db bootstrap as well as the upgrade code all execute a number of
statements, and have massive code dup around each statement execution. Instead
have one db_run_statements() that takes an array of strings and runs all.
Change-Id: I2721dfc0a9aadcc7f5ac81a1c0fa87452098996f
The osmo-hlr DB schema indicates a hlr_number column and references it as 3GPP
TS 23.008 chapter 2.4.6. However, chapter 2.4.6 refers to the "MSC number",
while the "HLR number" is chapter 2.4.7.
Taking a closer look, 2.4.6 says "The MSC number is [...] stored in the HLR",
while 2.4.7 says "The HLR number may be stored in the VLR". As quite obvious,
the HLR does not store the HLR number. This was a typo from the start.
The osmo-hlr code base so far does not use the hlr_number column at all, so we
get away with renaming the column without any effects on the code base.
However, let's rather make this a new schema version to be safe.
Change-Id: I527e8627b24b79f3e9eec32675c7f5a3a6d25440
This allows starting osmo-hlr to merely open the database, do upgrades if
necessary, and quit, without opening any ports.
So that no ports are opened, move the telnet VTY startup to below the database
check.
Needed for upcoming patch that introduces a db_upgrade test, in
I0961bab0e17cfde5b030576c5bc243c2b51d9dc4.
Change-Id: I1a4b3360690acd2cd3cffdadffbb00a28d421316
A subsequent commit will add a db_upgrade test, which verifies that the db
resulting from an upgrade is identical to one created from scratch in the new
version. That test currently would show a diff: an upgraded 'imei' column has
'default NULL', where a new db created in version 2 has no default value on the
imei column.
Fix the upgrade path to add an imei column without 'default NULL', so that
adding the upgrade test will result in success. The test is added in
I0961bab0e17cfde5b030576c5bc243c2b51d9dc4
Change-Id: I68a00014a3d603fcba8781470bc5285f78b538d0
Since March 15th 2017, libosmocore API logging_vty_add_cmds() had its
parameter removed (c65c5b4ea075ef6cef11fff9442ae0b15c1d6af7). However,
definition in C file doesn't contain "(void)", which means number of
parameters is undefined and thus compiler doesn't complain. Let's remove
parameters from all callers before enforcing "(void)" on it.
Related: OS#4138
Change-Id: I6d0dbbd83ce17ee798bfb6e30378ed1dbae19134
This change introduces an optional feature that allows to make
SQLite3 use talloc for all internal allocations. This would
facilitate finding memleaks. OsmoHLR needs to be configured
with '--enable-sqlite-talloc'.
full talloc report on 'OsmoHLR' (total 292168 bytes in 449 blocks)
struct osmo_gsup_server contains 162 bytes in 3 blocks (ref 0)
...
struct db_context contains 288407 bytes in 420 blocks (ref 0)
hlr.db contains 7 bytes in 1 blocks (ref 0)
SQLite3 contains 288192 bytes in 418 blocks (ref 0)
db.c:95 contains 48 bytes in 1 blocks (ref 0)
db.c:95 contains 2 bytes in 1 blocks (ref 0)
...
Unfortunately, old SQLite3 versions (such as 3.8.2) run out
of memory when trying to initialize a new database:
DDB ERROR db.c:88 (7) statement aborts at 3: []
DDB ERROR db.c:420 Unable to set Write-Ahead Logging: out of memory
DDB ERROR db.c:88 (7) statement aborts at 3: []
DDB ERROR db.c:238 Unable to prepare SQL statement
'SELECT name FROM sqlite_master WHERE type='table' AND name=?'
...
I've noticed a huge difference in heap usage footprint compared to
generic malloc. At the same time, the recent versions (at least
3.24.0), work just fine.
Change-Id: Icfe67ed0f063b63e6794f9516da3003d01cf20a7
Somehow both 'db_test_SOURCES' and 'db_test_LDADD' ended up in
'src/Makefile.am'. This causes automake / autoconf to complain.
Let's get rid of both useless declarations.
Furthermore, the actual 'db_test_LDADD' in 'tests/Makefile.am'
contained references to the source files from '$(top_srcdir)'.
Most likely, the original intention was to depend on the object
files in '$(top_builddir)'. Let's also fix this.
Change-Id: Ib2e436ed91d9b7551dc5b205329d468c2b0ced04
Replace commented out size check for Ki with a real check, and use it
consistently for Ki, K, OP and OPC. Add a test that sets all keys to the
wrong size and tries to read them.
Related: OS#2565
Change-Id: Ib8e8e9394fb65c6e7932ce9f8bebc321b99f7696
The SS payload is mandatory for GSUP PROC_SS_{REQ,RSP} messages
with session state BEGIN or CONTINUE, and optional for the END.
Make sure that it's present for both BEGIN and CONTINUE, consider
received message as incorrect otherwise. In case of the END, call
handle_ussd() / handle_ss() only if SS payload is present.
Change-Id: Ia71cabbf396bd1388e764a1749e953ac1782e307
Fixes: CID#188841
Allow updating the NAM (Network Access Mode) of subscribers with the
VTY. This is important for the subscriber create on demand use case
where subscribers get created without access to PS and CS NAM by
default. Regenerate hlr_vty_reference.xml.
Related: OS#2542
Change-Id: I231e03219355ebe6467d62ae2e40bef9d8303e3b
IMEIs (without the checksum) always have 14 digits. Replace the previous
check (length <= 14) with a proper one (length == 14) and set the buffer
to the right size. While at it, add the return code of
gsm48_decode_bc_number2() to the error log message.
I have tested with new TTCN3 tests, that the length check is working
properly now.
Related: OS#2541
Change-Id: I060a8db98fb882e4815d1709a5d85bc0143a73a6
The last LU time gets read from the database as string, parsed with
strptime to "struct tm", and then gets converted to time_t with mktime.
A recent behavior change in glibc's mktime implementation unconvered,
that we don't have tm.tm_isdst (daylight saving time) set properly. As
"struct tm" was not initialized, and strptime did not write to tm_isdst,
it was set to a random value. When it was not 0, db_test failed on UTC
systems with a more recent glibc (e.g. Ubuntu 19.04).
Fix this by zero-initializing "struct tm" and remove the previous
workaround that made db_test pass on UTC systems.
Related: OS#4026
Change-Id: Iebbbe42fc5cd43324206d9433ede67b39251389c
Add a new vty option and allow to optionally generate a random msisdn,
as well as setting the default NAM:
subscriber-create-on-demand (no-msisdn|<3-15>) (none|cs|ps|both)
Thanks to Vadim for the random MSISDN patch [1], which was squashed into
this one.
[1] Change-Id: I475c71f9902950fa7498855a616e1ec231fad6ac
Depends on: Idc74f4d94ad44b9fc1b6d43178f5f33d551ebfb1 (libosmocore)
Change-Id: I0c9fe93f5c24b5e9fefb513c4d049fb7ebd47ecd
Related: OS#2542
Check if a subscriber exists without generating an error log entry if
it does not. This is cheaper than db_subscr_get_by_msisdn(), as it
does not fetch the subscriber entry.
subscriber-create-on-demand will use this function to generate
a random unique MSISDN for new subscribers.
Related: OS#2542
Change-Id: Ibfbc408c966197682ba2b12d166ade4bfeb7abc2
Check if a subscriber exists without generating an error log entry if
it does not. This is cheaper than db_subscr_get_by_imsi(), as it does
not fetch the subscriber entry. subscriber-create-on-demand will use
this function.
Related: OS#2542
Change-Id: I63818c0dd4fd22b41dadeeba2a07a651b5454c54
Allow creating new subscribers without giving them access to CS or PS.
This will be used by the create-subscriber-on-demand feature.
Related: OS#2542
Change-Id: I1a6dd85387723dab5487c53b33d2d9ec6d05d006
Neels Hofmeyr
Oliver Smith
Harald Welte
Pau Espin