Each VLR requesting auth tuples should use a distinct IND pool for 3G
auth. So far we tied the IND to the GSUP peer connection; MSC and SGSN
were always distinct GSUP peers, they ended up using distinct INDs.
However, we have implemented a GSUP proxy, so that, in a distributed
setup, a remotely roaming subscriber has only one direct GSUP peer
proxying for both remote MSC and SGSN. That means as soon as a
subscriber roams to a different site, we would use the GSUP proxy name
to determine the IND instead of the separate MSC and SGSN. The site's
MSC and SGSN appear as the same client, get the same IND bucket, waste
SQNs rapidly and cause auth tuple generation load.
So instead of using the local client as IND, persistently keep a list of
VLR names and assign a different IND to each. Use the
gsup_req->source_name as indicator, which reflects the actual remote
VLR's name (remote MSC or SGSN).
Persist the site <-> IND assignments in the database.
Add an IND test to db_test.c
There was an earlier patch version that separated the IND pools by
cn_domain, but it turned out to add complex semantics, while only
solving one aspect of the "adjacent VLR" problem. We need a solution not
only for CS vs PS, but also for 2,3G vs 4G, and for sites that are
physically adjacent to each other. This patch version does not offer any
automatic solution for that -- as soon as more than 2^IND_bitlen
(usually 32) VLRs show up, it is the responsibility of the admin to
ensure the 'ind' table in the hlr.db does not have unfortunate IND
assignments. So far no VTY commands exist for that, they may be added in
the future.
Related: OS#4319
Change-Id: I6f0a6bbef3a27507605c3b4a0e1a89bdfd468374
Remove OpenSUSE bug report link, set version to @VERSION@, make it build
with CentOS 8 etc.
Related: OS#4550
Change-Id: I38f80d0f867a2bbaa09b5a42cad5028f23a8effa
These are seemingly orthogonal changes in one patch, because they are in fact
sufficiently intertwined that we are not willing to spend the time to separate
them. They are also refactoring changes, unlikely to make sense on their own.
** lu_fsm:
Attempting to make luop.c keep state about incoming GSUP requests made me find
shortcomings in several places:
- since it predates osmo_fsm, it is a state machine that does not strictly
enforce the order of state transitions or the right sequence of incoming
events.
- several places OSMO_ASSERT() on data received from the network.
- modifies the subscriber state before a LU is accepted.
- dead code about canceling a subscriber in a previous VLR. That would be a
good thing to actually do, which should also be trivial now that we record
vlr_name and sgsn_name, but I decided to remove the dead code for now.
To both step up the LU game *and* make it easier for me to integrate
osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now,
ample experience of writing osmo_fsms.
** osmo_gsup_req:
Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs /
communicate with remote MSCs via a proxy:
a) It is important that a response that osmo-hlr generates and that is sent
back to a requesting MSC contains all IEs that are needed to route it back to
the requester. Particularly source_name must become destination_name in the
response to be able to even reach the requesting MSC. Other fields are also
necessary to match, which were so far taken care of in individual numerous code
paths.
b) For some operations, the response to a GSUP request is generated
asynchronously (like Update Location Request -> Response, or taking the
response from an EUSE, or the upcoming proxying to a remote HLR). To be able to
feed a request message's information back into the response, we must thus keep
the request data around. Since struct osmo_gsup_message references a lot of
external data, usually with pointers directly into the received msgb, it is not
so trivial to pass GSUP message data around asynchronously, on its own.
osmo_gsup_req is the combined solution for both a and b: it keeps all data for
a GSUP message by taking ownership of the incoming msgb, and it provides an
explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that
all code paths trivially are definitely responding with the correct IEs set to
match the request's routing (by using osmo_gsup_make_response() recently added
to libosmocore).
Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to
incoming requests received on the GSUP server (above LU code being one of
them).
In fact, the same should be done on the client side. Hence osmo_gsup_req is
implemented in a server/client agnostic way, and is placed in
libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups,
using osmo_gsup_req in the related GSUP client is likely to resolve those
problems without much thinking required beyond making all code paths use it.
libosmo-gsupclient is hence added to osmo-hlr binary's own library
dependencies. It would have been added by the D-GSM proxy routing anyway, we
are just doing it a little sooner.
** cni_peer_id.c / osmo_ipa_name:
We so far handle an IPA unit name as pointer + size, or as just pointer with
implicit talloc size. To ease working with GSUP peer identification data, I
require:
- a non-allocated storage of an IPA Name. It brings the drawback of being
size limited, but our current implementation is anyway only able to handle
MSC and SGSN names of 31 characters (see struct hlr_subscriber).
- a single-argument handle for IPA Name,
- easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying
by simple assignment, a = b.
Hence this patch adds a osmo_ipa_name in cni_peer_id.h and cni_peer_id.c. Heavily
used in LU and osmo_gsup_req.
Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566
Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
As pointed out at https://github.com/libexpat/libexpat/issues/312
libtool does not play nice with clang sanitizer builds at all.
For those builds LD shoud be set to clang too (and LDFLAGS needs the
sanitizer flags as well), because the clang compiler driver knows how
linking to the sanitizer libs works, but then at a later stage libtool
fails to actually produce the shared libraries and the build fails. This
is fixed by this patch.
Addtionally LD_LIBRARY_PATH has no effect on conftest runs during
configure time, so the rpath needs to be set to the asan library path to
ensure the configure run does not fail due to a missing asan library,
i.e.:
SANS='-fsanitize=memory -fsanitize-recover=all -shared-libsan'
export CC=clang-10
ASANPATH=$(dirname `$CC -print-file-name=libclang_rt.asan-x86_64.so`)
export LDFLAGS="-Wl,-rpath,$ASANPATH $SANS $LDFLAGS"
Change-Id: Ib98b84ac156f52ecdbb7ae49eaaea35442527b22
Only build and run the test, if --enable-mslookup-client-mdns-test is
passed to ./configure. Enable that option in jenkins.sh.
Related: OS#4385
Change-Id: Ie0cd4b0c55a1fbb00c215aeec7dcd0c15805add3
Standalone program using libosmo-mslookup to easily integrate with programs
that want to connect services (SIP, SMS,...) to the current location of a
subscriber. Also useful for manual testing.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Resubmit of 637bbfcd92 after revert in
41fe362591.
Change-Id: Ie39d30e20461ab10ae3584863d8bfc6b76a12f37
Add the first actually useful lookup method to the mslookup library: multicast
DNS.
The server side is added in a subsequent commit, when the mslookup server is
implemented for the osmo-hlr program.
Use custom DNS encoding instead of libc-ares (which we use in OsmoSGSN
already), because libc-ares is only a DNS client implementation and we will
need both client and server.
Resubmit of f10463c5fc after being
reverted in 110a49f69f. This new version
skips the mslookup_client_mdns test if multicast is not supported in the
build environment. I have verified that it doesn't break the build
anymore in my own OBS namespace.
Related: OS#4237, OS#4361
Patch-by: osmith, nhofmeyr
Change-Id: I3c340627181b632dd6a0d577aa2ea2a7cd035c0c
This reverts commit f10463c5fc, as it
causes all OBS osmo-hlr builds to fail in the mslookup_client_mdns test.
Change-Id: I5aec5b59f304c7f732c4a31131beedf29c966d9d
This reverts commit 637bbfcd92, as it
is a follow-up commit to Change-Id I03a0ffa1d4dc1b24ac78a5ad0975bca90a49c728
which causes all OBS osmo-hlr builds to fail in the mslookup_client_mdns test.
Change-Id: I43084ac3b24684f17df43fefc82019e44baaa236
Standalone program using libosmo-mslookup to easily integrate with programs
that want to connect services (SIP, SMS,...) to the current location of a
subscriber. Also useful for manual testing.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Change-Id: Ie68a5c1db04fb4dff00dc3c774a1162f5b9fabf7
Add the first actually useful lookup method to the mslookup library: multicast
DNS.
The server side is added in a subsequent commit, when the mslookup server is
implemented for the osmo-hlr program.
Use custom DNS encoding instead of libc-ares (which we use in OsmoSGSN
already), because libc-ares is only a DNS client implementation and we will
need both client and server.
Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I03a0ffa1d4dc1b24ac78a5ad0975bca90a49c728
mslookup is a key concept in Distributed GSM, which allows querying the current
location of a subscriber in a number of cooperating but independent core
network sites, by arbitrary service names and by MSISDN/IMSI.
Add the abstract mslookup client library. An actual lookup method (besides
mslookup_client_fake.c) is added in a subsequent patch.
For a detailed overview of this and upcoming patches, please see the elaborate
comment at the top of mslookup.c.
Add as separate library, libosmo-mslookup, to allow adding D-GSM capability to
arbitrary client programs.
osmo-hlr will be the only mslookup server implementation, added in a subsequent
patch.
osmo-hlr itself will also use this library and act as an mslookup client, when
requesting the home HLR for locally unknown IMSIs.
Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I83487ab8aad1611eb02e997dafbcb8344da13df1
The test doesn't do much: it encodes an Insert Subscriber Data request for the
sole purpose to ensure the msgb is allocated large enough. A bug like that is
easily avoided statically.
Also, the lu functions will get refactored soon, it doesn't make sense to me to
drag this test along.
Change-Id: I42e1c72bf4cce8034f968cd4392773bf2b643c1b
Apply the same headers structure that we keep in most Osmocom source trees:
Keep noinst_HEADERS in include/osmocom/hlr and include them using
#include <osmocom/hlr/*.h>
The only header kept in src/ is db_bootstrap.h, because it is generated during
build time. If it was built in include/osmocom/hlr, we would need db.o to
depend on db_bootstrap.h in a different subdir, which automake can't do well.
Change-Id: Ic912fe27f545b85443c5fb713d8c3c8aac23c9ad
We have a database schema upgrade path, but so far nothing that verifies that
we don't break it. It almost seems like the user data weren't important to us!?
Add a db upgrade test:
- Create a db with an .sql dump taken from a db created with an old osmo-hlr,
producing DB schema version 0.
- Run osmo-hlr --db-upgrade --db-check
- Verify that the upgrade exited successfully.
- Verify that restarting with the upgraded DB works.
If python tests are enabled, also:
- create a new database using the new osmo-hlr (just built).
- replay a VTY transcript to create subscribers as in the old snapshot.
- replay some sql modifications as done in the old snapshot.
- Get a list of sorted table names,
- a list of their sorted columns with all their properties,
- and dump the table contents in a column- and value-sorted way.
- Compare the resulting dumps and error if there are any diffs.
(This is how I found the difference in the imei column that was fixed in
I68a00014a3d603fcba8781470bc5285f78b538d0)
Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4
Current code uses GSM23003_MSISDN_MAX_DIGITS, which is only available in
libosmocore 1.2.0 onwards. Let's update configure.ac accordingly.
Fixes: 2e403d6c3f
Change-Id: Iad03a8cf7a36bdc824ec2acc8fb8f9c27a1c0421
This change introduces an optional feature that allows to make
SQLite3 use talloc for all internal allocations. This would
facilitate finding memleaks. OsmoHLR needs to be configured
with '--enable-sqlite-talloc'.
full talloc report on 'OsmoHLR' (total 292168 bytes in 449 blocks)
struct osmo_gsup_server contains 162 bytes in 3 blocks (ref 0)
...
struct db_context contains 288407 bytes in 420 blocks (ref 0)
hlr.db contains 7 bytes in 1 blocks (ref 0)
SQLite3 contains 288192 bytes in 418 blocks (ref 0)
db.c:95 contains 48 bytes in 1 blocks (ref 0)
db.c:95 contains 2 bytes in 1 blocks (ref 0)
...
Unfortunately, old SQLite3 versions (such as 3.8.2) run out
of memory when trying to initialize a new database:
DDB ERROR db.c:88 (7) statement aborts at 3: []
DDB ERROR db.c:420 Unable to set Write-Ahead Logging: out of memory
DDB ERROR db.c:88 (7) statement aborts at 3: []
DDB ERROR db.c:238 Unable to prepare SQL statement
'SELECT name FROM sqlite_master WHERE type='table' AND name=?'
...
I've noticed a huge difference in heap usage footprint compared to
generic malloc. At the same time, the recent versions (at least
3.24.0), work just fine.
Change-Id: Icfe67ed0f063b63e6794f9516da3003d01cf20a7
Moved to doc/manuals/, with full commit history, in preceding merge commit.
Now incorporate in the build system.
Build with:
$ autoreconf -fi
$ ./configure --enable-manuals
$ make
Shared files from osmo-gsm-manuals.git are found automatically if
- the repository is checked out in ../osmo-gsm-manuals; or
- if it osmo-gsm-manuals was installed with "make install"; or
- OSMO_GSM_MANUALS_DIR is set.
Related: OS#3385
Change-Id: I52b7b06fddd77c6dc272004f434e9e7651f6b349
This imports the code from osmo-msc 6afef893e17bce67e4d4119acd34d480ed03ba77
with minimal changes to make it compile. Symbol renaming ot osmo_
prefix is done separately in a follow-up patch to have a as-clean-as-possible
import first.
Requires: libosmocore.git Change-Id Ie36729996abd30b84d1c30a09f62ebc6a9794950
Change-Id: Ief50054ad135551625b684ed8a0486f7af0b2940
Encode an Insert Subscr Data with is_ps == true to trigger the encoding bug
described in OS#3231, i.e. show that it is fixed.
Move osmo_gsup_addr_send() to a separate .c file, so that it can be overridden
in the regression test to just dump the msgb instead.
I used this test to reproduce issue OS#3231, and now that it's here we might as
well keep it, and possibly expand on it in the future.
Related: OS#3231
Change-Id: Id1453351758f3e1a9ff03bd99fefaf51886e77da
Provide a sane means of adding the -Werror compiler flag.
Currently, some of our jenkins.sh add -Werror by passing 'CFLAGS="-Werror"',
but that actually *overwrites* all the other CFLAGS we might want to have set.
Maintain these exceptions from -Werror:
a) deprecation (allow upstream to mark deprecation without breaking builds);
b) "#warning" pragmas (allow to remind ourselves of errors without breaking
builds)
As a last configure step before generating the output files, print the complete
CFLAGS and CPPFLAGS by means of AC_MSG_RESULT.
Change-Id: Id5c0740a37067cbe8986d52d63c6134769c71c47
Add ind_bitlen column to auc_3g to record each USIM's IND size according to
3GPP TS 33.102 -- default is 5 bits, as suggested by the spec.
Introduce auc_3g_ind to each connecting GSUP client to use as IND index for
generating auth tuples sent to this client.
With osmo_gsup_server_add_conn(), implement a scheme where clients receive
fixed auc_3g_ind indexes based on the order in which they connect; each new
connection takes the lowest unused auc_3g_ind, so in case one of the clients
restarts, it will most likely receive the same auc_3g_ind, and if one client
disconnects, no other clients' auc_3g_ind are affected.
Add gsup_server_test.c to test the auc_3g_ind index distribution scheme.
Depends: libosmocore I4eac5be0c0b2cede04464c4c3a0873102d952453 for llist_first
Related: OS#1969
Change-Id: If4501ed4ff8e923fa6fe8b80c44c5ad647a8ed60
Put to-text conversion of the 3GPP TS 55.205 PDF's section defining the test
vectors in tests/auc/gen_ts_55_205_test_sets/ts55_205_test_sets.txt and add
python script to generate auc_ts_55_205_test_sets.c from that at build time.
The generated auc_ts_55_205_test_sets.c runs through all 19 test sets,
verifying that our gsm_milenage() matches the reference data.
Change-Id: Idff9d757ab956179aa41ada2a223fd9f439aafbd
0.9.5 tags the version from which on DLGSUP is properly handled in
libosmocore's internal logging.
This bump follows up on previous osmo-hlr change
I74ab1a031d1ed144468b016294d2965eba5e7d1d.
Change-Id: I2af7a9eff3f116bd3bd02edd42e6e0a64f8280cd
Add configure.ac and Makefile.ams to build with
autoreconf && ./configure && make
like most other Osmocom projects.
Add jenkins.sh for a gerrit build job to verify patches.
Change-Id: I6b4419dd519f3d0a75235d0c22bf899f075347a3
Harald Welte
Neels Hofmeyr
Pau Espin
Oliver Smith
Eric Wild
Neels Hofmeyr