Commit Graph

113 Commits

Author SHA1 Message Date
Neels Hofmeyr 191f4b6eb3 db v6: determine 3G AUC IND from VLR name
Each VLR requesting auth tuples should use a distinct IND pool for 3G auth.  So
far we tied the IND to the GSUP peer connection; MSC and SGSN were always
distinct GSUP peers, they ended up using distinct INDs.

However, we have implemented a GSUP proxy, so that, in a distributed setup, a
remotely roaming subscriber has only one direct GSUP peer proxying for both
remote MSC and SGSN. That means as soon as a subscriber roams to a different
site, the site's MSC and SGSN compete for the same IND bucket, waste SQNs
rapidly and cause auth tuple generation load.

So instead of using the local client as IND, persistently keep a list of VLR
names and assign a different IND to each. Use the GSUP source_name as
indicator, which reflects the actual remote VLR's name.

Furthermore, make 100% sure that CS and PS pools can never collide, by keeping
all CS pools odd numbered and all PS pools even numbered.

Persist the site <-> IND assignments in the database.

Add an IND test to db_test.c

Related: OS#4319
Change-Id: I6f0a6bbef3a27507605c3b4a0e1a89bdfd468374
2019-12-12 21:51:27 +01:00
Neels Hofmeyr 20a757ee16 WIP: proxy routing refactor / SMS-over-GSUP
Change-Id: I43ad27f6d768df02abb3459ac4c43bb80cc1cbeb
2019-12-12 21:51:27 +01:00
Neels Hofmeyr f1e54d08ca drop error log for when a subscriber does not exist
Checking for existence of a subscriber and seeing that there is none is not
inherently an error. However, osmo-hlr currently logs on all occasions:

  DAUC ERROR Cannot read subscriber from db: MSISDN='1001': No such subscriber

This spams the ERROR log level. Particularly when a D-GSM setup does subscriber
existence checks for every incoming mslookup request, that potentially creates
constant ERROR logging.

The "No such subscriber" part comes from db_sel(), which might also return an
sqlite3_errmsg(). We still want those sqlite3_errmsg()es in the ERROR log.

Hence print an ERROR log only if db_sel() returns an rc != -ENOENT.

Change-Id: I5044e9b4519b948edc4e451cef0f7830d315619b
2019-12-12 21:51:27 +01:00
Neels Hofmeyr 417eb4a2ac D-GSM 3/n: implement roaming by mslookup in osmo-hlr
Add mslookup client to find remote home HLRs of unknown IMSIs, and
proxy/forward GSUP for those to the right remote HLR instances.

Add remote_hlr.c to manage one GSUP client per remote HLR GSUP address.

Add proxy.c to keep state about remotely handled IMSIs (remote GSUP address,
MSISDN, and probably more in future patches).  The mslookup_server that
determines whether a given MSISDN is attached locally now also needs to look in
the proxy record: it is always the osmo-hlr immediately peering for the MSC
that should respond to mslookup service address queries like SIP and SMPP.
(Only gsup.hlr service is always answered by the home HLR.)

Add dgsm.c to set up an mdns mslookup client, ask for IMSI homes, and to decide
which GSUP is handled locally and which needs to go to a remote HLR.

Add full VTY config and VTY tests.

For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).

Change-Id: I2fe453553c90e6ee527ed13a13089900efd488aa
2019-12-12 21:51:27 +01:00
Neels Hofmeyr a68d284e29 D-GSM 2/n: implement mDNS method of mslookup server
Implement the mslookup server's mDNS responder, to actually service remote
mslookup requests:
- VTY mslookup/server config with service names,
- the mslookup_mdns_server listening for mslookup requests,

For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).

Change-Id: I5cae6459090588b4dd292be90a5e8903432669d2
2019-12-12 21:51:27 +01:00
Neels Hofmeyr bdf5577f3d test_nodes.vty: remove cruft
This stuff is not testing osmo-hlr specific nodes, remove.

Change-Id: Ia11a209778b78ab02424e2abf3f9004fe97cf570
2019-12-12 21:51:27 +01:00
Neels Hofmeyr 35865d8647 db v5: prep for D-GSM: add vlr_via_proxy and sgsn_via_proxy
D-GSM will store in the HLR DB whether a locally connected MSC has attached the
subscriber (last_lu_seen[_ps]), or whether the attach happened via a GSUP proxy
from a different site.

Add columns for this separately in this patch.

Change-Id: I98c7b3870559ede84adf56e4bf111f53c7487745
2019-12-12 21:51:27 +01:00
Neels Hofmeyr 714b82e2c4 2/2: fixup: add osmo_gsup_peer_id with type enum and union
During code review it was requested to insert an ability to handle different
kinds of peer id, in order to be able to add a Global Title in the future.

Add this, but only in the publicly visible API. For osmo-hlr internal code, I
intend to push implementing this into the future, when a different peer
identification actually gets introduced.

This way we don't need to implement it now in all osmo-hlr code paths (save
time now), but still make all API users aware that this type may be extended in
the future.

Change-Id: Ide9dcdca283ab989240cfc6e53e9211862a199c5
2019-12-12 21:51:27 +01:00
Neels Hofmeyr cbf03ae5a4 1/2: refactor: add and use lu_fsm, osmo_gsup_req, osmo_ipa_name
These are seemingly orthogonal changes in one patch, because they are in fact
sufficiently intertwined that we are not willing to spend the time to separate
them. They are also refactoring changes, unlikely to make sense on their own.

** lu_fsm:

Attempting to make luop.c keep state about incoming GSUP requests made me find
shortcomings in several places:
- since it predates osmo_fsm, it is a state machine that does not strictly
  enforce the order of state transitions or the right sequence of incoming
  events.
- several places OSMO_ASSERT() on data received from the network.
- modifies the subscriber state before a LU is accepted.
- dead code about canceling a subscriber in a previous VLR. That would be a
  good thing to actually do, which should also be trivial now that we record
  vlr_name and sgsn_name, but I decided to remove the dead code for now.

To both step up the LU game *and* make it easier for me to integrate
osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now,
ample experience of writing osmo_fsms.

** osmo_gsup_req:

Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs /
communicate with remote MSCs via a proxy:

a) It is important that a response that osmo-hlr generates and that is sent
back to a requesting MSC contains all IEs that are needed to route it back to
the requester. Particularly source_name must become destination_name in the
response to be able to even reach the requesting MSC. Other fields are also
necessary to match, which were so far taken care of in individual numerous code
paths.

b) For some operations, the response to a GSUP request is generated
asynchronously (like Update Location Request -> Response, or taking the
response from an EUSE, or the upcoming proxying to a remote HLR). To be able to
feed a request message's information back into the response, we must thus keep
the request data around. Since struct osmo_gsup_message references a lot of
external data, usually with pointers directly into the received msgb, it is not
so trivial to pass GSUP message data around asynchronously, on its own.

osmo_gsup_req is the combined solution for both a and b: it keeps all data for
a GSUP message by taking ownership of the incoming msgb, and it provides an
explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that
all code paths trivially are definitely responding with the correct IEs set to
match the request's routing (by using osmo_gsup_make_response() recently added
to libosmocore).

Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to
incoming requests received on the GSUP server (above LU code being one of
them).

In fact, the same should be done on the client side. Hence osmo_gsup_req is
implemented in a server/client agnostic way, and is placed in
libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups,
using osmo_gsup_req in the related GSUP client is likely to resolve those
problems without much thinking required beyond making all code paths use it.

libosmo-gsupclient is hence added to osmo-hlr binary's own library
dependencies. It would have been added by the D-GSM proxy routing anyway, we
are just doing it a little sooner.

** gsup_peer_id.c / osmo_ipa_name:

We so far handle an IPA unit name as pointer + size, or as just pointer with
implicit talloc size. To ease working with GSUP peer identification data, I
require:

- a non-allocated storage of an IPA Name. It brings the drawback of being
  size limited, but our current implementation is anyway only able to handle
  MSC and SGSN names of 31 characters (see struct hlr_subscriber).
- a single-argument handle for IPA Name,
- easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying
  by simple assignment, a = b.

Hence this patch adds a osmo_ipa_name in gsup_peer_id.h and gsup_peer_id.c. Heavily
used in LU and osmo_gsup_req.

Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566
Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
2019-12-12 21:51:27 +01:00
Oliver Smith 4bd1100277 add mDNS lookup method to libosmo-mslookup
Add the first actually useful lookup method to the mslookup library: multicast
DNS.

The server side is added in a subsequent commit, when the mslookup server is
implemented for the osmo-hlr program.

Use custom DNS encoding instead of libc-ares (which we use in OsmoSGSN
already), because libc-ares is only a DNS client implementation and we will
need both client and server.

Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I03a0ffa1d4dc1b24ac78a5ad0975bca90a49c728
2019-12-12 21:51:27 +01:00
Oliver Smith 6871e5ea63 add libosmo-mslookup abstract client
mslookup is a key concept in Distributed GSM, which allows querying the current
location of a subscriber in a number of cooperating but independent core
network sites, by arbitrary service names and by MSISDN/IMSI.

Add the abstract mslookup client library. An actual lookup method (besides
mslookup_client_fake.c) is added in a subsequent patch.

For a detailed overview of this and upcoming patches, please see the elaborate
comment at the top of mslookup.c.

Add as separate library, libosmo-mslookup, to allow adding D-GSM capability to
arbitrary client programs.

osmo-hlr will be the only mslookup server implementation, added in a subsequent
patch.

osmo-hlr itself will also use this library and act as an mslookup client, when
requesting the home HLR for locally unknown IMSIs.

Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I83487ab8aad1611eb02e997dafbcb8344da13df1
2019-12-12 21:51:27 +01:00
Oliver Smith 276c5a7719 tests/auc: change back to python3
Change the interpreter of the python script back to python3, as it was
when the script was initially added in
Idff9d757ab956179aa41ada2a223fd9f439aafbd. In the meantime, it had been
changed to python2 to make it work with build slaves that were missing
python3, but this is not necessary anymore.

This should be merged shortly after osmo-python-tests was migrated to
python3, and the jenkins build slaves were (automatically) updated to
have the new osmo-python-tests installed.

Related: OS#2819
Depends: osmo-python-tests I3ffc3519bf6c22536a49dad7a966188ddad351a7
Change-Id: Ifbb8f8f5428657a1c2d4d6d1217f8e374be43aba
2019-12-10 14:37:25 +01:00
Harald Welte 06f5af22c8 AUC: Add support for setting the AMF separation bit to '1' for EUTRAN
Despite LTE/EUTRAN using the same authentication procedure (UMTS AKA)
as 3G, there's one difference: The "operator defined" 16bit AMF field
is reduced to 15 bits, with the first bit now being used as 'separation
bit'.  That bit should be '0' for 2G/3G (as it is right now) and '1'
for authentication vectores generated for authentication over
EUTRAN/EPS.

Depends: libosmocore I93850710ab55a605bf61b95063a69682a2899bb1 (OSMO_GSUP_RAT_TYPES_IE)
Change-Id: Ic766bc40f6126bb479bd0a05b0e96bec3e240008
2019-12-01 16:09:16 +01:00
Neels Hofmeyr 07e1602d2d db v4: add column last_lu_seen_ps
Location Updating procedures from both CS and PS overwrite the same
last_lu_seen field of a subscriber. For upcoming D-GSM it will be important to
distinguish those, because only CS attaches qualify for MSISDN lookup.

Add column last_lu_seen_ps, and upon PS LU, do not overwrite last_lu_seen, so
that last_lu_seen now only reflects CS LU.

In the VTY, dump both LU dates distinctively.

Change-Id: Id7fc50567211a0870ac0524f6dee94d4513781ba
2019-11-27 03:19:06 +01:00
Neels Hofmeyr abdfdb8a4a test_nodes.vty: tweak: add some '?' checks
I added these "by accident" when implementing D-GSM related VTY tests, now
submitting them separately.

Change-Id: I92a4245cae806270b00330403cc114017ab7af53
2019-11-25 13:58:51 +01:00
Neels Hofmeyr 7355d0ddfe remove gsup_test
The test doesn't do much: it encodes an Insert Subscriber Data request for the
sole purpose to ensure the msgb is allocated large enough. A bug like that is
easily avoided statically.

Also, the lu functions will get refactored soon, it doesn't make sense to me to
drag this test along.

Change-Id: I42e1c72bf4cce8034f968cd4392773bf2b643c1b
2019-11-25 13:58:51 +01:00
Neels Hofmeyr 6b305b4c30 Makefile convenience: add VTY_TEST var to run only one test
VTY transcript tests run all *.vty test scripts, and it is not so
trivial to figure out the test-db creation and cmdline to run only one
of them when debugging. Add VTY_TEST var, useful to pick one test on the
cmdline:

  cd tests
  make vty-test VTY_TEST=test_nodes.vty

Not all VTY tests leave files behind that match hlr_vty_test.db-*, so
make sure that make does not fail it they can't be deleted (rm -f).

Change-Id: I4ad7ddb31b2bfb668b3540cfef658417dd442375
2019-11-25 13:58:16 +01:00
Neels Hofmeyr 981e126686 db upgrade: remove some code dup
Instead of a switch() for each version number with identical switch cases
except for the function name, use an array of function pointers and loop.

Also print a success message after each individual version upgrade, instead of
only one in the end (see change in db_upgrade_test.ok).

Change-Id: I1736af3d9a3f02e29db836966ac15ce49f94737b
2019-11-25 13:13:18 +01:00
Neels Hofmeyr 7143f3a0cb fix upgrade test in presence of ~/.sqliterc
db_upgrade_test.sh:

- If an ~/.sqliterc file exists, it causes output of '-- Loading resources from
  ~/.sqliterc'. Use -batch option to omit that.

- To make sure that column headers are off when required, add -noheaders in
  some places.

Change-Id: I279a39984563594a4a3914b2ce3d803ad9468fe8
2019-11-25 13:13:18 +01:00
Neels Hofmeyr 2f75803e5d move headers to include/osmocom/hlr
Apply the same headers structure that we keep in most Osmocom source trees:
Keep noinst_HEADERS in include/osmocom/hlr and include them using
  #include <osmocom/hlr/*.h>

The only header kept in src/ is db_bootstrap.h, because it is generated during
build time. If it was built in include/osmocom/hlr, we would need db.o to
depend on db_bootstrap.h in a different subdir, which automake can't do well.

Change-Id: Ic912fe27f545b85443c5fb713d8c3c8aac23c9ad
2019-11-20 01:25:39 +01:00
Neels Hofmeyr a8045daeef hlr db schema 3: hlr_number -> msc_number
The osmo-hlr DB schema indicates a hlr_number column and references it as 3GPP
TS 23.008 chapter 2.4.6. However, chapter 2.4.6 refers to the "MSC number",
while the "HLR number" is chapter 2.4.7.

Taking a closer look, 2.4.6 says "The MSC number is [...] stored in the HLR",
while 2.4.7 says "The HLR number may be stored in the VLR". As quite obvious,
the HLR does not store the HLR number. This was a typo from the start.

The osmo-hlr code base so far does not use the hlr_number column at all, so we
get away with renaming the column without any effects on the code base.
However, let's rather make this a new schema version to be safe.

Change-Id: I527e8627b24b79f3e9eec32675c7f5a3a6d25440
2019-11-12 16:23:48 +00:00
Oliver Smith 4359b885d4 tests/db_upgrade: disable for old sqlite versions
Skip the test if the installed sqlite version is older than 3.16.2
(current version of debian 9). This prevents test failures caused by
the way we dump tables in the test, which does not work with older
versions.

This patch is a middle ground between reverting the db upgrade patch,
and spending lots of time to replace the table dumping code with
something that works with old sqlite versions to fix current build
failures in OBS.

Usually version checking is done in configure.ac, however I could not
find a good way to pass the result to testsuite.at. So I decided to
use pkg-config to do the test there.

Fixes: 5b65461d68 ("add db_upgrade test")
Related: https://lists.osmocom.org/pipermail/openbsc/2019-November/013063.html
Change-Id: I348c133003a95badbd6807d1519aa669115872fb
2019-11-12 13:39:00 +01:00
Neels Hofmeyr 5b65461d68 add db_upgrade test
We have a database schema upgrade path, but so far nothing that verifies that
we don't break it. It almost seems like the user data weren't important to us!?

Add a db upgrade test:
- Create a db with an .sql dump taken from a db created with an old osmo-hlr,
  producing DB schema version 0.
- Run osmo-hlr --db-upgrade --db-check
- Verify that the upgrade exited successfully.
- Verify that restarting with the upgraded DB works.

If python tests are enabled, also:
- create a new database using the new osmo-hlr (just built).
- replay a VTY transcript to create subscribers as in the old snapshot.
- replay some sql modifications as done in the old snapshot.
- Get a list of sorted table names,
- a list of their sorted columns with all their properties,
- and dump the table contents in a column- and value-sorted way.
- Compare the resulting dumps and error if there are any diffs.
(This is how I found the difference in the imei column that was fixed in
I68a00014a3d603fcba8781470bc5285f78b538d0)

Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4
2019-10-31 21:32:58 +01:00
Oliver Smith 2b0bf31183 tests/test_nodes.vty: check less libosmocore cmds
Use three dots to avoid checking for exact commands between help and
exit, which originate from libosmocore. This avoids test failues when
we slightly change the commands, like the change from "write file" to
"write file [PATH]" in [1] that is currently causing the vty tests to
fail.

[1] libosmocore I38edcf902a08b6bd0ebb9aa6fc1a7041421af525
Change-Id: I4a964b86195141e5a50705425206f3602f908999
2019-10-08 11:11:15 +02:00
Pau Espin 28f0774e34 tests: Fix db_test err file to expect error code name instead of value
Previous commit changed db_test to output code names to fix issues on
some platforms (I guess due to different error code values), but somehow
this log line was not updated.

Fixes: 8b860e54be
Change-Id: I598de6f83a86d528174d3d188596314572f5d70d
2019-08-13 11:59:09 +02:00
Ruben Undheim 8b860e54be Fix test for return codes on mipsel and alpha archs
Change-Id: Ia64f1d9f39fe2b1fb704f7b6c4d9cce93ab708cd
2019-08-12 08:33:33 +00:00
Vadim Yanitskiy fbd736ef37 src/db.c: integrate SQLite3 with talloc allocator
This change introduces an optional feature that allows to make
SQLite3 use talloc for all internal allocations. This would
facilitate finding memleaks. OsmoHLR needs to be configured
with '--enable-sqlite-talloc'.

  full talloc report on 'OsmoHLR' (total 292168 bytes in 449 blocks)
    struct osmo_gsup_server        contains    162 bytes in   3 blocks (ref 0)
      ...
    struct db_context              contains 288407 bytes in 420 blocks (ref 0)
      hlr.db                       contains      7 bytes in   1 blocks (ref 0)
    SQLite3                        contains 288192 bytes in 418 blocks (ref 0)
      db.c:95                      contains     48 bytes in   1 blocks (ref 0)
      db.c:95                      contains      2 bytes in   1 blocks (ref 0)
      ...

Unfortunately, old SQLite3 versions (such as 3.8.2) run out
of memory when trying to initialize a new database:

  DDB ERROR  db.c:88 (7) statement aborts at 3: []
  DDB ERROR  db.c:420 Unable to set Write-Ahead Logging: out of memory
  DDB ERROR  db.c:88 (7) statement aborts at 3: []
  DDB ERROR  db.c:238 Unable to prepare SQL statement
             'SELECT name FROM sqlite_master WHERE type='table' AND name=?'
  ...

I've noticed a huge difference in heap usage footprint compared to
generic malloc. At the same time, the recent versions (at least
3.24.0), work just fine.

Change-Id: Icfe67ed0f063b63e6794f9516da3003d01cf20a7
2019-07-30 17:15:17 +00:00
Vadim Yanitskiy dc30154fdf tests/db_test: close the database when test is finished
Change-Id: I96fedf9181e89e4d68815b04f494a9c2ecc0e057
2019-07-30 17:15:17 +00:00
Vadim Yanitskiy 37642177f9 build: fix mess with 'db_test_SOURCES' and 'db_test_LDADD'
Somehow both 'db_test_SOURCES' and 'db_test_LDADD' ended up in
'src/Makefile.am'. This causes automake / autoconf to complain.
Let's get rid of both useless declarations.

Furthermore, the actual 'db_test_LDADD' in 'tests/Makefile.am'
contained references to the source files from '$(top_srcdir)'.
Most likely, the original intention was to depend on the object
files in '$(top_builddir)'. Let's also fix this.

Change-Id: Ib2e436ed91d9b7551dc5b205329d468c2b0ced04
2019-07-30 17:15:17 +00:00
Oliver Smith 6401b90574 db_auc.c: verify hex key sizes read from DB
Replace commented out size check for Ki with a real check, and use it
consistently for Ki, K, OP and OPC. Add a test that sets all keys to the
wrong size and tries to read them.

Related: OS#2565
Change-Id: Ib8e8e9394fb65c6e7932ce9f8bebc321b99f7696
2019-07-25 14:52:20 +02:00
Oliver Smith 3b33b01fb0 VTY: add subscriber update network-access-mode
Allow updating the NAM (Network Access Mode) of subscribers with the
VTY. This is important for the subscriber create on demand use case
where subscribers get created without access to PS and CS NAM by
default. Regenerate hlr_vty_reference.xml.

Related: OS#2542
Change-Id: I231e03219355ebe6467d62ae2e40bef9d8303e3b
2019-07-15 14:13:46 +02:00
Oliver Smith 63de00cfc1 db_hlr: zero-initialize "struct tm"
The last LU time gets read from the database as string, parsed with
strptime to "struct tm", and then gets converted to time_t with mktime.

A recent behavior change in glibc's mktime implementation unconvered,
that we don't have tm.tm_isdst (daylight saving time) set properly. As
"struct tm" was not initialized, and strptime did not write to tm_isdst,
it was set to a random value. When it was not 0, db_test failed on UTC
systems with a more recent glibc (e.g. Ubuntu 19.04).

Fix this by zero-initializing "struct tm" and remove the previous
workaround that made db_test pass on UTC systems.

Related: OS#4026
Change-Id: Iebbbe42fc5cd43324206d9433ede67b39251389c
2019-06-04 12:41:54 +02:00
Oliver Smith 1a1398ed54 db_test: set timezone to work around mktime bug
Apply workaround for db_test failure on ubuntu 19.04 in OBS. When the
timezone is set to UTC (default in OBS), and the glibc version is 2.29,
the test case is failing sometimes (not always) with the following
errors in the test log:

DAUC Cannot convert LU timestamp '2019-05-26 03:05:03' to time_t: Value too large for defined

Force the timezone to be CET when running the test, so it passes again.
I found this workaround in a Fedora bugreport [1], and tested with an
ubuntu 19.04 system running in docker, that setting the environment
variable like this makes the test pass 25 times in a row. Without it, it
fails every two or three times. Then I verified in my OBS namespace,
that the test also passes in OBS again.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1653340

Related: OS#4026
Change-Id: Ic8080ba1914bb364169ab0c563b132a4ab9a9aab
2019-06-03 11:35:45 +02:00
Oliver Smith c7f1787c18 Create subscribers on demand
Add a new vty option and allow to optionally generate a random msisdn,
as well as setting the default NAM:

subscriber-create-on-demand (no-msisdn|<3-15>) (none|cs|ps|both)

Thanks to Vadim for the random MSISDN patch [1], which was squashed into
this one.

[1] Change-Id: I475c71f9902950fa7498855a616e1ec231fad6ac

Depends on: Idc74f4d94ad44b9fc1b6d43178f5f33d551ebfb1 (libosmocore)
Change-Id: I0c9fe93f5c24b5e9fefb513c4d049fb7ebd47ecd
Related: OS#2542
2019-05-19 14:42:46 +07:00
Vadim Yanitskiy c13599dc69 db_hlr.c: add db_subscr_exists_by_msisdn()
Check if a subscriber exists without generating an error log entry if
it does not. This is cheaper than db_subscr_get_by_msisdn(), as it
does not fetch the subscriber entry.

subscriber-create-on-demand will use this function to generate
a random unique MSISDN for new subscribers.

Related: OS#2542
Change-Id: Ibfbc408c966197682ba2b12d166ade4bfeb7abc2
2019-05-13 08:55:24 +02:00
Oliver Smith 6b73fd9678 db_hlr.c: add db_subscr_exists_by_imsi()
Check if a subscriber exists without generating an error log entry if
it does not. This is cheaper than db_subscr_get_by_imsi(), as it does
not fetch the subscriber entry. subscriber-create-on-demand will use
this function.

Related: OS#2542
Change-Id: I63818c0dd4fd22b41dadeeba2a07a651b5454c54
2019-05-13 08:55:24 +02:00
Oliver Smith cd2af5ead7 db_hlr.c: db_subscr_create(): add flags argument
Allow creating new subscribers without giving them access to CS or PS.
This will be used by the create-subscriber-on-demand feature.

Related: OS#2542
Change-Id: I1a6dd85387723dab5487c53b33d2d9ec6d05d006
2019-05-13 08:55:18 +02:00
Neels Hofmeyr 5857c595b3 osmo-hlr: allow configuring db path from cfg file
So far, the cmdline argument was the only way to set a database config file.
Add a similar config to VTY as 'hlr' / 'database'. The cmdline arg is stronger
than the 'database' cfg item. DB is not reloaded from VTY command.

Change-Id: I87b8673324e1e6225afb758fb4963ff3279ea3d8
2019-05-08 04:12:38 +00:00
Oliver Smith 3cf87fe22c tests: use -no-install libtool flag to avoid ./lt-* scripts
This ensures that the rpath of the generated binaries is set to use only
the just-compiled so-files and not any system-wide installed libraries
while avoiding the ugly shell script wrapper.

Change-Id: I927561289b17b313d52fb5c1da55e237fc1d33be
2019-03-19 13:04:49 +00:00
Max 20ddfdbc53 Enable statsd support
Change-Id: I00b8aa4e59028a4c1098a3bae034e8d8ddfbe681
2019-02-18 19:57:58 +00:00
Oliver Smith 02078b7d91 VTY: integrate IMEI
Display the IMEI in "subscriber ... show", allow showing and modifying
subscribers by their IMEI with: "subscriber imei ...". For debug
purposes (and to have proper VTY tests), make it possible to change the
IMEI with "subscriber ... update imei".

IMEIs are saved in the database without the 15th checksum number. When
the checksum gets passed, verify it and cut it off.

Related: OS#2541
Depends: I02b54cf01a674a1911c5c897fbec02240f88b521 (libosmocore)
Change-Id: I1af7b573ca2a1cb22497052665012d9c1acf3b30
2019-01-24 15:29:08 +00:00
Oliver Smith ef64b231dc VTY tests: fill DB before running test
Create a test_subscriber.vty.sql file with a dummy entry that has the
ID 100. All entries created in test_subscriber.vty have an ID > 100
now. This will be used in follow-up commit [1] to create a database
entry with an invalid IMEI value to test the related error handling
code path (that entry could not be created through the VTY).

[1]: change-id I1af7b573ca2a1cb22497052665012d9c1acf3b30
     "VTY: integrate IMEI"

Related: OS#3733
Change-Id: I48a3a503d7ca96798e2d5f70429b5fc36393420e
2019-01-24 15:29:08 +00:00
Oliver Smith 851814aa7c Optionally store IMEI in subscriber table
Add VTY config option "store-imei". When it is set, store the IMEI
sent from the VLR with CHECK-IMEI in the database.

Related: OS#2541
Change-Id: I09274ecbed64224f7ae305e09ede773931da2a57
2019-01-24 15:29:08 +00:00
Oliver Smith 81db389fd4 Add IMEI column to subscriber table
Extend the database scheme, add imei to the hlr_subscriber struct and
create db_subscr_update_imei_by_imsi() and db_subscr_get_by_imei(). The
new functions are used in db_test, and in follow-up commits [1], [2].

Upgrade DB schema to version 2. SQLite can only insert new columns at
the end of the table, so this happens when upgrading the database. In
new databases, the column is placed after the IMEISV column (where it
makes more sense in my opinion). This should not have any effect, as
we never rely on the order of the columns in the tables.

Follow-up commit [1] will make use of this column to save the IMEI as
received from the MSC/VLR with the Check-IMEI Procedure. It was
decided to use Check-IMEI instead of the recent Automatic Device
Detection Procedure (which would send the IMEISV) in OS#3733, because
with Check-IMEI we don't need to rely on very recent releases of the
specification.

[1] change-id I09274ecbed64224f7ae305e09ede773931da2a57
    "Optionally store IMEI in subscriber table"
[2] change-id I1af7b573ca2a1cb22497052665012d9c1acf3b30
    "VTY: integrate IMEI"

Depends: Id2d2a3a93b033bafc74c62e15297034bf4aafe61 (libosmocore)
Related: OS#2541
Change-Id: If232c80bea35d5c6864b889ae92d477eeaa3f45d
2019-01-24 15:29:08 +00:00
Vadim Yanitskiy d157a56361 SS/USSD: make NCSS session timeout configurable
It may happen that either the MS or an ESME would become
unresponsive, e.g. due to a bug, or a dropped message.
This is why we have SS session timeout, that prevents
keeping 'stalled' sessions forever.

Let's introduce a VTY option, which can be used to configure
this timer (by default it's set to 30 seconds):

hlr
  ...
  ! Use 0 to disable this timer
  ncss-guard-timeout 30

Change-Id: I971fc2cee6fd46d4d5d6dac6c634e0b22fff183d
Related: OS#3717
2018-12-05 19:45:34 +00:00
Stefan Sperling 8f3a7cce80 add database schema versioning to the HLR database
Make use of pragma user_version to store our database schema version.
The present schema is now identitifed as 'version 0', which is also
the default value for databases on which we never ran the statement
'pragma user_version' before.

Only bootstrap the database if it hasn't been bootstrapped yet.
Previously, bootstrap SQL statements ran every time osmo-hlr
opened the database, and any errors were being ignored in SQL.
Instead, we now first run a query which checks whether tables
already exist, and only create them if necessary.
This change will allow future schema updates to work properly.

Prepare for future schema upgrades by adding a new command-line
option which enables upgrades. This option defaults to 'false'
in order to avoid accidental upgrades.

Change-Id: I8aeaa9a404b622657cbc7138106f38aa6ad8d01b
Related: OS#2838
2018-12-04 14:13:28 +01:00
Neels Hofmeyr a820ea1f67 implement removal of MSISDN
Add the first "official" way to remove the MSISDN from a subscriber entry, to
go back to 'MSISDN: none' like just after 'subscriber create'.

Add VTY command 'subscriber <ID> update msisdn none' to drop the MSISDN from
the subscriber. (Like 'subscriber <ID> update aud3g none')

Add DB_STMT_DELETE_MSISDN_BY_IMSI.

In db_subscr_update_msisdn_by_imsi(), allow passing a NULL msisdn, and if NULL,
call above delete SQL statement.

Change-Id: I15419105ea461137776adb92d384d8985210c90e
2018-12-02 20:16:31 +01:00
Neels Hofmeyr 8aa780bf80 add 'show subscriber' command, alias for 'subscriber ... show'
When I wrote the osmo-hlr subscriber command, I failed to heed the common
'show foo' scheme and instead created a 'subscriber [...] show' command.
Relieve that weirdness by creating an alias that has 'show' at the start.

Arrange string macros so that the 'show subscriber' cmd doesn't end in a space
(the SUBSCR macro ends in a space ' ' to implicitly include the space to
commands like 'create', 'show', 'update').

Add the new command to test_nodes.vty and test_subscriber.vty.

Change-Id: I01ce9b0868302d40ed05c6a588316a194d6071e4
2018-12-02 20:06:24 +01:00
Harald Welte c88bdab96d fix VTY tests to expect well-formed output of logging configuration
In libosmocore Change-Id Ia75c7067284ea225cffe13ca71bad05a7747ae66
we fixed the generation (saving) of logging configuration to use
one level of indent, rather than the previous broken implementation
with two levels of indent.  This means we have to adjust the VTY
test expectations here.

Change-Id: I9282a59bbfad4cfc86e86c1212c74cccfe187ff8
2018-09-26 09:23:37 +02:00
Neels Hofmeyr ccdb970c57 make: always allow running python tests manually
Keep the rules to run the external-tests in tests/Makefile available for
invocation, to allow conveniently launching the tests manualy without the need
to pass --with-external-tests to ./configure first.

Change-Id: Ic28dbeabddee6b41af12b977e3fe59e663ee51a1
2018-09-18 15:09:05 +02:00