Commit Graph

27 Commits (cf535dbcbd438c8fb9bd9d64412532c71ae2c609)

Author SHA1 Message Date
Neels Hofmeyr 3f9d1977df db v6: determine 3G AUC IND from VLR name
Each VLR requesting auth tuples should use a distinct IND pool for 3G
auth.  So far we tied the IND to the GSUP peer connection; MSC and SGSN
were always distinct GSUP peers, they ended up using distinct INDs.

However, we have implemented a GSUP proxy, so that, in a distributed
setup, a remotely roaming subscriber has only one direct GSUP peer
proxying for both remote MSC and SGSN. That means as soon as a
subscriber roams to a different site, we would use the GSUP proxy name
to determine the IND instead of the separate MSC and SGSN. The site's
MSC and SGSN appear as the same client, get the same IND bucket, waste
SQNs rapidly and cause auth tuple generation load.

So instead of using the local client as IND, persistently keep a list of
VLR names and assign a different IND to each. Use the
gsup_req->source_name as indicator, which reflects the actual remote
VLR's name (remote MSC or SGSN).

Persist the site <-> IND assignments in the database.

Add an IND test to db_test.c

There was an earlier patch version that separated the IND pools by
cn_domain, but it turned out to add complex semantics, while only
solving one aspect of the "adjacent VLR" problem. We need a solution not
only for CS vs PS, but also for 2,3G vs 4G, and for sites that are
physically adjacent to each other. This patch version does not offer any
automatic solution for that -- as soon as more than 2^IND_bitlen
(usually 32) VLRs show up, it is the responsibility of the admin to
ensure the 'ind' table in the hlr.db does not have unfortunate IND
assignments. So far no VTY commands exist for that, they may be added in
the future.

Related: OS#4319
Change-Id: I6f0a6bbef3a27507605c3b4a0e1a89bdfd468374
2021-07-19 09:01:06 +00:00
Neels Hofmeyr 04c2375b38 db v5: prep for D-GSM: add vlr_via_proxy and sgsn_via_proxy
D-GSM will store in the HLR DB whether a locally connected MSC has attached the
subscriber (last_lu_seen[_ps]), or whether the attach happened via a GSUP proxy
from a different site.

Add columns for this separately in this patch.

Change-Id: I98c7b3870559ede84adf56e4bf111f53c7487745
2020-04-30 19:22:24 +02:00
Neels Hofmeyr 07e1602d2d db v4: add column last_lu_seen_ps
Location Updating procedures from both CS and PS overwrite the same
last_lu_seen field of a subscriber. For upcoming D-GSM it will be important to
distinguish those, because only CS attaches qualify for MSISDN lookup.

Add column last_lu_seen_ps, and upon PS LU, do not overwrite last_lu_seen, so
that last_lu_seen now only reflects CS LU.

In the VTY, dump both LU dates distinctively.

Change-Id: Id7fc50567211a0870ac0524f6dee94d4513781ba
2019-11-27 03:19:06 +01:00
Neels Hofmeyr a8045daeef hlr db schema 3: hlr_number -> msc_number
The osmo-hlr DB schema indicates a hlr_number column and references it as 3GPP
TS 23.008 chapter 2.4.6. However, chapter 2.4.6 refers to the "MSC number",
while the "HLR number" is chapter 2.4.7.

Taking a closer look, 2.4.6 says "The MSC number is [...] stored in the HLR",
while 2.4.7 says "The HLR number may be stored in the VLR". As quite obvious,
the HLR does not store the HLR number. This was a typo from the start.

The osmo-hlr code base so far does not use the hlr_number column at all, so we
get away with renaming the column without any effects on the code base.
However, let's rather make this a new schema version to be safe.

Change-Id: I527e8627b24b79f3e9eec32675c7f5a3a6d25440
2019-11-12 16:23:48 +00:00
Neels Hofmeyr f8ad67e7fc hlr.sql: move comment
Move a comment for ind_bitlen column to a separate line, so that it doesn't
show in PRAGMA_TABLE_INFO('subscriber').

An upcoming patch introduces db_upgrade_test, which dumps a sorted db schema.
In newer sqlite3 versions, a comment following a 'DEFAULT' keyword actually
shows up in the PRAGMA_TABLE_INFO() results (on my machine), but older versions
(on the build slaves) drop that comment. The ind_bitlen column is the only one
producing this odd side effect, because it is the last column and has no comma
between 'DEFAULT' and the comment.

The easiest way to get matching results across sqlite3 client versions is to
move the comment to above ind_bitlen instead of having it on the same line.
(Adding a comma doesn't work.)

Change-Id: Id66ad68dd3f22d533fc3a428223ea6ad0282bde0
2019-10-31 19:50:28 +01:00
Oliver Smith 81db389fd4 Add IMEI column to subscriber table
Extend the database scheme, add imei to the hlr_subscriber struct and
create db_subscr_update_imei_by_imsi() and db_subscr_get_by_imei(). The
new functions are used in db_test, and in follow-up commits [1], [2].

Upgrade DB schema to version 2. SQLite can only insert new columns at
the end of the table, so this happens when upgrading the database. In
new databases, the column is placed after the IMEISV column (where it
makes more sense in my opinion). This should not have any effect, as
we never rely on the order of the columns in the tables.

Follow-up commit [1] will make use of this column to save the IMEI as
received from the MSC/VLR with the Check-IMEI Procedure. It was
decided to use Check-IMEI instead of the recent Automatic Device
Detection Procedure (which would send the IMEISV) in OS#3733, because
with Check-IMEI we don't need to rely on very recent releases of the

[1] change-id I09274ecbed64224f7ae305e09ede773931da2a57
    "Optionally store IMEI in subscriber table"
[2] change-id I1af7b573ca2a1cb22497052665012d9c1acf3b30
    "VTY: integrate IMEI"

Depends: Id2d2a3a93b033bafc74c62e15297034bf4aafe61 (libosmocore)
Related: OS#2541
Change-Id: If232c80bea35d5c6864b889ae92d477eeaa3f45d
2019-01-24 15:29:08 +00:00
Stefan Sperling 638ba8cc04 store a timestamp of the last location update seen from a subscriber
Timestamps are stored in the HLR DB in the new 'last_lu_seen' column
of the 'subscriber' table, in UTC and in granularity of seconds.

At present, osmo-hlr only records these timestamps but otherwise
makes no use of them. Because the timestamps are stored in a
human-readable form, they may already provide value to external
processes which need this information. For example:

  sqlite> select imsi,last_lu_seen from subscriber;
  901990000000001|2018-12-04 14:17:12

I didn't bother adding additional tests because the code added
with this commit is already being exercised by several calls
to db_subscr_lu() in db_test.c.

This change requires a HLR DB schema update. Existing databases
won't be upgraded automatically. However, osmo-hlr will refuse
to operate with databases which are not upgraded.

Change-Id: Ibeb49d45aec18451a260a6654b8c51b8fc3bec50
Related: OS#2838
2018-12-07 11:50:06 +01:00
Stefan Sperling 8f3a7cce80 add database schema versioning to the HLR database
Make use of pragma user_version to store our database schema version.
The present schema is now identitifed as 'version 0', which is also
the default value for databases on which we never ran the statement
'pragma user_version' before.

Only bootstrap the database if it hasn't been bootstrapped yet.
Previously, bootstrap SQL statements ran every time osmo-hlr
opened the database, and any errors were being ignored in SQL.
Instead, we now first run a query which checks whether tables
already exist, and only create them if necessary.
This change will allow future schema updates to work properly.

Prepare for future schema upgrades by adding a new command-line
option which enables upgrades. This option defaults to 'false'
in order to avoid accidental upgrades.

Change-Id: I8aeaa9a404b622657cbc7138106f38aa6ad8d01b
Related: OS#2838
2018-12-04 14:13:28 +01:00
Pau Espin 6b274b95fc sql/Makefile: Create empty /var/lib/osmocom directory at install time
Otherwise osmo-hlr is unable to start correctly.

Change-Id: I1233fc9b3dc685561f79a34e1c32c459dc1aa685
2018-07-02 17:47:24 +02:00
Pau Espin edca4f88a6 sql/Makefile: Install sql files under doc/.../sql subdir
Change-Id: I1c9008d4692412a0cfe39d05216fc9c857cf1e8a
2018-07-02 17:44:35 +02:00
Pau Espin b9c1028cb0 sql/Makefile: Install hlr_data.sql as example together with hlr.sql
Change-Id: Id4a12252b309f03bb393fa26612c305744e14403
2018-07-02 17:27:44 +02:00
Martin Hauke 0b8f054b5f sql/ Make docsdir completely configurable
$(docdir) defaults to $(datadir)/doc/osmo-hlr

Change-Id: I77fa16d0edcf88a8e120921504cd088328077836
2018-06-24 17:14:04 +00:00
Neels Hofmeyr 88c91f6fb2 cosmetic: sql/hlr.sql: move comments
By moving the comments inside the table row definitions, they are dumped back
during 'sqlite3 hlr.db .dump'. When they are between SQL statements like before
this patch, the comments are lost.

Tweak wording.

Change-Id: I280c2e2d3e9b7f1dc632722724d9e1c54d041820
2017-10-28 16:49:33 +00:00
Neels Hofmeyr 7750d2cedc automatically create db tables on osmo-hlr invocation
If a database file is missing, osmo-hlr creates it, as is the default sqlite3
API behavior -- before this patch, that db file is created, but lacks useful
tables. Actually also create initial tables in it, as osmo-nitb did.

In effect, the 'vty-test' target in tests/ no longer needs to create
a database manually. (The 'ctrl-test' still does, because it also wants to add
subscriber data on top of the bare tables.)

Note: it could be desirable to bail if the desired database file does not
exist. That is however a different semantic from this patch; this is not
changing the fact that a db file is created, this just creates a usable one.

Note: I am about to add osmo-hlr-db-tool to do database migration from
osmo-nitb. For that, it is desirable to bootstrap a usable database, which is
the core reason for this patch.

Don't plainly duplicate hlr.sql to .c, but create db_bootstrap.h as a
BUILT_SOURCE from reading in sql/hlr.sql and mangling via sed to a list of SQL
statement strings. On each db_open(), run this bootstrap sequence.

In sql/hlr.sql, these tweaks are necessary:
* Add 'IF NOT EXISTS' to 'CREATE TABLE', so that the bootstrap sequence can be
  run on an already bootstrapped db.
* Drop the final comment at the bottom, which ended up being an empty SQL
  statement and causing sqlite3 API errors, seemed to have no purpose anyway.

Note: by composing the statement strings as multiline and including the SQL
comments, sqlite3 actually retains the comments contained in table definitions
and prints them back during 'sqlite3 hlr.db .dump'.

Change-Id: If77dbbfe1af3e66aaec91cb6295b687f37678636
2017-10-28 16:49:33 +00:00
Daniel Willmann 69f3860d28 hlr_data.sql: Insert ki and opc instead of op to example data
It depends on the cards whether you have op or opc, but the most cards in use
for 3G are using the opc. Change the example to reflect that.

Change-Id: I8f6051ea9b285ff6261bfe346cfc29d1167921f5
2017-06-03 14:55:56 +00:00
Neels Hofmeyr 1790c8246a install hlr.sql in prefix/doc/osmo-hlr/
In particular I need this to start a fresh osmo-hlr instance on the
osmo-gsm-tester. Might also come in handy during packaging?

Change-Id: I08e48375814ab93691892299d34909c6d0bf12a2
2017-05-22 19:52:26 +02:00
Neels Hofmeyr cab2fcd5b5 UMTS AKA: implement SQN increment according to SEQ and IND
Add ind_bitlen column to auc_3g to record each USIM's IND size according to
3GPP TS 33.102 -- default is 5 bits, as suggested by the spec.

Introduce auc_3g_ind to each connecting GSUP client to use as IND index for
generating auth tuples sent to this client.

With osmo_gsup_server_add_conn(), implement a scheme where clients receive
fixed auc_3g_ind indexes based on the order in which they connect; each new
connection takes the lowest unused auc_3g_ind, so in case one of the clients
restarts, it will most likely receive the same auc_3g_ind, and if one client
disconnects, no other clients' auc_3g_ind are affected.

Add gsup_server_test.c to test the auc_3g_ind index distribution scheme.

Depends: libosmocore I4eac5be0c0b2cede04464c4c3a0873102d952453 for llist_first
Related: OS#1969
Change-Id: If4501ed4ff8e923fa6fe8b80c44c5ad647a8ed60
2017-03-16 05:51:11 +01:00
Neels Hofmeyr cc785f0c43 sql: add unique constraints to IMSI and MSISDN
Todo for later: table subscriber_multi_msisdn possibly allows duplicating the
MSISDN, so we should drop this table or have all MSISDNs in one table separate
from 'subscriber'.

Change-Id: I5737106a232e416d67a10634e6270a7a89cf1b05
2017-02-01 19:13:27 +00:00
Neels Hofmeyr 8089d51f74 comment: sql: describe auc_2g and auc_3g columns
Change-Id: Ie4edc69ff11a83a4c0f79097f43a2cb206dfe405
2017-02-01 13:58:50 +01:00
Neels Hofmeyr d71dbec7b6 sql: auc_3g: set sqn NOT NULL DEFAULT 0
Change-Id: Ibb765f30295b441e563bb0e06ed39987f79a60d6
2017-02-01 13:58:50 +01:00
Neels Hofmeyr 862f1dc4fd cosmetic: sql: indicate VARCHAR size of key columns as 32
Notably this has no functional effect (according to ), but it can't hurt to indicate intent.

Change-Id: I2b0f9369318085c1482c6d2d8db56699466bfbf3
2017-02-01 13:58:50 +01:00
Neels Hofmeyr 24537b95bd sql: fix 3g_auc's column K data type
K is the SIM card's 128bit secret key, so the type should be VARCHAR like the
other key columns. The db code already reads the column as text and parses as
hex, so a VARCHAR column matches that.

Change-Id: Iaa8d33e303760bd15dcb7dc8bb8b9b24bf6c8f14
2017-02-01 13:58:50 +01:00
Neels Hofmeyr 2116ce2471 hlr.sql: typo in comment
Change-Id: I8f16944f966bd40540d5b5396873b873685c18e9
2017-01-19 15:50:59 +01:00
Neels Hofmeyr 40d8b01dea build with autoconf/automake, add script
Add and Makefile.ams to build with
  autoreconf && ./configure && make
like most other Osmocom projects.

Add for a gerrit build job to verify patches.

Change-Id: I6b4419dd519f3d0a75235d0c22bf899f075347a3
2016-12-11 01:07:26 +01:00
Harald Welte b18f0e04f3 implement PURGE-MS from VLR/SGSN to HLR
Using this procedure, the VLR/SGSN can set the cs/ps purged
flag for the subscriber.  We might not even need to store this
persistent in the database according to spec, but let's do it anyway, at
least until it turns out to be a performance issue.
2016-05-05 21:03:03 +02:00
Harald Welte 6122915304 hlr.sql: Don't use TRUE/FALSE as DEFAULT, they will be strings :(
SQLite has this crappy type system where you can put a string into
a boolean or integer column, which of course will break once you try to
read it as bool..

Also, add an index by IMSI to the SUBSCRIBER table.
2016-05-03 18:48:12 +02:00
Harald Welte e72cf55347 initial import of osmo-gsup-hlr code so far 2016-04-28 07:18:49 +02:00