Checking for existence of a subscriber and seeing that there is none is not
inherently an error. However, osmo-hlr currently logs on all occasions:
DAUC ERROR Cannot read subscriber from db: MSISDN='1001': No such subscriber
This spams the ERROR log level. Particularly when a D-GSM setup does subscriber
existence checks for every incoming mslookup request, that potentially creates
constant ERROR logging.
The "No such subscriber" part comes from db_sel(), which might also return an
sqlite3_errmsg(). We still want those sqlite3_errmsg()es in the ERROR log.
Hence print an ERROR log only if db_sel() returns an rc != -ENOENT.
Change-Id: I5044e9b4519b948edc4e451cef0f7830d315619b
D-GSM will store in the HLR DB whether a locally connected MSC has attached the
subscriber (last_lu_seen[_ps]), or whether the attach happened via a GSUP proxy
from a different site.
Add columns for this separately in this patch.
Change-Id: I98c7b3870559ede84adf56e4bf111f53c7487745
To be prepared for the future in public API, wrap the new osmo_ipa_name struct
in an enum-type and union called osmo_cni_peer.
During code review it was requested to insert an ability to handle different
kinds of peer id, in order to be able to add a Global Title in the future.
Use the generic osmo_cni_peer only in the publicly visible API. For osmo-hlr
internal code, I intend to postpone implementing this into the future, when a
different peer identification actually gets introduced.
This way we don't need to implement it now in all osmo-hlr code paths (save
time now), but still make all external API users aware that this type may be
extended in the future.
Change-Id: Ide9dcdca283ab989240cfc6e53e9211862a199c5
These are seemingly orthogonal changes in one patch, because they are in fact
sufficiently intertwined that we are not willing to spend the time to separate
them. They are also refactoring changes, unlikely to make sense on their own.
** lu_fsm:
Attempting to make luop.c keep state about incoming GSUP requests made me find
shortcomings in several places:
- since it predates osmo_fsm, it is a state machine that does not strictly
enforce the order of state transitions or the right sequence of incoming
events.
- several places OSMO_ASSERT() on data received from the network.
- modifies the subscriber state before a LU is accepted.
- dead code about canceling a subscriber in a previous VLR. That would be a
good thing to actually do, which should also be trivial now that we record
vlr_name and sgsn_name, but I decided to remove the dead code for now.
To both step up the LU game *and* make it easier for me to integrate
osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now,
ample experience of writing osmo_fsms.
** osmo_gsup_req:
Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs /
communicate with remote MSCs via a proxy:
a) It is important that a response that osmo-hlr generates and that is sent
back to a requesting MSC contains all IEs that are needed to route it back to
the requester. Particularly source_name must become destination_name in the
response to be able to even reach the requesting MSC. Other fields are also
necessary to match, which were so far taken care of in individual numerous code
paths.
b) For some operations, the response to a GSUP request is generated
asynchronously (like Update Location Request -> Response, or taking the
response from an EUSE, or the upcoming proxying to a remote HLR). To be able to
feed a request message's information back into the response, we must thus keep
the request data around. Since struct osmo_gsup_message references a lot of
external data, usually with pointers directly into the received msgb, it is not
so trivial to pass GSUP message data around asynchronously, on its own.
osmo_gsup_req is the combined solution for both a and b: it keeps all data for
a GSUP message by taking ownership of the incoming msgb, and it provides an
explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that
all code paths trivially are definitely responding with the correct IEs set to
match the request's routing (by using osmo_gsup_make_response() recently added
to libosmocore).
Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to
incoming requests received on the GSUP server (above LU code being one of
them).
In fact, the same should be done on the client side. Hence osmo_gsup_req is
implemented in a server/client agnostic way, and is placed in
libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups,
using osmo_gsup_req in the related GSUP client is likely to resolve those
problems without much thinking required beyond making all code paths use it.
libosmo-gsupclient is hence added to osmo-hlr binary's own library
dependencies. It would have been added by the D-GSM proxy routing anyway, we
are just doing it a little sooner.
** cni_peer_id.c / osmo_ipa_name:
We so far handle an IPA unit name as pointer + size, or as just pointer with
implicit talloc size. To ease working with GSUP peer identification data, I
require:
- a non-allocated storage of an IPA Name. It brings the drawback of being
size limited, but our current implementation is anyway only able to handle
MSC and SGSN names of 31 characters (see struct hlr_subscriber).
- a single-argument handle for IPA Name,
- easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying
by simple assignment, a = b.
Hence this patch adds a osmo_ipa_name in cni_peer_id.h and cni_peer_id.c. Heavily
used in LU and osmo_gsup_req.
Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566
Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
Location Updating procedures from both CS and PS overwrite the same
last_lu_seen field of a subscriber. For upcoming D-GSM it will be important to
distinguish those, because only CS attaches qualify for MSISDN lookup.
Add column last_lu_seen_ps, and upon PS LU, do not overwrite last_lu_seen, so
that last_lu_seen now only reflects CS LU.
In the VTY, dump both LU dates distinctively.
Change-Id: Id7fc50567211a0870ac0524f6dee94d4513781ba
Apply the same headers structure that we keep in most Osmocom source trees:
Keep noinst_HEADERS in include/osmocom/hlr and include them using
#include <osmocom/hlr/*.h>
The only header kept in src/ is db_bootstrap.h, because it is generated during
build time. If it was built in include/osmocom/hlr, we would need db.o to
depend on db_bootstrap.h in a different subdir, which automake can't do well.
Change-Id: Ic912fe27f545b85443c5fb713d8c3c8aac23c9ad
The last LU time gets read from the database as string, parsed with
strptime to "struct tm", and then gets converted to time_t with mktime.
A recent behavior change in glibc's mktime implementation unconvered,
that we don't have tm.tm_isdst (daylight saving time) set properly. As
"struct tm" was not initialized, and strptime did not write to tm_isdst,
it was set to a random value. When it was not 0, db_test failed on UTC
systems with a more recent glibc (e.g. Ubuntu 19.04).
Fix this by zero-initializing "struct tm" and remove the previous
workaround that made db_test pass on UTC systems.
Related: OS#4026
Change-Id: Iebbbe42fc5cd43324206d9433ede67b39251389c
Check if a subscriber exists without generating an error log entry if
it does not. This is cheaper than db_subscr_get_by_msisdn(), as it
does not fetch the subscriber entry.
subscriber-create-on-demand will use this function to generate
a random unique MSISDN for new subscribers.
Related: OS#2542
Change-Id: Ibfbc408c966197682ba2b12d166ade4bfeb7abc2
Check if a subscriber exists without generating an error log entry if
it does not. This is cheaper than db_subscr_get_by_imsi(), as it does
not fetch the subscriber entry. subscriber-create-on-demand will use
this function.
Related: OS#2542
Change-Id: I63818c0dd4fd22b41dadeeba2a07a651b5454c54
Allow creating new subscribers without giving them access to CS or PS.
This will be used by the create-subscriber-on-demand feature.
Related: OS#2542
Change-Id: I1a6dd85387723dab5487c53b33d2d9ec6d05d006
Extend the database scheme, add imei to the hlr_subscriber struct and
create db_subscr_update_imei_by_imsi() and db_subscr_get_by_imei(). The
new functions are used in db_test, and in follow-up commits [1], [2].
Upgrade DB schema to version 2. SQLite can only insert new columns at
the end of the table, so this happens when upgrading the database. In
new databases, the column is placed after the IMEISV column (where it
makes more sense in my opinion). This should not have any effect, as
we never rely on the order of the columns in the tables.
Follow-up commit [1] will make use of this column to save the IMEI as
received from the MSC/VLR with the Check-IMEI Procedure. It was
decided to use Check-IMEI instead of the recent Automatic Device
Detection Procedure (which would send the IMEISV) in OS#3733, because
with Check-IMEI we don't need to rely on very recent releases of the
specification.
[1] change-id I09274ecbed64224f7ae305e09ede773931da2a57
"Optionally store IMEI in subscriber table"
[2] change-id I1af7b573ca2a1cb22497052665012d9c1acf3b30
"VTY: integrate IMEI"
Depends: Id2d2a3a93b033bafc74c62e15297034bf4aafe61 (libosmocore)
Related: OS#2541
Change-Id: If232c80bea35d5c6864b889ae92d477eeaa3f45d
Read the subscriber's last location update timestamp from the
database and display it in the output of 'show subscriber'.
For example:
OsmoHLR> show subscriber id 1
ID: 1
IMSI: 123456789000000
MSISDN: 543210123456789
VLR number: 712
SGSN number: 5952
last LU seen: Fri Dec 7 11:30:51 2018 UTC
While the database stores the timestamp as a string, we
convert the timestamp into time_t for internal use.
This allows for flexible potential use of the timestamp
in contexts other than the VTY in the future.
The timestamp displayed in the VTY is created with ctime_r(3).
It does not match the format of the raw string in the database:
sqlite> select id,last_lu_seen from subscriber;
1|2018-12-07 11:30:51
Related: OS#2838
Change-Id: Ie180c434f02ffec0d4b2f651a73258a8126b2e1a
Avoid string concatenations without interleaving whitespace.
Some compilers don't like "foo""bar", they only like "foo" "bar".
Requested by: Pau
https://gerrit.osmocom.org/c/osmo-hlr/+/12121/5/src/db_hlr.c#637
Change-Id: Ic7a81114f9afbefcbd62d434720854cfdd4a2dd9
Timestamps are stored in the HLR DB in the new 'last_lu_seen' column
of the 'subscriber' table, in UTC and in granularity of seconds.
At present, osmo-hlr only records these timestamps but otherwise
makes no use of them. Because the timestamps are stored in a
human-readable form, they may already provide value to external
processes which need this information. For example:
sqlite> select imsi,last_lu_seen from subscriber;
901990000000001|2018-12-04 14:17:12
I didn't bother adding additional tests because the code added
with this commit is already being exercised by several calls
to db_subscr_lu() in db_test.c.
This change requires a HLR DB schema update. Existing databases
won't be upgraded automatically. However, osmo-hlr will refuse
to operate with databases which are not upgraded.
Change-Id: Ibeb49d45aec18451a260a6654b8c51b8fc3bec50
Related: OS#2838
Add the first "official" way to remove the MSISDN from a subscriber entry, to
go back to 'MSISDN: none' like just after 'subscriber create'.
Add VTY command 'subscriber <ID> update msisdn none' to drop the MSISDN from
the subscriber. (Like 'subscriber <ID> update aud3g none')
Add DB_STMT_DELETE_MSISDN_BY_IMSI.
In db_subscr_update_msisdn_by_imsi(), allow passing a NULL msisdn, and if NULL,
call above delete SQL statement.
Change-Id: I15419105ea461137776adb92d384d8985210c90e
Move macro copy_sqlite3_text_to_buf() to db.h, so it can be used in
hlr_db_tool.c.
Add _dbd_decode_binary() from libdbi to avoid depending on the entire libdbi
just for KI BLOB decoding. Add it in a separate file, copying its own license,
the lGPL.
Offer commandline option "import-nitb-db" to read in an old osmo-nitb database
and copy subscriber IMSIs and 2G auth data to OsmoHLR db format.
Anticipate future command line options like "import-csv", so keep the code
generalized.
Change-Id: I0dfa6ec033dd93161c1adc2ce1637195fe5b7a63
Rename SL3_TXT to more accurate copy_sqlite3_text_to_buf(), and use
osmo_strlcpy() instead of essentially dup'ing it.
The macro will also be used by hlr_db_tool.c in upcoming patch. This patch
prepares for a move to db.h.
Change-Id: I1dadeddddcfe0109195c09c0e706201b0df009cc
db_subscr_get_by_*() failed to clear the out-param struct, meaning that data
could remain in a struct even though it is not present in the database. Always
zero out the struct before writing to it.
Adjust the db_test to catch this error by writing "-invalid-data-" to each
struct before running db get functions.
Change-Id: I038bd437452c87841d709fcdd5ac30ab1356b2db
Add ability to add and remove auc_2g and auc_3g table rows with
db_subscr_update_aud_by_id().
In db_subscr_delete_by_id(), make sure that when deleting a subscriber, also
all auth data associated with that user ID is removed as well. A newly created
subscriber must not obtain the same auth tokens just by getting the same id.
Depends: libosmocore Idf75946eb0a84e145adad13fc7c78bb7a267aa0a
Change-Id: Icb11b5e059fb920447a9aa414db1819a0c020529
Use named parameters in the SQL statements.
Use db_bind_* functions to drop some code dup.
Adopt error handling (rc and logging) to match the other db functions: return
-ENOENT for unknown subscriber, -EIO for SQL failures.
Change-Id: Iad49d29b90a708c6cf55bfb3bcc02d9e29001a15
Use named parameters in the SQL statement.
Use db_bind_* functions to drop some code dup.
Use explicit subscriber id arg instead of subscriber struct.
Match return values and error logging to other db functions.
Change-Id: I35665e84ddbe54a6f218b24033df969ad2e669a0
Allow to set nam_ps and nam_cs from this same function, by adding the is_ps
arg.
Combine both NAM_PS stmts to DB_STMT_UPD_NAM_PS_BY_IMSI, add another such stmt
for CS. Use named parameters instead of parameter indexes.
Improve error return values as well as error logging to clearly indicate
whether the operation could not find the requested IMSI, or other errors
occured.
Adjust the single caller.
This prepares for upcoming VTY and possibly CTRL commands, and the error
handling introduced here has been or will be adopted by other functions in
previous or subsequent patches.
Change-Id: I6e70e15228f5bb10bee6758ae5dc9687d65839bd
Factor out the selected SQL columns as SEL_COLUMNS macro, so that each of the
new DB_STMTs will select identical columns: the old DB_STMT_SEL_BY_IMSI as well
as the new DB_STMT_SEL_BY_MSISDN and DB_STMT_SEL_BY_ID.
Add the new functions db_subscr_get_by_msisdn() and db_subscr_get_by_id() and
factor out common parts with db_subscr_get_by_imsi() to static db_sel().
Change-Id: I6d0ddd1b7e3f6b180b4b1b2663c5725d2a4a9428
These will be needed by VTY commands to create, delete and modify subscribers.
Auth data editing will follow in another patch.
The FIXME "also remove authentication data from auc_2g and auc_3g" will get
fixed in change-id Icb11b5e059fb920447a9aa414db1819a0c020529.
Change-Id: I725273d36234331093e7fff7d5f12f6be6ab2623
Before, it looked like the nul term was within the if () body (despite no body
being present).
While at it, also remove one of the two tabs of indenting and put the opening
'do {' on its own line.
Change-Id: I8d03433b6fba90f4e46814bc54636bc3a444cc46
In LOGHLR and LOGAUC, log IMSI='<imsi>' instead of just <imsi>:
In the log, it is not always obvious to the reader that the printed number
refers to an IMSI (vs. an MSISDN or in the future an IMEI).
In db_get_auth_data(), log "No such subscriber" instead of just "Unknown", to
clarify what exactly is meant.
Change-Id: I2ec8ab5e67d4e95083f6e39232fc91ebaa080cb8
There are more uses for a generalized db_bind_text(), and in an upcoming patch
there will be similar functions like db_bind_int().
Also, add argument param_name, optionally indicating a named SQL parameter to
bind to, which will be used in subsequent patches. So far, all callers pass
NULL to yield previous db_bind_imsi() behavior of binding to the first param.
Change-Id: I87bc46a23a724677e8319d6a4b032976b7ba9394
There will be more additions, _by_msisdn() and _by_id(), to serve the upcoming
VTY commands, to allow flexibly selecting subscribers as in the old OsmoNITB.
Change-Id: I32fa676ccc5c10eba834c4390c8a42476b9c1961
There are upcoming additions, and some seem too general without a proper common
prefix in the identifiers, like 'CREATE'.
Change-Id: I51b677db31a1ebbbc45dc7925074de7493fbde1f
This allows to check for subscriber's presence in DB without the need to
bother with unused structure allocation.
While at it also call to db_remove_reset() and return explicitly instead
of using goto to make it a bit easier to follow the code.
Change-Id: I83b0f4a5dacb97614721690ef55bc1311624a58e
* move common cleanup code into separate function
* add helper function for IMSI binding
* use errno.h instead of numbers
Change-Id: Iec81b56ab1ccc948807854a3947b04355a555c10
Using this procedure, the VLR/SGSN can set the cs/ps purged
flag for the subscriber. We might not even need to store this
persistent in the database according to spec, but let's do it anyway, at
least until it turns out to be a performance issue.
We also introduce a 'gsup_router' which enables us to route
a transaction to a given VLR. It works based on the SERIAL attribute
communicated at time of the IPA multiplex setup as part of the CCM
sub-protocol.
Neels Hofmeyr
Oliver Smith
Harald Welte