Create VTY commands and related database functions to get, allocate and
deallocate pseudonymous IMSIs. As this is a proof of concept, the code
to retrieve the next pseudonymous IMSI has no performance optimizations
and might not use a good entropy source (I did not verify).
Related: OS#4476
Change-Id: Ia93ee58d5e03c801eb774b9483a9a4417d031959
Add mslookup client to find remote home HLRs of unknown IMSIs, and
proxy/forward GSUP for those to the right remote HLR instances.
Add remote_hlr.c to manage one GSUP client per remote HLR GSUP address.
Add proxy.c to keep state about remotely handled IMSIs (remote GSUP address,
MSISDN, and probably more in future patches). The mslookup_server that
determines whether a given MSISDN is attached locally now also needs to look in
the proxy record: it is always the osmo-hlr immediately peering for the MSC
that should respond to mslookup service address queries like SIP and SMPP.
(Only gsup.hlr service is always answered by the home HLR.)
Add dgsm.c to set up an mdns mslookup client, ask for IMSI homes, and to decide
which GSUP is handled locally and which needs to go to a remote HLR.
Add full VTY config and VTY tests.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Change-Id: I2fe453553c90e6ee527ed13a13089900efd488aa
Implement the mslookup server's mDNS responder, to actually service remote
mslookup requests:
- VTY mslookup/server config with service names,
- the mslookup_mdns_server listening for mslookup requests,
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Change-Id: I5cae6459090588b4dd292be90a5e8903432669d2
Implement the mslookup server to service remote mslookup requests.
This patch merely adds the logic to answer incoming mslookup requests, an
actual method to receive requests (mDNS) follows in a subsequent patch.
- API to configure service names and addresses for the local site (per MSC).
- determine whether a subscriber is on a local MSC
(checking the local proxy will be added in subsequent patch that adds proxy
capability).
- VTY config follows in a subsequent patch.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Change-Id: Ife4a61d71926d08f310a1aeed9d9f1974f64178b
To be prepared for the future in public API, wrap the new osmo_ipa_name struct
in an enum-type and union called osmo_cni_peer.
During code review it was requested to insert an ability to handle different
kinds of peer id, in order to be able to add a Global Title in the future.
Use the generic osmo_cni_peer only in the publicly visible API. For osmo-hlr
internal code, I intend to postpone implementing this into the future, when a
different peer identification actually gets introduced.
This way we don't need to implement it now in all osmo-hlr code paths (save
time now), but still make all external API users aware that this type may be
extended in the future.
Change-Id: Ide9dcdca283ab989240cfc6e53e9211862a199c5
These are seemingly orthogonal changes in one patch, because they are in fact
sufficiently intertwined that we are not willing to spend the time to separate
them. They are also refactoring changes, unlikely to make sense on their own.
** lu_fsm:
Attempting to make luop.c keep state about incoming GSUP requests made me find
shortcomings in several places:
- since it predates osmo_fsm, it is a state machine that does not strictly
enforce the order of state transitions or the right sequence of incoming
events.
- several places OSMO_ASSERT() on data received from the network.
- modifies the subscriber state before a LU is accepted.
- dead code about canceling a subscriber in a previous VLR. That would be a
good thing to actually do, which should also be trivial now that we record
vlr_name and sgsn_name, but I decided to remove the dead code for now.
To both step up the LU game *and* make it easier for me to integrate
osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now,
ample experience of writing osmo_fsms.
** osmo_gsup_req:
Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs /
communicate with remote MSCs via a proxy:
a) It is important that a response that osmo-hlr generates and that is sent
back to a requesting MSC contains all IEs that are needed to route it back to
the requester. Particularly source_name must become destination_name in the
response to be able to even reach the requesting MSC. Other fields are also
necessary to match, which were so far taken care of in individual numerous code
paths.
b) For some operations, the response to a GSUP request is generated
asynchronously (like Update Location Request -> Response, or taking the
response from an EUSE, or the upcoming proxying to a remote HLR). To be able to
feed a request message's information back into the response, we must thus keep
the request data around. Since struct osmo_gsup_message references a lot of
external data, usually with pointers directly into the received msgb, it is not
so trivial to pass GSUP message data around asynchronously, on its own.
osmo_gsup_req is the combined solution for both a and b: it keeps all data for
a GSUP message by taking ownership of the incoming msgb, and it provides an
explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that
all code paths trivially are definitely responding with the correct IEs set to
match the request's routing (by using osmo_gsup_make_response() recently added
to libosmocore).
Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to
incoming requests received on the GSUP server (above LU code being one of
them).
In fact, the same should be done on the client side. Hence osmo_gsup_req is
implemented in a server/client agnostic way, and is placed in
libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups,
using osmo_gsup_req in the related GSUP client is likely to resolve those
problems without much thinking required beyond making all code paths use it.
libosmo-gsupclient is hence added to osmo-hlr binary's own library
dependencies. It would have been added by the D-GSM proxy routing anyway, we
are just doing it a little sooner.
** cni_peer_id.c / osmo_ipa_name:
We so far handle an IPA unit name as pointer + size, or as just pointer with
implicit talloc size. To ease working with GSUP peer identification data, I
require:
- a non-allocated storage of an IPA Name. It brings the drawback of being
size limited, but our current implementation is anyway only able to handle
MSC and SGSN names of 31 characters (see struct hlr_subscriber).
- a single-argument handle for IPA Name,
- easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying
by simple assignment, a = b.
Hence this patch adds a osmo_ipa_name in cni_peer_id.h and cni_peer_id.c. Heavily
used in LU and osmo_gsup_req.
Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566
Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
mslookup is a key concept in Distributed GSM, which allows querying the current
location of a subscriber in a number of cooperating but independent core
network sites, by arbitrary service names and by MSISDN/IMSI.
Add the abstract mslookup client library. An actual lookup method (besides
mslookup_client_fake.c) is added in a subsequent patch.
For a detailed overview of this and upcoming patches, please see the elaborate
comment at the top of mslookup.c.
Add as separate library, libosmo-mslookup, to allow adding D-GSM capability to
arbitrary client programs.
osmo-hlr will be the only mslookup server implementation, added in a subsequent
patch.
osmo-hlr itself will also use this library and act as an mslookup client, when
requesting the home HLR for locally unknown IMSIs.
Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I83487ab8aad1611eb02e997dafbcb8344da13df1
Apply the same headers structure that we keep in most Osmocom source trees:
Keep noinst_HEADERS in include/osmocom/hlr and include them using
#include <osmocom/hlr/*.h>
The only header kept in src/ is db_bootstrap.h, because it is generated during
build time. If it was built in include/osmocom/hlr, we would need db.o to
depend on db_bootstrap.h in a different subdir, which automake can't do well.
Change-Id: Ic912fe27f545b85443c5fb713d8c3c8aac23c9ad
This change introduces an optional feature that allows to make
SQLite3 use talloc for all internal allocations. This would
facilitate finding memleaks. OsmoHLR needs to be configured
with '--enable-sqlite-talloc'.
full talloc report on 'OsmoHLR' (total 292168 bytes in 449 blocks)
struct osmo_gsup_server contains 162 bytes in 3 blocks (ref 0)
...
struct db_context contains 288407 bytes in 420 blocks (ref 0)
hlr.db contains 7 bytes in 1 blocks (ref 0)
SQLite3 contains 288192 bytes in 418 blocks (ref 0)
db.c:95 contains 48 bytes in 1 blocks (ref 0)
db.c:95 contains 2 bytes in 1 blocks (ref 0)
...
Unfortunately, old SQLite3 versions (such as 3.8.2) run out
of memory when trying to initialize a new database:
DDB ERROR db.c:88 (7) statement aborts at 3: []
DDB ERROR db.c:420 Unable to set Write-Ahead Logging: out of memory
DDB ERROR db.c:88 (7) statement aborts at 3: []
DDB ERROR db.c:238 Unable to prepare SQL statement
'SELECT name FROM sqlite_master WHERE type='table' AND name=?'
...
I've noticed a huge difference in heap usage footprint compared to
generic malloc. At the same time, the recent versions (at least
3.24.0), work just fine.
Change-Id: Icfe67ed0f063b63e6794f9516da3003d01cf20a7
Somehow both 'db_test_SOURCES' and 'db_test_LDADD' ended up in
'src/Makefile.am'. This causes automake / autoconf to complain.
Let's get rid of both useless declarations.
Furthermore, the actual 'db_test_LDADD' in 'tests/Makefile.am'
contained references to the source files from '$(top_srcdir)'.
Most likely, the original intention was to depend on the object
files in '$(top_builddir)'. Let's also fix this.
Change-Id: Ib2e436ed91d9b7551dc5b205329d468c2b0ced04
This is a small program which simply echo's the USSD request message it
gets in a quote back to the sender. Its purpose is to illustrate how
EUSEs can be implemented using libosmo-gsup-client.
Change-Id: I3fb8554ca329cb609c591058254117006f665e73
This imports the code from osmo-msc 6afef893e17bce67e4d4119acd34d480ed03ba77
with minimal changes to make it compile. Symbol renaming ot osmo_
prefix is done separately in a follow-up patch to have a as-clean-as-possible
import first.
Requires: libosmocore.git Change-Id Ie36729996abd30b84d1c30a09f62ebc6a9794950
Change-Id: Ief50054ad135551625b684ed8a0486f7af0b2940
Encode an Insert Subscr Data with is_ps == true to trigger the encoding bug
described in OS#3231, i.e. show that it is fixed.
Move osmo_gsup_addr_send() to a separate .c file, so that it can be overridden
in the regression test to just dump the msgb instead.
I used this test to reproduce issue OS#3231, and now that it's here we might as
well keep it, and possibly expand on it in the future.
Related: OS#3231
Change-Id: Id1453351758f3e1a9ff03bd99fefaf51886e77da
Move macro copy_sqlite3_text_to_buf() to db.h, so it can be used in
hlr_db_tool.c.
Add _dbd_decode_binary() from libdbi to avoid depending on the entire libdbi
just for KI BLOB decoding. Add it in a separate file, copying its own license,
the lGPL.
Offer commandline option "import-nitb-db" to read in an old osmo-nitb database
and copy subscriber IMSIs and 2G auth data to OsmoHLR db format.
Anticipate future command line options like "import-csv", so keep the code
generalized.
Change-Id: I0dfa6ec033dd93161c1adc2ce1637195fe5b7a63
If a database file is missing, osmo-hlr creates it, as is the default sqlite3
API behavior -- before this patch, that db file is created, but lacks useful
tables. Actually also create initial tables in it, as osmo-nitb did.
In effect, the 'vty-test' target in tests/Makefile.am no longer needs to create
a database manually. (The 'ctrl-test' still does, because it also wants to add
subscriber data on top of the bare tables.)
Note: it could be desirable to bail if the desired database file does not
exist. That is however a different semantic from this patch; this is not
changing the fact that a db file is created, this just creates a usable one.
Note: I am about to add osmo-hlr-db-tool to do database migration from
osmo-nitb. For that, it is desirable to bootstrap a usable database, which is
the core reason for this patch.
Don't plainly duplicate hlr.sql to .c, but create db_bootstrap.h as a
BUILT_SOURCE from reading in sql/hlr.sql and mangling via sed to a list of SQL
statement strings. On each db_open(), run this bootstrap sequence.
In sql/hlr.sql, these tweaks are necessary:
* Add 'IF NOT EXISTS' to 'CREATE TABLE', so that the bootstrap sequence can be
run on an already bootstrapped db.
* Drop the final comment at the bottom, which ended up being an empty SQL
statement and causing sqlite3 API errors, seemed to have no purpose anyway.
Note: by composing the statement strings as multiline and including the SQL
comments, sqlite3 actually retains the comments contained in table definitions
and prints them back during 'sqlite3 hlr.db .dump'.
Change-Id: If77dbbfe1af3e66aaec91cb6295b687f37678636
Implement VTY commands for subscriber manipulation:
- create / delete subscriber
- modify MSISDN
- add/edit/remove 2G and 3G authentication data
- show by IMSI, MSISDN or DB ID.
(enable/disable CS/PS and purge/unpurge to follow later.)
Implement VTY unit tests for the new commands using new
osmo_verify_transcript_vty.py from osmo-python-tests.
Depends: libosmocore I1e94f5b0717b947d2a7a7d36bacdf04a75cb3522
osmo-python-tests Id47331009910e651372b9c9c76e12f2e8964cc2c
Change-Id: I42b3b70a0439a8f2e4964d7cc31e593c1f0d7537
Introduce g_hlr of type 'struct hlr' which holds pointers to all
globally accessible variables.
Change-Id: I275d3d54482f696e3378606b2406c7e0ad939e0f
Related: OS#1645
Create luop.(c|h) and move lu_operation and corresponding TX
functions there to facilitate re-use in upcoming control interface.
Change-Id: Ic55a45d56b37be2ba43d96f7da2af43b46af9813
Related: OS#1645
Add config file, mainly for logging control.
Open VTY on the OMSO_VTY_PORT_HLR added to libosmocore in
commit 92fa18e6b800a27aa064a5fb8321cddd7383ae20
aka change-id I08cb52d9399a27e6876e45da36f434708c4fddef.
Add hlr_vty.h/c for standard VTY setup.
Add -c option to pass config file.
Add --version option.
Change-Id: Iedb884345a597371a337b0c67eb6013b7d5d1ce1
Add configure.ac and Makefile.ams to build with
autoreconf && ./configure && make
like most other Osmocom projects.
Add jenkins.sh for a gerrit build job to verify patches.
Change-Id: I6b4419dd519f3d0a75235d0c22bf899f075347a3
Oliver Smith
Neels Hofmeyr
Harald Welte
Pau Espin
Neels Hofmeyr