osmo-hlr/include/osmocom/hlr/db.h

204 lines
6.6 KiB
C
Raw Normal View History

#pragma once
#include <stdbool.h>
#include <sqlite3.h>
#include <osmocom/gsupclient/cni_peer_id.h>
db v6: determine 3G AUC IND from VLR name Each VLR requesting auth tuples should use a distinct IND pool for 3G auth. So far we tied the IND to the GSUP peer connection; MSC and SGSN were always distinct GSUP peers, they ended up using distinct INDs. However, we have implemented a GSUP proxy, so that, in a distributed setup, a remotely roaming subscriber has only one direct GSUP peer proxying for both remote MSC and SGSN. That means as soon as a subscriber roams to a different site, we would use the GSUP proxy name to determine the IND instead of the separate MSC and SGSN. The site's MSC and SGSN appear as the same client, get the same IND bucket, waste SQNs rapidly and cause auth tuple generation load. So instead of using the local client as IND, persistently keep a list of VLR names and assign a different IND to each. Use the gsup_req->source_name as indicator, which reflects the actual remote VLR's name (remote MSC or SGSN). Persist the site <-> IND assignments in the database. Add an IND test to db_test.c There was an earlier patch version that separated the IND pools by cn_domain, but it turned out to add complex semantics, while only solving one aspect of the "adjacent VLR" problem. We need a solution not only for CS vs PS, but also for 2,3G vs 4G, and for sites that are physically adjacent to each other. This patch version does not offer any automatic solution for that -- as soon as more than 2^IND_bitlen (usually 32) VLRs show up, it is the responsibility of the admin to ensure the 'ind' table in the hlr.db does not have unfortunate IND assignments. So far no VTY commands exist for that, they may be added in the future. Related: OS#4319 Change-Id: I6f0a6bbef3a27507605c3b4a0e1a89bdfd468374
2019-12-12 03:04:53 +00:00
#include <osmocom/gsm/gsup.h>
1/2: refactor: add and use lu_fsm, osmo_gsup_req, osmo_ipa_name These are seemingly orthogonal changes in one patch, because they are in fact sufficiently intertwined that we are not willing to spend the time to separate them. They are also refactoring changes, unlikely to make sense on their own. ** lu_fsm: Attempting to make luop.c keep state about incoming GSUP requests made me find shortcomings in several places: - since it predates osmo_fsm, it is a state machine that does not strictly enforce the order of state transitions or the right sequence of incoming events. - several places OSMO_ASSERT() on data received from the network. - modifies the subscriber state before a LU is accepted. - dead code about canceling a subscriber in a previous VLR. That would be a good thing to actually do, which should also be trivial now that we record vlr_name and sgsn_name, but I decided to remove the dead code for now. To both step up the LU game *and* make it easier for me to integrate osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now, ample experience of writing osmo_fsms. ** osmo_gsup_req: Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs / communicate with remote MSCs via a proxy: a) It is important that a response that osmo-hlr generates and that is sent back to a requesting MSC contains all IEs that are needed to route it back to the requester. Particularly source_name must become destination_name in the response to be able to even reach the requesting MSC. Other fields are also necessary to match, which were so far taken care of in individual numerous code paths. b) For some operations, the response to a GSUP request is generated asynchronously (like Update Location Request -> Response, or taking the response from an EUSE, or the upcoming proxying to a remote HLR). To be able to feed a request message's information back into the response, we must thus keep the request data around. Since struct osmo_gsup_message references a lot of external data, usually with pointers directly into the received msgb, it is not so trivial to pass GSUP message data around asynchronously, on its own. osmo_gsup_req is the combined solution for both a and b: it keeps all data for a GSUP message by taking ownership of the incoming msgb, and it provides an explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that all code paths trivially are definitely responding with the correct IEs set to match the request's routing (by using osmo_gsup_make_response() recently added to libosmocore). Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to incoming requests received on the GSUP server (above LU code being one of them). In fact, the same should be done on the client side. Hence osmo_gsup_req is implemented in a server/client agnostic way, and is placed in libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups, using osmo_gsup_req in the related GSUP client is likely to resolve those problems without much thinking required beyond making all code paths use it. libosmo-gsupclient is hence added to osmo-hlr binary's own library dependencies. It would have been added by the D-GSM proxy routing anyway, we are just doing it a little sooner. ** cni_peer_id.c / osmo_ipa_name: We so far handle an IPA unit name as pointer + size, or as just pointer with implicit talloc size. To ease working with GSUP peer identification data, I require: - a non-allocated storage of an IPA Name. It brings the drawback of being size limited, but our current implementation is anyway only able to handle MSC and SGSN names of 31 characters (see struct hlr_subscriber). - a single-argument handle for IPA Name, - easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying by simple assignment, a = b. Hence this patch adds a osmo_ipa_name in cni_peer_id.h and cni_peer_id.c. Heavily used in LU and osmo_gsup_req. Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566 Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
2019-11-20 01:36:45 +00:00
struct hlr;
enum stmt_idx {
DB_STMT_SEL_ALL,
DB_STMT_SEL_ALL_ORDER_LAST_SEEN,
DB_STMT_SEL_FILTER_MSISDN,
DB_STMT_SEL_FILTER_IMSI,
DB_STMT_SEL_FILTER_IMEI,
DB_STMT_SEL_FILTER_CS,
DB_STMT_SEL_FILTER_PS,
DB_STMT_SEL_BY_IMSI,
DB_STMT_SEL_BY_MSISDN,
DB_STMT_SEL_BY_ID,
DB_STMT_SEL_BY_IMEI,
DB_STMT_UPD_VLR_BY_ID,
DB_STMT_UPD_SGSN_BY_ID,
DB_STMT_UPD_IMEI_BY_IMSI,
DB_STMT_AUC_BY_IMSI,
DB_STMT_AUC_UPD_SQN,
DB_STMT_UPD_PURGE_CS_BY_IMSI,
DB_STMT_UPD_PURGE_PS_BY_IMSI,
DB_STMT_UPD_NAM_PS_BY_IMSI,
DB_STMT_UPD_NAM_CS_BY_IMSI,
DB_STMT_SUBSCR_CREATE,
DB_STMT_DEL_BY_ID,
DB_STMT_SET_MSISDN_BY_IMSI,
DB_STMT_DELETE_MSISDN_BY_IMSI,
DB_STMT_AUC_2G_INSERT,
DB_STMT_AUC_2G_DELETE,
DB_STMT_AUC_3G_INSERT,
DB_STMT_AUC_3G_DELETE,
DB_STMT_SET_LAST_LU_SEEN,
DB_STMT_SET_LAST_LU_SEEN_PS,
DB_STMT_EXISTS_BY_IMSI,
DB_STMT_EXISTS_BY_MSISDN,
db v6: determine 3G AUC IND from VLR name Each VLR requesting auth tuples should use a distinct IND pool for 3G auth. So far we tied the IND to the GSUP peer connection; MSC and SGSN were always distinct GSUP peers, they ended up using distinct INDs. However, we have implemented a GSUP proxy, so that, in a distributed setup, a remotely roaming subscriber has only one direct GSUP peer proxying for both remote MSC and SGSN. That means as soon as a subscriber roams to a different site, we would use the GSUP proxy name to determine the IND instead of the separate MSC and SGSN. The site's MSC and SGSN appear as the same client, get the same IND bucket, waste SQNs rapidly and cause auth tuple generation load. So instead of using the local client as IND, persistently keep a list of VLR names and assign a different IND to each. Use the gsup_req->source_name as indicator, which reflects the actual remote VLR's name (remote MSC or SGSN). Persist the site <-> IND assignments in the database. Add an IND test to db_test.c There was an earlier patch version that separated the IND pools by cn_domain, but it turned out to add complex semantics, while only solving one aspect of the "adjacent VLR" problem. We need a solution not only for CS vs PS, but also for 2,3G vs 4G, and for sites that are physically adjacent to each other. This patch version does not offer any automatic solution for that -- as soon as more than 2^IND_bitlen (usually 32) VLRs show up, it is the responsibility of the admin to ensure the 'ind' table in the hlr.db does not have unfortunate IND assignments. So far no VTY commands exist for that, they may be added in the future. Related: OS#4319 Change-Id: I6f0a6bbef3a27507605c3b4a0e1a89bdfd468374
2019-12-12 03:04:53 +00:00
DB_STMT_IND_ADD,
DB_STMT_IND_SELECT,
DB_STMT_IND_DEL,
_NUM_DB_STMT
};
struct db_context {
char *fname;
sqlite3 *db;
sqlite3_stmt *stmt[_NUM_DB_STMT];
};
/* Optional feature to make SQLite3 using talloc */
#ifdef SQLITE_USE_TALLOC
int db_sqlite3_use_talloc(void *ctx);
#endif
void db_remove_reset(sqlite3_stmt *stmt);
bool db_bind_text(sqlite3_stmt *stmt, const char *param_name, const char *text);
bool db_bind_int(sqlite3_stmt *stmt, const char *param_name, int nr);
bool db_bind_int64(sqlite3_stmt *stmt, const char *param_name, int64_t nr);
bool db_bind_null(sqlite3_stmt *stmt, const char *param_name);
void db_close(struct db_context *dbc);
struct db_context *db_open(void *ctx, const char *fname, bool enable_sqlite3_logging, bool allow_upgrades);
#include <osmocom/crypt/auth.h>
/* obtain the authentication data for a given imsi */
int db_get_auth_data(struct db_context *dbc, const char *imsi,
struct osmo_sub_auth_data *aud2g,
struct osmo_sub_auth_data *aud3g,
int64_t *subscr_id);
int db_update_sqn(struct db_context *dbc, int64_t id,
uint64_t new_sqn);
int db_get_auc(struct db_context *dbc, const char *imsi,
unsigned int auc_3g_ind, struct osmo_auth_vector *vec,
unsigned int num_vec, const uint8_t *rand_auts,
const uint8_t *auts, bool separation_bit);
#include <osmocom/core/linuxlist.h>
#include <osmocom/gsm/protocol/gsm_23_003.h>
/* TODO: Get this from somewhere? */
#define GT_MAX_DIGITS 15
struct hlr_subscriber {
struct llist_head list;
int64_t id;
char imsi[GSM23003_IMSI_MAX_DIGITS+1];
char msisdn[GSM23003_MSISDN_MAX_DIGITS+1];
/* imeisv? */
char imei[GSM23003_IMEI_NUM_DIGITS+1];
char vlr_number[32];
char sgsn_number[32];
char sgsn_address[GT_MAX_DIGITS+1];
/* ggsn number + address */
/* gmlc number */
/* smsc number */
uint32_t periodic_lu_timer;
uint32_t periodic_rau_tau_timer;
bool nam_cs;
bool nam_ps;
uint32_t lmsi;
bool ms_purged_cs;
bool ms_purged_ps;
time_t last_lu_seen;
time_t last_lu_seen_ps;
/* talloc'd IPA unit name */
struct osmo_ipa_name vlr_via_proxy;
struct osmo_ipa_name sgsn_via_proxy;
};
/* A format string for use with strptime(3). This format string is
* used to parse the last_lu_seen column stored in the HLR database.
* See https://sqlite.org/lang_datefunc.html, function datetime(). */
#define DB_LAST_LU_SEEN_FMT "%Y-%m-%d %H:%M:%S"
/* Like struct osmo_sub_auth_data, but the keys are in hexdump representation.
* This is useful because SQLite requires them in hexdump format, and callers
* like the VTY and CTRL interface also have them available as hexdump to begin
* with. In the binary format, a VTY command would first need to hexparse,
* after which the db function would again hexdump, copying to separate
* buffers. The roundtrip can be saved by providing char* to begin with. */
struct sub_auth_data_str {
enum osmo_sub_auth_type type;
enum osmo_auth_algo algo;
union {
struct {
const char *opc;
const char *k;
uint64_t sqn;
int opc_is_op;
unsigned int ind_bitlen;
} umts;
struct {
const char *ki;
} gsm;
} u;
};
#define DB_SUBSCR_FLAG_NAM_CS (1 << 1)
#define DB_SUBSCR_FLAG_NAM_PS (1 << 2)
int db_subscr_create(struct db_context *dbc, const char *imsi, uint8_t flags);
int db_subscr_delete_by_id(struct db_context *dbc, int64_t subscr_id);
int db_subscr_update_msisdn_by_imsi(struct db_context *dbc, const char *imsi,
const char *msisdn);
int db_subscr_update_aud_by_id(struct db_context *dbc, int64_t subscr_id,
const struct sub_auth_data_str *aud);
int db_subscr_update_imei_by_imsi(struct db_context *dbc, const char* imsi, const char *imei);
int db_subscr_exists_by_imsi(struct db_context *dbc, const char *imsi);
int db_subscr_exists_by_msisdn(struct db_context *dbc, const char *msisdn);
int db_subscrs_get(struct db_context *dbc, const char *filter_type, const char *filter,
void (*get_cb)(struct hlr_subscriber *subscr, void *data), void *data,
int *count, const char **err);
int db_subscr_get_by_imsi(struct db_context *dbc, const char *imsi,
struct hlr_subscriber *subscr);
int db_subscr_get_by_msisdn(struct db_context *dbc, const char *msisdn,
struct hlr_subscriber *subscr);
int db_subscr_get_by_id(struct db_context *dbc, int64_t id,
struct hlr_subscriber *subscr);
int db_subscr_get_by_imei(struct db_context *dbc, const char *imei, struct hlr_subscriber *subscr);
int db_subscr_nam(struct db_context *dbc, const char *imsi, bool nam_val, bool is_ps);
int db_subscr_lu(struct db_context *dbc, int64_t subscr_id,
1/2: refactor: add and use lu_fsm, osmo_gsup_req, osmo_ipa_name These are seemingly orthogonal changes in one patch, because they are in fact sufficiently intertwined that we are not willing to spend the time to separate them. They are also refactoring changes, unlikely to make sense on their own. ** lu_fsm: Attempting to make luop.c keep state about incoming GSUP requests made me find shortcomings in several places: - since it predates osmo_fsm, it is a state machine that does not strictly enforce the order of state transitions or the right sequence of incoming events. - several places OSMO_ASSERT() on data received from the network. - modifies the subscriber state before a LU is accepted. - dead code about canceling a subscriber in a previous VLR. That would be a good thing to actually do, which should also be trivial now that we record vlr_name and sgsn_name, but I decided to remove the dead code for now. To both step up the LU game *and* make it easier for me to integrate osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now, ample experience of writing osmo_fsms. ** osmo_gsup_req: Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs / communicate with remote MSCs via a proxy: a) It is important that a response that osmo-hlr generates and that is sent back to a requesting MSC contains all IEs that are needed to route it back to the requester. Particularly source_name must become destination_name in the response to be able to even reach the requesting MSC. Other fields are also necessary to match, which were so far taken care of in individual numerous code paths. b) For some operations, the response to a GSUP request is generated asynchronously (like Update Location Request -> Response, or taking the response from an EUSE, or the upcoming proxying to a remote HLR). To be able to feed a request message's information back into the response, we must thus keep the request data around. Since struct osmo_gsup_message references a lot of external data, usually with pointers directly into the received msgb, it is not so trivial to pass GSUP message data around asynchronously, on its own. osmo_gsup_req is the combined solution for both a and b: it keeps all data for a GSUP message by taking ownership of the incoming msgb, and it provides an explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that all code paths trivially are definitely responding with the correct IEs set to match the request's routing (by using osmo_gsup_make_response() recently added to libosmocore). Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to incoming requests received on the GSUP server (above LU code being one of them). In fact, the same should be done on the client side. Hence osmo_gsup_req is implemented in a server/client agnostic way, and is placed in libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups, using osmo_gsup_req in the related GSUP client is likely to resolve those problems without much thinking required beyond making all code paths use it. libosmo-gsupclient is hence added to osmo-hlr binary's own library dependencies. It would have been added by the D-GSM proxy routing anyway, we are just doing it a little sooner. ** cni_peer_id.c / osmo_ipa_name: We so far handle an IPA unit name as pointer + size, or as just pointer with implicit talloc size. To ease working with GSUP peer identification data, I require: - a non-allocated storage of an IPA Name. It brings the drawback of being size limited, but our current implementation is anyway only able to handle MSC and SGSN names of 31 characters (see struct hlr_subscriber). - a single-argument handle for IPA Name, - easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying by simple assignment, a = b. Hence this patch adds a osmo_ipa_name in cni_peer_id.h and cni_peer_id.c. Heavily used in LU and osmo_gsup_req. Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566 Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
2019-11-20 01:36:45 +00:00
const struct osmo_ipa_name *vlr_name, bool is_ps,
const struct osmo_ipa_name *via_proxy);
int db_subscr_purge(struct db_context *dbc, const char *by_imsi,
bool purge_val, bool is_ps);
db v6: determine 3G AUC IND from VLR name Each VLR requesting auth tuples should use a distinct IND pool for 3G auth. So far we tied the IND to the GSUP peer connection; MSC and SGSN were always distinct GSUP peers, they ended up using distinct INDs. However, we have implemented a GSUP proxy, so that, in a distributed setup, a remotely roaming subscriber has only one direct GSUP peer proxying for both remote MSC and SGSN. That means as soon as a subscriber roams to a different site, we would use the GSUP proxy name to determine the IND instead of the separate MSC and SGSN. The site's MSC and SGSN appear as the same client, get the same IND bucket, waste SQNs rapidly and cause auth tuple generation load. So instead of using the local client as IND, persistently keep a list of VLR names and assign a different IND to each. Use the gsup_req->source_name as indicator, which reflects the actual remote VLR's name (remote MSC or SGSN). Persist the site <-> IND assignments in the database. Add an IND test to db_test.c There was an earlier patch version that separated the IND pools by cn_domain, but it turned out to add complex semantics, while only solving one aspect of the "adjacent VLR" problem. We need a solution not only for CS vs PS, but also for 2,3G vs 4G, and for sites that are physically adjacent to each other. This patch version does not offer any automatic solution for that -- as soon as more than 2^IND_bitlen (usually 32) VLRs show up, it is the responsibility of the admin to ensure the 'ind' table in the hlr.db does not have unfortunate IND assignments. So far no VTY commands exist for that, they may be added in the future. Related: OS#4319 Change-Id: I6f0a6bbef3a27507605c3b4a0e1a89bdfd468374
2019-12-12 03:04:53 +00:00
int db_ind(struct db_context *dbc, const struct osmo_cni_peer_id *vlr, unsigned int *ind);
int db_ind_del(struct db_context *dbc, const struct osmo_cni_peer_id *vlr);
/*! Call sqlite3_column_text() and copy result to a char[].
* \param[out] buf A char[] used as sizeof() arg(!) and osmo_strlcpy() target.
* \param[in] stmt An sqlite3_stmt*.
* \param[in] idx Index in stmt's returned columns.
*/
#define copy_sqlite3_text_to_buf(buf, stmt, idx) \
do { \
const char *_txt = (const char *) sqlite3_column_text(stmt, idx); \
osmo_strlcpy(buf, _txt, sizeof(buf)); \
} while (0)
1/2: refactor: add and use lu_fsm, osmo_gsup_req, osmo_ipa_name These are seemingly orthogonal changes in one patch, because they are in fact sufficiently intertwined that we are not willing to spend the time to separate them. They are also refactoring changes, unlikely to make sense on their own. ** lu_fsm: Attempting to make luop.c keep state about incoming GSUP requests made me find shortcomings in several places: - since it predates osmo_fsm, it is a state machine that does not strictly enforce the order of state transitions or the right sequence of incoming events. - several places OSMO_ASSERT() on data received from the network. - modifies the subscriber state before a LU is accepted. - dead code about canceling a subscriber in a previous VLR. That would be a good thing to actually do, which should also be trivial now that we record vlr_name and sgsn_name, but I decided to remove the dead code for now. To both step up the LU game *and* make it easier for me to integrate osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now, ample experience of writing osmo_fsms. ** osmo_gsup_req: Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs / communicate with remote MSCs via a proxy: a) It is important that a response that osmo-hlr generates and that is sent back to a requesting MSC contains all IEs that are needed to route it back to the requester. Particularly source_name must become destination_name in the response to be able to even reach the requesting MSC. Other fields are also necessary to match, which were so far taken care of in individual numerous code paths. b) For some operations, the response to a GSUP request is generated asynchronously (like Update Location Request -> Response, or taking the response from an EUSE, or the upcoming proxying to a remote HLR). To be able to feed a request message's information back into the response, we must thus keep the request data around. Since struct osmo_gsup_message references a lot of external data, usually with pointers directly into the received msgb, it is not so trivial to pass GSUP message data around asynchronously, on its own. osmo_gsup_req is the combined solution for both a and b: it keeps all data for a GSUP message by taking ownership of the incoming msgb, and it provides an explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that all code paths trivially are definitely responding with the correct IEs set to match the request's routing (by using osmo_gsup_make_response() recently added to libosmocore). Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to incoming requests received on the GSUP server (above LU code being one of them). In fact, the same should be done on the client side. Hence osmo_gsup_req is implemented in a server/client agnostic way, and is placed in libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups, using osmo_gsup_req in the related GSUP client is likely to resolve those problems without much thinking required beyond making all code paths use it. libosmo-gsupclient is hence added to osmo-hlr binary's own library dependencies. It would have been added by the D-GSM proxy routing anyway, we are just doing it a little sooner. ** cni_peer_id.c / osmo_ipa_name: We so far handle an IPA unit name as pointer + size, or as just pointer with implicit talloc size. To ease working with GSUP peer identification data, I require: - a non-allocated storage of an IPA Name. It brings the drawback of being size limited, but our current implementation is anyway only able to handle MSC and SGSN names of 31 characters (see struct hlr_subscriber). - a single-argument handle for IPA Name, - easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying by simple assignment, a = b. Hence this patch adds a osmo_ipa_name in cni_peer_id.h and cni_peer_id.c. Heavily used in LU and osmo_gsup_req. Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566 Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
2019-11-20 01:36:45 +00:00
/*! Call sqlite3_column_text() and copy result to a struct osmo_ipa_name.
* \param[out] ipa_name A struct osmo_ipa_name* to write to.
* \param[in] stmt An sqlite3_stmt*.
* \param[in] idx Index in stmt's returned columns.
*/
#define copy_sqlite3_text_to_ipa_name(ipa_name, stmt, idx) \
do { \
const char *_txt = (const char *) sqlite3_column_text(stmt, idx); \
osmo_ipa_name_set_str(ipa_name, _txt); \
} while (0)