Commit Graph

7 Commits

Author SHA1 Message Date
Vadim Yanitskiy fbd736ef37 src/db.c: integrate SQLite3 with talloc allocator
This change introduces an optional feature that allows to make
SQLite3 use talloc for all internal allocations. This would
facilitate finding memleaks. OsmoHLR needs to be configured
with '--enable-sqlite-talloc'.

  full talloc report on 'OsmoHLR' (total 292168 bytes in 449 blocks)
    struct osmo_gsup_server        contains    162 bytes in   3 blocks (ref 0)
      ...
    struct db_context              contains 288407 bytes in 420 blocks (ref 0)
      hlr.db                       contains      7 bytes in   1 blocks (ref 0)
    SQLite3                        contains 288192 bytes in 418 blocks (ref 0)
      db.c:95                      contains     48 bytes in   1 blocks (ref 0)
      db.c:95                      contains      2 bytes in   1 blocks (ref 0)
      ...

Unfortunately, old SQLite3 versions (such as 3.8.2) run out
of memory when trying to initialize a new database:

  DDB ERROR  db.c:88 (7) statement aborts at 3: []
  DDB ERROR  db.c:420 Unable to set Write-Ahead Logging: out of memory
  DDB ERROR  db.c:88 (7) statement aborts at 3: []
  DDB ERROR  db.c:238 Unable to prepare SQL statement
             'SELECT name FROM sqlite_master WHERE type='table' AND name=?'
  ...

I've noticed a huge difference in heap usage footprint compared to
generic malloc. At the same time, the recent versions (at least
3.24.0), work just fine.

Change-Id: Icfe67ed0f063b63e6794f9516da3003d01cf20a7
2019-07-30 17:15:17 +00:00
Vadim Yanitskiy 37642177f9 build: fix mess with 'db_test_SOURCES' and 'db_test_LDADD'
Somehow both 'db_test_SOURCES' and 'db_test_LDADD' ended up in
'src/Makefile.am'. This causes automake / autoconf to complain.
Let's get rid of both useless declarations.

Furthermore, the actual 'db_test_LDADD' in 'tests/Makefile.am'
contained references to the source files from '$(top_srcdir)'.
Most likely, the original intention was to depend on the object
files in '$(top_builddir)'. Let's also fix this.

Change-Id: Ib2e436ed91d9b7551dc5b205329d468c2b0ced04
2019-07-30 17:15:17 +00:00
Oliver Smith 3cf87fe22c tests: use -no-install libtool flag to avoid ./lt-* scripts
This ensures that the rpath of the generated binaries is set to use only
the just-compiled so-files and not any system-wide installed libraries
while avoiding the ugly shell script wrapper.

Change-Id: I927561289b17b313d52fb5c1da55e237fc1d33be
2019-03-19 13:04:49 +00:00
Neels Hofmeyr ba1605afdc fix build: db_test: missing LIBOSMOABIS_CFLAGS and _LIBS
Change-Id: I2539f5dc7a512a57ad36c460a11195ccbd84d7d6
2018-01-16 13:35:46 +01:00
Neels Hofmeyr 7750d2cedc automatically create db tables on osmo-hlr invocation
If a database file is missing, osmo-hlr creates it, as is the default sqlite3
API behavior -- before this patch, that db file is created, but lacks useful
tables. Actually also create initial tables in it, as osmo-nitb did.

In effect, the 'vty-test' target in tests/Makefile.am no longer needs to create
a database manually. (The 'ctrl-test' still does, because it also wants to add
subscriber data on top of the bare tables.)

Note: it could be desirable to bail if the desired database file does not
exist. That is however a different semantic from this patch; this is not
changing the fact that a db file is created, this just creates a usable one.

Note: I am about to add osmo-hlr-db-tool to do database migration from
osmo-nitb. For that, it is desirable to bootstrap a usable database, which is
the core reason for this patch.

Don't plainly duplicate hlr.sql to .c, but create db_bootstrap.h as a
BUILT_SOURCE from reading in sql/hlr.sql and mangling via sed to a list of SQL
statement strings. On each db_open(), run this bootstrap sequence.

In sql/hlr.sql, these tweaks are necessary:
* Add 'IF NOT EXISTS' to 'CREATE TABLE', so that the bootstrap sequence can be
  run on an already bootstrapped db.
* Drop the final comment at the bottom, which ended up being an empty SQL
  statement and causing sqlite3 API errors, seemed to have no purpose anyway.

Note: by composing the statement strings as multiline and including the SQL
comments, sqlite3 actually retains the comments contained in table definitions
and prints them back during 'sqlite3 hlr.db .dump'.

Change-Id: If77dbbfe1af3e66aaec91cb6295b687f37678636
2017-10-28 16:49:33 +00:00
Neels Hofmeyr cd7fa4502c db_test: fix *FLAGS
The -I includes should be in CFLAGS, not CPPFLAGS.

I noticed problems with it when trying to add an -I$(builddir) in an upcoming
patch that adds a BUILT_SOURCE, If77dbbfe1af3e66aaec91cb6295b687f37678636.

Change-Id: Ie57a04b7efc7a1e16cf0e3625d8ad2f0ef0089b0
2017-10-28 16:49:33 +00:00
Neels Hofmeyr 9850946013 add initial db_test: creating and deleting subscribers
Change-Id: I2a0d277f55162bf5ceb0fc7d50390f2994daed71
2017-10-11 22:32:19 +02:00