Commit Graph

1234 Commits

Author SHA1 Message Date
Pau Espin e1a26224e7 nitb: Introduce subscriber_delete API
Change-Id: I62829ed243857681bcc40a4a1861c5c583907f4b
2017-08-23 15:52:14 +02:00
Pau Espin 132e9fe6d3 nitb: ctrl: Remove unused return statement in subscriber_add
In case of failure an exception is raised. If everything goes well no
need to return True.

Change-Id: I5194eeb9208f973da12f2fa710f2b3e7acd78fb5
2017-08-23 15:50:50 +02:00
Pau Espin abc8fa5da1 hlr: fill auc_2g table when registering a subscriber
Change-Id: I491d093934c129a01baa1c5d7ada4acf370bf37d
2017-08-23 15:21:57 +02:00
Pau Espin b33fe85442 Add scenarios for each osmo-bts-trx type
This way we can run tests with a specific instance of an osmo-bts-trx,
for instance we may want to run some tests with an Ettus B200 and also
with a sysmoCell 5000.

Change-Id: I5fd78d79b8bfab8ccacc4666563b66b6da9f2bde
2017-08-23 12:56:40 +00:00
Pau Espin 404e150c69 bts_osmotrx: Support configuring bts addr, trx_remote_ip and launch_trx
We may want to support running a device which runs its own TRX
(osmo-trx or different implementation). Furthermore, this TRX may be
available in some specific hwardare rather than on the main unit.

This makes it easy to configure OsmoBtsTrx to launch it's own
osmo-trx or not. In case it is launched, all IPs are configured correctly
to ensure connection can be established.

Before this commit, osmo-trx was binding to 127.0.0.1. Now we can
support multiple osmo-trx being launched on the main unit.

Change-Id: I825ed1fc0c3fe75d196db90c1508283fbd04acf8
2017-08-23 12:56:40 +00:00
Pau Espin 5df8dd2a2a resources.conf: Remove empty line
Change-Id: I821bff68ce3a4a81a9deb79e6302bd7c341a8255
2017-08-23 12:56:40 +00:00
Pau Espin cd6ad9d519 modem: Allow tests to change KI
This is useful for tests that wants to fake a bad KI to test if auth and
encryption are working correctly.

Change-Id: I40cfed6f6b9239eea2b97ffa010b98c78f0cb9cb
2017-08-23 12:55:43 +00:00
Pau Espin ca126b1d82 Allow tests to select network encryption
Change-Id: I273eee44c095690d0c2e3994befa97edc42496ac
2017-08-23 12:55:30 +00:00
Pau Espin 66c96c3c8e ctrl: Add dbg log to print received data
This is useful to see for instance the returned values from NITB when
asked for active subscribers.

Change-Id: I5c616646da360457930888b54d9019b76f05fe2f
2017-08-23 10:48:07 +02:00
Pau Espin 8d7a2e343b Increase log verbosity for osmo-bts-trx
The log levels changed are useful to understand clock failures we see
from time to time in osmo-gsm-tester. See OS-2325.

Change-Id: Id74af3ccb2f3db3c1793c7ff7b7a60a534e78e81
2017-08-21 19:32:00 +02:00
Pau Espin 80a5c77821 bts_osmotrx: Allow LD_LIBRARY_PATH when loading osmo-trx
When splitting osmo-trx to its own class the env injection part was
removed because it seemed not needed as it has no dependencies included
by default in the trial.

However, it may still be required by the osmo-gsm-tester main unit to
launch osmo-gsm-tester with LD_PRELOAD_PATH to point to specific
versions of libuhd, so let's add the env injection back in there.

Change-Id: I069ba69f4207665c8c575fed0fc8ff164c571e36
2017-08-21 19:03:10 +02:00
Pau Espin 3169ffbaef esme: Remove temporary workaround for non-implemented deliver_sm
Openbsc has SMS Delivery Reports since recently, which means we are now
receiving deliver_sm messages and this workaround introduced in commit
db0d8ab4fc is not required anymore.

Closes OS-#2353.

Change-Id: Iecf3983bfc24fec8023ba5d589c2d7620ec5d921
2017-08-18 19:19:16 +00:00
Pau Espin 4773853cb0 Split osmo-trx from osmo-bts-trx
In the future, some environments/products may come with a trx interface
pre-installed. Start work to easily disable launching it.

Change-Id: I556c3e2ba16753393c7e70800c533a18122daeaa
2017-08-18 07:58:52 +00:00
Pau Espin 5adb5c7405 esme: user_message_referece values should be smaller than 256
SMPP specs say message reference is 16 bits field.
However, message reference field in GSM03.40 spec is only 8 bits.

Change-Id: I52968a10089d14e0e3b0f9be678c261e3f92939c
2017-08-17 17:58:02 +02:00
Pau Espin 69373b3201 esme: smpp pdu field is a string and needs conversion
I take the chance to also swap the two log parameters, because they are
included into a dictionary and shown in swapped order in the log anyway,
which distracts me every time I look at the values.

Change-Id: I6e6ec5b9980958de82ade40e4cd898dc6f1b889f
2017-08-17 17:05:08 +02:00
Pau Espin 74a7676eb1 util: ip_to_iface: Return secondary iface if present
In the following example setup, if osmo-nitb is listening
on 10.42.42.2, we want to return eth1:0 instead of eth1.

ip addr show eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0d:b9:35:3a:41 brd ff:ff:ff:ff:ff:ff
    inet 10.42.42.1/24 brd 10.42.42.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet 10.42.42.2/24 brd 10.42.42.255 scope global secondary eth1:0
       valid_lft forever preferred_lft forever
    inet 10.42.42.3/24 brd 10.42.42.255 scope global secondary eth1:1
       valid_lft forever preferred_lft forever

Change-Id: Ia66e6fd8138fc11a70437a573128aacf6a01b8ff
2017-08-10 12:12:07 +02:00
Pau Espin ac9c1bb32f esme: Add cleanup function to avoid cascade failure of tests
In commit 6100b629e5 I assumed esme
already had a cleanup function, which doesn't seem to be the case. This
means if a test calls esme.connect() and fails afterwards without
calling esme.disconnect(), the connection will be kept but as the test
will finish and the NITB is closed, then the connection will be kept
forever in an erroneous state. With the cleanup function we ensure we
will close the connection after the test.

Change-Id: Ia3a23bb8bbccfe7ac5c544521b5566164609f0a5
2017-08-10 10:59:43 +02:00
Neels Hofmeyr a10aeaea4b osmo-bts-trx config: remove settsc
In osmo-bts change I51aeb17661dfd63ff347f7b2c0d7ffa383ec814c the settsc config
item is being removed. To be able to test it, drop the settsc option from the
config.

Change-Id: I58002a84b6b0104de4fb5c7cade27b263e375c1f
2017-08-10 10:25:48 +02:00
Pau Espin 329b6f446c Set osmo-bts pcu-socket config to point to run dir
Otherwise osmo-bts stores the socket file in /tmp. If an earlier
instance doesn't finish cleanly, the file is left there and a new
instance will fail to start because it cannot create a new socket file
in the same place.

Change-Id: I5a1da23c45a4ac496fe765e0d78c52dae3e7808b
2017-08-09 11:01:52 +00:00
Neels Hofmeyr 4008b69bbe cosmetic: osmo-msc.cfg.tmpl: fix indent
Change-Id: Iefc3b06338cff4f5e51b7b03d9a5f92de18ca86a
2017-08-07 14:18:09 +02:00
Neels Hofmeyr a64ab702aa aoip: add newly needed osmo-msc point-code config
OsmoMSC no longer has a hard-coded point-code of 0.0.1. Configure it.

Change-Id: I54be984415445e695c583f35c419aed559ac5c84
2017-08-07 14:17:13 +02:00
Alexander Couzens 2ec075dede osmo-bsc.cfg.tmpl: add point code addresses
osmo-bsc requires atm to have point code addresses given for msc + bsc

Change-Id: I94f8a6892ad7e10001322cbc5a46687c0d532cb2
2017-08-07 13:45:08 +02:00
Pau Espin 6100b629e5 suite: Don't stop cleanup of objects on cleanup exception
Let's accept that during cleanup time some stuff may not be in a good
state, specially in case of a failure, but let's keep harder to clean
up everything as much as possible.

This should fix an issue in which after a test failure using an esme object,
an smpp connection is kept in a buggy state and never removed/cleaned up, and
makes all tests after it fail too because the conn is triggered at
poll() time:

   File "/home/jenkins/workspace/osmo-gsm-tester_run/osmo-gsm-tester/src/osmo_gsm_tester/suite.py", line 260, in run_tests
    self.objects_cleanup()
   File "/home/jenkins/workspace/osmo-gsm-tester_run/osmo-gsm-tester/src/osmo_gsm_tester/suite.py", line 199, in objects_cleanup
    obj.cleanup()
   File "/home/jenkins/workspace/osmo-gsm-tester_run/osmo-gsm-tester/src/osmo_gsm_tester/ofono_client.py", line 345, in cleanup
    self.dbus.cleanup()
   File "/home/jenkins/workspace/osmo-gsm-tester_run/osmo-gsm-tester/src/osmo_gsm_tester/ofono_client.py", line 170, in cleanup
    self.set_powered(False)
   File "/home/jenkins/workspace/osmo-gsm-tester_run/osmo-gsm-tester/src/osmo_gsm_tester/ofono_client.py", line 304, in set_powered
    self.set_bool('Powered', powered)
   File "/home/jenkins/workspace/osmo-gsm-tester_run/osmo-gsm-tester/src/osmo_gsm_tester/ofono_client.py", line 295, in set_bool
    event_loop.poll()
   File "/home/jenkins/workspace/osmo-gsm-tester_run/osmo-gsm-tester/src/osmo_gsm_tester/event_loop.py", line 39, in poll
    func()
   File "/home/jenkins/workspace/osmo-gsm-tester_run/osmo-gsm-tester/src/osmo_gsm_tester/esme.py", line 78, in poll
    self.client.poll()
   File "/usr/local/lib/python3.4/dist-packages/smpplib/client.py", line 321, in poll
    self.read_once(ignore_error_codes)
   File "/usr/local/lib/python3.4/dist-packages/smpplib/client.py", line 279, in read_once
    p = self.read_pdu()
   File "/usr/local/lib/python3.4/dist-packages/smpplib/client.py", line 206, in read_pdu
    raise exceptions.ConnectionError()
 smpplib.exceptions.ConnectionError

Change-Id: Ie7ef9284490f12f5cfd76c35b33b57eefab20eb6
2017-07-31 18:19:08 +02:00
Pau Espin 522ae68b08 esme: Log connection parameters of the SMSC
Change-Id: Ie4d3019503a68b02ace849defe618bdc74eacbd9
2017-07-31 17:25:15 +02:00
Pau Espin 91c75b38f1 esme: Fix race condition waiting for message response
Sometimes in test esme_ms_sms_transaction.py we get the following error:
esme_ms_sms_transaction.py:54: ERR: ValueError: list.remove(x): x not in list

This appears due to a race condition because sms_send is used several times,
which means we don't wait or sync until we receive the response before calling
sms_send_wait_resp. That means when we wait for response of message with
seqnum X, we may receive response from message seqnum X-1 which was
initiated by sms_send and thus was not stored in the self.pdus_pending
array. As it's not there, trying to remove it triggers an exception.

Change-Id: Idf49f40eb46be1448b328a5c338ddbc4547148ff
2017-07-20 22:51:52 +02:00
Pau Espin 4a22ac7d2c aoip_smpp: Disable check in esme_ms_sms_storeforward due to known issue
A test case checked in that test is known to be failing at the moment
due to missing correct implementation bits for AoIP software.

Comment it in a separate commit so that is easy to revert it once we
have it implemented.

See OsmoSMSC #2354 for more information.

Change-Id: I12f8dbe1d6dc9a554438fda1e38b2e5e3245c320
2017-07-20 16:56:08 +00:00
Pau Espin db0d8ab4fc Improve SMPP supported features and test coverage
esme: Add several bits to handle logic required by tests:
- Allow specifying the mode used to send an sms
- Add a parameter to ask to receive a Delivery receipt for that message
- Add sms_send_wait_resp API, which waits until the response message for
  a given smpp message is received when sending an sms.
- Add receipt_was_received API, which together with
  message_received_handler maintains state of the delivery receipts we
  asked for and were still not received. However, the check needs to be
  disabled for now because OsmoNITB doens't seem to be sending stuff
  properly, see OsmoNITB #2353.
- On message_received_handler, also print alert_notification messages,
  to show that there's actually a bug in OsmoNITB, see #2352.

Move old esme_ms_sms to esme_ms_sms_transaction, and explicitly state
that we are using that mode. On the same test, we can now enable the
part which asserts that sending an SMS to an msisdn with unknown
destination triggers an error. The issue was mainly that the error had
to come from the SMSC server response, not from the sent message, so
we have to wait for the response to have the failure triggered.

Finally, add esme_ms_sms_storeforward, which tests features for sms sent
using that mode, and uses the APIs described above.

Change-Id: Ia2c0c325fee14143deca8310312fc530cd9ce92e
2017-07-20 16:56:08 +00:00
Harald Welte ddf78124f8 remove timer section from osmo-{bts,bsc}.cfg.tmpl
As described in https://osmocom.org/issues/2380 and related gerrit
patch https://gerrit.osmocom.org/#/c/3306 the config files currently
contain timers with a 'zero' value, leading to issues.

The config file templates used in osmo-gsm-tester suffer actually
from exactly the same issue: They basically contain values that were
written back to the file based on the default initialization in libbsc,
which unfortunately was zero in some cases.

Let's remove those values, they are the (bad) default anyway.

libbsc patches in the pipeline will introduce reasonable defaults
and forbid the use of zero for timers.

Change-Id: I8cde29a597a17c2659b3b87268be4b12975f4bef
2017-07-20 11:54:49 +02:00
Pau Espin e5b0366849 contrib: Disable doxygen doc generation
The --disable-doxygen was recently added to libosmocore and
libosmo-netif. No need to manually remove documentation anymore.

Change-Id: I3ca3dd1c115fc6784e4dc26a5ff5bf35d65b4e22
2017-07-06 17:14:48 +02:00
Pau Espin c9dfe479cf default-suites: Add smpp
No need to run it for several BTS as the focus here is testing the core
network and interoperation with different BTS is already tested with the
sms suite. This way we avoid lossing extra time running the default
suite set.

Change-Id: Ie6458801ec1ecce63e08617d1e449047dc496e16
2017-07-05 11:35:44 +00:00
Pau Espin 3c35df3e28 contrib: Fix removal of doc directories
The docs are actually not removed because the shell takes
{libosmocore,libosmo-sccp} as a string literal if found inside quote
marks, see [1] for more information.

[1] https://unix.stackexchange.com/questions/67757/wildcards-inside-quotes

Change-Id: Ida8941525db2d6a80539ffda29fb4fa74dbb1e16
2017-07-05 10:53:13 +02:00
Pau Espin 800a697297 suite: Print process ending prematurely
Change-Id: Icec476ccc536611f771dcc7872f5190bbd0572e7
2017-07-03 18:34:09 +02:00
Pau Espin 90c23cc991 suite: Use event_loop.poll in interactive poll
This way we also poll dbus events from ofono_client

Change-Id: I06fe9a78d702ebb05c4ece78e14fcae9e2e74ee2
2017-07-03 13:12:41 +02:00
Pau Espin 0d8deec6ae Add microsecond accuracy to log timestamps
This is useful while debugging and trying to check events across other
outputs such as pcap files, process logs, etc.

Change-Id: I43bb5c6e9977189251802bc2b078c52eb046bab8
2017-07-03 10:39:56 +00:00
Pau Espin 58ff38d456 ofono_client: Power off modem during cleanup()
Change-Id: Ic0c509e77ababf385beb8b1d26659ca4d18f9331
2017-07-03 10:39:46 +00:00
Neels Hofmeyr db2d76bb52 jenkins: build msc using libosmo-sccp master
Since 81c0bcab068174036f6897e69311f413d94c74d4 was merged, it is not necessary
to build the neels/aoip branch anymore. Other patches on that branch are not
necessary either.

Change-Id: I850e0dd2fc61d59d56c0dd69ee00b6e19685fb95
2017-06-29 19:45:49 +02:00
Neels Hofmeyr 109ebca44c Revert "osmo-bts-trx config: remove settsc"
According to pespin, this makes all osmo-bts-trx test runs fail.
We will still figure out the details, but for the time being
let's go with the known working config.

This reverts commit 72ded64705.

Change-Id: I57a3db0d060203ad6540c9a3cc7735e273ee0048
2017-06-27 14:19:40 +00:00
Neels Hofmeyr 72ded64705 osmo-bts-trx config: remove settsc
In osmo-bts change I51aeb17661dfd63ff347f7b2c0d7ffa383ec814c the settsc config
item is being removed. To be able to test it, drop the settsc option from the
config.

Change-Id: I89b35fc587bbac6810511b2317eed24368858f37
2017-06-26 16:58:14 +02:00
Neels Hofmeyr b398b52ad5 jenkins: fix: recent change broke jenkins build
Recent commit 851802b927 introduces a build
branch, which works fine, but only on the first run. A second run in the same
repos can't delete the branch git is currently on.

I've had enough of git being impossibly difficult in UI to simply checkout a
hash or a branch now. Completely wipe out the git repository and clone a fresh
one every time.

Unfortunately, 'git clone -b' doesn't allow passing a commit hash, so we still
need to do a clone-then-checkout dance. At least now we know it will work the
same on every run.

Change-Id: I6aca4c53a796312248a189b815dfc1198a173ed9
2017-06-23 04:13:30 +02:00
Neels Hofmeyr 2581b50408 jenkins-build-common.sh: cosmetic: clear repos a bit later
If we rm -rf * and then checkout a branch, the log prints the entire file tree
as deleted. Instead, rm just before the git reset --hard, which avoids the
extra output.

Change-Id: Ib5b28a82f05d941eae8f3a2f468ef1e9d67e6180
2017-06-23 04:07:44 +02:00
Neels Hofmeyr 851802b927 jenkins-build-common.sh: fix being stuck on branch
It's hard to allow both branch names and git hashes. For a branch, we want to
prepend 'origin' to use the upstream branch. For a git hash, we don't.

We so far prepend 'origin/' if the current branch name doesn't resolve, but
that's not enough. If a local 'master' branch exists, we would stay on that
branch instead of origin/master.

Rather, prepend 'origin/' if 'origin/$branch' exists. Git hashes should not
exist as 'origin/123abc...' and used as-is, where branches from origin should
be updated to upstream by prepending 'origin/'.

Also create a local branch to build. Always force-remove the branch and
re-create from the origin/name or git hash. Keep the reset --hard for paranoia.

Change-Id: I2e610b357f8559c6b6ffb544eb0a952f04dd9f70
2017-06-23 03:52:26 +02:00
Neels Hofmeyr de79cb155d jenkins: aoip: move to libosmocore master
The patches needed for AoIP have now been merged to libosmocore master.

Change-Id: I31f627f24e0e111c2c3461dbdc88f0e54ebf0c8c
2017-06-23 03:01:43 +02:00
Neels Hofmeyr 42bebface8 add cmdline option -R to add src file log to stdout
Change-Id: I5b21ec30453dceb53e7d4cc972607971935ef10f
2017-06-22 06:43:04 +00:00
Neels Hofmeyr 3ce18fcac7 Revert "osmo-bts-trx: wait some longer before starting osmo-bts-trx"
With this patch, the same errors are still happening.
This is not a solution, so drop this again.

This reverts commit 6c8d9497f2.

Change-Id: I324e965fdf40a369e1bcebfa4b32d0a3a7b86eb3
2017-06-18 16:50:13 +00:00
Pau Espin 2d16f6fd2c Add support for SMPP testing
As defined in [1], the different related actors are implemented in this
commit: ESME and SMSC.

SMSC: In Osmocom, the SMSC is currently implemented inside the NITB or
the MSC. A new Smsc abstract class is created to shared code between the
NITB and the MSC, and also makes it easier for later when the SMSC is
splitted. ESMEs can be dynamically added to its configuration in a
similar way to how the BTSs are added.

ESME: A new class Esme is created which can be used by tests to control
an ESME to interact with the SMSC. The ESME functionalities are
implemented using python-smpplib. Required version of this library is at
least 43cc6f819ec76b2c0a9d36d1d439308634716227, which contains support
for python 3 and some required features to poll the socket.

This commit already contains a few tests which checks different
features and tests the API. Extending tested features or scenarios can be
later done quite easily.

The tests are not enabled by default right now, because there are several
of them in a suite and the ip_address resources are not freed after every
tests which ends up in the suite failing due to missing reserved
resources. All the tests run alone work though. When the issue is fixed
they can then be added to the default list of tests to be run.

[1] http://opensmpp.org/specs/SMPP_v3_4_Issue1_2.pdf

Change-Id: I14ca3cb009d6d646a449ca99b0200da12085c0da
2017-06-18 08:48:09 +00:00
Neels Hofmeyr 640c7760e9 jenkins-run: create bin.tgz separately from run.tgz
The binary tars are fairly large, and usually one wants to investigate the run
dir logs without any need to download the binaries. Archive the binary tars
separately from the run logs.

Change-Id: I23e6df0d30fc9ea7c5b48bb2364be075d910bfcd
2017-06-18 02:15:22 +02:00
Pau Espin 1dd2955cb1 Unuse resources after every test
Fix bug in put_all(). That method was unused before this commit.

Clean the process list after the processes are stopped,
otherwise the suite eventually fails with a 'process terminated
prematurely' error. Before it was not necessary because it was not
reused (a new suite run was created).

Change-Id: Iee12866045145544076c6c18786e1a54f18fc293
2017-06-17 17:01:20 +00:00
Pau Espin d091233934 Use a subdir of run_dir for each test
Processes created have the scope of the test, so we should store
everything in a per-suite_run/per-test directory, otherwise everything
is stored in the same trial run_dir directory and it's really messy.

Change-Id: I06be2dd21710e14c1337d13b1fe6c2f68f037957
2017-06-17 16:10:04 +00:00
Neels Hofmeyr 6c8d9497f2 osmo-bts-trx: wait some longer before starting osmo-bts-trx
We get sporadic clock skews when running osmo-bts-trx, causing the process to
end and the test to fail. Try to give some seconds for the osmo-trx process to
sort itself out, in the hope to avoid the clock skews.

Related: OS#2325

Change-Id: I82d29358498e7ad9fef28d409808168926e2f876
2017-06-17 17:44:06 +02:00
Pau Espin fbecf41f01 ofono_client: Cancel pending async dbus methods at cleanup time
This commit should fix issue described #2326, in which a test started
scanning and quickly afterwards failed, leaving the async Scan() method
running and the response arriving when we were already in another
suite/test.

Change-Id: If5e9bcac8e4114718243d178ff50d1ed8b30c95d
2017-06-14 12:27:42 +02:00