Commit Graph

135 Commits

Author SHA1 Message Date
Vadim Yanitskiy b2349bc359 trx_toolkit/data_msg.py: remove obsolete documentation
We do have TRXC/TRXD documentation in osmo-gsm-manuals repository.
These big comments are out of sync with what we have in the manuals,
so let's better remove them to avoid maintaining docs in several places.

Change-Id: I47786cf3039f712efadc85bc4e1c3ae89e79ff25
Related: OS#4006, SYS#4895
2021-05-03 20:39:10 +02:00
Vadim Yanitskiy 87c5f2c92d trx_toolkit/data_msg.py: update entries in enum Modulation
Change-Id: Ia96b263bbb162b6c40f2cf81616118cc297299a5
Related: OS#4006, SYS#4895
2021-05-03 20:39:05 +02:00
Vadim Yanitskiy a4127f272a trx_toolkit/test_codec.py: add missing sub-test with key=0x88
Change-Id: If8e2ebedc48fecc4c54c71e40cadb0f3273602c5
2021-05-03 20:35:46 +02:00
Vadim Yanitskiy eac05ae644 trx_toolkit/ctrl_if_trx.py: fix undefined variable 'trx'
Change-Id: I5c06fa5183b4d04cbc0c68327a062b320477344c
2021-05-03 20:35:46 +02:00
Vadim Yanitskiy b988408166 trx_toolkit/fake_trx.py: use raw string notation for regex
This way there is no need to escape backslashes in regex.

Change-Id: I1b2b6675851275bd5285ffc287410535b22055ae
2021-05-03 20:35:46 +02:00
Vadim Yanitskiy bd73f0920d trx_toolkit/gsm_shared.py: s/GSM_BURST_LEN/GMSK_BURST_LEN/g
Change-Id: I7268196eb9fd822f0e7b65899e4c83c48a20ba5b
2021-04-30 22:31:40 +02:00
Vadim Yanitskiy a2618b789a trx_toolkit/trxd_proto.py: fix missing field name for codec.Spare
Change-Id: I110dbcebf86cf978f2de7275a91c48b999fade32
Fixes: I482f72fd9305c51f43a0339d03904fb693d90ac9
Related: OS#4006, SYS#4895
2021-04-30 22:31:40 +02:00
Vadim Yanitskiy 8fc13352aa trx_toolkit/trxd_proto.py: fix encoding of TRXDv0 Tx PDUs
Change-Id: I9da59f7e7c47dd1cac833839165ba05de6b18144
Related: OS#4006, SYS#4895
2021-04-30 22:31:40 +02:00
Vadim Yanitskiy 393158ac08 trx_toolkit: change coding of 'PWR' and 'SCPIR' fields
During an internal discussion, it was decided to keep field 'PWR'
as-is and move 'SCPIR' into a separate octet.  This is easier to
parse, less confusing, and would save us some CPU cycles.

Change-Id: I482f72fd9305c51f43a0339d03904fb693d90ac9
Related: OS#4006, SYS#4895
2021-04-15 21:51:11 +02:00
Vadim Yanitskiy d4900f93ee trx_toolkit: define TRXDv2 PDUs using declarative codec
Change-Id: If356d285006c0b9b57879d0499b8144eca820cab
Related: OS#4006, SYS#4895
2021-04-12 01:51:14 +02:00
Vadim Yanitskiy 7c3a566b52 trx_toolkit: define TRXDv0/v1 PDUs using declarative codec
Change-Id: I739ae5da22c603fb2cf1e84d3a79fb1a6e7343b6
Related: OS#4006, SYS#4895
2021-04-08 03:20:39 +02:00
Vadim Yanitskiy 33f456dda3 trx_toolkit: make codec.py work with EOL Python 3.5
Our build system is based on Debian 9 and EOL Python 3.5, so we have
to maintain backwards compatibility (sigh).  Some type hints moved
to comments, some had to be commented out completely.  Hopefully,
we can 'un-vandalize' the code by reverting this change once there
will be no requirement to support EOL stuff.

Change-Id: I7211cfbb7549b6e700aa3dd44464ff762fd51185
Related: OS#4006, SYS#4895
2021-04-08 03:20:39 +02:00
Vadim Yanitskiy a772337e27 trx_toolkit: check in simple declarative codec
Change-Id: I7ff46b278c59af3720ee7f3950ea5a8b2f1313e1
Related: OS#4006, SYS#4895
2021-04-08 03:20:39 +02:00
Vadim Yanitskiy b00601d7ee trx_toolkit/data_msg.py: add type hints to static methods
Change-Id: I06fd8bc7418700de40467fd63a08da2bc2abcea2
2021-04-08 03:20:39 +02:00
Vadim Yanitskiy 45cc12d665 trx_toolkit/udp_link.py: cosmetic: setblocking() takes bool
Change-Id: I9c4fa8cb7c237f3b9e34ba236b6507da5c668aee
2021-03-14 00:57:43 +01:00
Vadim Yanitskiy c8754d47db trx_toolkit/data_msg.py: use uppercase for KNOWN_VERSIONS
Change-Id: Ie4dcc8e91115c514ace243a95a9b44a17cd976b4
Related: OS#4006, SYS#4895
2021-03-01 15:02:35 +00:00
Vadim Yanitskiy ba3e1f7c2a trx_toolkit/data_msg.py: use tuple for DATAMSG.known_versions
Change-Id: I79de5fe478297e1850d9582426888411a0189d3f
Related: OS#4006, SYS#4895
2021-03-01 15:02:35 +00:00
Vadim Yanitskiy f9938b3a54 trx_toolkit/data_msg.py: decorate abstract methods as such
Change-Id: I27fdcfdabc2b5318ab3e958d2e5446e670fe9035
Related: OS#4006, SYS#4895
2021-03-01 15:02:35 +00:00
Vadim Yanitskiy 7a31e98936 trx_toolkit/data_msg.py: convert comments into docstrings
Change-Id: I856b54fd1baca4ae0edd2aa59be6a76372cef667
Related: OS#4006, SYS#4895
2021-03-01 15:02:35 +00:00
Vadim Yanitskiy 904d128c4d trx_toolkit/data_msg.py: do not instantiate abstract class DATAMSG
Change-Id: I811f67ae030ccd93e1c6a421097cec36ca01cbf9
Related: OS#4006, SYS#4895
2021-03-01 15:02:35 +00:00
Vadim Yanitskiy 02830f3bd2 trx_toolkit/clck_gen.py: fix: Thread.isAlive() was removed
As it turns out, in Python < 3.9 class Thread defines 'is_alive'
and 'isAlive = is_alive'.  In Python 3.9 the later has been
removed, so fake_trx.py crashes on receipt of 'POWEROFF':

  File "/home/wmn/wmn/osmocom/bb/src/target/trx_toolkit/clck_gen.py",
    line 63, in running
      return self._thread.isAlive()
  AttributeError: 'Thread' object has no attribute 'isAlive'

See https://bugs.python.org/issue35283 for more details.

Change-Id: Id441d76dddb659958803d507e0fb028fb06422a7
2020-12-25 14:40:34 +01:00
Pau Espin 1687d3dea3 fake_trx: Implement RFMUTE TRXC cmd
Change-Id: I67d16858cd70cb0527c1da77bd3787d5e53100b4
2020-08-26 18:04:57 +02:00
Vadim Yanitskiy 4e4babc7db trx_toolkit/data_if.py: fix: handle encoding exceptions
Change-Id: I78163d41be3a912da1dd8c0543b1c3af3a0649fa
Related: OS#4681
2020-07-27 02:58:25 +07:00
Vadim Yanitskiy 129c976c3a trx_toolkit/data_if.py: do not validate TRXD message twice
DATAMSG.gen_msg() does validete the message before encoding.

Change-Id: Ia3691b3c18778cf7a1f16c71bef5c0b2e6241190
Related: OS#4681
2020-07-27 02:25:20 +07:00
Vadim Yanitskiy d4ed09df57 Revert "trx_toolkit/transceiver.py: implement the transmit burst queue"
This reverts commit 6e1c82d298.
Unfortunately, solving one problem it introduced even more regressions.

Change-Id: If29b4f6718cbc8af18fe18a5e3eca3912e8af01e
Related: OS#4658
2020-07-22 19:59:06 +07:00
Vadim Yanitskiy 7eb355ab3e trx_toolkit/clck_gen.py: remove unused import of 'time' module
Change-Id: I40628d32409543c9f4b40b7268a4538b4671102d
2020-07-16 14:16:34 +07:00
Vadim Yanitskiy 1e20f8e042 trx_toolkit: get rid of Python2 specific workarounds
Change-Id: I16c63205c9133d964048588c25867ac7c310f951
2020-07-16 13:58:34 +07:00
Vadim Yanitskiy 49673e4073 trx_toolkit: use python3 in shebang of executable scripts
TRX Toolkit is still backwards compatible with Python2, but Python3
does much better in terms of performance.  Also, on Debian Stretch
that is used as a base for our Docker images, Python 2.7 is still
the default.  Let's require Python3 in shebang.

Change-Id: I8a1d7c59d3b5d49ec2ed94a7c77905e02134f216
2020-07-16 13:46:19 +07:00
Vadim Yanitskiy 270af48118 trx_toolkit: remove shebang from non-executable scripts
Change-Id: I5ddc531a4e98d4d6f8672d6ef14034fce605ba3d
2020-07-16 13:42:56 +07:00
Vadim Yanitskiy 6e1c82d298 trx_toolkit/transceiver.py: implement the transmit burst queue
In order to reflect the UL/DL delay caused by the premature burst
scheduling (a.k.a. 'fn-advance') in a virtual environment, the
Transceiver implementation now queues all to be transmitted bursts,
so they remain in the queue until the appropriate time of transmission.

The API user is supposed to call recv_data_msg() in order to obtain
a L12TRX message on the TRXD (data) inteface, so it gets queued by
this function.  Then, to ensure the timeous transmission, the user
of this implementation needs to call clck_tick() on each TDMA
frame.  Both functions are thread-safe (queue mutex).

In a multi-trx configuration, the use of queue additionally ensures
proper burst aggregation on multiple TRXD connections, so all L12TRX
messages are guaranteed to be sent in the right order, i.e. with
monolithically-increasing TDMA frame numbers.

Of course, this change increases the overall CPU usage, given that
each transceiver gets its own queue, and we need to serve them all
on every TDMA frame.  According to my measurements, when running
test cases from ttcn3-bts-test, the average load is ~50% higher
than what it used to be.  Still not significantly high, though.

Change-Id: Ie66ef9667dc8d156ad578ce324941a816c07c105
Related: OS#4658, OS#4546
2020-07-14 17:21:14 +07:00
Vadim Yanitskiy f262caca79 trx_toolkit/clck_gen.py: support optional clock handler
Change-Id: I85b2182d9835ed035cf370e45ea039ac6a7e8405
2020-07-14 17:21:10 +07:00
Vadim Yanitskiy 8d19fbef57 trx_toolkit/clck_gen.py: fix TDMA clock counter wrapping
Change-Id: I157447c7610402f6d62d2b74c9f04fcaa0bc1724
2020-07-14 17:20:14 +07:00
Vadim Yanitskiy 93beb3f5c5 trx_toolkit/clck_gen.py: call send_clck_ind() on every TDMA frame
Change-Id: I6d53e5266fa3b1f2eb55822d1c14975789b202ed
2020-07-14 17:17:38 +07:00
Vadim Yanitskiy e2aaeb59b3 trx_toolkit/fake_trx.py: move Rx burst handling to Transceiver
Change-Id: Ic1f44bfb21ac3173e9530a0a9966cd5e64b8bd48
2020-07-13 08:51:19 +00:00
Vadim Yanitskiy 37c81fea95 trx_toolkit/fake_trx.py: avoid using TRXList.__getitem__()
Running with cProfile shows that there are quite a lot calls:

  469896    0.254    0.000    0.254    0.000 trx_list.py:37(__getitem__)

Let's better avoid using it in performance critical parts.

Change-Id: I2bbc0a2af8218af0b9a02d8e16d4216cf602892a
2020-07-13 08:51:19 +00:00
Vadim Yanitskiy bb0155d0e7 trx_toolkit/burst_fwd.py: inherit trx list API from TRXList
Change-Id: I1c589888991add435d88517094c7b4a7db93cbae
2020-07-13 08:51:19 +00:00
Pau Espin 3ca971eb2e fake_trx: Support SETPOWER and NOMTXPOWER TRXC cmds
By default RSSI on the Rx side is computed based on transmitter's
tx power and then substracting the the Rx path loss.
If FAKE_RSSI is used, then the values in there are used instead.

A default hardcoded value of tx nominal power = 50 dBm is set to keep
old behavior of RSSI=-60dB after calculations.

Change-Id: I3ee1a32ca22c3272e66b3ca78e4f67d283844c80
2020-06-03 20:22:59 +02:00
Vadim Yanitskiy cab00398c2 trx_toolkit: cosmetic: get rid of 'i' where it is not used
Change-Id: I00126a90446e5f3fb77a46be9d7d5dbff89fa221
2020-05-22 18:48:23 +07:00
Vadim Yanitskiy 0a6e083e8a trx_toolkit/data_dump.py: fix return value of parse_msg()
Jenkins build #2516 has uncovered a problem in DATADumpFile.parse_msg():

  ======================================================================
  FAIL: test_parse_empty (test_data_dump.DATADump_Test)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/build/src/target/trx_toolkit/test_data_dump.py",
         line 138, in test_parse_empty
      self.assertEqual(msg, False)
  AssertionError: None != False

I did a quick investigation, and figured out that this failure
happens when trying to call parse_msg() with idx == 0, because
DATADumpFile._seek2msg() basically does nothing in this case
and thus always returns True. The None itself comes from
DATADumpFile._parse_msg().

Let's ensure that DATADumpFile.parse_msg() always returns None,
even if DATADumpFile._seek2msg() fails. Also, update the unit
test, so we always test a wide range of 'idx' values.

Change-Id: Ifcfa9c5208636a0f9309f5ba8e47d282dc6a03f4
2020-05-22 18:22:32 +07:00
Vadim Yanitskiy 7ec1c1ccc8 trx_toolkit/transceiver.py: add frequency hopping support
There are two ways to implement frequency hopping:

  a) The Transceiver is configured with the hopping parameters, in
     particular HSN, MAIO, and the list of ARFCNs (channels), so the
     actual Rx/Tx frequencies are changed by the Transceiver itself
     depending on the current TDMA frame number.

  b) The L1 maintains several Transceivers (two or more), so each
     instance is assigned one dedicated RF carrier frequency, and
     hence the number of available hopping frequencies is equal to
     the number of Transceivers. In this case, it's the task of
     the L1 to commutate bursts between Transceivers (frequencies).

Variant a) is commonly known as "synthesizer frequency hopping"
whereas b) is known as "baseband frequency hopping".

For the MS side, a) is preferred, because a phone usually has only
one Transceiver (per RAT). On the other hand, b) is more suitable
for the BTS side, because it's relatively easy to implement and
there is no technical limitation on the amount of Transceivers.

FakeTRX obviously does support b) since multi-TRX feature has been
implemented, as well as a) by resolving UL/DL frequencies using a
preconfigured (by the L1) set of the hopping parameters. The later
can be enabled using the SETFH control command:

  CMD SETFH <HSN> <MAIO> <RXF1> <TXF1> [... <RXFN> <TXFN>]

where <RXFN> and <TXFN> is a pair of Rx/Tx frequencies (in kHz)
corresponding to one ARFCN the Mobile Allocation. Note that the
channel list is expected to be sorted in ascending order.

NOTE: in the current implementation, mode a) applies to the whole
Transceiver and all its timeslots, so using in for the BTS side
does not make any sense (imagine BCCH hopping together with DCCH).

Change-Id: I587e4f5da67c7b7f28e010ed46b24622c31a3fdd
2020-05-17 14:36:12 +07:00
Vadim Yanitskiy 220e60184f trx_toolkit/gsm_shared.py: implement hopping sequence generation
Based on firmware/layer1/rfch.c:rfch_hop_seq_gen() by Sylvain Munaut.

Change-Id: I9ecabfef6f5a4e4180956c6a019c386ccb1c9acd
2020-05-17 07:26:07 +00:00
Vadim Yanitskiy 2db781a5d7 trx_toolkit/rand_burst_gen.py: use list comprehension
See previous commit, TL;DR this approach is significantly faster.

Change-Id: I5dc0dda89443d2763bfae50cc402724935cc91b3
2020-05-16 20:17:18 +00:00
Vadim Yanitskiy 86b621b36b trx_toolkit/data_msg.py: use list comprehension for bit conversion
This approach is much better than buf.append() in terms of performance.
Consider the following bit conversion benchmark code:

  usbits = [random.randint(0, 254) for i in range(GSM_BURST_LEN)]
  ubits = [int(b > 128) for b in usbits]

  for i in range(100000):
      sbits = DATAMSG.usbit2sbit(usbits)
      assert(DATAMSG.sbit2usbit(sbits) == usbits)

      sbits = DATAMSG.ubit2sbit(ubits)
      assert(DATAMSG.sbit2ubit(sbits) == ubits)

=== Before this patch:

 59603795 function calls (59603761 primitive calls) in 11.357 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
 59200093    3.389    0.000    3.389    0.000 {method 'append' of 'list' objects}
   100000    2.212    0.000    3.062    0.000 data_msg.py:191(usbit2sbit)
   100000    1.920    0.000    2.762    0.000 data_msg.py:214(sbit2ubit)
   100000    1.835    0.000    2.677    0.000 data_msg.py:204(sbit2usbit)
   100000    1.760    0.000    2.613    0.000 data_msg.py:224(ubit2sbit)

=== After this patch:

  803794 function calls (803760 primitive calls) in 3.547 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   100000    1.284    0.000    1.284    0.000 data_msg.py:203(<listcomp>)
   100000    0.864    0.000    0.864    0.000 data_msg.py:193(<listcomp>)
   100000    0.523    0.000    0.523    0.000 data_msg.py:198(<listcomp>)
   100000    0.500    0.000    0.500    0.000 data_msg.py:208(<listcomp>)
        1    0.237    0.237    3.547    3.547 data_msg.py:25(<module>)
   100000    0.035    0.000    0.899    0.000 data_msg.py:191(usbit2sbit)
   100000    0.035    0.000    0.558    0.000 data_msg.py:196(sbit2usbit)
   100000    0.033    0.000    0.533    0.000 data_msg.py:206(ubit2sbit)
   100000    0.033    0.000    1.317    0.000 data_msg.py:201(sbit2ubit)

So the new implementation is ~70% faster in this case, and takes
significantly less function calls according to cProfile [1].

[1] https://docs.python.org/3.8/library/profile.html

Change-Id: I01c07160064c8107e5db7d913ac6dec6fc419945
2020-05-16 20:17:18 +00:00
Vadim Yanitskiy 75ae23674c trx_toolkit/app_common: add options to enable time printing
Change-Id: Ie5d14a261e17af554f7132b03d58549a4831dcdb
2020-04-09 04:44:49 +07:00
Vadim Yanitskiy ec71203e79 trx_toolkit/app_common: introduce auxiliary add_log_handler()
Change-Id: Ied32764cf1c34dc7e0f746f4f085ea20168775cb
2020-04-09 04:42:45 +07:00
Vadim Yanitskiy 930d7240bf trx_toolkit/trx_sniff.py: add options to filter bursts by RSSI
Change-Id: I16dd29d2f1e14e634029195599fa49a9be9219ab
2020-03-30 19:22:47 +07:00
Vadim Yanitskiy b1f0772002 trx_toolkit/trx_sniff.py: add option to ignore NOPE / IDLE indications
Change-Id: If51052af04289f10bfaefd5374049908de05319a
2020-03-30 19:20:32 +07:00
Vadim Yanitskiy bbd1edccae trx_toolkit/trx_sniff.py: pass the whole msg to burst_pass_filter()
Change-Id: I3da62249a4d62078b79ce5e79c86923e59c1e457
2020-03-30 19:00:45 +07:00
Vadim Yanitskiy 7a49fc31c1 trx_toolkit/data_dump.py: rewrite unit tests to use unittest framework
Change-Id: I8b934c15ba96d856aa79d10bf296d1446f043dd1
2019-12-31 17:03:12 +00:00
Vadim Yanitskiy 5cee398277 trx_toolkit/data_msg.py: rewrite unit tests to use unittest framework
Change-Id: Ia0cc7447b193a705e994078d16f3902339219916
2019-12-31 17:03:12 +00:00