Commit Graph

144 Commits

Author SHA1 Message Date
Vadim Yanitskiy 2db781a5d7 trx_toolkit/rand_burst_gen.py: use list comprehension
See previous commit, TL;DR this approach is significantly faster.

Change-Id: I5dc0dda89443d2763bfae50cc402724935cc91b3
2020-05-16 20:17:18 +00:00
Vadim Yanitskiy 86b621b36b trx_toolkit/data_msg.py: use list comprehension for bit conversion
This approach is much better than buf.append() in terms of performance.
Consider the following bit conversion benchmark code:

  usbits = [random.randint(0, 254) for i in range(GSM_BURST_LEN)]
  ubits = [int(b > 128) for b in usbits]

  for i in range(100000):
      sbits = DATAMSG.usbit2sbit(usbits)
      assert(DATAMSG.sbit2usbit(sbits) == usbits)

      sbits = DATAMSG.ubit2sbit(ubits)
      assert(DATAMSG.sbit2ubit(sbits) == ubits)

=== Before this patch:

 59603795 function calls (59603761 primitive calls) in 11.357 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
 59200093    3.389    0.000    3.389    0.000 {method 'append' of 'list' objects}
   100000    2.212    0.000    3.062    0.000 data_msg.py:191(usbit2sbit)
   100000    1.920    0.000    2.762    0.000 data_msg.py:214(sbit2ubit)
   100000    1.835    0.000    2.677    0.000 data_msg.py:204(sbit2usbit)
   100000    1.760    0.000    2.613    0.000 data_msg.py:224(ubit2sbit)

=== After this patch:

  803794 function calls (803760 primitive calls) in 3.547 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   100000    1.284    0.000    1.284    0.000 data_msg.py:203(<listcomp>)
   100000    0.864    0.000    0.864    0.000 data_msg.py:193(<listcomp>)
   100000    0.523    0.000    0.523    0.000 data_msg.py:198(<listcomp>)
   100000    0.500    0.000    0.500    0.000 data_msg.py:208(<listcomp>)
        1    0.237    0.237    3.547    3.547 data_msg.py:25(<module>)
   100000    0.035    0.000    0.899    0.000 data_msg.py:191(usbit2sbit)
   100000    0.035    0.000    0.558    0.000 data_msg.py:196(sbit2usbit)
   100000    0.033    0.000    0.533    0.000 data_msg.py:206(ubit2sbit)
   100000    0.033    0.000    1.317    0.000 data_msg.py:201(sbit2ubit)

So the new implementation is ~70% faster in this case, and takes
significantly less function calls according to cProfile [1].

[1] https://docs.python.org/3.8/library/profile.html

Change-Id: I01c07160064c8107e5db7d913ac6dec6fc419945
2020-05-16 20:17:18 +00:00
Vadim Yanitskiy 75ae23674c trx_toolkit/app_common: add options to enable time printing
Change-Id: Ie5d14a261e17af554f7132b03d58549a4831dcdb
2020-04-09 04:44:49 +07:00
Vadim Yanitskiy ec71203e79 trx_toolkit/app_common: introduce auxiliary add_log_handler()
Change-Id: Ied32764cf1c34dc7e0f746f4f085ea20168775cb
2020-04-09 04:42:45 +07:00
Vadim Yanitskiy 930d7240bf trx_toolkit/trx_sniff.py: add options to filter bursts by RSSI
Change-Id: I16dd29d2f1e14e634029195599fa49a9be9219ab
2020-03-30 19:22:47 +07:00
Vadim Yanitskiy b1f0772002 trx_toolkit/trx_sniff.py: add option to ignore NOPE / IDLE indications
Change-Id: If51052af04289f10bfaefd5374049908de05319a
2020-03-30 19:20:32 +07:00
Vadim Yanitskiy bbd1edccae trx_toolkit/trx_sniff.py: pass the whole msg to burst_pass_filter()
Change-Id: I3da62249a4d62078b79ce5e79c86923e59c1e457
2020-03-30 19:00:45 +07:00
Vadim Yanitskiy 7a49fc31c1 trx_toolkit/data_dump.py: rewrite unit tests to use unittest framework
Change-Id: I8b934c15ba96d856aa79d10bf296d1446f043dd1
2019-12-31 17:03:12 +00:00
Vadim Yanitskiy 5cee398277 trx_toolkit/data_msg.py: rewrite unit tests to use unittest framework
Change-Id: Ia0cc7447b193a705e994078d16f3902339219916
2019-12-31 17:03:12 +00:00
Vadim Yanitskiy a15fbcbdf3 trx_toolkit/clck_gen.py: drop unneeded debug print()
Change-Id: I372af77d5b0b24fa38e304b782ca5b3d0888211b
2019-11-24 02:24:42 +07:00
Vadim Yanitskiy 6b0946ee69 trx_toolkit/clck_gen.py: turn CLCKGen's thread into a daemon
If the main thread crashes, the CLCKGen's thread would never stop.
It would also happen if the main thread terminates without calling
CLCKGen.stop().  Let's prevent this by creating a daemon thread.

Change-Id: I9d41c5baa25fa0a263758414a164c1bded25e04e
2019-11-24 02:24:42 +07:00
Vadim Yanitskiy baf07c4be2 trx_toolkit/clck_gen.py: refactor CLCKGen to use a single thread
The previous approach was based on threading.Timer, so on each clock
iteration one thread spawned another new thread.  So far it worked
well, but such frequent spawning involves an additional overhead.

After this change, CLCKGen.start() allocates and starts a new thread,
that periodically sends clock indications and sleep()s during the
indication intervals.  The CLCKGen.stop() in its turn terminates
that thread and frees the memory.

Change-Id: Ibe477eb0a1ee2193c1ff16452a407be7e858b2ef
2019-11-24 02:24:34 +07:00
Vadim Yanitskiy ae8e5ad648 trx_toolkit/fake_trx.py: send NOPE.ind in case of path loss simulation
Since TRXD header version 1, we should send NOPE indications to the
L1 side in absence of TRX2L1 bursts, and IDLE indications during
IDLE TDMA frames (basically noise measurements).

This change is the first step towards the goal: if a given burst
is to be dropped due to the path loss simulation (see FAKE_DROP),
mark the carrier TRX2L1 message as NOPE.ind and send anyway.

Change-Id: Iabd0af665e3108d23a908638f943a5b689986e2c
Related: OS#3428, OS#2975
2019-11-20 15:05:57 +00:00
Vadim Yanitskiy 7108c28c38 trx_toolkit/data_msg.py: fix: NOPE.ind also contains C/I filed
Change-Id: I281fb7387a83fec7e097ebf8650c95713d3f70e9
2019-11-20 15:05:57 +00:00
Vadim Yanitskiy b1ae186c55 trx_toolkit/fake_trx.py: refactor L12TRX -> TRX2L1 burst transformation
The burst transformation in BurstForwarder.forward_msg() used to be
done only once, so then the resulting message was distributed over
the list of connected (and active) transceivers.

This approach limits the path loss simulation capabilities, because
a reference to the same message is passed to FakeTRX.send_data_msg().
If one transceiver changes (or removes) the burst bits, the other
transceivers would not receive the original message.

Let's do the transformation individually for each transceiver,
so the original message will always remain unchanged.

Change-Id: Ia016a3a9bb6e9f17182a7168aa5a501ae9b9978b
2019-11-20 15:05:57 +00:00
Vadim Yanitskiy 2605d96720 trx_toolkit: fix: do not use 'is' / 'is not' with string and numerical literals
Since version 3.8, Python warnins us that using the "is" and "is not"
operators with string and numerical literals is a bad idea. Let's
avoid this and use the classical '==' and '!=' operators instead.

Change-Id: Iaed86d630ac1e0b9b4f72bbf3c788e325783456d
Bug description: https://bugs.python.org/issue34850
2019-11-17 17:46:48 +07:00
Martin Hauke 1f7a2ab5d3 Fix common misspellings and typos
Change-Id: I962b42871693f33b1054d43d195817e9cd84bb64
2019-10-17 08:07:39 +00:00
Vadim Yanitskiy 5afc0e4cb0 trx_toolkit/data_msg.py: enrich some ValueError messages
Change-Id: Icdf0d136a9b820cfaec534e2604204da9ea42092
2019-08-29 18:02:04 +02:00
Vadim Yanitskiy 171773b3e8 trx_toolkit/data_msg.py: also print burst length in desc_hdr()
Change-Id: Ia0036cbf70a3736a7f9779e030e05221cba8add9
2019-08-29 18:02:00 +02:00
Vadim Yanitskiy 59903772fa trx_toolkit/data_msg.py: fix: make sure header version is known
Before using DATA_MSG.HDR_LEN, we need to make sure that a parsed
header version is known and supported. Otherwise we will get an
IndexError exception.

Change-Id: Ie1887aa8709da1a2a287aa58a7873e72c0b4ed33
2019-08-29 15:09:40 +02:00
Vadim Yanitskiy df86074abe trx_toolkit/data_msg.py: fix message length check in parse_msg()
Unlike DATA_MSG.HDR_LEN, the CHDR_LEN is a constant that defines
length of the common header, which is mandatory for every version.
DATA_MSG.HDR_LEN in its turn defines length of the whole header,
including the version specific fields. Thus we need to know the
header version before using it.

In DATA_MSG.parse_msg() we need to parse the common header first,
so then we know the version and length of the whole header. After
that we can safely use DATA_MSG.HDR_LEN.

Change-Id: I2809f5f96209eed64bdabf7a15575144313f7cc9
2019-08-29 15:04:28 +02:00
Vadim Yanitskiy 5fed799df3 trx_toolkit/trx_sniff.py: also print header for incorrect messages
Change-Id: I376f1e7af46750e53305109cf4a9f64427b9960c
2019-08-27 21:13:31 +02:00
Vadim Yanitskiy 83a41a63b2 trx_toolkit/trx_sniff.py: also validate() sniffed messages
Change-Id: Ica3e4702fdabfdae0c1025b650ff32c8e85a6edf
2019-08-27 21:06:58 +02:00
Vadim Yanitskiy 78199f4f7b trx_toolkit/trx_sniff.py: print exact message parsing error
For sure, the following message is much more informative:

  Ignoring an incorrect message: Unhandled version 12

than:

  Failed to parse message, dropping...

NOTE: since the way of printing exceptions is different in both
Python versions, I had to drop Python 2 support.

Change-Id: I5fb02ce508c58ff94e47accc0ed655939eb53062
2019-08-27 21:04:47 +02:00
Vadim Yanitskiy 6af5d8da2c trx_toolkit/data_msg.py: raise exceptions from validate() methods
Raising exceptions is a Pythonic way to handle errors, which in this
particular case will help us to know *why* exactly a given message
is incorrect or incomplete.

Change-Id: Ia961f83c717066af61699c80536468392b8ce064
2019-08-27 20:30:23 +02:00
Pau Espin 2cc3392706 trx_toolkit: Fixes in TRXD documentation
Change-Id: I126a8aed6b2bac7a620e95f06ecb98642a63b5f0
2019-08-05 16:24:40 +02:00
Vadim Yanitskiy fe981d65da trx_toolkit/trx_sniff.py: support additional capture filter
Change-Id: I761debb2e1c411f2c6d489eac0adf32060966a4c
2019-07-24 05:44:19 +07:00
Vadim Yanitskiy 710503bc4c trx_toolkit/trx_sniff.py: support sniffing on multiple ports
Change-Id: I76b818c673b98c427b5621ddb852f947c74557d6
2019-07-24 05:44:19 +07:00
Vadim Yanitskiy e92c9cd923 trx_toolkit/trx_sniff.py: add support for reading from PCAP file
Change-Id: I8a3481fbea5f6c917ae5684d0b5b806f4a76ff78
2019-07-24 05:44:19 +07:00
Vadim Yanitskiy 7cde195e30 trx_toolkit/burst_gen.py: basic TRXD header version 1 support
Change-Id: Icd4505d211816dc80e91d65094be92f2aed856bd
2019-07-24 05:44:19 +07:00
Vadim Yanitskiy 88e1620502 trx_toolkit: fix compatibility with Python 3: 'is' vs '=='
Change-Id: I8ba05c04f206578cf61df58573c24cba1d6fba52
2019-07-24 05:44:19 +07:00
Vadim Yanitskiy d206f0cc2f trx_toolkit/rand_burst_gen.py: fix compatibility with Python 3
Change-Id: I362881af83664f94be09d40f43dfb23d18a35136
2019-07-24 05:44:19 +07:00
Vadim Yanitskiy 06e3b4fa54 trx_toolkit/trx_sniff.py: fix compatibility with Python 3
Change-Id: I9cd9b7baae31045c6495b90df2517d32772098ed
2019-07-24 02:05:35 +07:00
Alexander Couzens 998ed796c3 target/*.py: shebang: use /usr/bin/env python
Use the system default python instead of a hardcoded python2
Allow to use python2 and python3.

Change-Id: Iab185759b574eff1ca1b189dcbb4e1a3eec52132
2019-07-16 22:46:12 +00:00
Vadim Yanitskiy 3d9a1d6e48 trx_toolkit/fake_trx.py: introduce a TRXC command for C/I simulation
C/I (Carrier-to-Interference ratio) is a value in cB (centiBels),
computed from the training sequence of each received burst,
by comparing the "ideal" training sequence with the received one.

This change introduces a new command similar to FAKE_TOA and FAKE_RSSI,
so it can be used by TTCN-3 test case 'TC_pcu_data_ind_lqual_cb' to
verify that the link quality measurements are delivered to the PCU.

Change-Id: I7080effbbc1022d1884c6d6f0cb580eba8e514ff
Related: OS#1855
2019-07-16 14:52:24 +07:00
Vadim Yanitskiy 13ec32d380 trx_toolkit/ctrl_if_trx.py: implement TRXD header version negotiation
Messages on DATA interface may have different header formats, defined
by a version number, which can be negotiated on the control interface.
By default, the Transceiver will use the legacy header version (0).

The header format negotiation can be initiated by the L1 using the
'SETFORMAT' command. If the requested version is not supported by
the transceiver, status code of the response message should indicate
a preferred (basically, the latest) version. The format of this
message is the following:

  L1 -> TRX: CMD SETFORMAT VER_REQ
  L1 <- TRX: RSP SETFORMAT VER_RSP VER_REQ

where:

  - VER_REQ is the requested version (suggested by the L1),
  - VER_RSP is either the applied version if matches VER_REQ,
    or a preferred version if VER_REQ is not supported.

If the transceiver indicates VER_RSP different than VER_REQ, the L1
is supposed to reinitiate the version negotiation using the suggested
VER_RSP. For example:

  L1 -> TRX: CMD SETFORMAT 2
  L1 <- TRX: RSP SETFORMAT 1 2

  L1 -> TRX: CMD SETFORMAT 1
  L1 <- TRX: RSP SETFORMAT 1 1

If no suitable VER_RSP is found, or the VER_REQ is incorrect,
the status code in the response shall be -1.

As soon as VER_RSP matches VER_REQ in the response, the process
of negotiation is complete. Changing the header version is
supposed to be done before POWERON, but can be also done after.

Change-Id: I8d441b2559863d2dbd680db371062e4f3a2f9ff9
Related: OS#4006
2019-07-16 14:52:24 +07:00
Vadim Yanitskiy ebf676597b trx_toolkit/fake_trx.py: basic TRXD version 0x01 support
Since the new TRXD header format has been introduced, FakeTRX needs
to be able to fill it correctly. In particular, the following:

  - Modulation, which can be determined from the burst length;
  - Training Sequence Code (and set), which needs to be detected
    by comparing the burst bits of L12TRX message against known
    training sequences (only GMSK and the default TS set for now);
  - C/I (Carrier-to-Interference ratio), which can be simulated
    later on, as instructed on the TRXC interface ('FAKE_CI').

The actual TRXD header version is stored in the instance of class
DATAInterface. By default (at startup), legacy version 0 is used.
The version negotiation is supposed to be performed on the TRXC
interface, and to be implemented in a follow-up change.

Different Transceivers may use different header versions, thus in
FakeTRX.send_data_msg() we need to override the original version
of the L12TRX message, and generate the corresponding PDU.

Limitations:

  - NOPE / IDLE indications are not (yet) supported;
  - TSC detection: GMSK modulation only.

Change-Id: I164f5ae4ce7694d6e324aab927a04e96d489ebd8
Related: OS#4006
2019-07-16 04:07:34 +00:00
Vadim Yanitskiy 6780e47b6c trx_toolkit/rand_burst_gen.py: use TrainingSeqGMSK and BurstType enums
Change-Id: I8a3faceae4a8d9b57d86d42600db839da073dad6
2019-07-16 04:07:34 +00:00
Vadim Yanitskiy dfa43a3892 trx_toolkit/gsm_shared.py: introduce a new enum TrainingSeqGMSK
Training Sequences are defined in 3GPP TS 45.002, and used by the
transceiver for detecting bursts. This change introduces an enum
with training sequences for GMSK for Access and Normal bursts.

This enumeration is needed for the follow-up changes that implement
TRXD header version 1 support, and can now be used by RandBurstGen.

Change-Id: If3bf102019ef53d6ee9ad230ef98bb45845b5af5
2019-07-16 04:07:34 +00:00
Vadim Yanitskiy 62825cc8eb trx_toolkit/data_msg.py: legacy flag is only for version 0x00
Since version 0x01, the burst bits are encoded as L16V,
so appending two dummy octets doesn't make sense.

Change-Id: I4d6c0bf54649d636ea6cb3fa2f37486b6619d5b3
2019-07-02 13:48:06 +07:00
Vadim Yanitskiy dce8a24a2c trx_toolkit/data_msg.py: introduce header coding version 0x01
The new version adds the following fields to the TRX2L1 message,
keeping the L12TRX message unchanged:

  +------+-----+-----+-----+--------------------+
  | RSSI | ToA | MTS | C/I | soft-bits (254..0) |
  +------+-----+-----+-----+--------------------+

  - MTS (1 octet)  - Modulation and Training Sequence info, and
  - C/I (2 octets) - Carrier-to-Interference ratio (big endian).

== Coding of MTS: Modulation and Training Sequence info

3GPP TS 45.002 version 15.1.0 defines several modulation types,
and a few sets of training sequences for each type. The most
common are GMSK and 8-PSK (which is used in EDGE).

  +-----------------+---------------------------------------+
  | 7 6 5 4 3 2 1 0 | bit numbers (value range)             |
  +-----------------+---------------------------------------+
  | . . . . . X X X | Training Sequence Code (0..7)         |
  +-----------------+---------------------------------------+
  | . X X X X . . . | Modulation, TS set number (see below) |
  +-----------------+---------------------------------------+
  | X . . . . . . . | IDLE / nope frame indication (0 or 1) |
  +-----------------+---------------------------------------+

The bit number 7 (MSB) is set to high when either nothing has been
detected, or during IDLE frames, so we can deliver noise levels,
and avoid clock gaps on the L1 side. Other bits are ignored,
and should be set to low (0) in this case.

== Coding of modulation and TS set number

GMSK has 4 sets of training sequences (see tables 5.2.3a-d),
while 8-PSK (see tables 5.2.3f-g) and the others have 2 sets.
Access and Synchronization bursts also have several synch.
sequences.

  +-----------------+---------------------------------------+
  | 7 6 5 4 3 2 1 0 | bit numbers (value range)             |
  +-----------------+---------------------------------------+
  | . 0 0 X X . . . | GMSK, 4 TS sets (0..3)                |
  +-----------------+---------------------------------------+
  | . 0 1 0 X . . . | 8-PSK, 2 TS sets (0..1)               |
  +-----------------+---------------------------------------+
  | . 0 1 1 X . . . | AQPSK, 2 TS sets (0..1)               |
  +-----------------+---------------------------------------+
  | . 1 0 0 X . . . | 16QAM, 2 TS sets (0..1)               |
  +-----------------+---------------------------------------+
  | . 1 0 1 X . . . | 32QAM, 2 TS sets (0..1)               |
  +-----------------+---------------------------------------+
  | . 1 1 1 X . . . | RESERVED (0)                          |
  +-----------------+---------------------------------------+

== C/I: Carrier-to-Interference ratio

The C/I value is computed from the training sequence of each burst,
where we can compare the "ideal" training sequence with the actual
training sequence, and then express that difference in centiBels.

Change-Id: Ie810c5a482d1c908994e8cdd32a2ea641ae7cedd
Related: OS#4006, OS#1855
2019-07-02 13:46:10 +07:00
Vadim Yanitskiy af60c1bb10 trx_toolkit/data_msg.py: implement header version coding
It may be necessary to extend the message specific header with
more information. Since this is not a TLV-based protocol, we
need to include the header format version.

  +-----------------+------------------------+
  | 7 6 5 4 3 2 1 0 | bit numbers            |
  +-----------------+------------------------+
  | X X X X . . . . | header version (0..15) |
  +-----------------+------------------------+
  | . . . . . X X X | TDMA TN (0..7)         |
  +-----------------+------------------------+
  | . . . . X . . . | RESERVED (0)           |
  +-----------------+------------------------+

Instead of prepending an additional byte, it was decided to use
4 MSB bits of the first octet, which used to be zero-initialized
due to the value range of TDMA TN. Therefore, the current header
format has implicit version 0x00.

Otherwise Wireshark (or trx_sniff.py) would need to guess the
header version, or alternatively follow the control channel
looking for the version setting command.

The reserved bit number 3 can be used in the future to extend
the TDMA TN range to (0..15), in case anybody would need
to transfer UMTS bursts.

Change-Id: Idb0377d66290eb9c15d6998a5806a84fa2e5dd02
Related: OS#4006
2019-07-01 20:05:48 +07:00
Vadim Yanitskiy 851d35c675 trx_toolkit/data_msg.py: add burst randomization helpers
Change-Id: Idf1393d3d1f04e6c60b356b797a18e6f77b23554
2019-07-01 17:39:45 +07:00
Vadim Yanitskiy 7cb120e841 trx_toolkit/data_msg.py: mark bit conversion methods as @staticmethod
Change-Id: I2f9fdd514908f186b1c6c043ee9b31c27a396900
2019-07-01 17:37:00 +07:00
Vadim Yanitskiy 696fe71ac1 trx_toolkit/data_msg.py: add basic class documentation
Change-Id: I538bc96e5e24d3b7e344e4dbe2877bf60c13c720
Related# OS#4006
2019-06-24 14:18:09 +00:00
Vadim Yanitskiy 1ff9780366 trx_toolkit/data_msg.py: inline both gen_fn() and parse_fn()
Both functions are never used outside of both gen_msg() and parse_msg().
AFAIR, they were more complicated until we started to use struct, but
now they can be easily inlined.

Change-Id: Ie64b271cf502f3df23b32f4b14a1e2b551a0f794
2019-06-24 14:18:09 +00:00
Vadim Yanitskiy 84bd7b2746 trx_toolkit/data_msg.py: drop double field initialization
Those fields are being initialized by __init__().

Change-Id: Ibf71be552a7eb5dab1d096421a8557514294683e
2019-06-24 14:18:09 +00:00
Vadim Yanitskiy 7a007e82a8 trx_toolkit/data_msg.py: tests: use random reference data
Having fn = 1024 and tn = 0 in all tests decreases the chances
to spot encoding / decoding bugs of higher or lower values.
Let's randomize the reference data before all the tests.

Change-Id: Id3c5be9faaf0bef727b975c7182098af0cec6e71
2019-06-24 14:18:09 +00:00
Vadim Yanitskiy 6e9808662f trx_toolkit/data_msg.py: fix: extend RSSI value range to [-47..-120]
Change-Id: I9fce3462db14bd273a1498762cc9293fc888b45a
2019-06-01 01:42:43 +07:00
Vadim Yanitskiy ab7118a2aa trx_toolkit/clck_gen.py: avoid logging \0-terminator
Change-Id: I93da2e8ba9d3fda944b8171bc42e49063c925f9c
2019-05-17 19:40:31 +07:00
Vadim Yanitskiy cca67cfe94 trx_toolkit/data_msg.py: fix TOA256 MIN/MAX constants
The old TOA256 range was bigger than we can actually store:

  struct.error: 'h' format requires -32768 <= number <= 32767

Change-Id: I5d4e1fea9d07f2c49f01e6644d1c0d1dc8cf4e40
2019-04-24 15:12:03 +07:00
Vadim Yanitskiy 59d20f6d5e trx_toolkit/README: update fake_trx.py description
Since Ibff31fb3a958a714c828d0dea7e87d47f778fd80, fake_trx.py does
support multiple transceivers. Let's update its description.

Change-Id: I6e4351693da3a1f7e3eadd8e11971c34044dde20
2019-01-14 23:52:31 +00:00
Vadim Yanitskiy c09894924a trx_toolkit/transceiver.py: add optional transceiver name
Since fake_trx.py can handle multiple transceivers, it may be useful
to name transceivers. If transceiver has some name, it will appear
in logging messages, for example:

  [INFO] transceiver.py:104 Init transceiver 'BTS@127.0.0.1:5700'
  [INFO] transceiver.py:104 Init transceiver 'MS@127.0.0.1:6700'
  [INFO] transceiver.py:104 Init transceiver '127.0.0.1:5700/1'

This change additionally assigns names to the both default
transceivers, and extends the '--trx' option with ability
to specify some name, for example:

  --trx foo@127.0.0.1:5700 or --trx bar@127.0.0.1:5700/1
  --trx ipv6@[2001:0db8:85a3:0000:0000:8a2e:0370:7334]:6700

Change-Id: I2f58f02e7819bb008b8aab1a8bf9e0adeb2e44ec
2019-01-14 23:52:31 +00:00
Vadim Yanitskiy 21d99e65fb trx_toolkit/transceiver.py: add init logging message
Change-Id: I9a8b327748fbfcbae438cb531cd0b2767b6ca4dd
2019-01-14 23:52:31 +00:00
Vadim Yanitskiy 95025a2389 trx_toolkit: use 'TRXC' and 'TRXD' in logging messages
One can confuse TRX control interface with libosmoctrl's one.
TRX toolkit is not using libosmoctrl, and will never do. But,
in order to avoid this confusion, and potential confusion of
DATA interface, let's call them 'TRXC' and 'TRXD' in logging.

Change-Id: I67b1e850094cf8e279777c45c7544886be42a009
2019-01-14 23:52:31 +00:00
Vadim Yanitskiy 6b040c6e06 trx_toolkit/{ctrl,data}_if.py: add init log message
Change-Id: I1a2caf6999ed4f33df76328e48ff5076d166d9fe
2019-01-14 23:52:31 +00:00
Vadim Yanitskiy ba28c3efc1 trx_toolkit/fake_trx.py: enrich TRX logging messages
Since fake_trx.py can handle multiple transceivers, it makes sense
to print some info in logging messages about transceivers they
belong to. This acvieved by defining __str__() for Transceiver.

Some examples:

  [DEBUG] ctrl_if_trx.py:83 (127.0.0.1:5700) Recv POWEROFF cmd
  [INFO] ctrl_if_trx.py:85 (127.0.0.1:5700) Stopping transceiver...

  [DEBUG] ctrl_if_trx.py:95 (127.0.0.1:5700/1) Recv RXTUNE cmd
  [DEBUG] ctrl_if_trx.py:102 (127.0.0.1:5700/1) Recv TXTUNE cmd
  [DEBUG] ctrl_if_trx.py:155 (127.0.0.1:5700/1) Ignore CMD SETTSC
  [DEBUG] ctrl_if_trx.py:155 (127.0.0.1:5700/1) Ignore CMD SETPOWER

Change-Id: I1f706790a2da226f1418f89d2cfbb55baa6ea624
2019-01-14 23:52:31 +00:00
Vadim Yanitskiy 71fee86391 trx_toolkit/fake_trx.py: move init code to __init__()
There should be no code in run() that does initialization,
__init__() is the best place for this. This change allows
to import the Application class from fake_trx.py, and
run it from script (e.g. for testing).

Change-Id: I84969630348a189d237cc98354e568421839a37b
2019-01-14 23:52:31 +00:00
Vadim Yanitskiy e4e3e6facd trx_toolkit/fake_trx.py: fix: update trx_list of FakePM
It seems in Ice44e2b22566b3652ef6d43896055963b13ab185 I forgot
to do this, so all measurements triggered by MEASURE command
were incorrect (always noise). Let's fix this!

Change-Id: I155f118b2d3e3b23eb148fe7e2630790f8fcd18c
2019-01-13 16:33:31 +07:00
Vadim Yanitskiy 786e97a0dc trx_toolkit/fake_trx.py: handle multiple transceivers
Since Ice44e2b22566b3652ef6d43896055963b13ab185 is merged, the class
hierarchy has become much more flexible, so it's possible to create
multiple Transceiver / FakeTRX instances and distribute bursts using
a single instance of BurstForwarder.

This change introduces a new command line option, that can be used
to specify additional transceivers. Please note that fake_trx.py
still initializes a pair of BTS and BB transceivers by default.

The new option has the following format:

  --trx REMOTE_ADDR:BASE_PORT[/TRX_NUM]

Some examples for IPv4 and IPv6:

  --trx 127.0.0.1:5703
  --trx [2001:0db8:85a3:0000:0000:8a2e:0370:7334]:6703

If optional TRX_NUM > 0 is specified, e.g.:

  --trx 127.0.0.1:5700/1

then this transceiver is considered as a child of another one.
See I7e97b7f32dde7ab74779133e9d7504f1d0fce60c for details.

Change-Id: Ibff31fb3a958a714c828d0dea7e87d47f778fd80
2019-01-13 16:33:12 +07:00
Vadim Yanitskiy 42bce2bd5b trx_toolkit/transceiver.py: add support for child transceivers
A BTS can (optionally) have more than one transceiver. In this case
additional (let's say child) transceivers basically share the same
clock source of the first transceiver, and being powered on / off
as soon as the first transceiver is powered on / off.

Change-Id: I7e97b7f32dde7ab74779133e9d7504f1d0fce60c
2019-01-13 16:05:27 +07:00
Vadim Yanitskiy 433b761390 trx_toolkit/burst_fwd.py: fix trx_list cross-reference
It was discovered that using an empty list as default argument
value does result into the cross-reference, i.e. all instances
of BurstForwarder would reference the same trx_list object.

This is not an expected behaviour, let's fix this.

Change-Id: Id71185de05b0ebc5adb105b10fad2cbde5f800b1
2019-01-13 15:39:27 +07:00
Vadim Yanitskiy 152a2da8d2 trx_toolkit/fake_trx.py: refactor global class hierarchy
This change is a big step towards handling of multiple transceivers
in a single process, i.e. multiple MS and multiple BTS connections.

The old class hierarchy wasn't flexible enough, because initially
fake_trx was designed as a bridge between OsmocomBB and OsmoBTS,
but not as the burst router. There were two separate, but 90%
similar implementations of the CTRL interface, two variations
of each simulation parameter - one for UL, another for DL.

The following new classes are introduced:

  - Transceiver - represents a single transceiver, that can be
    used as for the BTS side, as for the MS side. Each instance
    has its own CTRL, DATA, and (optionally) CLCK interfaces,
    among with basic state variables, such as both RX / TX freq.,
    power state (running or idle) and list of active timeslots.

  - CTRLInterfaceTRX - unified control interface handler for
    common transceiver management commands, such as POWERON,
    RXTUNE, and SETSLOT. Deprecates both CTRLInterface{BB|BTS}.

  - FakeTRX - basically, a child of Transceiver, extended with
    RF path (burst loss, RSSI, TA, ToA) simulation. Implements
    a custom CTRL command handler for CTRLInterfaceTRX.

The following classes were refactored:

  - BurstForwarder - still performs burst forwarding, but now
    it doesn't store any simulation parameters, and doesn't
    know who is BTS, and who is MS. Actually, BurstForwarder
    transforms each L12TRX message into a TRX2L1 message, and
    dispatches it between running transceivers with matching
    RX frequency and matching timeslot.

  - FakePM - still generates random RSSI values, but doesn't
    distinguish between MS and BTS anymore. As soon as a
    measurement request is received, it attempts to find
    at least one running TRX on a given frequency.

Please note that fake_trx.py still does handle only a single pair
of MS and BTS. No regressions have been observed. Both new and
refactored classes were documented.

Change-Id: Ice44e2b22566b3652ef6d43896055963b13ab185
Related: OS#3667
2018-12-18 05:47:11 +07:00
Vadim Yanitskiy 1197382e8d trx_toolkit/data_if.py: add optional legacy message coding flag
In I6b9a8b611ea1e9badc4d9ddf13aa9e237028e39a an optional legacy
message coding mode was introduced. Let's add the corresponding
argument to send_msg() and pass it to gen_msg().

Change-Id: I6b9a8b611ea1e9badc4d9ddf13aa9e237028e39a
2018-12-18 05:47:11 +07:00
Vadim Yanitskiy e4cddb5ef1 trx_toolkit/data_msg.py: add optional legacy message coding flag
Some transceivers (e.g. OsmoTRX) have inherited a rudiment from
OpenBTS - two dummy bytes at the end of TRX2L1 messages. Despite
they are absolutely useless, some L1 implementations, such as
trxcon and OpenBTS, still do expect them when checking
the message length.

Let's add an optional (disabled by default) argument to gen_msg(),
that would enable adding those two dummy bytes.

Change-Id: I0cf1314c399411886420176704cadd6e6d84787f
2018-12-18 05:47:11 +07:00
Vadim Yanitskiy 8b3d49bd34 trx_toolkit/data_msg.py: use struct API for fn and toa256 coding
The built-in struct module is already used for toa256 decoding,
so let's use it for toa256 encoding, and TDMA frame number
coding too - no need to (re)implement the wheel!

Change-Id: I10d2e15ac57a0524e9bc1c80ed6a0f6f5a263436
2018-12-18 05:47:11 +07:00
Vadim Yanitskiy 016361cda1 trx_toolkit/ctrl_if_bb.py: drop meaningless warnings
A timeslot can be reconfigured at runtime, this is normal.
We should neither complain nor reject such commands.

Change-Id: I0a69ebceed5aa724093e6d1b23faad8c16705055
2018-12-16 07:14:11 +07:00
Vadim Yanitskiy 2e32260040 trx_toolkit/burst_send.py: exit if DATA dump parsing failed
False is not iterable, so we should properly handle the case
when parsing of the whole DATA dump was failed (e.g. due to
incorrect offset specified).

Change-Id: I5443efb39bb9d3377290ce7ec5e34016cae0edb2
2018-12-16 07:14:11 +07:00
Vadim Yanitskiy 7dacdcbd09 trx_toolkit/burst_send.py: fix: use proper argparse group
Found using Flake8, F841 "local variable 'pf_group' is assigned
to but never used". The filtering related options should be
defined in 'pf_group' group, not in 'cnt_group'.

Change-Id: I15d17c134cbbbd54d761113a56c1f83910ab6407
2018-12-16 07:14:11 +07:00
Vadim Yanitskiy 1bea8faa7d trx_toolkit: fix missing / useless imports
Found using Flake8:

  - data_if.py:57:4: F405 'log' may be undefined, or defined
                     from star imports: data_msg
  - clck_gen.py:29:1: F401 'time' imported but unused
  - clck_gen.py:30:1: F401 'sys' imported but unused
  - trx_sniff.py:28:1: F401 'signal' imported but unused

Change-Id: Id0c42319b445db218b77fd5e99a9a0a89724281d
2018-12-16 07:14:11 +07:00
Vadim Yanitskiy 810b696b8d trx_toolkit/data_if.py: add message parsing methods
This change extends DATAInterface class with new methods:

  - recv_raw_data() - read raw data from socket;
  - recv_l12trx_msg() - read raw data and parse as L12TRX;
  - recv_trx2l1_msg() - read raw data and parse as TRX2L1;

which would simplify the further usage of this class.

Change-Id: I761c4e63864622d3882b8f9c80ea43b58f092cb1
2018-12-12 00:43:18 +07:00
Vadim Yanitskiy 1dcc45beff trx_toolkit/ctrl_if.py: read data from socket in handle_rx()
It makes much more sense to read data from socket in handle_rx(),
instead of expecting a buffer with received data from caller.

Change-Id: I83479c60c54e36a2a7582714a6043090585957ae
2018-12-11 23:58:18 +07:00
Vadim Yanitskiy 7e9821f289 trx_toolkit/ctrl_if_*.py: transparently pass UDPLink arguments
There is no need to (re)define the arguments of UDPLink's constructor.
Let's use non-keyworded variable length argument list (*args).

Change-Id: Ia312a5e15ce88d5f7e8d76c4ea8c93c59d91be5a
2018-12-11 23:47:13 +07:00
Vadim Yanitskiy 63be9da190 trx_toolkit/ctrl_if_bb.py: fix SETTA command handling
Since I8fd2a2ab7784b38bde5ebcfd0359b7e2cb53f5a7, SETTA command
handling was broken, because the range limitation was removed
together with argument parsing. Let's fix this.

Change-Id: If582af3849359866de129504cc5b2dc6d64edbd5
2018-12-10 05:57:46 +07:00
Vadim Yanitskiy 59debdcd07 trx_toolkit: merge copyright.py into app_common.py
Since we have introduced ApplicationBase class, that are used
by all existing applications, let's merge the copyright
printing helper into it.

Change-Id: I8b70ec2dd08cb2ffed733d2c4e1215b094f8d3d5
2018-12-08 02:10:50 +07:00
Vadim Yanitskiy cdf349bc64 trx_toolkit: introduce logging configuration arguments
Before this change, it was impossible to configure logging
parameters from command line, such as log level and format.

This change introduces the following optional arguments:

  --log-level - logging level for stderr (by default, DEBUG);
  --log-format - logging message format for stderr;

  --log-file-name - enable logging to a given file;
  --log-file-level - logging level for file (by default, DEBUG);
  --log-file-format - logging message format for file;

which are defined in a new class called ApplicationBase, so
all existing applications should inherit them now.

Change-Id: Ic3b0440cd73946ad444bd7e48feb7a92d45f6488
2018-12-08 02:10:20 +07:00
Vadim Yanitskiy 5cb7e287ed trx_toolkit/fake_trx.py: migrate from getopt to argparse
Change-Id: I0218a328671c591b20b0666dc7c39ba865892acc
2018-12-07 08:20:37 +07:00
Vadim Yanitskiy 060b2729d1 trx_toolkit/trx_sniff.py: migrate from getopt to argparse
Change-Id: Id1dacaa32134bfa68344e6c48310390cdd85cdc9
2018-12-07 08:08:49 +07:00
Vadim Yanitskiy 66b7e78eb1 trx_toolkit/ctrl_cmd.py: migrate from getopt to argparse
Change-Id: Ib9bf5ed7384c4f34478975cefe03880eb2fc8069
2018-12-07 08:08:45 +07:00
Vadim Yanitskiy 3fee14b5a3 trx_toolkit/burst_send.py: migrate from getopt to argparse
Change-Id: I1be66aa022a79aa1683f0e6cfebaed568b1736b1
2018-12-07 07:15:45 +07:00
Vadim Yanitskiy 51dd1dc703 trx_toolkit/burst_gen.py: migrate from getopt to argparse
Change-Id: I7eb3f2e2713f1f97293bd47a2eae3b140f63fb59
2018-12-07 06:52:59 +07:00
Vadim Yanitskiy 54a8c02a80 trx_toolkit/burst_gen.py: fix short version of --burst-type
Both '--bind-addr' and '--burst-type' had the same short '-b'.
Let's use the upper case version for '--burst-type'.

Change-Id: Ib8a46e25cbc6266c3e147582f9e8045362270151
2018-12-07 05:54:26 +07:00
Vadim Yanitskiy 6bab6acee6 trx_toolkit: use generic logging module instead of print()
There are multiple advantages of using Python's logging module:

  - advanced message formatting (file name, line number, etc.),
  - multiple logging targets (e.g. stderr, file, socket),
  - logging levels (e.g. DEBUG, INFO, ERROR),
  - the pythonic way ;)

so, let's replace multiple print() calls by logging calls,
add use the following logging message format by default:

  [%(levelname)s] %(filename)s:%(lineno)d %(message)s

Examples:

  [INFO] ctrl_if_bts.py:57 Starting transceiver...
  [DEBUG] clck_gen.py:87 IND CLOCK 26826
  [DEBUG] ctrl_if_bts.py:71 Recv POWEROFF cmd
  [INFO] ctrl_if_bts.py:73 Stopping transceiver...
  [INFO] fake_trx.py:127 Shutting down...

Please note that there is no way to filter messages by logging
level yet. This is to be introduced soon, together with argparse.

Change-Id: I7fcafabafe8323b58990997a47afdd48b6d1f357
2018-12-07 05:36:20 +07:00
Vadim Yanitskiy 72c8296bfe trx_toolkit/fake_trx.py: drop randomization related cmd options
The randomization of both UL/DL RSSI and ToA values is optional,
and can be configured from the control interface (see both
FAKE_RSSI and FAKE_TOA commands).

The command line options for enabling / disabling the randomization
were redundant, so let's get rid of them and check if the
corresponding treshold value is set.

Change-Id: I6adc13b8989ade2fab895673525c0ca17bf9b3f2
2018-12-07 03:06:54 +07:00
Vadim Yanitskiy 6c4d9b6284 trx_toolkit/burst_fwd.py: also pass-filter UL bursts by time-slot
For some reason, the time-slot pass-filtering was only done for
DL bursts, but not for UL bursts. BurstForwarder shall not pass
UL bursts for unconfigured time-slots too.

Let's also print a warning if an UL burst is sent on a not
configured time-slot, i.e. before sending SETSLOT command.

Change-Id: Idb7f5b212e5814aeff8ca8bc875ad066674267cd
2018-12-07 03:06:54 +07:00
Vadim Yanitskiy cadbce066b trx_toolkit/burst_fwd.py: properly pass-filter multiple time-slots
Previously it was only possible to configure a single time-slot
that would be pass-filtered by a BurstForwarder instance. In some
applications it would be useful to configure multiple time-slots,
so let's refactor the time-slot pass-filtering algorithm.

Change-Id: Ie1490adaf7a7c62c966aeb60c1898eaf3b5a1e84
2018-12-07 03:06:49 +07:00
Vadim Yanitskiy 8d70f9d9fe trx_toolkit/burst_fwd.py: document and refactor BurstForwarder
Instead of having all configuration variables of BurstForwarder
initialized in the class heading, let's introduce two functions
for initialization (resetting to defaults) of both UL/DL params.
This would allow to reset a BurstForwarder instance from the
control interface in follow-up patches.

Let's also introduce some basic documentation for the class
fields, which were defined in the heading previously.

Change-Id: I6b1bf41cf22f01a7e7ecc91c625fb0d2bf4bfeac
2018-12-06 07:42:07 +07:00
Pau Espin 5516f2cad3 fake_trx: introduce CTRL commands for RSSI simulation
Since FakeTRX is a proxy, it basically emulates transceiver for
both osmo-bts-trx and trxcon, hence there are two separate and
independent control interfaces (usually, ports 6701 and 5701).

All simulation commands (see 'FAKE_*') are usually implemented
on both interfaces separately:

  - ctrl_if_bb.py - simulation commands affecting Uplink,
  - ctrl_if_bts.py - simulation commands affecting Downlink.

This change introduces the 'FAKE_RSSI' command for both CTRL
interfaces in two variations:

  - absolute: CMD FAKE_RSSI <BASE> <THRESH>
  - relative: CMD FAKE_RSSI <+-BASE_DELTA>

where 'THRESH' affects optional value randomization.

Change-Id: Ic01c31fb0304345dd7337c3ee1c7ee3c2d3e8460
2018-10-23 00:00:13 +02:00
Vadim Yanitskiy 05d88069d4 trx_toolkit/fake_trx: drop Timing Advance range limitation
Despite the correct range of Timing Advance value is [0..63],
there is a special feature in OsmocomBB which allows one to
simulate the distance between both MS and a BTS by playing
with the signal delay.

So, let's drop the range limitation.

Change-Id: I8fd2a2ab7784b38bde5ebcfd0359b7e2cb53f5a7
2018-09-07 09:02:19 +00:00
Vadim Yanitskiy a50d3fff72 trx_toolkit/fake_trx: introduce basic path loss simulation
This change introduces a couple of new CTRL commands for path loss
simulation, in particular a possibility to drop some amount of
bursts according to some TDMA frame period, separately for both
Uplink and Downlink directions.

Examples:

  FAKE_DROP 4 - drop 4 consistent (period=1) bursts,
  FAKE_DROP 16 2 - drop 16 even bursts (period=2).

Change-Id: Ib210138a03e2377c79875a4ff2f2bb58a43cfa46
Related: OS#3428
2018-08-02 05:30:49 +07:00
Vadim Yanitskiy b914cfd488 trx_toolkit/burst_fwd.py: separate burst preprocessing
This change separates burst preprocessing (i.e. both RSSI and ToA
calculation) from BurstForwarder.transform_msg() because it's not
actually related to the message transformation process.

Change-Id: Ia7ad970593f38d9a9401975eb6dae67cd0c94e11
2018-08-02 03:50:57 +07:00
Vadim Yanitskiy fd33dcc202 trx_toolkit/trx_sniff.py: fix memleak: don't store packets
The Scapy itself was the actual cause of continuously growing
memory consumption. It was configured to store the captured
packets, what isn't required for this tool.

Change-Id: I0c6d9b76398e148b7febd94aa37aa2fa22d19b3f
2018-07-21 00:14:58 +07:00
Pau Espin 55afe0072b trx_toolkit: Add cmdline arg to set bind addr
Previous hardcoded default of 0.0.0.0 was inappropiate in some
scenarios, as it sets the SRC addr of the packets sent through the
socket based on the routing.

For instance, if iface IF1 has assigned two IP addresses A and B,
A being the first addr of the interface, and osmo-bts-trx is
configured with "osmotrx ip local A" and "osmotrx ip remote B",
the following happens:

  CMD POWER OFF src=A:5801 dst=B:5701
  RSP POWER OFF src=A:5701 dst=A:5701 <-- A is assigned as src addr.

But osmo-bts-trx is waiting for packets from B:5701, and the packet
is dropped with ICMP Unreachable. If addr binding is forced in
fake_trx to B, then everthing's fine.

Let's extend the UDPLink in order to allow manual, but optional
setting of bind address, and add a corresponding cmdline
argument to all executables.

Change-Id: I7be18fef40967fb7551f4115f22cbbd9cdb0840d
2018-04-04 17:14:26 +00:00
Vadim Yanitskiy 23914b9cf8 Rename 'fake_trx' to 'trx_toolkit'
This toolkit has branched out into several different tools for
TRX interface hacking, and creating a virtual Um-interface
(FakeTRX) is only one of its potential applications.

Change-Id: I56bcbc76b9c273d6b469a2bb68ddc46f3980e835
2018-03-13 02:10:02 +07:00