Commit Graph

27 Commits (8dbf714e962327aad69242ca2634b0c05da341a5)

Author SHA1 Message Date
Harald Welte 75a58d1a87 Add new pySim.ota library, implement SIM OTA crypto
This introduces a hierarchy of classes implementing

* ETS TS 102 225 (general command structure)
* 3GPP TS 31.115 (dialects for SMS-PP)

In this initial patch only the SMS "dialect" is supported,
but it is foreseen that USSD/SMSCB/HTTPS dialects can be
added at a later point.

Change-Id: I193ff4712c8503279c017b4b1324f0c3d38b9f84
2022-09-08 15:45:55 +02:00
Harald Welte 21caf32e3d Introduce APDU/TPDU trace decoder
This introduces a new pySim.apdu module hierarchy, which contains
classes that represent TPDU/APDUs as exchanged between

It contains instruction level decoders for SELECT, READ BINARY and
friends, and then uses the pySim.filesystem.Runtime{Lchan,State} classes
to keep track of the currently selected EF/DF/ADF for each logical
channel, and uses the file-specific decoder classes of pySim to decode
the actual file content that is being read or written.

This provides a much more meaningful decode of protocol traces than
wireshark will ever be able to give us.

Furthermore, there's the new pySim.apdu_source set of classes which
provides "input plugins" for obtaining APDU traces in a variety of
formats.  So far, GSMTAP UDP live capture and pyshark based RSPRO
live and pcap file reading are imlpemented.

Change-Id: I862d93163d495a294364168f7818641e47b18c0a
Closes: OS#5126
2022-07-23 12:18:57 +02:00
Harald Welte 785d484709 utils: Fix bugs in DataObject encoders
The DataObject is some weird / rarely used different code than the
normal TLV encoder/decoder.  It has apparently so far only been used
for decoding, without testing the encoding side, resulting in related

Let's fix those that I encountered today, and add a test case.

Change-Id: I31370066f43c22fc3ce9e2b9ee75986a652f6fc4
2022-04-05 14:33:00 +02:00
Harald Welte 81f4b4058b Extend unit test coverage for construct, add [some] tests for TLV
Change-Id: I3470e0b2e978221aa0c1e46a4b65f71f71abef2e
2022-02-14 00:41:24 +01:00
Harald Welte d0519e0c37 construct: Add Construct for variable-length int 'GreedyInteger'
We have a number of integers with variable-length encoding, so
add a Construct for this.  Naming inspired by GreedyBytes.

Related to

Change-Id: Ic6049b74ea3705fda24855f34b4a1d5f2c9327f7
2022-02-14 00:41:24 +01:00
Bjoern Riemer ffee89a031 add missing bit definition for NG-RAN in xAcT
when encoding the AcT value bit 11 is correctly set
when NG-RAN is present in the string representation,
however the decoding of bit 11 was missing.
Adds tests for the decoder as well.

Change-Id: I910df28c4c59ec94cce9603377786325f6d8c1a3
2022-01-22 12:58:00 +00:00
Harald Welte 9f3b44d6ff utils: COMPREHENSION-TLV support
Change-Id: I8d969382b73fa152ee09c456fa4aee428fb36285
2021-05-29 22:13:56 +02:00
Harald Welte e7506036bd Introduce unit test for bertlv_parse_one()
Change-Id: I3adbe22afd4b6503a7454de39b7663e9ede8995f
2021-05-25 09:43:13 +02:00
Harald Welte de02718631 add unit tests for BER-TLV encoder/decoder functions
... and while at it resolve a bug in bertlv_parse_len()
discovered by those new tests.

Change-Id: I9f14dafab4f712c29224c4eb25cacab7885e2b68
2021-05-25 09:43:13 +02:00
Philipp Maier e7d417955d ts_51_011, utils: fix Access Technology Identifier coding
When the Access Technology Identifier encoder sets the bits for E-UTRAN
it does not respect that bit "100" is also a valid bit combination that
encodes E-UTRAN WB-S1 and E-UTRAN NB-S1. Lets encode this bit
combination if the user is just specifying "E-UTRAN" without further
spefication of WB or NB.

The decoder only looks at bit 14 and decodes "1xx" always to "E-UTRAN".
This is not specific enough. Lets make sure that the decoder is
complementary to the encoder.

Change-Id: Ibfe8883a05f9ad6988d8e212cb9a598229954296
Related: OS#4963
2021-05-03 17:08:37 +02:00
Philipp Maier b919f8bd75 utils: fix dec_xplmn_w_act() and format_xplmn_w_act()
The function dec_xplmn_w_act(), which is also used by
format_xplmn_w_act() is using integer numbers as MCC/MNC representation.
This causes various problems since the information about leading zeros
gets lost.

Change-Id: I57f7dff80f48071ef9a3732ae1088882b127a6d4
2021-05-03 15:08:27 +00:00
Philipp Maier 6c5cd8031d utils: fix mcc/mnc encoding in dec_plmn (EF_PLMNsel)
The dec_plmn function takes an hexstring and returns the decoded MCC and
MNC as integer values. The result is then used by the json encoder in
EF_PLMNsel, which means the json output will contrary to the input, use
integer values instead of strings.

This is not correct since there may be leading zeros (e.g. mnc 01 and
001 both exist are different) which must be retained in order to know
the correct length of the MNC.

Related: OS#4963
Change-Id: I393e04836814d992d2a6d0a4e4e01850976d6e81
2021-05-03 15:07:50 +00:00
Philipp Maier b46cb3ffa2 utils: fix encoding of EF.MSISDN
The encoding of EF.MSISDN is a bit unstrutured. The encoder function
does not return a valid result since it lacks the parameters
Capability/Configuration2 Record Identifier and Extension5 Record
Identifier, which are mandatory but can be set to 0xFF. Also the
encoder gets its input from pySim-shell, so it should have some
more input validation, especially when the user encodes an empty
string. The encoder and decoder function also do not have unit-tests.

Since the encoder now adds the missing two bytes by isself this does
not have to be done manually anymore, so needs to be

For the encoder is used from Unfortunately
it is used wrongly there. The optional Alpha Identifier is required
here as well.

Related: OS#4963
Change-Id: Iee5369b3e3ba7fa1155facc8fa824bc60e33b55b
2021-04-23 15:52:10 +02:00
Philipp Maier 1bd664da9e tests: remove .example files for simcard tests
The folder pysim-testdata already contains testdata that can also be
used as examples. The .example files are from a time where the testdata
was not kept inside the repository. Since we decided to keep the test
data in the repository as well those file are redundant.

Change-Id: Iee34cad74b50755e1007506f909da9766fa8412e
2021-04-08 15:37:32 +00:00
Merlin Chlosta 05ca36b3f3 Add decoder/encoder for EF.SUCI_Calc_Info
Change-Id: I848a766e6d00be497c7db905475e0681cce197ac
2021-04-02 14:10:10 +02:00
Vadim Yanitskiy 46c49d5256 tests/ update expectations for format_xplmn_w_act()
Change-Id: I520328e3490cc3a333d2daad84e745d115196626
2021-03-12 01:12:11 +01:00
Vadim Yanitskiy c8458e2477 pySim/ fix 3-digit MNC encoding in enc_plmn()
The bug that was attempted to be fixed in [1] actually was in the
encoding API - pySim.utils.enc_plmn().  According to 3GPP TS 31.102,
which points to TS 24.008, the three-digit (E)HPLMN shall be encoded
as shown below (ASCII-art interpretation):

    0   1   2   3   4   5   6   7
  |  MCC Digit 2  |  MCC Digit 1  |
  |  MNC Digit 3  |  MCC Digit 3  |
  |  MNC Digit 2  |  MNC Digit 1  |

while pySim.utils.enc_plmn() would produce the following:

    0   1   2   3   4   5   6   7
  |  MCC Digit 2  |  MCC Digit 1  |
  |  MNC Digit 1  |  MCC Digit 3  |
  |  MNC Digit 3  |  MNC Digit 2  |

Initially the _decoding_ API was correct, but then got changed in
[1] to follow buggy pySim's encoding API.  As a result, a (E)HPLMN
programmed with would look correct if verified by, but the actual file content would be wrong.

This situation shows that our 'program-read-match' build verification
approach alone is insignificant.  The lack of unit test coverage,
at least for the core parts of the project, makes it possible to have
symmetrical bugs in both encoding and decoding API parts unnoticed.

This problem was found while trying to enable dead unit tests in [3].
Change [1] that introduced a symmetrical bug is reverted in [2].

Change-Id: Ic7612502e1bb0d280133dabbcb5cb146fc6997e5
Related: [1] I799469206f87e930d8888367890babcb8ebe23a9
Related: [2] If6bf5383988ad442e275efc7c5a159327d104879
Related: [3] I4d4facfabc75187acd5238ff4d0f26022bd58f82
2021-03-12 01:00:32 +01:00
Vadim Yanitskiy 4ae7c49076 pySim/ prepare this to be executed on Jenkins
As it turns out, we had this set of unit tests since 2018, but
so far they were not executed during the build verification.

Let's fix this:

  * run unittest in discovery mode for all files in 'tests/' (commented out);
  * rename this file, so it can be automatically detected and executed;
  * properly import the API to be tested.

Currently 2 out of 16 unit tests are failing, so we need to get
them passing first and then uncomment the unittest execution.

Change-Id: I4d4facfabc75187acd5238ff4d0f26022bd58f82
2021-03-11 23:54:15 +01:00
Denis 'GNUtoo' Carikli 79f5b6080b Python 2 is deprecated, remove backwards compatibility chunks
pySim has already been migrated to Python 3 in another change [1],
and the build verification has been migrated to Debian 10 with
Python 3.7.  However, there is still some backwards compatibility
code left.  Let's get rid of it.

[1] Ic78da9c03e99f59d142c93394051bbc2751f0205

Signed-off-by: Denis 'GNUtoo' Carikli <>
Tweaked-by: Vadim Yanitskiy <>
Change-Id: I430d173535e0cd5bb895b9dfc9070cbc40cfc8ff
2021-03-07 20:33:46 +01:00
Philipp Maier ff84c23839 pySim-prog, pySim-read, do not echo reader id
pySim-prog and pySim-read currently echo back the pcsc reader id (or
baudrate/socket, depending on the interface used). This makes the output
unecessarly undeterministic, which becomes a problem when verifying the
putput in tests. Lets not echo those variable, user supplied parameters
back. Also lets move the code that does the initalization to utils, so
that it can be used from pySim-prog and from pySim-read (code dup).

Change-Id: I243cc332f075d007b1c111292effcc610e874eb3
Related: OS#4503
2020-05-12 18:11:38 +00:00
Supreeth Herle 5a541016fb add support for MSISDN programming
This change implements programming of EF.MSISDN as per 3GPP TS 31.102,
sections 4.2.26 and, excluding the following fields:

  - Alpha Identifier (currently 'FF'O * 20),
  - Capability/Configuration1 Record Identifier ('FF'O),
  - Extension1 Record Identifier ('FF'O).

This feature is introduced exclusively for sysmoUSIM-SJS1.
Othere SIM card types need to be tested.

Change-Id: Ie033a0ffc3697ae562eaa7a241a0f6af6c2b0594
2020-02-15 04:57:20 +07:00
Philipp Maier 07cd481954 Fixup: Fix automated tests
The Change I12e6b46787efb39c5745f4e7f3cdcca9209881b8 was not as
effective as expected. Diff is used wrongly so that no lines are
compared. Lets fix this

Change-Id: I1601d8a2b3e1c07fe1eba375ea8deae3d50bbef0
2019-12-31 18:02:42 +01:00
Philipp Maier 4e724391e0 Fix automated tests
- The .ok files currently dictate in which pysical reader device the
card must be placed. Lets remove this dependncy to make the setup more
reliable. Testing in which reade a card is placed is not in the scope of
our tests.

- Fix bug in (test runner), so that the veriable $ADM_HEX
gets reset after the execution of the tests.

Change-Id: I12e6b46787efb39c5745f4e7f3cdcca9209881b8
2019-12-16 15:13:27 +01:00
Daniel Willmann f432b2ba96 pySim-prog: Add option for hex ADM keys
pySim-prog would implicitly try to use the raw or hex-escaped format
depending on the length of the parameter, now there is the option "-A"
to explicitly specify the hex-escaped ADM1 key. Explicitly use the "-A" option to pass the hex adm1 key
for wavemobile cards

Change-Id: Id75a03482aa7f8cc3bdbb8d5967f1e8ab45c179a
2019-09-03 20:11:48 +02:00
Philipp Maier c555e18ebb tests: add example test data for Wavemobile-SIM
Change-Id: If69cd2d8af6d544155b3088f77eb6ea4b789901d
Related: SYS#4245
2018-09-13 09:43:26 +02:00
Philipp Maier 11a2e3ca10 tests: add example file for sysmosim-gr1
Change-Id: I36cfa81f9029bfaedd2e802a2d709596dbeb20ab
Related: OS#3405
2018-08-23 10:44:51 +02:00
Philipp Maier 7f340851fc tests: add test program to verify (and
Pysim now supports quite a number of different cards. Estimating
if changes in pysim introce regressions becomes increasingly difficult

The script that is added with this patch is intended to run as
atomated testsuit on real cards attached to a test system. However,
it can also be used by developers locally to check for regressions.

Change-Id: I8c6f95998272333bc757b34e3ab6be004e8cd674
Related: OS#3376
2018-07-19 07:55:42 +00:00