Commit Graph

733 Commits

Author SHA1 Message Date
Harald Welte 8dbf714e96 gsm_r: Fix decoding of EF.FN
This fixes the below exception when trying to decode records of EF.FN:

EXCEPTION of type 'TypeError' occurred with message: 'unsupported operand type(s) for &: 'str' and 'int''

Change-Id: I3723a0d59f862fa818bea1622fe43a7b56c92847
Related: OS#5784
2023-01-24 14:37:18 +01:00
Harald Welte e6d7b14f43 gsm_r: Fix typo (it's EF.FN, not EF.EN)
Related: OS#5784
Change-Id: I2c97a02973d2a1eda2cea5412391144726bb0525
2023-01-24 14:37:13 +01:00
Harald Welte bc7437d3b6 pySim-trace: Also consider SW 91xx as successful
Change-Id: I9e4170721be30342bdce7fb4beeefd1927263ca6
2023-01-24 13:50:51 +01:00
Harald Welte 7489947046 pySim-trace: Fix missing MANAGE CHANNEL decode
old output:

00 MANAGE CHANNEL 01       9110

new output:

00 MANAGE CHANNEL 01       9110 {'mode': 'open_channel', 'created_channel': 1}

Change-Id: Iac5b24c14d2b68d526ab347462b72548b8731b30
2023-01-24 13:50:51 +01:00
Harald Welte c95f6e2124 pySim-trace: Add support for reading GSMTAP from pcap files
So far we supported
* GSMTAP live traces via a UDP socket
* RSPRO traces from pcap files (or live)

We were lacking support for reading GSMTAP stored in pcap, which
is what this patch implements.

Change-Id: I46d42774b39a2735500ff5804206ddcfa545568c
2023-01-24 13:50:51 +01:00
Philipp Maier 284ac104af cards: also program EF.AD under ADF.USIM
DF.GSM and ADF.USIM have an EF.AD with nearly the same contents. Usually
there is one file physically present and the other is just a link.
Apparantly this is not always the case for sysmo-ismi-sja2 cards, so
lets program EF.AD in both locations.

Change-Id: Ic9dd4acc8d9a72acbb7376ddf3e2128125d4a8f5
Related: OS#5830
2023-01-19 10:32:13 +01:00
Philipp Maier de0cf1648c cards: fix typo
Change-Id: I81a6074776bdf67b7bea359fe7a24f906936f46d
2023-01-03 13:30:02 +01:00
Philipp Maier 4237ccfb45 pySim-prog: add python docstring for read_params_csv
Change-Id: I098ff56ef38208f2f321194625ff4279ece2023c
2022-12-20 11:33:03 +01:00
Philipp Maier 5f0cb3c5f2 pySim-prog: rename write_parameters function.
The function name "write_parameters" is very generic and since it is
called during the programming cycle it should be made clear that it is
not about writing parameters to the card.

Change-Id: Idaba672987230d7d0dd500409f9fe0b94ba39370
2022-12-20 11:33:03 +01:00
Philipp Maier cbb8c02d25 pySim-prog: make dry-run more realistic
The process_card function has a dry-run mode where one can test
parameters without actually writing to the card. However, the dry-run
feature also does not perform read operations and connects to the card
reader at a different point in time. Lets be more accurate here and
perform all operations a normal programming cycle would perform but
without calling the card.program() method.

Change-Id: I3653bada1ad26bcf75109a935105273ee9d1525c
2022-12-20 11:33:03 +01:00
Philipp Maier 0a8d9f05b8 cards: check length of mnc more restrictively
Since we now ensure that mnc always has a valid length lets make the
check in cards.py more strict.

Related: OS#5830
Change-Id: Iee8f25416e0cc3be96dff025affb1dc11d919fcd
2022-12-20 11:33:03 +01:00
Philipp Maier 32c0434540 pySim-prog: fix handling of mnclen parameter.
The handling of the mnclen parameter does not work. Lets fix it so that
it can be used again with CSV and normal card programming. Lets ensure
that depending on the parameter and the defaults it is always ensured
that the mnc string has the correct length so that lower layers can
deduct the length of the mnc properly by the string length of the mnc.

Change-Id: I48a109296efcd7a3f37c183ff53c5fc9544e3cfc
Related: OS#5830
2022-12-20 11:21:07 +01:00
Philipp Maier 2688ddf459 pySim-prog: clean up csv file reader function
The function that goes through the CSV file and searches for either IMSI
or ICCID or picks a specific line by number is very hard to read and
understand. Lets clean it up and add useful error messages

Change-Id: I7ae995aa3297e77b983e59c75e1c3ef17e1d7cd4
Related: OS#5830
2022-12-20 11:12:52 +01:00
Philipp Maier 4f888a0414 sysmocom_sja2: simplify and fix op/opc decoder/encoder
The decoder/encoder of that decodes the EF.xSIM_AUTH_KEY files has an
overcomplicated handling for op/and opc. There is a condition that
checks if milenage is configured and another one that checks if the
string is recognized as OP or OPc. Both is not correct and seems not to
work (op and opc is always displayed as "null")

The encoder/decoder should focus on the physical file layout and
regardless of any other conriguration the OP/OPc field is physically
present and should be displayd and presented for editing.

Change-Id: I6fa3a07e5e473273498d3f13d4cfa33743b787e1
2022-12-02 12:38:08 +01:00
Christian Amsüss 5d26311efc OTA: Adjust IV length for AES
Change-Id: I854c844418244c100c328f9e76c0f37850d3db00
2022-11-25 04:00:55 +01:00
Oliver Smith 8e45b75711 contrib/jenkins.sh: split test/pylint/docs
Split the jenkins job up in three parts, so each of them can run in
parallel, and the test part that has to run on a specific node (and
blocks it while running), finishes faster.

Don't install depends of pylint/docs jobs as they will run in docker
and the depends get installed once in the container.

Related: OS#5497
Depends: docker-playground Id5c75725d2fab46b29773fa4f637fa2d73fa7291
Depends: osmo-ci Iea4f15fd9c9f8f36cb8d638c48da000eafe746a4
Change-Id: I5245c529db729e209d78a02ab9c917a90d0e0206
2022-11-04 13:13:14 +01:00
Oliver Smith 0529c1906d docs: allow overriding OSMO_GSM_MANUALS_DIR
Related: OS#5497
Change-Id: I433217b7aa1cdcddc52a89721e03e44b417bacb1
2022-10-21 16:24:47 +02:00
Oliver Smith 507b5271ac contrib/jenkins.sh: set PYTHONUNBUFFERED=1
Make sure all python output is printed immediatelly.

Change-Id: I5d334bbc34e4df39ac54472642299c567894f449
2022-10-18 16:50:03 +02:00
Vadim Yanitskiy 4e64e72766 Revert "contrib/jenkins.sh: pylint v2.15 is unstable, pin v2.14.5"
This reverts commit 12175d3588.

The upstream has fixed the regression:

https://github.com/PyCQA/pylint/issues/7375

which was actually not in pylint itself but in its dependency:

https://github.com/PyCQA/astroid/pull/1763
https://github.com/PyCQA/astroid/releases/tag/v2.12.6

Change-Id: I1bf36e0c6db14a10ff4eab57bae238401dbd7fd0
Closes: OS#5668
2022-09-14 05:00:33 +00:00
Harald Welte 75a58d1a87 Add new pySim.ota library, implement SIM OTA crypto
This introduces a hierarchy of classes implementing

* ETS TS 102 225 (general command structure)
* 3GPP TS 31.115 (dialects for SMS-PP)

In this initial patch only the SMS "dialect" is supported,
but it is foreseen that USSD/SMSCB/HTTPS dialects can be
added at a later point.

Change-Id: I193ff4712c8503279c017b4b1324f0c3d38b9f84
2022-09-08 15:45:55 +02:00
Vadim Yanitskiy 7d05e49f11 README.md: update installation instructions for Debian
Change-Id: Icefa33570a34960a4fff145f3c1b6585d867605c
2022-09-05 23:15:11 +07:00
Vadim Yanitskiy 98ea2a0f7a README.md: update git URLs (git -> https; gitea)
Change-Id: Ia86979f656557e442b0f432b0646aa7661c293e9
2022-09-05 23:15:11 +07:00
Vadim Yanitskiy 0a8d27ad7a README.md: list recent dependencies from requirements.txt
Change-Id: Ia486dbc7f630c1404e51728b5353cf5a0d643415
2022-09-05 23:15:11 +07:00
Vadim Yanitskiy 9550a0a45b README.md: fix module name: s/serial/pyserial/
Change-Id: I5fd308fb161cd5bd5f702845691296877e523248
2022-09-05 23:15:11 +07:00
Vadim Yanitskiy b5eaf14991 README.md,requirements.txt: add missing construct version info
Change-Id: I90da0df431f0d7dbfa4aa428366fbf0e35db388f
Related: OS#5666
2022-09-05 23:15:10 +07:00
Vadim Yanitskiy bdac3f61be Bump minimum required construct version to v2.9.51
With this version I can get all unittests passing:

  python -m unittest discover tests/

We're passing argument 'path' to stream_read_entire(), which was
added in [1] and become available since v2.9.51.

Change-Id: I4223c83570d333ad8d79bc2aa2d8bcc580156cff
Related: [1] bfe71315b027e18e62f00ec4de75043992fd2316 construct.git
Related: OS#5666
2022-09-05 23:15:10 +07:00
Vadim Yanitskiy 05d30eb666 construct: use Python's API for int<->bytes conversion
Argument 'signed' was added in [1] and become available since v2.10.63.
Therefore using bytes2integer() and integer2bytes() from construct.core
bumps the minimum required version of construct to v2.10.63.  For
instance, debian:bullseye currently ships v2.10.58.

There is no strict requirement to use construct's API, so let's use
Python's API instead.  This allows using older construct versions
from the v2.9.xx family.

Change-Id: I613dbfebe993f9c19003635371941710fc1b1236
Related: [1] 660ddbe2d9a351731ad7976351adbf413809a715 construct.git
Related: OS#5666
2022-09-05 23:15:10 +07:00
Vadim Yanitskiy 7800f9d356 contrib/jenkins.sh: install dependencies from requirements.txt
Change-Id: I99af496e9a3758ea624ca484f4fbc51b262ffaf4
2022-09-05 23:15:10 +07:00
Vadim Yanitskiy 7ce04a5a29 contrib/jenkins.sh: execute this script with -x and -e
-x  Print commands and their arguments as they are executed
  -e  Exit immediately if a command exits with a non-zero status

Change-Id: I13af70ef770936bec00b050b6c4f988e53ee2833
2022-09-05 23:15:06 +07:00
Vadim Yanitskiy b3ea021b32 contrib/jenkins.sh: speed up pylint by running multiple processes
Use multiple processes to speed up pylint.  Specifying -j0 will
auto-detect the number of processors available to use.

On AMD Ryzen 7 3700X this significantly reduces the exec time:

  $ time python -m pylint -j1 ... pySim *.py
  real    0m12.409s
  user    0m12.149s
  sys     0m0.136s

  $ time python -m pylint -j0 ... pySim *.py
  real    0m5.541s
  user    0m58.496s
  sys     0m1.213s

Change-Id: I76d1696c27ddcab358526f807c4a0a7f0d4c85d4
2022-08-30 17:15:53 +07:00
Vadim Yanitskiy 12175d3588 contrib/jenkins.sh: pylint v2.15 is unstable, pin v2.14.5
pylint v2.15 is crashing, let's fall-back to a known to work v2.14.5.

Change-Id: Ie29be6ec6631ff2b3d8cd6b2dd9ac0ed8f505e4f
Related: https://github.com/PyCQA/pylint/issues/7375
Related: OS#5668
2022-08-30 17:12:03 +07:00
Christian Amsüss 59f3b1154f proactive: Send a Terminal Response automatically after a Fetch
Change-Id: I43bc994e7517b5907fb40a98d84797c54056c47d
2022-08-21 11:54:33 +00:00
Christian Amsüss 98552ef1bd proactive: Avoid clobbering the output of the command that triggered the FETCH
Change-Id: I2b794a5c5bc808b9703b4bc679c119341a0ed41c
2022-08-21 11:54:00 +00:00
Harald Welte cab26c728c pySim-shell: Use pySim.cat definitions to print decoded proactive cmds
Register a ProactiveHandler with pySim.transport and call the decoder
from pySim.cat to print a decoded version:

Example usage (exact data only works on my specific card due to the
encrpyted payload):

pySIM-shell (MF/ADF.USIM)> envelope_sms 400881214365877ff6227052000000000302700000201506393535b000118dd46f4ad6b015922f62292350d60af4af191adcbbc35cf4
FETCH: d0378103011300820281838b2c410008812143658700f621027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c
SendShortMessage(CommandDetails({'command_number': 1, 'type_of_command': 19, 'command_qualifier': 0}),DeviceIdentities({'source_dev_id': 'uicc', 'dest_dev_id': 'network'}),SMS_TPDU({'tpdu': '410008812143658700f621027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c'}))
SW: 9000, data: d0378103011300820281838b2c410008812143658700f621027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c

Change-Id: Ia4cdf06a44f46184d0da318bdf67077bc8ac9a1a
2022-08-06 18:56:42 +02:00
Harald Welte fd476b4d62 pySim.transport: Add mechanism for handling for CAT/USAT proactive cmds
This introduces an optional argument to the LinkBase class constructor,
where the application can pass an instance of a ProactiveHandler derived
class in order to handle the proactive commands that the LinkBase is
automatically fetching whenever the card indicates so.

Change-Id: I844504e2fc1b27ce4fc7ede20b2307e698baa0f6
2022-08-06 18:56:42 +02:00
Harald Welte 5a4891a5b7 Add TLV definitions for *a lot more* CAT / USAT data objects
This adds deciding for the bulk of the TLV objects used in the
ETSI CAT (Card Application Toolkit) and 3GPP USAT (USIM Application
Toolkit) systems.

This patch just adds the definitions, but doesn't use them anywhere yet.

Change-Id: I0c66912dbc10164e040e2fec358cef13c45a66ec
2022-08-06 18:56:42 +02:00
Harald Welte 7d8029eb23 tlv: Use self._compute_tag() method rather than direct self.tag
The TLV_IE.from_tlv() method is part of a base class that is inherited
by more specific classes.  The official way to obtain the tag is the
inherited-class-provided self._compute_tag() method, and *not* a direct
reference to the self.tag member.

This allows for some more obscure TLV parsers, such as the upcoming one
for Proactive Commands in the CAT/OTA context.

Change-Id: I0cd70e31567edc5a0584336efcb5e4282734f6dd
2022-08-06 13:19:16 +02:00
Harald Welte f56b6b2a1c ts_31_102: Add missing imports for envelope_sms command
The envelope_sms command fails due to some missing imports prior to
this patch.

Change-Id: I98e692745e7e1cfbc64b88b248700b1e54915b96
2022-07-30 16:37:01 +02:00
Harald Welte 51b3abb000 ts_31_102: Fix terminal_profile, envelope and envelope_sms commands
In commit Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5 I accidentially
broke those commands by adding argparse definitions for better
documentation.  When adding the  @cmd2.with_argparser decorator,
the method argument changes from the raw string to an argparse.Namespace
object.

This patch fixes the below exception:

pySIM-shell (MF/ADF.USIM)> terminal_profile ffffffff
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/cmd2/cmd2.py", line 2129, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/usr/local/lib/python3.10/dist-packages/cmd2/cmd2.py", line 2559, in onecmd
    stop = func(statement)
  File "/usr/local/lib/python3.10/dist-packages/cmd2/decorators.py", line 336, in cmd_wrapper
    return func(*args_list, **kwargs)
  File "/space/home/laforge/projects/git/pysim/pySim/ts_31_102.py", line 1274, in do_terminal_profile
    (data, sw) = self._cmd.card._scc.terminal_profile(arg)
  File "/space/home/laforge/projects/git/pysim/pySim/commands.py", line 583, in terminal_profile
    data_length = len(payload) // 2
TypeError: object of type 'Namespace' has no len()

Change-Id: Ia861eeb2970627d3ecfd0ca73f75ca571c6885b2
Fixes: Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5
2022-07-30 16:37:01 +02:00
Harald Welte 7416d463a4 Fix printing of SwMatchError after introduction of logical channels
the interpret_sw() method was moved from RuntimeState to RuntimeLchan
in Change-Id I7aa994b625467d4e46a2edd8123240b930305360 - but the code
in pySim/exceptions.py was not adjusted accordingly.

Change-Id: I0614436c99c6a6ebc22c4dc14fb361c5f5f16686
2022-07-30 16:37:01 +02:00
Harald Welte 93c34aac89 apdu/ts_102_221: SELECT: allow select of SELF
While in the pySim-shell, it's useful to filter the currently selected
file from the choice of available files for select, this doesn't apply
for the tracing case: It's perfectly valid for the UE to SELECT the
file that's already selected right now.  The operation basically
becomes equivalent to a STATUS.

Change-Id: I1a20fb3ba70426333ac34448c6cb782c51363965
2022-07-25 14:25:11 +02:00
Harald Welte dcc689d9c4 apdu/ts_102_221: SELECT: allow select of parent/ancestor DFs
We need to pass the 'PARENT' flag to get_selectables() to be able
to track SELECT on any of the parent/ancestor DF FID.

Change-Id: Ia7ac627d5edccb97160c90688d720d887fad6ec7
2022-07-25 14:25:11 +02:00
Harald Welte f5ff1b896e filesystem: We can select not just immediate parent DF but all ancestors
I didn't check the specs, but at least experience with real-world cards
(and modems) shows that it's not just permitted to select the immediate
parent DF, but all ancestors of the currently selected file.

So adjust the get_selectables() method to not just return the immediate
parent, but to recurse all the way up and report the FID of any ancestor
DF.

Change-Id: Ic9037aa9a13af6fb0c2c22b673aa4afa78575b49
2022-07-25 14:25:11 +02:00
Harald Welte 8e9c844130 apdu/ts_102_221: Fix SELECT of 3f00
In order to be able to explicitly select the MF via 3f00,
we need to pass the 'MF' to get_selectables(), so the record
is included in the list of selectable files from the current
working directory.

Change-Id: I27085896142fe547a6e93e01e63e59bbc65c8b8a
2022-07-24 11:56:35 +02:00
Harald Welte 498361f3b5 apdu/ts_102_221: Implement SELECT case "df_ef_or_mf_by_file_id"
This was [sadly] simply missing from the implementation so far.

Change-Id: I7bbd13ce29f5adc1ca3ca01bffabbe02dd17db20
2022-07-24 11:56:35 +02:00
Harald Welte d2c177b396 filesystem.py: Make CardDF.get_selectables() respect the flags
All other get_selectables() understand a flag like 'FIDS' to request
only the hexadecimal FIDs and not the file names.  However, the
CardEF.get_selectables() ignored those flags and unconditionally
returned the names.

Change-Id: Icdc37cae3eecd36d167da76c30224b9d48c844fd
2022-07-24 11:56:35 +02:00
Harald Welte 86d698d310 pySim-trace: Don't print argparse object at start-up
Change-Id: I881471d026457d8ffcfdbd412c7aae0d0bff9344
2022-07-24 10:23:50 +02:00
Harald Welte 72c5b2d796 pySim-trace: Fix --no-suppress-{select.status} command line arguments
The Tracer implemented those options and the argparser handled it,
but we didn't ever connect the two.

Change-Id: I7d7d5fc475a8d09efdb63d3d6f1cc1de1996687b
2022-07-24 10:23:50 +02:00
Harald Welte c61fbf4daa pySim-trace: Support SELECT with empty response body
If the modem/UE doesn't ask for the FCP to be returned, a SELECT
can exit with 9000 and no response body.  Don't crash in that case.

Change-Id: I66788717bec921bc54575e60f3f81adc80584dbc
2022-07-24 09:46:11 +02:00
Harald Welte 04897d5f25 sim-rest-server: Report meaningful error message if PIN is blocked
Instead of a cryptic backtrace, we now return a meaningful error like this:

{"error": {"message": "Security Status not satisfied - Card PIN enabled?", "status_word": "6982"}

Change-Id: I6dafd37dfd9fa3d52ca2c2e5ec37a6d274ba651b
Closes: OS#5606
2022-07-23 14:07:00 +02:00