Commit Graph

22 Commits

Author SHA1 Message Date
Harald Welte 21caf32e3d Introduce APDU/TPDU trace decoder
This introduces a new pySim.apdu module hierarchy, which contains
classes that represent TPDU/APDUs as exchanged between
SIM/UICC/USIM/ISIM card and UE.

It contains instruction level decoders for SELECT, READ BINARY and
friends, and then uses the pySim.filesystem.Runtime{Lchan,State} classes
to keep track of the currently selected EF/DF/ADF for each logical
channel, and uses the file-specific decoder classes of pySim to decode
the actual file content that is being read or written.

This provides a much more meaningful decode of protocol traces than
wireshark will ever be able to give us.

Furthermore, there's the new pySim.apdu_source set of classes which
provides "input plugins" for obtaining APDU traces in a variety of
formats.  So far, GSMTAP UDP live capture and pyshark based RSPRO
live and pcap file reading are imlpemented.

Change-Id: I862d93163d495a294364168f7818641e47b18c0a
Closes: OS#5126
2022-07-23 12:18:57 +02:00
Harald Welte 362d2d0433 publish also the HTML manual for pySim
Change-Id: I124d59626f40538da30f729beca0e40bf6b2b915
Closes: OS#5270
2022-02-09 21:02:38 +01:00
Harald Welte 5981aa5c7f contrib/jenkins.sh: Fix PUBLISH
There is no $base in this script, and the current form renders:

make: *** /docs: No such file or directory.  Stop.
Build step 'Execute shell' marked build as failure

Change-Id: Ifcf27f7497daeb285dfb364bff20d0c861c77dcb
Related: OS#5271
2022-02-09 18:02:31 +01:00
Harald Welte 6b08fc3b49 contrib/jenkins.sh: first upload manuals, then execute physical tests
The tests with physical cards should not prevent upload of the manuals

Change-Id: I8aecb4ce211cbcc3890886ef24b04d01c510b6da
2022-02-09 17:01:46 +01:00
Harald Welte f03894aa65 update pyyaml dependency to >= 5.1
5.1 was the version introducing pyyaml.FullLoader which we're using,
see https://pyyaml.org/wiki/PyYAML#history

Change-Id: I0f2fa08ceeac2759218e85ad5bdce3ef951d0b74
2022-02-09 15:06:43 +01:00
Vadim Yanitskiy fc769e2fdb contrib/jenkins.sh: make pylint warn about unnecessary semicolon
Change-Id: I7793e30501ad109c95b207cbfada50596de17cda
Related: OS#5292
2021-11-05 16:22:06 +03:00
Harald Welte 1a4e9fd163 cmd2: Constrain version to >= 1.3.0 but < 2.0.0
2.0.0 introduces several incompatible changes, see
https://github.com/python-cmd2/cmd2/blob/master/CHANGELOG.md
as well as https://github.com/python-cmd2/cmd2/issues/1120

As we want to be able to use what distributions ship, let's stay
with 1.x for now.  If piip is used, use 1.5

Change-Id: Iecc953269d5ae9ed9f31b829743c63bdfd29fa61
2021-06-11 23:47:26 +02:00
Robert Falkenberg b07a3e9c87 Add codecs for EF_SPN and GSM strings via construct
This will replace the hand-crafted codec for EF_SPN
by a struct definition using the construct library.
Old encoders are updated and kept for API compatibility
but are not used internally anymore.

New data structures:
* Rpad(Adapter): Right-padded bytestring (0xff, adjustable)
* GsmStringAdapter(Adapter): Codec for "SMS default 7-bit
	coded alphabet as defined int TS 23.038" using
	the gsm0338 library.
* GsmString(n): Convenient wrapper of both above

Adjustments:
* utils: update+deprecate old dec_spn(), enc_spn()
* remove refs to deprecated functions

Change-Id: Ia1d3a3835933bac0002b7c52511481dd8094b994
2021-05-10 06:15:39 +02:00
Harald Welte 4ae228afc7 Implement EF.ARR (Access Rule Reference) decoding
The Access Mode (AM) and Security Condition (SC) DOs are incredibly
convoluted, so we need a lot of code to properly decode them.

Change-Id: If4f0725a849d41fd93de327ed00996d8179f2b0e
2021-05-04 13:24:07 +02:00
Vadim Yanitskiy e9fe09baff contrib/jenkins.sh: run pylint to find potential errors
Change-Id: Ib4cdd0fa824329569f973925ba3a46656a8c8296
2021-05-02 20:22:49 +00:00
Harald Welte e4759fd5cd contrib/jenkins.sh: Build and publish PDF manual
Change-Id: I3f01e93dd5a25d26feb3d067a171244a20f0f8e5
2021-04-11 12:20:29 +02:00
Harald Welte e0f9ef1606 integrate 'construct' python library
'construct' is a declarative symmetric encoder/decoder for user
specified binary formats.  It should come in extremely handy in
tools like pySim.

We start the integration by adding transport methods for transceiving
APDUs with built-in encoding of the command data and decoding of the
response data.

Change-Id: Ibf457aa8b9480a8db5979defcfafd67674303f6c
2021-04-11 12:20:29 +02:00
Harald Welte 0d4e98a2ac pySim-shell: JSONpath support for updating files/records
Change-Id: Iad09b3d878b8b58ad34cb549c80f8a6eb3149faa
2021-04-07 16:45:12 +00:00
Harald Welte d36f6943d9 docs/shell.rst: Auto-generate shell command reference
We use a slightly modified version of sphinx-argparse
(with patch https://github.com/alex-rudakov/sphinx-argparse/pull/136 applied)
in order to generate the command reference for each shell command in the
manual.

As the upstream repository seems unmaintained for ~2 years, let's use
the osmocom 'fork' with that above-mentioned patch applied.

Change-Id: I134f267cf53c7ecbc8cbfb33a8766d63bf4a8582
2021-04-06 21:16:16 +02:00
Harald Welte 5da7a72c99 jenkins.sh: Build documentation as part of build verification
Change-Id: Iea7547c81fcd3e4268da23c18ad56d841aaf7535
2021-04-03 19:58:43 +00:00
Vadim Yanitskiy d61da8a64c contrib/jenkins.sh: enable automatic execution of unit tests
Change-Id: I7b4bb49efd5e6ae284da063b7899e368ea4f1e22
Related: I4d4facfabc75187acd5238ff4d0f26022bd58f82
2021-03-12 01:13:15 +01:00
Vadim Yanitskiy 4ae7c49076 pySim/utils_test.py: prepare this to be executed on Jenkins
As it turns out, we had this set of unit tests since 2018, but
so far they were not executed during the build verification.

Let's fix this:

  * run unittest in discovery mode for all files in 'tests/' (commented out);
  * rename this file, so it can be automatically detected and executed;
  * properly import the API to be tested.

Currently 2 out of 16 unit tests are failing, so we need to get
them passing first and then uncomment the unittest execution.

Change-Id: I4d4facfabc75187acd5238ff4d0f26022bd58f82
2021-03-11 23:54:15 +01:00
Harald Welte b2edd14475 Add a new pySim-shell program
pySim-prog was nice when there were only 5 parameters on a SIM that we
could program, and where the use case was pretty limited.  Today, we
have SIM/USIM/ISIM cards with hundreds of files and even more parameters
to program.  We cannot add a command line argument for each file to
pySim-prog.

Instead, this introduces an interactive command-line shell / REPL,
in which one can navigate the file system of the card, read and update
files both in raw format and in decoded/parsed format.

The idea is primarily inspired by Henryk Ploatz' venerable
cyberflex-shell, but implemented on a more modern basis using
the cmd2 python module.

See https://lists.osmocom.org/pipermail/simtrace/2021-January/000860.html
and https://lists.osmocom.org/pipermail/simtrace/2021-February/000864.html
for some related background.

Most code by Harald Welte. Some bug fixes by Philipp Maier
have been squashed.

Change-Id: Iad117596e922223bdc1e5b956f84844b7c577e02
Related: OS#4963
2021-03-03 08:43:38 +01:00
Daniel Willmann de07b95f84 Fix invocation python2 -> python3
Change-Id: Ic78da9c03e99f59d142c93394051bbc2751f0205
2020-10-19 10:32:34 +02:00
Philipp Maier c5b422e2ca Add support for automatic card handling
When using the batch mode of pySim-prog, the user has to insert/remove
the cards from the cardreader manually. This is fine for small batches,
but for high volume batches this method is not applicable.

This patch adds support for the integration of an automatic card handler
machine. The user can freely configure a custom commandline that is
executed when a card should be inserted or moved to a good/bad
collection bin.

Change-Id: Icfed3cad7927b92816723d75603b78e1a4b87ef1
Related: SYS#4654
2019-09-12 11:21:47 +02:00
Harald Welte 91d4ec7e7b import pysim-testdata of automatic testing to this repository
This test data is used by the jenkins build verification for pySim,
and it was previously located locally on the build slave.

By moving the testdata to this repository, any contributor can
modify both the code and the expected test results simultaneously.

Change-Id: I6714b091a114035d6aab8ba750c5f2b86e438467
2019-05-10 17:03:58 +02:00
Alexander Couzens 4731232aae contrib: add jenkins.sh
Run the tests.

Change-Id: I9308f097bf77d82e5ef78d1be551baf127726f74
2018-07-19 23:46:38 +02:00