Commit Graph

405 Commits

Author SHA1 Message Date
Harald Welte 8fe1d202c7 pySim-read: Migrate over to use shared argparse from transport
Now that we have a shared argparse definition for all reader related
options in the transport module, use that.

Change-Id: I12ca1a484a5d6e84820d9761c9701f8a94381f66
2021-04-22 09:11:23 +02:00
Harald Welte 28c2431f9c Move reader related argument parser to transport module
Ideally that shared definition would be used by all programs,
rather than copy+pasting it.  Unfortunately pySim-{read,prog}
are still using optparse and first need to be converted to
argparse.

Change-Id: If77f53850e1ca65f42cf1dca3e0f460dac1b0d1a
2021-04-22 09:11:23 +02:00
Philipp Maier fe1fb037b5 filesystem: fix wrong helpstring for update_record_decoded
the helpstring of update_record_decoded mentions hex bytes for the data
parameter, but it should be mentioned as abstract json data like in
update_binary_decoded

Change-Id: Ibae2ab49054ac5dd6fcccddd28c98d886403dac9
Related: OS#4963
2021-04-21 15:13:52 +02:00
Philipp Maier 0adabf6161 filesystem: fix wrong comment
The property rec_len is not a tuple, it is a set.

Related: OS#4963
Change-Id: I366772c62d0bb5a6400ce5b431eb94ac9248dccc
2021-04-21 15:13:52 +02:00
Philipp Maier 80ce71f58c pySim-shell: separate export summary with a headline
the export summary is printed after the log entry for the last file
without separation. This is confusing because it looks like if the
summary would refer to the last file only. Lets add a headline to make
clear that the last few lines are the "Export summary"

Change-Id: I90771e525b2b114bdb41a8e90d298ca991c09c3d
Related: OS#4963
2021-04-21 15:11:34 +02:00
Philipp Maier 04be9d6033 pysim-testdata/sysmoISIM-SJA2: change card in test rig
The sysmo-usim-sja2 which is currently used for testing somewhat old. In
order to have a defined state here lets exchange it with a new card.
This will also ensure that we have a card with the most recent card
profile in the tester.

Change-Id: Ife13823d0b0a16b1f327dd882f91d0c95af65e43
2021-04-21 12:32:24 +02:00
Robert Falkenberg bba2bd4348 setup: add (nested) library pySim.transport
* setup.py did not install pySim.transport sub-library

Change-Id: Ice026a28c4843121591f3404225c7f6aec4df6c7
2021-04-16 12:43:11 +02:00
Robert Falkenberg 9d16fbca4a Use construct for EF_AD in pySim-{shell, prog, read}.py, cards.py
Also serves as example for RFU (reserved for future use) fields
which should not always be reset to zero in case they have been
set on the uSIM for some reason.
See pySim/ts_51_011.py, class EF_AD.

* Add definitions for RFU {Flag, Bits, Byte, Bytes}
* Use IntEnum for OP_MODE (convenient auto completion)
* Remove obsolete definitions and imports
* Update test results for all SIMs (opmode strings are shortened)

Change-Id: I65e0a426f80a619fec38856a30e590f0e726b554
2021-04-13 11:27:37 +00:00
Harald Welte c8ff026782 pySim-shell: Introduce logical grouping of arguments
Change-Id: Id80a68e615fc1e457bde3dff8817776826fc6d8e
2021-04-11 12:35:25 +02:00
Harald Welte f2e761ce20 pySim-shell: Migrate from optparse to argparse for the main()
We're using argparse internally for all shell commands, and can
use that to auto-generate command reference in the manual.

Let's switch to argparse for the main program, too - and generate
the related reference in the manual.

Change-Id: I77c946dbeb9f746fe3d8051173e59462dc2fb5e2
2021-04-11 12:35:25 +02:00
Harald Welte e4759fd5cd contrib/jenkins.sh: Build and publish PDF manual
Change-Id: I3f01e93dd5a25d26feb3d067a171244a20f0f8e5
2021-04-11 12:20:29 +02:00
Harald Welte 348c195e97 rename manual to be consistent with osmo-gsm-manuals projects
Change-Id: Iad1d96f2b0f95a010b0b30da00d3eef754c1ec44
2021-04-11 12:20:29 +02:00
Harald Welte c9cdce3e02 fix various typos all over the code
Change-Id: Ic8392a951bf94f67b51e35bed95d0e856f7a9250
2021-04-11 12:20:29 +02:00
Harald Welte 790b2709bd lots of file definitions for classic SIM and USIM
Change-Id: I91475df4a5eaca423473aaebba8c69c54c9a0c1a
2021-04-11 12:20:29 +02:00
Harald Welte 9853e247a2 ts_31_102.py: switch EF.Keys from imperative to declarative codec
Change-Id: I215a61c1e44ee2c5df70a8ca96344b4dd4d93c00
2021-04-11 12:20:29 +02:00
Harald Welte 2db5cfb525 filesystem: Introduce 'construct' support
This allows for pure declarative encoders/decoders for linear fixed
and transparent EFs.

Change-Id: Id0765ec3ddb8c044838ce47d0ff8740720a32b15
2021-04-11 12:20:29 +02:00
Harald Welte 703f933b40 pySim-shell: Add open_channel + close_channel commands
Change-Id: I53d9d7f7720eb5f10956bff74ea7ba9fd3b3bd19
2021-04-11 12:20:29 +02:00
Harald Welte a463161ae2 pySim-shell: Adds support for DEACTIVATE FILE + ACTIVATE FILE
Change-Id: I22207dde20f991b0a22dea8f5dd695a0ec99da33
2021-04-11 12:20:29 +02:00
Harald Welte 15fae98d2e pySim-shell: Authenticate (3g) support
This adds support for AUTHENTICATE to the USIM and ISIM application,
based on the newly-introduced 'construct' encoder/decoder support.

Change-Id: Id5697463e29c3dceff98bcf80f5400f7f2bcaa6c
2021-04-11 12:20:29 +02:00
Harald Welte e0f9ef1606 integrate 'construct' python library
'construct' is a declarative symmetric encoder/decoder for user
specified binary formats.  It should come in extremely handy in
tools like pySim.

We start the integration by adding transport methods for transceiving
APDUs with built-in encoding of the command data and decoding of the
response data.

Change-Id: Ibf457aa8b9480a8db5979defcfafd67674303f6c
2021-04-11 12:20:29 +02:00
Robert Falkenberg d0505bdb55 WIP: Add option to set UE operation mode in EF_AD (Administrative Data)
Use ``--opmode=OPMODE`` in cmdline mode or column ``OPMODE`` in csv mode
to specify OPMODE as listed below.

Details:
The ``EF_AD`` field contains administrative data (AD).
It consists of four bytes ``B1``, ``B2``, ``B3``, ``B4``,
and optionally further bytes for future use.

Previous implementation only sets the MNC field appropriately
(located in `B4`) and sets all other bits/bytes to 0.

However, `B1` also defines the *UE operation mode* (see below).
For type approval operations, such as testing with a test uSIM,
this value could be set to `0x80` rather than `0x00`(= normal operation).
This may unlock some UE capabilities that are restricted in
normal operation mode.

Excerpt from [ETSI TS 131 102, 4.2.18](https://www.etsi.org/deliver/etsi_ts/131100_131199/131102/04.15.00_60/ts_131102v041500p.pdf):

```
B1 - UE operation mode:
	Coding:
	Initial value
	- '00' normal operation.
	- '80' type approval operations.
	- '01' normal operation + specific facilities.
	- '81' type approval operations + specific facilities.
	- '02' maintenance (off line).
	- '04' cell test operation.

B2 - Additional information:
	Coding:
	Reserved for future use

B3 - Additional information:
	Coding:
	- B3.b1: OFM setting (Ciphering Indicator)
	- B3.others: Reserved for future use

B4 - Length of MNC in the IMSI:
	Coding:
	- B4.b4..B4.b1: length:  '0010' (= 2) or '0011' (=3)
	- B4.others: Reserved for future use
```

**Legend:** Byte X, bit Y: BX.bY

Further reading: https://nickvsnetworking.com/usim-basics/

Change-Id: Ie9040c6b127c268878a0845ed73d0918ec6bbb08
2021-04-11 09:35:36 +00:00
Harald Welte 08028d02dc README.md: Mention user manual
Change-Id: I6e40166a2ed7ba9f352b466ce85651e20da591e9
2021-04-11 11:21:39 +02:00
Harald Welte 11ee243c47 README.md: We have switched to gerrit for review
Change-Id: I4ee30c411a2dfc2cf39840af2517f28c1989099f
2021-04-11 11:21:39 +02:00
Harald Welte e7887d7af5 README.md: Update dependencies
Change-Id: Ib4028b57aff6bf3d42a83b3942c55f2a1c133172
2021-04-11 11:21:39 +02:00
Harald Welte edee155773 setup.py: add missing comma in list of dependencies
Change-Id: If93d8c67cdfacc1f30307c8cfb38924fb2cbabbe
2021-04-10 18:41:15 +02:00
Harald Welte b00e893e73 pySim-shell: Use poutput_json() whenever applicable
poutput_json() uses a customs JSONEncoder, and hence all JSON
prenting should go through it.

Change-Id: Ie023669e77311350ade4f8dbc65431e71b586052
2021-04-10 18:41:15 +02:00
Harald Welte 5e749a79ac extend JSONEncoder to serialze 'bytes' style objects as hex strings
This means we can skip a lot of code that manually converts from
bytes to hex before JSON serialization.

Change-Id: I9c9eff0556d9d196e64553b5276e162f69d0c18f
2021-04-10 18:41:15 +02:00
Harald Welte 7829d8a357 shell: Add 'apdu_trace' settable parameter for hex-dumping APDUs
Change-Id: I0c957c0b86473413f31e4bd8bc4e633fc1470222
2021-04-10 18:41:15 +02:00
Harald Welte eb05b2f60e transport: Pass arbitrary kwargs to base-class constructor
Change-Id: I3cd5ba87cf53409ea97196d5789ed28eef072c68
2021-04-10 18:41:15 +02:00
Harald Welte c34f9405f1 transport: Make all calls go through base class send_apdu_raw()
This allows us to add APDU tracing at one central location in the code

Change-Id: Id0593a2e6d846cc3151443f1022ae7ee030e6673
2021-04-10 18:41:15 +02:00
Harald Welte 4145d3c4af shell: add edit_{record,binary}_decoded shell commands
This allows the user to edit the file/record contents in its
JSON representation inside the standard system text editor.

Change-Id: Icf6a6e8529e7664c5645519fb4bdd55b35f34664
2021-04-10 18:41:15 +02:00
Harald Welte fe8a74490a pySim/filesystem: Fix more Python 3.5 compatibility issues
Change-Id: I277e86a0e6b16d53d2b5b48b74915ac9a48a4211
2021-04-10 11:52:24 +02:00
Harald Welte 0912d9f3f2 Wavemobile: adjust test expectations about binary EF.AD content
Change-Id: Ib6ebe063a4d0d90b1cdc9bc7ec0773cef8d87fbc
2021-04-10 11:49:08 +02:00
Vadim Yanitskiy a0040792bf pySim/filesystem.py: fix compatibility with Python 3.5
Change-Id: Ia4021551bcd28e6020958964f6f4e9b6dc989d94
Related: OS#5111
2021-04-10 09:48:43 +00:00
Philipp Maier f408a40039 pySim-shell: tree/export: catch errors during DF selection
When a DF is selected, then the method walk() does not catch the
related execption, which causes the command to stop immediately. Lets
also catch those exceptions and generate appropriate error messages
during export.

Change-Id: I24ccf64965e7b0756c3db77eb2084b22c357a570
Related: OS#4963
2021-04-09 21:19:32 +02:00
Philipp Maier 13e258dc85 pySim-shell: be sure that startup script file exists
When a startup script file is specified that does not exists pySim-shell
skips the script and starts normally. This is dangerous because if
pySim-shell is called by a shellscript with a nonexisting script file
the shellscript may hang forever because there is no script that exists
pySim-shell again

Change-Id: I4ff2226c8852727aa23357aa54d1e2d480bfaf2d
Related: OS#4963
2021-04-09 17:46:15 +02:00
Philipp Maier 1bd664da9e tests: remove .example files for simcard tests
The folder pysim-testdata already contains testdata that can also be
used as examples. The .example files are from a time where the testdata
was not kept inside the repository. Since we decided to keep the test
data in the repository as well those file are redundant.

Change-Id: Iee34cad74b50755e1007506f909da9766fa8412e
2021-04-08 15:37:32 +00:00
Philipp Maier dc4402e54f pySim-shell: fix wrong reference to iccid.
It should be self.iccid instead of self._cmd.iccid

Change-Id: Icb7378929187a4fc39a8f1ddbaa18291f16caf79
Related: OS#4963
2021-04-08 15:37:08 +00:00
Harald Welte 850b72ad58 shell: New 'read_records' and 'read_records_decoded' commands
Those commands can be used to read all the records available in the
selected file.

Change-Id: If457b4e02bde8aa6db8cc329411f94411c100bc9
2021-04-07 16:45:12 +00:00
Harald Welte 0d4e98a2ac pySim-shell: JSONpath support for updating files/records
Change-Id: Iad09b3d878b8b58ad34cb549c80f8a6eb3149faa
2021-04-07 16:45:12 +00:00
Robert Falkenberg 75487aed90 Use zero padding for EF['ACC'] field
The ``EF_ACC`` field defines the access control class (ACC)
for a subscriber.

Without this patch, the implementation adds padding 1 towards
the most significant bits if the input is shorter than 2 bytes.

However, it should be padded with 0, otherwise additional ACCs
are allocated to the subscriber. (Probably only a single bit
shall be set to 1)

Excerpt from [ETSI TS 131 102, 4.2.15](https://www.etsi.org/deliver/etsi_ts/131100_131199/131102/04.15.00_60/ts_131102v041500p.pdf):

```
EF_ACC: Two bytes: B1, B2

B1.b8...B1.b4: high priority users (class 15...11)
B1.b3: always 0
B1.b2...B1.b2 and B2.b7...B2.b0: normal priority users (class 9...0) - to be evenly distributed across subscribers
```

**Legend:** Byte X, bit Y: BX.bY


Change-Id: I1b8dc01a6c48adad1ed8158de59b12519ed688e9
2021-04-07 16:23:11 +00:00
Philipp Maier c98ef8a79d ts_102_221.py: fix fixup_fcp_proprietary_tlv_map()
The function fixup_fcp_proprietary_tlv_map() addes propritary TLV
tags in the range of d0 to ff to the TLV map. However, the spec defines
this range as b7 and b8 of the first tag byte set to 1. This results
in a range from c0 to ff. See also ETSI TS 102 221, section 11.1.1.4.6.0

Change-Id: I8359527c9ff303b257b181b87dc440f27735ece9
Related: OS#4963
2021-04-07 16:21:52 +00:00
Robert Falkenberg 5459536c7b SysmoISIM-SJA2: Add option to set Service Provider Name (SPN)
Same implementation as for sysmoUSIM-SJS1

Change-Id: I3a9dd2fe85126584758ea4cfa127f9cd14ab0c7d
2021-04-07 12:18:29 +02:00
Harald Welte d7a7e17a48 utils.py: Add missing dec_plmn function.
This function is being used e.g. for ADF.USIM/EF.FPLMN entries.

The EF_PLMNsel class also already uses a function by this name, we just
haven't had any actual implementation around.

Change-Id: Iacb45c90bb6491ebb89a477a85ef1f3129b38788
2021-04-06 22:53:18 +00:00
Harald Welte f431189a9e pySim/filesystem: Remove left-over debug print statements
the print statements in read_binary_decoded and update_binary_decoded
should have been removed a long time ago.

Change-Id: I9ccc61c426a755fae9008d0717d579fa2da0ef7c
2021-04-07 00:33:02 +02:00
Harald Welte 9ae33c8ad9 docs: Documentation for classic pySim-{prog,read} tools
Particularly the documentation for pySim-prog is far from being
complete, but it's a start.

Change-Id: Ic1932e62a5d7cf33e0dd74cb071cfa7f27c6e497
2021-04-06 21:16:16 +02:00
Harald Welte d36f6943d9 docs/shell.rst: Auto-generate shell command reference
We use a slightly modified version of sphinx-argparse
(with patch https://github.com/alex-rudakov/sphinx-argparse/pull/136 applied)
in order to generate the command reference for each shell command in the
manual.

As the upstream repository seems unmaintained for ~2 years, let's use
the osmocom 'fork' with that above-mentioned patch applied.

Change-Id: I134f267cf53c7ecbc8cbfb33a8766d63bf4a8582
2021-04-06 21:16:16 +02:00
Harald Welte 1748b9372c pySim-shell: Add settable parameter on JSON pretty-printing
Change-Id: Ic095c96733de2b0f359bfe067cd719d38712faff
2021-04-06 21:16:16 +02:00
Harald Welte c9baa4d915 ts_51_011: Full encoder/decoder for EF.AD
The EF.AD class only had a partial decoder and no encoder before this
patch.

You can now do things like

pySIM-shell (MF/ADF.USIM/EF.AD)> read_binary_decoded
{
    "ms_operation_mode": "normal_and_specific_facilities",
    "specific_facilities": {
        "ofm": false
    },
    "len_of_mnc_in_imsi": 2
}
pySIM-shell (MF/ADF.USIM/EF.AD)> update_binary_decoded '{"ms_operation_mode": "normal_and_specific_facilities", "specific_facilities": {"ofm": false}, "len_of_mnc_in_imsi": 3}'

not quite all that elegant yet, but working at all.

Change-Id: Id2cb66cb26b6bd08befe9f8468b0b0773da842b1
2021-04-06 20:20:57 +02:00
Harald Welte bcad86c08c pySim-shell: Add '--oneline' to read_{binary,record}_decoded
This allows for single-line output, rather than the default JSON
pretty-printing

Change-Id: I08e0e7b6f0d796626f4d6c3e9a2622c1ee0c210a
2021-04-06 20:20:57 +02:00