Commit Graph

474 Commits

Author SHA1 Message Date
Harald Welte fb50621570 filesystem: Introduce support for TLV parser
This adds an easy way for files to make use of the pySim.tlv parser.

All a file has to do is to specify a _tlv member which points to
either a TLV_IE or a TLV_IE_Collection instance.

Change-Id: I59f456b4223ec88081e91cee168b654c69bcb5f4
2021-06-05 10:47:17 +02:00
Harald Welte bb3b5df8bf Introduce new object-oriented TLV parser/decoder/encoder
This introduces a new TLV library that heavily builds upon python object
oriented concepts.  Contrary to classic TLV parsers it doesn't focus on
the structure of Tag, Length and binary Value only, but it supports
actual decoding/interpretation of the value part into some kind of JSON
serializable dict.  The latter can be achieved by imperative
encode/decode methods, or by using our existing declarative 'construct'
based approach.

The TLV library supports both BER-TLV and COMPREHENSION-TLV for both
nested and non-nested TLV definitions.

As an example we include TLV definitions for a number of CAT (Card
Application Toolkit) IEs.

Change-Id: I7fc1699443bc9d8a4e7cdd2687af9af7cc03c30e
2021-06-05 10:47:17 +02:00
Harald Welte 8f892fbbe8 ts_102_221: Add construct for contents of EF.UMPC
Change-Id: I7c63ccca90ab34b0d6ac6c990eeb53279ef2cd8d
2021-06-05 10:47:17 +02:00
Harald Welte 07c7b1f592 construct: Recursive normalization of construct parse result
If we want to use construct parse results to generate JSON serializable
dicts, we need to

* apply the filter_dict() operation recursively, and
* simplify the construct Container and ListContainer classes to
  a simple dict and/or list.

We introduce a pySim.construct.parse_construct() helper which is
subsequently used from all pySim.filesystem caller sites.

Change-Id: I319414eb69808ef65895293832bb30519f45949d
2021-06-05 10:47:17 +02:00
Harald Welte 7fca85b42c utils: Make filter_dict() transparently pass non-dict
Change-Id: Ia1802101a62e21f1ce894d80728f939bf3da5a39
2021-06-05 10:47:17 +02:00
Harald Welte f0885b1042 utils: Add bertlv_encode_tag()
We so far had decoders for BER-TLV tags, but no encoder yet.

Change-Id: I4183546bed9d6232ddcefad764f4e67afcf8b2ed
2021-05-30 19:27:37 +02:00
Harald Welte 6912b1b67d utils: Add 'raw' version of TLV tag decoders
The existing {comprehension,ber}tlv_parse_tag() functions are
decoding the tag to a high level of detail.  However, all the 3GPP
specs seem to deal with the 'raw' version, i.e something like
0xD1 as a single-byte tag with the class + constructed fields already
shifted next to the actual tag value.

Let's accommodate that with new *_parse_tag_raw() functions.

Change-Id: Ib50946bfb3b3ecd7942c423ac0f98b6c07649224
2021-05-29 22:21:38 +02:00
Harald Welte 9f3b44d6ff utils: COMPREHENSION-TLV support
Change-Id: I8d969382b73fa152ee09c456fa4aee428fb36285
2021-05-29 22:13:56 +02:00
Harald Welte 485692bc77 shell: Fix activate_file + deactivate_file commands
We cannot re-activate a deactivated file after we have selected somethng
else, as SELECT will fail on the deactivated file.  Hence, the
deactivate_file command needs to be used with a file name as argument.

Change-Id: Ief4d2bf8ea90497a8f25d1986aeea935c615f9bb
2021-05-25 22:23:00 +02:00
Harald Welte 34b05d3707 shell: Add 'status' command to issue STATUS APDU
This can be used to get the FCP of the currently selected file.

Change-Id: I65c97adadd831ca2daa5a0dbb52a37999f8514fd
2021-05-25 22:23:00 +02:00
Harald Welte 8dfd6d070f transport: Add support for SW 6Cxx
According to ETSI TS 102 221 Section 7.2.2.3.1 Table 7.1 the UICC
may respond with SW 6Cxx to tell us to re-issue the command with
a modified P3/Le.

Change-Id: Ia7e6202bbd0f61034a985ecf76d0542d959922ce
2021-05-25 22:23:00 +02:00
Harald Welte e7506036bd Introduce unit test for bertlv_parse_one()
Change-Id: I3adbe22afd4b6503a7454de39b7663e9ede8995f
2021-05-25 09:43:13 +02:00
Harald Welte c1475307c8 bertlv_parse_one: Also return remainder after end of TLV
Change-Id: I10ebd87f72ee934561118b768108e5dc76277660
2021-05-25 09:43:13 +02:00
Harald Welte de02718631 add unit tests for BER-TLV encoder/decoder functions
... and while at it resolve a bug in bertlv_parse_len()
discovered by those new tests.

Change-Id: I9f14dafab4f712c29224c4eb25cacab7885e2b68
2021-05-25 09:43:13 +02:00
Harald Welte c781ab85bc contrib: Add sim-rest-{server,client}.py
sim-rest-server.py can be used to provide a RESTful API to allow remote
clients to perform the authentication command against a SIM card in a
PC/SC reader.

sim-rest-client.py is an example client against sim-rest-server.py
which can be used to test the functionality of sim-rest-server.py.

Change-Id: I738ca3109ab038d4f5595cc1dab6a49087df5886
2021-05-25 09:43:13 +02:00
Harald Welte 0f96c02815 commands: remove superfluous getter/setter for cla_byte property
There's little point in having a getter+setter for a property if
all it does is assigning a value to an attribute of self.  That
works without any property methods

Change-Id: Id214cc83a29e8aa88f4e1413e07b419285c1b7ff
2021-05-23 12:35:29 +00:00
Harald Welte 951f263de7 commands: resolve inconsistency on sel_ctrl
The code uses self.sel_ctrl everywhere except in the two @property
methods, where the _sel_ctrl variable is used.  Let's just abandon
those property methods and make sure all users directly use the
[public] sel_ctrl member variable.

Change-Id: I10362300c1cf7b493d89bf71bbd3a10c80ef9a49
2021-05-23 12:35:29 +00:00
Robert Falkenberg 5933c3b88d pySim-read: adjust meaning of HPLMN/OPLMN flags in EF_SPN
The updated wording better reflects the actual meaning
of a set or unset flag, especially as OPLMN is inverted.

Change-Id: I65c6f0e9bc1a12a4a74c4274eebb8e612296888f
2021-05-23 10:06:08 +00:00
Philipp Maier 86b09da61d ts_31_103: finish decoder and fix encoder for EF.PCSCF
The encoder/decoder functions in class EF_PCSCF look rather unfinshed
because of problems with dec_addr_tlv(), since those problems are fixed
by a previous patch we can now finish the decoder function and fix the
decoder as well.

Change-Id: I7613b8b71624dc5802aca93163788a2a2d4ca345
Related: OS#4963
2021-05-23 10:05:50 +00:00
Philipp Maier be18f2a419 utils: split string formatting from dec_addr_tlv
The function dec_addr_tlv() takes an encoded FQDN or IPv4 address and
fromats it into a human readable string that contains the human readable
form and the encoded hex form. Unfortunately this limits the usecase of
dec_addr_tlv. Lets split the string generation into a separate function
so that we can use dec_addr_tlv universally

Change-Id: Id017b0786089adac4d6c5be688742eaa9699e529
Related: OS#4963
2021-05-23 10:05:50 +00:00
Philipp Maier 42804d7803 commands: pad short data input in update_record()
The method update_record as a "force_len" parameter, which is somewhat
irretatating. Some explainatory comments and a reformat of the if
statement will help to make it more understandable to the api user.

In the non force_len case the method determines the record length from
the select response and throws an exception if the data input does not
match that length. This makes sense if the data input exceeds the
record length of the file but if the data input is less then the record
length the situation is fixable by padding the input with 0xff. This
also a quite common case because in some situation it is not guaranteed
that the data will fill the entire record.

Change-Id: I9a5df0e46c3dd2e87d447c5c01cf15844b0eed07
Related: OS#4963
2021-05-23 10:05:50 +00:00
Harald Welte 59f9a38623 commands: check for status word in USIM authenticate command
Change-Id: I4c7e7261dd597cef0825826b36d50a144efa90d9
2021-05-22 09:32:39 +02:00
Harald Welte 2db843e7b5 transport/pcsc: Raise exception if reader number is out of range
Change-Id: I0acd6900feabb1cfa03b84f24903c6b746a6bdea
2021-05-22 09:32:39 +02:00
Philipp Maier fc5f28db3b cards: populate name property in Card, UsimCard and IsimCard
Even though Card, UsimCard and IsimCard are abstract classes which are
normally only used to inherit from mit may make sense to pre-populate
the name property with some meaningful value.

Change-Id: Id643e1f83718aea073e7200aecbf2db2def8652f
2021-05-21 16:46:00 +02:00
Philipp Maier e2c59a8b91 pySim_prog: remove unused import for dec_addr_tlv
The function dec_addr_tlv is imported from utils, but not used

Change-Id: I2a962d544f288259f16c1dca92715953d1c24d82
Related: OS#4963
2021-05-21 15:26:52 +02:00
Robert Falkenberg f658c55745 Update README
* Add instructions for convenient install on Archlinux
* Update hyperlinks, replace http with https
* Fix incorrect implicit code markup by explicit markup
* Fix Typos, etc.
* Adjust headlines

Change-Id: I96ac0f7caea8a28d2bbeba9e54911b4bd44aaad5
2021-05-17 18:08:45 +02:00
Robert Falkenberg b07a3e9c87 Add codecs for EF_SPN and GSM strings via construct
This will replace the hand-crafted codec for EF_SPN
by a struct definition using the construct library.
Old encoders are updated and kept for API compatibility
but are not used internally anymore.

New data structures:
* Rpad(Adapter): Right-padded bytestring (0xff, adjustable)
* GsmStringAdapter(Adapter): Codec for "SMS default 7-bit
	coded alphabet as defined int TS 23.038" using
	the gsm0338 library.
* GsmString(n): Convenient wrapper of both above

Adjustments:
* utils: update+deprecate old dec_spn(), enc_spn()
* remove refs to deprecated functions

Change-Id: Ia1d3a3835933bac0002b7c52511481dd8094b994
2021-05-10 06:15:39 +02:00
Philipp Maier c957ce8adc ts_51_011: fix encoder of EF_SPN:
The encoder for EF_SPN is passing the 'spn' parameter (which is a list)
directly to enc_spn without taking it apart first.

Change-Id: I0a405793c8909d4279e634b93dcb76e5cb2963f3
Related: OS#4963
2021-05-07 21:52:30 +00:00
Robert Falkenberg 90f7497d6d ts_102_221: add missing TLV key '9B'/'target_ef' for sysmoUSIM-SJS1
Change-Id: I8131bfbbdeb50ddb4d6a06c16586238a36582b5e
2021-05-06 20:46:11 +02:00
Robert Falkenberg dddcc60f2d ModemATCommandLink: improve response time for "+CME ERROR"
Change-Id: I41af33c1898f5ed3d1c5238e45f956c6ceab2826
2021-05-06 20:36:55 +02:00
Robert Falkenberg 7cb7c78ca8 ModemATCommandLink: add/adjust some logging
Change-Id: I303506a751b4a34d83c18bc097e0cfb0517ee82c
2021-05-06 20:23:00 +02:00
Robert Falkenberg e5a5ffb44f ModemATCommandLink: return lower case hexstring
Change-Id: Id56e92962c1d75b832b42516099f97aac5a9d1d3
2021-05-06 09:51:29 +02:00
Philipp Maier 4210a7001d pySim-read: fix wrong comment
Change-Id: Idcbbc6e964f7932a10d55f7f28646f278c994129
2021-05-05 14:22:23 +02:00
Harald Welte 7743c20d2a docs/shell.rst: Document verify_adm and tree commands
Change-Id: I8afd061bc7b93a5488dd1fc135a73b9d7c75e0bb
2021-05-04 13:24:07 +02:00
Harald Welte 903bdaaca8 transport: Mark more methods as abstractmethod
Change-Id: Ied3dbd07fdd0d3fa9bbe2dd7dd674700cf13bf63
2021-05-04 13:24:07 +02:00
Harald Welte daf2b392f0 shell: Add 'reset' command to reset the card
At some points during an interactive session or a script one may want
to reset the card.

Change-Id: I992eb3e0ed52f7941a5fb44f28a42e22ebd49301
2021-05-04 13:24:07 +02:00
Harald Welte 917d98c1a5 BER-TLV EF support (command, filesystem, shell)
This adds support for a new EF file type: BER-TLV files.  They are
different from transparent and linear fixed EFs in that they neither
operate on a byte stream nor fixed-sized records, but on BER-TLV encoded
objects.  One can specify a tag value, and the card will return the
entire TLV for that tag.

As indicated in the spec, the magic tag value 0x5C (92) will return a
list of tags existing in the file.

Change-Id: Ibfcce757dcd477fd0d6857f64fbb4346d6d62e63
2021-05-04 13:24:07 +02:00
Harald Welte fc4833ec20 ts_31_103: Use EF_ARR decoder from TS 102 221
We already used that decoder also in ts_31_102

Change-Id: Iaab9f038544b5914405568b072731d3847c9d017
2021-05-04 13:24:07 +02:00
Harald Welte 4ae228afc7 Implement EF.ARR (Access Rule Reference) decoding
The Access Mode (AM) and Security Condition (SC) DOs are incredibly
convoluted, so we need a lot of code to properly decode them.

Change-Id: If4f0725a849d41fd93de327ed00996d8179f2b0e
2021-05-04 13:24:07 +02:00
Harald Welte 90441436a0 utils: Introduce CommandSet abstraction
This will allow us to match INS -> name and add more related
bits in the future (e.g. for decoding APDU traces)

Change-Id: I314ff15186dc05778ea12363cac0a310b6c7713c
2021-05-04 13:24:01 +02:00
Harald Welte 3de6ca2d20 utils: Introduce DataObject representation
Represents DataObject (DO) in the sense of ISO 7816-4.  Contrary to
'normal' TLVs where one simply has any number of different TLVs that may
occur in any order at any point, ISO 7816 has the habit of specifying
TLV data but with very specific ordering, or specific choices of tags at
specific points in a stream.  This is represented by DataObjectChoice,
DataObjectCollection and DataObjectSequence classes.

Change-Id: Iac18e7665481c9323cc7d22a3cd93e3da7869deb
2021-05-03 21:45:39 +02:00
Philipp Maier f39a4cb369 utils: specify type of parameter name in enc_spn
Related: OS#4963
Change-Id: I43a1e68afe9e756346bc0cfe8bda4ac665ac6c54
2021-05-03 17:08:37 +02:00
Philipp Maier e7d417955d ts_51_011, utils: fix Access Technology Identifier coding
When the Access Technology Identifier encoder sets the bits for E-UTRAN
it does not respect that bit "100" is also a valid bit combination that
encodes E-UTRAN WB-S1 and E-UTRAN NB-S1. Lets encode this bit
combination if the user is just specifying "E-UTRAN" without further
spefication of WB or NB.

The decoder only looks at bit 14 and decodes "1xx" always to "E-UTRAN".
This is not specific enough. Lets make sure that the decoder is
complementary to the encoder.

Change-Id: Ibfe8883a05f9ad6988d8e212cb9a598229954296
Related: OS#4963
2021-05-03 17:08:37 +02:00
Philipp Maier b919f8bd75 utils: fix dec_xplmn_w_act() and format_xplmn_w_act()
The function dec_xplmn_w_act(), which is also used by
format_xplmn_w_act() is using integer numbers as MCC/MNC representation.
This causes various problems since the information about leading zeros
gets lost.

Change-Id: I57f7dff80f48071ef9a3732ae1088882b127a6d4
2021-05-03 15:08:27 +00:00
Philipp Maier 6c5cd8031d utils: fix mcc/mnc encoding in dec_plmn (EF_PLMNsel)
The dec_plmn function takes an hexstring and returns the decoded MCC and
MNC as integer values. The result is then used by the json encoder in
EF_PLMNsel, which means the json output will contrary to the input, use
integer values instead of strings.

This is not correct since there may be leading zeros (e.g. mnc 01 and
001 both exist are different) which must be retained in order to know
the correct length of the MNC.

Related: OS#4963
Change-Id: I393e04836814d992d2a6d0a4e4e01850976d6e81
2021-05-03 15:07:50 +00:00
Philipp Maier e6f8d683e1 utils: specify paremeters of enc_plmn() as Hexstr
To prevent missunderstandings when using enc_plmn(), specify the input
and return parameters as Hexstr.

Change-Id: I57cf8e2de357650aef2a06fbffc7615ccb2a45b4
Related: OS#4963
2021-05-03 14:19:41 +00:00
Vadim Yanitskiy 52efc0372c ModemATCommandLink: fix AttributeError exception in __del__()
self._sl will not be assigned if serial.Serial() fails.

Change-Id: I179a7efd92eaa3469a17b6ef252b3949c44ea6ea
2021-05-03 14:13:14 +00:00
Robert Falkenberg 18fb82bd38 Speed up AT command interface (~130 times faster)
Previous implementation waits 300ms for response after
each command issued. But many commands finish earlier.

This patch improves the command execution time by frequently
checking for the response to complete (i.e. ends with
OK or ERROR), or the occurence of a timeout (default 200ms).

Timeout can be adapted per command to support long response
times of certain commands like AT+COPS=? (network search)

Execution time benchmark (20 AT commands/responses):
Previous: 6.010s (100.0%)
New code: 0.045s (  0.7%)

Change-Id: I69b1cbc0a20d54791e5800bf27ebafc2c8606d93
2021-05-03 10:06:10 +02:00
Vadim Yanitskiy e9fe09baff contrib/jenkins.sh: run pylint to find potential errors
Change-Id: Ib4cdd0fa824329569f973925ba3a46656a8c8296
2021-05-02 20:22:49 +00:00
Vadim Yanitskiy 895fa6f83c [pylint] Fix referencing undefined variable 'shutil'
Let's just use the scope limited TemporaryDirectory() instead, so
the temporary directory will be removed by Python automatically.

pySim/filesystem.py:679:16: E0602: Undefined variable 'shutil' (undefined-variable)

Change-Id: I4ea833fd79f4342c33899124379be509ba1e35ed
2021-05-02 20:22:15 +00:00