Commit Graph

24 Commits

Author SHA1 Message Date
Harald Welte 6db681924c pylint: tlv.py
pySim/tlv.py:29:0: W0401: Wildcard import pySim.exceptions (wildcard-import)
pySim/tlv.py:43:4: C0204: Metaclass class method __new__ should have 'mcs' as first argument (bad-mcs-classmethod-argument)
pySim/tlv.py:66:4: C0204: Metaclass class method __new__ should have 'mcs' as first argument (bad-mcs-classmethod-argument)
pySim/tlv.py:89:11: C0121: Comparison 'self.decoded == None' should be 'self.decoded is None' (singleton-comparison)
pySim/tlv.py:170:8: R1703: The if statement can be replaced with 'return bool(test)' (simplifiable-if-statement)
pySim/tlv.py:202:4: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/tlv.py:257:4: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/tlv.py:308:4: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/tlv.py:383:15: C0121: Comparison 'tag == None' should be 'tag is None' (singleton-comparison)
pySim/tlv.py:382:17: W0612: Unused variable 'r' (unused-variable)
pySim/tlv.py:389:16: W0612: Unused variable 'dec' (unused-variable)
pySim/tlv.py:461:22: R1718: Consider using a set comprehension (consider-using-set-comprehension)
pySim/tlv.py:473:0: C0206: Consider iterating with .items() (consider-using-dict-items)
pySim/tlv.py:473:58: C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary)
pySim/tlv.py:20:0: W0611: Unused Optional imported from typing (unused-import)
pySim/tlv.py:20:0: W0611: Unused Dict imported from typing (unused-import)
pySim/tlv.py:20:0: W0611: Unused Any imported from typing (unused-import)
pySim/tlv.py:21:0: W0611: Unused bidict imported from bidict (unused-import)
pySim/tlv.py:28:0: W0611: Unused LV imported from pySim.construct (unused-import)
pySim/tlv.py:28:0: W0611: Unused HexAdapter imported from pySim.construct (unused-import)
pySim/tlv.py:28:0: W0611: Unused BcdAdapter imported from pySim.construct (unused-import)
pySim/tlv.py:28:0: W0611: Unused BitsRFU imported from pySim.construct (unused-import)
pySim/tlv.py:28:0: W0611: Unused GsmStringAdapter imported from pySim.construct (unused-import)
pySim/tlv.py:29:0: W0614: Unused import(s) NoCardError, ProtocolError, ReaderError and SwMatchError from wildcard import of pySim.exceptions (unused-wildcard-import)

Change-Id: Ic22d00d3ae73ad81167276d9482b7b86a04476ba
2024-02-05 17:54:30 +01:00
Harald Welte e55fcf66bf Be more conservative in our imports
Try to avoid '*' from anything into various modules, polluting the
namespace.

Change-Id: Iba749d18e1863ded88ba2d2183e2e8d718b2d612
2024-02-04 21:27:00 +01:00
Harald Welte 1f7a9bd5b4 TLV: Add DGI encoding of "GP Scripting Language Annex B"
The DGI encoding is specified in Annex B of the
"GlobalPlatform Systems Scripting Language Specification v1.1.0"

which is an "archived" specification that is no longer published
by GlobalPlatform, despite it being referenced from the GlobalPlatform
Card Specification v2.3, which is the basis of the GSMA eSIM
specifications.

For some reason it was the belief of the specification authors that
yet another format of TLV encoding is needed, in addition to the BER-TLV
and COMPREHENSION-TLV used by the very same specifications.

The encoding of the tag is not really specified anywhere, but I've only
seen 16-bit examples.  The encoding of the length is specified and
implemented here accordingly.

Change-Id: Ie29ab7eb39f3165f3d695fcc1f02051338095697
2024-01-14 17:42:01 +01:00
Harald Welte 9a48aea263 fileystem/tlv: remove unused imports
Change-Id: I519c7792c7fbe18be63ddc77d211f0d034afcd1f
2024-01-02 21:13:30 +01:00
Harald Welte 69bdcf5022 Fix TLV_IE_Collection.from_tlv in certain situations
The existing code used to produce an empty output in situations where a
TLV_IE_Collection would be parsed from a single TLV only with some
additional trailing padding:

>>> from pySim.utils import h2b
>>> from pySim.ts_31_102 import EF_CSGT
>>> t = EF_CSGT.Csgt_TLV_Collection()
>>> t.from_tlv(h2b('8906810300666f6fff'))
[TextCsgType(foo)]
>>> t.to_dict()
[]

This was caused by an early return (actually returning the decoded
result) but *without updating self.children*.

Change-Id: I1c84ccf698c6ff7e7f14242f9aaf7d15ac2239f4
2023-12-29 18:51:25 +01:00
Harald Welte a9b21bdb1f tlv: Fix from_dict() symmetry
the to_dict() method generates a {class_name: value} dictionary,
for both the nested and non-nested case.  However, before this patch,
the from_dict() method expects a plain list of child IE dicts
in the nested case.  This is illogical.

Let's make sure from_dict always expectes a {class_name: value} dict
for both nested and non-nested situations.

Change-Id: I07e4feb3800b420d8be7aae8911f828f1da9dab8
2023-12-27 15:14:48 +00:00
Harald Welte caef0df663 construct/tlv: Pass optional 'context' into construct decoder/encoder
The context is some opaque dictionary that can be used by the
constructs; let's allow the caller of parse_construct,  from_bytes,
from_tlv to specify it.

Also, when decoding a TLV_IE_Collection, pass the decode results of
existing siblings via the construct.

Change-Id: I021016aaa09cddf9d36521c1a54b468ec49ff54d
2023-12-23 09:15:47 +00:00
Harald Welte 0f7d48ed69 tlv: Fix encoding of zero-valued TLVs
If a TLV was elementary (no nested IEs), and it had only a single
integer content whose value is 0, we erroneously encoded that as
zero-length TLV (len=0, no value part):

>>> rf = pySim.euicc.RefreshFlag(decoded=0);
>>> rf.to_bytes()
b''
>>> rf.to_tlv()
b'\x81\x00'

After this change it is correct:

>>> rf = pySim.euicc.RefreshFlag(decoded=0);
>>> rf.to_bytes()
b'\x00'
>>> rf.to_tlv()
b'\x81\x01\x00'

Change-Id: I5f4c0555cff7df9ccfc4a56da12766d1bf89122f
2023-12-07 13:19:52 +00:00
Harald Welte 880db37356 flatten_dict_lists(): Don't flatten lists with duplicate keys
If we have a list of dicts, and we flatten that into a dict: Only
do that if there are no dicts with duplocate key values in the list,
as otherwise we will loose information during the transformation.

Change-Id: I7f6d03bf323a153f3172853a3ef171cbec8aece7
Closes: OS#6288
2023-12-06 09:02:38 +01:00
Harald Welte 6663218ab8 docs: Fix docstring syntax to avoid warnings
pySim/tlv.py:docstring of pySim.tlv.IE.from_bytes:1: ERROR: Unknown target name: "part".
pySim/tlv.py:docstring of pySim.tlv.IE.to_bytes:1: ERROR: Unknown target name: "part".

Change-Id: I170176910c4519005b9276dbe5854aaaecb58efb
2023-11-02 21:45:54 +00:00
Harald Welte 2352f2dcdd pySim/tlv.py: Fix TLV_IE_Collection from_dict with nested collections
This is all quite complicated.  In general, the TLV_IE.to_dict() method
obviously is expected to return a dict (with key equal to the snake-case
name of the class, value to the decode IE value).  This single-entry
dict can then be passed back to the from_dict() method to build the
binary representation.

However, with a TLV_IE_Collection, any TLV_IE can occur any number of
times, so we need an array to represent it (dict would need unique key,
which doesn't exist in multiple instances of same TLV IE).  Hence, the
TLV_IE_Collection.to_dict() method actually returns a list of dicts,
rather than a dict itself.  Each dict in the list represents one TLV_IE.

When encoding such a TLV_IE_Collection back from the list-of-dicts, we
so far didn't handle this special case and tried to de-serialize with
a class-name-keyed dict, which doesn't work.

This patch fixes a regression in the aram_store_ref_ar_do pySim-shell
command which got introduced in Change-Id I3dd5204510e5c32ef1c4a999258d87cb3f1df8c8

While we're fixing it, add some additional comments to why things are
how they are.

Change-Id: Ibdd30cf1652c864f167b1b655b49a87941e15fd5
2023-07-11 08:42:12 +02:00
Harald Welte ba955b650e pySim/tlv.py: Don't create an exception from within raise
An invalid variable used in a raise ValueError() would cause a further
exception, depriving the user of a meaningful error message.

Change-Id: I6eb31b91bd69c311f07ff259a424edc58b57529a
2023-07-11 08:42:12 +02:00
Harald Welte 30de9fd8ab TLV_IE_Collection: use snake-style names during from_dict()
The TLV_IE_Collection, just like the individual TLV classes, do
use their snake-style names when converting from binary to dict
using the to_dict() method.  It is inconsistent (and a bug) to
expect the CamelCase names during encoding (from_dict).  After all,
we want the output of to_dict() to be used as input to from_dict().

Change-Id: Iabd1ad98c3878659d123eef919c22ca824886f8a
2023-07-11 08:42:12 +02:00
Harald Welte 579ac3ec0e tlv: Fix IE.from_dict() method
The existing IE.from_dict() method *supposedly* accepts a dict as
input value, but it actually expects the raw decoded value, unless it is
a nested IE.  This is inconsistent in various ways, and results in a bug
visible at a higher layer, such as files like EF.{DOMAIN,IMPI,IMPU},
which are transparent files containing a single BER-TLV IE.

Decoding such files worked, but re-encoding them did not, due to the
fact that we'd pass a dict to the from_dict method, which then gets
assigned to self.decoded and further passed along to any later actual
encoder function like to_bytes or to_tlv.  In that instance, the dict
might be handed to a self._construct which has no idea how to process
the dict, as it expects the raw decoded value.

Change-Id: I3dd5204510e5c32ef1c4a999258d87cb3f1df8c8
Closes: OS#6073
Related: OS#6072
2023-06-27 09:29:37 +02:00
Harald Welte 7d8029eb23 tlv: Use self._compute_tag() method rather than direct self.tag
The TLV_IE.from_tlv() method is part of a base class that is inherited
by more specific classes.  The official way to obtain the tag is the
inherited-class-provided self._compute_tag() method, and *not* a direct
reference to the self.tag member.

This allows for some more obscure TLV parsers, such as the upcoming one
for Proactive Commands in the CAT/OTA context.

Change-Id: I0cd70e31567edc5a0584336efcb5e4282734f6dd
2022-08-06 13:19:16 +02:00
Harald Welte ea600a8451 tlv: Make NotImplementedError more verbose
This helps to understand immediately _what_ is not implemented for which
type.

Change-Id: I017eb4828e9deee80338024c41c93c0f78db3f3b
2022-07-20 19:35:58 +02:00
Harald Welte e4a6eafc6f tlv: Don't raise exception if somebody passes empty data to TLV decoder
Change-Id: Id46994029d9b3cd6b67f4f7ee619466602cc8142
2022-02-14 00:44:55 +01:00
Harald Welte e8d177d88f tlv: Convert CamelCase class name to snake_case in json
Our hand-written JSON so far is using snake_case identifiers,
while the JSON generated by the pySim.tlv classes use the class
names as keys, which LooksQuiteDifferent.

So let's auto-convert the CamelCase into something that reflects
our existing notion.

Change-Id: Id55929ef03dc48cb668e6ba7e99b6b291680a42f
2022-02-12 08:33:37 +00:00
Harald Welte 9a2a6691b0 tlv: Function for flattening the list-of-dict output of TLV decoder
Before:
{
    "FcpTemplate": [
        {
            "FileDescriptor": {
                "shareable": true,
                "file_type": "df",
                "structure": "no_info_given"
            }
        },
        {
            "FileIdentifier": "3f00"
        },
        {
            "ProprietaryInformation": [
                {
                    "UiccCharacteristics": "71"
                },
                {
                    "AvailableMemory": 123052
                }
            ]
        },
        {
            "LifeCycleStatusInteger": "operational_activated"
        },
        {
            "SecurityAttribReferenced": {
                "ef_arr_file_id": "2f06",
                "ef_arr_record_nr": 2
            }
        },
        {
            "PinStatusTemplate_DO": [
                {
                    "PS_DO": "40"
                },
                {
                    "KeyReference": 1
                },
                {
                    "KeyReference": 129
                }
            ]
        },
        {
            "TotalFileSize": 187809
        }
    ]
}

After:
{
    "FcpTemplate": {
        "FileDescriptor": {
            "shareable": true,
            "file_type": "df",
            "structure": "no_info_given"
        },
        "FileIdentifier": "3f00",
        "ProprietaryInformation": {
            "UiccCharacteristics": "71",
            "AvailableMemory": 123052
        },
        "LifeCycleStatusInteger": "operational_activated",
        "SecurityAttribReferenced": {
            "ef_arr_file_id": "2f06",
            "ef_arr_record_nr": 2
        },
        "PinStatusTemplate_DO": {
            "PS_DO": "40",
            "KeyReference": 129
        },
        "TotalFileSize": 187809
    }
}

Change-Id: Ia5ad8f1d3b0d47ebdb1856b0feaba120bad3eef9
2022-02-11 17:13:01 +01:00
Harald Welte c91085e744 cosmetic: Switch to consistent four-spaces indent; run autopep8
We had a mixture of tab and 4space based indenting, which is a bad
idea.  4space is the standard in python, so convert all our code to
that.  The result unfortuantely still shoed even more inconsistencies,
so I've decided to run autopep8 on the entire code base.

Change-Id: I4a4b1b444a2f43fab05fc5d2c8a7dd6ddecb5f07
2022-02-11 13:32:58 +01:00
Harald Welte 5895380a45 tlv: Fix recursive initialization from_dict()
When calling from_dict() on a hierarchy of nested BER_TLV_IE,
only the first/outer layer of TLV_IE_Collection would get its
'decoded' initialized correctly from the dict.  Subsequent layers
were not, as the 'decoded=' was passed as parameter during instance
initialization.  If we first instantiate the class and then call the
from_dict() method, the recursive initialization down the full hierarchy
works as expected.

Change-Id: I795a33ed8dfa8454dc9079c189ab7b2ba64a3b72
2021-10-21 14:12:13 +02:00
Harald Welte 04c1302b68 tlv: Don't require encoder/decoder methods for TLV without value
There are instances where a TLV IE is used as just a flag, i.e.
length zero and no value part.  In those situations, it would require
a lot of boilerplate code to require the TLV_IE class definitions to
have _to_bytes/_from_bytes methods that do nothing.

So instead, add a shortcut: If we want to encode 'None', then return
b'', and if we want to decode b'' return None.

Change-Id: Ie8eb2830e8eefa81e94b8b8b157062c085aeb777
2021-10-21 14:12:13 +02:00
Harald Welte fb50621570 filesystem: Introduce support for TLV parser
This adds an easy way for files to make use of the pySim.tlv parser.

All a file has to do is to specify a _tlv member which points to
either a TLV_IE or a TLV_IE_Collection instance.

Change-Id: I59f456b4223ec88081e91cee168b654c69bcb5f4
2021-06-05 10:47:17 +02:00
Harald Welte bb3b5df8bf Introduce new object-oriented TLV parser/decoder/encoder
This introduces a new TLV library that heavily builds upon python object
oriented concepts.  Contrary to classic TLV parsers it doesn't focus on
the structure of Tag, Length and binary Value only, but it supports
actual decoding/interpretation of the value part into some kind of JSON
serializable dict.  The latter can be achieved by imperative
encode/decode methods, or by using our existing declarative 'construct'
based approach.

The TLV library supports both BER-TLV and COMPREHENSION-TLV for both
nested and non-nested TLV definitions.

As an example we include TLV definitions for a number of CAT (Card
Application Toolkit) IEs.

Change-Id: I7fc1699443bc9d8a4e7cdd2687af9af7cc03c30e
2021-06-05 10:47:17 +02:00