Commit Graph

632 Commits

Author SHA1 Message Date
Harald Welte aefd0649a2 pySim-shell: Add 'decode_hex' command for transparent + linear EF
These commands can be used to decode a user-provided hex-string,
instead of decoding the data read from the file.  This is useful
for quickly manually decoding some values read from other locations,
such as e.g. copy+pasted from a eSIM profile in ASN.1 value notation.

Change-Id: I81f73bce2c26e3e5dfc7538d223bb2d2483c7fa0
2022-03-01 16:48:22 +00:00
Harald Welte 34eb504b3b Initial support for GlobalPlatform
One can now select the Issuer Security Domain (hard-coded to
a000000003000000) and issue get_data requests.  FCI and other TLV
objects are dcoded, e.g.

pySIM-shell (MF)> select ADF.ISD
{
    "application_id": "a000000003000000",
    "proprietary_data": {
        "maximum_length_of_data_field_in_command_message": 255
    }
}
pySIM-shell (MF/ADF.ISD)> get_data CardData
{
    "card_data": [
        {
            "card_recognition_data": [
                {
                    "object_identifier": "2a864886fc6b01"
                },
                {
                    "card_management_type_and_version": [
                        {
                            "object_identifier": "2a864886fc6b02020101"
                        }
                    ]
                },
                {
                    "card_identification_scheme": [
                        {
                            "object_identifier": "2a864886fc6b03"
                        }
                    ]
                },
                {
                    "secure_channel_protocol_of_isd": [
                        {
                            "object_identifier": "2a864886fc6b040215"
                        }
                    ]
                }
            ]
        }
    ]
}

Change-Id: If11267d45ab7aa371eea8c143abd9320c32b54d0
2022-03-01 16:32:15 +00:00
Harald Welte a037762b04 ts_31_102: Further decode TAI in EF.OPL5G
The TAI is not just an opaque bytestring but it consists of 3 fields.

Change-Id: Ie5a5ce74713deb0e151218ae553d3f3d96cef17d
2022-02-25 15:45:09 +01:00
Harald Welte 3a5afff022 ts_31_102: Further decode LAI in EF_LOCI
Change-Id: I21d9356e541eb320848a373804781ae0bef7d012
2022-02-25 15:45:02 +01:00
Harald Welte 1459e45005 ts_51_011: Better decode of EF_OPL LAI
before:
{
    "lai": "62f2300000fffe",
    "pnn_record_id": 1
}

after:
{
    "lai": {
        "mcc_mnc": "262f03",
        "lac_min": "0000",
        "lac_max": "fffe"
    },
    "pnn_record_id": 1
}

Change-Id: I82581220e9c33a8e67cbefd5dfeb40bbc2c31179
2022-02-25 15:44:26 +01:00
Harald Welte 22a1cdde25 ts_51_011: Properly decode EF.OPL
The OPL has 7 bytes "LAI" as the LAI actually contains a LAC
range (so two more bytes for the end of the 16bit range).

Change-Id: I74bcf10b0a8977af0f2844044a812c5780af1706
2022-02-25 15:31:16 +01:00
Harald Welte dd45d8ee3b ts_31_102: Fix decoding of UServiceTable
range(0,7) in python is 0..6, and not 0..7, so we need range(0.8)
to produce the desired range covering all bits of a byte.

This resulted in services 8,16,24,... not being displayed in
the decoded output of EF.UST / EF.IST.

Change-Id: I22bbc481de342685352bf5b13d54931d3f37f9b7
2022-02-25 15:31:16 +01:00
Harald Welte 4ebeebffca ts_102_221: Fix decoding the 'num_of_rec' field
It is a 8bit integer, not a 16bit integer.  See TS 102 221 11.1.1.4.3

Change-Id: I3e258547dad21a248650cfbc02e0576268d3b3fd
2022-02-25 09:48:20 +01:00
Harald Welte 5e9bd93bbd ts_102_221: properly decode short file identifier
The SFI TLV contanins not the raw SFI, but it contains the SFI
shifted to left by 3 bits (for some strange reason).  So let's
un-shift it.

Change-Id: Ibc69b99010d2a25cbb69b6a3d1585d0cb63f1345
2022-02-25 09:37:40 +01:00
Harald Welte fa578bd601 add scripts/deactivate-ims.script to deactivate IMS related services
Change-Id: I0cd93c8fa0024dd9d93647c565190abe94d3097e
2022-02-21 09:57:09 +01:00
Harald Welte c89a1a99ca Add scripts/deacivate-5g.script
This script can be used to deactivate all 5G related services and files.

Change-Id: I5dc3e9f0ae76a7ae57484e5a3369e11ff02c7eca
2022-02-17 12:42:14 +01:00
Harald Welte 12af793d4b doc: Improve documentation in various places
* don't duplicate information between .rst files and docstrings
* if there's more than a trivial single-line documentation, put it as
  docstring into the python source and use ".. argparse" to pul it into
  the manual
* add documentation for some commands for which it was missing
* show one level deeper in the navigation table, listing the commands

Change-Id: Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5
2022-02-15 16:40:45 +01:00
Harald Welte d01bd3632c docs: Document missing 'status' command in 7816 section
Change-Id: I9af85a36bc4f24c3a22b9b2a6b8e2abd86edfe4e
2022-02-15 15:56:48 +01:00
Harald Welte 799c354827 shell: Proper argparser (for help + manual) activate_file
Change-Id: I5929ae3deff4d15b5db4a1d866576271c57a955f
2022-02-15 15:56:28 +01:00
Harald Welte 2bb17f3df9 pySim-shell: export: Add FCP template to export
The FCP template provides us a lot of context, like the permissions of
a given file.  Let's make it part of the 'export' output, both in raw
and in decoded form.

Change-Id: I05f17bbebd7a9b3535204b821900851a5f66e88f
Closes: OS#5457
2022-02-15 15:41:55 +01:00
Harald Welte 9e241435cc docs/legcay.txt: Point to pySim-shell as replacement
Change-Id: I9ca6b9d8c35e23be2ec8752107bb7d1e4f6f9bc1
2022-02-15 15:38:19 +01:00
Harald Welte 3c9b784825 pySim-shell: support TS 102 222 administrative commands
This adds support for creating/deleting and terminating files,
as well as support for permanent card termination.

Change-Id: I5b1ffb1334afa18d62beb642268066a30deb7ea6
2022-02-15 15:35:36 +01:00
Harald Welte 747a978478 ts_102_221: Implement File Descriptor using construct
This automatically adds encoding support, which is needed for upcoming
CREATE FILE support.

Change-Id: Ia40dba4aab6ceb9d81fd170f7efa8dad1f9b43d0
2022-02-15 15:35:36 +01:00
Harald Welte ee670bc1c6 pySim-shell: Allow selecting of deep paths like DF.GSM/EF.IMSI
With this patch applied, users can directly enter commands like

select DF.GSM/EF.IMSI or
select ADF.USIM/DF.5GS/EF.5GAUTHKEYS

This feature doesn't have tabl completion, so it's mostly useful
for when you know what to select, or for use within scripts.

Change-Id: I681a132eb2df4b2aba4c2ccbdd21c6d5b88443e3
2022-02-15 15:35:36 +01:00
Harald Welte 226b866f51 ts_31_103: TLV definitions for IMS, XCAP and MudMid configuration
Change-Id: I9a90ee978db668a70259eb48085ff5384cf696d6
2022-02-15 15:35:36 +01:00
Harald Welte 540adb0ee6 ts_51_011: EF_CMI: Decoder the alpha_id string
Change-Id: I45efe29ab98972945b4257229a995815f5632536
2022-02-15 15:35:36 +01:00
Harald Welte 1e73d228f4 ts_51_011: Convert EF_ADN and EF_ACC to Construct
this has the benefit of providing encoding support for free.

Change-Id: I31c118082e92892486c3688de2197c0c6dd2750e
2022-02-15 15:35:36 +01:00
Harald Welte bc0e209a9f ts_51_011: Proper decode of EF.SMSP
Full decode of the SSM Parameters File

Change-Id: Iac5bb87ed3350978dc8b207f052510fdba2e4883
2022-02-15 15:35:35 +01:00
Harald Welte 3bb516b2b1 Improve IST/UST check documentation (for the user manual)
Change-Id: I18093d795721f2e729eff858c8922edde9e84451
2022-02-15 15:35:35 +01:00
Harald Welte aceb2a548a ust_service_check: proper treatment of files in sub-directories
We must not only consider files in the current directory (ADF.USIM)
but also in its sub-directories.  This requires us to be able to
determine the path we need to traverse between the currently selected
file (EF.UST) and the respective file in some other directory,
which is implemented via CardFile.build_select_path_to().

Change-Id: I61797fefa9dafa36a8a62c11aa2cfaeecb015740
2022-02-15 15:35:35 +01:00
Harald Welte 419bb496e1 ts_31_102: service annotations for DF.{5GS,WLAN,HNB}
We had service annotations only for ADF.USIM so far, but not for
the related sub-directories.

Change-Id: Iaa56a26ba53eaf18fce14845ae07a27c52a2c58a
Note: The code doesn't make use of them in any reasonable way yet!
2022-02-15 15:35:35 +01:00
Harald Welte fa8b8d1160 ts_31_102: Use perror() instead of poutput() for errors
This adds colorization and ensures they go to stderr and not stdout

Change-Id: I34b8f974b4ff13002679c4700bdf604db7d7f3cd
2022-02-15 15:35:35 +01:00
Harald Welte 82f75c200f ts_31_102: Add more EF.UST checks to 'ust_service_check' command
* check for service dependencies listed in TS 31.102
* print number of errors encountered

Change-Id: Id47f8f2c8de299bbf91243d0c8900d22a7d35b10
2022-02-15 15:35:35 +01:00
Harald Welte d53918c3e1 filesystem: Fix CardMF.get_app_names()
This function was not used and doesn't work without this patch.

Change-Id: Id3dad7d97fe29a25792d2f8f0e879666c1d9c136
2022-02-15 15:35:35 +01:00
Harald Welte 6ca2fa7a5d Split EF.UST handling from EF.IST and EF.SST
The existing code had the following serious problems:
* when trying to update EF.SST or EF.IST, it would write to EF.UST !
* shell commands were called ust_* even for the EST/IST files

Let's introduce the proper separation between what is shared and what
is file-specific.

Change-Id: Ie55669ca37a4762fac9f71b1db528ca67056e8dd
2022-02-15 15:35:35 +01:00
Harald Welte 4c5e2310fa ts_31_102: Add "ust_service_check" command.
This command performs a consistency check between the services activated
in EF.UST/EF.IST and the files that should (or should not) be
active/selectable for the given service.

Produces output like:

Checking service No 48 (inactive)
  ERROR: File EF(EF.MWIS) is selectable but should not!
Checking service No 49 (active)
  ERROR: File EF(EF.CFIS) is not selectable (SW=6a82) but should!

Change-Id: Iea7166959e2015eb8fa34d86036560c9e42ce4d3
2022-02-15 15:35:35 +01:00
Harald Welte d16d904c57 README.md: Remove old usage examples, refer to user manual instead
We want people to use pySim-shell and should not mislead them by
having usage examples of old tools in README.md.  Also, all
documentation should be in the manuals, let's try to have bits
and pieces in various places.

Change-Id: I8c07a2e0778ab95fb42be6074acb80874e681d20
2022-02-15 15:35:35 +01:00
Harald Welte 3729c47651 commands: Add method to select parent DF ("cd ..")
This is useful when walking around the filesystem tree.

Change-Id: Ib256c1b7319f2b5f9a06200fb96854ecb2b7f6bb
2022-02-14 00:51:27 +01:00
Harald Welte a630a3cd28 cosmetic: Remove extraneous empty lines between spec-section-comment and class
This is an artefact of the recent autopep8 re-formatting.

Change-Id: I8b0e7781719d69e18856ada2f482de2c5396bcc3
2022-02-14 00:51:27 +01:00
Harald Welte 6169c72f82 USIM + ISIM: Specify the services associated with each file
This allows us [in a future patch] to perform consistency checking,
whether files exist for services not activated in EF.{UST,IST} or
vice-versa: Services are activated by files are not present or
deactivated.

Change-Id: I94bd1c3f9e977767553000077dd003423ed6dbd1
2022-02-14 00:51:27 +01:00
Harald Welte 9170fbf08d filesystem: Maintain a 'service' attribute for all files on a card
This can be populated by card profiles with the SST/IST/UST service
that is associated with the file.

Change-Id: I3b3f74b691368fa09967ecb377a9f7a6d8af7869
2022-02-14 00:51:22 +01:00
Harald Welte afb8d3f925 pySim-shell: introduce 'apdu' command for sending raw APDU to card
This can be useful when playing around with cards, for example
sending commands for which pySim-shell doesn't yet have proper support.

Change-Id: Ib504431d26ed2b6f71f77a143ff0a7fb4f5ea02e
2022-02-14 00:48:16 +01:00
Harald Welte 08b11abc2f pySim-shell: export: allow export as JSON instead of hex
The primary use case of the --json option is to systematically execute
all of our decoder classes in order to find bugs.  As we don't have
encoders for all files yet, the output generated by 'export --json'
will in many cases not be executable as script again, unlike the normal
'export' output.

Change-Id: Idd820f8e3af70ebcbf82037b56fd2ae9655afbc5
2022-02-14 00:48:16 +01:00
Harald Welte c8c3327b6e ts_102_221: Proper parsing of FCP using pySim.tlv instead of pytlv
pytlv is a nightmare of shortcomings, let's abandon it in favor of
our own meanwhile-created pySim.tlv.  This has the added benefit
that unknown tags finally no longer raise exceptions.

Change-Id: Ic8e0e0ddf915949670d620630d4ceb02a9116471
Closes: OS#5414
2022-02-14 00:48:11 +01:00
Harald Welte e4a6eafc6f tlv: Don't raise exception if somebody passes empty data to TLV decoder
Change-Id: Id46994029d9b3cd6b67f4f7ee619466602cc8142
2022-02-14 00:44:55 +01:00
Harald Welte c975251a48 filesystem: Don't pass empty string to parse_select_response()
This happens e.g. when selecting the ARA-M applet on sysmoISIM-SJA2:

pySIM-shell (MF)> select ADF.ARA-M
-> 00a4040409 a00000015141434c00
<- 9000:
Traceback (most recent call last):
  File "/space/home/laforge/.local/lib/python3.9/site-packages/cmd2/cmd2.py", line 2064, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/space/home/laforge/.local/lib/python3.9/site-packages/cmd2/cmd2.py", line 2494, in onecmd
    stop = func(statement)
  File "/space/home/laforge/projects/git/pysim/./pySim-shell.py", line 750, in do_select
    fcp_dec = self._cmd.rs.select(path, self._cmd)
  File "/space/home/laforge/projects/git/pysim/pySim/filesystem.py", line 1314, in select
    select_resp = f.decode_select_response(data)
  File "/space/home/laforge/projects/git/pysim/pySim/filesystem.py", line 193, in decode_select_response
    return self.parent.decode_select_response(data_hex)
  File "/space/home/laforge/projects/git/pysim/pySim/filesystem.py", line 378, in decode_select_response
    return profile.decode_select_response(data_hex)
  File "/space/home/laforge/projects/git/pysim/pySim/ts_102_221.py", line 796, in decode_select_response
    t.from_tlv(h2b(resp_hex))
  File "/space/home/laforge/projects/git/pysim/pySim/tlv.py", line 231, in from_tlv
    (rawtag, remainder) = self.__class__._parse_tag_raw(do)
  File "/space/home/laforge/projects/git/pysim/pySim/tlv.py", line 258, in _parse_tag_raw
    return bertlv_parse_tag_raw(do)
  File "/space/home/laforge/projects/git/pysim/pySim/utils.py", line 208, in bertlv_parse_tag_raw
    if binary[0] == 0xff and len(binary) == 1 or binary[0] == 0xff and binary[1] == 0xff:
IndexError: bytearray index out of range
EXCEPTION of type 'IndexError' occurred with message: 'bytearray index out of range'

Change-Id: I910e6deba27d1483dff1e986c89f1a1b2165f49b
2022-02-14 00:44:55 +01:00
Harald Welte 81f4b4058b Extend unit test coverage for construct, add [some] tests for TLV
Change-Id: I3470e0b2e978221aa0c1e46a4b65f71f71abef2e
2022-02-14 00:41:24 +01:00
Harald Welte d0519e0c37 construct: Add Construct for variable-length int 'GreedyInteger'
We have a number of integers with variable-length encoding, so
add a Construct for this.  Naming inspired by GreedyBytes.

Related to https://github.com/construct/construct/issues/962

Change-Id: Ic6049b74ea3705fda24855f34b4a1d5f2c9327f7
2022-02-14 00:41:24 +01:00
Harald Welte e8d177d88f tlv: Convert CamelCase class name to snake_case in json
Our hand-written JSON so far is using snake_case identifiers,
while the JSON generated by the pySim.tlv classes use the class
names as keys, which LooksQuiteDifferent.

So let's auto-convert the CamelCase into something that reflects
our existing notion.

Change-Id: Id55929ef03dc48cb668e6ba7e99b6b291680a42f
2022-02-12 08:33:37 +00:00
Harald Welte 9a2a6691b0 tlv: Function for flattening the list-of-dict output of TLV decoder
Before:
{
    "FcpTemplate": [
        {
            "FileDescriptor": {
                "shareable": true,
                "file_type": "df",
                "structure": "no_info_given"
            }
        },
        {
            "FileIdentifier": "3f00"
        },
        {
            "ProprietaryInformation": [
                {
                    "UiccCharacteristics": "71"
                },
                {
                    "AvailableMemory": 123052
                }
            ]
        },
        {
            "LifeCycleStatusInteger": "operational_activated"
        },
        {
            "SecurityAttribReferenced": {
                "ef_arr_file_id": "2f06",
                "ef_arr_record_nr": 2
            }
        },
        {
            "PinStatusTemplate_DO": [
                {
                    "PS_DO": "40"
                },
                {
                    "KeyReference": 1
                },
                {
                    "KeyReference": 129
                }
            ]
        },
        {
            "TotalFileSize": 187809
        }
    ]
}

After:
{
    "FcpTemplate": {
        "FileDescriptor": {
            "shareable": true,
            "file_type": "df",
            "structure": "no_info_given"
        },
        "FileIdentifier": "3f00",
        "ProprietaryInformation": {
            "UiccCharacteristics": "71",
            "AvailableMemory": 123052
        },
        "LifeCycleStatusInteger": "operational_activated",
        "SecurityAttribReferenced": {
            "ef_arr_file_id": "2f06",
            "ef_arr_record_nr": 2
        },
        "PinStatusTemplate_DO": {
            "PS_DO": "40",
            "KeyReference": 129
        },
        "TotalFileSize": 187809
    }
}

Change-Id: Ia5ad8f1d3b0d47ebdb1856b0feaba120bad3eef9
2022-02-11 17:13:01 +01:00
Harald Welte 425038ffbc utils: Fix missing Optional[] in type annotations
Thanks to Vadim for pointing this out

Change-Id: I6e7d3725f28410d66580e88f2271d2b240d1f98e
2022-02-11 13:32:58 +01:00
Harald Welte c91085e744 cosmetic: Switch to consistent four-spaces indent; run autopep8
We had a mixture of tab and 4space based indenting, which is a bad
idea.  4space is the standard in python, so convert all our code to
that.  The result unfortuantely still shoed even more inconsistencies,
so I've decided to run autopep8 on the entire code base.

Change-Id: I4a4b1b444a2f43fab05fc5d2c8a7dd6ddecb5f07
2022-02-11 13:32:58 +01:00
Harald Welte 181c7c5930 ts_102_221: Implement proper parsing of EF.DIR
EF.DIR can not only contain the AID + Label of TS 102 221, but can
also contain any of the DOs specified in ISO7816-4.  Let's imoplement
this based on the modern pySim.tlv parser

Change-Id: I875eb49e1f0370428c2eae69af84f5483bd5b1fc
Closes: OS#5410
2022-02-11 13:01:55 +01:00
Harald Welte ca60ac253e filesystem.py: Accept both a class or an instance as TLV._tlv member
As we've seen in recent patches, this has been a source of bugs, so
let's be tolerant and deal with both.

Change-Id: I0a5ec2a860104ffe4524c647105a42505ac394d6
2022-02-10 18:15:25 +01:00
Harald Welte 6551627cb8 ts_31_102: TLV._tlv must point to the class, not an instance
In Change-Id I6d7c1bf49a8eaf3d8e50fb12888bf3d5b46b6c55 we fixed the
filesystem code to assume the self._tlv memper is a reference to a
class, and not an instance (as this is what the majority of the code
did).

However, it seems thre wer two instances where we actually had _tlv
reference an instance.  Change that to class so it's the same all over
the code base.

Change-Id: Ie4878ad6a92feafe47e375c4f5f3f198921e1e95
2022-02-10 17:53:11 +01:00