timothyclaeys / pycose Goto Github PK

Hi,
@TimothyClaeys
First of all, great effort putting this library together 👍

In the docs I see that ECDH-ES with direct CEK derivation is supported:
ECDH-ES + HKDF-256 | ✔️ | ECDH ES w/ HKDF - generate key directly

Is there an example or test case that I could find sample of generating Encrypt message in ECDH-ES direct CEK derivation scheme

The base class CoseKey parameter key_ops is currently typed in the API as Optional[KeyOps] when it should be something more like Optional[Set[KeyOps]] because the actual COSE_Key definition from RFC8152 defines it as CDDL type [+ (tstr/int)] and there's no meaning to duplicate values in that list.

I'm using this library for my thesis and I have to compare AES GCM with AES CCM, but I don't know how to choose one algorithm instead of the other.

I'm working with nonce = 12 byte because of GCM recommendation and I saw in hazmat/cryptography that they only provide one AES CCM algorithm where nonce has to be between 7-13 byte. Why did you create more than one?

pycose doesn't allow to modify the unprotected header of an already signed message. Being able to do so is useful for countersigning, since the countersignature is computed based on the original signature and then embedded as new parameter in the unprotected header. Decoding an existing message followed by msg.encode(sign=False) strips the original signature instead of retaining it.

COSE requires that alg must be authenticated (external_aad or protected header) and crit must be in the protected header. pycose reads both parameters from the unprotected header if missing in the protected header.

The get_attr method should be extended to check whether a parameter is required to be in the protected bucket. This requires another field in the attribute class.

There is a function to randomly generate COSE keys, but no way to programmatically split out the private key part.

As a workaround, I'm generating keys and then remove the d (which at least works the same way in OKP and EC2 keys), but for the general case (RSA having been recently added, albeit irrelevant to my EDHOC use case) the key type would need to know how to strip the private parts out of this.

The function could have a signature def public_part(self: CoseKey) -> CoseKey, and could raise ValueError if key does not contain any public information (seems one can construct an OKP from a d alone, not sure if the x can be derived then).

Relatedly, it may also make sense to remove any unneeded public parts from a key -- but I don't know if that has any practical applications (in local storage it's probably fine to keep the public parts around, Python systems are typically not constrained).

In #88 a packaging bug was found that was not discovered by CI because tests currently don't run on the installed package but directly in the source tree.

I think running tests locally inside the source tree is fine, but CI should go the extra mile and test against the installed package.

Here's one possible way to do it:

pip install .
mkdir tmp
cd tmp
pytest ../tests

This won't work currently as the tests have an __init__.py and import some helper functions from tests.conftest (conftest.py file in the tests/ folder).

Let's discuss what the best option is in this issue.

EDIT: This is addressed in #90.

Currently the EC2Key and OKPKey classes use the cryptography library internally to do work but don't expose API to use these intermediate representations. When interoperating between cose and other libraries it would be convenient to have a standard API to get these representations.

Without this API an application needs to do some special data manipulation which duplicates already-present internal state of functions within those classes.

For my own use and as an example the RSAKey class in my branch uses the following API (but the names are arbitrary and I have no special need for them):

    @classmethod
    def from_cryptograpy_key_obj(cls, ext_key) -> 'RSAKey':

and

    def to_cryptograpy_key_obj(self):

Hi,
Its not clear to me how to serialize a COSE Key using cbor2. I would expect something like this:

import cbor2
from cose import OKP, CoseAlgorithms, KeyOps

k=OKP.generate_key(algorithm=CoseAlgorithms.EDDSA, key_ops=KeyOps.VERIFY)
cbor2.dumps(k.encode())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_cbor2.CBOREncodeTypeError: cannot serialize type <aenum 'KTY'>

Im pretty sure I can "hand"-code this structure manually like this (although I would certainly prefer not to):

import cbor2
cbor2.dumps([{-1:6, -2:<x coordinate as bytestring>, 1: 1, 2: <kid>}]

Am I missing something extremely obvious here? The pycose docs also seem to indicate you can encode using cbor directly:

From https://pycose.readthedocs.io/en/latest/cose/keys/okp.html

(...)
encode(*argv)
    Encode the provided key words in dictionary. The dictionary can then be encoded with CBOR.
(...)

When installing cose with pip3 install cose on RBPi - with Python v3.10.0 I get the "Could not find a version that staisfies the requirement cose (from versions: ) No Matching distribution found for cose"

Is Python 3.10 or RBPi not supported?

I will try to explain my problem:

Setup.py is based on dynamically resolved requirements.txt file. It sounds very comfortable for developer, but hurts user when switching to newer version of cose. Now when you create dist ( that will be later on pushed on pypi) it takes most recent (in moment of creating dist) packages from requirements.txt file, so as a result user when user update cose, he have to have i.e most recent cryptography in version 3.4.6, when in reality it still works fine with older versions of cryptography (i.e 3.1.1). It is real pain when production environment has older versions of packages and each of them have to be updated everytime cose is updated.

I would suggest to specify setup install_requires (as in biggier open source projects) in more controlled and libarate way for user.

For example take a look at:

https://github.com/Pylons/pyramid/blob/master/setup.py

so i.e

cryptography >= 3.1.1

would allow user to have 3.4.6 as well as older versions.

Regards,
Mateusz Zaborski

Once #94 is implemented, get_attr(name) will validate any location requirements for standard header parameters. However, some clients may use unknown header parameters or want to enforce some standard header parameters to be in the protected header when not required by the spec.

To support this, get_attr should get another argument to enforce location.

When installing the current develop branch of cose there is now an upstream library version dependency issue. When attempting to run an application which imports cose I now see fatal exceptions:

pkg_resources.ContextualVersionConflict: (docutils 0.17 (/usr/local/lib/python3.8/site-packages), Requirement.parse('docutils<0.17'), {'sphinx-rtd-theme'})
pkg_resources.DistributionNotFound: The 'docutils<0.17' distribution was not found and is required by sphinx-rtd-theme

I'm installing cose with pip, which seems to not properly handle the two (min-version, max-version) constraints on the docutils package.

As a workaround I can pre-install sphinx-rtd-theme==0.5.0 or docutils<0.17 but it appears that the dependency from cose to the Sphinx tools is not really runtime, but doc building. Can the sphinx dependencies simply be moved out of the runtime requirements.txt?

I see that the reverse of this was done in 1103acb, any reason for that?

I opened this ticket to track changes to the cose.extensions.x509 module for the new cose API (from #41). If this is already being worked I will wait on it. If not, I can propose changes to get the module import-able and working again.

The current first failure is using the old API:

ImportError: cannot import name 'CoseAlgorithms' from 'cose.algorithms'

I am getting this error using cose-sign1 with the ES256 algorithm.

This is an error with the single signature

I am confident that the signature is correct as it works on other programs.

Is there any way around this error?

Hi,

I think that in:

the returned object does not have a curve type set:

I'm also wondering what is the need for algorithm specification here:

Does the algorithm that the key will be used with need to be set at the key creation time?

attrs is not used in pycose as far as I see.

When running on Ubuntu 20.04, which requires an explicit backend argument for many cryptographic operations there are a few places in cose which do not supply the required argument and fail with an exception similar to

  File ".../cose/keys/ec2.py", line 100, in __init__
    public_nums = ec.derive_private_key(int.from_bytes(d, byteorder="big"),
TypeError: derive_private_key() missing 1 required positional argument: 'backend'

I'm going to patch up a branch which allows my uses to work, but it's not an exhaustive search for missing backend arguments.

The new API for encoding with multiple recipients is much improved from the old API.

Currently when using an algorithm which itself requires an ephemeral sender key (e.g., EcdhEsA256KW) the recipient encoding disallows an external (application-defined) ephemeral key with the error:

cose.exceptions.CoseException: Unrelated ephemeral public key found in COSE message header

This seems overly-strict and disallows deterministic testing. Could this guard be relaxed so that an application could generate its own ephemeral key? The guard can still check that it's the correct key type, curve, etc.

Also, the CoseRecipient._setup_ephemeral_key() function generates a COSE key with an default-empty key_ops member. The cose library doesn't seem to care that the key_ops is empty but it seems like it should either be absent or initialized to include DeriveKeyOp.

In situations where signing happens offline, for example through a device, it is useful to be able to compose a Sign1Message, and to set an externally-provided signature on it before encoding. If the signature property (https://github.com/TimothyClaeys/pycose/blob/master/pycose/messages/sign1message.py#L42) was made settable, this would become easily possible.

I have tested that this works today by setting _signature directly, but this is obviously not supported usage.

It seems possible to set the signature by passing in an already signed COSE message through from_cose_obj, but this isn't really a solution here, since the composition of that message is the goal itself.

Hello, I have a EC2 Key:

<COSE_Key(EC2Key): {'EC2KpY': "b'\\xcc\\x01\\x99R\\xcc' ... (32 B)", 'EC2KpX': "b'_F\\xbf\\xa8\\xde' ... (32 B)", 'EC2KpCurve': 'P256', 'KpKty': 'KtyEC2', 'KpAlg': 'Es256'}>

and when I try to encode:

msg = Sign1Message(
   		 	phdr = {Algorithm: 'Es256', KID: b'EC2'},
    		payload = nonce_bytes
    		)
key_as_dict =  CoseKey.from_dict(key)
msg.key = key_as_dict
encoded = msg.encode()

I get:

Traceback (most recent call last):
  File "/Users/Chris/Developer/PassKeysTest/API/cert.py", line 153, in <module>
    key = extractKey(
  File "/Users/Chris/Developer/PassKeysTest/API/cert.py", line 136, in extractKey
    encoded = msg.encode()
  File "/opt/homebrew/lib/python3.9/site-packages/cose/messages/sign1message.py", line 67, in encode
    message = [self.phdr_encoded, self.uhdr_encoded, self.payload, self.compute_signature()]
  File "/opt/homebrew/lib/python3.9/site-packages/cose/messages/signcommon.py", line 65, in compute_signature
    return alg.sign(key=self.key, data=self._sig_structure)
  File "/opt/homebrew/lib/python3.9/site-packages/cose/algorithms.py", line 185, in sign
    sk = SigningKey.from_secret_exponent(int(hexlify(key.d), 16), curve=cls.get_curve())
ValueError: invalid literal for int() with base 16: b''

The method it's calling is:

 @classmethod
    def sign(cls, key: 'EC2', data: bytes) -> bytes:
        sk = SigningKey.from_secret_exponent(int(hexlify(key.d), 16), curve=cls.get_curve())

        return sk.sign_deterministic(data, hashfunc=cls.get_hash_func())

....but key.d doesn't exist in this key.....

The current setup.cfg configures the package search within the "pycose" directory, but that is the actual package root so the "pycose" package itself is not installed when I run pip install . from the checkout root.

I think that line should be changed to where = . or similar to capture the whole "pycose" package.

The text on https://pypi.org/project/cose/ is confusing as it says that pycose is old code. Since this has been fixed now and pycose is the way forward, there should be a last release for cose to update the README on PyPI to say that this package won't receive further updates and people should switch to pycose.

I think this means:

• Create a branch off of v0.9.dev8, maybe named release/0.9.x
• Create a PR branch targeting release/0.9.x
• Make the necessary README changes in that branch
• Edit the version in setup.py to 0.9.dev9
• Create a PR for review & merge it into release/0.9.x
• Create a GitHub release with git tag v0.9.dev9 off of release/0.9.x
• Push to PyPI

I've observed that EC2.generate_key() may result in a key that is incorrecty encoded.

Notice that in the following lines the x and y byte arrays length depends on the x and y values:

If we assume that x and y are randomly distributed over the entire curve, there is a pretty high (a around 1/256 per coordinate I think) chance that a coordinate integer representation will start with at least 8 leading zero bits. In such case, the coordinate will be represented by a byte array shorter than the length of private key.

If key with leading zero bytes in either x or y is encoded using encode() method, the result does not follow the specs.
RFC8152 (13.1.1) states that EC2 points must be encoded with leading zero octets preserved.

Sample code:

TRIAL_COUNT = 4096
cases = 0
for i in range(TRIAL_COUNT):
    ec2_test = EC2.generate_key(CoseAlgorithms.ECDH_ES_HKDF_256, KeyOps.DERIVE_KEY, CoseEllipticCurves.P_256)
    ec2_encoded = ec2_test.encode('x', 'y')
    if len(ec2_encoded[EC2.EC2Prm.X]) != 32 or len(ec2_encoded[EC2.EC2Prm.Y]) != 32:
        cases += 1
        print('Found a bug:')
        print(ec2_encoded)
print('Bug rate: {}'.format(cases/TRIAL_COUNT))

In pycose/keys/symmetric.py, there are several checks on key length which exclude keys for HS384 and HS512 algorithms.

if key_len not in [16, 24, 32]:

Here's how I reproduce this issue

from pycose.keys import SymmetricKey, keyops
from pycose.algorithms import HMAC256, HMAC384, HMAC512
from binascii import hexlify
hashes = [
    ["HS256", HMAC256],
    ["HS384", HMAC384],
    ["HS512", HMAC512]
]

for [name, alg] in hashes:
    print(f"{alg} - {alg.get_digest_length()}")
    key = SymmetricKey.generate_key(alg.get_digest_length())
    key.kid = b"[email protected]"
    key.key_ops = [keyops.MacCreateOp, keyops.MacVerifyOp]
    key.alg = alg
    print(hexlify(key.encode()))

This may be corrected by updating said list to

if key_len not in [16, 24, 32, 48, 64]:

Other implementations do support this length, for example in rust: https://github.com/tramires/cose-rust/blob/main/src/algs.rs#L1006

Perhaps I'm missing something, but it seems verify_signature() expects the message key to be set to the SigningKey and there is no way to verify it with just the VerifyingKey?

This is the only place where the ecdsa library is used:

I'm wondering whether the following functions of the cryptography package would be sufficient to handle this:

• EllipticCurvePublicKey.from_encoded_point(curve, data)
• ec.derive_private_key(private_value, curve)
• EllipticCurvePrivateKey.sign(data, sig_alg)
• EllipticCurvePublicKey.verify(signature, data, sig_alg)
• encode_dss_signature(r, s)
• decode_dss_signature(sig)

If the above is not a perfect fit, what is missing?

I really appreciate all of the effort that's gone into this library. For my purposes I want to produce some examples using RSA keypairs, and I'm comfortable hammering in an RSA key class in a local fork then later making a merge request.

This ticket is really to note the missing behavior and make sure there's no parallel effort.

I've observed that _enc_structure is used as external AAD to authenticate the message. However _enc_structure seems to be calculated from different things including IV. But my message can be only authenticated if the "real" external AAD is used. Can I somehow overwrite _enc_structure?

I'm probably missing something but I don't see how to sign or verify with detached content, which RFC 8152 allows. (Reason: we are wondering whether to add COSE signing to GRASP, RFC 8990, and I'd like to prototype it.)

Related to openwsn-berkeley/py-edhoc#21.

The readme mentions Sign and Sign0 cases, while the COSE standard specifies the same differentiation (i.e. either many or 1 recipient), but calls it Sign and Sign1.

In fact the pycose docs and code seem to follow this standard as well: https://pycose.readthedocs.io/en/latest/examples.html
Only the readme is off.

I think this is roughly:

1. Ensure version is set to the next SemVer version in setup.cfg
2. Create a new GitHub release with git tag v<x>.<y>.<z>, e.g., v1.2.3
3. Wait for CI to run for the new git tag
4. Make sure that CI ran successfully and pushed the package to PyPI

The function _CoseAttribute.from_id() has an argument allow_unknown_attributes=False but it is not possible to use this from the outside.

...
  File ".../site-packages/cose/messages/cosebase.py", line 50, in __init__
    CoseBase._transform_headers(self._phdr, phdr)
  File ".../site-packages/cose/messages/cosebase.py", line 159, in _transform_headers
    hp = CoseHeaderAttribute.from_id(_header_attribute)
  File ".../site-packages/cose/utils.py", line 43, in from_id
    raise CoseException(f"Unknown COSE attribute with value: [{cls.__name__} - {attribute}]")

I'm attempting to decode a COSE message with an "x5t" parameter, and what happens when attempting to decode is (only showing the top of the stack):

  File "PATH/pycose/cose/cosebase.py", line 19, in from_cose_obj
    uhdr = cls._parse_header(cose_obj.pop(0))
  File "PATH/pycose/cose/cosebase.py", line 93, in _parse_header
    new_hdr[attr.headers.CoseHeaderKeys(k)] = parse_func(v)
TypeError: __init__() missing 1 required positional argument: 'certificate'

The registration in CoseHeaderKeys uses the class constructor, which doesn't actually decode the header encoded value. Also, the current X5T class does not actually store the thumbprint value so there's no way to decode an actual header value.

It may make better sense for the X5T to actually contain the thumbprint bytestring with some static functions to separately: decode a header value and compute a thumbprint (as the current constructor does).

With the different implementations of CoseKey encode function, there are some discrepancies with how the attribute names are handled (adding or stripping leading underscore) and so the API breaks on some attr names. I'm not sure if I was using it correctly but I have made a patch which at least makes the uses consistent for SymmetricKey.

Hi,

I've noticed that in case of empty protected header, the resulting COSE structure header is set to an empty bstr:

In examples from COSE-WG empty header is an empty (0-element) byte array serialized to a bstr:
https://github.com/cose-wg/Examples/blob/master/sign1-tests/sign-pass-01.json

D2                                      # tag(18)
   84                                   # array(4)
      41                                # bytes(1)
         A0                             # "\xA0"
      A2                                # map(2)
         01                             # unsigned(1)
         26                             # negative(6)
         04                             # unsigned(4)
         42                             # bytes(2)
            3131                        # "11"
      54                                # bytes(20)
         546869732069732074686520636F6E74656E742E # "This is the content."
      58 40                             # bytes(64)

I'm not sure if both encodings are valid. What do you think?

Any chance to get a new release out on PyPI? Much appreciated! :)

When constructing a key using EC2.generate_key the curve type is not a constructor argument.
The EC2.crv member must be set after the generation.

The crit parameter lists header parameters that should be considered critical. It's an array of labels, where a label is an int or a tstr. Currently, pycose raises an error if the label is not of type int.

You probably know why I'm here: I'm trying to verify a COSE_Sign1 message against a public key for COVID Passport verification. And I'm stuck because I don't know which methods I should use and how. So I'm asking for help here, considering I'm already able to get a COSE_Sign1 object:
<COSE_Sign1: [{'Algorithm': 'Es256', 'KID': b'|b\xee\xbe\x0e\xa7\xe7\t'}, {}, b'\xa4\x01dCN' ... (228 B), b'\x17\x04c\xe8\xd3' ... (64 B)]>
I'm very bad at Python and just trying pieces of code, the usual stuff, and I'm already struggling with trying to get the value for KID from this COSE_Sign1 object, then I'd like to call verify_signature() but I'm not sure how to import the public key (which I can retrieve, knowing the KID, the Algorithm, etc.).
I believe I'm close, though, but any pointer to put me on the right tracks would be much welcome.

Is it possible to add a cbor.tag at the beginning of the COSE signature? Many COSE signatures have these tags in front of them. it would only be a numerical value of "6" or "15" for example.

It would be good if the CoseKey.key_ops setter did some structural checking of values. I am migrating from old to new cose API and accidentaly left behind a non-list key_ops assignment which results in a strange behavior seen below.
It took some troubleshooting to figure out my problem, which could have been found easier if the setter raised an exception instead of doing what appears to be splitting a str value input.

k1 = EC2Key(
    kid=b'ExampleEC2',
    key_ops=[keyops.DeriveKeyOp],
)
print(k1)
k2 = EC2Key(
    kid=b'ExampleEC2',
    key_ops=keyops.DeriveKeyOp,
)
print(k2)

will result in the output:

<COSE_Key(EC2Key): {'KpKty': 'KtyEC2', 'KpKid': b'ExampleEC2', 'KpKeyOps': ['DeriveKeyOp']}>
<COSE_Key(EC2Key): {'KpKty': 'KtyEC2', 'KpKid': b'ExampleEC2', 'KpKeyOps': ['D', 'e', 'r', 'i', 'v', 'e', 'K', 'e', 'y', 'O', 'p']}>

It seems the complete package requires Sphinx, even though it is just used for generating the documentation. As Sphinx depends on a lot of other packages, this results in a lot of unneeded dependencies for normal use.

The KDF interface has an optional "salt" parameter but when it is used by EC2 and OKP classes there is no way to provide a salt alongside or as part of the CoseKDFContext parameter. This means that the algorithm parameter "salt" (-20) cannot be passed into the KDF.

Am I misunderstanding the interface, or is this parameter just missing?

The master branch should be protected with the following guards at minimum:

• At least one approving review
• Passes CI

For direct key-wrap algorithms like A128KW, the current implementation of KeyWrap._compute_kek() returns just the k value and assumes that the key_ops is WrapOp or UnwrapOp for encrypt() and decrypt() respectively.
This will allow a situation where a KEK is identified (e.g., by KID header) and found but has some other incompatible key_ops.

If the _compute_kek() returned the whole SymmetricKey, including its key_ops, then this check would be valid. As it is, the verify() following _compute_kek() is useless because the key_ops are forced instead of being read-in.

Hi,
I just used the message = CoseMessage.decode(some_bytes) to decrypt bytes, and I found the message.key is None. But once I try to encrypt it back with a key equals to None, I get a cose.exceptions.CoseException. Is it supposed to act like this or I did something wrong ?

cose.headers.KID: "foo" produces a tstr value which is not according to the spec. Either a string KID is automatically encoded into a UTF-8 buffer (making it bstr), or an error should be thrown.