Merge the Mbed Crypto development branch a little after
mbedcrypto-1.0.0 into the PSA Crypto API 1.0 beta branch a little
after beta 2.
Summary of merge conflicts:
* Some features (psa_copy_key, public key format without
SubjectPublicKeyInfo wrapping) went into both sides, but with a few
improvements on the implementation side. For those, take the
implementation side.
* The key derivation API changed considerably on the API side. This
merge commit generally goes with the updated API except in the tests
where it keeps some aspects of the implementation.
Due to the divergence between the two branches on key derivation and
key agreement, test_suite_psa_crypto does not compile. This will be
resolved in subsequent commits.
Extend hash bad order test in line with the new bad order tests for MAC
and cipher, covering more cases and making comments and test layout
consistent.
Ensure that when doing hash operations out of order, PSA_ERROR_BAD_STATE
is returned as documented in crypto.h and the PSA Crypto specification.
In multipart cipher tests, test that each step of psa_cipher_update
produces output of the expected length. The length is hard-coded in
the test data since it depends on the mode.
The length of the output of psa_cipher_finish is effectively tested
because it's the total output length minus the length produced by the
update steps.
Test data obtained with Python+PyCrypto:
AES.new(key, mode=AES.MODE_CTR, counter=Crypto.Util.Counter.new(128, initial_value=0x2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a)).encrypt(plaintext.decode('hex')).encode('hex')
For must-fail asymmetric decryption tests, add an output size parameter
so that tests can directly control what output buffer size they allocate
and use independently from the key size used. This enables better
testing of behavior with various output buffer sizes.
Test a few cases. The logic to combine the constraint is similar to
the logic to combine the source and target, so it's ok to have less
parameter domain coverage for constraints.
Split the testing into tests that exercise policies in
test_suite_psa_crypto and tests that exercise slot content (slot
states, key material) in test_suite_psa_crypto_slot_management.
Test various cases of source and target policies with and without
wildcards. Missing: testing of the policy constraint on psa_copy_key
itself.
Test several key types (raw data, AES, RSA). Test with the
source or target being persistent.
Add failure tests (incompatible policies, source slot empty, target
slot occupied).
Remove front matter from our EC key format, to make it just the contents
of an ECPoint as defined by SEC1 section 2.3.3.
As a consequence of the simplification, remove the restriction on not
being able to use an ECDH key with ECDSA. There is no longer any OID
specified when importing a key, so we can't reject importing of an ECDH
key for the purpose of ECDSA based on the OID.
Remove pkcs-1 and rsaEncryption front matter from RSA public keys. Move
code that was shared between RSA and other key types (like EC keys) to
be used only with non-RSA keys.
Change the key derivation API to take inputs in multiple steps,
instead of a single one-site-fits-poorly function.
Conflicts:
* include/psa/crypto.h: merge independent changes in the documentation
of psa_key_agreement (public_key from the work on public key formats
vs general description and other parameters in the work on key derivation).
* tests/suites/test_suite_psa_crypto.data: update the key agreement
tests from the work on key derivation to the format from the work on
public key formats.
* tests/suites/test_suite_psa_crypto_metadata.function: reconcile the
addition of unrelated ALG_IS_xxx macros
Get rid of "key selection" algorithms (of which there was only one:
raw key selection).
Encode key agreement by combining a raw key agreement with a KDF,
rather than passing the KDF as an argument of a key agreement macro.
Remove front matter from our EC key format, to make it just the contents
of an ECPoint as defined by SEC1 section 2.3.3.
As a consequence of the simplification, remove the restriction on not
being able to use an ECDH key with ECDSA. There is no longer any OID
specified when importing a key, so we can't reject importing of an ECDH
key for the purpose of ECDSA based on the OID.
You can use PSA_ALG_ANY_HASH to build the algorithm value for a
hash-and-sign algorithm in a policy. Then the policy allows usage with
this hash-and-sign family with any hash.
Test that PSA_ALG_ANY_HASH-based policies allow a specific hash, but
not a different hash-and-sign family. Test that PSA_ALG_ANY_HASH is
not valid for operations, only in policies.
Remove pkcs-1 and rsaEncryption front matter from RSA public keys. Move
code that was shared between RSA and other key types (like EC keys) to
be used only with non-RSA keys.
Add new initializers for cipher operation objects and use them in our
tests and library code. Prefer using the macro initializers due to their
straightforwardness.
Add new initializers for MAC operation objects and use them in our tests
and library code. Prefer using the macro initializers due to their
straightforwardness.
Add new initializers for hash operation objects and use them in our
tests and library code. Prefer using the macro initializers due to their
straightforwardness.
Add new initializers for key policies and use them in our docs, example
programs, tests, and library code. Prefer using the macro initializers
due to their straightforwardness.
This commit finishes the removal of support for direct access to key
slots in psa_crypto.c.
This marks the end of the necessary phase of the transition to key
handles. The code should subsequently be refactored to move key slot
management from psa_crypto.c to psa_crypto_slot_management.c.
Switch from the direct use of slot numbers to handles allocated by
psa_allocate_key.
This commit does not affect persistent key tests except for the one
test function in test_suite_psa_crypto that uses persistent keys
(persistent_key_load_key_from_storage).
The general principle for each function is:
* Change `psa_key_slot_t slot` to `psa_key_handle_t handle`.
* Call psa_allocate_key() before setting the policy of the slot,
or before creating key material in functions that don't set a policy.
* Some PSA_ERROR_EMPTY_SLOT errors become PSA_ERROR_INVALID_HANDLE
because there is now a distinction between not having a valid
handle, and having a valid handle to a slot that doesn't contain key
material.
* In tests that use symmetric keys, calculate the max_bits parameters
of psa_allocate_key() from the key data size. In tests where the key
may be asymmetric, call an auxiliary macro KEY_BITS_FROM_DATA which
returns an overapproximation. There's no good way to find a good
value for max_bits with the API, I think the API should be tweaked.
Implement psa_allocate_key, psa_open_key, psa_create_key,
psa_close_key.
Add support for keys designated to handles to psa_get_key_slot, and
thereby to the whole API.
Allocated and non-allocated keys can coexist. This is a temporary
stage in order to transition from the use of direct slot numbers to
allocated handles only. Once all the tests and sample programs have
been migrated to use handles, the implementation will be simplified
and made more robust with support for handles only.
Add missing compilation guards that broke the build if either GCM or
CCM was not defined.
Add missing guards on test cases that require GCM or CBC.
The build and tests now pass for any subset of {MBEDTLS_CCM_C,
MBEDTLS_GCM_C}. There are still unused variables warnings if neither
is defined.
Allow use of persistent keys, including configuring them, importing and
exporting them, and destroying them.
When getting a slot using psa_get_key_slot, there are 3 scenarios that
can occur if the keys lifetime is persistent:
1. Key type is PSA_KEY_TYPE_NONE, no persistent storage entry:
- The key slot is treated as a standard empty key slot
2. Key type is PSA_KEY_TYPE_NONE, persistent storage entry exists:
- Attempt to load the key from persistent storage
3. Key type is not PSA_KEY_TYPE_NONE:
- As checking persistent storage on every use of the key could
be expensive, the persistent key is assumed to be saved in
persistent storage, the in-memory key is continued to be used.
There was no test case of ECDH with anything other than
PSA_ALG_SELECT_RAW. Exercise the code path from ECDH through a
"proper" KDF.
ECDH shared secret copied from an existing test, HKDF output
calculated with Cryptodome.
In ECDH key agreement, allow a public key with the OID id-ECDH, not
just a public key with the OID id-ecPublicKey.
Public keys with the OID id-ECDH are not permitted by psa_import_key,
at least for now. There would be no way to use the key for a key
agreement operation anyway in the current API.
Add test cases that do key agreement with raw selection in pieces, to
validate that selection works even when the application doesn't read
everything in one chunk.
A key selection algorithm is similar to a key derivation algorithm in
that it takes a secret input and produces a secret output stream.
However, unlike key derivation algorithms, there is no expectation
that the input cannot be reconstructed from the output. Key selection
algorithms are exclusively meant to be used on the output of a key
agreement algorithm to select chunks of the shared secret.
On key import and key generation, for RSA, reject key sizes that are
not a multiple of 8. Such keys are not well-supported in Mbed TLS and
are hardly ever used in practice.
The previous commit removed support for non-byte-aligned keys at the
PSA level. This commit actively rejects such keys and adds
corresponding tests (test keys generated with "openssl genrsa").
Simplify the test case "PSA export a slot after a failed import of an
EC keypair": use an invalid private value for the specified curve. Now
the dependencies match the test data, so this fixes curves.pl.
Update some test data from the asymmetric_apis_coverage branch that
wasn't updated to the new format from the
psa-asymmetric-format-raw_private_key branch.
1. New test for testing bad order of hash function calls.
2. Removed test hash_update_bad_paths since it's test scenario
was moved to the new test.
3. Moved some scenarios from test hash_verify_bad_paths to
the new test.
1. Rename hash_bad_paths to hash_verify_bad_paths
2. Add test hash_update_bad_paths
3. Add test hash_finish_bad_paths
The different scenarios tested as part of hash_bad_paths are
moved to the relevant test.
streamline the API for the test test_derive_invalid_generator_state: by removing
the key_data parameter.
This parameter is not important for test flow and can be hard-coded.
Add boundary test cases for private key validity for a short
Weierstrass curve (0 < d < n).
Remove obsolete test cases "valid key but wrong curve". With the new
format, the private key representation does not contain an encoding of
the curve.
In preparation for the import/export format change for private
elliptic curve keys from RFC 5915 to the raw secret value, transform the
test data to the new format.
Tests will not pass until the implementation has been changed to the
new format and some test cases and test functions have been adjusted.
I used the script below to look for lines containing a
PSA_KEY_TYPE_ECC_KEYPAIR and change the first hex string in the
line with an ASN.1 header that looks like the beginning of an RFC 5915
ECPrivateKey. This always happens to be a private key input.
perl -a -F: -i -pe 'sub pad { local ($_) = @_; s/^00// if length == $digits + 2; die if length > $digits; sprintf("\"%0${digits}s\"", $_) } if ($F[0] !~ /\W/ && /:PSA_KEY_TYPE_ECC_KEYPAIR\( *PSA_ECC_CURVE_[A-Z_]+([0-9]+)/) {$digits = int(($1+7)/8)*2; s/"30(?:[0-7].|81..|82....)02010104(..)([0-9a-f]+)"/pad(substr($2, 0, hex($1)*2))/ie}' tests/suites/test_suite_psa_crypto.data
In the test function for export_public_key, don't just check the
length of the result. Compare the actual result to the expected
result.
Take an extra argument that allows using an export buffer that's
larger or smaller than needed. Zero is the size given by
PSA_KEY_EXPORT_MAX_SIZE.
Don't check the output of psa_get_key_information. That's useful in
import_export because it tests both import and export, but not in
import_export_public_key whose goal is only to test public key export.
This commit adjusts the existing test data but does not add new test
cases.
Key derivation test now uses an indirect way to test generator validity
as the direct way previously used isn't compatible with the PSA IPC
implementation. Additional bad path test for the generator added
to check basic bad-path scenarios.
Add comments noting that the maximum length of a MAC must fit in
PSA_ALG_MAC_TRUNCATION_MASK. Add a unit test that verifies that the
maximum MAC size fits.
Pass the nonce first, then the AD, then the input. This is the order
in which the data is processed and it's the order of the parameters to
the API functions.
OFB and CFB are streaming modes. XTS is a not a cipher mode but it
doesn't use a separate padding step. This leaves only CBC as a block
cipher mode that needs a padding step.
Since CBC is the only mode that uses a separate padding step, and is
likely to remain the only mode in the future, encode the padding mode
directly in the algorithm constant, rather than building up an
algorithm value from a chaining mode and a padding mode. This greatly
simplifies the interface as well as some parts of the implementation.
In psa_generator_import_key, if generating a DES or 3DES key, set the
parity bits.
Add tests for deriving a DES key. Also test deriving an AES key while
I'm at it.
In psa_generator_hkdf_read, return BAD_STATE if we're trying to
construct more output than the algorithm allows. This can't happen
through the API due to the capacity limit, but it could potentially
happen in an internal call.
Also add a test case that verifies that we can set up HKDF with its
maximum capacity and read up to the maximum capacity.
New key type PSA_KEY_TYPE_DERIVE. New usage flag PSA_KEY_USAGE_DERIVE.
New function psa_key_derivation.
No key derivation algorithm is implemented yet. The code may not
compile with -Wunused.
Write some unit test code for psa_key_derivation. Most of it cannot be
used yet due to the lack of a key derivation algorithm.
Add a label argument to all asymmetric encryption test functions
(currently empty in all tests, but that will change soon).
In asymmetric_encrypt and asymmetric_decrypt, with an empty label,
test with both a null pointer and a non-null pointer.
Although RSASSA-PSS defines its input as a message to be hashed, we
implement a sign-the-hash function. This function can take an input
which isn't a hash, so don't restrict the size of the input, any more
than Mbed TLS does.
Remove a redundant check that hash_length fits in unsigned int for the
sake of Mbed TLS RSA functions.
Test that PSS accepts inputs of various lengths. For PKCS#1 v1.5
signature in raw mode, test the maximum input length.
This required tweaking exercise_signature_key to use a payload size
for the signature based on the algorithm, since our implementation of
PSS requires that the input size matches the hash size. This would
also be the case for PKCS#1 v1.5 with a specified hash.
* No test depends on MBEDTLS_PK_C except via MBEDTLS_PK_PARSE_C, so
remove MBEDTLS_PK_C and keep only MBEDTLS_PK_PARSE_C.
* Add MBEDTLS_PK_WRITE_C for pk export tests.
* Add MBEDTLS_GENPRIME for RSA key generation tests.
* Add dependencies to AEAD tests.
* Add missing dependencies to many RSA tests.
* Add a test for decryption with invalid padding.
* Add a test for encryption with input too large.
* In negative tests, pass input whose length matches the key length,
unless that's what the test is about.
Change most asymmetric_verify to use public keys (they were all using
key pairs before). Keep one test with an RSA key pair and one with an
EC key pair.
Revise the test function asymmetric_encrypt_fail into
asymmetric_encrypt and use it for positive tests as well. Get the
expected output length from PSA_ASYMMETRIC_ENCRYPT_OUTPUT_SIZE. Check
the actual output length against test data.
Add positive test cases for encryption: one with an RSA public
key (this is the only test for encryption with a public key rather
than a key pair) and one with a key pair.
Add tests of key policy checks for MAC, cipher, AEAD, asymmetric
encryption and asymmetric signature. For each category, test
with/without the requisite usage flag in each direction, and test
algorithm mismatch.
Change the representation of an ECDSA signature from the ASN.1 DER
encoding used in TLS and X.509, to the concatenation of r and s
in big-endian order with a fixed size. A fixed size helps memory and
buffer management and this representation is generally easier to use
for anything that doesn't require the ASN.1 representation. This is
the same representation as PKCS#11 (Cryptoki) except that PKCS#11
allows r and s to be truncated (both to the same length), which
complicates the implementation and negates the advantage of a
fixed-size representation.
* Distinguish randomized ECDSA from deterministic ECDSA.
* Deterministic ECDSA needs to be parametrized by a hash.
* Randomized ECDSA only uses the hash for the initial hash step,
but add ECDSA(hash) algorithms anyway so that all the signature
algorithms encode the initial hashing step.
* Add brief documentation for the ECDSA signature mechanisms.
* Also define DSA signature mechanisms while I'm at it. There were
already key types for DSA.
* PSS needs to be parametrized by a hash.
* Don't use `_MGF1` in the names of macros for OAEP and PSS. No one
ever uses anything else.
* Add brief documentation for the RSA signature mechanisms.
Add a negative test for import where the expected key is an EC key
with the correct key size, but the wrong curve. Change the test that
tries to import an RSA key when an EC key is expected to have the
expected key size.
Because exporting-public a symmetric key fails, we have no reasonable
expectation that the exported key length has any value at all other than
something obviously incorrect or "empty", like a key with a length of 0.
Our current implementation explicitly sets the exported key length to 0
on errors, so test for this. Fix the "PSA import/export-public: cannot
export-public a symmetric key" test to expect a key length of 0 instead
of 162.
In the test generate_random, focus on testing that psa_generate_random
is writing all the bytes of the output buffer and no more. Add a check
that it is writing to each byte of the output buffer. Do not try to
look for repeating output as the structure of a unit test isn't likely
to catch that sort of problem anyway.
Also add what was missing in the test suite to support block ciphers
with a block size that isn't 16.
Fix some buggy test data that passed only due to problems with DES
support in the product.
In psa_hash_start, psa_mac_start and psa_cipher_setup, return
PSA_ERROR_INVALID_ARGUMENT rather than PSA_ERROR_NOT_SUPPORTED when
the algorithm parameter is not the right category.
When psa_mac_start(), psa_encrypt_setup() or psa_cipher_setup()
failed, depending on when the failure happened, it was possible that
psa_mac_abort() or psa_cipher_abort() would crash because it would try
to call a free() function uninitialized data in the operation
structure. Refactor the functions so that they initialize the
operation structure before doing anything else.
Add non-regression tests and a few more positive and negative unit
tests for psa_mac_start() and psa_cipher_setup() (the latter via
psa_encrypt_setip()).
Exporting an asymmetric key only worked if the target buffer had
exactly the right size, because psa_export_key uses
mbedtls_pk_write_key_der or mbedtls_pk_write_pubkey_der and these
functions write to the end of the buffer, which psa_export_key did not
correct for. Fix this by moving the data to the beginning of the
buffer if necessary.
Add non-regression tests.
* init-deinit
* import-export
* policies
* lifetime
* hash
* MAC
* cipher
* AEAD
* asymmetric sign
* asymmetric verify
* asymmetric encrypt-decrypt
This commit only moves test functions and test cases around. It does
not modify, add or remove tests.
psa_import_key must check that the imported key data matches the
expected key type. Implement the missing check for EC keys that the
curve is the expected one.
Instead of rolling our own list of elliptic curve identifiers, use one
from somewhere. Pick TLS because it's the right size (16 bits) and
it's as good as any.
Always adding things at the end tends to create merge conflicts.
Adding in the middle in this way makes the order more logical in
addition to avoiding conflicts.
* PKCS#7 padding always adds at least one byte of padding, so test
data with plaintext length = ciphertext length could not have been
correct.
* CTR has plaintext length = ciphertext length, so test data with
differing lengths could not have been correct.
Only whitespace changes.
* Remove tabs.
* Remove trailing whitespace.
* Correct some misindented lines.
* Normalize whitespace around some punctuation.
* Split some lines to avoid going over 80 columns.
