Merge the latest state of the target branch (mbedtls/development) into the
pull request to merge mbed-crypto into mbedtls.
Conflicts:
* ChangeLog: add/add conflict. Resolve by using the usual section order.
Rename identifiers containing double-underscore (`__`) to avoid `__`.
The reason to avoid double-underscore is that all identifiers
containing double-underscore are reserved in C++. Rename all such
identifiers that appear in any public header, including ssl_internal.h
which is in principle private but in practice is installed with the
public headers.
This commit makes check-names.sh pass.
```
perl -i -pe 's/\bMBEDTLS_SSL__ECP_RESTARTABLE\b/MBEDTLS_SSL_ECP_RESTARTABLE_ENABLED/g; s/\bMBEDTLS_KEY_EXCHANGE_(_\w+)_(_\w+)\b/MBEDTLS_KEY_EXCHANGE${1}${2}/g' include/mbedtls/*.h library/*.c programs/*/*.c scripts/data_files/rename-1.3-2.0.txt tests/suites/*.function
```
Look for changes that remove X509/SSL functionality.
```
git diff 'HEAD^{/^Merge}~1' HEAD --diff-filter=M -- . ':!library/error.c' ':!library/version_features.c' ':!programs/test/query_config.c' ':!visualc' ':!*.pdf' ':!*.der' | grep -E "^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_"
```
All of these removals are in `config.h` or `check_config.h`. Selectively revert the differences in these two files.
```
git diff 'HEAD^{/^Merge}~1' 'HEAD^{/^Merge}' include/mbedtls/config.h include/mbedtls/check_config.h | git apply -p1 -R
```
* `include/mbedtls/check_config.h`:
* ARIA for GCM: don't remove it.
* `MBEDTLS_PSA_CRYPTO_SE_C`: don't remove it.
* `MBEDTLS_SHA512_NO_SHA384`: don't remove it.
* `MBEDTLS_SSL_DTLS_CONNECTION_ID`: restore it.
* `include/mbedtls/config.h`:
* warning about `MBEDTLS_ECDSA_SIGN_ALT`: don't remove it.
* `MBEDTLS_CTR_DRBG_USE_128_BIT_KEY` full paragraph: don't remove it.
* `MBEDTLS_PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER`: don't remove it.
* `MBEDTLS_SHA512_SMALLER`…: don't remove it.
* `MBEDTLS_SSL_RECORD_CHECKING`: restore it.
* `MBEDTLS_SSL_CONTEXT_SERIALIZATION`: restore it.
* `MBEDTLS_USE_PSA_CRYPTO` note: don't restore the tls version.
* `MBEDTLS_USE_PSA_CRYPTO` warning: restore the tls version.
* `MBEDTLS_CMAC_C`: restore it to being disabled by default. It's a minor API change in Mbed TLS because it changes the layout of `mbedtls_cipher_context_t`.
* `MBEDTLS_CTR_DRBG_C`: don't restore the older version of the description from tls.
* `MBEDTLS_GCM_C`: don't restore the older description from tls.
* `MBEDTLS_PSA_CRYPTO_C`: don't restore `crypto/`.
* `MBEDTLS_PSA_CRYPTO_SE_C`: don't remove it.
* `MBEDTLS_PSA_CRYPTO_STORAGE_C`: don't restore `crypto/`. Don't disable it by default.
* `MBEDTLS_PSA_ITS_FILE_C`: don't restore, like for ``MBEDTLS_PSA_CRYPTO_STORAGE_C``.
* `MBEDTLS_CTR_DRBG_USE_128_BIT_KEY` single line: don't restore it since there is now a full paragraph in the proper section above.
* `MBEDTLS_SSL_CID_IN_LEN_MAX`…: restore it.
* `MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_CERTIFICATES`: restore the version without a space to format the line like the others.
Merge `unremove-non-crypto` into `mbedtls/development`. The branch
`unremove-non-crypto` was obtained by starting from `mbed-crypto/development`,
then reverting many commits that removed X.509 and TLS functionality when Mbed
Crypto forked from Mbed TLS (the “unremoval”), then make a few tweaks to
facilitate the merge.
The unremoval step restored old versions of some tls files. If a file doesn't
exist in mbed-crypto, check out the mbedtls version, regardless of what
happened during the unremoval of tls files in the crypto tree. Also
unconditionally take the mbedtls version of a few files where the
modifications are completely project-specific and are not relevant in
mbed-crypto:
* `.github/issue_template.md`: completely different. We may want to reconcile
them independently as a follow-up.
* `.travis.yml`: would only be reverted to an earlier tls version.
* `README.md`: completely different. We may want to reconcile them
independently as a follow-up.
* `doxygen/input/doc_mainpage.h`: the changes in crypto were minimal and not
relevant except as a stopgap as mbed-crypto did not have its own product
versioning in the Doxygen documentation.
* `tests/.jenkins/Jenkinsfile`: completely different.
* `tests/data_files/Makefile`: there were no changes in mbed-crypto,
but the unremoval step restored an old version.
Shell script for everything to do after the merge apart from the conflict
resolution:
```
tls_files=($(comm -23 <(git ls-tree -r --name-only HEAD) <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development MERGE_HEAD))))
tls_files+=($tls_files .github/issue_template.md .travis.yml README.md doxygen/input/doc_mainpage.h tests/.jenkins/Jenkinsfile tests/data_files/Makefile)
git checkout --theirs HEAD -- $tls_files
git add -- $tls_files
```
Resolve the remaining conflicts:
* `library/CMakeLists.txt`:
* Keep the TLS definition of `src_crypto`
* `USE_SHARED_MBEDTLS_LIBRARY`: keep all three libraries, with both
`include` and `crypto/include` in `target_include_directories`, all with
version `2.21.0`.
* `programs/Makefile`:
* Reconcile the APPS lists (add/add from a differently-formatted common
ancestor): insert the `psa/*` from crypto into the tls list.
* Keep the `fuzz` target defined only in tls version.
* Keep the recipe (only in tls version) cleaning `ssl_pthread_server`
stuff for the `clean` target.
* `scripts/config.py`:
* `include_in_full`: add/add conflict. Keep both.
* `tests/scripts/all.sh`:
* `component_test_no_use_psa_crypto_full_cmake_asan`: partially old
version in crypto. Take the tls version.
* `component_test_malloc_0_null` and more: take
`component_test_malloc_0_null` from crypto (with `config.py` rather than
`config.pl`, and with `$ASAN_FLAGS` rather than an explicit list), but
add the call to `ssl-opt.sh` from tls. Take the other components from
crypto.
With this commit, building and running the unit tests with both `make ` and
`cmake` work in the default configuration on Linux. Other platforms, build
systems and configurations are likely not to work, and there is some
regression in test coverage.
There is some loss of functionality because the unremoval step restored older
versions of tls content. This commit contains the latest tls version of
tls-only files, but some changes from the tls side in files that existed on
both sides have regressed. Most problematic changes are hunks that remove some
tls-specific feature and contain either a C preprocessor symbol identifying a
tls-specific module or option, or the name of a tls-specific file. Hunks
that remove a tls-specific preprocessor symbol can be identified with the
regular expression `^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_`.
Subsequent commits will revert a few parts of the patch from this merge commit
in order to restore the tls functionality that it removes, ensure that the
test coverage includes what was covered in either branch, and fix test
failures.
This reverts commit 9afb2e9921.
Conflicts:
* include/CMakeLists.txt
* "Make config.h available" comment: there has been a change
adjacent to where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
* compat.sh: there has been a change immediately before where it was
removed. Just re-add what was removed.
This reverts commit 1c66e48670.
Conflicts:
* include/mbedtls/check_config.h:
* MBEDTLS_SSL_PROTO_SSL3: there has been an addition (of
MBEDTLS_SHA512_NO_SHA384) at the place where it was removed. Re-add it
after (alphabetical order).
* MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
Re-add it after (alphabetical order).
* MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
after (alphabetical order).
* include/mbedtls/config.h:
* MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
Re-add it after (alphabetical order).
* MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
after (alphabetical order).
* library/version_features.c: re-generate by running
scripts/generate_features.pl.
* programs/test/query_config.c: re-generate by running
scripts/generate_query_config.pl.
* scripts/config.pl: this file has been replaced by config.py. Port
the reversed changes to config.py:
* Revert removing three symbols from the list of symbols to
exclude from full.
* Revert removing one symbol (MBEDTLS_NET_C) from the list of symbols
to exclude from baremetal.
* scripts/footprint.sh:
* Re-add the line to unset MBEDTLS_NET_C, but with config.py instead of
config.pl.
* tests/scripts/all.sh:
* component_test_no_platform: re-add the line to unset MBEDTLS_NET_C, but
with config.py instead of config.pl.
* component_build_arm_none_eabi_gcc,
component_build_arm_none_eabi_gcc_no_udbl_division,
component_build_arm_none_eabi_gcc_no_64bit_multiplication,
component_build_armcc: these components now use the baremetal
configuration, so they do not need to turn off MBEDTLS_NET_C explicitly.
This reverts commit bb1f701212.
* include/mbedtls/check_config.h:
* MBEDTLS_X509_RSASSA_PSS_SUPPORT: there has been an addition (of
MBEDTLS_SHA512_NO_SHA384) at the place where it was removed.
Re-add it before MBEDTLS_SHA512_NO_SHA384 to keep it grouped
with MBEDTLS_RSA_C.
Conflicts:
* scripts/config.pl: this file has been replaced by config.py. Port
the reversed changes to config.py:
* Revert removing three symbols from the list of symbols to
exclude from full.
Add a description of MBEDTLS_ERR_SSL_PEER_CLOSE_NOTIFY to the
documentation, as suggested by hanno-arm.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Add a conditional buffer resizing feature. Introduce tests exercising
it in various setups (serialization, renegotiation, mfl manipulations).
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
This patch changes the compatibility API defined in crypto_compat.h
to static inline functions as the previous macro definitions were
causing issues for the C pre-processor when included in projects
which need to redefine the PSA function names. Making it static
inline function solves this problem neatly and also modern compilers
do a good job at inlining the function which makes the need for making
it a macro redundant.
Signed-off-by: Soby Mathew <soby.mathew@arm.com>
* iotssl-2652-deprecate-pkcs11:
Group PKCS11_C entries in check_config.h
Clarify that what we're dropping is pkcs11-helper support
Fix typo in doxy docs for ssl_pkcs11_sign()
Add missing docs to PKCS#11 public funcs
Wrap PKCS1 module with DEPRECATED_REMOVED
Fix deprecated docs for PKCS1
Deprecate MBEDTLS_PKCS11_C functions
Add ChangeLog entry for MBEDTLS_PKCS11_C deprecation
Deprecate MBEDTLS_PKCS11_C feature
Files deleted by us: keep them deleted.
```
git rm $(git status -s | sed -n 's/^DU //p')
```
Individual files with conflicts:
* `README.md`: keep the crypto version.
* `doxygen/input/doc_mainpage.h`: keep the crypto version (with an obsolete Mbed Crypto version number).
* `include/mbedtls/error.h`:
* `ERROR`: similar additions made through parallel commits, with only whitespace differences. Align with the tls version.
* `library/CMakeLists.txt`: keep the crypto version.
* `library/Makefile`: keep the crypto version.
* `scripts/generate_errors.pl`: keep the crypto version (the relevant changes were made through parallel commits).
* `tests/scripts/check-test-cases.py`:
* `Results`: keep the crypto version, which has both the new argument to the constructor (added in crypto only) and the class docstring (added through parallel commits).
* `tests/suites/helpers.function`:
* `ARRAY_LENGTH`, `ASSERT_ALLOC`: additions in the same location. Keep both, in indifferent order.
* `tests/suites/target_test.function`:
* `receive_uint32`: keep the crypto version which has an additional bug fix. The tls changes made in tls are irrelevant after this bug fix.
* `visualc/VS2010/mbedTLS.vcxproj`: run `scripts/generate_visualc_files.pl`.
Review of non-conflicting changes:
* `all.sh`: 1 change.
* zlib test components: don't add them.
* `include/CMakeLists.txt`: 1 change.
* `target_include_directories`: doesn't work as is (different target name). Don't take the change.
* All other non-conflicting changes: take them.
Adapt to the change of encoding of elliptic curve key types in PSA
crypto. Before, an EC key type encoded the TLS curve identifier. Now
the EC key type only includes an ad hoc curve family identifier, and
determining the exact curve requires both the key type and size. This
commit moves from the old encoding and old definitions from
crypto/include/mbedtls/psa_util.h to the new encoding and definitions
from the immediately preceding crypto submodule update.
Change the encoding of key types, EC curve families and DH group
families to make the low-order bit a parity bit (with even parity).
This ensures that distinct key type values always have a Hamming
distance of at least 2, which makes it easier for implementations to
resist single bit flips.
All key types now have an encoding on 32 bits where the bottom 16 bits
are zero. Change to using 16 bits only.
Keep 32 bits for key types in storage, but move the significant
half-word from the top to the bottom.
Likewise, change EC curve and DH group families from 32 bits out of
which the top 8 and bottom 16 bits are zero, to 8 bits only.
Reorder psa_core_key_attributes_t to avoid padding.
Remove the values of curve encodings that are based on the TLS registry
and include the curve size, keeping only the new encoding that merely
encodes a curve family in 8 bits.
Keep the old constant names as aliases for the new values and
deprecate the old names.
Define constants for ECC curve families and DH group families. These
constants have 0x0000 in the lower 16 bits of the key type.
Support these constants in the implementation and in the PSA metadata
tests.
Switch the slot management and secure element driver HAL tests to the
new curve encodings. This requires SE driver code to become slightly
more clever when figuring out the bit-size of an imported EC key since
it now needs to take the data size into account.
Switch some documentation to the new encodings.
Remove the macro PSA_ECC_CURVE_BITS which can no longer be implemented.
Change the representation of psa_ecc_curve_t and psa_dh_group_t from
the IETF 16-bit encoding to a custom 24-bit encoding where the upper 8
bits represent a curve family and the lower 16 bits are the key size
in bits. Families are based on naming and mathematical similarity,
with sufficiently precise families that no two curves in a family have
the same bit size (for example SECP-R1 and SECP-R2 are two different
families).
As a consequence, the lower 16 bits of a key type value are always
either the key size or 0.
Don't assume that the PSA encoding of elliptic curves is identical to
the TLS encoding. This is currently true but about to change.
The new implementation only works when MBEDTLS_ECP_C is defined. This
is ok because the function is only used with MBEDTLS_ECP_C defined.
Internally, use the corresponding function from psa_crypto.c instead.
Externally, this function is not used in Mbed TLS and is documented as
"may change at any time".
Key types are now encoded through a category in the upper 4 bits (bits
28-31) and a type-within-category in the next 11 bits (bits 17-27),
with bit 16 unused and bits 0-15 only used for the EC curve or DH
group.
For symmetric keys, bits 20-22 encode the block size (0x0=stream,
0x3=8B, 0x4=16B).
Change the numerical encoding of values for symmetric key types to
have 0000 as the lower 16 bits. Now the lower 16 bits are only used
for key types that have a subtype (EC curve or DH group).
Whether a parameter should be const is an implementation detail of the
function, so don't declare a parameter of psa_hash_compare as
const. (This only applies to parameters themselves, not to objects
that pointer parameters points to.)
Before the initial seeding, reseed_counter used to always be 0. Now, the
value depends on whether or not the user has explicitly set the amount
of data to get from the nonce (via e.g.
mbedtls_ctr_drbg_set_nonce_len()). Add comments to clarify the possible
values reseed_counter can have before the initial seeding.
One of the error codes was already reserved, this commit just makes it
explicit. The other one is a new error code for initializing return
values in the library: `MBEDTLS_ERR_ERROR_CORRUPTION_DETECTED` should
not be returned by the library. If it is returned, then it is surely a
bug in the library or somebody is tampering with the device.
