Merge `unremove-non-crypto` into `mbedtls/development`. The branch
`unremove-non-crypto` was obtained by starting from `mbed-crypto/development`,
then reverting many commits that removed X.509 and TLS functionality when Mbed
Crypto forked from Mbed TLS (the “unremoval”), then make a few tweaks to
facilitate the merge.
The unremoval step restored old versions of some tls files. If a file doesn't
exist in mbed-crypto, check out the mbedtls version, regardless of what
happened during the unremoval of tls files in the crypto tree. Also
unconditionally take the mbedtls version of a few files where the
modifications are completely project-specific and are not relevant in
mbed-crypto:
* `.github/issue_template.md`: completely different. We may want to reconcile
them independently as a follow-up.
* `.travis.yml`: would only be reverted to an earlier tls version.
* `README.md`: completely different. We may want to reconcile them
independently as a follow-up.
* `doxygen/input/doc_mainpage.h`: the changes in crypto were minimal and not
relevant except as a stopgap as mbed-crypto did not have its own product
versioning in the Doxygen documentation.
* `tests/.jenkins/Jenkinsfile`: completely different.
* `tests/data_files/Makefile`: there were no changes in mbed-crypto,
but the unremoval step restored an old version.
Shell script for everything to do after the merge apart from the conflict
resolution:
```
tls_files=($(comm -23 <(git ls-tree -r --name-only HEAD) <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development MERGE_HEAD))))
tls_files+=($tls_files .github/issue_template.md .travis.yml README.md doxygen/input/doc_mainpage.h tests/.jenkins/Jenkinsfile tests/data_files/Makefile)
git checkout --theirs HEAD -- $tls_files
git add -- $tls_files
```
Resolve the remaining conflicts:
* `library/CMakeLists.txt`:
* Keep the TLS definition of `src_crypto`
* `USE_SHARED_MBEDTLS_LIBRARY`: keep all three libraries, with both
`include` and `crypto/include` in `target_include_directories`, all with
version `2.21.0`.
* `programs/Makefile`:
* Reconcile the APPS lists (add/add from a differently-formatted common
ancestor): insert the `psa/*` from crypto into the tls list.
* Keep the `fuzz` target defined only in tls version.
* Keep the recipe (only in tls version) cleaning `ssl_pthread_server`
stuff for the `clean` target.
* `scripts/config.py`:
* `include_in_full`: add/add conflict. Keep both.
* `tests/scripts/all.sh`:
* `component_test_no_use_psa_crypto_full_cmake_asan`: partially old
version in crypto. Take the tls version.
* `component_test_malloc_0_null` and more: take
`component_test_malloc_0_null` from crypto (with `config.py` rather than
`config.pl`, and with `$ASAN_FLAGS` rather than an explicit list), but
add the call to `ssl-opt.sh` from tls. Take the other components from
crypto.
With this commit, building and running the unit tests with both `make ` and
`cmake` work in the default configuration on Linux. Other platforms, build
systems and configurations are likely not to work, and there is some
regression in test coverage.
There is some loss of functionality because the unremoval step restored older
versions of tls content. This commit contains the latest tls version of
tls-only files, but some changes from the tls side in files that existed on
both sides have regressed. Most problematic changes are hunks that remove some
tls-specific feature and contain either a C preprocessor symbol identifying a
tls-specific module or option, or the name of a tls-specific file. Hunks
that remove a tls-specific preprocessor symbol can be identified with the
regular expression `^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_`.
Subsequent commits will revert a few parts of the patch from this merge commit
in order to restore the tls functionality that it removes, ensure that the
test coverage includes what was covered in either branch, and fix test
failures.
This reverts commit d832f187f7.
Conflicts:
* CMakeLists.txt:
* USE_PKCS11_HELPER_LIBRARY: there has been a change immediately before
where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
* USE_PKCS11_HELPER_LIBRARY: there has been a change immediately before
where it was removed. Just re-add what was removed.
This reverts commit d874a1fd14.
Conflicts:
* CMakeLists.txt:
* ENABLE_ZLIB_SUPPORT: there has been a change immediately after
where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
* ENABLE_ZLIB_SUPPORT: there has been a change immediately after
where it was removed. Just re-add what was removed.
This reverts commit 8298d70bee.
Conflicts:
* library/Makefile: removal of SOEXT_X509 and SOEXT_TLS vs change of
value of SOEXT_CRYPTO. Keep all, with the new value of SOEXT_CRYPTO.
This commit adds the newly created copy ssl_msg.c of ssl_tls.c
to the build system but guards its content by an `#if 0 ... #endif`
preprocessor guard in order to avoid compilation failures resulting
from code duplication. This guard will be removed once the contents
of ssl_tls.c and ssl_msg.c have been made disjoint.
This commit is the first in a series of commits aiming to split
the content of ssl_tls.c in two files ssl_tls.c and ssl_msg.c.
As a first step, this commit replaces ssl_tls.c by two identical
copies ssl_tls_old.c and ssl_msg.c. Even though the file
ssl_tls_old.c will subsequently be renamed back into ssl_tls.c,
this approach retains the git history in both files.
Manually edit ChangeLog to ensure correct placement of ChangeLog notes.
* origin/pr/2799: (42 commits)
Handle deleting non-existant files on Windows
Update submodule
Use 3rdparty headers from the submodule
Add Everest components to all.sh
3rdparty: Add config checks for Everest
Fix macros in benchmark.c
Update generated files
3rdparty: Fix inclusion order of CMakeLists.txt
Fix trailing whitespace
ECDH: Fix inclusion of platform.h for proper use of MBEDTLS_ERR_PLATFORM_FEATURE_UNSUPPORTED
ECDH: Fix use of ECDH API in full handshake benchmark
ECDH: Removed unnecessary calls to mbedtls_ecp_group_load in ECDH benchmark
ECDH: Fix Everest x25519 make_public
Fix file permissions
3rdparty: Rename THIRDPARTY_OBJECTS
3rdparty: Update description of MBEDTLS_ECDH_VARIANT_EVEREST_ENABLED
3rdparty: Fix Makefile coding conventions
ECDSA: Refactor return value checks for mbedtls_ecdsa_can_do
Add a changelog entry for Everest ECDH (X25519)
Document that curve lists can include partially-supported curves
...
Conflict resolution:
* `scripts/config.pl`:
Take the exclusion of `MBEDTLS_PSA_CRYPTO_SE_C` from the API branch.
Take the removal of `MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C` (obsolete) from
the development branch.
* `tests/scripts/all.sh`:
Multiple instances of factoring a sequence of `config.pl` calls into
a mere `config.pl baremetal` in the development branch, and a change in
the composition of `baremetal` in the API branch. In each case, take the
version from development.
* `tests/suites/test_suite_psa_crypto_slot_management.function`:
A function became non-static in development and disappeared in the API
branch. Keep the version from the API branch. Functions need to be
non-static if they're defined but unused in some configurations,
which is not the case for any function in this file at the moment.
* `tests/suites/test_suite_psa_crypto.function`:
Consecutive changes in the two branches, reconciled.
In the generic message digest abstraction, instead of storing method
pointers in the per-algorithm data structure and using wrapper
functions as those methods, call the per-algorithm function directly.
This saves some code size (2336B -> 2043B for md with all algorithms
enabled on M0+ with gcc -Os). This should also make it easier to
optimize the case when a single algorithm is supported. In addition,
this is a very slight security improvement since it removes one
opportunity for a buffer overflow to directly turn into letting the
attacker overwrite a pointer to a function pointer.
This commit does not modify the documented API. However, it removes
the possibility for users to define their own hash implementations and
use them by building their own md_info.
Changing mbedtls_md_context to contain a md type identifier rather
than a pointer to an info structure would save a few more bytes and a
few more runtime memory accesses, but would be a major API break since
a lot of code uses `const mbedtls_md_info *` to keep track of which
hash is in use.
Remove use of CMAKE_SOURCE_DIR in case mbedtls is built from within
another CMake project. Define MBEDTLS_DIR to ${CMAKE_CURRENT_SOURCE_DIR}
in the main CMakeLists.txt file and refer to that when defining target
include paths to enable mbedtls to be built as a sub project.
Fixes#2609
Signed-off-by: Ashley Duncan <ashes.man@gmail.com>
Signed-off-by: Jaeden Amero <jaeden.amero@arm.com>
Remove use of CMAKE_SOURCE_DIR in case mbedtls is built from within
another CMake project. Define MBEDTLS_DIR to ${CMAKE_CURRENT_SOURCE_DIR}
in the main CMakeLists.txt file and refer to that when defining target
include paths to enable mbedtls to be built as a sub project.
Fixes https://github.com/ARMmbed/mbedtls/issues/2609
Signed-off-by: Ashley Duncan <ashes.man@gmail.com>
Signed-off-by: Jaeden Amero <jaeden.amero@arm.com>
Increase the SO versions of libmbedx509 and libmbedtls due to the
addition of fields in publicly visible (non-opaque) structs:
- mbedtls_ssl_config
- mbedtls_ssl_context
- mbedtls_x509_crt
When building as a submodule of a parent project, like Mbed TLS, use the
parent projects generated source files (error.c, version.c,
version_features.c)
Resolve conflicts actions:
- Reject path changes to config.h
- Reject submodule-related changes in build scripts (Makefile,
CMakeLists.txt)
- Add oid test suite to list of tests in tests/CMakeLists.txt,
rejecting any test filtering related changes (which TLS uses to avoid
duplicating crypto tests)
- Add legacy ECDH test to all.sh without including
all.sh tests that depend on SSL
Since the ITS API has stabilized and we don't plan to make use of more
than ITS, we don't need an abstraction layer between key storage and
key storage over ITS. Merge the ITS code into the generic storage
module.
Now that we have ITS over files, we no longer need a direct backend
for key storage over files. Remove psa_crypto_storage_file and its
tests.
Switch MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C and MBEDTLS_PSA_ITS_FILE_C on
by default. This preserves functionality and test coverage in the
default configuration, but forgets any key previously stored using the
file backend.
Resolve conflicts by performing the following.
- Take the upstream Mbed TLS ChangeLog verbatim.
- Reject changes to Makefiles and CMake that are related to using Mbed
Crypto as a submodule. It doesn't make sense to use Mbed Crypto as a
submodule of itself.
- Reject README changes, as Mbed Crypto has its own, different README.
- Reject PSA-related changes to config.h. We don't want to disable the
availability of the PSA Crypto API by default in the Mbed Crypto
config.h.
- Don't inadvertently revert dead code removal in
mbedtls_cipher_write_tag() which was added in f2a7529403 ("Fix
double return statement in cipher.c")
- Where Mbed Crypto already had some MBEDTLS_USE_PSA_CRYPTO code (from
past companion PRs) take the latest version from Mbed TLS which
includes integration with MBEDTLS_CHECK_PARAMS.
- Update the version of the shared library files to match what's
currently present in Mbed TLS.
- Reject removal of testing with PSA from config full tests.
- Resolve conflicts in test tests/suites/helpers.function, where both
Mbed Crypto and Mbed TLS both added documentation for TEST_ASSERT.
Combine text from both documentation efforts.
- Reject adding a submodule of ourselves.
- Reject addition of submodule tests in all.sh.
- Reject addition of submodule to library path in
tests/scripts/run-test-suites.pl.
- Avoid using USE_CRYPTO_SUBMODULE=1 in
component_test_use_psa_crypto_full_cmake_asan() in all.sh.
Additional changes to temporarily enable running tests:
ssl_srv.c and test_suite_ecdh use mbedtls_ecp_group_load instead of
mbedtls_ecdh_setup
test_suite_ctr_drbg uses mbedtls_ctr_drbg_update instead of
mbedtls_ctr_drbg_update_ret
Use `cmake -D CMAKE_BUILD_TYPE=Asan` rather than manually setting
`-fsanitize=address`. This lets cmake determine the necessary compiler
and linker flags.
With UNSAFE_BUILD on, force -Wno-error. This is necessary to build
with MBEDTLS_TEST_NULL_ENTROPY.
Use `cmake -D CMAKE_BUILD_TYPE=Asan` rather than manually setting
`-fsanitize=address`. This lets cmake determine the necessary compiler
and linker flags.
With UNSAFE_BUILD on, force -Wno-error. This is necessary to build
with MBEDTLS_TEST_NULL_ENTROPY.
Implement psa_allocate_key, psa_open_key, psa_create_key,
psa_close_key.
Add support for keys designated to handles to psa_get_key_slot, and
thereby to the whole API.
Allocated and non-allocated keys can coexist. This is a temporary
stage in order to transition from the use of direct slot numbers to
allocated handles only. Once all the tests and sample programs have
been migrated to use handles, the implementation will be simplified
and made more robust with support for handles only.
