* origin/pr/2532: (29 commits)
Document and test flags in x509_verify
Fix style issues and a typo
Fix name to function call
Address comments for x509 tests
Address review comments regarding ssl_client2 and ssl tests
Remove mbedtls_ from the static function name
Change docs according to review comments
Change the verify function naming
Fix ssl_client2 and ssl_server2 if !PLATFORM_C
Correct placement of usage macro in ssl_client2
Update version_features.c
Remove trailing whitespace in test_suite_x509parse.function
Update query_config.c
Add ssl-opt.sh tests for trusted CA callbacks
Only run X.509 CRT verification tests with CA callback tests if !CRL
Minor fixes to CA callback tests
Declare CA callback type even if feature is disabled
Implement X.509 CRT verification using CA callback
Add prototype for CRT verification with static and dynamic CA list
Make use of CA callback if present when verifying peer CRT chain
...
The documentation doesn't explicitly say whether it's allowed or not.
This currently works with the default software implementation, but
only by accident. It isn't guaranteed to work with new ciphers or with
alternative implementations of individual ciphers, and it doesn't work
with the PSA wrappers. So don't do it.
Run ssl-opt.sh on x86_32 with ASan. This may detect bugs that only
show up on 32-bit platforms, for example due to size_t overflow.
For this component, turn off some memory management features that are
not useful, potentially slow, and may reduce ASan's effectiveness at
catching buffer overflows.
Merge the Mbed Crypto development branch a little after
mbedcrypto-1.0.0 into the PSA Crypto API 1.0 beta branch a little
after beta 2.
Summary of merge conflicts:
* Some features (psa_copy_key, public key format without
SubjectPublicKeyInfo wrapping) went into both sides, but with a few
improvements on the implementation side. For those, take the
implementation side.
* The key derivation API changed considerably on the API side. This
merge commit generally goes with the updated API except in the tests
where it keeps some aspects of the implementation.
Due to the divergence between the two branches on key derivation and
key agreement, test_suite_psa_crypto does not compile. This will be
resolved in subsequent commits.
* origin/pr/2464:
Allow main() to lack a docstring.
Silence pylint
check-files.py: readability improvement in permission check
check-files.py: use class fields for class-wide constants
check-files.py: clean up class structure
abi_check.py: Document more methods
check-files.py: document some classes and methods
Fix pylint errors going uncaught
Call pylint3, not pylint
New, documented pylint configuration
When doing ABI/API checking, its useful to have a list of all the
identifiers that are defined in the internal header files, as we
do not promise compatibility for them. This option allows for a
simple method of getting them for use with the ABI checking script.
* origin/pr/2192:
Increase okm_hex buffer to contain null character
Minor modifications to hkdf test
Add explanation for okm_string size
Update ChangeLog
Reduce buffer size of okm
Reduce Stack usage of hkdf test function
It was failing to set the key in the ENCRYPT direction before encrypting.
This just happened to work for GCM and CCM.
After re-encrypting, compare the length to the expected ciphertext
length not the plaintext length. Again this just happens to work for
GCM and CCM since they do not perform any kind of padding.
Resolve conflicts actions:
- Reject path changes to config.h
- Reject submodule-related changes in build scripts (Makefile,
CMakeLists.txt)
- Add oid test suite to list of tests in tests/CMakeLists.txt,
rejecting any test filtering related changes (which TLS uses to avoid
duplicating crypto tests)
- Add legacy ECDH test to all.sh without including
all.sh tests that depend on SSL
* restricted/pr/551:
ECP: Clarify test descriptions
ECP: remove extra whitespaces
Fix ECDH secret export for Mongomery curves
Improve ECP test names
Make ecp_get_type public
Add more tests for ecp_read_key
ECP: Catch unsupported import/export
Improve documentation of mbedtls_ecp_read_key
Fix typo in ECP module
Remove unnecessary cast from ECP test
Improve mbedtls_ecp_point_read_binary tests
Add Montgomery points to ecp_point_write_binary
ECDH: Add test vectors for Curve25519
Add little endian export to Bignum
Add mbedtls_ecp_read_key
Add Montgomery points to ecp_point_read_binary
Add little endian import to Bignum
Ensure this merge passes tests by auto-generating query_config.c, adding
MBEDTLS_ECDH_LEGACY_CONTEXT to it.
* restricted/pr/552:
Fix mbedtls_ecdh_get_params with new ECDH context
Test undefining MBEDTLS_ECDH_LEGACY_CONTEXT in all.sh
Define MBEDTLS_ECDH_LEGACY_CONTEXT in config.h
Add changelog entry for mbedtls_ecdh_get_params robustness
Fix ecdh_get_params with mismatching group
Add test case for ecdh_get_params with mismatching group
Add test case for ecdh_calc_secret
Fix typo in documentation
Ensure tests pass when the submodule is used by updating the list of
crypto tests to include test_suite_oid in both tests/CMakeLists.txt and
tests/Makefile.
* origin/pr/2531:
Add changeLog entry
Add certificate policy of type any policy id
* origin/pr/2509:
all.sh: Generate seedfile for crypto submodule tests
Update crypto submodule to test with private headers
tests: Use globbing in test suite exclusion list
Update crypto submodule to Mbed Crypto development
tests: Test crypto via the crypto submodule
Some tests use internal-only header files, which are stored in the
'library' folder, and therefore need the library folder passed in on the
include path. For non-submoudle builds, this is set globally in the
top-level CMakeLists.txt file. For submodule builds, this is set through
target includes to a path only meaningful when Mbed Crypto is built as a
submodule.
MBEDTLS_PSA_HAS_ITS_IO is not really useful since it doesn't actually
enable anything except the entropy seed file support, which only
requires the ITS interface and not a native implemetation. Remove it.
Since there is now a single storage backend, we don't need a backend
interface. Make the functions that were declared in
psa_crypto_storage_backend.h and are now both defined and used in
psa_crypto_storage.c static, except for psa_is_key_present_in_storage
which is used by the gray-box tests and is now declared in
psa_crypto_storage.h.
Now that we have ITS over files, we no longer need a direct backend
for key storage over files. Remove psa_crypto_storage_file and its
tests.
Switch MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C and MBEDTLS_PSA_ITS_FILE_C on
by default. This preserves functionality and test coverage in the
default configuration, but forgets any key previously stored using the
file backend.
Test the crypto implementation via tests from the Mbed Crypto submodule
instead of at the Mbed TLS top level.
The version test is the only test that is tested from both TLS and
Crypto, despite being entirely in libmbedcrypto. This is because the
test data is code-gen'd from the version updating script and the version
between Mbed TLS and Mbed Crypto don't necessarily always agree. The
test data must come from the top level module, as only the top level
module will have test data that matches the expected version.
Prepend ".crypto" to tests that came from the crypto submodule. This
allows, when this project is used as a submodule, for tests with names
the same between the parent and this project when used as a submodule to
both be built and run.
The version test suite is duplicated between Mbed TLS and Mbed Crypto.
Use TLS's copy and not Crypto's copy when Crypto is used as a submodule
of TLS.
The version test is the only test that is tested from both TLS and
Crypto, despite being entirely in libmbedcrypto. This is because the
test data is code-gen'd from the version updating script and the version
between Mbed TLS and Mbed Crypto don't necessarily always agree. The
test data must come from the top level module, as only the top level
module will have test data that matches the expected version.
For Makefiles, enable overriding where includes can come from in order
to enable the parent module to set the include path. This allows the
parent module to specify that its config.h should be used, even when the
submodule when built standalone would use a different config.h.
For CMake, always look in the parent's include folder and our own. List
the parent's include folder first, so that preference is given to parent
include files.
As the SSL programs, like ssl_client2 and ssl_server2, are dependent on
SSL and therefore about to be removed, the only consumer of query_config
is the query_compile_time_config test. As such, it makes sense to move
query_config to be next to what uses it.
* origin/development: (113 commits)
Update query_config.c
Fix failure in SSLv3 per-version suites test
Adjust DES exclude lists in test scripts
Clarify 3DES changes in ChangeLog
Fix documentation for 3DES removal
Exclude 3DES tests in test scripts
Fix wording of ChangeLog and 3DES_REMOVE docs
Reduce priority of 3DES ciphersuites
Fix unused variable warning in ssl_parse_certificate_coordinate()
Update the crypto submodule to a78c958
Fix ChangeLog entry to correct release version
Fix typo in x509write test data
Add ChangeLog entry for unused bits in bitstrings
Improve docs for named bitstrings and their usage
Add tests for (named) bitstring to suite_asn1write
Add new function mbedtls_asn1_write_named_bitstring()
Add missing compile time guard in ssl_client2
Update programs/ssl/query_config.c
ssl_client2: Reset peer CRT info string on reconnect
Add further debug statements on assertion failures
...
* origin/pr/1520:
Use certificates from data_files and refer them
Specify server certificate to use in SHA-1 test
refactor CA and SRV certificates into separate blocks
refactor SHA-1 certificate defintions and assignment
refactor server SHA-1 certificate definition into a new block
define TEST_SRV_CRT_RSA_SOME in similar logic to TEST_CA_CRT_RSA_SOME
server SHA-256 certificate now follows the same logic as CA SHA-256 certificate
add entry to ChangeLog
* public/pr/2421: (68 commits)
Fix unused variable warning in ssl_parse_certificate_coordinate()
Add missing compile time guard in ssl_client2
Update programs/ssl/query_config.c
ssl_client2: Reset peer CRT info string on reconnect
Add further debug statements on assertion failures
Fix typo in documentation of ssl_parse_certificate_chain()
Add debug output in case of assertion failure
Fix typo in SSL ticket documentation
Add config sanity check for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
ssl_client2: Zeroize peer CRT info buffer when reconnecting
Reintroduce numerous ssl-opt.sh tests if !MBEDTLS_SSL_KEEP_PEER_CERT
ssl_client2: Extract peer CRT info from verification callback
Improve documentation of mbedtls_ssl_get_peer_cert()
Improve documentation of MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
Fix indentation of Doxygen comment in ssl_internal.h
Set peer CRT length only after successful allocation
Remove question in comment about verify flags on cli vs. server
Remove misleading and redundant guard around restartable ECC field
Add test for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE to all.sh
Free peer CRT chain immediately after verifying it
...
* restricted/pr/528:
Update query_config.c
Fix failure in SSLv3 per-version suites test
Adjust DES exclude lists in test scripts
Clarify 3DES changes in ChangeLog
Fix documentation for 3DES removal
Exclude 3DES tests in test scripts
Fix wording of ChangeLog and 3DES_REMOVE docs
Reduce priority of 3DES ciphersuites
* public/pr/2028:
Update the crypto submodule to a78c958
Fix ChangeLog entry to correct release version
Fix typo in x509write test data
Add ChangeLog entry for unused bits in bitstrings
Improve docs for named bitstrings and their usage
Add tests for (named) bitstring to suite_asn1write
Add new function mbedtls_asn1_write_named_bitstring()
The test used 3DES as the suite for SSLv3, which now makes the handshake fails
with "no ciphersuite in common", failing the test as well. Use Camellia
instead (as there are not enough AES ciphersuites before TLS 1.2 to
distinguish between the 3 versions).
Document some dependencies, but not all. Just trying to avoid introducing new
issues by using a new cipher here, not trying to make it perfect, which is a
much larger task out of scope of this commit.
Additional work done as part of merge:
- Run ./tests/scripts/check-generated-files.sh and check in the
resulting changes to programs/ssl/query_config.c
Line issue trackers are conceptually a subclass of file issue
trackers: they're file issue trackers where issues arise from checking
each line independently. So make it an actual subclass.
Pylint pointed out the design smell: there was an abstract method that
wasn't always overridden in concrete child classes.
Make check-python-files.sh run pylint on all *.py files (in
directories where they are known to be present), rather than list
files explicitly.
Fix a bug whereby the return status of check-python-files.sh was only
based on the last file passing, i.e. errors in other files were
effectively ignored.
Make check-python-files.sh run pylint unconditionally. Since pylint3
is not critical, make all.sh to skip running check-python-files.sh if
pylint3 is not available.
The pylint configuration in .pylint was a modified version of the
output of `pylint --generate-rcfile` from an unknown version of
pylint. Replace it with a file that only contains settings that are
modified from the default, with an explanation of why each setting is
modified.
The new .pylintrc was written from scratch, based on the output of
pylint on the current version of the files and on a judgement of what
to silence generically, what to silence on a case-by-case basis and
what to fix.
The ecp_get_type function comes handy in higher level modules and tests
as well. It is not inline anymore, to enable alternative implementations
to implement it for themselves.
mbedtls_ecp_read_key() module returned without an error even when
importing keys corresponding to the requested group was not
implemented.
We change this and return an error when the requested group is not
supported and make the remaining import/export functions more robust.
Commit "Smoke-test operation contexts after setup+abort" replaced
{failed-setup; abort} sequences by {failed-setup; successful-setup}.
We want to test that, but we also want to test {failed-setup; abort}.
So test {failed-setup; abort; failed-setup; successful-setup}.
After a successful setup followed by abort, or after a failed setup
from an inactive state, a context must be usable. Test this for
hash, MAC and cipher contexts.
Renamed the tests because they are explicitly testing Curve25519 and
nothing else. Improved test coverage, test documentation and extended
in-code documentation with a specific reference to the standard as well.
The library is able to perform computations and cryptographic schemes on
curves with x coordinate ladder representation. Here we add the
capability to export such points.
The function `mbedtls_mpi_write_binary()` writes big endian byte order,
but we need to be able to write little endian in some caseses. (For
example when handling keys corresponding to Montgomery curves.)
Used `echo xx | tac -rs ..` to transform the test data to little endian.
The private keys used in ECDH differ in the case of Weierstrass and
Montgomery curves. They have different constraints, the former is based
on big endian, the latter little endian byte order. The fundamental
approach is different too:
- Weierstrass keys have to be in the right interval, otherwise they are
rejected.
- Any byte array of the right size is a valid Montgomery key and it
needs to be masked before interpreting it as a number.
Historically it was sufficient to use mbedtls_mpi_read_binary() to read
private keys, but as a preparation to improve support for Montgomery
curves we add mbedtls_ecp_read_key() to enable uniform treatment of EC
keys.
For the masking the `mbedtls_mpi_set_bit()` function is used. This is
suboptimal but seems to provide the best trade-off at this time.
Alternatives considered:
- Making a copy of the input buffer (less efficient)
- removing the `const` constraint from the input buffer (breaks the api
and makes it less user friendly)
- applying the mask directly to the limbs (violates the api between the
modules and creates and unwanted dependency)
The library is able to perform computations and cryptographic schemes on
curves with x coordinate ladder representation. Here we add the
capability to import such points.
The function `mbedtls_mpi_read_binary()` expects big endian byte order,
but we need to be able to read from little endian in some caseses. (For
example when handling keys corresponding to Montgomery curves.)
Used `echo xx | tac -rs .. | tr [a-z] [A-Z]` to transform the test data
to little endian and `echo "ibase=16;xx" | bc` to convert to decimal.
Additional work done as part of merge:
- Run ./tests/scripts/check-generated-files.sh and check in the
resulting changes to programs/ssl/query_config.c
Add a test case for doing an ECDH calculation by calling
mbedtls_ecdh_get_params on both keys, with keys belonging to
different groups. This should fail, but currently passes.
Calling psa_*_setup() twice on a MAC, cipher, or hash context should
result in a PSA_ERROR_BAD_STATE error because the operation has already
been set up.
Fixes#10
Extend hash bad order test in line with the new bad order tests for MAC
and cipher, covering more cases and making comments and test layout
consistent.
Ensure that when doing hash operations out of order, PSA_ERROR_BAD_STATE
is returned as documented in crypto.h and the PSA Crypto specification.
In multipart cipher tests, test that each step of psa_cipher_update
produces output of the expected length. The length is hard-coded in
the test data since it depends on the mode.
The length of the output of psa_cipher_finish is effectively tested
because it's the total output length minus the length produced by the
update steps.
Test data obtained with Python+PyCrypto:
AES.new(key, mode=AES.MODE_CTR, counter=Crypto.Util.Counter.new(128, initial_value=0x2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a)).encrypt(plaintext.decode('hex')).encode('hex')
The output length can be equal to the input length.
This wasn't noticed at runtime because we happened to only test with
CBC with the first chunk being a partial block.
Some calls to psa_cipher_finish or psa_cipher_update append to a
buffer. Several of these calls were not calculating the offset into
the buffer or the remaining buffer size correctly.
This did not lead to buffer overflows before because the buffer sizes
were sufficiently large for our test inputs. This did not lead to
incorrect output when the test was designed to append but actually
wrote too early because all the existing test cases either have no
output from finish (stream cipher) or have no output from update (CBC,
with less than one block of input).
The test function pkcs1_rsaes_v15_encrypt gets its fake-random input
for padding from a test parameter. In one test case, the parameter was
too short, causing a fallback to rand(). The reference output depends
on this random input, so the test data was correct only for a platform
with one particular rand() implementation. Supply sufficient
fake-random input so that rand() isn't called.
Check generator validity (i.e. that alg has been initialized) before
allowing reads from the generator or allowing reads of the generator's
capacity.
This aligns our implementation with the documented error code behavior
in our crypto.h and the PSA Crypto API.
PSA spec now defines more generic PSA storage types instead of the ITS
specific ones. This is necessary in order to integrate with
the newer implementation of PSA ITS landing in Mbed OS soon.
Changes include the following:
- psa_status_t replaces psa_its_status_t
- psa_storage_info_t replaces psa_its_info_t
- psa_storage_uid_t replaces psa_its_uid_t
Resolve conflicts by performing the following.
- Take the upstream Mbed TLS ChangeLog verbatim.
- Reject changes to Makefiles and CMake that are related to using Mbed
Crypto as a submodule. It doesn't make sense to use Mbed Crypto as a
submodule of itself.
- Reject README changes, as Mbed Crypto has its own, different README.
- Reject PSA-related changes to config.h. We don't want to disable the
availability of the PSA Crypto API by default in the Mbed Crypto
config.h.
- Don't inadvertently revert dead code removal in
mbedtls_cipher_write_tag() which was added in f2a7529403 ("Fix
double return statement in cipher.c")
- Where Mbed Crypto already had some MBEDTLS_USE_PSA_CRYPTO code (from
past companion PRs) take the latest version from Mbed TLS which
includes integration with MBEDTLS_CHECK_PARAMS.
- Update the version of the shared library files to match what's
currently present in Mbed TLS.
- Reject removal of testing with PSA from config full tests.
- Resolve conflicts in test tests/suites/helpers.function, where both
Mbed Crypto and Mbed TLS both added documentation for TEST_ASSERT.
Combine text from both documentation efforts.
- Reject adding a submodule of ourselves.
- Reject addition of submodule tests in all.sh.
- Reject addition of submodule to library path in
tests/scripts/run-test-suites.pl.
- Avoid using USE_CRYPTO_SUBMODULE=1 in
component_test_use_psa_crypto_full_cmake_asan() in all.sh.
When using PSA with MBEDTLS_ENTROPY_NV_SEED, some test suites
require the seed file for PSA initialization, which was normally generated
later, when entropy tests were run. This change creates an initial seedfile
in all.sh.
Some of the types may in principle be wider than `unsigned`, so use
`unsigned long` in printf.
Add support for signed types: a status is a signed value, and
preferentially printed in decimal.
Don't unconditionally enable PSA Crypto for all tests. Only enable it in
tests that require it. This allows crypto tests to check that
psa_crypto_init() fails when it is supposed to fail, since we want to
perform some action in a test, and then call psa_crypto_init() and check
the result without it having been called previously.
Set the CMake-observed variable `CTEST_OUTPUT_ON_FAILURE`, so that when
a "make test" run by CMake fails, verbose test output about the detail
of failure is available.
The existing test `x509parse_crt()` for X.509 CRT parsing
so far used the generic parsing API `mbedtls_x509_crt_parse()`
capable of parsing both PEM encoded and DER encoded certficates,
but was actually only used with DER encoded input data. Moreover,
as the purpose of the test is the testing of the core DER X.509 parsing
functionality, not the PEM vs. DER dispatch (which is now already tested
in the various `x509_crt_info()` tests), the call can be replaced with a
direct call to `mbedtls_x509_parse_crt_der()`.
This commit does that, and further adds to the test an analogous
call to the new API `mbedtls_x509_parse_crt_der_nocopy()` to test
copyless parsing of X.509 certificates.
Test that freshly-initialized contexts exhibit default behavior through
the API. Do this without depending on the internal representation of the
contexts. This provides better portability of our tests on compilers
like MSVC.
For must-fail asymmetric decryption tests, add an output size parameter
so that tests can directly control what output buffer size they allocate
and use independently from the key size used. This enables better
testing of behavior with various output buffer sizes.
When RSA decrypting, unlike with RSA encrypting, we sometimes expect the
output length will be less than the key size. For instance, in the case
where the plaintext is zero-length we expect the output length of the
decryption to be zero-length as well, not key size in length.
For must-fail tests, we don't expect output-buffer-sized RSA-decryption,
only that the output length is less than or equal to the output size, so
these tests remain unchanged.
Change the must-pass tests to expect that the actual output size is
equal to the expected length of the output buffer instead of always
being the key size.
Merge a development version of Mbed TLS 2.16.0 that doesn't have
parameter validation into development.
The following conflicts were resolved:
- Update ChangeLog to include release notes merged from development so
far, with a version of "2.14.0+01b34fb316a5" and release date of
"xxxx-xx-xx" to show this is not a released version, but instead a
snapshot of the development branch equivalent to version of the 2.14.0
with additional commits from the mbedtls/development branch up through
01b34fb316 included. Entries added for unreleased versions of Mbed
Crypto remain at the top of the file for Mbed TLS 2.xx.x.
- Replace the Mbed Crypto version of
mbedtls_rsa_rsaes_pkcs1_v15_decrypt() with the version from Mbed TLS
which fixes timing variations and memory access variations that could
lead to a Bleichenbacher-style padding oracle attack. This will
prevent using psa_asymmetric_decrypt() with zero-length output buffers
until a follow up commit is made to restore this capability.
- In ssl_srv.c, include changes for both the new ECDH interface and
opaque PSK as already added to development previously.
In one place, exercise_key was used in a such a way that if the test
failed inside exercise_key, the test suite would correctly report the
test as failed but would not report the exact location of the failure.
Fix this.
Add documentation for exercise_key that explains how to use it.
When all.sh invokes check_headers_in_cpp, a backup config.h exists. This
causes a stray difference vs cpp_dummy_build.cpp. Fix by only collecting
the *.h files in include/mbedtls.
Change-Id: Ifd415027e856858579a6699538f06fc49c793570
Additional changes to temporarily enable running tests:
ssl_srv.c and test_suite_ecdh use mbedtls_ecp_group_load instead of
mbedtls_ecdh_setup
test_suite_ctr_drbg uses mbedtls_ctr_drbg_update instead of
mbedtls_ctr_drbg_update_ret
test_psa_constant_names.py was originally written before the split of
crypto.h into crypto_values.h and more, so it now needs to read
crypto_values.h as well.
In both generate_psa_constants.py and test_psa_constant_names.py, read
crypto_extra.h as well. We don't currently define any value there, but
it's plausible that we will one day.
Calls to PSA_ALG_AEAD_WITH_DEFAULT_TAG_LENGTH and
PSA_ALG_FULL_LENGTH_MAC are not in canonical form, so exclude them
from the list of constructor macros to test.
