Moved the new component_test_depends_pkalgs_psa to after the
component_test_depends_pkalgs test to be more consistent.
Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
Needed to make additional fixes so that when MBEDTLS_USE_PSA_CRYPTO
is defined, the depends-hashes test will succeed. There are two
versions of the ecdsa_verify_wrap() function, one with
MBEDTLS_USE_PSA_CRYPTO and when when it is not enabled. The non PSA
version is not using the md_alg parameter since it is not required.
The PSA version was using that parameter to derive a different value
it needed for PSA_ALG_ECDSA. The arguement of PSA_ALG_ECDSA is
ignored for psa_sign_hash and psa_verify_hash. It is present because
it is used and must be a valid hash, not zero, for psa_sign_hash
(but not psa_verify_hash) with PSA_ALG_DETERMINISTIC_ECDSA, and it is
needed for psa_sign_message and psa_verify_message which are not
implemented yet. The local parameter now uses PSA_ALG_ECDSA_ANY for
the verify function to avoid using the md_alg parameter and avoids
returning incorrect error codes.
Fixes#3587
Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
Trying to compile in the PSA accelerator test driver under MBEDTLS_TEST_HOOKS
turned out to be awkward regarding existing builds. We'll put it under a
custom (not in config.h) define instead, since it's something that only
should happen in test.
Signed-off-by: Steven Cooreman <steven.cooreman@silabs.com>
GCC up to 4.x defaults to C89. On our CI, we run the arm-none-eabi-gcc
version from Ubuntu 16.04 on Travis, and that's 4.9, so the gcc-arm
builds started failing on Travis when we introduced a C99 construct in
the configurations that we test on arm on Travis. Other builds, and
Jenkins CI, are not affected because they use GCC 5.x or newer.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Add new test (test_depends_curves_psa) to all.sh to confirm
that test is passing when MBEDTLS_USE_PSA_CRYPTO is defined.
Fix#3294
Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
Add guards in pk_wrap.c to ensure if ECDSA is not defined, errors
are returned.
Remove warnings in pk.c for unused variables.
Add new test (test_depends_pkalgs_psa) to all.sh to confirm
when USE_PSA_CRYPTO is defined that features are working properly.
Fix#3294
Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
* development:
Update copyright notices to use Linux Foundation guidance
Undef ASSERT before defining it to ensure that no previous definition has sneaked in through included files.
Add ChangeLog entry for X.509 CN-type vulnerability
Improve documentation of cn in x509_crt_verify()
Fix comparison between different name types
Add test: DNS names should not match IP addresses
Remove obsolete buildbot reference in compat.sh
Fix misuse of printf in shell script
Fix added proxy command when IPv6 is used
Simplify test syntax
Fix logic error in setting client port
ssl-opt.sh: include test name in log files
ssl-opt.sh: remove old buildbot-specific condition
ssl-opt.sh: add proxy to all DTLS tests
Signed-off-by: Dan Handley <dan.handley@arm.com>
Currently the new component in all.sh fails because
mbedtls_ssl_cf_memcpy_offset() is not actually constant flow - this is on
purpose to be able to verify that the new test works.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
As a result, the copyright of contributors other than Arm is now
acknowledged, and the years of publishing are no longer tracked in the
source files.
Also remove the now-redundant lines declaring that the files are part of
MbedTLS.
This commit was generated using the following script:
# ========================
#!/bin/sh
# Find files
find '(' -path './.git' -o -path './3rdparty' ')' -prune -o -type f -print | xargs sed -bi '
# Replace copyright attribution line
s/Copyright.*Arm.*/Copyright The Mbed TLS Contributors/I
# Remove redundant declaration and the preceding line
$!N
/This file is part of Mbed TLS/Id
P
D
'
# ========================
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
RFC5280 does not state that the `revocationDate` should be checked.
In addition, when no time source is available (i.e., when MBEDTLS_HAVE_TIME_DATE is not defined), `mbedtls_x509_time_is_past` always returns 0. This results in the CRL not being checked at all.
https://tools.ietf.org/html/rfc5280
Signed-off-by: Raoul Strackx <raoul.strackx@fortanix.com>
* development: (55 commits)
Log change as bugfix
Add changelog entry
Clarify updates to the persistent state in storage
With multiple applicable transparent drivers, the order is unspecified
Minor clarifications
Give some examples of purpsoses of pure-software transparent driver
Fix typos
Add a link to the PSA API specification
Explain locations vs lifetimes
Initialize key pointer in ecdh to NULL
Add buffer zeroization when ecp_write_key fails
Simplified key slot deletion
Style fixes
Use arc4random_buf instead of rand on NetBSD
Apply review feedback
Update open question section about public key storage
Remove the paragraph about declaring application needs
Change driver persistent data to a callback interface
Rework and expand key management in opaque drivers
Fix typos and copypasta
...
tests/scripts/curves.pl tests the library with a single curve enabled.
This uses the legacy ECDH context and the default ECDH implementation.
For Curve25519, there is an alternative implementation, which is
Everest. Test this. This also tests the new ECDH context, which
Everest requires.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This option allows to test the constant-flow nature of selected code, using
MemSan and the fundamental observation behind ctgrind that the set of
operations allowed on undefined memory by dynamic analysers is the same as the
set of operations allowed on secret data to avoid leaking it to a local
attacker via side channels, namely, any operation except branching and
dereferencing.
(This isn't the full story, as on some CPUs some instructions have variable
execution depending on the inputs, most notably division and on some cores
multiplication. However, testing that no branch or memory access depends on
secret data is already a good start.)
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This makes the implementation of mbedtls_param_failed()
for testing purpose available to programs. Thus removing
the ad-hoc implementations in programs.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
This commit was generated using the following script:
# ========================
#!/bin/sh
# Find scripts
find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i '
# Remove Mbed TLS declaration if it occurs before the copyright line
1,/Copyright.*Arm/I {
/This file is part of/,$ {
/Copyright.*Arm/I! d
}
}
# Convert non-standard header in scripts/abi_check.py to the format used in the other scripts
/"""/,/"""/ {
# Cut copyright declaration
/Copyright.*Arm/I {
h
N
d
}
# Paste copyright declaration
/"""/ {
x
/./ {
s/^/# / # Add #
x # Replace orignal buffer with Copyright declaration
p # Print original buffer, insert newline
i\
s/.*// # Clear original buffer
}
x
}
}
/Copyright.*Arm/I {
# Print copyright declaration
p
# Read the two lines immediately following the copyright declaration
N
N
# Insert Apache header if it is missing
/SPDX/! {
i\
# SPDX-License-Identifier: Apache-2.0\
#\
# Licensed under the Apache License, Version 2.0 (the "License"); you may\
# not use this file except in compliance with the License.\
# You may obtain a copy of the License at\
#\
# http://www.apache.org/licenses/LICENSE-2.0\
#\
# Unless required by applicable law or agreed to in writing, software\
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
# See the License for the specific language governing permissions and\
# limitations under the License.
# Insert Mbed TLS declaration if it is missing
/This file is part of/! i\
#\
# This file is part of Mbed TLS (https://tls.mbed.org)
}
# Clear copyright declaration from buffer
D
}
'
# ========================
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
While pure sh doesn't have a concept of local variables, we can partially
emulate them by unsetting variables before we exit the function, and use the
convention of giving them lowercase names to distinguish from global
variables.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Since dd prints everything on stderr, both normal status update and actual
errors when they occur, redirecting that to /dev/null is a trade-off that's
acceptable in quiet mode (typically used on a developer's machine and the
developer will re-run in non-quiet mode if anything fails without sufficient
detail in the output), but not that much in non-quiet mode.
For example, if our dd invocation fails because the disk in full on a CI
machine, we want the error to be reported at the time we invoke dd, and not
later when a seemingly unrelated test fails due to an incorrect seedfile.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Currently we draw pseudo-random numbers at the beginning and end of the main
loop. With ECP_RESTARTABLE, it's possible that between those two occasions we
returned from the multiplication function, hence lost our internal DRBG
context that lives in this function's stack frame. This would result in the
same pseudo-random numbers being used for blinding in multiple places. While
it's not immediately clear that this would give rise to an attack, it's also
absolutely not clear that it doesn't. So let's avoid that by using a DRBG
context that lives inside the restart context and persists across
return/resume cycles. That way the RESTARTABLE case uses exactly the
same pseudo-random numbers as the non-restartable case.
Testing and compile-time options:
- The case ECP_RESTARTABLE && !ECP_NO_INTERNAL_RNG is already tested by
component_test_no_use_psa_crypto_full_cmake_asan.
- The case ECP_RESTARTABLE && ECP_NO_INTERNAL_RNG didn't have a pre-existing
test so a component is added.
Testing and runtime options: when ECP_RESTARTABLE is enabled, the test suites
already contain cases where restart happens and cases where it doesn't
(because the operation is short enough or because restart is disabled (NULL
restart context)).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
No effect so far, except on dependency checking, as the feature it's meant to
disable isn't implemented yet (so the descriptions in config.h and the
ChangeLog entry are anticipation for now).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Similarly to the recently-added tests for dependencies on CTR_DRBG:
constrained environments will probably want only one DRBG module, and we
should make sure that tests pass in such a configuration.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The primary purpose is to use it to run all.sh -k -q in the pre-push hook, but
this can be useful in any circumstance where you're not interested in the full
output from each component and just want a short summary of which components
were run (and if any failed).
Note that only stdout from components is suppressed, stderr is preserved so
that errors are reported. This means components should avoid printing to
stderr in normal usage (ie in the absence of errors).
Currently all the `check_*` components obey this convention except:
- check_generate_test_code: unittest prints progress to stderr
- check_test_cases: lots of non-fatal warnings printed to stderr
These components will be fixed in follow-up commits.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
People who prefer to rely on HMAC_DRBG (for example because they use it for
deterministic ECDSA and don't want a second DRBG for code size reasons) should
be able to build and run the tests suites without CTR_DRBG.
Ideally we should make sure the level of testing (SSL) is the same regardless
of which DRBG modules is enabled, but that's a more significant piece of work.
For now, just ensure everything builds and `make test` passes.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This is supposed to be for GCC (or a compiler with a compatible
command line interface) targeting arm-none-eabi, so name it
accordingly.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Make it possible to use a compiler that isn't in $PATH, or that's
installed with a different name, or even a compiler for a different
target such as arm-linux-gnueabi.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Almost everything the selftest program does is in the test suites. But
just in case run the selftest program itself once in the full
configuration, and once in the default configuration with ASan, in
addition to running it out of box.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use "no_deprecated" both in the name of the configuration and in the
name of all.sh components, rather than a mixture of "no_deprecated"
and "non_deprecated".
Make all.sh component names more consistent.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
build_deprecated combined the testing of deprecated features, and
testing of the build without deprecated features. Also, it violated the
component naming convention by being called build_xxx but running tests.
Replace it by:
* test_default_no_deprecated: check that you can remove deprecated
features from the default build.
* test_full_no_deprecated: check that the library builds when deprecated
features are disabled (and incidentally that the tests run).
* test_no_deprecated_warning: check that there are no warnings when
deprecated features are disabled and MBEDTLS_DEPRECATED_WARNING
is enabled.
* test_deprecated: test the deprecated features.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Ensure that there is a build with -pedantic in the full config, not
just in "exotic" configurations.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
It started out as be experimental, but it is now robust enough not to
break the rest, so there's no reason to leave it out.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The intended logic around MBEDTLS_xxx_ALT is to exclude them from full
because they require the alternative implementation of one or more
library functions, except that MBEDTLS_PLATFORM_xxx_ALT are different:
they're alternative implementations of a platform function and they
have a built-in default, so they should be included in full. Document
this.
Fix a bug whereby MBEDTLS_PLATFORM_xxx_ALT didn't catch symbols where
xxx contains an underscore. As a consequence,
MBEDTLS_PLATFORM_GMTIME_R_ALT and MBEDTLS_PLATFORM_NV_SEED_ALT are now
enabled in the full config. Explicitly exclude
MBEDTLS_PLATFORM_SETUP_TEARDOWN_ALT because it behaves like the
non-platform ones, requiring an extra build-time dependency.
Explicitly exclude MBEDTLS_PLATFORM_NV_SEED_ALT from baremetal
because it requires MBEDTLS_ENTROPY_NV_SEED, and likewise explicitly
unset it from builds that unset MBEDTLS_ENTROPY_NV_SEED.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
check_python_files was optional in all.sh because we used to have CI
machines where pylint wasn't available. But this had the downside that
check_python_files kept breaking because it wasn't checked in the CI.
Now our CI has pylint and check_python_files should not be optional.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Merge the latest state of the target branch (mbedtls/development) into the
pull request to merge mbed-crypto into mbedtls.
Conflicts:
* ChangeLog: add/add conflict. Resolve by using the usual section order.
The merge of mbed-crypto removed some tls coverage. Restore it. Also
remove references to the `crypto` subdirectory brought by the mbedtls
side of the merge. In more detail:
* `tests/scripts/all.sh`:
* `fuzz` in comments (×2): restore it.
* `CTEST_OUTPUT_ON_FAILURE=1`: don't remove it.
* `cd crypto` for `make clean`: don't restore it.
* `cleanup`: do restore `programs/fuzz/Makefile`. Don't go into `crypto`. Keep only one copy of the calls to `rm` in `cmake_subproject`.
* Comment legacy options: don't remove it.
* `crypto/Makefile` and `pre_check_seedfile`: don't restore either. See below regarding the lack of need for `pre_check_seedfile`.
* blank line in `pre_print_configuration`: restore it.
* blank line before `#### Build and test`: restore it.
* SSL tests in `component_test_full_cmake_gcc_asan` and zlib components: restore it.
* `component_test_no_pem_no_fs` (×2): the merge placed two copies in different locations. Reconcile them: unset PSA storage like in crypto, and call `ssl-opt.sh` like in tls. Put the merged version at the tls location.
* `component_test_everest`: do add it at the tls location.
* `component_test_small_mbedtls_ssl_dtls_max_buffering`: restore the tls value.
* `component_test_new_ecdh_context`…: move `component_test_new_ecdh_context` before `component_test_everest` and add a calls to `compat.sh` and `ssl-opt.sh` like in `component_test_everest`. Remove the redundant crypto-only `component_test_everest`. Don't remove `component_test_psa_collect_statuses`.
* `component_test_full_cmake_clang`: don't remove `clang` in the `msg` call. Don't remove the call to `test_psa_constant_names.py`.
* `component_test_full_make_gcc_o0`: remove it. It's subsumed by `component_test_gcc_opt`.
* `component_build_deprecated`: don't remove anything.
* `component_test_memory_buffer_allocator`: restore `ssl-opt.sh`.
* `component_test_when_no_ciphersuites_have_mac`: restore it.
* `component_test_platform_calloc_macro`: don't restore `unset MBEDTLS_MEMORY_BUFFER_ALLOC_C` which is now redundant. Don't restore explicit flags instead of `$ASAN_CFLAGS`.
* `component_test_aes_fewer_tables`…: don't remove it.
* `component_test_m32_o1`: restore SSL testing.
* `component_test_m32_everest`: restore SSL testing.
* `component_test_min_mpi_window_size`…: don't remove it.
* `component_test_valgrind`: do restore the tls version of the comment.
* `run_component`: don't remove the seedfile creation. This is better than `pre_check_seedfile` (see below).
* `pre_check_seedfile`: don't restore it. `pre_check_seedfile` (from tls) creates a seedfile once and for all. This is not good enough if a component fails in such a way as to leave a broken seedfile, or if a component leaves a seedfile with a size that's wrong for the next component to run. Instead (from crypto), `run_component` creates a sufficiently large seedfile before each component.
Merge `unremove-non-crypto` into `mbedtls/development`. The branch
`unremove-non-crypto` was obtained by starting from `mbed-crypto/development`,
then reverting many commits that removed X.509 and TLS functionality when Mbed
Crypto forked from Mbed TLS (the “unremoval”), then make a few tweaks to
facilitate the merge.
The unremoval step restored old versions of some tls files. If a file doesn't
exist in mbed-crypto, check out the mbedtls version, regardless of what
happened during the unremoval of tls files in the crypto tree. Also
unconditionally take the mbedtls version of a few files where the
modifications are completely project-specific and are not relevant in
mbed-crypto:
* `.github/issue_template.md`: completely different. We may want to reconcile
them independently as a follow-up.
* `.travis.yml`: would only be reverted to an earlier tls version.
* `README.md`: completely different. We may want to reconcile them
independently as a follow-up.
* `doxygen/input/doc_mainpage.h`: the changes in crypto were minimal and not
relevant except as a stopgap as mbed-crypto did not have its own product
versioning in the Doxygen documentation.
* `tests/.jenkins/Jenkinsfile`: completely different.
* `tests/data_files/Makefile`: there were no changes in mbed-crypto,
but the unremoval step restored an old version.
Shell script for everything to do after the merge apart from the conflict
resolution:
```
tls_files=($(comm -23 <(git ls-tree -r --name-only HEAD) <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development MERGE_HEAD))))
tls_files+=($tls_files .github/issue_template.md .travis.yml README.md doxygen/input/doc_mainpage.h tests/.jenkins/Jenkinsfile tests/data_files/Makefile)
git checkout --theirs HEAD -- $tls_files
git add -- $tls_files
```
Resolve the remaining conflicts:
* `library/CMakeLists.txt`:
* Keep the TLS definition of `src_crypto`
* `USE_SHARED_MBEDTLS_LIBRARY`: keep all three libraries, with both
`include` and `crypto/include` in `target_include_directories`, all with
version `2.21.0`.
* `programs/Makefile`:
* Reconcile the APPS lists (add/add from a differently-formatted common
ancestor): insert the `psa/*` from crypto into the tls list.
* Keep the `fuzz` target defined only in tls version.
* Keep the recipe (only in tls version) cleaning `ssl_pthread_server`
stuff for the `clean` target.
* `scripts/config.py`:
* `include_in_full`: add/add conflict. Keep both.
* `tests/scripts/all.sh`:
* `component_test_no_use_psa_crypto_full_cmake_asan`: partially old
version in crypto. Take the tls version.
* `component_test_malloc_0_null` and more: take
`component_test_malloc_0_null` from crypto (with `config.py` rather than
`config.pl`, and with `$ASAN_FLAGS` rather than an explicit list), but
add the call to `ssl-opt.sh` from tls. Take the other components from
crypto.
With this commit, building and running the unit tests with both `make ` and
`cmake` work in the default configuration on Linux. Other platforms, build
systems and configurations are likely not to work, and there is some
regression in test coverage.
There is some loss of functionality because the unremoval step restored older
versions of tls content. This commit contains the latest tls version of
tls-only files, but some changes from the tls side in files that existed on
both sides have regressed. Most problematic changes are hunks that remove some
tls-specific feature and contain either a C preprocessor symbol identifying a
tls-specific module or option, or the name of a tls-specific file. Hunks
that remove a tls-specific preprocessor symbol can be identified with the
regular expression `^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_`.
Subsequent commits will revert a few parts of the patch from this merge commit
in order to restore the tls functionality that it removes, ensure that the
test coverage includes what was covered in either branch, and fix test
failures.
This reverts commit 9b90f2e294.
Conflicts:
* tests/scripts/all.sh: do the same changes, dancing around the new
outcome file feature and components added in the same places.
Make sure that the components that are getting added back are at the
same locations as where they are now in mbedtls.
This reverts commit 1c66e48670.
Conflicts:
* include/mbedtls/check_config.h:
* MBEDTLS_SSL_PROTO_SSL3: there has been an addition (of
MBEDTLS_SHA512_NO_SHA384) at the place where it was removed. Re-add it
after (alphabetical order).
* MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
Re-add it after (alphabetical order).
* MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
after (alphabetical order).
* include/mbedtls/config.h:
* MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
Re-add it after (alphabetical order).
* MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
after (alphabetical order).
* library/version_features.c: re-generate by running
scripts/generate_features.pl.
* programs/test/query_config.c: re-generate by running
scripts/generate_query_config.pl.
* scripts/config.pl: this file has been replaced by config.py. Port
the reversed changes to config.py:
* Revert removing three symbols from the list of symbols to
exclude from full.
* Revert removing one symbol (MBEDTLS_NET_C) from the list of symbols
to exclude from baremetal.
* scripts/footprint.sh:
* Re-add the line to unset MBEDTLS_NET_C, but with config.py instead of
config.pl.
* tests/scripts/all.sh:
* component_test_no_platform: re-add the line to unset MBEDTLS_NET_C, but
with config.py instead of config.pl.
* component_build_arm_none_eabi_gcc,
component_build_arm_none_eabi_gcc_no_udbl_division,
component_build_arm_none_eabi_gcc_no_64bit_multiplication,
component_build_armcc: these components now use the baremetal
configuration, so they do not need to turn off MBEDTLS_NET_C explicitly.
Pylint when installed as a distro package can be installed as pylint3, whilst as
a PEP egg, it can be installed as pylint.
This commit changes the scripts to first use pylint if installed, and optionally
look for pylint3 if not installed. This is to allow a preference for the PEP
version over the distro version, assuming the PEP one is more likely to be
the correct one.
Signed-off-by: Simon Butcher <simon.butcher@arm.com>
The .pl version is now a compat wrapper around the .py script. Better call the
.py script directly.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Exercise the feature alone, with record splitting and DTLS connection ID.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Signed-off-by: Darryl Green <darryl.green@arm.com>
Exercise the library functions with calloc returning NULL for a size
of 0. Make this a separate job with UBSan (and ASan) to detect
places where we try to dereference the result of calloc(0) or to do
things like
buf = calloc(size, 1);
if (buf == NULL && size != 0) return INSUFFICIENT_MEMORY;
memcpy(buf, source, size);
which has undefined behavior when buf is NULL at the memcpy call even
if size is 0.
This is needed because other test components jobs either use the system
malloc which returns non-NULL on Linux and FreeBSD, or the
memory_buffer_alloc malloc which returns NULL but does not give as
useful feedback with ASan (because the whole heap is a single C
object).
Goals:
* Build with common compilers with common options, so that we don't
miss a (potentially useful) warning only triggered with certain
build options.
* A previous commit removed -O0 test jobs, leaving only the one with
-m32. We have inline assembly that is disabled with -O0, falling
back to generic C code. This commit restores a test that runs the
generic C code on a 64-bit platform.
For each of the crypto-only presets, run the build and check that the
resulting libmbedx509 and libmbedtls are empty.
Don't bother testing, because for each crypto-only preset, another
component builds that plus the x509 and tls parts and tests
everything.
We already have a specific component in all.sh for testing SSLv3, we don't
need to also test it in components that aren't specifically about it.
Previously config.py full enabled SSLv3, but it no longer does since it is
deprecated.
When MBEDTLS_TEST_DEPRECATED is defined, run some additional tests to
validate deprecated PSA macros. We don't need to test deprecated
features extensively, but we should at least ensure that they don't
break the build.
Add some code to component_build_deprecated in all.sh to run these
tests with MBEDTLS_DEPRECATED_WARNING enabled. The tests are also
executed when MBEDTLS_DEPRECATED_WARNING and
MBEDTLS_DEPRECATED_REMOVED are both disabled.
The size of the seedfile used by the entropy module when
MBEDTLS_ENTROPY_NV_SEED is enabled is 32 byte when
MBEDTLS_ENTROPY_FORCE_SHA256 is enabled or MBEDTLS_SHA512_C is
disabled, and 64 bytes otherwise. A larger seedfile is ok on
entry (the code just grabs the first N bytes), but a smaller seedfile
is not ok. Therefore, if you run a component with a 32-byte seedfile
and then a component with a 64-byte seedfile, the second component
fails in the unit tests (up to test_suite_entropy which erases the
seedfile and creates a fresh one).
This is ok up to now because we only enable MBEDTLS_ENTROPY_NV_SEED
together with MBEDTLS_ENTROPY_FORCE_SHA256. But it prevents enabling
MBEDTLS_ENTROPY_NV_SEED without MBEDTLS_ENTROPY_FORCE_SHA256.
To fix this, unconditionally create a seedfile before each component.
Enabling MBEDTLS_MEMORY_BUFFER_ALLOC_C module together with
MBEDTLS_PLATFORM_MEMORY causes the library to use its own malloc
replacement. This makes memory management analyzers such as ASan
largely ineffective. We now test MBEDTLS_MEMORY_BUFFER_ALLOC_C
separately. Disable it in the "full" config.
This mirrors a change that was made in Mbed TLS on config.pl and had
not been ported to Mbed Crypto yet.
With this commit, config.py is aligned in Mbed Crypto and Mbed TLS.
First deal with deleted files.
* Files deleted by us: keep them deleted.
* Files deleted by them, whether modified by us or not: keep our version.
```
git rm $(git status -s | sed -n 's/^DU //p')
git reset -- $(git status -s | sed -n 's/^D //p')
git checkout -- $(git status -s | sed -n 's/^ D //p')
git add -- $(git status -s | sed -n 's/^UD //p')
```
Individual files with conflicts:
* `3rdparty/everest/library/Hacl_Curve25519_joined.c`: spurious conflict because git mistakenly identified this file as a rename. Keep our version.
* `README.md`: conflict due to their change in a paragraph that doesn't exist in our version. Keep our version of this paragraph.
* `docs/architecture/Makefile`: near-identical additions. Adapt the definition of `all_markdown` and include the clean target.
* `doxygen/input/docs_mainpage.h`: conflict in the version number. Keep our version number.
* `include/mbedtls/config.h`: two delete/modify conflicts. Keep the removed chunks out.
* `library/CMakeLists.txt`: discard all their changes as they are not relevant.
* `library/Makefile`:
* Discard the added chunk about the crypto submodule starting with `INCLUDING_FROM_MBEDTLS:=1`.
* delete/modify: keep the removed chunk out.
* library build: This is almost delete/modify. Their changes are mostly not applicable. Do keep the `libmbedcrypto.$(DLEXT): | libmbedcrypto.a` order dependency.
* `.c.o`: `-o` was added on both sides but in a different place. Change to their place.
* `library/error.c`: to be regenerated.
* `library/version_features.c`: to be regenerated.
* `programs/Makefile`: Most of the changes are not relevant. The one relevant change is in the `clean` target for Windows; adapt it by removing `/S` from our version.
* `programs/test/query_config.c`: to be regenerated.
* `scripts/config.py`: added in parallel on both sides. Keep our version.
* `scripts/footprint.sh`: parallel changes. Keep our version.
* `scripts/generate_visualc_files.pl`: one delete/modify conflict. Keep the removed chunks out.
* `tests/Makefile`: discard all of their changes.
* `tests/scripts/all.sh`:
* `pre_initialize_variables` add `append_outcome`: add it.
* `pre_initialize_variables` add `ASAN_CFLAGS`: already there, keep our version.
* `pre_parse_command_line` add `--no-append-outcome`: add it.
* `pre_parse_command_line` add `--outcome-file`: add it.
* `pre_print_configuration`: add `MBEDTLS_TEST_OUTCOME_FILE`.
* Several changes in SSL-specific components: keep our version without them.
* Several changes where `config.pl` was changed to `config.py` and there was an adjacent difference: keep our version.
* Changes regarding the inclusion of `MBEDTLS_MEMORY_xxx`: ignore them here, they will be normalized in a subsequent commit.
* `component_test_full_cmake_gcc_asan`: add it without the TLS tests.
* `component_test_no_use_psa_crypto_full_cmake_asan`: keep the fixed `msg`, discard other changes.
* `component_test_memory_buffer_allocator_backtrace`, `component_test_memory_buffer_allocator`: add them without the TLS tests.
* `component_test_m32_everest`: added in parallel on both sides. Keep our version.
* `tests/scripts/check-names.sh`, `tests/scripts/list-enum-consts.pl`, `tests/scripts/list-identifiers.sh`, ``tests/scripts/list-macros.sh`: discard all of their changes.
* `tests/scripts/test-ref-configs.pl`: the change in the conflict is not relevant, so keep our version there.
* `visualc/VS2010/*.vcxproj`: to be regenerated.
Regenerate files:
```
scripts/generate_visualc_files.pl
git add visualc/VS2010/*.vcxproj
scripts/generate_errors.pl
git add library/error.c
scripts/generate_features.pl
git add library/version_features.c
scripts/generate_query_config.pl
git add programs/test/query_config.c
```
Rejected changes in non-conflicting files:
* `CMakeLists.txt`: discard their addition which has already been side-ported.
* `doxygen/mbedtls.doxyfile`: keep the version number change. Discard the changes related to `../crypto` paths.
Keep the following changes after examination:
* `.travis.yml`: all of their changes are relevant.
* `include/mbedtls/error.h`: do keep their changes. Even though Crypto doesn't use TLS errors, it must not encroach on TLS's allocated numbers.
* `tests/scripts/check-test-cases.py`: keep the code dealing with `ssl-opt.sh`. It works correctly when the file is not present.
git grep -Fl /config.pl | xargs sed -i -e 's!/config\.pl!/config.py!g'
Also:
* Change one comment in include/mbedtls/check_config.h.
* Change PERL to PYTHON in CMakeLists.txt.
MBEDTLS_PK_SIGNATURE_MAX_SIZE is tested in Mbed Crypto. Its effect on
Mbed TLS is also tested via the X.509 tests. The case of
MBEDTLS_MPI_MAX_SIZE < MBEDTLS_ECDSA_MAX_LEN, for which this component
was added as a regression test, is covered by config-suite-b.h which
is tested via test-ref-configs.pl.
Enabling memory_buffer_alloc is slow and makes ASan ineffective. We
have a patch pending to remove it from the full config. In the
meantime, disable it explicitly.
Some sanitizers default to displaying an error message and recovering.
This could result in a test being recorded as passing despite a
complaint from the sanitizer. Turn off sanitizer recovery to avoid
this risk.
Some sanitizers default to displaying an error message and recovering.
This could result in a test being recorded as passing despite a
complaint from the sanitizer. Turn off sanitizer recovery to avoid
this risk.
