Commit Graph

15985 Commits

Author SHA1 Message Date
Gilles Peskine
8bfe15f291 Disable wildcards when checking for unsupported components
Otherwise $COMMAND_LINE_COMPONENTS would try to expand wildcard patterns
based on files in the current directory.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-10-11 14:29:16 +01:00
Gilles Peskine
76d40fa977 Improve the detection of keep-going commands
Have simpler patterns related to 'test' (the central objective being to keep
going if 'make test' or 'tests/...' fails, but not if 'make tests' fails).

Add 'cd' as a can't-keep-going command.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-10-11 14:29:16 +01:00
Gilles Peskine
202e9b4ed4 Improve --error-test reporting
Count invocations from 1 to n instead of n to 1.

Explain how changing the loop variable would cause an error if the function
was not executed in a subshell.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-10-11 14:29:16 +01:00
Gilles Peskine
cb4bfac119 Clarify some comments
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-10-11 14:29:16 +01:00
Gilles Peskine
ef64e7fb3b Don't unconditionally restore **/Makefile
all.sh restores **/Makefile from git in case the version in the worktree was
from doing a cmake in-tree build. Instead of doing this unconditionally, do
it only if the toplevel Makefile seems to have been automatically
generated (by cmake or otherwise, e.g. by mbedtls-prepare-build). This way
all.sh no longer silently wipes changes made to Makefile but not committed yet.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-10-11 14:29:16 +01:00
Gilles Peskine
24bdf02e4e Don't restore *config.h before backing it up
Back up the config files at the beginning of all.sh, rather than before each
component. In particular, create the backup before running cleanup for the
first time. This fixes #3139 (all.sh using a config.h.bak from a previous
job), and makes all.sh more robust against accidentally using a modified
config.h midway through because a component messed with the backup.

Use a different extension (*.all.bak rather than *.bak) for the backups.
This is necessary to ensure that auxiliary scripts such as depends*.pl that
make their own backup don't remove all.sh's backup, which the code from this
commit does not support.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-10-11 14:29:16 +01:00
Gilles Peskine
9b35640e3f Generate cpp_dummy_build.cpp dynamically
Generate programs/test/cpp_dummy_build.cpp dynamically instead of
maintaining it manually. This removes the need to update it when the list of
headers changes.

Include all the headers unconditionally except for the ones that cannot be
included directly.

Support this dynamic generation both with make and with cmake.

Adapt all.sh accordingly. Remove the redundant C build from
component_build_default_make_gcc_and_cxx (it was also done in
component_test_default_out_of_box), leaving a component_test_make_cxx. Also
run the C++ program, because why not. Do this in the full configuration
which may catch a bit more problems in headers.

Fixes #2570 for good.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Signed-off-by: Thomas Daubney <>
2021-10-11 14:29:16 +01:00
Gilles Peskine
f313336c01 Remove barely-used redirect functions
redirect_out was no longer used and redirect_err was only used to
quiet dd. Change the dd invocation to only print diagnostics on
error (on platforms where this is possible).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:59:00 +01:00
Gilles Peskine
7c220d7d37 Keep going after a shell "[" a.k.a. "test" fails
This is necessary to actually keep going and finish the
component-specific cleanup in component_test_cmake_out_of_source if
ssl-opt.err is non-empty.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:59:00 +01:00
Gilles Peskine
1d55995d32 Remove code that is useless now that components run in a subshell
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:59:00 +01:00
Gilles Peskine
e8056c5e31 Complain if an unsupported component is explicitly requested
In all.sh, when an explicit list of components is specified, error out
if one of the components is not known or not supported. Patterns that
happen to match zero components are still effectively ignored.

Fix #2783

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:59:00 +01:00
Gilles Peskine
39a3b11006 Better not function
In the `not` function, in keep-going mode, arrange to report the
failing command (rather than `"$@"`).

Note that the `!` keyword should not be used, because failures with
`!` are not reported properly.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:59:00 +01:00
Gilles Peskine
4848d7bb45 Fix double reporting when the last command of a function fails
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:59:00 +01:00
Gilles Peskine
f6d29c6a9e Stop dispatching through obsolete functions
Remove the obsolete functions record_status and if_build_succeeded.
They didn't affect error detection, but they made error reporting
worse since $BASH_COMMAND would be the unexpanded "$@".

Keep the function definitions for the sake of pull requests using them
that may still be in flight.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:58:55 +01:00
Gilles Peskine
0a7984f1dd component_test_cmake_out_of_source: simplify and fix error handling
Remove ssl-opt.err even if it's empty.

Call cat unconditionally: it'll have no visible effect if the file is
empty.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:55:21 +01:00
Gilles Peskine
8ab2994eb5 Detect errors on the left-hand side of a pipeline
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:55:21 +01:00
Gilles Peskine
7105a33906 Run each component in a subshell and handle errors more robustly
This commit completely rewrites keep-going mode. Instead of relying
solely on "set -e", which has some subtle limitations (such as being
off anywhere inside a conditional), use an ERR trap to record errors.

Run each component in a subshell. This way a component can set
environment variables, change the current directory, etc., without
affecting other components.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:55:21 +01:00
Gilles Peskine
a5eb22d434 Add --error-test option to test error detection and reporting
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:55:21 +01:00
Gilles Peskine
3de7be8b88 Switch all.sh to bash
This will let us use bash features that are not found in some other sh
implementations, such as DEBUG and ERR traps, "set -o pipefail", etc.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-29 14:55:21 +01:00
Gilles Peskine
27bb62bc81
Merge pull request #4985 from gilles-peskine-arm/check-names-rewrite-2.2x
Backport 2.x: Rewrite check-names.sh in python
2021-09-29 12:43:09 +02:00
Gilles Peskine
1c39975a27 Fix typo
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-28 10:13:45 +02:00
Gilles Peskine
ee20f3698a Remove check-names.sh and now-unused helper scripts
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-27 20:14:12 +02:00
Gilles Peskine
31da67beb7 Switch to the new Python implementation of check-names
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-27 20:13:00 +02:00
Gilles Peskine
d47f636a19 Adapt source file names from Mbed TLS 3.0 to 2.27
* There's no compat-2.x.h in Mbed TLS 2.x, but there's a compat-1.3.h which
  similarly needs to be excluded.
* mbedtls_config.h is called config.h.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-27 20:12:00 +02:00
Yuto Takano
5b4caf21de Fix typos pointed out by check_names
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-09-27 20:08:01 +02:00
Gilles Peskine
7bf5205581 More robust handling of excluded files
Don't try to enumerate excluded files. List included files, and remove names
from the list if they match an excluded-file pattern.

This resolves the problem that the script could get into an infinite loop
due to the use of recursive globbing. Unfortunately, Python's recursive
globs follows symbolic links to directories, which leads to an infinite loop
if a symbolic link points to an ancestor of the directory that contains it.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-27 20:01:24 +02:00
Gilles Peskine
8266b5b0b4 Copy check_names.py and friends from development
Copy the following files:
    tests/scripts/check_names.py
    tests/scripts/list_internal_identifiers.py
    tests/scripts/list-identifiers.sh

Copy the version from b19be6b5f3c9a92b5b17fa27e16901f132f1a310, which is the
result of merging https://github.com/ARMmbed/mbedtls/pull/1638 into the
development branch.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-27 20:01:08 +02:00
Gilles Peskine
186c0216b0
Merge pull request #4978 from davidhorstmann-arm/2.x-fix-aarch64-asm-constraints
Backport 2.x: Fix aarch64 assembly for bignum multiplication
2021-09-27 09:01:12 +02:00
Gilles Peskine
97cd76988d
Merge pull request #4979 from gilles-peskine-arm/doc-use-psa-crypto-2.x
Backport 2.x: Document effects of MBEDTLS_USE_PSA_CRYPTO
2021-09-25 09:25:36 +02:00
Manuel Pégourié-Gonnard
fec7ef8270 Mention areas that are not (well) tested.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-09-24 20:44:03 +02:00
Manuel Pégourié-Gonnard
ee20baf6e1 Clarify that 1.3 is excluded
Don't mention "TLS 1.2 only" for PSK, as that could give the impression
that the other things about TLS are supported beyond 1.2, which isn't
the case currently.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-09-24 20:44:03 +02:00
Manuel Pégourié-Gonnard
12ab49aaf7 Improve wording and fix some typos.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-09-24 20:44:03 +02:00
Manuel Pégourié-Gonnard
b4113bac9a Clarify wording of "not covered" section
The section is about things that are not covered, but some lists are
about things that are covered, which was very confusing.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-09-24 20:44:03 +02:00
Manuel Pégourié-Gonnard
1fa923a5bc Fix inaccuracy in key exchange summary
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-09-24 20:44:03 +02:00
Manuel Pégourié-Gonnard
6cf7d94ab4 Document parts not covered by USE_PSA_CRYPTO
Also, remove the section about design considerations for now. It's
probably more suitable for a developer-oriented document that would also
include considerations about possible paths for the future, which would
better be separated from user documentation (separating the certain that
is now, from the uncertain that might or might not be later).

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-09-24 20:44:03 +02:00
Manuel Pégourié-Gonnard
b52b91d949 Remove warning about PSA Crypto being beta
The API reached 1.0.0 some time ago, and we've caught up with the
incompatible changes already.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-09-24 20:44:03 +02:00
Manuel Pégourié-Gonnard
04bc6875a0 Document current effects of USE_PSA_CRYPTO
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-09-24 20:44:03 +02:00
Manuel Pégourié-Gonnard
00b72fc35f Add docs/use-psa-crypto.md
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-09-24 20:44:03 +02:00
David Horstmann
27d8b5c680 Combine changelog entries for muladdc assembly fix
Combine the changelog entries for the memory constraints fix on
aarch64 and amd64, since these are essentially fixing the same
issue.

Signed-off-by: David Horstmann <david.horstmann@arm.com>
2021-09-24 16:11:15 +01:00
Gilles Peskine
7358065203
Merge pull request #4970 from jclab-joseph/pr/fix/build-alpine_2.x
Backport 2.x: Fix test code to can be built on alpine #4969
2021-09-24 15:04:09 +02:00
David Horstmann
a23be22308 Fix aarch64 assembly for bignum multiplication
Add memory constraints to the aarch64 inline assembly in MULADDC_STOP.
This fixes an issue where Clang 12 and 13 were generating
non-functional code on aarch64 platforms. See #4962, #4943
for further details.

Signed-off-by: David Horstmann <david.horstmann@arm.com>
2021-09-24 09:47:01 +01:00
joseph
00f4eae025 Fix test code to can be built on alpine
Signed-off-by: joseph <joseph@jc-lab.net>
2021-09-23 20:58:45 +09:00
Gilles Peskine
02e17c0aa5
Merge pull request #4941 from gilles-peskine-arm/muladdc-amd64-memory-2.x
Backport 2.x: Fix x86_64 assembly for bignum multiplication
2021-09-20 22:23:53 +02:00
Gilles Peskine
3c0f304848
Merge pull request #4954 from SiliconLabs/backport_4878
Backport 2.x: Remove dependency of built-in keys on storage
2021-09-20 22:20:19 +02:00
Gilles Peskine
184a688d51 Update the list of issues fixed
This had actually been reported multiple times.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-15 17:07:09 +02:00
Gilles Peskine
c6d977eae5 x86_64 MULADDC assembly: add missing constraints about memory
MULADDC_CORE reads from (%%rsi) and writes to (%%rdi). This fragment is
repeated up to 16 times, and %%rsi and %%rdi are s and d on entry
respectively. Hence the complete asm statement reads 16 64-bit words
from memory starting at s, and writes 16 64-bit words starting at d.

Without any declaration of modified memory, Clang 12 and Clang 13 generated
non-working code for mbedtls_mpi_mod_exp. The constraints make the unit
tests pass with Clang 12.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-15 15:47:37 +02:00
Gilles Peskine
394b9f2d2c
Merge pull request #4898 from mstarzyk-mobica/disable_defaults_sha1
Remove MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_CERTIFICATES in 2.28
2021-09-14 11:10:30 +02:00
Janos Follath
2079aa7838
Merge pull request #4939 from gilles-peskine-arm/psa_cipher_update_ecp-unused_parameter-2.x
Backport 2.x: Fix parameter set but unused on psa_cipher_update_ecb
2021-09-13 13:55:16 +01:00
Gilles Peskine
0390016096 Fix the size in bytes
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-13 12:54:46 +02:00
Gilles Peskine
a63ba6cd93 psa_cipher_update_ecb: remove parameter output_size
This parameter was set but not used, which was pointless. Clang 14 detects
this and legitimately complains.

Remove the parameter. This is an internal function, only called once. The
caller already has a sufficient check on the output buffer size which
applies in more cases, so there is no real gain in robustness in adding the
same check inside the internal function.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-09-13 12:54:46 +02:00