Ensure the unique part fits in the 66 columns that the test runner displays.
Leave room for an additional distinguisher on signature key policy negative
test cases.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The test cases strictly replicate a subset of the test cases for
PSA_ALG_RSA_PSS. The subset validates that PSA_ALG_RSA_PSS_ANY_SALT is
recognized wherever PSA_ALG_RSA_PSS is.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This is a variant of PSA_ALG_RSA_PSS which currently has exactly the same
behavior, but is intended to have a different behavior when verifying
signatures.
In a subsequent commit, PSA_ALG_RSA_PSS will change to requiring the salt
length to be what it would produce when signing, as is currently documented,
whereas PSA_ALG_RSA_PSS_ANY_SALT will retain the current behavior of
allowing any salt length (including 0).
Changes in this commit:
* New algorithm constructor PSA_ALG_RSA_PSS_ANY_SALT.
* New predicates PSA_ALG_IS_RSA_PSS_STANDARD_SALT (corresponding to
PSA_ALG_RSA_PSS) and PSA_ALG_IS_RSA_PSS_ANY_SALT (corresponding to
PSA_ALG_RSA_PSS_ANY_SALT).
* Support for the new predicates in macro_collector.py (needed for
generate_psa_constant_names).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use case pattern matching instead of multiline split, given there is
only the well formatted PIDs to match on this should be safe.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
On machines with more modern kernels (>5.4 from testing so far) the
useage of -b seems to conflict with the usage of -p. Whilst the usage of
-b seems like a good idea to avoid blocks as we are tight looping on it,
the usage of -p seems to require the usage of stat() (specifically in
/proc) which -b forbids. All you get is a load of warnings
(suppressable by -w) but never a positive result, which means that all
servers are reported as "Failed to start". We are not keen on losing
-b, so instead parse the output of lsof (using -F to format it) to
check the if PIDs that it outputs match that we are looking for.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
digits is also a local variable in host_test.function, leading to compilers
complaining about that shadowing the global variable in
test_suite_base64.function.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This is part of the definition of the encoding, not a choice of test
parameter, so keep it with the test code.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Add unit tests for mask_of_range(), enc_char() and dec_value().
When constant-flow testing is enabled, verify that these functions are
constant-flow.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When TEST_EQUAL fails, show the two numerical values in the test log (only
with host_test). The values are printed in hexa and signed decimal.
The arguments of TEST_EQUAL must now be integers, not pointers or floats.
The current implementation requires them to fit in unsigned long long
Signed values no larger than long long will work too. The implementation
uses unsigned long long rather than uintmax_t to reduce portability
concerns. The snprintf function must support "%llx" and "%lld".
For this purpose, add room for two lines of text to the mbedtls_test_info
structure. This adds 154 bytes of global data.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Improve the code structure in case we want to add other similar conditions
later. Document better what we're doing, and document why we're doing it.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Palliative for https://github.com/ARMmbed/mbedtls/issues/3377. If a test
case fails due to an unexpected resend, allow retrying, like in the case of
a client timeout.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This was causing some tests using the openssl s_client to not connect -
I suspect this was due to localhost (at least on my machine) resolving
to ::1 rather than 127.0.0.1. Note that the error seen would have been
that the session file specified with -sess_out did not get created.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
Missing wildcards meant that some servers were not identified as DTLS,
which lead to port checking on TCP rather than UDP, and thus mistakenly
cancelling tests as the server had not come up.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
Otherwise $COMMAND_LINE_COMPONENTS would try to expand wildcard patterns
based on files in the current directory.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Have simpler patterns related to 'test' (the central objective being to keep
going if 'make test' or 'tests/...' fails, but not if 'make tests' fails).
Add 'cd' as a can't-keep-going command.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Count invocations from 1 to n instead of n to 1.
Explain how changing the loop variable would cause an error if the function
was not executed in a subshell.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
all.sh restores **/Makefile from git in case the version in the worktree was
from doing a cmake in-tree build. Instead of doing this unconditionally, do
it only if the toplevel Makefile seems to have been automatically
generated (by cmake or otherwise, e.g. by mbedtls-prepare-build). This way
all.sh no longer silently wipes changes made to Makefile but not committed yet.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Back up the config files at the beginning of all.sh, rather than before each
component. In particular, create the backup before running cleanup for the
first time. This fixes#3139 (all.sh using a config.h.bak from a previous
job), and makes all.sh more robust against accidentally using a modified
config.h midway through because a component messed with the backup.
Use a different extension (*.all.bak rather than *.bak) for the backups.
This is necessary to ensure that auxiliary scripts such as depends*.pl that
make their own backup don't remove all.sh's backup, which the code from this
commit does not support.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Generate programs/test/cpp_dummy_build.cpp dynamically instead of
maintaining it manually. This removes the need to update it when the list of
headers changes.
Include all the headers unconditionally except for the ones that cannot be
included directly.
Support this dynamic generation both with make and with cmake.
Adapt all.sh accordingly. Remove the redundant C build from
component_build_default_make_gcc_and_cxx (it was also done in
component_test_default_out_of_box), leaving a component_test_make_cxx. Also
run the C++ program, because why not. Do this in the full configuration
which may catch a bit more problems in headers.
Fixes#2570 for good.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Signed-off-by: Thomas Daubney <>
Use the encoding from an upcoming version of the specification.
Add as much (or as little) testing as is currently present for Camellia.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Don't default to unbridled -j, which causes a load spike and isn't really
faster.
"Number of CPUs" is implemented here as a reasonable compromise between
portability, correctness and simplicity. This is just a default that can be
overridden by setting MAKEFLAGS in the environment.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Declare all AES and DES functions that return int as needing to have
their result checked, and do check the result in our code.
A DES or AES block operation can fail in alternative implementations of
mbedtls_internal_aes_encrypt() (under MBEDTLS_AES_ENCRYPT_ALT),
mbedtls_internal_aes_decrypt() (under MBEDTLS_AES_DECRYPT_ALT),
mbedtls_des_crypt_ecb() (under MBEDTLS_DES_CRYPT_ECB_ALT),
mbedtls_des3_crypt_ecb() (under MBEDTLS_DES3_CRYPT_ECB_ALT).
A failure can happen if the accelerator peripheral is in a bad state.
Several block modes were not catching the error.
This commit does the following code changes, grouped together to avoid
having an intermediate commit where the build fails:
* Add MBEDTLS_CHECK_RETURN to all functions returning int in aes.h and des.h.
* Fix all places where this causes a GCC warning, indicating that our code
was not properly checking the result of an AES operation:
* In library code: on failure, goto exit and return ret.
* In pkey programs: goto exit.
* In the benchmark program: exit (not ideal since there's no error
message, but it's what the code currently does for failures).
* In test code: TEST_ASSERT.
* Changelog entry.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
redirect_out was no longer used and redirect_err was only used to
quiet dd. Change the dd invocation to only print diagnostics on
error (on platforms where this is possible).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This is necessary to actually keep going and finish the
component-specific cleanup in component_test_cmake_out_of_source if
ssl-opt.err is non-empty.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In all.sh, when an explicit list of components is specified, error out
if one of the components is not known or not supported. Patterns that
happen to match zero components are still effectively ignored.
Fix#2783
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In the `not` function, in keep-going mode, arrange to report the
failing command (rather than `"$@"`).
Note that the `!` keyword should not be used, because failures with
`!` are not reported properly.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Remove the obsolete functions record_status and if_build_succeeded.
They didn't affect error detection, but they made error reporting
worse since $BASH_COMMAND would be the unexpanded "$@".
Keep the function definitions for the sake of pull requests using them
that may still be in flight.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Remove ssl-opt.err even if it's empty.
Call cat unconditionally: it'll have no visible effect if the file is
empty.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This commit completely rewrites keep-going mode. Instead of relying
solely on "set -e", which has some subtle limitations (such as being
off anywhere inside a conditional), use an ERR trap to record errors.
Run each component in a subshell. This way a component can set
environment variables, change the current directory, etc., without
affecting other components.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This will let us use bash features that are not found in some other sh
implementations, such as DEBUG and ERR traps, "set -o pipefail", etc.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
* There's no compat-2.x.h in Mbed TLS 2.x, but there's a compat-1.3.h which
similarly needs to be excluded.
* mbedtls_config.h is called config.h.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Don't try to enumerate excluded files. List included files, and remove names
from the list if they match an excluded-file pattern.
This resolves the problem that the script could get into an infinite loop
due to the use of recursive globbing. Unfortunately, Python's recursive
globs follows symbolic links to directories, which leads to an infinite loop
if a symbolic link points to an ancestor of the directory that contains it.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Copy the following files:
tests/scripts/check_names.py
tests/scripts/list_internal_identifiers.py
tests/scripts/list-identifiers.sh
Copy the version from b19be6b5f3c9a92b5b17fa27e16901f132f1a310, which is the
result of merging https://github.com/ARMmbed/mbedtls/pull/1638 into the
development branch.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
To facilitate maintenance and to make it easier to reproduce all.sh builds
manually, remove the long, repeated list of -D options from
component_test_psa_crypto_config_basic and component_test_psa_crypto_drivers
and put it in a header file instead.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When building with make, `make test` runs `run-test-suites.pl` which has a
verbose mode that reports the failing test cases, but it didn't provide a
way to enable this verbose mode. With the present commit, you can run `make
test TEST_FLAGS=-v` to use verbose mode.
Base the default for verbose mode on the same environment variable that
`make test` uses when building with CMake: default off, but enabled if
`CTEST_OUTPUT_ON_FAILURE` is true. In particular, verbose mode will now be
on when building from `all.sh`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
It was unmaintained and untested, and the fear of breaking it was holding us
back. Resolves#4934.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The psa_open_key API depends on MBEDTLS_PSA_CRYPTO_STORAGE_C.
This is unnecessary for builtin keys and so is fixed.
Updated an open_fail test vector keeping with the same.
Signed-off-by: Archana <archana.madhavan@silabs.com>
Was getting errors like:
In file included from /usr/include/limits.h:25:0,
from /usr/lib/gcc-cross/arm-linux-gnueabi/5/include-fixed/limits.h:168,
from /usr/lib/gcc-cross/arm-linux-gnueabi/5/include-fixed/syslimits.h:7,
from /usr/lib/gcc-cross/arm-linux-gnueabi/5/include-fixed/limits.h:34,
from ../include/mbedtls/check_config.h:30,
from ../include/mbedtls/build_info.h:81,
from common.h:26,
from asn1write.c:20:
/usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or directory
There are two packages to choose from: armhf or armel. Since the comment
in all.sh says we're trying to be close to Debian's "armel"
architecture, choose that, and fix a comment that was mentioning
gnueabihf for no apparent reason.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Currently it can't be mandatory, since we can't install the required toolchain
on Jenkins right away.
Also, while at it, remove `SHELL='sh -x'` from the other arm5vte component; it
was a leftover from debugging.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Call the output size macros specifically with asymmetric keys, which
would cause a crash (and thus test fail) should this fix get regressed.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
The previous implementation was hard to understand and could in principle
fail to notice if there was a test case failure and the writing of the
line "Note: $TOTAL_FAIL failures." failed. KISS.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Save the "Test Report Summary" to a file. This can help both CI scripts and
human readers who want the summary after the fact without having to copy the
console output.
Take care to exit with a nonzero status if there is a failure while
generating the test report summary.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The makefiles look for python3 on Unix-like systems where python is often
Python 2. This uses sh code so it doesn't work on Windows. On Windows, the
makefiles just assume that python is Python 3.
The code was incorrectly deciding not to try python3 based on WINDOWS_BUILD,
which indicates that the build is *for* Windows. Switch to checking WINDOWS,
which indicates that the build is *on* Windows.
Fix#4774
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This file had temporary MBEDTLS_xxx dependencies because it was created when
support for PSA_WANT_xxx was still incomplete. Switch to the PSA_WANT_xxx
dependencies
This fixes the bug that "PSA storage read: AES-GCM+CTR" was never executed
because there was a typo in a dependency.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Previously the check was convoluted. This has been simplified
and given a more appropriate suggestion as per gilles suggestion
Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
- Reword the comment on config.h to suggest that
`MAX_INTERMEDIATE_CA` may not exist in the config.
- Add a comment explaining why the tests are more restrictive
than necessary.
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
- Abstract out repetitive checks for IN and OUT content lens
- Remove unclear comment and redundant echo
- Add content length constraints in Renegotiation with fragment length test
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
- Replace last remaining dependency on config.py with query_config
- Replace hard exit with `requires_config_value_at_least` and
`requires_config_value_at_most` to maintain the same effect
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
- Replace calls to config.py for MAX_IN_LEN and MAX_OUT_LEN with
`get_config_value_or_default`
- Remove hard exit when MAX_IN/OUT_LEN < 4096, replace with
`requires_config_value_at_least`
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
Restore the optimization done in
HEAD^{/Speed up the generation of storage format test cases}
which was lost during refactoring made when adding support for
implicit usage flags.
There are still more than one call to the C compiler, but the extra
calls are only for some key usage test cases.
This is an internal refactoring. This commit does not change the
output of generate_psa_tests.py
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In order to for tests to pass from the previous commit (which it mandatory for all pk verify/sign
functions to be given a hash_len that is exactly equal to the message digest length of md_alg) the
hash_len that is supplied to the fucntion cannot be MBEDTLS_MD_MAX_SIZE. This would result in all tests failing. Since the md alg for all of these funtions are SHA256, we can use mbedtls functions to get
the required length of a SHA256 digest (32 bytes). Then that number can be used for allocating the
hash buffer.
Signed-off-by: Nick Child <nick.child@ibm.com>
It is enough only one test case for a key type, algorithm pair when
testing the implicit usage flags.
Signed-off-by: gabor-mezei-arm <gabor.mezei@arm.com>
Add test cases validating that if a stored key only had the hash policy,
then after loading it psa_get_key_attributes reports that it also has the
message policy, and the key can be used with message functions.
Signed-off-by: gabor-mezei-arm <gabor.mezei@arm.com>
