* origin/development: (36 commits)
Do not build fuzz on windows
No booleans and import config
Removing space before opening parenthesis
Style corrections
Syntax fix
Fixes warnings from MSVC
Add a linker flag to enable gcov in basic-build-test.sh
tests: Limit each log to 10 GiB
Fix parsing issue when int parameter is in base 16
checks MBEDTLS_PEM_PARSE_C
Restore programs/fuzz/Makefile after in-tree cmake
Move fuzz directory to programs
Documentation for corpus generation
Restore tests/fuzz/Makefile after in-tree cmake
Adding ifdefs to avoid warnings for unused globals
Adds LDFLAGS fsanitize=address
Refactor receive_uint32()
Refactor get_byte function
Make the script portable to both pythons
Update the test encoding to support python3
...
* origin/pr/1622: (29 commits)
Do not build fuzz on windows
No booleans and import config
Removing space before opening parenthesis
Style corrections
Syntax fix
Fixes warnings from MSVC
Add a linker flag to enable gcov in basic-build-test.sh
checks MBEDTLS_PEM_PARSE_C
Restore programs/fuzz/Makefile after in-tree cmake
Move fuzz directory to programs
Documentation for corpus generation
Restore tests/fuzz/Makefile after in-tree cmake
Adding ifdefs to avoid warnings for unused globals
Adds LDFLAGS fsanitize=address
Ignore compiled object files and executables
Also clean the fuzz subdirectory
copyediting README.md
Protecting client/server fuzz targts with ifdefs
Makefile support 1
Fuzz README and direct compilation
...
* origin/development: (33 commits)
Test with MBEDTLS_ECP_RESTARTABLE
Allow TODO in code
Use the docstring in the command line help
Split _abi_compliance_command into smaller functions
Record the commits that were compared
Document how to build the typical argument for -s
Allow running /somewhere/else/path/to/abi_check.py
Warn if VLAs are used
Remove redundant compiler flag
Consistently spell -Wextra
Update Mbed Crypto to contain mbed-crypto#152
Improve compatibility with firewalled networks
Dockerfile: apt -> apt-get
Change Docker container to bionic
Clean up file prologue comments
Add docker-based test scripts
ChangeLog: Add ChangeLog entry for #2681
Allow declarations after statements
CMake: Add a subdirectory build regression test
README: Enable builds as a CMake subproject
...
We accidentally disabled testing with MBEDTLS_ECP_RESTARTABLE. Re-enable
testing with restartable ECP when MBEDTLS_USE_PSA_CRYPTO is not set.
Fixes 971dea3745 ("Enable USE_PSA_CRYPTO with config.pl full")
* restricted/pr/573:
Remove redundant config.pl call
Add a test for signing content with a long ECDSA key
Add documentation notes about the required size of the signature buffers
Add missing MBEDTLS_ECP_C dependencies in check_config.h
Change size of preallocated buffer for pk_sign() calls
* origin/pr/2697:
Update crypto submodule
Add all.sh component that exercises invalid_param checks
Remove mbedtls_param_failed from programs
Make it easier to define MBEDTLS_PARAM_FAILED as assert
Make test suites compatible with #include <assert.h>
Pass -m32 to the linker as well
Don't systematically rebuild programs
With the change to the full config, there were no longer any tests
that exercise invalid-parameter behavior. The test suite exercises
invalid-parameter behavior by calling TEST_INVALID_PARAM and friends,
relying on the test suite's mbedtls_check_param function. This
function is only enabled if MBEDTLS_CHECK_PARAMS is defined but not
MBEDTLS_CHECK_PARAMS_ASSERT.
Add a component to all.sh that enables MBEDTLS_CHECK_PARAMS but
disables MBEDTLS_CHECK_PARAMS_ASSERT and doesn't define
MBEDTLS_PARAM_FAILED. This way, the xxx_invalid_param() tests do run.
Since sample programs don't provide a mbedtls_check_param function,
this component doesn't build the sample programs.
* origin/pr/2260:
Update crypto submodule
Remove heading spaces in tests/data_files/Makefile
Re-generate library/certs.c from script
Add new line at the end of test-ca2.key.enc
Use strict syntax to annotate origin of test data in certs.c
Add run to all.sh exercising !MBEDTLS_PEM_PARSE_C + !MBEDTLS_FS_IO
Allow DHM self test to run without MBEDTLS_PEM_PARSE_C
ssl-opt.sh: Auto-skip tests that use files if MBEDTLS_FS_IO unset
Document origin of hardcoded certificates in library/certs.c
Adapt ChangeLog
Rename server1.der to server1.crt.der
Add DER encoded files to git tree
Add build instructions to generate DER versions of CRTs and keys
Document "none" value for ca_path/ca_file in ssl_client2/ssl_server2
ssl_server2: Skip CA setup if `ca_path` or `ca_file` argument "none"
ssl_client2: Skip CA setup if `ca_path` or `ca_file` argument "none"
Correct white spaces in ssl_server2 and ssl_client2
Adapt ssl_client2 to parse DER encoded test CRTs if PEM is disabled
Adapt ssl_server2 to parse DER encoded test CRTs if PEM is disabled
Due to the way the current PK API works, it may have not been clear
for the library clients, how big output buffers they should pass
to the signing functions. Depending on the key type they depend on
MPI or EC specific compile-time constants.
Inside the library, there were places, where it was assumed that
the MPI size will always be enough, even for ECDSA signatures.
However, for very small sizes of the MBEDTLS_MPI_MAX_SIZE and
sufficiently large key, the EC signature could exceed the MPI size
and cause a stack overflow.
This test establishes both conditions -- small MPI size and the use
of a long ECDSA key -- and attempts to sign an arbitrary file.
This can cause a stack overvlow if the signature buffers are not
big enough, therefore the test is performed for an ASan build.
For unit tests and sample programs, CFLAGS=-m32 is enough to get a
32-bit build, because these programs are all compiled directly
from *.c to the executable in one shot. But with makefile rules that
first build object files and then link them, LDFLAGS=-m32 is also
needed.
When testing a configuration where no ciphersuites have MAC, via
component_test_when_no_ciphersuites_have_mac(), perform a targeted test
of only encrypt-then-MAC tests within ssl-opt.sh.
When MBEDTLS_SSL_ENCRYPT_THEN_MAC is enabled, but not
MBEDTLS_SSL_SOME_MODES_USE_MAC, mbedtls_ssl_derive_keys() and
build_transforms() will attempt to use a non-existent `encrypt_then_mac`
field in the ssl_transform.
Compile [ 93.7%]: ssl_tls.c
[Error] ssl_tls.c@865,14: 'mbedtls_ssl_transform {aka struct mbedtls_ssl_transform}' ha
s no member named 'encrypt_then_mac'
[ERROR] ./mbed-os/features/mbedtls/src/ssl_tls.c: In function 'mbedtls_ssl_derive_keys'
:
./mbed-os/features/mbedtls/src/ssl_tls.c:865:14: error: 'mbedtls_ssl_transform {aka str
uct mbedtls_ssl_transform}' has no member named 'encrypt_then_mac'
transform->encrypt_then_mac = session->encrypt_then_mac;
^~
Change mbedtls_ssl_derive_keys() and build_transforms() to only access
`encrypt_then_mac` if `encrypt_then_mac` is actually present.
Add a regression test to detect when we have regressions with
configurations that do not include any MAC ciphersuites.
Fixes d56ed2491b ("Reduce size of `ssl_transform` if no MAC ciphersuite is enabled")
* origin/pr/2403: (24 commits)
crypto: Update to Mbed Crypto 8907b019e7
Create seedfile before running tests
crypto: Update to Mbed Crypto 81f9539037
ssl_cli.c : add explicit casting to unsigned char
Generating visualc files - let Mbed TLS take precedence over crypto
Add a link to the seedfile for out-of-tree cmake builds
Adjust visual studio file generation to always use the crypto submodule
all.sh: unparallelize mingw tests
all.sh - disable parallelization for shared target tests
config.pl: disable PSA_ITS_FILE and PSA_CRYPTO_STORAGE for baremetal
all.sh: unset crypto storage define in a psa full config cmake asan test
all.sh: unset FS_IO-dependent defines for tests that do not have it
curves.pl - change test script to not depend on the implementation
Export the submodule flag to sub-cmakes
Disable MBEDTLS_ECP_RESTARTABLE in full config
Export the submodule flag to sub-makes
Force the usage of crypto submodule
Fix crypto submodule usage in Makefile
Documentation rewording
Typo fixes in documentation
...
Sometimes, when building the shared target with crypto submodule, one could
get an "No rule to make target '../crypto/library/libmbedcrypto.so'" error.
This is due to two reasons - building in parallel and (probably) an
incomplete/incorrect list of dependencies for certain targets. The proposed
solution is to disable parallel builds with crypto submodule for now.
An issue has been raised here: https://github.com/ARMmbed/mbedtls/issues/2634
Sometimes, when building the shared target with crypto submodule, one could
get an "No rule to make target '../crypto/library/libmbedcrypto.so'" error.
This is due to two reasons - building in parallel and (probably) an
incomplete/incorrect list of dependencies for certain targets. The proposed
solution is to disable parallel builds with crypto submodule for now.
An issue has been raised here: https://github.com/ARMmbed/mbedtls/issues/2634
Previously it was disabled as too experimental, which no longer holds. Also,
this option introduces new APIs, so it's not only about an internal
alternative (as the comment in config.pl used to state) - people who request a
full config should get all of the available APIs.
Adapt all.sh: now all builds with full config will also test this option, and
builds with the default config will test without it. Just to be sure, let's
have a build with full config minus this option.
Update documentation of MBEDTLS_USE_PSA_CRYPTO to reflect the status of the
new APIs it enables in Mbed TLS and why they're still opt-in.
Also enable it in scripts/config.pl full, as well as two storage options that
were only blacklisted from full config because they depended on
MBEDTLS_PSA_CRYPTO_C.
Adapt tests in all.sh:
- tests with submodule enabled (default) no longer need to enable it
explicitly, and no longer need runtime tests, as those are now handled by
all other test cases in this script
- tests with submodule disabled (old default) now need to disable it
explicitly, and execute some runtime tests, as those are no longer tested
anywhere else in this script
Adapt documentation in Readme: remove the section "building with submodule"
and replace it with a new section before the other building sections.
Purposefully don't document how to build not from the submodule, as that
option is going away soon.
Run ssl-opt.sh on x86_32 with ASan. This may detect bugs that only
show up on 32-bit platforms, for example due to size_t overflow.
For this component, turn off some memory management features that are
not useful, potentially slow, and may reduce ASan's effectiveness at
catching buffer overflows.
* origin/pr/2464:
Allow main() to lack a docstring.
Silence pylint
check-files.py: readability improvement in permission check
check-files.py: use class fields for class-wide constants
check-files.py: clean up class structure
abi_check.py: Document more methods
check-files.py: document some classes and methods
Fix pylint errors going uncaught
Call pylint3, not pylint
New, documented pylint configuration
Ensure this merge passes tests by auto-generating query_config.c, adding
MBEDTLS_ECDH_LEGACY_CONTEXT to it.
* restricted/pr/552:
Fix mbedtls_ecdh_get_params with new ECDH context
Test undefining MBEDTLS_ECDH_LEGACY_CONTEXT in all.sh
Define MBEDTLS_ECDH_LEGACY_CONTEXT in config.h
Add changelog entry for mbedtls_ecdh_get_params robustness
Fix ecdh_get_params with mismatching group
Add test case for ecdh_get_params with mismatching group
Add test case for ecdh_calc_secret
Fix typo in documentation
* public/pr/2421: (68 commits)
Fix unused variable warning in ssl_parse_certificate_coordinate()
Add missing compile time guard in ssl_client2
Update programs/ssl/query_config.c
ssl_client2: Reset peer CRT info string on reconnect
Add further debug statements on assertion failures
Fix typo in documentation of ssl_parse_certificate_chain()
Add debug output in case of assertion failure
Fix typo in SSL ticket documentation
Add config sanity check for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
ssl_client2: Zeroize peer CRT info buffer when reconnecting
Reintroduce numerous ssl-opt.sh tests if !MBEDTLS_SSL_KEEP_PEER_CERT
ssl_client2: Extract peer CRT info from verification callback
Improve documentation of mbedtls_ssl_get_peer_cert()
Improve documentation of MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
Fix indentation of Doxygen comment in ssl_internal.h
Set peer CRT length only after successful allocation
Remove question in comment about verify flags on cli vs. server
Remove misleading and redundant guard around restartable ECC field
Add test for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE to all.sh
Free peer CRT chain immediately after verifying it
...
Make check-python-files.sh run pylint on all *.py files (in
directories where they are known to be present), rather than list
files explicitly.
Fix a bug whereby the return status of check-python-files.sh was only
based on the last file passing, i.e. errors in other files were
effectively ignored.
Make check-python-files.sh run pylint unconditionally. Since pylint3
is not critical, make all.sh to skip running check-python-files.sh if
pylint3 is not available.
When using PSA with MBEDTLS_ENTROPY_NV_SEED, some test suites
require the seed file for PSA initialization, which was normally generated
later, when entropy tests were run. This change creates an initial seedfile
in all.sh.
When all.sh invokes check_headers_in_cpp, a backup config.h exists. This
causes a stray difference vs cpp_dummy_build.cpp. Fix by only collecting
the *.h files in include/mbedtls.
Change-Id: Ifd415027e856858579a6699538f06fc49c793570
Additional changes to temporarily enable running tests:
ssl_srv.c and test_suite_ecdh use mbedtls_ecp_group_load instead of
mbedtls_ecdh_setup
test_suite_ctr_drbg uses mbedtls_ctr_drbg_update instead of
mbedtls_ctr_drbg_update_ret
Wildcard patterns now work with command line COMPONENT arguments
without --except as well as with. You can now run e.g.
`all.sh "check_*` to run all the sanity checks.
After backing up and restoring config.h, `git diff-files` may report
it as potentially-changed because it isn't sure whether the index is
up to date. Use `git diff` instead: it actually reads the file.
Only look for armcc if component_build_armcc is to be executed,
instead of requiring the option --no-armcc.
You can still pass --no-armcc, but it's no longer required when
listing components to run. With no list of components or an exclude
list on the command line, --no-armcc is equivalent to having
build_armcc in the exclude list.
Build the list of components to run in $RUN_COMPONENTS as part of
command line parsing. After parsing the command line, it no longer
matters how this list was built.
Extract the list of available components by looking for definitions of
functions called component_xxx. The previous code explicitly listed
all components in run_all_components, which opened the risk of
forgetting to list a component there.
Add a conditional execution facility: if a function support_xxx exists
and returns false then component_xxx is not executed (except when the
command line lists an explicit set of components to execute).
MAKEFLAGS was set to -j if it was already set, instead of being set if
not previously set as intended. So now all.sh will do parallel builds
if invoked without MAKEFLAGS in the environment.
Don't bail out of all.sh if the OS isn't Linux. We only expect
everything to pass on a recent Linux x86_64, but it's useful to call
all.sh to run some components on any platform.
In all.sh, always run both MemorySanitizer and Valgrind. Valgrind is
slower than ASan and MSan but finds some things that they don't.
Run MSan unconditionally, not just on Linux/x86_64. MSan is supported
on some other OSes and CPUs these days.
Use `all.sh --except test_memsan` if you want to omit MSan because it
isn't supported on your platform. Use `all.sh --except test_memcheck`
if you want to omit Valgrind because it's too slow.
Make the test scripts more portable (tested on FreeBSD): don't insist
on GNU sed, and recognize amd64 as well as x86_64 for `uname -m`. The
`make` utility must still be GNU make.
Call `set disable-randomization off` only if it seems to be supported.
The goal is to neither get an error about disable-randomization not
being supported (e.g. on FreeBSD), nor get an error if it is supported
but fails (e.g. on Ubuntu).
Only fiddle with disable-randomization from all.sh, which cares
because it reports the failure of ASLR disabling as an error. If a
developer invokes the Gdb script manually, a warning about ASLR
doesn't matter.
Use `cmake -D CMAKE_BUILD_TYPE=Asan` rather than manually setting
`-fsanitize=address`. This lets cmake determine the necessary compiler
and linker flags.
With UNSAFE_BUILD on, force -Wno-error. This is necessary to build
with MBEDTLS_TEST_NULL_ENTROPY.
Merge the work on all.sh that was done on mbedtls-2.14.0 with the
changes from mbedtls-2.14.0 to mbedtls-2.16.0.
There is a merge conflict in test/scripts/all.sh, which is the only
file that was modified in the all.sh work branch. I resolved it by
taking the copy from the all.sh branch and applying the changes
between mbedtls-2.14.0 and mbedtls-2.16.0. These changes consisted of
two commits:
* "Add tests to all.sh for CHECK_PARAMS edge cases": adds two
test components which are reproduced here as
test_check_params_without_platform and component_test_check_params_silent.
* "tests: Backup config.h before modifying it": moot because the
component framework introduced in the all.sh branch backs up config.h
systematically.
In all.sh, always save config.h before running a component, instead of
doing it manually in each component that requires it (except when we
forget, which has happened). This would break a script that requires
config.h.bak not to exist, but we don't have any of those.
Call cleanup from run_component instead of calling it from each
individual component function.
Clean up after each component rather than before. With the new
structure it makes more sense for each component to leave the place
clean. Run cleanup once at the beginning to start from a clean slate.
Move almost all the code of this script into functions. There is no
intended behavior change. The goal of this commit is to make
subsequent improvements easier to follow.
A very large number of lines have been reintended. To see what's going
on, ignore whitespace differences (e.g. diff -w).
I followed the following rules:
* Minimize the amount of code that gets moved.
* Don't change anything to what gets executed or displayed.
* Almost all the code must end up in a function.
* One function does one thing. For most of the code, that's from one
"cleanup" to the next.
* The test sequence functions (run_XXX) are independent.
The change mostly amounts to putting chunks of code into a function
and calling the functions in order. A few test runs are conditional;
in those cases the conditional is around the function call.
It's good to make a backup of config.h before modifying it, so that when
"cleanup" runs the next test has a clean default config.h to start from.
Fixes 840af0a9ae ("Add tests to all.sh for CHECK_PARAMS edge cases")
Add a test to ensure that when the crypto submodule is not used, the crypto
library is not created and that libmbedcrypto.a does not contain symbols
from files contained within the crypto submodule.
- Check that libmbedcrypto was not built at the top level.
- Check that we've built libmbedcrypto with the correct files. Build
libmbedcrypto with debug symbols and verify that files used are from the
crypto submodule.
- Check that config.h is handled properly. Enable a feature at the top
level that the crypto library submodule has disabled in its config.h, and
check that the library symboles indicate that the feature is present in
libmbedcrypto.
- Ensure basic functionality of the resulting build with a run of
`make test` and `ssl-opt.sh`.
all.sh's cleanup function would not entirely remove CMakeFiles due to a
missing -o in its fine command. Add a -o after prune, so that the find for
CMakeFiles can succeed.
This commit adds a test to tests/scripts/all.sh exercising an
ASan build of the default configuration with
MBEDTLS_PLATFORM_MEMORY enabled,
MBEDTLS_PLATFORM_CALLOC_MACRO set to std calloc
MBEDTLS_PLATFORM_FREE_MACRO set to std free
(This should functionally be indistinguishable from a default build)
