Commit Graph

633 Commits

Author SHA1 Message Date
Manuel Pégourié-Gonnard
f2e2902c5a Add detection for zlib headers to all.sh 2020-01-24 10:44:13 +01:00
Manuel Pégourié-Gonnard
95e04490fa Add all.sh components with ZLIB enabled
ZLIB support is deprecated, but until it's removed it should still be tested.
2020-01-24 10:44:13 +01:00
Darryl Green
fb5faa2582 Fix some pylint warnings
Add docstrings where they were missing and fix a too-long line
2019-12-20 15:14:59 +00:00
Gilles Peskine
2e86a206f7 Remove component designed to test MAX_SIGNATURE_SIZE
MBEDTLS_PK_SIGNATURE_MAX_SIZE is tested in Mbed Crypto. Its effect on
Mbed TLS is also tested via the X.509 tests. The case of
MBEDTLS_MPI_MAX_SIZE < MBEDTLS_ECDSA_MAX_LEN, for which this component
was added as a regression test, is covered by config-suite-b.h which
is tested via test-ref-configs.pl.
2019-11-13 15:32:11 +01:00
Gilles Peskine
ce35cb3cc7 'make test' must fail if Asan fails
When running 'make test' with GNU make, if a test suite program
displays "PASSED", this was automatically counted as a pass. This
would in particular count as passing:
* A test suite with the substring "PASSED" in a test description.
* A test suite where all the test cases succeeded, but the final
  cleanup failed, in particular if a sanitizer reported a memory leak.

Use the test executable's return status instead to determine whether
the test suite passed. It's always 0 on PASSED unless the executable's
cleanup code fails, and it's never 0 on any failure.

Fix ARMmbed/mbed-crypto#303
2019-10-21 20:10:21 +02:00
Gilles Peskine
5ca393f7b8 Asan make builds: avoid sanitizer recovery
Some sanitizers default to displaying an error message and recovering.
This could result in a test being recorded as passing despite a
complaint from the sanitizer. Turn off sanitizer recovery to avoid
this risk.
2019-10-21 20:10:12 +02:00
Gilles Peskine
2558d30f93 Use UBsan in addition to Asan with 'make test'
When building with make with the address sanitizer enabled, also
enable the undefined behavior sanitizer.
2019-10-21 20:09:22 +02:00
Gilles Peskine
8fd5942229 Unify ASan options in make builds
Use a common set of options when building with Asan without CMake.
2019-10-21 20:09:13 +02:00
Jaeden Amero
719ae74253
Merge pull request #2884 from AndrzejKurek/iotssl-2886-fix-parallel-make-jobs-2
Fix parallel make jobs for shared target
2019-10-18 14:18:55 +01:00
Andrzej Kurek
232e8f959a Enable parallel shared target tests
Now that the dependency issues for shared target are fixed, the feature can be
enabled in tests.
2019-10-07 09:24:43 -04:00
Jaeden Amero
230b87a1ea Merge remote-tracking branch 'origin/pr/2843' into development
* origin/pr/2843: (26 commits)
  Make hyperlink a hyperlink in every markdown flavor
  Update the crypto submodule to be the same as development
  Document test case descriptions
  Restore MBEDTLS_TEST_OUTCOME_FILE after test_default_out_of_box
  ssl-opt.sh: Fix some test case descriptions
  Reject non-ASCII characters in test case descriptions
  Process input files as binary
  Factor description-checking code into a common function
  Fix cosmetic error in warnings
  Fix regex matching run_test calls in ssl-opt.sh
  all.sh: run check-test-cases.py
  Better information messages for quick checks
  Fix configuration short name in key-exchanges.pl
  Make test case descriptions unique
  New test script check-test-cases.py
  Document the test outcome file
  Create infrastructure for architecture documents in Markdown
  all.sh --outcome-file creates an outcome file
  Set meaningful test configuration names when running tests
  ssl-opt: remove semicolons from test case descriptions
  ...
2019-10-02 18:01:32 +01:00
Gilles Peskine
717cd76e8a Restore MBEDTLS_TEST_OUTCOME_FILE after test_default_out_of_box
Since components run in the main process, unsetting
MBEDTLS_TEST_OUTCOME_FILE unset it in subsequent components as well.
To avoid this, save and restore the value.

(Making each component run in a subshell would be a better solution,
but it would be a much bigger change.)
2019-09-27 20:24:33 +02:00
Gilles Peskine
57870e8c67 Reject non-ASCII characters in test case descriptions
Don't require that all the tools we use to process test outcomes are
Unicode-clean.
2019-09-24 19:21:20 +02:00
Gilles Peskine
f12ad58a1d Process input files as binary
Don't die if there's a non-ASCII character and we're running in an
ASCII environment.
2019-09-24 19:21:20 +02:00
Gilles Peskine
32b9421f12 Factor description-checking code into a common function
Behavior change: some error messages are slightly different.
2019-09-24 19:21:20 +02:00
Gilles Peskine
283df2e90c Fix cosmetic error in warnings 2019-09-24 19:21:20 +02:00
Gilles Peskine
168858f52d Fix regex matching run_test calls in ssl-opt.sh
No descriptions were processed before due to bugs in the regex.

Support \" inside double-quoted strings.
2019-09-24 19:21:19 +02:00
Gilles Peskine
895868bc82 all.sh: run check-test-cases.py 2019-09-24 19:21:19 +02:00
Gilles Peskine
600bb694ac Better information messages for quick checks
Call them "check" rather than "test" to distinguish them from tests
that build and run code, and for consistency with the component names.
2019-09-24 19:21:19 +02:00
Gilles Peskine
a9478bab08 Fix configuration short name in key-exchanges.pl
This is testing with $kex, not without $kex, so use $kex, not "-$kex".

In test-ref-configs.pl, use $conf rather than "$conf". This is purely
a matter of Perl coding style.
2019-09-24 19:21:19 +02:00
Gilles Peskine
ba94b58127 New test script check-test-cases.py
This script checks test case descriptions in test_suite_*.data and
ssl-opt.sh.

It reports the following issues:
* Error: forbidden character in a test case description.
* Error: Duplicate test description.
* Warning: Test description is too long.
2019-09-24 19:21:19 +02:00
Gilles Peskine
67ffdafde6 all.sh --outcome-file creates an outcome file
By default, remove the outcome file before starting. With
--append-outcome, append to the existing outcome file if there is one.
2019-09-24 19:21:19 +02:00
Gilles Peskine
16a25e005d Add set+get tests
The tests were not covering get for a symbol with a value. No symbol
has an uncommented value in the default config.h. (Actually there's
_CRT_SECURE_NO_DEPRECATE, but that's a bit of a hack that this script
is not expected to handle, so don't use it).

Add tests of "get FOO" after "set FOO" and "set FOO value", so that we
have coverage for "get FOO" when "FOO" has a value.
2019-09-19 12:19:24 +02:00
Gilles Peskine
fd7ad33ee9 Consolidate tests for set with/without values
We currently test setting a symbol with a value even if it didn't
originally had one and vice versa. So there's no need to have separate
lists of symbols to test with. Just test everything we want to test
with each symbol.
2019-09-19 12:18:23 +02:00
Gilles Peskine
9004a1768b Set meaningful test configuration names when running tests
Set MBEDTLS_TEST_PLATFORM and MBEDTLS_TEST_CONFIGURATION to meaningful
values in all.sh. These environment variables are used when
writing an outcome file, which happens if MBEDTLS_TEST_OUTCOME_FILE is
also set.

When running one of the try-multiple-configuration scripts, set
MBEDTLS_TEST_CONFIGURATION to a value that uniquely describes the
configuration.
2019-09-18 17:44:29 +02:00
Gilles Peskine
a708dae94b Add comment to help syntax highlighting in editors 2019-09-18 17:44:29 +02:00
Gilles Peskine
61695e70f8 config.py testing: also test the get command 2019-09-13 15:17:57 +02:00
Gilles Peskine
f686042554 Fix config.py output when a symbol has acquired or lost a value
Normally a valueless symbol remains valueless and a symbol with a
value keeps having one. But just in case a symbol does get changed
from valueless to having a value, make sure there's a space between
the symbol and the value. And if a symbol gets changed from having a
value to valueless, strip trailing whitespace.

Add corresponding tests.

Also fix the case of a valueless symbol added with the set method,
which would have resulted in attempting to use None as a string. This
only happened with the Python API, not with the command line API.
2019-09-13 11:04:24 +02:00
Gilles Peskine
e3734bd13a Remove redundant test case 2019-09-13 11:04:24 +02:00
Gilles Peskine
878acd6490 Test script for config.py
Run config.py with various options and store the results in files.

This script also supports the now-removed config.pl.

This is a framework to run non-regression tests on config.py: run it
with the old version, run it with the new version, and compare the
output.

This is deliberately not a functional test suite so that we don't need
to maintain a set of known outputs. When something changes in
config.py (or config.h), run the script before, run it after, and
check manually whether any differences in the output are acceptable.
2019-09-13 11:04:24 +02:00
Gilles Peskine
5d46f6a89b Invoke config.py instead of config.pl
git grep -Fl /config.pl | xargs sed -i -e 's!/config\.pl!/config.py!g'

Also:
* Change one comment in include/mbedtls/check_config.h.
* Change PERL to PYTHON in CMakeLists.txt.
2019-09-13 11:04:23 +02:00
Jaeden Amero
62be28b525 Merge remote-tracking branch 'origin/pr/2469' into development
* origin/pr/2469:
  Enable MBEDTLS_MEMORY_DEBUG in memory buffer alloc test in all.sh
  Remove unnecessary memory buffer alloc unsets
  Disable DTLS proxy tests for MEMORY_BUFFER_ALLOC test
  all.sh: restructure memory allocator tests
  Add missing dependency in memory buffer alloc set in all.sh
  Don't set MBEDTLS_MEMORY_DEBUG through `scripts/config.pl full`
  Add cfg dep MBEDTLS_MEMORY_DEBUG->MBEDTLS_MEMORY_BUFFER_ALLOC_C
  Fix memory leak in CSR test suite on failure
  Fix a memory leak in x509write test suite
  Add all.sh run with full config and ASan enabled
  Add all.sh run with MBEDTLS_MEMORY_BUFFER_ALLOC_C enabled
  Update documentation of exceptions for `config.pl full`
  Adapt all.sh to removal of buffer allocator from full config
  Disable memory buffer allocator in full config
  Check dependencies of MBEDTLS_MEMORY_BACKTRACE in check_config.h
2019-09-12 16:45:55 +01:00
Andrzej Kurek
9f409f6aec Enable MBEDTLS_MEMORY_DEBUG in memory buffer alloc test in all.sh 2019-09-10 04:28:55 -04:00
Andrzej Kurek
4b3a45e190 Remove unnecessary memory buffer alloc unsets
This define is turned off by default
2019-09-06 07:47:56 -04:00
Andrzej Kurek
1e56d2c3de Disable DTLS proxy tests for MEMORY_BUFFER_ALLOC test 2019-09-06 07:44:37 -04:00
Andrzej Kurek
69f20aae77 all.sh: restructure memory allocator tests
Run basic tests and ssl-opt with memory backtrace disabled, then
run basic tests only with it enabled.
2019-09-06 07:44:37 -04:00
Hanno Becker
d7064202ea Add missing dependency in memory buffer alloc set in all.sh 2019-09-06 07:44:37 -04:00
Hanno Becker
0163551aa0 Add all.sh run with full config and ASan enabled 2019-09-06 07:44:37 -04:00
Hanno Becker
0fb9ba2760 Add all.sh run with MBEDTLS_MEMORY_BUFFER_ALLOC_C enabled
With the removal of MBEDTLS_MEMORY_BUFFER_ALLOC_C from the
full config, there are no tests for it remaining in all.sh.
This commit adds a build as well as runs of `make test` and
`ssl-opt.sh` with MBEDTLS_MEMORY_BUFFER_ALLOC_C enabled to all.sh.
2019-09-06 07:44:37 -04:00
Unknown
790c281f51 Adapt all.sh to removal of buffer allocator from full config
Previously, numerous all.sh tests manually disabled the buffer allocator
or memory backtracting after setting a full config as the starting point.

With the removal of MBEDTLS_MEMORY_BACKTRACE and MBEDTLS_MEMORY_BUFFER_ALLOC_C
from full configs, this is no longer necessary.
2019-09-06 07:44:37 -04:00
Jaeden Amero
c9c4ca3f40 Merge remote-tracking branch 'origin/development' into development-restricted
* origin/development:
  Fix copypasta in msg
  When not using PSA crypto, disable it
  Disable MEMORY_BUFFER_ALLOC with ASan
  Remove config.pl calls with no effect
  ssl-opt.sh: wait for proxy to start before running the script further
  Adapt ChangeLog
  Fix mpi_bigendian_to_host() on bigendian systems
2019-09-05 18:14:14 +01:00
Jaeden Amero
ba7f4d1484 Merge remote-tracking branch 'origin/pr/2771' into development
* origin/pr/2771:
  Fix copypasta in msg
  When not using PSA crypto, disable it
  Disable MEMORY_BUFFER_ALLOC with ASan
  Remove config.pl calls with no effect
2019-09-05 14:23:55 +01:00
Gilles Peskine
dc3a179995 Fix copypasta in msg 2019-09-03 14:11:36 +02:00
Gilles Peskine
6ce30722d0 When not using PSA crypto, disable it
In the test with the full config without MBEDTLS_USE_PSA_CRYPTO, don't
build MBEDTLS_PSA_CRYPTO_C, since it isn't supposed to be used.
2019-09-03 14:11:36 +02:00
Gilles Peskine
751bb4c0e1 Disable MEMORY_BUFFER_ALLOC with ASan
MBEDTLS_MEMORY_BUFFER_ALLOC_C makes ASan mostly ineffective since it
hides allocations. So disable it when testing with ASan.
2019-09-03 14:11:36 +02:00
Gilles Peskine
c6f1c84663 Remove config.pl calls with no effect
When MBEDTLS_MEMORY_BUFFER_ALLOC_C is disabled, other
MBEDTLS_MEMORY_xxx options have no effect, so don't bother unsetting
them explicitly.
2019-09-03 14:11:36 +02:00
Jaeden Amero
4cf0e7e4d2 Merge remote-tracking branch 'origin/development' into development-restricted
* origin/development: (42 commits)
  Handle deleting non-existant files on Windows
  Update submodule
  Use 3rdparty headers from the submodule
  Add Everest components to all.sh
  3rdparty: Add config checks for Everest
  Fix macros in benchmark.c
  Update generated files
  3rdparty: Fix inclusion order of CMakeLists.txt
  Fix trailing whitespace
  ECDH: Fix inclusion of platform.h for proper use of MBEDTLS_ERR_PLATFORM_FEATURE_UNSUPPORTED
  ECDH: Fix use of ECDH API in full handshake benchmark
  ECDH: Removed unnecessary calls to mbedtls_ecp_group_load in ECDH benchmark
  ECDH: Fix Everest x25519 make_public
  Fix file permissions
  3rdparty: Rename THIRDPARTY_OBJECTS
  3rdparty: Update description of MBEDTLS_ECDH_VARIANT_EVEREST_ENABLED
  3rdparty: Fix Makefile coding conventions
  ECDSA: Refactor return value checks for mbedtls_ecdsa_can_do
  Add a changelog entry for Everest ECDH (X25519)
  Document that curve lists can include partially-supported curves
  ...
2019-08-30 16:24:18 +01:00
Gilles Peskine
0c6b79979c Add Everest components to all.sh
Test a native build and a 32-bit build. For variety, the native build
is with CMake and clang, and the 32-bit build is with GNU make and
gcc.
2019-08-29 16:12:38 +01:00
Christoph M. Wintersteiger
37eb90617a 3rdparty: Fix Makefile coding conventions 2019-08-29 16:12:38 +01:00
Christoph M. Wintersteiger
9b33e7d7d7 ECDH: Exclude FStar and Hacl* from exported symbol checks 2019-08-29 16:12:38 +01:00