Commit Graph

428 Commits

Author SHA1 Message Date
Jaeden Amero
ee427b26d4 tests: Backup config.h before modifying it
It's good to make a backup of config.h before modifying it, so that when
"cleanup" runs the next test has a clean default config.h to start from.

Fixes 840af0a9ae ("Add tests to all.sh for CHECK_PARAMS edge cases")
2018-12-18 21:14:28 +00:00
Manuel Pégourié-Gonnard
840af0a9ae Add tests to all.sh for CHECK_PARAMS edge cases 2018-12-11 12:28:56 +01:00
Jaeden Amero
faafcd83a8 Merge remote-tracking branch 'upstream-public/pr/2223' into development 2018-12-07 16:16:24 +00:00
Jaeden Amero
d07ef47150 Merge remote-tracking branch 'upstream-public/pr/2061' into development 2018-12-07 16:07:21 +00:00
Jaeden Amero
60ca6e58b6 test: Make basic-build-test.sh see summary statuses
We've changed the behavior of "-v" to no longer output test summary
statuses. Update basic-build-test.sh to use the test runner's verbosity
option "-v 2", so that the basic-build-test.sh script can get the summary
statuses it needs.
2018-12-07 13:06:24 +00:00
Jaeden Amero
8ef1f48a52 Merge remote-tracking branch 'upstream-public/pr/2219' into development 2018-12-06 16:16:36 +00:00
Jaeden Amero
3c0941ec64 Merge remote-tracking branch 'upstream-public/pr/2172' into development 2018-12-06 16:09:08 +00:00
Gilles Peskine
c117d5928c check-files: detect merge artifacts
Detect Git merge artifacts. These are lines starting with "<<<<<<",
"|||||||" or ">>>>>>>" followed by a space, or containing just
"=======". For "=======", exempt Markdown files, because this can be
used to underline a title, as a compromise between false negatives and
false positives.
2018-11-23 21:11:52 +01:00
Gilles Peskine
043980585c Factor record_issue into its own method 2018-11-23 21:11:30 +01:00
Jaeden Amero
80a23a5bc4 check-files: Don't check same-named files
The check-files script contains the strings "TODO" and "todo" in order to
search for files that contain TODO items. So, any check-files script would
need to be excluded from the list of files that gets checked for "TODO".
Normally, the script excludes itself from checks, but with the addition of
the crypto submodule, there is another copy of the script present from the
project root. We must avoid checking check-files scripts for TODO items.
This also helps if you run check-files from another working tree in your
working tree.
2018-11-23 13:14:38 +00:00
Simon Butcher
53b45ec881 Merge remote-tracking branch 'public/pr/2079' into development-restricted-proposed 2018-11-07 12:58:44 +00:00
Ron Eldor
c242eea732 Change data file suffix for ott
Change the suffix of the data files searched in `mbedtls_test.py`
to `datax` as the generated files have this suffix.
2018-11-05 16:22:36 +02:00
Jaeden Amero
8396a71449 test: Enable multiple levels of verbosity
Enable passing a number to "-v" in order to set the level of verbosity.
Print detailed test failure information at verbosity level 1 or higher.
Display summary messages at the verbosity level 2 or higher. Print
detailed test information at verbosity level 3 or higher, whether the
test failed or not. This enables a more readable output style that
includes detailed failure information when a failure occurs.
2018-11-02 13:25:16 +00:00
Jaeden Amero
f4b521dd10 test: Use GetOpt::Long for argument parsing
Simplify argument parsing by using a core perl library for parsing
arguments.
2018-11-02 13:25:16 +00:00
Jaeden Amero
79e4f4e933 test: Print verbosely on failures in verbose mode
Update the test runner to print detail about why the test failed when it
fails, if the runner is running in verbose mode.
2018-11-02 13:25:16 +00:00
Simon Butcher
3869384ddb Merge remote-tracking branch 'public/pr/2031' into development 2018-10-27 18:34:18 +01:00
Simon Butcher
78dd2e55bf Merge remote-tracking branch 'public/pr/1806' into development 2018-10-27 17:59:17 +01:00
Gilles Peskine
96f3b4ee42 Remove yotta support from check-files.py
Complements "Remove Yotta support from the docs, tests and build scripts".
2018-10-16 14:10:21 +02:00
Hanno Becker
e5fecec7bf Add test for MBEDTLS_PLATFORM_{CALLOC/FREE}_MACRO to all.sh
This commit adds a test to tests/scripts/all.sh exercising an
ASan build of the default configuration with

MBEDTLS_PLATFORM_MEMORY enabled,
MBEDTLS_PLATFORM_CALLOC_MACRO set to std calloc
MBEDTLS_PLATFORM_FREE_MACRO   set to std free

(This should functionally be indistinguishable from a default build)
2018-10-11 11:04:20 +01:00
Gilles Peskine
95c5575e12 check-files: exclude .git and third-party files
Exclude ".git" directories anywhere. This avoids spurious errors in git
checkouts that contain branch names that look like a file
check-files.py would check. Fix #1713

Exclude "mbed-os" anywhere and "examples" from the root. Switch to the
new mechanism to exclude "yotta/module". These are directories where
we store third-party files that do not need to match our preferences.

Exclude "cov-int" from the root. Fix #1691
2018-10-02 13:13:24 +02:00
Simon Butcher
6e3606e4f6 Fix run-test-suites.pl to screen for files
Changes run-test-suites.pl to filter out directories, and select only files
as on OSX, test coverage tests create .dSYM directories which were being
accidentally selected to execute.
2018-09-30 21:53:16 +01:00
Gilles Peskine
427df37f84 Don't try to disable ASLR
We don't need to disable ASLR, so don't try. If gdb tries but fails,
the test runs normally, but all.sh then trips up because it sees
`warning: Error disabling address space randomization: Operation not permitted`
and interprets it as an error that indicates a test failure.
2018-09-28 14:31:16 +02:00
Gilles Peskine
5c39d7a972 Remove redundant check in all.sh
test -s can't fail if the subsequent grep succeeds.
2018-09-28 14:31:16 +02:00
Gilles Peskine
bd90a8c002 In keep-going mode, don't hard-fail on some tests
Add if_build_succeeded in front of the invocation of some test runs
where it was missing.
2018-09-28 14:31:16 +02:00
Simon Butcher
3ad2efdc82 Remove Yotta support from the docs, tests and build scripts
Yotta is no longer supported by Mbed TLS, so has been removed. Specifically, the
following changes have been made:
 * references to yotta have been removed from the main readme and build
   instructions
 * the yotta module directory and build script has been removed
 * yotta has been removed from test scripts such as all.sh and check-names.sh
 * yotta has been removed from other files that that referenced it such as the
   doxyfile and the bn_mul.h header
 * yotta specific configurations and references have been removed from config.h
2018-09-26 17:55:09 +01:00
Gilles Peskine
487cdf5516 In keep-going mode, don't hard-fail on some auxiliary script
Add record_status in front of the invocation of several scripts where
it was missing.
2018-09-26 16:50:38 +02:00
Simon Butcher
68dbc94720 Merge remote-tracking branch 'public/pr/1951' into development 2018-08-30 00:56:56 +01:00
Hanno Becker
2f5aa4c64e all.sh: Add builds allowing to test dropping buffered messages
This commit adds two builds to all.sh which use a value of
MBEDTLS_SSL_DTLS_MAX_BUFFERING that allows to run the
reordering tests in ssl-opt.sh introduced in the last commit.
2018-08-24 14:48:11 +01:00
Mohammad Azim Khan
5cb7017077 Less obscure test suites template 2018-08-06 11:42:56 +01:00
Mohammad Azim Khan
d2d0112ca8 Style fixes 2018-08-06 11:42:56 +01:00
Mohammad Azim Khan
440d8737c6 Fix macro validation regex 2018-08-06 11:42:56 +01:00
Mohammad Azim Khan
ee6529eb83 Add generate_test_code.py unit tests in all.sh 2018-08-06 11:42:56 +01:00
Mohammad Azim Khan
21798105f6 Pylint tests in all.sh 2018-08-06 11:42:56 +01:00
Mohammad Azim Khan
539aa06f64 Fix Python 2 & 3 compatibility in test_generate_test_code.py 2018-08-06 11:42:56 +01:00
Mohammad Azim Khan
32cbcdac8f Fix Pylint errors in Python scripts 2018-08-06 11:42:56 +01:00
Azim Khan
630281349e Replaced escaped_split() logic with regex 2018-08-06 11:42:56 +01:00
Azim Khan
fcdf685302 Make test function parsing robust
This commit enhances parsing of the test function in generate_test_code.py for
cases where return type and function name are on separate lines.
2018-08-06 11:42:56 +01:00
Azim Khan
4084ec7ae5 Fixed unit tests in test_generate_test_code.py 2018-08-06 11:42:56 +01:00
Azim Khan
8d686bfdb1 Incorporated code revoew comments. 2018-08-06 11:42:56 +01:00
Azim Khan
b31aa44e16 Fix style errors reported by pylint 2018-08-06 11:42:56 +01:00
Azim Khan
aee05bbe70 Add test suite framework summary 2018-08-06 11:42:56 +01:00
Azim Khan
5fcca46a3a Rename HexParam_t -> data_t for consistent coding style 2018-08-06 11:42:06 +01:00
Azim Khan
951a2c8898 Improve documentation in mbedtls_test.py 2018-08-06 11:42:06 +01:00
Azim Khan
e3b26af7c0 Improve documentation in generate_test_code.py 2018-08-06 11:42:06 +01:00
Azim Khan
b98e6eec58 Replace asserts with exceptions in mbedtls_test.py 2018-08-06 11:42:06 +01:00
Azim Khan
040b6a2281 Wrap code to 79 character limit 2018-08-06 11:42:06 +01:00
Azim Khan
936ea9302a Strip whitespaces added by decode() function 2018-08-06 11:42:06 +01:00
Mohammad Azim Khan
00c4b090c1 Change intermediate data file extension to .datax 2018-08-06 11:42:06 +01:00
Mohammad Azim Khan
8f6e8cfcc7 Print line number with data file error 2018-08-06 11:40:58 +01:00
Mohammad Azim Khan
3b06f226e9 Replace asserts with exceptions in generate_test_code.py 2018-08-06 11:40:58 +01:00