Commit Graph

308 Commits

Author SHA1 Message Date
Jaeden Amero
5758d8cb9c test: Print verbosely on failures in verbose mode
Update the test runner to print detail about why the test failed when it
fails, if the runner is running in verbose mode.
2018-11-02 17:30:24 +00:00
Simon Butcher
ec3f9c362d Merge remote-tracking branch 'public/pr/2032' into mbedtls-2.7 2018-10-27 18:34:46 +01:00
Gilles Peskine
3400b4dbd6 check-files: exclude .git and third-party files
Exclude ".git" directories anywhere. This avoids spurious errors in git
checkouts that contain branch names that look like a file
check-files.py would check. Fix #1713

Exclude "mbed-os" anywhere and "examples" from the root. Switch to the
new mechanism to exclude "yotta/module". These are directories where
we store third-party files that do not need to match our preferences.

Exclude "cov-int" from the root. Fix #1691
2018-10-02 13:14:58 +02:00
Simon Butcher
eb219391fa Fix run-test-suites.pl to screen for files
Changes run-test-suites.pl to filter out directories, and select only files
as on OSX, test coverage tests create .dSYM directories which were being
accidentally selected to execute.
2018-09-30 21:57:34 +01:00
Simon Butcher
d620f6f56b Adds a filter to run-test-suites.pl to exclude data files
The run-test-suites.pl script was executing all files of the form 'test_suite*'
which were either executable or ended with a .exe extension.

On some filesystems, such as through network shares or VMs, which are
abstracting one set of file permissions to Unix permissions, may set the
executable permissions on all files, whether they're executable or not.

That was leading to the run-test-suites.pl script to attempt to execute the .c
intermediate files because they followed the form 'test_suite_*.c'. This change
now excludes them, just in case they accidentally have execute permissions.
2018-09-27 17:29:44 +01:00
Gilles Peskine
b46c59e2a7 In keep-going mode, don't hard-fail on some tests
Add if_build_succeeded in front of the invocation of some test runs
where it was missing.
2018-09-27 10:12:58 +02:00
Gilles Peskine
732826d265 In keep-going mode, don't hard-fail on some auxiliary script
Add record_status in front of the invocation of several scripts where
it was missing.
2018-09-27 10:08:31 +02:00
Simon Butcher
d7126d7009 Merge remote-tracking branch 'public/pr/779' into mbedtls-2.7 2018-07-24 13:38:44 +01:00
Simon Butcher
bd40916dfa Merge remote-tracking branch 'public/pr/1872' into mbedtls-2.7 2018-07-24 12:12:43 +01:00
Simon Butcher
948f264302 Add additional i386 tests to all.sh
Added an additional i386 test to all.sh, to allow one test with -O0 which
compiles out inline assembly, and one to test with -01 which includes the inline
assembly.
2018-07-23 13:41:25 +01:00
Jaeden Amero
5113bdec6e all.sh: Return error on keep-going failure
When calling all.sh from a script and using "--keep-going", errors were
sometimes missed due to all.sh always returning 0 "success" return code.
Return 1 if there is any failure encountered during a "keep-going" run.
2018-07-23 10:24:31 +01:00
Simon Butcher
7c6b84102d Expand i386 all.sh tests to full config ASan builds
The i386 test builds were only building the default configuration and had
no address sanitisation. This commit expands the test configuration to the full
configuration in all.sh and builds with ASan for when the test suites are
executed.
2018-07-20 21:34:04 +01:00
Simon Butcher
0e342f77fc Merge remote-tracking branch 'public/pr/1390' into mbedtls-2.7 2018-06-27 11:11:34 +01:00
Simon Butcher
f15cfd5d04 Merge remote-tracking branch 'public/pr/1557' into mbedtls-2.7 2018-06-27 11:07:50 +01:00
Simon Butcher
8c83673eb2 Merge remote-tracking branch 'public/pr/1708' into mbedtls-2.7 2018-06-12 17:26:55 +01:00
Simon Butcher
856870952a Merge remote-tracking branch 'public/pr/1709' into mbedtls-2.7 2018-06-12 17:25:19 +01:00
Simon Butcher
ee3a3d4a72 Merge remote-tracking branch 'public/pr/1470' into mbedtls-2.7 2018-06-11 11:30:33 +01:00
Simon Butcher
bb5e1c3973 Fix multiple quality issues in the source
This PR fixes multiple issues in the source code to address issues raised by
tests/scripts/check-files.py. Specifically:
 * incorrect file permissions
 * missing newline at the end of files
 * trailing whitespace
 * Tabs present
 * TODOs in the souce code
2018-06-08 11:14:43 +01:00
Darryl Green
bd38c3b89f Add check-files.py to all.sh 2018-06-05 11:57:12 +01:00
Darryl Green
da02eb310c Add script for source integrity checking 2018-06-05 11:57:01 +01:00
Simon Butcher
e83b1ae201 Merge remote-tracking branch 'public/pr/1606' into mbedtls-2.7 2018-06-01 19:34:44 +01:00
Jaeden Amero
11d5551d0a Merge remote-tracking branch 'upstream-public/pr/1487' into mbedtls-2.7-proposed 2018-05-04 11:06:21 +01:00
Andres AG
b7b420b51c Fix uninitialized var in check-generated-files.sh 2018-05-01 21:01:22 +01:00
Andres Amaya Garcia
7dae108fe8 Check generated-visualc-files in check-generated-files 2018-05-01 21:01:18 +01:00
fbrosson
3a7457136e Backport 2.7: Use "#!/usr/bin/env perl" as shebang line. 2018-04-04 22:26:56 +00:00
Gilles Peskine
595c84a7b1 Merge remote-tracking branch 'upstream-public/pr/1500' into mbedtls-2.7-proposed 2018-04-01 12:41:29 +02:00
Andres Amaya Garcia
28d97e1dfc Fix shared library lookup on Mac OS X when running tests 2018-03-27 20:04:20 +01:00
Gilles Peskine
763da6e550 all.sh --keep-going: properly handle multiple-builds scripts
In keep-going mode, if a multiple-builds script fails, record its
status and keep going.
2018-03-23 02:18:33 +01:00
Gilles Peskine
2cfeb887b4 Merge tag 'mbedtls-2.7.2' into iotssl-1381-x509-verify-refactor-2.7-restricted
Conflict resolution:

* ChangeLog
* tests/data_files/Makefile: concurrent additions, order irrelevant
* tests/data_files/test-ca.opensslconf: concurrent additions, order irrelevant
* tests/scripts/all.sh: one comment change conflicted with a code
  addition. In addition some of the additions in the
  iotssl-1381-x509-verify-refactor-restricted branch need support for
  keep-going mode, this will be added in a subsequent commit.
2018-03-23 02:12:44 +01:00
Gilles Peskine
0114ffc76b all.sh: Verify out-of-tree testing with CMake
Run a test case in ssl-opt.sh to validate that testing works in an
out-of-tree CMake build.
2018-03-21 12:29:20 +01:00
Gilles Peskine
a71d64c74f all.sh: fix cleanup happening during an out-of-tree build 2018-03-21 12:29:15 +01:00
Gilles Peskine
31b07e2833 all.sh: be more conservative when cleaning up CMake artefacts
Only delete things that we expect to find, to avoid deleting other
things that people might have lying around in their build tree.
Explicitly skip .git to avoid e.g. accidentally matching a branch
name.
2018-03-21 12:29:08 +01:00
Gilles Peskine
19ceb7104c all.sh: add opposites to all boolean options
All options can now be overridden by a subsequent option, e.g.
"all.sh --foo --no-foo" is equivalent to "all.sh --no-foo". This
allows making wrapper scripts with default options and occasionally
overriding those options when running the wrapper script.
2018-03-21 08:48:40 +01:00
Gilles Peskine
54933e95bd all.sh: option parsing: reduce vertical spread
Only whitespace changes.
2018-03-21 08:48:33 +01:00
Gilles Peskine
53038ebecc all.sh: with --no-armcc, don't call armcc from output_env.sh
When not running armcc, don't try to invoke armcc at all, not even to
report its version.
2018-03-21 08:48:26 +01:00
Gilles Peskine
955d70459d Merge remote-tracking branch 'upstream-restricted/pr/460' into mbedtls-2.7-restricted-proposed 2018-03-13 17:24:33 +01:00
Gilles Peskine
4848b97bc7 Merge remote-tracking branch 'upstream-public/pr/1249' into mbedtls-2.7-proposed 2018-03-11 00:48:17 +01:00
Hanno Becker
69d45cce5d Add a run with RSA_NO_CRT to all.sh 2018-03-09 10:46:23 +00:00
Manuel Pégourié-Gonnard
fa973e022a Document choice of script exit code 2018-03-05 13:23:27 +01:00
Gert van Dijk
ab41f04554 Tests: depends-pkalgs.pl - disable less options
Rather than disabling SSL & Key exchanges as a whole, only disable those
options required by reverse dependencies.

GitHub issue #1040 https://github.com/ARMmbed/mbedtls/issues/1040
See also discussion in PR #1074.
https://github.com/ARMmbed/mbedtls/pull/1074#issuecomment-327096303
2018-03-05 13:20:36 +01:00
Manuel Pégourié-Gonnard
fcc4348ee2 Improve some comments 2018-03-05 13:20:21 +01:00
Manuel Pégourié-Gonnard
2f1633e002 Improve some comments, fix some typos 2018-03-05 13:19:41 +01:00
Manuel Pégourié-Gonnard
b26b28a7e4 Make some perl scripts usable with git bisect run
For that they need to return between 0 and 124 on error, while die returns
255, causing bisect-run to abort.
2018-03-05 13:16:15 +01:00
Manuel Pégourié-Gonnard
4d9e7cb66b Fix usage of CFLAGS with cmake in all.sh
With cmake, CFLAGS has to be set when invoking cmake, not make (which totally
ignores the value of CFLAGS when it runs and only keeps the one from cmake).

Also, in that case the flags were either redundant (-Werror etc) or wrong
(-std=c99 -pedantic) as some parts of the library will not build with
-pedantic (see the other -pedantic tests, which are correct, for what needs to
be disabled).
2018-03-05 13:00:03 +01:00
Manuel Pégourié-Gonnard
1ddd682843 Fix depends_on:pk_alg in test suites 2018-03-05 12:58:51 +01:00
Manuel Pégourié-Gonnard
153b5ef696 Add new test script depends-pkalgs.pl 2018-03-05 12:58:42 +01:00
Manuel Pégourié-Gonnard
056eab5ff0 Fix usage of {curves,key-exchanges}.pl in all.sh 2018-03-05 12:58:22 +01:00
Manuel Pégourié-Gonnard
483a776019 Fix missing depends_on:SHA/MD in x509 tests 2018-03-05 12:57:36 +01:00
Manuel Pégourié-Gonnard
73fc679293 Add new test script depends-hashes.pl
This is step 1 of a plan to get rid once and for all of missing depends_on in
the X509 test suite (step 2 will be RSA/ECDSA, and step 0 was curves.pl).
2018-03-05 12:57:16 +01:00
Gilles Peskine
b4ef45b4f5 all.sh: build with -O -Werror in the full config
Build with MBEDTLS_DEPRECATED_REMOVED and MBEDTLS_DEPRECATED_WARNING
separately.

Do these builds with `-O -Werror -Wall -Wextra` to catch a maximum of
issues while we're at it. Do one with gcc and one with clang for
variety. This caught an uninitialized variable warning in cmac.c that
builds without -O didn't catch.
2018-03-01 22:23:50 +01:00