Commit Graph

188 Commits

Author SHA1 Message Date
Gilles Peskine
a8bf9867d4 all.sh: --keep-going mode
Add --keep-going mode to all.sh. In this mode, if a test fails, keep
running the subsequent tests. If a build fails, skip any tests of this
build and move on to the next tests. Errors in infrastructure, such as
git or cmake runs, remain fatal. Print an error summary at the end of
the run, and return a nonzero code if there was any failure.

In known terminal types, use color to highlight errors.

On a fatal signal, interrupt the run and report the errors so far.
2017-12-20 13:51:41 +01:00
Gilles Peskine
a6901f48cb all.sh: cleaned up usage output 2017-12-19 18:41:28 +01:00
Gilles Peskine
30dfbafb82 all.sh: indent 2017-12-19 18:37:30 +01:00
Manuel Pégourié-Gonnard
86bc448e75 Merge remote-tracking branch 'public/pr/1118' into mbedtls-2.1
* public/pr/1118:
  Allow comments in test data files
2017-12-19 12:19:52 +01:00
Gilles Peskine
49349bacb9 Merge remote-tracking branch 'upstream-public/pr/1153' into mbedtls-2.1 2017-11-29 20:53:58 +01:00
Gilles Peskine
25aa833ac3 Merge branch 'pr_1082' into mbedtls-2.1 2017-11-28 18:33:50 +01:00
Hanno Becker
9293592843 Add test case for RSA_NO_CRT to all.sh 2017-11-06 15:07:09 +00:00
Hanno Becker
4f9973efb9 Add build and ssl-opt.sh run for !SSL_RENEGOTIATION to all.sh 2017-10-24 11:56:28 +01:00
Hanno Becker
797c084394 Add tests for disabled MFL-extension to all.sh
This commit adds a build with default config except
MBEDTLS_SSL_MAX_FRAGMENT_LENGTH to all.sh, as well as a run of the MFL-related
tests in ssl-opt.sh.
2017-10-19 15:49:21 +01:00
Gilles Peskine
e38900b1a1 Allow comments in test data files 2017-10-02 11:26:09 +02:00
Hanno Becker
276d530abe Support negative dependencies in test cases
The entropy test suite uses a negative dependency "depends_on:!CONFIG_FLAG" for one of its tests. This kind of
dependency (running a test only if some configuration flag is not defined) is currently not supported and instead
results in the respective test case being dropped.

This commit adds support for negative dependencies in test cases.
2017-09-08 10:13:57 +01:00
Manuel Pégourié-Gonnard
b55f613601 Merge remote-tracking branch 'gilles/iotssl-1223/mbedtls-2.1' into mbedtls-2.1
* gilles/iotssl-1223/mbedtls-2.1:
  Fix FALLBACK_SCSV parsing
2017-06-06 20:12:51 +02:00
Gilles Peskine
b49351d1aa all.sh: test with SHA-1 enabled
Enabling SHA-1 for certificates is deprecated but we still want it to work.

Thanks to @andresag01
2017-06-06 19:09:04 +02:00
Gilles Peskine
39e2981b22 Fix FALLBACK_SCSV parsing
Fixed a bug in ssl_srv.c when parsing TLS_FALLBACK_SCSV in the
ciphersuite list that caused it to miss it sometimes. Reported by Hugo
Leisink as issue #810. Fix initially by @andreasag01; this commit
isolates the bug fix and adds a non-regression test.
2017-05-16 18:03:02 +02:00
Andres Amaya Garcia
d8fcfe7850 Fix check-doxy-blocks.pl to run from root dir
Modify tests/scripts/check-doxy-blocks.pl to ensure that:
  * It can only be run from the mbed TLS root directory.
  * An error code is returned to the environment when a potential error
    in the source code is found.
2017-05-13 19:22:03 +01:00
Andres AG
f083b31fb3 Fix generate_code.pl to handle escaped : 2017-02-02 14:43:44 +00:00
Simon Butcher
d7f1902342 Updated test script all.sh
Changes to increase the release test coverage and also allow testing with
two different versions of OpenSSL and GNUTLS for legacy features.
2016-10-14 10:48:17 +01:00
Simon Butcher
02b8d481f6 Adapt SSLv3 test configuration for new default
Removed SSLv3 test suites from system tests in 'compat.sh' script, and
added SSLv3 specific tests to 'all.sh'
2016-03-15 20:39:52 +00:00
Manuel Pégourié-Gonnard
f10f85f676 Add test for yotta debug build 2016-01-08 15:08:49 +01:00
Manuel Pégourié-Gonnard
bb81b4a009 Make ar invocation more portable
armar doesn't understand the syntax without dash. OTOH, the syntax with dash
is the only one specified by POSIX, and it's accepted by GNU ar, BSD ar (as
bundled with OS X) and armar, so it looks like the most portable syntax.

fixes #386
2016-01-08 14:52:14 +01:00
Simon Butcher
73156357ed Disable Yotta tests from 'all tests' script
Yotta tests not supported in 2.1 branch
2015-11-04 00:36:30 +00:00
Manuel Pégourié-Gonnard
d13585f1b3 Small improvement to test script 2015-10-30 16:56:30 +01:00
Manuel Pégourié-Gonnard
134ca18fbc Add key-exchanges.pl to test list 2015-10-28 13:17:18 +01:00
Manuel Pégourié-Gonnard
fe3affdad2 Add -Werror to reduced configs test scripts 2015-10-28 13:17:08 +01:00
Manuel Pégourié-Gonnard
4b56e755af Add script to test configs with single key exchanges 2015-10-28 13:15:23 +01:00
Manuel Pégourié-Gonnard
c4cbc94d44 Small fix to 'make test' script
When the tests fail they don't display the number of skipped and run test
2015-10-27 10:29:26 +01:00
Manuel Pégourié-Gonnard
2f056a0aee Try to run yotta update for yotta build test
But accept failures in case we're offline
2015-09-18 14:37:54 +02:00
Manuel Pégourié-Gonnard
e33316c607 Add test build of shared libs for windows 2015-08-07 13:22:37 +02:00
Manuel Pégourié-Gonnard
32da9f66a8 Add support for MBEDTLS_USER_CONFIG_FILE 2015-08-06 09:57:54 +02:00
Manuel Pégourié-Gonnard
77d56bb6c2 Add yotta test builds to all.sh 2015-08-06 09:57:53 +02:00
Manuel Pégourié-Gonnard
bf6ed08aaa Fix list-symbols.sh
- make it work on Linux
- use all three libraries
2015-08-04 17:46:21 +02:00
Manuel Pégourié-Gonnard
9afdc83d77 Fix bashisms in test scripts 2015-08-04 17:15:13 +02:00
Manuel Pégourié-Gonnard
e88b49323e Fix run-test-suite.pl with coverage 2015-07-15 12:31:12 +02:00
Manuel Pégourié-Gonnard
1780f89ecf Some more scripts simplified
make now has an alias test = check
2015-07-08 22:08:02 +01:00
Manuel Pégourié-Gonnard
3d404b4763 Simplify some scripts
ssl-opt.sh and compat.sh can now be run from root too
2015-07-08 21:59:16 +01:00
Manuel Pégourié-Gonnard
85113848bd Use Perl to run tests suites with make
- works on Windows too!
- we need Perl to generate/build the test suites anyway
- easier & more flexible (eg, now count total number of tests run)
2015-07-08 21:36:06 +01:00
Manuel Pégourié-Gonnard
ae738c29eb Fix warning from Perl 5.21 in helper script 2015-07-01 19:32:00 +02:00
Manuel Pégourié-Gonnard
6dc2651f02 Cosmetics 2015-06-25 10:59:57 +02:00
Manuel Pégourié-Gonnard
9b06abe1d1 Add a shared build with make on Linux to all.sh 2015-06-25 10:59:57 +02:00
Manuel Pégourié-Gonnard
e058ea2ed8 More consistent toolchains in all.sh 2015-06-25 10:59:57 +02:00
Manuel Pégourié-Gonnard
52fa38a214 Better cleanup in all.sh 2015-06-23 18:44:10 +02:00
Manuel Pégourié-Gonnard
bcf13bab5d Fix issue with MemSan and entropy
Due to the recent change about entropy sources strength, it is no longer
acceptable to just disable the platform source. So, instead "fix" it so that
it is clear to MemSan that memory is initialized.

I tried __attribute__((no_sanitize_memory)) and MemSan's blacklist file, but
couldn't seem to get them to work.
2015-06-22 18:25:41 +02:00
Manuel Pégourié-Gonnard
bbc60db221 Adjustments for armcc in all.sh
- no HAVE_TIME -> no HAVE_TIME_DATE
- check if --help works, so that the test is cleanly skipped if license server
  cannot be reached
2015-06-22 14:41:37 +02:00
Manuel Pégourié-Gonnard
3d4755bec4 Merge branch 'mbedtls-1.3' into development
* mbedtls-1.3:
  Fix all.sh for recent config.pl change

Conflicts:
	tests/scripts/all.sh
2015-06-03 14:03:17 +01:00
Manuel Pégourié-Gonnard
721e6bbf71 Fix all.sh for recent config.pl change 2015-06-03 13:38:20 +01:00
Manuel Pégourié-Gonnard
7ee5ddd798 Merge branch 'mbedtls-1.3' into development
* mbedtls-1.3:
  Fix compile errors with NO_STD_FUNCTIONS
  Expand config.pl's notion of "full"
  Ack external bugfix in Changelog
  FIx misplaced Changelog entry (oops)
  Fix compile bug: incompatible declaration of polarssl_exit in platform.c
  Fix contributor's name in Changelog
2015-06-03 10:33:55 +01:00
Manuel Pégourié-Gonnard
dccb80b7e5 Fix compile errors with NO_STD_FUNCTIONS 2015-06-03 10:20:33 +01:00
Manuel Pégourié-Gonnard
718593681e Use -std=c99 only for library 2015-06-02 16:39:22 +01:00
Manuel Pégourié-Gonnard
f78e4de6f4 Fix warnings from -pedantic 2015-05-29 10:52:14 +02:00
Manuel Pégourié-Gonnard
009a2640f6 Add test build with -std=c99 2015-05-29 10:31:13 +02:00
Manuel Pégourié-Gonnard
6ad5d35ba9 Catch a few more likely typos
I often mistype MBEDLTS for MBEDTLS
2015-05-28 17:28:38 +02:00
Manuel Pégourié-Gonnard
66b8e956bb Add test build with all except ssl_cli/srv.c 2015-05-20 11:14:58 +02:00
Manuel Pégourié-Gonnard
5a8d56d3a3 Rm hard dependency of DTLS on TIMING_C 2015-05-13 10:10:00 +02:00
Manuel Pégourié-Gonnard
e2650c8238 Merge branch 'mbedtls-1.3' into development
* commit '23c0608':
  Fix bug in generate_code.pl
  Fix typo in contributor name (oops!)
2015-04-17 20:39:50 +02:00
Manuel Pégourié-Gonnard
23c0608e28 Fix bug in generate_code.pl
The following did fail:

Test 1
foo:SOME_CONSTANT:"string"

Test 2
foo:OTHER_CONSTANT:"string"

due to the first string actually including the second "foo" up to (but no
including) the colon.
2015-04-17 17:24:25 +02:00
Manuel Pégourié-Gonnard
d09a6b53b5 Move test scripts to tests/scripts 2015-04-10 11:34:22 +02:00
Manuel Pégourié-Gonnard
f9aae834a2 Add test for typos in check-names.sh 2015-04-09 13:52:55 +02:00
Manuel Pégourié-Gonnard
a687baf195 Add check-names.sh to all.sh 2015-04-09 13:52:55 +02:00
Manuel Pégourié-Gonnard
e137ea6f64 Add test script check-names.sh 2015-04-09 10:53:11 +02:00
Manuel Pégourié-Gonnard
d1ddd29e81 Adapt error messages in tests/scripts/list-* 2015-04-09 10:15:10 +02:00
Manuel Pégourié-Gonnard
0edba1a8ee Move temporary list scripts to tests/scripts 2015-04-09 10:12:44 +02:00
Manuel Pégourié-Gonnard
2cf5a7c98e The Great Renaming
A simple execution of tmp/invoke-rename.pl
2015-04-08 13:25:31 +02:00
Manuel Pégourié-Gonnard
8a89fcd13f Rm references to removed config options
POLARSSL_MEMORY_C POLARSSL_PBKDF2_C DISABLE_RENEGO
2015-04-03 18:46:55 +02:00
Manuel Pégourié-Gonnard
ba2e72c007 Merge branch 'mbedtls-1.3' into development
* mbedtls-1.3:
  Fix issue with armcc test in all.sh
2015-03-25 18:09:21 +01:00
Manuel Pégourié-Gonnard
f1002f8582 Fix issue with armcc test in all.sh 2015-03-25 17:09:16 +01:00
Manuel Pégourié-Gonnard
ea0920f079 Adjust test scripts to new RC4 defaults 2015-03-24 10:14:23 +01:00
Manuel Pégourié-Gonnard
29c28f961d Merge branch 'mbedtls-1.3' into development
* mbedtls-1.3:
  Fix gcc detection in check_config.h
2015-03-23 15:27:25 +01:00
Manuel Pégourié-Gonnard
757ca00396 Fix gcc detection in check_config.h 2015-03-23 15:24:07 +01:00
Manuel Pégourié-Gonnard
8a80318df2 Merge branch 'mbedtls-1.3' into development
* mbedtls-1.3:
  Update generated file
  Update Changelog for deprecation config flags
  Fix tests to work with DEPRECATED_REMOVED
  Add POLARSSL_DEPRECATED_{WARNING,REMOVED}
  Suppress clang warning we don't want
2015-03-23 14:31:25 +01:00
Manuel Pégourié-Gonnard
e46c6c38c9 Fix tests to work with DEPRECATED_REMOVED 2015-03-23 14:11:11 +01:00
Manuel Pégourié-Gonnard
0bf112e7dc Remove small configuration based on RC4 2015-03-20 17:46:40 +00:00
Manuel Pégourié-Gonnard
b295e4fdc4 Fix duplication in all.sh 2015-03-19 13:40:37 +00:00
Manuel Pégourié-Gonnard
cc0d084820 Merge branch 'mbedtls-1.3' into development
* mbedtls-1.3:
  Actually use armcc for the armcc test ^^'
  Add more -O level variety in all.sh
  Document recent make changes
  build: Makefile: cleanup CFLAGS
  build: Makefile: cleanup LDFLAGS
  build: Makefile: simplify root Makefile
  build: Makefile: remove bashism

Conflicts:
	programs/Makefile
2015-03-13 16:32:40 +00:00
Manuel Pégourié-Gonnard
129e41322b Actually use armcc for the armcc test ^^' 2015-03-13 17:29:18 +01:00
Manuel Pégourié-Gonnard
61fe8b0290 Add more -O level variety in all.sh 2015-03-13 14:33:16 +00:00
Manuel Pégourié-Gonnard
7f8099773e Rename include directory to mbedtls 2015-03-10 11:23:56 +00:00
Manuel Pégourié-Gonnard
129db08c90 Rm polarssl compat targets from Makefiles 2015-03-10 11:23:56 +00:00
Manuel Pégourié-Gonnard
b251a20a7e Merge branch 'development' into dtls
* development:
  Fix missing/misplaced #include's
2015-02-17 15:48:15 +00:00
Manuel Pégourié-Gonnard
981732bb8e Fix missing/misplaced #include's 2015-02-17 15:47:31 +00:00
Manuel Pégourié-Gonnard
01bb5f9a00 Update dependencies in all.sh 2015-02-17 09:36:15 +00:00
Manuel Pégourié-Gonnard
6448bceeb6 Add mingw cross-compile test to all.sh 2015-02-16 17:18:36 +01:00
Manuel Pégourié-Gonnard
6ca4076468 Update all.sh for new dependency 2015-02-13 15:57:35 +00:00
Manuel Pégourié-Gonnard
b3b8e43a12 New test script for generated files 2015-02-13 14:53:53 +00:00
Manuel Pégourié-Gonnard
a71780e475 Add test for no platform.c in all.sh 2015-02-13 13:56:55 +00:00
Manuel Pégourié-Gonnard
c5c5939577 Add armcc to all.sh if available 2015-02-10 17:38:54 +01:00
Manuel Pégourié-Gonnard
edb2dc93c6 Add some cross-compile runs to all.sh 2015-02-10 14:36:31 +00:00
Rich Evans
00ab47026b cleanup library and some basic tests. Includes, add guards to includes 2015-02-10 11:28:46 +00:00
Rich Evans
920aa9c8ee change more references to (f)printf to polarssl_(f)printf to allow overriding 2015-01-28 15:28:28 +01:00
Rich Evans
f4253c72f0 modify tests/scripts/generate_code.pl to allow overriding of from command line 2015-01-28 15:28:27 +01:00
Manuel Pégourié-Gonnard
392d3dde0b Adapt all.sh for OS X (no MemSan) 2015-01-26 16:44:38 +00:00
Manuel Pégourié-Gonnard
1e77a96683 Fix memsan false positive on Linux >= 3.17 2015-01-26 16:44:37 +00:00
Manuel Pégourié-Gonnard
e4f6edcda1 Rename to mbed TLS in tests/ 2015-01-22 16:43:54 +00:00
Paul Bakker
d7e2483bfc Merge miscellaneous fixes into development 2015-01-13 16:04:38 +01:00
Manuel Pégourié-Gonnard
76c99a01a1 Fix Gnu-ism in script 2014-12-11 10:33:43 +01:00
Manuel Pégourié-Gonnard
ea29d152c7 Add recursion.pl to all.sh 2014-11-20 17:32:33 +01:00
Manuel Pégourié-Gonnard
89d69b398c Fix 3DES -> DES in all.sh (+ time estimates) 2014-11-20 16:36:08 +01:00
Manuel Pégourié-Gonnard
246978d97d Add curves.pl to all.sh 2014-11-20 16:36:08 +01:00
Manuel Pégourié-Gonnard
9bda9b3b92 Rework all.sh to use MSan instead of valgrind 2014-11-20 16:36:08 +01:00
Manuel Pégourié-Gonnard
2727dc1e09 Add script to test depends on individual curves 2014-11-20 16:36:08 +01:00
Manuel Pégourié-Gonnard
61bc57af99 Optimize all.sh for new build options 2014-08-14 18:33:00 +02:00