Commit Graph

459 Commits

Author SHA1 Message Date
Jaeden Amero
5758d8cb9c test: Print verbosely on failures in verbose mode
Update the test runner to print detail about why the test failed when it
fails, if the runner is running in verbose mode.
2018-11-02 17:30:24 +00:00
Simon Butcher
ec3f9c362d Merge remote-tracking branch 'public/pr/2032' into mbedtls-2.7 2018-10-27 18:34:46 +01:00
Hanno Becker
72311b468d Add test for MBEDTLS_PLATFORM_{CALLOC/FREE}_MACRO to all.sh
This commit adds a test to tests/scripts/all.sh exercising an
ASan build of the default configuration with

MBEDTLS_PLATFORM_MEMORY enabled,
MBEDTLS_PLATFORM_CALLOC_MACRO set to std calloc
MBEDTLS_PLATFORM_FREE_MACRO   set to std free

(This should functionally be indistinguishable from a default build)
2018-10-11 11:10:09 +01:00
Gilles Peskine
3400b4dbd6 check-files: exclude .git and third-party files
Exclude ".git" directories anywhere. This avoids spurious errors in git
checkouts that contain branch names that look like a file
check-files.py would check. Fix #1713

Exclude "mbed-os" anywhere and "examples" from the root. Switch to the
new mechanism to exclude "yotta/module". These are directories where
we store third-party files that do not need to match our preferences.

Exclude "cov-int" from the root. Fix #1691
2018-10-02 13:14:58 +02:00
Simon Butcher
eb219391fa Fix run-test-suites.pl to screen for files
Changes run-test-suites.pl to filter out directories, and select only files
as on OSX, test coverage tests create .dSYM directories which were being
accidentally selected to execute.
2018-09-30 21:57:34 +01:00
Simon Butcher
d620f6f56b Adds a filter to run-test-suites.pl to exclude data files
The run-test-suites.pl script was executing all files of the form 'test_suite*'
which were either executable or ended with a .exe extension.

On some filesystems, such as through network shares or VMs, which are
abstracting one set of file permissions to Unix permissions, may set the
executable permissions on all files, whether they're executable or not.

That was leading to the run-test-suites.pl script to attempt to execute the .c
intermediate files because they followed the form 'test_suite_*.c'. This change
now excludes them, just in case they accidentally have execute permissions.
2018-09-27 17:29:44 +01:00
Gilles Peskine
b46c59e2a7 In keep-going mode, don't hard-fail on some tests
Add if_build_succeeded in front of the invocation of some test runs
where it was missing.
2018-09-27 10:12:58 +02:00
Gilles Peskine
732826d265 In keep-going mode, don't hard-fail on some auxiliary script
Add record_status in front of the invocation of several scripts where
it was missing.
2018-09-27 10:08:31 +02:00
Simon Butcher
d7126d7009 Merge remote-tracking branch 'public/pr/779' into mbedtls-2.7 2018-07-24 13:38:44 +01:00
Simon Butcher
bd40916dfa Merge remote-tracking branch 'public/pr/1872' into mbedtls-2.7 2018-07-24 12:12:43 +01:00
Simon Butcher
948f264302 Add additional i386 tests to all.sh
Added an additional i386 test to all.sh, to allow one test with -O0 which
compiles out inline assembly, and one to test with -01 which includes the inline
assembly.
2018-07-23 13:41:25 +01:00
Jaeden Amero
5113bdec6e all.sh: Return error on keep-going failure
When calling all.sh from a script and using "--keep-going", errors were
sometimes missed due to all.sh always returning 0 "success" return code.
Return 1 if there is any failure encountered during a "keep-going" run.
2018-07-23 10:24:31 +01:00
Simon Butcher
7c6b84102d Expand i386 all.sh tests to full config ASan builds
The i386 test builds were only building the default configuration and had
no address sanitisation. This commit expands the test configuration to the full
configuration in all.sh and builds with ASan for when the test suites are
executed.
2018-07-20 21:34:04 +01:00
Simon Butcher
0e342f77fc Merge remote-tracking branch 'public/pr/1390' into mbedtls-2.7 2018-06-27 11:11:34 +01:00
Simon Butcher
f15cfd5d04 Merge remote-tracking branch 'public/pr/1557' into mbedtls-2.7 2018-06-27 11:07:50 +01:00
Simon Butcher
8c83673eb2 Merge remote-tracking branch 'public/pr/1708' into mbedtls-2.7 2018-06-12 17:26:55 +01:00
Simon Butcher
856870952a Merge remote-tracking branch 'public/pr/1709' into mbedtls-2.7 2018-06-12 17:25:19 +01:00
Simon Butcher
ee3a3d4a72 Merge remote-tracking branch 'public/pr/1470' into mbedtls-2.7 2018-06-11 11:30:33 +01:00
Simon Butcher
bb5e1c3973 Fix multiple quality issues in the source
This PR fixes multiple issues in the source code to address issues raised by
tests/scripts/check-files.py. Specifically:
 * incorrect file permissions
 * missing newline at the end of files
 * trailing whitespace
 * Tabs present
 * TODOs in the souce code
2018-06-08 11:14:43 +01:00
Darryl Green
bd38c3b89f Add check-files.py to all.sh 2018-06-05 11:57:12 +01:00
Darryl Green
da02eb310c Add script for source integrity checking 2018-06-05 11:57:01 +01:00
Simon Butcher
e83b1ae201 Merge remote-tracking branch 'public/pr/1606' into mbedtls-2.7 2018-06-01 19:34:44 +01:00
Jaeden Amero
11d5551d0a Merge remote-tracking branch 'upstream-public/pr/1487' into mbedtls-2.7-proposed 2018-05-04 11:06:21 +01:00
Andres AG
b7b420b51c Fix uninitialized var in check-generated-files.sh 2018-05-01 21:01:22 +01:00
Andres Amaya Garcia
7dae108fe8 Check generated-visualc-files in check-generated-files 2018-05-01 21:01:18 +01:00
fbrosson
3a7457136e Backport 2.7: Use "#!/usr/bin/env perl" as shebang line. 2018-04-04 22:26:56 +00:00
Gilles Peskine
595c84a7b1 Merge remote-tracking branch 'upstream-public/pr/1500' into mbedtls-2.7-proposed 2018-04-01 12:41:29 +02:00
Andres Amaya Garcia
28d97e1dfc Fix shared library lookup on Mac OS X when running tests 2018-03-27 20:04:20 +01:00
Gilles Peskine
763da6e550 all.sh --keep-going: properly handle multiple-builds scripts
In keep-going mode, if a multiple-builds script fails, record its
status and keep going.
2018-03-23 02:18:33 +01:00
Gilles Peskine
2cfeb887b4 Merge tag 'mbedtls-2.7.2' into iotssl-1381-x509-verify-refactor-2.7-restricted
Conflict resolution:

* ChangeLog
* tests/data_files/Makefile: concurrent additions, order irrelevant
* tests/data_files/test-ca.opensslconf: concurrent additions, order irrelevant
* tests/scripts/all.sh: one comment change conflicted with a code
  addition. In addition some of the additions in the
  iotssl-1381-x509-verify-refactor-restricted branch need support for
  keep-going mode, this will be added in a subsequent commit.
2018-03-23 02:12:44 +01:00
Gilles Peskine
0114ffc76b all.sh: Verify out-of-tree testing with CMake
Run a test case in ssl-opt.sh to validate that testing works in an
out-of-tree CMake build.
2018-03-21 12:29:20 +01:00
Gilles Peskine
a71d64c74f all.sh: fix cleanup happening during an out-of-tree build 2018-03-21 12:29:15 +01:00
Gilles Peskine
31b07e2833 all.sh: be more conservative when cleaning up CMake artefacts
Only delete things that we expect to find, to avoid deleting other
things that people might have lying around in their build tree.
Explicitly skip .git to avoid e.g. accidentally matching a branch
name.
2018-03-21 12:29:08 +01:00
Gilles Peskine
19ceb7104c all.sh: add opposites to all boolean options
All options can now be overridden by a subsequent option, e.g.
"all.sh --foo --no-foo" is equivalent to "all.sh --no-foo". This
allows making wrapper scripts with default options and occasionally
overriding those options when running the wrapper script.
2018-03-21 08:48:40 +01:00
Gilles Peskine
54933e95bd all.sh: option parsing: reduce vertical spread
Only whitespace changes.
2018-03-21 08:48:33 +01:00
Gilles Peskine
53038ebecc all.sh: with --no-armcc, don't call armcc from output_env.sh
When not running armcc, don't try to invoke armcc at all, not even to
report its version.
2018-03-21 08:48:26 +01:00
Gilles Peskine
955d70459d Merge remote-tracking branch 'upstream-restricted/pr/460' into mbedtls-2.7-restricted-proposed 2018-03-13 17:24:33 +01:00
Gilles Peskine
4848b97bc7 Merge remote-tracking branch 'upstream-public/pr/1249' into mbedtls-2.7-proposed 2018-03-11 00:48:17 +01:00
Hanno Becker
69d45cce5d Add a run with RSA_NO_CRT to all.sh 2018-03-09 10:46:23 +00:00
Manuel Pégourié-Gonnard
fa973e022a Document choice of script exit code 2018-03-05 13:23:27 +01:00
Gert van Dijk
ab41f04554 Tests: depends-pkalgs.pl - disable less options
Rather than disabling SSL & Key exchanges as a whole, only disable those
options required by reverse dependencies.

GitHub issue #1040 https://github.com/ARMmbed/mbedtls/issues/1040
See also discussion in PR #1074.
https://github.com/ARMmbed/mbedtls/pull/1074#issuecomment-327096303
2018-03-05 13:20:36 +01:00
Manuel Pégourié-Gonnard
fcc4348ee2 Improve some comments 2018-03-05 13:20:21 +01:00
Manuel Pégourié-Gonnard
2f1633e002 Improve some comments, fix some typos 2018-03-05 13:19:41 +01:00
Manuel Pégourié-Gonnard
b26b28a7e4 Make some perl scripts usable with git bisect run
For that they need to return between 0 and 124 on error, while die returns
255, causing bisect-run to abort.
2018-03-05 13:16:15 +01:00
Manuel Pégourié-Gonnard
4d9e7cb66b Fix usage of CFLAGS with cmake in all.sh
With cmake, CFLAGS has to be set when invoking cmake, not make (which totally
ignores the value of CFLAGS when it runs and only keeps the one from cmake).

Also, in that case the flags were either redundant (-Werror etc) or wrong
(-std=c99 -pedantic) as some parts of the library will not build with
-pedantic (see the other -pedantic tests, which are correct, for what needs to
be disabled).
2018-03-05 13:00:03 +01:00
Manuel Pégourié-Gonnard
1ddd682843 Fix depends_on:pk_alg in test suites 2018-03-05 12:58:51 +01:00
Manuel Pégourié-Gonnard
153b5ef696 Add new test script depends-pkalgs.pl 2018-03-05 12:58:42 +01:00
Manuel Pégourié-Gonnard
056eab5ff0 Fix usage of {curves,key-exchanges}.pl in all.sh 2018-03-05 12:58:22 +01:00
Manuel Pégourié-Gonnard
483a776019 Fix missing depends_on:SHA/MD in x509 tests 2018-03-05 12:57:36 +01:00
Manuel Pégourié-Gonnard
73fc679293 Add new test script depends-hashes.pl
This is step 1 of a plan to get rid once and for all of missing depends_on in
the X509 test suite (step 2 will be RSA/ECDSA, and step 0 was curves.pl).
2018-03-05 12:57:16 +01:00
Gilles Peskine
b4ef45b4f5 all.sh: build with -O -Werror in the full config
Build with MBEDTLS_DEPRECATED_REMOVED and MBEDTLS_DEPRECATED_WARNING
separately.

Do these builds with `-O -Werror -Wall -Wextra` to catch a maximum of
issues while we're at it. Do one with gcc and one with clang for
variety. This caught an uninitialized variable warning in cmac.c that
builds without -O didn't catch.
2018-03-01 22:23:50 +01:00
Gilles Peskine
0afe624fff all.sh: add test with MBEDTLS_DEPRECATED_REMOVED 2018-02-21 19:28:12 +01:00
Gilles Peskine
14c3c0610e Test with 32-bit and 64-bit bignum limbs on all architectures
Build with MBEDTLS_HAVE_INT32 and MBEDTLS_HAVE_INT64 on all
architectures, not just x86_64. These two modes should work on all
platforms (except embedded environments where 64-bit division is not
available).

Also run the unit tests.

Correct the description: this is not "N-bit compilation", but "N-bit
bignum limbs".
2018-01-29 21:31:16 +01:00
Gilles Peskine
48e689e6be Remove duplicate build run
Don't compile twice with MBEDTLS_HAVE_INT64.

But do test with MBEDTLS_HAVE_INT32.
2018-01-29 21:31:16 +01:00
Andres Amaya Garcia
f4fbdda602 Add test command for 64-bit ILP32 in all.sh 2018-01-29 20:51:55 +01:00
Hanno Becker
d485c319a5 Make small corrections to all.sh
Correct gcc flags in !MBEDTLS_SSL_CLI_C test (preexisting) and build and test
for RSA_NO_CRT in ASan mode.
2018-01-05 13:03:53 +00:00
Hanno Becker
88683b2c6d Correct all.sh and config.h after merge commit
- Adapt the change in all.sh to the new keep-going mode
- Restore alphabetical order of configuration flags for
  alternative implementations in config.h and rebuild
  library/version_features.c
2018-01-04 18:48:32 +00:00
Hanno Becker
8bc74d6f2f Merge branch 'development' into iotssl-1619 2018-01-03 10:24:02 +00:00
Hanno Becker
e963efa110 Don't limit RSA_NO_CRT test in all.sh to 64-bit systems
Compilation and test for the `MBEDTLS_RSA_NO_CRT` option were
previously guarded by a check for 64-bit systems, for which there
is no reason. This commit moves both outside of the guard.
2018-01-03 10:03:43 +00:00
Gilles Peskine
9736b9d59a all.sh --keep-going: work if TERM is unset 2018-01-02 21:54:17 +01:00
Hanno Becker
32297e8314 Merge branch 'development' into iotssl-1619 2017-12-22 10:24:32 +00:00
Gilles Peskine
192c72f7a1 all.sh: add some documentation 2017-12-21 16:54:14 +01:00
Gilles Peskine
bca6ab9d38 all.sh: new option --no-armcc
With this option, don't run anything that requires armcc or yotta, so
the script can run offline.
2017-12-21 15:21:31 +01:00
Gilles Peskine
2a22a8041c all.sh: add --yotta to go with --no-yotta
Add --yotta which is currently a no-op but may not remain so if we
decide to make no-yotta the default in the future.
2017-12-21 15:19:00 +01:00
Gilles Peskine
7c6521688a all.sh: --keep-going mode
Add --keep-going mode to all.sh. In this mode, if a test fails, keep
running the subsequent tests. If a build fails, skip any tests of this
build and move on to the next tests. Errors in infrastructure, such as
git or cmake runs, remain fatal. Print an error summary at the end of
the run, and return a nonzero code if there was any failure.

In known terminal types, use color to highlight errors.

On a fatal signal, interrupt the run and report the errors so far.
2017-12-21 15:17:21 +01:00
Gilles Peskine
709346aed8 all.sh: cleaned up usage output 2017-12-21 15:16:11 +01:00
Gilles Peskine
7ad603e662 all.sh: indent 2017-12-21 15:13:27 +01:00
Gilles Peskine
da519251d4 Add --no-yotta option to all.sh
The Yotta tools break in some environments and it's useful to be able
to run the rest of all.sh nonetheless.
2017-11-30 14:24:33 +01:00
Gilles Peskine
0884f4811b Merge remote-tracking branch 'upstream-public/pr/1141' into development 2017-11-29 20:50:59 +01:00
Gilles Peskine
4daffe236a Merge branch 'pr_1025' into development
Merge PR #1025 + ChangeLog entry
2017-11-28 18:23:53 +01:00
Gilles Peskine
68306ed31f Merge remote-tracking branch 'upstream-public/pr/1094' into development 2017-11-23 20:02:46 +01:00
Hanno Becker
134c2ab891 Add build and ssl-opt.sh run for !SSL_RENEGOTIATION to all.sh 2017-10-17 11:03:50 +01:00
Gilles Peskine
b04e2c3d81 Allow comments in test data files 2017-10-06 11:58:50 +01:00
Gilles Peskine
26182edd0c Allow comments in test data files 2017-09-29 15:45:12 +02:00
Hanno Becker
5175ac6e13 Add tests for disabled MFL-extension to all.sh
This commit adds a build with default config except
MBEDTLS_SSL_MAX_FRAGMENT_LENGTH to all.sh, as well as a run of the MFL-related
tests in ssl-opt.sh.
2017-09-18 16:11:39 +01:00
Hanno Becker
bf37b10370 Add test run for RSA_NO_CRT to all.sh 2017-08-23 16:17:28 +01:00
Ron Eldor
bf007d297d Pre push hook script
Add git_hook folder, and pre-push script,
to be soft linked from .git/hooks/pre-push
2017-07-27 21:44:34 +01:00
Simon Butcher
2c4d558873 Fixes test for MBEDTLS_NO_UDBL_DIVISION
The test for MBEDTLS_NO_UDBL_DIVISION wasn't preserving it's own config.h
for the next test.

Also added comments to ARM Compiler 6 tests to better explain them.
2017-07-27 21:44:34 +01:00
Andres Amaya Garcia
f755bb3adf Remove MBEDTLS_TYPE_UDBL tests from all.sh 2017-07-27 21:44:33 +01:00
Andres Amaya Garcia
465db7eba1 Fix no 64-bit division test in all.sh 2017-07-27 21:44:33 +01:00
Andres Amaya Garcia
9946783218 Add tests for 64 and 32-bit int types compilation 2017-07-27 21:44:33 +01:00
Gilles Peskine
b1a977f5a7 MBEDTLS_NO_INT64_DIVISION -> MBEDTLS_NO_UDBL_DIVISION
Changed the option to disable the use of 64-bit division, to an option
to disable the use of double-width division, whether that's 64 or 128-bit.
2017-07-27 21:44:33 +01:00
Andres Amaya Garcia
5e873fb464 Add all.sh test to force 32-bit compilation 2017-07-27 21:44:33 +01:00
Ron Eldor
fcb7491a49 Pre push hook script
Add git_hook folder, and pre-push script,
to be soft linked from .git/hooks/pre-push
2017-07-24 14:25:26 +02:00
Simon Butcher
51aaa99473 Fixes test for MBEDTLS_NO_UDBL_DIVISION
The test for MBEDTLS_NO_UDBL_DIVISION wasn't preserving it's own config.h
for the next test.

Also added comments to ARM Compiler 6 tests to better explain them.
2017-07-23 13:42:36 +02:00
Hanno Becker
f058f34b5a Support negative dependencies in test cases
The entropy test suite uses a negative dependency "depends_on:!CONFIG_FLAG" for one of its tests. This kind of
dependency (running a test only if some configuration flag is not defined) is currently not supported and instead
results in the respective test case being dropped.

This commit adds support for negative dependencies in test cases.
2017-07-23 10:41:04 +01:00
Andres Amaya Garcia
c327aa1542 Remove MBEDTLS_TYPE_UDBL tests from all.sh 2017-07-22 11:53:56 +02:00
Andres Amaya Garcia
6fb65864a2 Fix no 64-bit division test in all.sh 2017-07-22 11:53:56 +02:00
Andres Amaya Garcia
33264d7a96 Add tests for 64 and 32-bit int types compilation 2017-07-22 11:53:56 +02:00
Gilles Peskine
9a9adcd6aa MBEDTLS_NO_INT64_DIVISION -> MBEDTLS_NO_UDBL_DIVISION
Changed the option to disable the use of 64-bit division, to an option
to disable the use of double-width division, whether that's 64 or 128-bit.
2017-07-22 11:53:56 +02:00
Andres Amaya Garcia
dd29c2f2c3 Add all.sh test to force 32-bit compilation 2017-07-22 11:53:56 +02:00
Manuel Pégourié-Gonnard
ddc6e52cc1 Merge remote-tracking branch 'gilles/iotssl-1223/development' into development
* gilles/iotssl-1223/development:
  Fix FALLBACK_SCSV parsing
2017-06-06 20:11:36 +02:00
Manuel Pégourié-Gonnard
383a118338 Merge remote-tracking branch 'gilles/IOTSSL-1330/development' into development
* gilles/IOTSSL-1330/development:
  Changelog entry for the bug fixes
  SSLv3: when refusing renegotiation, stop processing
  Ignore failures when sending fatal alerts
  Cleaned up double variable declaration
  Code portability fix
  Added changelog entry
  Send TLS alerts in many more cases
  Skip all non-executables in run-test-suites.pl
  SSL tests: server requires auth, client has no certificate
  Balanced braces across preprocessor conditionals
  Support setting the ports on the command line
2017-06-06 19:22:41 +02:00
Gilles Peskine
2a458daa11 all.sh: test with SHA-1 enabled
Enabling SHA-1 for certificates is deprecated but we still want it to work.

Thanks to @andresag01
2017-06-06 18:44:14 +02:00
Gilles Peskine
d50177fdf3 Fix FALLBACK_SCSV parsing
Fixed a bug in ssl_srv.c when parsing TLS_FALLBACK_SCSV in the
ciphersuite list that caused it to miss it sometimes. Reported by Hugo
Leisink as issue #810. Fix initially by @andreasag01; this commit
isolates the bug fix and adds a non-regression test.
2017-05-16 17:53:03 +02:00
Andres Amaya Garcia
d3f0f5e1c8 Fix check-doxy-blocks.pl to run from root dir
Modify tests/scripts/check-doxy-blocks.pl to ensure that:
  * It can only be run from the mbed TLS root directory.
  * An error code is returned to the environment when a potential error
    in the source code is found.
2017-05-12 00:16:50 +01:00
Gilles Peskine
071db41627 Skip all non-executables in run-test-suites.pl
The script was blacklisting extensions. Rather than keep adding to the
extensions, whitelist executables.
2017-05-03 16:26:47 +02:00
Simon Butcher
9cfdf2caa7 Merge branch 'development-restricted' 2017-02-15 23:55:22 +02:00
Simon Butcher
ba6aa63578 Fix all.sh test builds with recent glibc and clang
Fixes strict C99 builds in all.sh with glibc version >2.19 where platform support
wasn't being compiled in automatically.

Also fixes C99 syntax with armclang.
2017-02-15 23:25:08 +02:00
Simon Butcher
53716395b2 Clarify use of armcc in all.sh 2017-02-15 23:23:22 +02:00
Andres AG
0911f0880c Add -march argument to armc6 build tests 2017-02-15 23:23:22 +02:00
Andres AG
7f08d7a35a Modify output_env.sh to make it extensible 2017-02-15 23:23:22 +02:00
Andres AG
9e3fba09fd Add ARM Compiler 6 build tests to all.sh 2017-02-15 23:23:22 +02:00
Simon Butcher
1e6f5ac127 Make mingw test build a requirement of all.sh
Changed the mingw test build to be a required test of the all.sh script.
2017-02-15 09:08:26 +00:00
Simon Butcher
4ae4fdcd99 Fix mingw test build to avoid executing the tests
Changed the mingw build target to avoid building mingw test suites and
then attempting to run them which was failing on Linux.
2017-02-15 09:08:26 +00:00
Simon Butcher
9510cc1a18 Make the test builds much stricter for warnings
Tighten up the test options in all.sh, test-ref-configs.pl and curves.pl
to ensure the builds are strict for all warnings, warnings are treated
as errors, and that wherever possible builds are strict to the C99
standard. (Note that builds that use the Unix sockets API cannot be).
2017-02-15 09:08:26 +00:00
Andres AG
b9d3db68c6 Use MAKEFLAGS to pass args to make in all.sh
Modify the script at tests/scripts/all.sh to export the variable
MAKEFLAGS with -j if it was not set before. This should decrease the
total runtime of tests/scripts/all.sh by letting make run multiple jobs
in parallel. Also, add a check at the top of the script to cause a
failure if the environment is not Linux.
2017-02-15 09:08:26 +00:00
Andres AG
9060d4da08 Fix generate_code.pl to handle escaped : 2017-02-02 14:38:13 +00:00
Andres AG
9839360a10 Fix all.sh check_tools function to handle paths 2017-01-31 17:04:45 +00:00
Simon Butcher
cb587009d6 Fix all.sh test builds with recent glibc and clang
Fixes strict C99 builds in all.sh with glibc version >2.19 where platform support
wasn't being compiled in automatically.

Also fixes C99 syntax with armclang.
2017-01-06 16:14:44 +00:00
Simon Butcher
105e856143 Merge branch 'gcc-compiler-warnings' 2017-01-05 18:26:40 +00:00
Simon Butcher
49f00bd81d Clarify use of armcc in all.sh 2017-01-05 16:20:56 +00:00
Andres AG
a5cd973047 Add -march argument to armc6 build tests 2017-01-05 10:15:00 +00:00
Andres AG
31f9b5bdd8 Modify output_env.sh to make it extensible 2017-01-05 10:14:51 +00:00
Andres AG
87bb577cfb Add ARM Compiler 6 build tests to all.sh 2017-01-03 10:12:11 +00:00
Simon Butcher
710dd4fdd6 Merge branch 'speedup_tests'
Pull Request #533 - Use MAKEFLAGS to pass args to make in all.sh

Modify the script at tests/scripts/all.sh to export the variable
MAKEFLAGS with -j if it was not set before. This should decrease the
total runtime of tests/scripts/all.sh by letting make run multiple jobs
in parallel. Also, add a check at the top of the script to cause a
failure if the environment is not Linux.w
2016-12-30 19:09:40 +00:00
Simon Butcher
002bc6262b Make mingw test build a requirement of all.sh
Changed the mingw test build to be a required test of the all.sh script.
2016-11-17 09:27:45 +00:00
Simon Butcher
91aef33832 Fix mingw test build to avoid executing the tests
Changed the mingw build target to avoid building mingw test suites and
then attempting to run them which was failing on Linux.
2016-11-17 09:22:47 +00:00
Simon Butcher
f95c1764fa Make the test builds much stricter for warnings
Tighten up the test options in all.sh, test-ref-configs.pl and curves.pl
to ensure the builds are strict for all warnings, warnings are treated
as errors, and that wherever possible builds are strict to the C99
standard. (Note that builds that use the Unix sockets API cannot be).
2016-11-17 09:22:46 +00:00
Andres AG
7770ea82d5 Add seed cmdline arg to test scripts 2016-10-12 00:32:17 +01:00
Andres AG
b2fdd04f6a Fix scripts to support multiple openssl and gnutls 2016-09-27 16:04:44 +01:00
Andres AG
788aa4a812 Rename net.{c,h} to net_sockets.{c,h}
The library/net.c and its corresponding include/mbedtls/net.h file are
renamed to library/net_sockets.c and include/mbedtls/net_sockets.h
respectively. This is to avoid naming collisions in projects which also
have files with the common name 'net'.
2016-09-26 23:23:52 +01:00
Andres AG
38495a3824 Use MAKEFLAGS to pass args to make in all.sh
Modify the script at tests/scripts/all.sh to export the variable
MAKEFLAGS with -j if it was not set before. This should decrease the
total runtime of tests/scripts/all.sh by letting make run multiple jobs
in parallel. Also, add a check at the top of the script to cause a
failure if the environment is not Linux.
2016-09-22 17:01:13 +01:00
Simon Butcher
41eeccf76b Fix merge of OpenSSL/GNUTLS version options in all.sh 2016-09-15 20:45:53 +01:00
Andres AG
f07314e787 Allow cmake 'out-of-source' builds
Allow mbed TLS to be build in a subdirectory. Also add a test in
all.sh
2016-09-15 20:45:53 +01:00
Andres AG
d9eba4ba3d Allow alternate openssl and gnutls in all.sh
Allow the user to specify alternative openssl, gnutls-cli and
gnutls-serv binaries to execute legacy tests in all.sh.
2016-09-15 20:45:53 +01:00
Simon Butcher
8e3afc7332 Removes duplicated selftests from all.sh
All selftests are now called from the test suites so don't need to be executed
explicitly in tests/scripts/all.sh
2016-09-15 17:13:08 +01:00
Andres AG
dc192215f4 Allow cmake 'out-of-source' builds
Allow mbed TLS to be build in a subdirectory. Also add a test in
all.sh
2016-09-06 20:38:40 +01:00
Paul Bakker
4b8bc5228a Removed running selftest program from basic-build-test.sh
All self tests are run in the unit test framework as well.
2016-08-25 16:36:35 +01:00
Simon Butcher
4df5eaf2b0 Fixes build configuration for armcc test build in all.sh
In the test script tests/scripts/all.sh the armcc build is a baremetal build
and doesn't use MBEDTLS_HAVE_TIME therefore the test configuration
MBEDTLS_PLATFORM_TIME_ALT which is dependent on it must be unset.
2016-08-24 22:58:31 +03:00
Simon Butcher
a4ed19c7c0 Fixes test script all.sh to output errors from armcc
The test script tests/scripts/all.sh exits on first error, although it also
attempted to redirect error output from armcc and then output it after armcc
had completed. This never occurred because as soon as armcc failed the script
would end and the redirected output wouldn't be displayed.

This change removes that redirection.
2016-08-24 22:37:43 +03:00
Janos Follath
b72c67804a Add script to print build environment info. (#539)
* Add script to print build environment info.

The new script is also included in:
- all.sh
- basic-build-test.sh

* Tidy up environment reporting script.

Changes include:
- making the echo calls portable
- removing unnecessary brackets
- using more efficient checks for the existance of commands
- correcting typos and copyright year

* Update references to output_env.sh
2016-07-19 14:54:17 +01:00
Simon Butcher
b92834324f Fixes all.sh for full config
MBEDTLS_PLATFORM_TIME_ALT was accidentally left in the full config test
leading to linker problems.
2016-07-13 14:47:07 +01:00
Simon Butcher
1c71965d46 Fixes armcc builds in all.sh
MBEDTLS_NV_SEED needs to be disabled in builds without filesystem IO.
2016-06-27 19:02:12 +01:00
Simon Butcher
eebf1b9380 Various fixes for NV SEED feature in all.sh 2016-06-27 01:42:39 +01:00
Simon Butcher
284b4c9927 Fixes all.sh script for filesystem IO test build
MBEDTLS_ENTROPY_NV_SEED is dependent on platform code unless an
alternative implementation is provided,  therefore needs to be disabled
in the disabled filesystem IO build.
2016-06-26 15:44:34 +01:00
Simon Butcher
ab069c6b46 Merge branch 'development' into development-restricted 2016-06-23 21:42:26 +01:00
Simon Butcher
71ebc58932 Changes to check-names.sh script to accept yotta constants 2016-06-23 20:02:07 +01:00
Janos Follath
15ab7ed0f3 Merge branch 'development' into development-restricted
Conflicts:
	programs/pkey/rsa_decrypt.c
	programs/pkey/rsa_encrypt.c
	programs/test/selftest.c
2016-06-14 09:20:46 +01:00
Simon Butcher
ab5df40054 Rename the 'no entropy' feature to MBEDTLS_TEST_NULL_ENTROPY
Following review and for clarity, changed the name of the feature to 'null
entropy'.
2016-06-11 02:31:21 +01:00
Janos Follath
d2e7ff746d Fix test message for entropy safety switch. 2016-06-09 14:12:02 +01:00
Janos Follath
06c54000f3 Add test for the entropy safety switch feature. 2016-06-09 13:57:40 +01:00
Janos Follath
c4191e475b Make basic-build-test.sh clean up after itself. 2016-06-07 14:52:35 +01:00
Simon Butcher
9c22e7311c Merge branch 'development' 2016-05-24 13:25:46 +01:00
Simon Butcher
cbb9075c54 Adds parallel builds to basic-build-test.sh
To speed up test time, added parallel builds
2016-05-23 14:29:32 +01:00
Simon Butcher
ae79124905 Disables backtrace config from basic-build-test.sh
The configuration MBEDTLS_MEMORY_BACKTRACE is intended for debug and
is not necessary for test coverage. Because it causes timing problems
in some tests the configuration has been removed as it's not present in
equivalent tests in the all.sh test script.
2016-05-23 14:29:30 +01:00
SimonB
4a3b023172 Adds line numbering in errors for test helpers
Adds to the 'generate_code.pl' tool, support to insert line numbers
before test suite helper code.
2016-05-23 14:29:29 +01:00
SimonB
525b792823 Fixes off by 1 error reported in line number errors 2016-05-23 14:29:29 +01:00
SimonB
ab07135165 Adds line number substitution in test cases
Expanded generate_code.pl to substitute !LINE_NO! in test cases.
2016-05-23 14:29:29 +01:00
Simon Butcher
bc4d9c1faa Disables backtrace config from basic-build-test.sh
The configuration MBEDTLS_MEMORY_BACKTRACE is intended for debug and
is not necessary for test coverage. Because it causes timing problems
in some tests the configuration has been removed as it's not present in
equivalent tests in the all.sh test script.
2016-05-10 21:20:32 +01:00
Simon Butcher
938f65c452 Merge 'development' into development 2016-05-10 20:58:54 +01:00
SimonB
37f2620db6 Adds line numbering in errors for test helpers
Adds to the 'generate_code.pl' tool, support to insert line numbers
before test suite helper code.
2016-05-05 14:25:03 +01:00
SimonB
43dba3d94e Fixes off by 1 error reported in line number errors 2016-05-05 14:25:03 +01:00
SimonB
c1d2eb3fd6 Adds line number substitution in test cases
Expanded generate_code.pl to substitute !LINE_NO! in test cases.
2016-05-05 14:25:03 +01:00
Simon Butcher
e4a46f696f Merge branch 'development' 2016-04-27 18:44:37 +01:00
SimonB
1594210a49 Adds better support to debug generated code
The commit adds to the generate_code.pl script support to add #line directives
to generated code to allow build breaks to be more easily found from the
generated code.
2016-04-26 14:46:56 +01:00
Janos Follath
35d48cb338 Fix missing cleanup in all.sh 2016-04-22 14:45:00 +01:00
SimonB
8ca7bc42d0 Adds verbose mode to the test suites
Added a verbose option to the generated test suites which can list the
dependencies not met for skipped test cases.
Also clarifies internal interfaces between the main_test.function and test code,
and fixed a bug on calculating available tests in run-test-suites.pl.
2016-04-17 23:24:50 +01:00
SimonB
098a3b5025 Makes basic-build-test.sh test the full config
Previously the test worked on the default configuration which missed deprecated
or legacy features. This change tests the full configuration and all available
tests.
2016-04-16 21:56:59 +01:00
SimonB
2e23c82753 Adds check to avoid overwriting files
Adds check to avoid accidental overwriting of config.h or the yotta module, as
well as a force option to override any changes.
2016-04-16 21:54:39 +01:00
Simon Butcher
6eaf3659ea Fixes Travis post-mortem script dump following review
Changes made:
 * Added copyright and project statement
 * Limited size of each file to dump to 1Mbyte
 * Changed name of script
2016-04-10 15:11:27 +01:00
Simon Butcher
371a9e0ae9 Merge branch 'iotssl-683-travis-failing-intermittently' 2016-04-10 14:24:31 +01:00
Simon Butcher
c2b0efcebe Fix the basic test build script to always build
The test script, 'basic-build-test.sh', wasn't consistently building with
symbols and coverage data, nor doing a forced rebuild.
2016-03-21 22:40:13 +00:00
Janos Follath
831a65ffa7 Make Travis more chatty.
Include the logs in the report after failing.
2016-03-21 14:52:25 +00:00
Simon Butcher
f1547632dc Fixes to style following review
Made code spacing consistent with guidelines, and corrected the misnamed test
steps in basic-build-test.sh
2016-03-14 23:12:32 +00:00
Simon Butcher
ab0c51d782 Fix minor issues with basic test script
Following fixes:
 * In the test script, 'basic-build-test.sh', the total number of functions had
   a broken RE, and was picking up the number of tested functions.
 * Titles of tests was misleading
 * The 'run-test-suites.pl' script was mistaking dSYM directories as test
   suites to be executed.
2016-03-14 23:12:31 +00:00
SimonB
21ab9d7b8b Add a script to execute the basic tests
basic-build-test.sh executes the most obvious and common test suites and creates
a test report including coverage data.
2016-03-14 23:12:30 +00:00
SimonB
75f3caa408 Fix unit test script to ignore coverage data
The script run-test-suite.pl was mistaking .gcda code coverage data files as
test suites. The files are now ignored.
2016-03-14 23:12:30 +00:00
SimonB
ad8fbc066c Add test result breakdown to test suites script
Added a --verbose switch to 'run-test-suite.pl' to summarise the pass/fail/skip
results of each test suite, and summary for all executed tests.
2016-03-14 23:12:30 +00:00
Simon Butcher
f413b6fffe Fix to stop all.sh corrupting config.h
The test script all.sh was persisting the SSL3 configuration in config.h
through more tests than intended and not restoring the config the end.
2016-03-14 22:32:42 +00:00
Simon Butcher
3ea7f52fdf Update interop tests to default configuration
Removed SSLv3 from the default tests in compat.sh, and adapted the test
cases in all.sh to include an additional SSLv3 regression test suite.
2016-03-09 19:32:11 +00:00
Simon Butcher
bc6a486b2f Fix the 'all tests' script for baremetal builds
Fixes the test script test/scripts/all.sh which was failing at the
baremetal ARM builds due to the entropy platform check introduced in
7ff4b77.
2016-03-09 19:32:11 +00:00
Simon Butcher
3000f78b0b Add copright, and better documentation to curves.pl
The purpose and use of the test script, curves.pl was not obvious without
reading the source code, plus the file was missing a copyright statement.
2016-03-09 19:32:11 +00:00
Simon Butcher
64d60da4f6 Fix typos and add copyright statement to generate_code.pl 2016-03-09 19:32:10 +00:00
SimonB
0269dad5e5 Refactored test suite template code
Restructed test suite helper and main code to support tests suite helper
functions, changed C++ comments to C-style, and made the generated
source code more navigable.
2016-03-09 19:32:10 +00:00
SimonB
152ea18037 Added support for per test suite helper functions
Added to generate_code.pl:
    - support for per test suite helper functions
    - description of the structure of the files the script uses to construct
      the test suite file
    - delimiters through the source code to make the machine generated code
      easier to understand
2016-03-09 19:32:10 +00:00
SimonB
3ddf35526a Clarified purpose and usage of generate_code.pl
Added comments to explain purpose and usage of generate_code.pl
2016-03-09 19:32:09 +00:00
Simon Butcher
ef50c0da6e Added script to split the test case data files
Script generate-afl-tests.sh will split the test suite data files into
individual test case files, suitable for fuzzing.
2016-03-09 19:32:09 +00:00
Simon Butcher
9a3ee57c84 Merge branch 'fixes' into development 2016-01-13 02:08:02 +00:00
Manuel Pégourié-Gonnard
d091ed1911 Add scripts/apidoc_full.sh
This re-introduces the apidoc with full config.h, but hopefully with the race
conditions and other issues that the previous implementation had.

Adapt doxygen test script to use that new script, and also check for errors
in addition to warnings while at it.
2016-01-12 14:48:03 +00:00
Manuel Pégourié-Gonnard
de7ae7b2e9 Exclude some warnings from the doxygen test
Apparently travis has an old version of doxygen that doesn't know all tags in
our config. That's not something we care about, we only want to know about
warnings in our doxygen content
2016-01-08 17:01:59 +01:00
Manuel Pégourié-Gonnard
259b08a5d2 Add -s (short) option to all.sh
On my machine, that reduces running time from about 30 minutes to less than 10
minutes, while maintaining a good probability of catching the most likely
issues in practice.
2016-01-08 16:49:17 +01:00
Manuel Pégourié-Gonnard
1d552e7583 Add test script for doxygen warnings 2016-01-08 16:49:17 +01:00
Manuel Pégourié-Gonnard
d2655ac2dc Add test for yotta debug build 2016-01-08 15:05:11 +01:00
Manuel Pégourié-Gonnard
3551901cd1 Make ar invocation more portable
armar doesn't understand the syntax without dash. OTOH, the syntax with dash
is the only one specified by POSIX, and it's accepted by GNU ar, BSD ar (as
bundled with OS X) and armar, so it looks like the most portable syntax.

fixes #386
2016-01-07 13:55:05 +01:00
Manuel Pégourié-Gonnard
067523ef98 Small improvement to test script 2015-10-30 11:15:40 +01:00
Simon Butcher
204606238c Merge branch 'development' into misc 2015-10-27 16:57:34 +00:00
Simon Butcher
e357a64355 Merge pull request #328 from ARMmbed/iotssl-461-ecjpake-finalization
Iotssl 461 ecjpake finalization
2015-10-27 00:08:31 +00:00
Manuel Pégourié-Gonnard
503a5efef1 Add key-exchanges.pl to test list 2015-10-23 09:04:45 +02:00
Manuel Pégourié-Gonnard
50bd2607b8 Add -Werror to reduced configs test scripts 2015-10-23 08:53:34 +02:00
Manuel Pégourié-Gonnard
5df9216c9e Add script to test configs with single key exchanges 2015-10-22 16:11:39 +02:00
Manuel Pégourié-Gonnard
c8cd2c6577 Small fix to 'make test' script
When the tests fail they don't display the number of skipped and run test
2015-10-20 17:01:10 +02:00
Manuel Pégourié-Gonnard
8a7a189220 Fix curves.pl for ECJPAKE disabled by default 2015-10-20 16:56:12 +02:00
Manuel Pégourié-Gonnard
fadacb9d0b Merge branch 'development' into iotssl-461-ecjpake-finalization
* development: (73 commits)
  Bump yotta dependencies version
  Fix typo in documentation
  Corrected misleading fn description in ssl_cache.h
  Corrected URL/reference to MPI library
  Fix yotta dependencies
  Fix minor spelling mistake in programs/pkey/gen_key.c
  Bump version to 2.1.2
  Fix CVE number in ChangeLog
  Add 'inline' workaround where needed
  Fix references to non-standard SIZE_T_MAX
  Fix yotta version dependencies again
  Upgrade yotta dependency versions
  Fix compile error in net.c with musl libc
  Add missing warning in doc
  Remove inline workaround when not useful
  Fix macroization of inline in C++
  Changed attribution for Guido Vranken
  Merge of IOTSSL-476 - Random malloc in pem_read()
  Fix for IOTSSL-473 Double free error
  Fix potential overflow in CertificateRequest
  ...

Conflicts:
	include/mbedtls/ssl_internal.h
	library/ssl_cli.c
2015-10-20 15:00:29 +02:00
Manuel Pégourié-Gonnard
ca700b2371 Add config-thread.h to test-ref-configs.pl 2015-10-20 14:56:04 +02:00
Manuel Pégourié-Gonnard
eb47b870b1 Rework test-ref-configs.pl to also use ssl-opt.sh 2015-10-20 14:56:04 +02:00
Manuel Pégourié-Gonnard
2f056a0aee Try to run yotta update for yotta build test
But accept failures in case we're offline
2015-09-18 14:37:54 +02:00
Manuel Pégourié-Gonnard
6657b8da3b Fix curve-dependency test 2015-09-17 13:46:21 +02:00
Manuel Pégourié-Gonnard
e33316c607 Add test build of shared libs for windows 2015-08-07 13:22:37 +02:00
Manuel Pégourié-Gonnard
32da9f66a8 Add support for MBEDTLS_USER_CONFIG_FILE 2015-08-06 09:57:54 +02:00