Commit Graph

380 Commits

Author SHA1 Message Date
Gilles Peskine
72c8c5b352 Merge remote-tracking branch 'upstream-crypto/development' into psa-api-1.0-beta-merge_development_20190801
Conflict resolution:
* `scripts/config.pl`:
  Take the exclusion of `MBEDTLS_PSA_CRYPTO_SE_C` from the API branch.
  Take the removal of `MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C` (obsolete) from
  the development branch.
* `tests/scripts/all.sh`:
  Multiple instances of factoring a sequence of `config.pl` calls into
  a mere `config.pl baremetal` in the development branch, and a change in
  the composition of `baremetal` in the API branch. In each case, take the
  version from development.
* `tests/suites/test_suite_psa_crypto_slot_management.function`:
  A function became non-static in development and disappeared in the API
  branch. Keep the version from the API branch. Functions need to be
  non-static if they're defined but unused in some configurations,
  which is not the case for any function in this file at the moment.
* `tests/suites/test_suite_psa_crypto.function`:
  Consecutive changes in the two branches, reconciled.
2019-07-31 17:47:49 +02:00
Jaeden Amero
8dd1690993 Merge remote-tracking branch 'tls/development' into development
Resolve conflicts by performing the following operations:
- Reject changes related to building a crypto submodule, since Mbed
  Crypto is the crypto submodule.
- Reject X.509, NET, and SSL changes.
- Reject changes to README, as Mbed Crypto is a different project from
  Mbed TLS, with a different README.
- Avoid adding mention of ssl-opt.sh in a comment near some modified
  code in include/CMakeLists.txt (around where ENABLE_TESTING as added).
- Align config.pl in Mbed TLS with config.pl in Mbed Crypto where PSA
  options are concerned, to make future merging easier. There is no
  reason for the two to be different in this regard, now that Mbed TLS
  always depends on Mbed Crypto. Remaining differences are only the
  PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER option and the absence of X.509,
  NET, and SSL related options in Mbed Crypto's config.pl.
- Align config.h in Mbed Crypto with Mbed TLS's copy, with a few notable
  exceptions:
  - Leave CMAC on by default.
  - Leave storage on by default (including ITS emulation).
  - Avoid documenting the PSA Crypto API as is in beta stage in
    documentation for MBEDTLS_PSA_CRYPTO_C.
  The only remaining differences are a lack of X.509, NET, and SSL
  options in Mbed Crypto's config.h, as well as an additional
  Mbed-Crypto-specific PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER option.
  Documentation for the check params feature and related macros is also
  updated to match Mbed TLS's description.
- Reject tests/data_files/Makefile changes to generate DER versions of
  CRTs and keys, as none of those are used by Mbed Crypto tests.
- Add the "no PEM and no filesystem" test to all.sh, without ssl-opt.sh
  run, as Mbed Crypto doesn't have ssl-opt.sh. Also remove use of PSA
  Crypto storage and ITS emulation, since those depend on filesystem
  support.
- Reject addition of test when no ciphersuites have MAC to all.sh, as
  the option being tested, MBEDTLS_SSL_SOME_MODES_USE_MAC, is not
  present in Mbed Crypto.
- Use baremetal config in all.sh, as Mbed Crypto's baremetal
  configuration does exclude the net module (as it doesn't exist in Mbed
  Crypto)
- Reject cmake_subproject_build changes, continuing to link only
  libmbedcrypto.
- Reject changes to visualc and associated templates. Mbed Crypto
  doesn't need additional logic to handle submodule-sourced headers.
- Avoid adding fuzzers from Mbed TLS. The only relevant fuzzers are the
  privkey and pubkey fuzzers, but non-trivial work would be required to
  integrate those into Mbed Crypto (more than is comfortable in a merge
  commit).
- Reject addition of Docker wrappers for compat.sh and ssl-opt.sh, as
  those are not present in Mbed Crypto.
- Remove calls to SSL-related scripts from basic-in-docker.sh

Fix test errors by performing the following:
- Avoid using a link that Doxygen can't seem to resolve in Mbed Crypto,
  but can resolve in Mbed TLS. In documentation for
  MBEDTLS_CHECK_PARAMS, don't attempt to link to MBEDTLS_PARAM_FAILED.

* origin/development: (339 commits)
  Do not build fuzz on windows
  No booleans and import config
  Removing space before opening parenthesis
  Style corrections
  Syntax fix
  Fixes warnings from MSVC
  Add a linker flag to enable gcov in basic-build-test.sh
  Update crypto submodule to a revision with the HAVEGE header changes
  Test with MBEDTLS_ECP_RESTARTABLE
  Allow TODO in code
  Use the docstring in the command line help
  Split _abi_compliance_command into smaller functions
  Record the commits that were compared
  Document how to build the typical argument for -s
  Allow running /somewhere/else/path/to/abi_check.py
  tests: Limit each log to 10 GiB
  Warn if VLAs are used
  Remove redundant compiler flag
  Consistently spell -Wextra
  Fix parsing issue when int parameter is in base 16
  ...
2019-07-31 10:37:53 +01:00
Gilles Peskine
d0e66b00fb Turn off secure element support by default
Secure element support is not yet usable in the real world. Only part
of the feature is implemented and the part that's implemented is not
sufficient for real-world uses. A lot of error handling is missing,
and there are no tests.

This commit should be reverted once the feature has stabilized.
2019-07-24 13:52:51 +02:00
Jaeden Amero
b8e4ae18cf Remove certs.h
certs.h is not needed in Mbed Crypto. No programs or other library code
use it.
2019-07-15 15:52:25 +01:00
Gilles Peskine
ada828f36a Split _abi_compliance_command into smaller functions
This makes the code easier to read and pacifies pylint.
2019-07-04 19:20:35 +02:00
Gilles Peskine
3e2da4acf2 Record the commits that were compared
Record the commit ID in addition to the symbolic name of the version
being tested. This makes it easier to figure out what has been
compared when reading logs that don't always indicate explicitly what
things like HEAD are.

This makes the title of HTML reports somewhat verbose, but I think
that's a small price to pay.
2019-07-04 19:06:54 +02:00
Gilles Peskine
b6ce234c57 Document how to build the typical argument for -s 2019-07-04 19:00:31 +02:00
Gilles Peskine
6aa32ccfae Allow running /somewhere/else/path/to/abi_check.py
Don't require abi_check.py to be the one in scripts/ under the current
directory.
2019-07-04 18:59:36 +02:00
Gilles Peskine
a8ade16ffd Gate secure element support by a separate config option
Secure element support has its own source file, and in addition
requires many hooks in other files. This is a nontrivial amount of
code, so make it optional (but default on).
2019-06-26 20:01:35 +02:00
Jaeden Amero
66b7edb108 Merge remote-tracking branch 'origin/pr/2711' into development
* origin/pr/2711:
  programs: Make `make clean` clean all programs always
  ssl_tls: Enable Suite B with subset of ECP curves
  windows: Fix Release x64 configuration
  platform: Include stdarg.h where needed
  timing: Remove redundant include file
  net_sockets: Fix typo in net_would_block()
2019-06-21 14:09:10 +01:00
Jaeden Amero
e2d5b9e5cc Merge remote-tracking branch 'origin/pr/2690' into development
* origin/pr/2690:
  Making version features easily ROM-able when using Arm C compiler.
2019-06-21 13:20:22 +01:00
Jaeden Amero
4f4af6e2ef windows: Fix Release x64 configuration
Inherit PlatformToolset from the project configuration. This allow the
project to configure PlatformToolset, and aligns the Release x64 build
with other build types.

Fixes #1430
2019-06-20 10:58:50 +01:00
Jaeden Amero
d042198419 Merge remote-tracking branch 'origin/pr/2675' into development
* origin/pr/2675:
  Show removed symbols in abi check
2019-06-14 08:54:21 +01:00
Jaeden Amero
39a8a26cff Merge remote-tracking branch 'origin/pr/2661' into development
* origin/pr/2661:
  Remove all abi dumps, not just ones shared between versions
2019-06-14 08:51:38 +01:00
Máté Varga
c5de4623e8 Making version features easily ROM-able when using Arm C compiler. 2019-06-12 12:26:37 +02:00
Darryl Green
c6f874bfea Show removed symbols in abi check 2019-06-05 12:57:50 +01:00
Gilles Peskine
a3b93ff893 Make docstring style consistent
Use PEP 257 indented docstring style, mostly: always with """, with the
terminating """ on a separate line if the docstring is more than one
line, and with all lines indented to the opening """.

This commit does not change the text to keep the first paragraph single-line.
2019-06-03 11:23:56 +02:00
Darryl Green
f2688e2fe2 Remove all abi dumps, not just ones shared between versions
While the abi-checking script handled comparing only the modules
that were shared between the old and new versions correctly, the
cleanup of the abi dumps only removed what was shared. Change the
cleanup logic to remove all abi dumps instead.
2019-05-29 11:29:08 +01:00
Gilles Peskine
54f544581a Pacify Pylint
Pass Pylint by cleaning up the code where possible and silencing
Pylint where I know better.

No behavior change.
2019-05-29 09:58:59 +02:00
Gilles Peskine
42a0a0aeea Obey Python naming and method structure conventions
* Rename internal methods and fields to start with an underscore.
* Rename global constants to uppercase.
* Change methods that don't use self to be class methods or static
  methods as appropriate.

No behavior change in this commit.
2019-05-29 09:58:59 +02:00
Gilles Peskine
9d4d750041 Add a bit of documentation 2019-05-29 09:58:59 +02:00
Gilles Peskine
0bbad741f4 Switch script to Python3
generate_psa_constants.py was accidentally declared with an
implicitly-Python2 shebang.
2019-05-29 09:58:59 +02:00
Gilles Peskine
2938268fb4 Merge remote-tracking branch 'upstream-crypto/development' into psa-api-1.0-beta-merge_development_20190524
Conflicts:
* library/ssl_cli.c, library/ssl_tls.c:
  Removed on the development branch. Keep them removed.
* include/psa/crypto_extra.h, library/psa_crypto_storage.c,
  tests/suites/test_suite_psa_crypto.data,
  tests/suites/test_suite_psa_crypto.function,
  tests/suites/test_suite_psa_crypto_persistent_key.data,
  tests/suites/test_suite_psa_crypto_slot_management.data,
  tests/suites/test_suite_psa_crypto_slot_management.function:
  Modified on the development branch only to implement the enrollment
  algorithm, which has been reimplemented on the API branch.
  Keep the API branch.
2019-05-29 09:57:29 +02:00
Andrzej Kurek
9f3d39d733 Generating visualc files - let Mbed TLS take precedence over crypto
Change the order of include directories so that Mbed TLS is first.
2019-05-23 03:01:35 -04:00
Andrzej Kurek
b78cf2b261 Adjust visual studio file generation to always use the crypto submodule
Add crypto include to include paths
2019-05-23 03:01:35 -04:00
Andrzej Kurek
9c754a9892 config.pl: disable PSA_ITS_FILE and PSA_CRYPTO_STORAGE for baremetal 2019-05-23 03:01:35 -04:00
Andrzej Kurek
03c35a5744 Disable MBEDTLS_ECP_RESTARTABLE in full config
MBEDTLS_ECP_RESTARTABLE is not compatible with MBEDTLS_USE_PSA_CRYPTO.
2019-05-23 03:01:35 -04:00
Manuel Pégourié-Gonnard
971dea3745 Enable USE_PSA_CRYPTO with config.pl full
Previously it was disabled as too experimental, which no longer holds. Also,
this option introduces new APIs, so it's not only about an internal
alternative (as the comment in config.pl used to state) - people who request a
full config should get all of the available APIs.

Adapt all.sh: now all builds with full config will also test this option, and
builds with the default config will test without it. Just to be sure, let's
have a build with full config minus this option.

Update documentation of MBEDTLS_USE_PSA_CRYPTO to reflect the status of the
new APIs it enables in Mbed TLS and why they're still opt-in.
2019-05-23 03:01:35 -04:00
Manuel Pégourié-Gonnard
de7636e2dd Enable MBEDTLS_PSA_CRYPTO_C by default
Also enable it in scripts/config.pl full, as well as two storage options that
were only blacklisted from full config because they depended on
MBEDTLS_PSA_CRYPTO_C.
2019-05-23 03:01:35 -04:00
Jaeden Amero
3d07ffade2 Merge remote-tracking branch 'tls/development' into development
Resolve conflicts by performing the following operations:
    - Reject changes to files removed during the creation of Mbed Crypto
      from Mbed TLS.
    - Reject the addition of certificates that would not be used by any
      tests, including rejecting the addition of Makefile rules to
      generate these certificates.
    - Reject changes to error.c referencing modules that are not part of
      Mbed Crypto.

* origin/development: (80 commits)
  Style fix
  Fix test data
  Update test data
  Add some negative test cases
  Fix minor issues
  Add ChangeLog entry about listing all SAN
  Remove unneeded whitespaces
  Fix mingw CI failures
  Initialize psa_crypto in ssl test
  Check that SAN is not malformed when parsing
  Documentation fixes
  Fix ChangeLog entry
  Fix missing tls version test failures
  Fix typo
  Fix ChangeLog entry location
  Add changeLog entry
  Add test for export keys functionality
  Add function to retrieve the tls_prf type
  Add tests for the public tls_prf API
  Add public API for tls_prf
  ...
2019-05-21 08:57:44 +01:00
Gilles Peskine
dcaefae849 Parametrize Diffie-Hellman keys by a group identifier
Parametrize finite-field Diffie-Hellman key types with a DH group
identifier, in the same way elliptic curve keys are parametrized with
an EC curve identifier.

Define the DH groups from the TLS registry (these are the groups from
RFC 7919).

Replicate the macro definitions and the metadata tests from elliptic
curve identifiers to DH group identifiers.

Define PSA_DH_GROUP_CUSTOM as an implementation-specific extension for
which domain parameters are used to specify the group.
2019-05-16 19:51:10 +02:00
Darryl Green
d2dba36f44 Only use submodule if present
Enabling the USE_CRYPTO_SUBMODULE option causes problems if the
crypto submodule isn't present. For example, when building
mbed-crypto as a submodule, it should use error.c from the parent
project if USE_CRYPTO_SUBMODULE is set. However if the parent
project isn't present, then the build will fail. Only enable it
if the submodule actually exists.
2019-05-09 13:17:53 +01:00
Jaeden Amero
7b3603c6d8 Merge remote-tracking branch 'tls/development' into development
Resolve merge conflicts by performing the following actions:

- Reject changes to deleted files.
- Reject changes to generate_errors.pl and generate_visualc_files.pl.
  Don't add an 'include-crypto' option which would attempt to use the
  non-existent crypto submodule.
- list-identifiers.sh had the `--internal` option added to it, which
  lists identifiers only in internal headers. Add PSA-specific internal
  headers to list-identifiers.sh.

* origin/development: (40 commits)
  Document the scripts behaviour further
  Use check_output instead of Popen
  all.sh: Require i686-w64-mingw32-gcc version >= 6
  generate_visualc_files.pl: add mbedtls source shadowing by crypto
  generate_errors.pl: refactor and simplify the code
  Start unused variable with underscore
  Correct documentation
  generate_errors.pl: typo fix
  revert changes to generate_features.pl and generate_query_config.pl
  Check that the report directory is a directory
  Use namespaces instead of full classes
  Fix pylint issues
  Don't put abi dumps in subfolders
  Add verbose switch to silence all output except the final report
  Fetch the remote crypto branch, rather than cloning it
  Prefix internal functions with underscore
  Add RepoVersion class to make handling of many arguments easier
  Reduce indentation levels
  Improve documentation
  Use optional arguments for setting repositories
  ...
2019-05-01 14:12:43 +01:00
Jaeden Amero
ed05b29ea3 scripts: Remove unneeded scripts 2019-04-25 11:46:21 +01:00
Jaeden Amero
356acc82ad scripts: Remove dependency on NET 2019-04-25 11:46:21 +01:00
Jaeden Amero
43a450c858 scripts: Remove dependency on X.509 2019-04-25 11:46:21 +01:00
Jaeden Amero
b58ff9541b scripts: Remove dependency on TLS 2019-04-25 11:46:21 +01:00
Jaeden Amero
a4308b29a4 Remove unused TLS, NET, and X.509 files
We've removed all software that depends on or uses the TLS, NET, and
X.509 modules. This means TLS, NET, and X.509 are unused and can be
removed. Remove TLS, NET, and X.509.
2019-04-25 11:46:21 +01:00
Jaeden Amero
bb1f701212 config: Remove X.509 options
Note that this fails check-names.sh because options that TLS and X.509
files use are no longer present in config.h.
2019-04-25 11:46:21 +01:00
Jaeden Amero
1c66e48670 config: Remove TLS and NET options
Remove TLS and NET options from config files and scripts.

Note that this fails check-names.sh because options that TLS and NET
files use are no longer present in config.h.
2019-04-25 11:46:21 +01:00
Jaeden Amero
8298d70bee Only build libmbedcrypto
Update build scripts and tools to only build or update libmbedcrypto.
2019-04-25 11:46:21 +01:00
Jaeden Amero
5c03c65a66 Merge remote-tracking branch 'origin/pr/2547' into development
* origin/pr/2547:
  generate_visualc_files.pl: add mbedtls source shadowing by crypto
  generate_errors.pl: refactor and simplify the code
  generate_errors.pl: typo fix
  revert changes to generate_features.pl and generate_query_config.pl
  generate_errors.pl: add mbedtls header shadowing by crypto headers
  Add an option to use crypto source files in generated visual c project
  Add description of generate_query_config.pl argument
  Add crypto includes when generating features in generate_features.pl
  Include crypto config when generating query config
  Add crypto includes when generating errors in generate_errors.pl
2019-04-24 11:21:56 +01:00
Darryl Green
b2ee0b8782 Use check_output instead of Popen 2019-04-18 09:24:38 +01:00
Gilles Peskine
882e57ecba psa_constant_names: support key agreement algorithms 2019-04-18 09:42:21 +02:00
Jaeden Amero
521dbc67da Merge remote-tracking branch 'tls/development' into development
Merge Mbed TLS at f790a6cbee into Mbed Crypto.

Resolve conflicts by performing the following:
    - Reject changes to README.md
    - Don't add crypto as a submodule
    - Remove test/ssl_cert_test from programs/Makefile
    - Add cipher.nist_kw test to tests/CMakeLists.txt
    - Reject removal of crypto-specific all.sh tests
    - Reject update to SSL-specific portion of component_test_valgrind
      in all.sh
    - Reject addition of ssl-opt.sh testing to component_test_m32_o1 in
      all.sh

* tls/development: (87 commits)
  Call mbedtls_cipher_free() to reset a cipher context
  Don't call mbedtls_cipher_setkey twice
  Update crypto submodule
  Minor fixes in get certificate policies oid test
  Add certificate policy oid x509 extension
  cpp_dummy_build: Add missing header psa_util.h
  Clarify comment mangled by an earlier refactoring
  Add an "out-of-box" component
  Run ssl-opt.sh on 32-bit runtime
  Don't use debug level 1 for informational messages
  Skip uncritical unsupported extensions
  Give credit to OSS-Fuzz for #2404
  all.sh: remove component_test_new_ecdh_context
  Remove crypto-only related components from all.sh
  Remove ssl_cert_test sample app
  Make CRT callback tests more robust
  Rename constant in client2.c
  Document and test flags in x509_verify
  Fix style issues and a typo
  Fix a rebase error
  ...
2019-04-17 12:12:24 +01:00
Andrzej Kurek
021dc3f226 generate_visualc_files.pl: add mbedtls source shadowing by crypto
Running the generation script with "include_crypto" input parameter set to 1
makes the mbedtls sources being overshadowed by crypto sources. 
In case of any duplicate sources, crypto ones take precedence.
2019-04-12 10:51:27 -04:00
Andrzej Kurek
463f049ef0 generate_errors.pl: refactor and simplify the code 2019-04-12 10:35:01 -04:00
Darryl Green
f025d5395e Start unused variable with underscore 2019-04-12 15:18:02 +01:00
Darryl Green
f67e349863 Correct documentation 2019-04-12 15:17:02 +01:00
Andrzej Kurek
e90205f9e6 generate_errors.pl: typo fix 2019-04-12 09:49:30 -04:00