Commit Graph

12904 Commits

Author SHA1 Message Date
Simon Butcher
3aa6405007 Correct comment on the configuration option in x509.c
In x509.c, the self-test code is dependent on MBEDTLS_CERTS_C and
MBEDTLS_SHA256_C being enabled. At some point in the recent past that dependency
was on MBEDTLS_SHA1_C but changed to SHA256, but the comment wasn't updated.

This commit updates the comment.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-03-27 17:16:15 +00:00
Simon Butcher
432e7023b1 Add additional sanity checks to check_config.h
Additional sanity checks in check_config.h to ensure:
    * if test certificates are included (MBEDTLS_CERTS_C) there must be also be
      support for the core X509 feature (MBEDTLS_X509_USE_C). This has a
      secondary dependency on the public key abstraction layer (MBEDTLS_PK_C),
      necessary as the certificates will either be signed by RSA or ECDSA, and
      therefore need to be part of the library.
    * if any of the TLS protocols are defined (MBEDTLS_SSL_PROTO_xxx) then a
      key exchange method must also be defined (MBEDTLS_KEY_EXCHANGE_xxx).

Anyone who knows the library will probably not make these mistakes or will
quickly diagnose and fix them, but it is possible to compile and link both
configurations if you build only the library and not the example programs, and
therefore users may not realise immediately that there's a mistake, only
discovering it at runtime.

These checks may therefore save someone some time.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-03-27 17:16:15 +00:00
Manuel Pégourié-Gonnard
6f8aaba524
Merge pull request #3131 from AndrzejKurek/makefile-remove-priority-dependency
makefile: remove unneeded priority dependency from library makefile
2020-03-27 13:06:16 +01:00
Gilles Peskine
1596c5e9cb
Merge pull request #3133 from gilles-peskine-arm/changelog-date-pylint
assemble_changelog.py: Reformatting to pacify pylint
2020-03-27 11:29:37 +01:00
Gilles Peskine
ac0f0860f1 Reformatting to pacify pylint
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 10:56:45 +01:00
Gilles Peskine
4457a0a489
Merge pull request #3128 from gilles-peskine-arm/changelog-date
Sort changelog entries by their merge date
2020-03-27 10:42:43 +01:00
Gilles Peskine
42f384c186 Mbed Crypto isn't a separate project anymore
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 09:23:38 +01:00
Gilles Peskine
98a53aa399 Follow renames when looking for the creation of a file
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:50:11 +01:00
Gilles Peskine
13dc634282 Simplify the matching of the last line
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:49:44 +01:00
Gilles Peskine
7fa3eb7d44 Simplify sorting
Calling sort_key explicitly is marginally faster, but less readable.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:41:32 +01:00
Gilles Peskine
28af958ea4 Documentation improvements
Document that git is needed.

Be clearer about the entry sort key being an entry sort key, not just
a merge order. Be clearer about what "merge order" means.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:39:18 +01:00
Andrzej Kurek
98911cdd3a
makefile: remove unneeded priority dependency from library makefile
This change was first introduced in 8af3923 - see this commit for more background.
After the removal of crypto directory, there are no targets that require a 
crypto library with the directory prefix, so there's also no need for the priority
dependency to be declared. This commit removes it.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2020-03-26 07:18:42 -04:00
Gilles Peskine
8f46bbf46f Sort changelog entries by their merge date
Changelog entry files were listed in reverse alphabetical order of the
file name, by happenstance. Now, changelog entry files are listed in
the order in which the changes were merged. More precisely: look for
the git commit where the entry file was created, and look where this
commit was merged into the current branch. List older merges first.
List never-merged commits in date order after all the merged ones.
List never-committed files in file timestamp order after all the
committed ones.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 11:58:00 +01:00
Gilles Peskine
27a1facd48 Factor out list_files_to_merge
No behavior changes.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 11:57:54 +01:00
Manuel Pégourié-Gonnard
2451ac698d
Merge pull request #3126 from gilles-peskine-arm/merge-crypto-stragglers-20200325
Merge mbed-crypto stragglers 2020-03-25
2020-03-25 22:12:30 +01:00
Gilles Peskine
854770b84e Temporarily remove ChangeLog.md
All the content that is currently in ChangeLog.md is the result of
merging Mbed Crypto, and that content is already present in
ChangeLog. Since we aren't using ChangeLog.md yet, remove it for now.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-25 17:13:19 +01:00
Gilles Peskine
5d324b203a Merge remote-tracking branch 'upstream-crypto/development' into HEAD
Conflicts:
* .github/issue_template.md, .github/pull_request_template.md:
  The changes in Mbed Crypto are not relevant to Mbed TLS. Keep the
  mbedtls versions.
2020-03-25 15:47:43 +01:00
Gilles Peskine
8bad3ab45f
Merge pull request #300 from gilles-peskine-arm/assemble-changelog
Changelog assembly script
2020-03-25 15:17:04 +01:00
Jaeden Amero
dfd5ea239d
Merge pull request #3117 from AndrzejKurek/null_buffer_tests
Increase test coverage by adding AES and CAMELLIA empty buffer tests
2020-03-25 18:04:48 +04:00
Manuel Pégourié-Gonnard
8b0187e6f4
Merge pull request #392 from mpg/no-new-issue-or-pr
Have issue and PR templates redirect to Mbed TLS
2020-03-25 09:55:33 +01:00
Manuel Pégourié-Gonnard
cdcb187eb2 Have issue and PR templates redirect to Mbed TLS
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-03-25 09:31:23 +01:00
Gilles Peskine
d414c32a16
Merge pull request #3113 from mpg/ssl-opt-names
Improve some test names in ssl-opt.sh
2020-03-24 19:48:09 +01:00
Andrzej Kurek
1b7a780e65
Increase test coverage by adding AES and CAMELLIA empty buffer tests
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2020-03-24 13:18:58 -04:00
Gilles Peskine
7a054d1def
Merge pull request #3094 from gilles-peskine-arm/merge-crypto-docs
Merge documentation from Mbed Crypto
2020-03-24 14:19:34 +01:00
Gilles Peskine
37b61814cb
Merge pull request #3085 from gilles-peskine-arm/merge-crypto-unremoved-20200304
Merge of Mbed Crypto

Merge Mbed Crypto back into Mbed TLS. See https://github.com/ARMmbed/mbed-crypto/issues/374 for more details.
2020-03-24 13:56:15 +01:00
Manuel Pégourié-Gonnard
9e2c80fa24 Improve some test names in ssl-opt.sh
- "Default" should only be used for tests that actually use the defaults (ie,
  not passing options on the command line, except maybe debug/dtls)
- All tests in the "Encrypt then MAC" group should start with that string as a
  common prefix

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-03-24 10:53:39 +01:00
Gilles Peskine
3ed2de9f54 Refresh generated files 2020-03-23 21:54:07 +01:00
Gilles Peskine
f03bd81be0 Place MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH in alphabetical order
The automatic merge of include/mbedtls/config.h had placed
MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH before
MBEDTLS_SSL_TRUNCATED_HMAC_COMPAT.
2020-03-23 18:13:58 +01:00
Gilles Peskine
5e7d6fd240 Merge 'mbedtls/development' into merge-crypto-unremoved-20200304
Merge the latest state of the target branch (mbedtls/development) into the
pull request to merge mbed-crypto into mbedtls.

Conflicts:

* ChangeLog: add/add conflict. Resolve by using the usual section order.
2020-03-23 18:02:07 +01:00
Gilles Peskine
5ec51f923a Future-proof make clean under Windows
In Windows cmd, `del foo` succeeds if `foo` doesn't exist, but
`del *.ext` fails if `*.ext` doesn't match any files. Therefore we use
the idiom `if exist *.ext del *.ext` to delete files matching
wildcards.

We've accidentally used `if exist $(some_list) del $(some_list)`, and
that's wrong, because it's only syntactically correct if
`$(some_list)` contains exactly one element. As long as `$(some_list)`
contains actual file names and not wildcards, just use `del $(some_list)`.
2020-03-23 17:55:32 +01:00
Gilles Peskine
a647f1296e Delete third-party objects in cleanup 2020-03-23 17:55:32 +01:00
Gilles Peskine
049fefff90 Add dependencies on !MBEDTLS_SHA512_NO_SHA384 to x509/ssl unit tests
Add dependencies on !MBEDTLS_SHA512_NO_SHA384 to X.509 and SSL unit
tests that use SHA-384 (identified based on having a description that
contains "SHA384" or "SHA-384").
2020-03-23 17:55:32 +01:00
Gilles Peskine
6eb45f99f0 Add missing dependency on MBEDTLS_CIPHER_MODE_CBC 2020-03-23 17:55:32 +01:00
Gilles Peskine
19e841e19f Fix warning about a potentially initialized variable
The variable drop_len is only used under conditions where it has been
initialized, but this is somewhat fragile and not obvious for a
compiler to see.
2020-03-23 17:55:32 +01:00
Gilles Peskine
eccd888717 Rename identifiers containing double-underscore
Rename identifiers containing double-underscore (`__`) to avoid `__`.
The reason to avoid double-underscore is that all identifiers
containing double-underscore are reserved in C++. Rename all such
identifiers that appear in any public header, including ssl_internal.h
which is in principle private but in practice is installed with the
public headers.

This commit makes check-names.sh pass.

```
perl -i -pe 's/\bMBEDTLS_SSL__ECP_RESTARTABLE\b/MBEDTLS_SSL_ECP_RESTARTABLE_ENABLED/g; s/\bMBEDTLS_KEY_EXCHANGE_(_\w+)_(_\w+)\b/MBEDTLS_KEY_EXCHANGE${1}${2}/g' include/mbedtls/*.h library/*.c programs/*/*.c scripts/data_files/rename-1.3-2.0.txt tests/suites/*.function
```
2020-03-23 17:55:32 +01:00
Gilles Peskine
2f084fe02c Add changelog entries about the crypto submodule
Add a changelog entry for an already-released version to indicate when
the crypto submodule became mandatory.

Add a changelog entry for the removal of the crypto submodule.
2020-03-23 17:55:32 +01:00
Gilles Peskine
7415f2fd12 Add comment to help syntax highlighting in editors
Restore this change which was accidentally reverted during the merge
of Mbed Crypto.
2020-03-23 17:55:32 +01:00
Gilles Peskine
0a968f7c7f Fix the shared library build with CMake
Fix an error in the earlier merge.
2020-03-23 17:55:32 +01:00
Gilles Peskine
55d5353b3a Remove lingering build code referencing the crypto submodule
Remove code guarded by `USE_CRYPTO_SUBMODULE`. It's dead now that
crypto can no longer be a submodule.

In `library/Makefile`:

* Replace `$(CRYPTO_INCLUDE)` with the single include directory
  `-I../include`.
* Remove references to `$(OBJS_CRYPTO)` when it's in addition to the
  local objects (`*.o`) since `$(OBJS_CRYPTO)` is now a subset of the
  local objects.
* Merge modules that were duplicated between the mbedtls and the
  mbed-crypto repositories back into the single list for `OBJS_CRYPTO`.
2020-03-23 17:55:32 +01:00
Gilles Peskine
f8c2965d11 Remove the crypto submodule
All crypto files are now in the main tree.
2020-03-23 17:55:31 +01:00
Gilles Peskine
aae57bffd9 Remove remaining references to the crypto subdirectory 2020-03-23 17:55:31 +01:00
Gilles Peskine
18a9d5b425 Use an indirect reference to detect the 3rdparty directory 2020-03-23 17:55:31 +01:00
Gilles Peskine
37940d923e Restore linking against all libraries in the cmake_subproject test 2020-03-23 17:55:31 +01:00
Gilles Peskine
4973196e5a Restore fuzz to the cmake build 2020-03-23 17:55:31 +01:00
Gilles Peskine
d2e77b51a1 Restore the library SO version in Makefile 2020-03-23 17:55:31 +01:00
Gilles Peskine
4e616f893b Remove config-default.h
This file was the default mbedtls configuration in the mbedtls-psa
fork. It is no longer relevant.
2020-03-23 17:55:31 +01:00
Gilles Peskine
b22d0cfa99 Re-generate automatically generated files
```
scripts/generate_features.pl
scripts/generate_errors.pl
scripts/generate_query_config.pl
scripts/generate_visualc_files.pl
```
2020-03-23 17:55:31 +01:00
Gilles Peskine
d3d02900f2 Restore X509 and SSL functionality
Look for changes that remove X509/SSL functionality.
```
git diff 'HEAD^{/^Merge}~1' HEAD --diff-filter=M -- . ':!library/error.c' ':!library/version_features.c' ':!programs/test/query_config.c' ':!visualc' ':!*.pdf' ':!*.der' | grep -E "^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_"
```

All of these removals are in `config.h` or `check_config.h`. Selectively revert the differences in these two files.

```
git diff 'HEAD^{/^Merge}~1' 'HEAD^{/^Merge}' include/mbedtls/config.h include/mbedtls/check_config.h | git apply -p1 -R
```

* `include/mbedtls/check_config.h`:
    * ARIA for GCM: don't remove it.
    * `MBEDTLS_PSA_CRYPTO_SE_C`: don't remove it.
    * `MBEDTLS_SHA512_NO_SHA384`: don't remove it.
    * `MBEDTLS_SSL_DTLS_CONNECTION_ID`: restore it.
* `include/mbedtls/config.h`:
    * warning about `MBEDTLS_ECDSA_SIGN_ALT`: don't remove it.
    * `MBEDTLS_CTR_DRBG_USE_128_BIT_KEY` full paragraph: don't remove it.
    * `MBEDTLS_PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER`: don't remove it.
    * `MBEDTLS_SHA512_SMALLER`…: don't remove it.
    * `MBEDTLS_SSL_RECORD_CHECKING`: restore it.
    * `MBEDTLS_SSL_CONTEXT_SERIALIZATION`: restore it.
    * `MBEDTLS_USE_PSA_CRYPTO` note: don't restore the tls version.
    * `MBEDTLS_USE_PSA_CRYPTO` warning: restore the tls version.
    * `MBEDTLS_CMAC_C`: restore it to being disabled by default. It's a minor API change in Mbed TLS because it changes the layout of `mbedtls_cipher_context_t`.
    * `MBEDTLS_CTR_DRBG_C`: don't restore the older version of the description from tls.
    * `MBEDTLS_GCM_C`: don't restore the older description from tls.
    * `MBEDTLS_PSA_CRYPTO_C`: don't restore `crypto/`.
    * `MBEDTLS_PSA_CRYPTO_SE_C`: don't remove it.
    * `MBEDTLS_PSA_CRYPTO_STORAGE_C`: don't restore `crypto/`. Don't disable it by default.
    * `MBEDTLS_PSA_ITS_FILE_C`: don't restore, like for ``MBEDTLS_PSA_CRYPTO_STORAGE_C``.
    * `MBEDTLS_CTR_DRBG_USE_128_BIT_KEY` single line: don't restore it since there is now a full paragraph in the proper section above.
    * `MBEDTLS_SSL_CID_IN_LEN_MAX`…: restore it.
    * `MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_CERTIFICATES`: restore the version without a space to format the line like the others.
2020-03-23 17:55:31 +01:00
Gilles Peskine
a57a80e213 Restore full tls coverage to remaining scripts
Look for any other invocation of test scripts that was removed: look for a change that removes a line (`^-.*…`) containing one of the names of a test script (without its path because it may be used with a relative path). Look for `ssl-opt.sh` and `compat.sh`, as well any file in `tests/scripts` that only exists in tls.
```
git diff 'HEAD^{/^Merge}~1' HEAD --diff-filter=M -- . ':!library/error.c' ':!library/version_features.c' ':!programs/test/query_config.c' ':!visualc' ':!*.pdf' ':!*.der' | grep -E "^-.*($(comm -23 <(git ls-tree -r --name-only 'HEAD^{/^Merge}~1') <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development 'HEAD^{/^Merge}^2')) | sed -n 's!^tests/scripts/!!p' | sed 's/\./\\./g' | tr '\n' '|')ssl-opt\.sh|compat\.sh)" | grep -v '^---'
```
This only turns up changes in `basic-in-docker.sh`.
2020-03-23 17:55:31 +01:00
Gilles Peskine
636c26ad75 Restore full tls coverage to all.sh
The merge of mbed-crypto removed some tls coverage. Restore it. Also
remove references to the `crypto` subdirectory brought by the mbedtls
side of the merge. In more detail:

* `tests/scripts/all.sh`:
    * `fuzz` in comments (×2): restore it.
    * `CTEST_OUTPUT_ON_FAILURE=1`: don't remove it.
    * `cd crypto` for `make clean`: don't restore it.
    * `cleanup`: do restore `programs/fuzz/Makefile`. Don't go into `crypto`. Keep only one copy of the calls to `rm` in `cmake_subproject`.
    * Comment legacy options: don't remove it.
    * `crypto/Makefile` and `pre_check_seedfile`: don't restore either. See below regarding the lack of need for `pre_check_seedfile`.
    * blank line in `pre_print_configuration`: restore it.
    * blank line before `#### Build and test`: restore it.
    * SSL tests in `component_test_full_cmake_gcc_asan` and zlib components: restore it.
    * `component_test_no_pem_no_fs` (×2): the merge placed two copies in different locations. Reconcile them: unset PSA storage like in crypto, and call `ssl-opt.sh` like in tls. Put the merged version at the tls location.
    * `component_test_everest`: do add it at the tls location.
    * `component_test_small_mbedtls_ssl_dtls_max_buffering`: restore the tls value.
    * `component_test_new_ecdh_context`…: move `component_test_new_ecdh_context` before `component_test_everest` and add a calls to `compat.sh` and `ssl-opt.sh` like in `component_test_everest`. Remove the redundant crypto-only `component_test_everest`. Don't remove `component_test_psa_collect_statuses`.
    * `component_test_full_cmake_clang`: don't remove `clang` in the `msg` call. Don't remove the call to `test_psa_constant_names.py`.
    * `component_test_full_make_gcc_o0`: remove it. It's subsumed by `component_test_gcc_opt`.
    * `component_build_deprecated`: don't remove anything.
    * `component_test_memory_buffer_allocator`: restore `ssl-opt.sh`.
    * `component_test_when_no_ciphersuites_have_mac`: restore it.
    * `component_test_platform_calloc_macro`: don't restore `unset MBEDTLS_MEMORY_BUFFER_ALLOC_C` which is now redundant. Don't restore explicit flags instead of `$ASAN_CFLAGS`.
    * `component_test_aes_fewer_tables`…: don't remove it.
    * `component_test_m32_o1`: restore SSL testing.
    * `component_test_m32_everest`: restore SSL testing.
    * `component_test_min_mpi_window_size`…: don't remove it.
    * `component_test_valgrind`: do restore the tls version of the comment.
    * `run_component`: don't remove the seedfile creation. This is better than `pre_check_seedfile` (see below).
    * `pre_check_seedfile`: don't restore it. `pre_check_seedfile` (from tls) creates a seedfile once and for all. This is not good enough if a component fails in such a way as to leave a broken seedfile, or if a component leaves a seedfile with a size that's wrong for the next component to run. Instead (from crypto), `run_component` creates a sufficiently large seedfile before each component.
2020-03-23 17:55:31 +01:00