This is meant to be a drop-in replacement for config.pl which can
additionally be used as a library in a Python script.
So far this script supports the commands 'get', 'set' and 'realfull'
but not the other built-in configurations.
assemble_changelog.py reads changelog entries from ChangeLog.d/*.md
and merges them into ChangeLog.md.
The changelog entries are merged into the first version in
ChangeLog.md. The script does not yet support creating a new version
in ChangeLog.md.
The changelog entries are merged in alphabetical order of the file
names. Future versions of this script are likely to adopt a different
order that reflects the git history of the entries.
Normally a valueless symbol remains valueless and a symbol with a
value keeps having one. But just in case a symbol does get changed
from valueless to having a value, make sure there's a space between
the symbol and the value. And if a symbol gets changed from having a
value to valueless, strip trailing whitespace.
Add corresponding tests.
Also fix the case of a valueless symbol added with the set method,
which would have resulted in attempting to use None as a string. This
only happened with the Python API, not with the command line API.
By default, this script looks for include/mbedtls/config.h relative to
the current directory. This allows running config.py from outside the
build tree.
To support out-of-tree builds where config.h and config.py are in the
source tree and the current directory is in the build tree, also try
DIRECTORY_CONTAINING_SCRIPT/../include/mbedtls/config.h, and the
equivalent with symbolic links traversed.
git grep -Fl /config.pl | xargs sed -i -e 's!/config\.pl!/config.py!g'
Also:
* Change one comment in include/mbedtls/check_config.h.
* Change PERL to PYTHON in CMakeLists.txt.
They're easier to maintain that way. The old lists were partly
alphabetized, partly based on config.h order, and partly in the order
in which symbols had been added to config.pl.
Also fix 'realfull' to only affect the appropriate sections.
Tested to produce the same results as config.pl on the default
configuration. This commit deliberately contains a direct copy the
lists of symbol names from config.pl.
This is meant to be a drop-in replacement for config.pl which can
additionally be used as a library in a Python script.
So far this script supports the commands 'get', 'set' and 'realfull'
but not the other built-in configurations.
This being the first 3rdparty-contribution, we may want to consider the
structure of the project file generation scripts. Perhaps add small,
constribution-specific scripts to each directory in 3rdparty instead of adding
all constraints to generate_visualc_files.pl?
This being the first 3rdparty-contribution, we may want to consider the
structure of the project file generation scripts. Perhaps add small,
constribution-specific scripts to each directory in 3rdparty instead of adding
all constraints to generate_visualc_files.pl?
Conflicts and changes:
* Files that do not exist in Mbed Crypto and have changed in Mbed TLS:
These files should not exist in Mbed Crypto. Keep them deleted.
* tests/data_files/test-ca.server1.db: new file in Mbed TLS, don't create
it in Mbed Crypto.
* tests/data_files/rsa_pkcs1_1024_clear.pem: do create this file in
Mbed Crypto. I don't see why it would be kept out.
* tests/data_files/Makefile: don't take any of the changes in sections
that have been removed in Crypto. Do take in the certificate
expiration date updates and the extra .crt.der rules (even if Crypto
doesn't actually use those certificates: removing them would be out
of scope of the present merge).
* tests/suites/helpers.function: consecutive additions, take
both (order indifferent).
Conflict resolution:
* `scripts/config.pl`:
Take the exclusion of `MBEDTLS_PSA_CRYPTO_SE_C` from the API branch.
Take the removal of `MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C` (obsolete) from
the development branch.
* `tests/scripts/all.sh`:
Multiple instances of factoring a sequence of `config.pl` calls into
a mere `config.pl baremetal` in the development branch, and a change in
the composition of `baremetal` in the API branch. In each case, take the
version from development.
* `tests/suites/test_suite_psa_crypto_slot_management.function`:
A function became non-static in development and disappeared in the API
branch. Keep the version from the API branch. Functions need to be
non-static if they're defined but unused in some configurations,
which is not the case for any function in this file at the moment.
* `tests/suites/test_suite_psa_crypto.function`:
Consecutive changes in the two branches, reconciled.
Resolve conflicts by performing the following operations:
- Reject changes related to building a crypto submodule, since Mbed
Crypto is the crypto submodule.
- Reject X.509, NET, and SSL changes.
- Reject changes to README, as Mbed Crypto is a different project from
Mbed TLS, with a different README.
- Avoid adding mention of ssl-opt.sh in a comment near some modified
code in include/CMakeLists.txt (around where ENABLE_TESTING as added).
- Align config.pl in Mbed TLS with config.pl in Mbed Crypto where PSA
options are concerned, to make future merging easier. There is no
reason for the two to be different in this regard, now that Mbed TLS
always depends on Mbed Crypto. Remaining differences are only the
PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER option and the absence of X.509,
NET, and SSL related options in Mbed Crypto's config.pl.
- Align config.h in Mbed Crypto with Mbed TLS's copy, with a few notable
exceptions:
- Leave CMAC on by default.
- Leave storage on by default (including ITS emulation).
- Avoid documenting the PSA Crypto API as is in beta stage in
documentation for MBEDTLS_PSA_CRYPTO_C.
The only remaining differences are a lack of X.509, NET, and SSL
options in Mbed Crypto's config.h, as well as an additional
Mbed-Crypto-specific PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER option.
Documentation for the check params feature and related macros is also
updated to match Mbed TLS's description.
- Reject tests/data_files/Makefile changes to generate DER versions of
CRTs and keys, as none of those are used by Mbed Crypto tests.
- Add the "no PEM and no filesystem" test to all.sh, without ssl-opt.sh
run, as Mbed Crypto doesn't have ssl-opt.sh. Also remove use of PSA
Crypto storage and ITS emulation, since those depend on filesystem
support.
- Reject addition of test when no ciphersuites have MAC to all.sh, as
the option being tested, MBEDTLS_SSL_SOME_MODES_USE_MAC, is not
present in Mbed Crypto.
- Use baremetal config in all.sh, as Mbed Crypto's baremetal
configuration does exclude the net module (as it doesn't exist in Mbed
Crypto)
- Reject cmake_subproject_build changes, continuing to link only
libmbedcrypto.
- Reject changes to visualc and associated templates. Mbed Crypto
doesn't need additional logic to handle submodule-sourced headers.
- Avoid adding fuzzers from Mbed TLS. The only relevant fuzzers are the
privkey and pubkey fuzzers, but non-trivial work would be required to
integrate those into Mbed Crypto (more than is comfortable in a merge
commit).
- Reject addition of Docker wrappers for compat.sh and ssl-opt.sh, as
those are not present in Mbed Crypto.
- Remove calls to SSL-related scripts from basic-in-docker.sh
Fix test errors by performing the following:
- Avoid using a link that Doxygen can't seem to resolve in Mbed Crypto,
but can resolve in Mbed TLS. In documentation for
MBEDTLS_CHECK_PARAMS, don't attempt to link to MBEDTLS_PARAM_FAILED.
* origin/development: (339 commits)
Do not build fuzz on windows
No booleans and import config
Removing space before opening parenthesis
Style corrections
Syntax fix
Fixes warnings from MSVC
Add a linker flag to enable gcov in basic-build-test.sh
Update crypto submodule to a revision with the HAVEGE header changes
Test with MBEDTLS_ECP_RESTARTABLE
Allow TODO in code
Use the docstring in the command line help
Split _abi_compliance_command into smaller functions
Record the commits that were compared
Document how to build the typical argument for -s
Allow running /somewhere/else/path/to/abi_check.py
tests: Limit each log to 10 GiB
Warn if VLAs are used
Remove redundant compiler flag
Consistently spell -Wextra
Fix parsing issue when int parameter is in base 16
...
Due to how the checking script is run in docker, worktree_rev is
ambiguous when running rev-parse. We're running it in the checked
out worktree, so we can use HEAD instead, which is unambiguous.
Secure element support is not yet usable in the real world. Only part
of the feature is implemented and the part that's implemented is not
sufficient for real-world uses. A lot of error handling is missing,
and there are no tests.
This commit should be reverted once the feature has stabilized.
Record the commit ID in addition to the symbolic name of the version
being tested. This makes it easier to figure out what has been
compared when reading logs that don't always indicate explicitly what
things like HEAD are.
This makes the title of HTML reports somewhat verbose, but I think
that's a small price to pay.
Secure element support has its own source file, and in addition
requires many hooks in other files. This is a nontrivial amount of
code, so make it optional (but default on).
* origin/pr/2711:
programs: Make `make clean` clean all programs always
ssl_tls: Enable Suite B with subset of ECP curves
windows: Fix Release x64 configuration
platform: Include stdarg.h where needed
timing: Remove redundant include file
net_sockets: Fix typo in net_would_block()
Inherit PlatformToolset from the project configuration. This allow the
project to configure PlatformToolset, and aligns the Release x64 build
with other build types.
Fixes#1430
Use PEP 257 indented docstring style, mostly: always with """, with the
terminating """ on a separate line if the docstring is more than one
line, and with all lines indented to the opening """.
This commit does not change the text to keep the first paragraph single-line.
While the abi-checking script handled comparing only the modules
that were shared between the old and new versions correctly, the
cleanup of the abi dumps only removed what was shared. Change the
cleanup logic to remove all abi dumps instead.
* Rename internal methods and fields to start with an underscore.
* Rename global constants to uppercase.
* Change methods that don't use self to be class methods or static
methods as appropriate.
No behavior change in this commit.
Conflicts:
* library/ssl_cli.c, library/ssl_tls.c:
Removed on the development branch. Keep them removed.
* include/psa/crypto_extra.h, library/psa_crypto_storage.c,
tests/suites/test_suite_psa_crypto.data,
tests/suites/test_suite_psa_crypto.function,
tests/suites/test_suite_psa_crypto_persistent_key.data,
tests/suites/test_suite_psa_crypto_slot_management.data,
tests/suites/test_suite_psa_crypto_slot_management.function:
Modified on the development branch only to implement the enrollment
algorithm, which has been reimplemented on the API branch.
Keep the API branch.
Previously it was disabled as too experimental, which no longer holds. Also,
this option introduces new APIs, so it's not only about an internal
alternative (as the comment in config.pl used to state) - people who request a
full config should get all of the available APIs.
Adapt all.sh: now all builds with full config will also test this option, and
builds with the default config will test without it. Just to be sure, let's
have a build with full config minus this option.
Update documentation of MBEDTLS_USE_PSA_CRYPTO to reflect the status of the
new APIs it enables in Mbed TLS and why they're still opt-in.
Also enable it in scripts/config.pl full, as well as two storage options that
were only blacklisted from full config because they depended on
MBEDTLS_PSA_CRYPTO_C.
Resolve conflicts by performing the following operations:
- Reject changes to files removed during the creation of Mbed Crypto
from Mbed TLS.
- Reject the addition of certificates that would not be used by any
tests, including rejecting the addition of Makefile rules to
generate these certificates.
- Reject changes to error.c referencing modules that are not part of
Mbed Crypto.
* origin/development: (80 commits)
Style fix
Fix test data
Update test data
Add some negative test cases
Fix minor issues
Add ChangeLog entry about listing all SAN
Remove unneeded whitespaces
Fix mingw CI failures
Initialize psa_crypto in ssl test
Check that SAN is not malformed when parsing
Documentation fixes
Fix ChangeLog entry
Fix missing tls version test failures
Fix typo
Fix ChangeLog entry location
Add changeLog entry
Add test for export keys functionality
Add function to retrieve the tls_prf type
Add tests for the public tls_prf API
Add public API for tls_prf
...
Parametrize finite-field Diffie-Hellman key types with a DH group
identifier, in the same way elliptic curve keys are parametrized with
an EC curve identifier.
Define the DH groups from the TLS registry (these are the groups from
RFC 7919).
Replicate the macro definitions and the metadata tests from elliptic
curve identifiers to DH group identifiers.
Define PSA_DH_GROUP_CUSTOM as an implementation-specific extension for
which domain parameters are used to specify the group.
Enabling the USE_CRYPTO_SUBMODULE option causes problems if the
crypto submodule isn't present. For example, when building
mbed-crypto as a submodule, it should use error.c from the parent
project if USE_CRYPTO_SUBMODULE is set. However if the parent
project isn't present, then the build will fail. Only enable it
if the submodule actually exists.
Resolve merge conflicts by performing the following actions:
- Reject changes to deleted files.
- Reject changes to generate_errors.pl and generate_visualc_files.pl.
Don't add an 'include-crypto' option which would attempt to use the
non-existent crypto submodule.
- list-identifiers.sh had the `--internal` option added to it, which
lists identifiers only in internal headers. Add PSA-specific internal
headers to list-identifiers.sh.
* origin/development: (40 commits)
Document the scripts behaviour further
Use check_output instead of Popen
all.sh: Require i686-w64-mingw32-gcc version >= 6
generate_visualc_files.pl: add mbedtls source shadowing by crypto
generate_errors.pl: refactor and simplify the code
Start unused variable with underscore
Correct documentation
generate_errors.pl: typo fix
revert changes to generate_features.pl and generate_query_config.pl
Check that the report directory is a directory
Use namespaces instead of full classes
Fix pylint issues
Don't put abi dumps in subfolders
Add verbose switch to silence all output except the final report
Fetch the remote crypto branch, rather than cloning it
Prefix internal functions with underscore
Add RepoVersion class to make handling of many arguments easier
Reduce indentation levels
Improve documentation
Use optional arguments for setting repositories
...
We've removed all software that depends on or uses the TLS, NET, and
X.509 modules. This means TLS, NET, and X.509 are unused and can be
removed. Remove TLS, NET, and X.509.
Remove TLS and NET options from config files and scripts.
Note that this fails check-names.sh because options that TLS and NET
files use are no longer present in config.h.
* origin/pr/2547:
generate_visualc_files.pl: add mbedtls source shadowing by crypto
generate_errors.pl: refactor and simplify the code
generate_errors.pl: typo fix
revert changes to generate_features.pl and generate_query_config.pl
generate_errors.pl: add mbedtls header shadowing by crypto headers
Add an option to use crypto source files in generated visual c project
Add description of generate_query_config.pl argument
Add crypto includes when generating features in generate_features.pl
Include crypto config when generating query config
Add crypto includes when generating errors in generate_errors.pl
Merge Mbed TLS at f790a6cbee into Mbed Crypto.
Resolve conflicts by performing the following:
- Reject changes to README.md
- Don't add crypto as a submodule
- Remove test/ssl_cert_test from programs/Makefile
- Add cipher.nist_kw test to tests/CMakeLists.txt
- Reject removal of crypto-specific all.sh tests
- Reject update to SSL-specific portion of component_test_valgrind
in all.sh
- Reject addition of ssl-opt.sh testing to component_test_m32_o1 in
all.sh
* tls/development: (87 commits)
Call mbedtls_cipher_free() to reset a cipher context
Don't call mbedtls_cipher_setkey twice
Update crypto submodule
Minor fixes in get certificate policies oid test
Add certificate policy oid x509 extension
cpp_dummy_build: Add missing header psa_util.h
Clarify comment mangled by an earlier refactoring
Add an "out-of-box" component
Run ssl-opt.sh on 32-bit runtime
Don't use debug level 1 for informational messages
Skip uncritical unsupported extensions
Give credit to OSS-Fuzz for #2404
all.sh: remove component_test_new_ecdh_context
Remove crypto-only related components from all.sh
Remove ssl_cert_test sample app
Make CRT callback tests more robust
Rename constant in client2.c
Document and test flags in x509_verify
Fix style issues and a typo
Fix a rebase error
...
Running the generation script with "include_crypto" input parameter set to 1
makes the mbedtls sources being overshadowed by crypto sources.
In case of any duplicate sources, crypto ones take precedence.
There are a number of arguments being passed around, nearly all of
which are duplicated between the old and new versions. Moving these
into a separate class should hopefully make it simpler to follow
what is being done.
As before with wanting to compare revisions across different
repositories, the ability to select the crypto submodule from a
different repository is useful.
We may wish to compare ABI/API between Mbed TLS and Mbed Crypto,
which will cause issues as not all .so files are shared. Only
compare .so files which both libraries have.
As going forward we will have Crypto in a submodule, we will need to
be able to check ABI compatibility between versions using different
submodule versions. For TLS versions that support the submodule, we
will always build using the submodule.
If the Crypto submodule is used, libmbedcrypto.so is not in the main
library folder, but in crypto/library instead. Given this, the script
searches for *.so files and notes their path, in order to create the
dumps correctly.
By default abi-compliance-checker will check the entire ABI/API.
There are internal identifiers that we do not promise compatibility
for, so we want the ability to skip them when checking the ABI/API.
* origin/pr/2464:
Allow main() to lack a docstring.
Silence pylint
check-files.py: readability improvement in permission check
check-files.py: use class fields for class-wide constants
check-files.py: clean up class structure
abi_check.py: Document more methods
check-files.py: document some classes and methods
Fix pylint errors going uncaught
Call pylint3, not pylint
New, documented pylint configuration
Update the crypto submodule to the top of the Mbed Crypto development
branch. This brings in a version of Mbed Crypto that enables building
Mbed Crypto tests that depend on private headers, like
'psa_crypto_invasive.h'.
This also requires updating our config.h to include new configuration
options added to Mbed Crypto. MBEDTLS_PSA_ITS_FILE_C replaces
MBEDTLS_PSA_CRYPTO_STORAGE_FILE_C and MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C.
MBEDTLS_PSA_INJECT_ENTROPY replaces MBEDTLS_PSA_HAS_ITS_IO.
MBEDTLS_PSA_HAS_ITS_IO is not really useful since it doesn't actually
enable anything except the entropy seed file support, which only
requires the ITS interface and not a native implemetation. Remove it.
Now that we have ITS over files, we no longer need a direct backend
for key storage over files. Remove psa_crypto_storage_file and its
tests.
Switch MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C and MBEDTLS_PSA_ITS_FILE_C on
by default. This preserves functionality and test coverage in the
default configuration, but forgets any key previously stored using the
file backend.
As the SSL programs, like ssl_client2 and ssl_server2, are dependent on
SSL and therefore about to be removed, the only consumer of query_config
is the query_compile_time_config test. As such, it makes sense to move
query_config to be next to what uses it.
* origin/development: (113 commits)
Update query_config.c
Fix failure in SSLv3 per-version suites test
Adjust DES exclude lists in test scripts
Clarify 3DES changes in ChangeLog
Fix documentation for 3DES removal
Exclude 3DES tests in test scripts
Fix wording of ChangeLog and 3DES_REMOVE docs
Reduce priority of 3DES ciphersuites
Fix unused variable warning in ssl_parse_certificate_coordinate()
Update the crypto submodule to a78c958
Fix ChangeLog entry to correct release version
Fix typo in x509write test data
Add ChangeLog entry for unused bits in bitstrings
Improve docs for named bitstrings and their usage
Add tests for (named) bitstring to suite_asn1write
Add new function mbedtls_asn1_write_named_bitstring()
Add missing compile time guard in ssl_client2
Update programs/ssl/query_config.c
ssl_client2: Reset peer CRT info string on reconnect
Add further debug statements on assertion failures
...
Additional work done as part of merge:
- Run ./tests/scripts/check-generated-files.sh and check in the
resulting changes to programs/ssl/query_config.c
Additional work done as part of merge:
- Run ./tests/scripts/check-generated-files.sh and check in the
resulting changes to programs/ssl/query_config.c
Declare the owner as psa_key_owner_id_t, of which an implementation
must be provided separately.
Make this a configuration option
MBEDTLS_PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER, to make the conditional
compilation flow easier to follow. Declare it in config.h to
pacify check_names.sh.
Support for a specific implementation of psa_key_owner_id_t in storage
backends will come in a subsequent commit.
Resolve conflicts by performing the following.
- Take the upstream Mbed TLS ChangeLog verbatim.
- Reject changes to Makefiles and CMake that are related to using Mbed
Crypto as a submodule. It doesn't make sense to use Mbed Crypto as a
submodule of itself.
- Reject README changes, as Mbed Crypto has its own, different README.
- Reject PSA-related changes to config.h. We don't want to disable the
availability of the PSA Crypto API by default in the Mbed Crypto
config.h.
- Don't inadvertently revert dead code removal in
mbedtls_cipher_write_tag() which was added in f2a7529403 ("Fix
double return statement in cipher.c")
- Where Mbed Crypto already had some MBEDTLS_USE_PSA_CRYPTO code (from
past companion PRs) take the latest version from Mbed TLS which
includes integration with MBEDTLS_CHECK_PARAMS.
- Update the version of the shared library files to match what's
currently present in Mbed TLS.
- Reject removal of testing with PSA from config full tests.
- Resolve conflicts in test tests/suites/helpers.function, where both
Mbed Crypto and Mbed TLS both added documentation for TEST_ASSERT.
Combine text from both documentation efforts.
- Reject adding a submodule of ourselves.
- Reject addition of submodule tests in all.sh.
- Reject addition of submodule to library path in
tests/scripts/run-test-suites.pl.
- Avoid using USE_CRYPTO_SUBMODULE=1 in
component_test_use_psa_crypto_full_cmake_asan() in all.sh.
test_psa_constant_names.py was originally written before the split of
crypto.h into crypto_values.h and more, so it now needs to read
crypto_values.h as well.
In both generate_psa_constants.py and test_psa_constant_names.py, read
crypto_extra.h as well. We don't currently define any value there, but
it's plausible that we will one day.
Some parts of the library, and crypto drivers, need to see key types,
algorithms, policies, etc. but not API functions. Move portable
integral types and macros to build and analyze values of these types
to a separate headers crypto_types.h and crypto_values.h.
No functional changes, code was only moved from crypto.h to the new headers.
