The signature of mbedtls_mpi_cmp_mpi_ct() meant to support using it in
place of mbedtls_mpi_cmp_mpi(). This meant full comparison functionality
and a signed result.
To make the function more universal and friendly to constant time
coding, we change the result type to unsigned. Theoretically, we could
encode the comparison result in an unsigned value, but it would be less
intuitive.
Therefore we won't be able to represent the result as unsigned anymore
and the functionality will be constrained to checking if the first
operand is less than the second. This is sufficient to support the
current use case and to check any relationship between MPIs.
The only drawback is that we need to call the function twice when
checking for equality, but this can be optimised later if an when it is
needed.
Enabling memory_buffer_alloc is slow and makes ASan ineffective. We
have a patch pending to remove it from the full config. In the
meantime, disable it explicitly.
None of the test cases in tests_suite_memory_buffer_alloc actually
need MBEDTLS_MEMORY_DEBUG. Some have additional checks when
MBEDTLS_MEMORY_DEBUG but all are useful even without it. So enable
them all and #ifdef out the parts that require DEBUG.
The test case "Memory buffer small buffer" emits a message
"FATAL: verification of first header failed". In this test case, it's
actually expected, but it looks weird to see this message from a
passing test. Add a comment that states this explicitly, and modify
the test description to indicate that the failure is expected, and
change the test function name to be more accurate.
Fix#309
The default entropy nonce length is either zero or nonzero depending
on the desired security strength and the entropy length.
The implementation calculates the actual entropy nonce length from the
actual entropy length, and therefore it doesn't need a constant that
indicates the default entropy nonce length. A portable application may
be interested in this constant, however. And our test code could
definitely use it.
Define a constant MBEDTLS_CTR_DRBG_ENTROPY_NONCE_LEN and use it in
test code. Previously, test_suite_ctr_drbg had knowledge about the
default entropy nonce length built in and test_suite_psa_crypto_init
failed. Now both use MBEDTLS_CTR_DRBG_ENTROPY_NONCE_LEN.
This change means that the test ctr_drbg_entropy_usage no longer
validates that the default entropy nonce length is sensible. So add a
new test that checks that the default entropy length and the default
entropy nonce length are sufficient to ensure the expected security
strength.
Change the default entropy nonce length to be nonzero in some cases.
Specifically, the default nonce length is now set in such a way that
the entropy input during the initial seeding always contains enough
entropy to achieve the maximum possible security strength per
NIST SP 800-90A given the key size and entropy length.
If MBEDTLS_CTR_DRBG_ENTROPY_LEN is kept to its default value,
mbedtls_ctr_drbg_seed() now grabs extra entropy for a nonce if
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY is disabled and either
MBEDTLS_ENTROPY_FORCE_SHA256 is enabled or MBEDTLS_SHA512_C is
disabled. If MBEDTLS_CTR_DRBG_USE_128_BIT_KEY is enabled, or if
the entropy module uses SHA-512, then the default value of
MBEDTLS_CTR_DRBG_ENTROPY_LEN does not require a second call to the
entropy function to achieve the maximum security strength.
This choice of default nonce size guarantees NIST compliance with the
maximum security strength while keeping backward compatibility and
performance high: in configurations that do not require grabbing more
entropy, the code will not grab more entropy than before.
When running 'make test' with GNU make, if a test suite program
displays "PASSED", this was automatically counted as a pass. This
would in particular count as passing:
* A test suite with the substring "PASSED" in a test description.
* A test suite where all the test cases succeeded, but the final
cleanup failed, in particular if a sanitizer reported a memory leak.
Use the test executable's return status instead to determine whether
the test suite passed. It's always 0 on PASSED unless the executable's
cleanup code fails, and it's never 0 on any failure.
FixARMmbed/mbed-crypto#303
Some sanitizers default to displaying an error message and recovering.
This could result in a test being recorded as passing despite a
complaint from the sanitizer. Turn off sanitizer recovery to avoid
this risk.
When running 'make test' with GNU make, if a test suite program
displays "PASSED", this was automatically counted as a pass. This
would in particular count as passing:
* A test suite with the substring "PASSED" in a test description.
* A test suite where all the test cases succeeded, but the final
cleanup failed, in particular if a sanitizer reported a memory leak.
Use the test executable's return status instead to determine whether
the test suite passed. It's always 0 on PASSED unless the executable's
cleanup code fails, and it's never 0 on any failure.
FixARMmbed/mbed-crypto#303
Some sanitizers default to displaying an error message and recovering.
This could result in a test being recorded as passing despite a
complaint from the sanitizer. Turn off sanitizer recovery to avoid
this risk.
config-symmetric-only.h enables MBEDTLS_ENTROPY_NV_SEED so it needs a
seedfile. Create it in test-ref-configs.pl so that the script works on
its own, even if it is not invoked by all.sh.
Add a reference configuration with most symmetric cryptographic
algorithms enabled, but without any asymmetric cryptography. This
checks that we don't have spurious unexpected dependencies on
asymmetric-only modules such as bignum.
Keep HAVE_ASM disabled because it's platform-specific.
Keep HAVEGE disabled because it's untested and not recommended.
Keep MEMORY_BUFFER_ALLOC out because it isn't related to cryptography
at all and it makes memory sanitizers ineffective.
Keep THREADING disabled because it requires special build options.
mbedtls_ctr_drbg_seed() always set the entropy length to the default,
so a call to mbedtls_ctr_drbg_set_entropy_len() before seed() had no
effect. Change this to the more intuitive behavior that
set_entropy_len() sets the entropy length and seed() respects that and
only uses the default entropy length if there was no call to
set_entropy_len().
This removes the need for the test-only function
mbedtls_ctr_drbg_seed_entropy_len(). Just call
mbedtls_ctr_drbg_set_entropy_len() followed by
mbedtls_ctr_drbg_seed(), it works now.
Consolidate the invalid-handle tests from test_suite_psa_crypto and
test_suite_psa_crypto_slot_management. Start with the code in
test_suite_psa_crypto_slot_management and adapt it to test one invalid
handle value per run of the test function.
mbedtls_asn1_get_int() and mbedtls_asn1_get_mpi() behave differently
on negative INTEGERs (0200). Don't change the library behavior for now
because this might break interoperability in some applications. Change
the test function to the library behavior.
Fix the test data with negative INTEGERs. These test cases were
previously not run (they were introduced but deliberately deactivated
in 27d806fab4). The test data was
actually wrong: ASN.1 uses two's complement, which has no negative 0,
and some encodings were wrong. Now the tests have correct data, and
the test code rectifies the expected data to match the library
behavior.
mbedtls_asn1_get_int() and mbedtls_asn1_get_mpi() behave differently
on an empty INTEGER (0200). Don't change the library behavior for now
because this might break interoperability in some applications. Write
a test function that matches the library behavior.
When the asn1parse module is enabled but the bignum module is
disabled, the asn1parse test suite did not work. Fix this.
* Fix a syntax error in get_integer() (label immediately followed by a
closing brace).
* Fix an unused variable in get_integer().
* Fix `TEST_ASSERT( *p == q );` in nested_parse() failing because `*p`
was not set.
* Fix nested_parse() not outputting the length of what it parsed.
Add some ECDSA test cases where the hash is shorter or longer than the
key length, to check that the API doesn't enforce a relationship
between the two.
For the sign_deterministic tests, the keys are
tests/data_files/ec_256_prv.pem and tests/data_files/ec_384_prv.pem
and the signatures were obtained with Python Cryptodome:
from binascii import hexlify, unhexlify
from Crypto.Hash import SHA256, SHA384
from Crypto.PublicKey import ECC
from Crypto.Signature import DSS
k2 = ECC.import_key(unhexlify("3077020101042049c9a8c18c4b885638c431cf1df1c994131609b580d4fd43a0cab17db2f13eeea00a06082a8648ce3d030107a144034200047772656f814b399279d5e1f1781fac6f099a3c5ca1b0e35351834b08b65e0b572590cdaf8f769361bcf34acfc11e5e074e8426bdde04be6e653945449617de45"))
SHA384.new(b'hello').hexdigest()
hexlify(DSS.new(k2, 'deterministic-rfc6979').sign(SHA384.new(b'hello')))
k3 = ECC.import_key(unhexlify("3081a402010104303f5d8d9be280b5696cc5cc9f94cf8af7e6b61dd6592b2ab2b3a4c607450417ec327dcdcaed7c10053d719a0574f0a76aa00706052b81040022a16403620004d9c662b50ba29ca47990450e043aeaf4f0c69b15676d112f622a71c93059af999691c5680d2b44d111579db12f4a413a2ed5c45fcfb67b5b63e00b91ebe59d09a6b1ac2c0c4282aa12317ed5914f999bc488bb132e8342cc36f2ca5e3379c747"))
SHA256.new(b'hello').hexdigest()
hexlify(DSS.new(k3, 'deterministic-rfc6979').sign(SHA256.new(b'hello')))
There were tests to ensure that each entropy source reaches its
threshold, but no test that covers the total amount of entropy. Add
test cases with a known set of entropy sources and make sure that we
always gather at least MBEDTLS_ENTROPY_BLOCK_SIZE bytes from a strong
source.
Always pass a context object to entropy_dummy_source. This lets us
write tests that register more than one source and keep track of how
many times each one is called.
* origin/pr/2843: (26 commits)
Make hyperlink a hyperlink in every markdown flavor
Update the crypto submodule to be the same as development
Document test case descriptions
Restore MBEDTLS_TEST_OUTCOME_FILE after test_default_out_of_box
ssl-opt.sh: Fix some test case descriptions
Reject non-ASCII characters in test case descriptions
Process input files as binary
Factor description-checking code into a common function
Fix cosmetic error in warnings
Fix regex matching run_test calls in ssl-opt.sh
all.sh: run check-test-cases.py
Better information messages for quick checks
Fix configuration short name in key-exchanges.pl
Make test case descriptions unique
New test script check-test-cases.py
Document the test outcome file
Create infrastructure for architecture documents in Markdown
all.sh --outcome-file creates an outcome file
Set meaningful test configuration names when running tests
ssl-opt: remove semicolons from test case descriptions
...
Add invasive checks that peek at the stored persistent data after some
successful import, generation or destruction operations and after
reinitialization to ensure that the persistent data in storage has the
expected content.
Add a parameter to the p_validate_slot_number method to allow the
driver to modify the persistent data.
With the current structure of the core, the persistent data is already
updated. All it took was adding a way to modify it.
When registering a key in a secure element, go through the transaction
mechanism. This makes the code simpler, at the expense of a few extra
storage operations. Given that registering a key is typically very
rare over the lifetime of a device, this is an acceptable loss.
Drivers must now have a p_validate_slot_number method, otherwise
registering a key is not possible. This reduces the risk that due to a
mistake during the integration of a device, an application might claim
a slot in a way that is not supported by the driver.
Since components run in the main process, unsetting
MBEDTLS_TEST_OUTCOME_FILE unset it in subsequent components as well.
To avoid this, save and restore the value.
(Making each component run in a subshell would be a better solution,
but it would be a much bigger change.)
Fix copypasta in some test cases with
MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_CERTIFICATES enabled.
Add unique suffix to the two
"DTLS fragmenting: proxy MTU: auto-reduction" test cases.
This is testing with $kex, not without $kex, so use $kex, not "-$kex".
In test-ref-configs.pl, use $conf rather than "$conf". This is purely
a matter of Perl coding style.
This script checks test case descriptions in test_suite_*.data and
ssl-opt.sh.
It reports the following issues:
* Error: forbidden character in a test case description.
* Error: Duplicate test description.
* Warning: Test description is too long.
If none of the inputs to a key derivation is a
PSA_KEY_DERIVATION_INPUT_SECRET passed with
psa_key_derivation_input_key(), forbid
psa_key_derivation_output_key(). It usually doesn't make sense to
derive a key object if the secret isn't itself a proper key.
After passing some inputs, try getting one byte of output, just to
check that this succeeds (for a valid sequence of inputs) or fails
with BAD_STATE (for an invalid sequence of inputs). Either output a
1-byte key or a 1-byte buffer depending on the test data.
The test data was expanded as follows:
* Output key type (or not a key): same as the SECRET input if success
is expected, otherwise NONE.
* Expected status: PSA_SUCCESS after valid inputs, BAD_STATE after any
invalid input.
Allow a direct input as the SECRET input step in a key derivation, in
addition to allowing DERIVE keys. This makes it easier for
applications to run a key derivation where the "secret" input is
obtained from somewhere else. This makes it possible for the "secret"
input to be empty (keys cannot be empty), which some protocols do (for
example the IV derivation in EAP-TLS).
Conversely, allow a RAW_DATA key as the INFO/LABEL/SALT/SEED input to a key
derivation, in addition to allowing direct inputs. This doesn't
improve security, but removes a step when a personalization parameter
is stored in the key store, and allows this personalization parameter
to remain opaque.
Add test cases that explore step/key-type-and-keyhood combinations.
This commit only makes derive_input more flexible so that the key
derivation API can be tested with different key types and raw data for
each input step. The behavior of the test cases remains the same.
Exercise the library functions with calloc returning NULL for a size
of 0. Make this a separate job with UBSan (and ASan) to detect
places where we try to dereference the result of calloc(0) or to do
things like
buf = calloc(size, 1);
if (buf == NULL && size != 0) return INSUFFICIENT_MEMORY;
memcpy(buf, source, size);
which has undefined behavior when buf is NULL at the memcpy call even
if size is 0.
This is needed because other test components jobs either use the system
malloc which returns non-NULL on Linux and FreeBSD, or the
memory_buffer_alloc malloc which returns NULL but does not give as
useful feedback with ASan (because the whole heap is a single C
object).
The uint32 is given as a bigendian stream, in the tests, however,
the char buffer that collected the stream read it as is,
without converting it. Add a temporary buffer, to call `greentea_getc()`
8 times, and then put it in the correct endianity for input to `unhexify()`.
Reduce the stack usage of the `test_suite_pkcs1_v21` by reducing the
size of the buffers used in the tests, to a reasonable big enough size,
and change the size sent to the API to sizeof output.
Reduce the stack usage of the `test_suite_rsa` by reducing the
size of the buffers used in the tests, to a reasonable big enough size,
and change the data size to decrypt in the data file.
* origin/pr/2765: (28 commits)
Add set+get tests
Consolidate tests for set with/without values
config.py testing: also test the get command
Compatibility redirect: add copyright notice
Compatibility redirect: if python3 is not available, try python
Fix config.py output when a symbol has acquired or lost a value
Remove redundant test case
cmake: update interpreter requirement for the test suite generator
cmake: fix Python requirement
Test script for config.py
Documentation improvements
Fix "#define ... not found" error when using the default file name
Fix "--force set" without a value sneaking a None in
Fix --force requiring an argument
Fix Config.unset() making the name known
Also search config.h near the script
Report an error if switching to Python fails
Fix 'config.py set' without --force
Fix encoding errors
Print help when invoked with no arguments
...
The current test generator code accepts multiple colons as a
separator, but this is just happenstance due to how the code, it isn't
robust. Replace "::" by ":", which is more future-proof and allows
simple separator-based navigation.
Make check-test-cases.py pass.
Prior to this commit, there were many repeated test descriptions, but
none with the same test data and dependencies and comments, as checked
with the following command:
for x in tests/suites/*.data; do perl -00 -ne 'warn "$ARGV: $. = $seen{$_}\n" if $seen{$_}; $seen{$_}=$.' $x; done
Wherever a test suite contains multiple test cases with the exact same
description, add " [#1]", " [#2]", etc. to make the descriptions
unique. We don't currently use this particular arrangement of
punctuation, so all occurrences of " [#" were added by this script.
I used the following ad hoc code:
import sys
def fix_test_suite(data_file_name):
in_paragraph = False
total = {}
index = {}
lines = None
with open(data_file_name) as data_file:
lines = list(data_file.readlines())
for line in lines:
if line == '\n':
in_paragraph = False
continue
if line.startswith('#'):
continue
if not in_paragraph:
# This is a test case description line.
total[line] = total.get(line, 0) + 1
index[line] = 0
in_paragraph = True
with open(data_file_name, 'w') as data_file:
for line in lines:
if line in total and total[line] > 1:
index[line] += 1
line = '%s [#%d]\n' % (line[:-1], index[line])
data_file.write(line)
for data_file_name in sys.argv[1:]:
fix_test_suite(data_file_name)
A test case for 32+0 was present three times, evidently overeager
copy-paste. Replace the duplicates by test cases that read more than
32 bytes, which exercises HKDF a little more (32 bytes is significant
because HKDF-SHA-256 produces output in blocks of 32 bytes).
I obtained the test data by running our implementation, because we're
confident in our implementation now thanks to other test cases: this
data is useful as a non-regression test.
There should have been a good-saltlen test case and a bad-saltlen test
case for both sizes 522 and 528, but the 522-bad-saltlen test case was
missing and the 528-good-saltlen test case was repeated. Fix this.
Don't use semicolons in test case descriptions. The test outcome file
is a semicolon-separated CSV file without quotes to keep things
simple, so fields in that file may not contain semicolons.
The tests were not covering get for a symbol with a value. No symbol
has an uncommented value in the default config.h. (Actually there's
_CRT_SECURE_NO_DEPRECATE, but that's a bit of a hack that this script
is not expected to handle, so don't use it).
Add tests of "get FOO" after "set FOO" and "set FOO value", so that we
have coverage for "get FOO" when "FOO" has a value.
We currently test setting a symbol with a value even if it didn't
originally had one and vice versa. So there's no need to have separate
lists of symbols to test with. Just test everything we want to test
with each symbol.
Set MBEDTLS_TEST_PLATFORM and MBEDTLS_TEST_CONFIGURATION to meaningful
values in all.sh. These environment variables are used when
writing an outcome file, which happens if MBEDTLS_TEST_OUTCOME_FILE is
also set.
When running one of the try-multiple-configuration scripts, set
MBEDTLS_TEST_CONFIGURATION to a value that uniquely describes the
configuration.
Don't use semicolons in test case descriptions. The test outcome file
is a semicolon-separated CSV file without quotes to keep things
simple, so fields in that file may not contain semicolons.
If the environment variable MBEDTLS_TEST_OUTCOME_FILE is set, then for
each test case, write a line to the file with the given name, of the
form
PLATFORM;CONFIGURATION;ssl-opt;TEST CASE DESCRIPTION;PASS/FAIL/SKIP;CAUSE
PLATFORM and CONFIGURATION come from the environment variables
MBEDTLS_TEST_PLATFORM and MBEDTLS_TEST_CONFIGURATION. If these
variables are unset, the script uses some easily-calculated values.
If the environment variable MBEDTLS_TEST_OUTCOME_FILE is set, then for
each test case, write a line to the file with the given name, of the
form
PLATFORM;CONFIGURATION;TEST SUITE;TEST CASE DESCRIPTION;PASS/FAIL/SKIP;CAUSE
PLATFORM and CONFIGURATION come from the environment variables
MBEDTLS_TEST_PLATFORM and MBEDTLS_TEST_CONFIGURATION.
Errors while writing the test outcome file are not considered fatal,
and are not reported except for an error initially opening the file.
This is in line with other write errors that are not checked.
The signature must have exactly the same length as the key, it can't
be longer. Fix#258
If the signature doesn't have the correct size, that's an invalid
signature, not a problem with an output buffer size. Fix the error code.
Add test cases.
Normally a valueless symbol remains valueless and a symbol with a
value keeps having one. But just in case a symbol does get changed
from valueless to having a value, make sure there's a space between
the symbol and the value. And if a symbol gets changed from having a
value to valueless, strip trailing whitespace.
Add corresponding tests.
Also fix the case of a valueless symbol added with the set method,
which would have resulted in attempting to use None as a string. This
only happened with the Python API, not with the command line API.
The test suite generator has been a Python script for a long time,
but tests/CMakeLists.txt still looked for Perl. The reference to
PYTHON_INTERP only worked due to a call to find_package(PythonInterp)
in the toplevel CMakeLists.txt, and cmake would not have printed the
expected error message if python was not available.
Run config.py with various options and store the results in files.
This script also supports the now-removed config.pl.
This is a framework to run non-regression tests on config.py: run it
with the old version, run it with the new version, and compare the
output.
This is deliberately not a functional test suite so that we don't need
to maintain a set of known outputs. When something changes in
config.py (or config.h), run the script before, run it after, and
check manually whether any differences in the output are acceptable.
git grep -Fl /config.pl | xargs sed -i -e 's!/config\.pl!/config.py!g'
Also:
* Change one comment in include/mbedtls/check_config.h.
* Change PERL to PYTHON in CMakeLists.txt.
In psa_asymmetric_sign, immediately reject an empty signature buffer.
This can never be right.
Add test cases (one RSA and one ECDSA).
Change the SE HAL mock tests not to use an empty signature buffer.
Add tests for derivation.
Test both 7 bits and 9 bits, in case the implementation truncated the
bit size down and 7 was rejected as 0 rather than because it isn't a
multiple of 8.
There is no corresponding test for import because import determines
the key size from the key data, which is always a whole number of bytes.
* origin/pr/2469:
Enable MBEDTLS_MEMORY_DEBUG in memory buffer alloc test in all.sh
Remove unnecessary memory buffer alloc unsets
Disable DTLS proxy tests for MEMORY_BUFFER_ALLOC test
all.sh: restructure memory allocator tests
Add missing dependency in memory buffer alloc set in all.sh
Don't set MBEDTLS_MEMORY_DEBUG through `scripts/config.pl full`
Add cfg dep MBEDTLS_MEMORY_DEBUG->MBEDTLS_MEMORY_BUFFER_ALLOC_C
Fix memory leak in CSR test suite on failure
Fix a memory leak in x509write test suite
Add all.sh run with full config and ASan enabled
Add all.sh run with MBEDTLS_MEMORY_BUFFER_ALLOC_C enabled
Update documentation of exceptions for `config.pl full`
Adapt all.sh to removal of buffer allocator from full config
Disable memory buffer allocator in full config
Check dependencies of MBEDTLS_MEMORY_BACKTRACE in check_config.h
Tweak test data for one test case to not rely on mbedtls_asn1_get_int
lacking support for leading zeros. Instead, use a number that is
actually out of range for int.
Tweak test data for one test case to not rely on
mbedtls_asn1_get_bitstring_null rejecting bitstrings shorter than two
octets. Instead, try bit strings that are genuinely invalid, or have a
nonzero number of unused bits.
Add a test case with a correct empty signature. This is commented out
because asn1parse currently does not support this. Uncomment it when
asn1parse is updated to support this.
The new macro ASSERT_ALLOC_WEAK does not fail the test case if the
memory allocation fails. This is useful for tests that allocate a
large amount of memory, but that aren't useful on platforms where
allocating such a large amount is not possible.
Ideally this macro should mark the test as skipped. We don't yet have
a facility for that but we're working on it. Once we have a skip
functionality, this macro should be changed to use it.
Use the test-many-sizes framework for string writes as
well (previously, it was only used for booleans and integers). This
way, more edge cases are tested with less test code.
This commit removes buffer overwrite checks. Instead of these checks,
run the test suite under a memory sanitizer (which we do in our CI).
Omit negative integers and MPIs that would result in values that look
like negative INTEGERs, since the library doesn't respect the
specifications there, but fixing it has a serious risk of breaking
interoperability when ASN.1 is used in X.509 and other
cryptography-related applications.
Add self-contained ASN.1 parsing tests, so that ASN.1 parsing is not
solely tested through X.509 and TLS.
The tests cover every function and almost complete line coverage in
asn1parse.c.
A few test cases containing negative and edge case INTEGER values are
deliberately deactivated because the historical library behavior is at
odds with official specifications, but changing the behavior might
break interoperability.
Other than that, these tests revealed a couple of minor bugs which
will be fixed in subsequent commits.
Add infrastructure to run unit tests and collect the return values for
every PSA API function that returns psa_status_t.
./tests/scripts/psa_collect_statuses.py >statuses.txt
