To test a file name exactly, prepend a / to the base name.
files_to_check actually checks suffixes, not file names, so rename it
to extensions_to_check.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The identifiers of the unmet dependencies of a test case are
stored in a buffer of fixed size that can be potentially too
small to store all the unmet dependencies. Indicate in test
report if some unmet dependencies are missing.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Fix potential buffer overflow when tracking the unmet dependencies
of a test case. The identifiers of unmet dependencies are stored
in an array of fixed size. Ensure that we don't overrun the array.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Use size_t for some variables that are array indices.
Use unsigned for some variables that are counts of "small" things.
This is a backport of commit 3c1c8ea3e7.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Warn about a gotcha that caused a bug in development.
Ensure that it's at least zeroed out, rather than uninitialized, in
non-verbose mode.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
There are currently 4 tests in ssl-opt.sh with either -C "resend" or -S
"resend", that is, asserting that no retransmission will occur. They sometimes
fail on loaded CI machines as one side doesn't send a message fast enough,
causing the other side to retransmit, causing the test to fail.
(For the "reconnect" test there was an other issue causing random failures,
fixed in a previous commit, but even after that fix the test would still
sometimes randomly fail, even if much more rarely.)
While it's a hard problem to fix in a general and perfect way, in practice the
probability of failures can be drastically reduced by making the timeout
values much larger.
For some tests, where retransmissions are actually expected, this would have
the negative effect of increasing the average running time of the test, as
each side would wait for longer before it starts retransmission, so we have a
trade-off between average running time and probability of spurious failures.
But for tests where retransmission is not expected, there is no such trade-off
as the expected running time of the test (assuming the code is correct most of
the time) is not impacted by the timeout value. So the only negative effect of
increasing the timeout value is on the worst-case running time on the test,
which is much less important, as test should only fail quite rarely.
This commit addresses the easy case of tests that don't expect retransmission
by increasing the value of their timeout range to 10s-20s. This value
corresponds to the value used for tests that assert `-S "autoreduction"` which
are in the same case and where the current value seems acceptable so far.
It also represents an increase, compared to the values before this commit, of
a factor 20 for the "reconnect" tests which were frequently observed to fail
in the CI, and of a factor 10 for the first two "DTLS proxy" tests, which were
observed to fail much less frequently, so hopefully the new values are enough
to reduce the probability of spurious failures to an acceptable level.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The server must check client reachability (we chose to do that by checking a
cookie) before destroying the existing association (RFC 6347 section 4.2.8).
Let's make sure we do, by having a proxy-in-the-middle inject a ClientHello -
the server should notice, but not destroy the connection.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
- "Default" should only be used for tests that actually use the defaults (ie,
not passing options on the command line, except maybe debug/dtls)
- All tests in the "Encrypt then MAC" group should start with that string as a
common prefix
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This commit only addresses the timeouts in the "DTLS proxy: 3d, ..." tests.
The discrepancy with the 2.16 branch became apparent for some of these tests
when backporting the previous commit (skip_close_nofity), so let's align the
whole series for consistency and to make future backporting easier.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The ssl-opt.sh test cases using session resumption tend to fail occasionally
on the CI due to a race condition in how ssl_server2 and ssl_client2 handle
the reconnection cycle.
The server does the following in order:
- S1 send application data
- S2 send a close_notify alert
- S3 close the client socket
- S4 wait for a "new connection" (actually a new datagram)
- S5 start a handshake
The client does the following in order:
- C1 wait for and read application data from the server
- C2 send a close_notify alert
- C3 close the server socket
- C4 reset session data and re-open a server socket
- C5 start a handshake
If the client has been able to send the close_notify (C2) and if has been
delivered to the server before if closes the client socket (S3), when the
server reaches S4, the datagram that we start the new connection will be the
ClientHello and everything will be fine.
However if S3 wins the race and happens before the close_notify is delivered,
in S4 the close_notify is what will be seen as the first datagram in a new
connection, and then in S5 this will rightfully be rejected as not being a
valid ClientHello and the server will close the connection (and go wait for
another one). The client will then fail to read from the socket and exit
non-zero and the ssl-opt.sh harness will correctly report this as a failure.
In order to avoid this race condition in test using ssl_client2 and
ssl_server2, this commits introduces a new command-line option
skip_close_notify to ssl_client2 and uses it in all ssl-opt.sh tests that use
session resumption with DTLS and ssl_server2.
This works because ssl_server2 knows how many messages it expects in each
direction and in what order, and closes the connection after that rather than
relying on close_notify (which is also why there was a race in the first
place).
Tests that use another server (in practice there are two of them, using
OpenSSL as a server) wouldn't work with skip_close_notify, as the server won't
close the connection until the client sends a close_notify, but for the same
reason they don't need it (there is no race between receiving close_notify and
closing as the former is the cause of the later).
An alternative approach would be to make ssl_server2 keep the connection open
until it receives a close_notify. Unfortunately it creates problems for tests
where we simulate a lossy network, as the close_notify could be lost (and the
client can't retransmit it). We could modify udp_proxy with an option to never
drop alert messages, but when TLS 1.3 comes that would no longer work as the
type of messages will be encrypted.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
(Only the top-level ones, ie, for each call to eg asn1_get_mpi(), ensure
there's at least one test case that makes this call fail in one way, but don't
test the various ways to make asn1_get_mpi fail - that should be covered
elsewhere.)
- the new checks added by the previous commits needed exercising
- existing tests sometimes had wrong descriptions or where passing for the
wrong reason (eg with the "length mismatch" test, the function actually
failed before reaching the length check)
- while at it, add tests for the rest as well
The valid minimal-size key was generated with:
openssl genrsa 128 2>/dev/null | openssl rsa -outform der 2>/dev/null | xxd -p
Goals:
* Build with common compilers with common options, so that we don't
miss a (potentially useful) warning only triggered with certain
build options.
* A previous commit removed -O0 test jobs, leaving only the one with
-m32. We have inline assembly that is disabled with -O0, falling
back to generic C code. This commit restores a test that runs the
generic C code on a 64-bit platform.
In the 2.7 branch, test-ca.crt has all the components of its Subject name
encoded as PrintableString, because it's generated with our cert_write
program, and our code writes all components that way until Mbed TLS 2.14.
But the default RSA SHA-256 certificate, server2-sha256.crt, has the O and CN
components of its Issuer name encoded as UTF8String, because it was generated
with OpenSSL and that's what OpenSSL does, regardless of how those components
were encoded in the CA's Subject name.
This triggers some overly strict behaviour in some libraries, most notably NSS
and GnuTLS (of interest to us in ssl-opt.sh) which won't recognize the trusted
root as a possible parent for the presented certificate, see for example:
https://github.com/ARMmbed/mbedtls/issues/1033
Fortunately, we have at our disposal a version of test-ca.crt with encodings
matching the ones in server2-sha256.crt, in the file test-ca_utf8.crt. So
let's append that to gnutls-cli's list of trusted roots, so that it recognizes
certs signed by this CA but with the O and CN components as UTF8String.
Note: Since https://github.com/ARMmbed/mbedtls/pull/1641 was merged (in Mbed
TLS 2.14), we changed how we encode those components, so in the 2.16 branch,
cert_write generates test-ca.crt with encodings that matches the ones used by
openssl when generating server2-sha256.crt, so the issue of gnutls-cli
rejecting server2-sha256.crt is specific to the 2.7 branch.
The splitting of this test into two versions depending on whether SHA-1 was
allowed by the server was a mistake in
5d2511c4d4 - the test has nothing to do with
SHA-1 in the first place, as the server doesn't request a certificate from
the client so it doesn't matter if the server accepts SHA-1 or not.
While the whole script makes (often implicit) assumptions about the version of
GnuTLS used, generally speaking it should work out of the box with the version
packaged on our reference testing platform, which is Ubuntu 16.04 so far.
With the update from Jan 8 2020 (3.4.10-4ubuntu1.6), the patches for rejecting
SHA-1 in certificate signatures were backported, so we should avoid presenting
SHA-1 signed certificates to a GnuTLS peer in ssl-opt.sh.
* origin/mbedtls-2.7:
Enable more test cases without MBEDTLS_MEMORY_DEBUG
More accurate test case description
Clarify that the "FATAL" message is expected
Note that mbedtls_ctr_drbg_seed() must not be called twice
Fix CTR_DRBG benchmark
Changelog entry for xxx_drbg_set_entropy_len before xxx_drbg_seed
CTR_DRBG: support set_entropy_len() before seed()
CTR_DRBG: Don't use functions before they're defined
HMAC_DRBG: support set_entropy_len() before seed()
None of the test cases in tests_suite_memory_buffer_alloc actually
need MBEDTLS_MEMORY_DEBUG. Some have additional checks when
MBEDTLS_MEMORY_DEBUG but all are useful even without it. So enable
them all and #ifdef out the parts that require DEBUG.
The test case "Memory buffer small buffer" emits a message
"FATAL: verification of first header failed". In this test case, it's
actually expected, but it looks weird to see this message from a
passing test. Add a comment that states this explicitly, and modify
the test description to indicate that the failure is expected, and
change the test function name to be more accurate.
Fix#309
The corner case tests were designed for 32 and 64 bit limbs
independently and performed only on the target platform. On the other
platform they are not corner cases anymore, but we can still exercise
them.
The corner case tests were designed for 64 bit limbs and failed on 32
bit platforms because the numbers in the test ended up being stored in a
different number of limbs and the function (correctly) returnd an error
upon receiving them.
The signature of mbedtls_mpi_cmp_mpi_ct() meant to support using it in
place of mbedtls_mpi_cmp_mpi(). This meant full comparison functionality
and a signed result.
To make the function more universal and friendly to constant time
coding, we change the result type to unsigned. Theoretically, we could
encode the comparison result in an unsigned value, but it would be less
intuitive.
Therefore we won't be able to represent the result as unsigned anymore
and the functionality will be constrained to checking if the first
operand is less than the second. This is sufficient to support the
current use case and to check any relationship between MPIs.
The only drawback is that we need to call the function twice when
checking for equality, but this can be optimised later if an when it is
needed.
mbedtls_ctr_drbg_seed() always set the entropy length to the default,
so a call to mbedtls_ctr_drbg_set_entropy_len() before seed() had no
effect. Change this to the more intuitive behavior that
set_entropy_len() sets the entropy length and seed() respects that and
only uses the default entropy length if there was no call to
set_entropy_len().
The former test-only function mbedtls_ctr_drbg_seed_entropy_len() is
no longer used, but keep it for strict ABI compatibility.
When running 'make test' with GNU make, if a test suite program
displays "PASSED", this was automatically counted as a pass. This
would in particular count as passing:
* A test suite with the substring "PASSED" in a test description.
* A test suite where all the test cases succeeded, but the final
cleanup failed, in particular if a sanitizer reported a memory leak.
Use the test executable's return status instead to determine whether
the test suite passed. It's always 0 on PASSED unless the executable's
cleanup code fails, and it's never 0 on any failure.
FixARMmbed/mbed-crypto#303
Some sanitizers default to displaying an error message and recovering.
This could result in a test being recorded as passing despite a
complaint from the sanitizer. Turn off sanitizer recovery to avoid
this risk.
* origin/pr/2704:
Adapt auth_crypt_tv usage to 2.7
Add missing dependencies in test_suite_cipher.gcm
Adapt ChangeLog
Add NIST AES GCM test vectors to single-step cipher API test suite
Reduce the stack usage of the `test_suite_pkcs1_v21` by reducing the
size of the buffers used in the tests, to a reasonable big enough size,
and change the size sent to the API to sizeof output.
Reduce the stack usage of the `test_suite_rsa` by reducing the
size of the buffers used in the tests, to a reasonable big enough size,
and change the data size to decrypt in the data file.
Exercise the library functions with calloc returning NULL for a size
of 0. Make this a separate job with UBSan (and ASan) to detect
places where we try to dereference the result of calloc(0) or to do
things like
buf = calloc(size, 1);
if (buf == NULL && size != 0) return INSUFFICIENT_MEMORY;
memcpy(buf, source, size);
which has undefined behavior when buf is NULL at the memcpy call even
if size is 0.
This is needed because other test components jobs either use the system
malloc which returns non-NULL on Linux and FreeBSD, or the
memory_buffer_alloc malloc which returns NULL but does not give as
useful feedback with ASan (because the whole heap is a single C
object).
With the removal of MBEDTLS_MEMORY_BUFFER_ALLOC_C from the
full config, there are no tests for it remaining in all.sh.
This commit adds a build as well as runs of `make test` and
`ssl-opt.sh` with MBEDTLS_MEMORY_BUFFER_ALLOC_C enabled to all.sh.
Previously, numerous all.sh tests manually disabled the buffer allocator
or memory backtracting after setting a full config as the starting point.
With the removal of MBEDTLS_MEMORY_BACKTRACE and MBEDTLS_MEMORY_BUFFER_ALLOC_C
from full configs, this is no longer necessary.
compat.sh used to skip OpenSSL altogether for DTLS 1.2, because older
versions of OpenSSL didn't support it. But these days it is supported.
We don't want to use DTLS 1.2 with OpenSSL unconditionally, because we
still use legacy versions of OpenSSL to test with legacy ciphers. So
check whether the version we're using supports it.
This test case was only executed if the SHA-512 module was enabled and
MBEDTLS_ENTROPY_FORCE_SHA256 was not enabled, so "config.pl full"
didn't have a chance to reach it even if that enabled
MBEDTLS_PLATFORM_NV_SEED_ALT.
Now all it takes to enable this test is MBEDTLS_PLATFORM_NV_SEED_ALT
and its requirements, and the near-ubiquitous MD module.
Call mbedtls_entropy_free on test failure.
Restore the previous NV seed functions which the call to
mbedtls_platform_set_nv_seed() changed. This didn't break anything,
but only because the NV seed functions used for these tests happened
to work for the tests that got executed later in the .data file.
Limit log output in compat.sh and ssl-opt.sh, in case of failures with
these scripts where they may output seemingly unlimited length error
logs.
Note that ulimit -f uses units of 512 bytes, so we use 10 * 1024 * 1024
* 2 to get 10 GiB.
* origin/mbedtls-2.7:
Split _abi_compliance_command into smaller functions
Record the commits that were compared
Document how to build the typical argument for -s
Allow running /somewhere/else/path/to/abi_check.py
Allow TODO in code
Use the docstring in the command line help
* restricted/pr/581:
Remove unnecessary empty line
Add a test for signing content with a long ECDSA key
Add documentation notes about the required size of the signature buffers
Add missing MBEDTLS_ECP_C dependencies in check_config.h
Change size of preallocated buffer for pk_sign() calls
