From 508caf528ac3504f4757271c1d12dfb2d4a3967d Mon Sep 17 00:00:00 2001
From: Gilles Peskine <Gilles.Peskine@arm.com>
Date: Mon, 16 Sep 2019 16:29:15 +0200
Subject: [PATCH] Document the test outcome file

---
 docs/architecture/Makefile                  |  1 +
 docs/architecture/testing/test-framework.md | 28 +++++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 docs/architecture/testing/test-framework.md

diff --git a/docs/architecture/Makefile b/docs/architecture/Makefile
index 4873daeb8..2b2a11b68 100644
--- a/docs/architecture/Makefile
+++ b/docs/architecture/Makefile
@@ -3,6 +3,7 @@ PANDOC = pandoc
 default: all
 
 all_markdown = \
+	       testing/test-framework.md \
 	       # This line is intentionally left blank
 
 html: $(all_markdown:.md=.html)
diff --git a/docs/architecture/testing/test-framework.md b/docs/architecture/testing/test-framework.md
new file mode 100644
index 000000000..270a3f9be
--- /dev/null
+++ b/docs/architecture/testing/test-framework.md
@@ -0,0 +1,28 @@
+# Mbed TLS test framework
+
+This document is an overview of the Mbed TLS test framework and test tools.
+
+This document is incomplete. You can help by expanding it.
+
+## Running tests
+
+### Outcome file
+
+#### Generating an outcome file
+
+Unit tests and `ssl-opt.sh` record the outcome of each test case in a **test outcome file**. This feature is enabled if the environment variable `MBEDTLS_TEST_OUTCOME_FILE` is set. Set it to the path of the desired file.
+
+If you run `all.sh --outcome-file test-outcome.csv`, this collects the outcome of all the test cases in `test-outcome.csv`.
+
+#### Outcome file format
+
+The outcome file is in a CSV format using `;` (semicolon) as the delimiter and no quoting. This means that fields may not contain newlines or semicolons. There is no title line.
+
+The outcome file has 6 fields:
+
+* **Platform**: a description of the platform, e.g. `Linux-x86_64` or `Linux-x86_64-gcc7-msan`.
+* **Configuration**: a unique description of the configuration (`config.h`).
+* **Test suite**: `test_suite_xxx` or `ssl-opt`.
+* **Test case**: the description of the test case.
+* **Result**: one of `PASS`, `SKIP` or `FAIL`.
+* **Cause**: more information explaining the result.