bogo: implement most client-side ECH tests

Getting bogo test coverage working for ECH requires taking a (temporary)
dev-dep. on the provider-example crate so we can use the Rust Crypto
HPKE provider. This in turn means that we now have two Rustls versions
in tree, the src crate and the older version being used by the
provider example by way of hickory-dns. This will fall away when proper
HPKE support is implemented with one/both built-in crypto providers but
in the meantime requires some small adjustments in CI and the runme
script. In general the scope of a proper HPKE impl will be much less
invasive than implementing ECH so we can use this hack for now and
revisit shortly.

The bogo shim also requires some updates to support new command line
flags. Additionally in order to be able to assert some details in errors
(e.g. that an ECH required err contained expected retry configs) we have
to pipe the `Options` struct deeper into the client/server processing
logic.

To support ECH bogo testing w/ aws-lc-rs as the provider requires taking
a patch on an unreleased aws-lc-rs fix that increases
a `MAX_HKDF_INFO_LEN` constant beyond 80 bytes. In practice when
computing ECH confirmation the info fed through the HKDF interface can
be larger than 80 bytes and without this change the operation fails with
an output length error.

Beyond these changes, it's worth discussing the ignored tests. They're
either not applicable, or need upstream bogo fixes:

"TLS-ECH-Server*":
  We ignore all the TLS-ECH-Server tests. We haven't implemented server
  side ECH yet

"TLS-ECH-Client-ExpectECHOuterExtensions"
"TLS-ECH-Client-CompressSupportedVersions":
  These rely on extension compression between inner/outer hellos. NYI.

"TLS-ECH-Client-SelectECHConfig"
"TLS-ECH-Client-NoSupportedConfigs"
  These are meant to test unsupported configs are handled correctly: we
  happen to support the HPKE ciphersuites that make them "unsupported".
  There's a fix for this upstream we can take later.

"TLS-ECH-Client-SkipInvalidPublicName*":
  Our name validation allows underscores in names. We also don't
  fallback to GREASE when there are no valid ECH configs.

"TLS-ECH-Client-NoSupportedConfigs-GREASE":
  We don't fallback to GREASE for no ECH configs.

"TLS-ECH-Client-Reject-ResumeInnerSession-TLS13":
  This test expects an unexpected extension error in the resumption
  connection, but this only happens if the outer hello didn't include
  GREASE PSK. BoringSSL's impl doesn't. Ours does. As a result we
  produce `:ECH_REJECTED:` instead of :UNEXPECTED_EXTENSION:` and have
  to ignore this test.

"TLS-ECH-Client-TLS12-RejectRetryConfigs"
"TLS-ECH-Client-Reject-NoClientCertificate-TLS12"
"TLS-ECH-Client-Reject-TLS12"
"TLS-ECH-Client-Reject-ResumeInnerSession-TLS12"
"TLS-ECH-GREASE-Client-TLS12-RejectRetryConfigs"
"TLS-ECH-Client-Reject-EarlyDataRejected-TLS12"
"TLS-ECH-Client-Reject-NoClientCertificate-TLS12-Async"
  We never offer/support TLS 1.2 when using ECH. There's no fallback to
  plaintext or GREASE for a server that won't support TLS 1.3

Author: cpu

.github/workflows/build.yml

@@ -232,13 +232,13 @@ jobs:
         uses: dtolnay/rust-toolchain@nightly
 
       - name: Smoke-test benchmark program (ring)
-        run: cargo run -p rustls --profile=bench --locked --example bench
+        run: cargo run -p rustls@0.23.5 --profile=bench --locked --example bench
 
       - name: Smoke-test benchmark program (aws-lc-rs)
-        run: cargo run -p rustls --profile=bench --locked --example bench --no-default-features --features aws_lc_rs,tls12,std
+        run: cargo run -p rustls@0.23.5 --profile=bench --locked --example bench --no-default-features --features aws_lc_rs,tls12,std
 
       - name: Smoke-test benchmark program (fips)
-        run: cargo run -p rustls --profile=bench --locked --example bench --no-default-features --features fips,tls12,std
+        run: cargo run -p rustls@0.23.5 --profile=bench --locked --example bench --no-default-features --features fips,tls12,std
 
       - name: Run micro-benchmarks
         run: cargo bench --locked --all-features
@@ -258,7 +258,7 @@ jobs:
         uses: dtolnay/rust-toolchain@nightly
 
       - name: cargo doc (rustls; all features)
-        run: cargo doc --locked --all-features --no-deps --document-private-items --package rustls
+        run: cargo doc --locked --all-features --no-deps --document-private-items --package rustls@0.23.5
         env:
           RUSTDOCFLAGS: -Dwarnings
 
@@ -403,7 +403,7 @@ jobs:
           components: clippy
         # because examples enable rustls' features, `--workspace --no-default-features` is not
         # the same as `--package rustls --no-default-features` so run it separately
-      - run: cargo clippy --locked --package rustls --no-default-features --all-targets -- --deny warnings
+      - run: cargo clippy --locked --package rustls@0.23.5 --no-default-features --all-targets -- --deny warnings
       - run: cargo clippy --locked --workspace --all-features --all-targets -- --deny warnings
         # not part of the workspace
       - run: cargo clippy --locked --manifest-path=fuzz/Cargo.toml --all-features --all-targets -- --deny warnings
@@ -420,7 +420,7 @@ jobs:
         uses: dtolnay/rust-toolchain@nightly
         with:
           components: clippy
-      - run: cargo clippy --locked --package rustls --no-default-features --all-targets
+      - run: cargo clippy --locked --package rustls@0.23.5 --no-default-features --all-targets
       - run: cargo clippy --locked --workspace --all-features --all-targets
       - run: cargo clippy --locked --manifest-path=fuzz/Cargo.toml --all-features --all-targets
 

Cargo.lock

@@ -31,3 +31,6 @@ lto = "yes"
 # TODO(XXX): Remove this once 0.25 is released - we want the ECH fixes from
 #            https://github.com/hickory-dns/hickory-dns/pull/2183
 hickory-resolver = { git = "https://github.com/hickory-dns/hickory-dns", rev = "6334a01430088ead8642cafaee592ec7bf49831f" }
+
+# TODO(XXX): Remove this once upstream HKDF info size limit increased.
+aws-lc-rs = { git = "https://github.com/cpu/aws-lc-rs", rev = "58f8fe83a42fb65a7efb65f4fe84234211318221" }

Cargo.toml

@@ -31,7 +31,21 @@
     "ServerOCSPCallback*": "",
     "CertCompression*": "not implemented",
     "DuplicateCertCompressionExt*": "",
-    "TLS-ECH-*": "",
+    "TLS-ECH-Server*": "ECH server support NYI",
+    "TLS-ECH-Client-ExpectECHOuterExtensions": "ECH extension compression NYI",
+    "TLS-ECH-Client-CompressSupportedVersions": "ECH extension compression NYI",
+    "TLS-ECH-Client-SelectECHConfig": "TODO(XXX): re-enable after upstream bogo fix",
+    "TLS-ECH-Client-NoSupportedConfigs": "TODO(XXX): re-enable after upstream bogo fix",
+    "TLS-ECH-Client-SkipInvalidPublicName*": "we allow underscore names, don't fallback on no ECH configs",
+    "TLS-ECH-Client-NoSupportedConfigs-GREASE": "we don't fallback to GREASE for no ECH configs",
+    "TLS-ECH-Client-TLS12-RejectRetryConfigs": "we disable TLS1.2 w/ ECH",
+    "TLS-ECH-Client-Reject-NoClientCertificate-TLS12": "we disable TLS1.2 w/ ECH",
+    "TLS-ECH-Client-Reject-TLS12": "we disable TLS1.2 w/ ECH",
+    "TLS-ECH-Client-Reject-ResumeInnerSession-TLS12": "we disable TLS1.2 w/ ECH",
+    "TLS-ECH-GREASE-Client-TLS12-RejectRetryConfigs": "we disable TLS1.2 w/ ECH",
+    "TLS-ECH-Client-Reject-EarlyDataRejected-TLS12": "we disable TLS1.2 w/ ECH",
+    "TLS-ECH-Client-Reject-NoClientCertificate-TLS12-Async": "we disable TLS1.2 w/ ECH",
+    "TLS-ECH-Client-Reject-ResumeInnerSession-TLS13": "assumes no outter GREASE PSK, we send GREASE PSK",
     "ALPS-*": "",
     "*Kyber*": "",
     "ExtraClientEncryptedExtension-*": "we don't implement ALPS",

bogo/config.json.in

@@ -7,11 +7,11 @@ set -xe
 
 case ${BOGO_SHIM_PROVIDER:-aws-lc-rs} in
   ring)
-      cargo build -p rustls --example bogo_shim --no-default-features --features ring,tls12,logging,std
+      cargo build -p rustls@0.23.5 --example bogo_shim --no-default-features --features ring,tls12,logging,std
       cpp -P -DRING config.json.in -oconfig.json
       ;;
   aws-lc-rs)
-      cargo build -p rustls --example bogo_shim --no-default-features --features aws_lc_rs,tls12,logging,std
+      cargo build -p rustls@0.23.5 --example bogo_shim --no-default-features --features aws_lc_rs,tls12,logging,std
       cpp -P -DAWS_LC_RS config.json.in -oconfig.json
       ;;
   existing)

bogo/runme

@@ -48,6 +48,8 @@ rcgen = { version = "0.13", default-features = false, features = ["aws_lc_rs", "
 rustls-pemfile = "2"
 time = { version = "0.3.6", default-features = false }
 webpki-roots = "0.26"
+# TODO(@cpu): Remove once HPKE provider backed by aws-lc-rs and/or ring is available.
+rustls-provider-example = { path = "../provider-example" }
 
 [target.'cfg(not(target_env = "msvc"))'.dev-dependencies]
 tikv-jemallocator = "0.5"