Unexpected matching rule of . appears in the hostname verification func

[x509-name-testing]

Describe the bug
The prefix . (0x2e) is ignored in matching CommonName (CN) when using public_key:pkix_verify_hostname. E.g., the [CN]='..a'(0x2e2e61) can match the [input]='a'(0x61). According to the rules in Stringprep(RFC3454, RFC4518) and PKIX(RFC 5280, RFC 6125), there is no rule about ignoring dot (.) at the prefix of string. It would be a name confusion if two strings are falsely matched in the hostname verification task. So we want to inquire if this is a bug. Looking forward to your views.

To Reproduce
Example code for calling OTP's API for hostname verification.

-module(test_dn).
-export([verify/0]).
verify() ->
	{ok, [[EE]]} = init:get_argument(ee),
	{ok, EEPemBin} = file:read_file(EE),
	[EEPemEntries] = public_key:pem_decode(EEPemBin),
	{_, EeBytes, _} = EEPemEntries,
	EeCertOtp = public_key:pkix_decode_cert(EeBytes, otp),

	{ok, [[Name]]} = init:get_argument(n),
	io:format("~p: ", [Name]),
	case public_key:pkix_verify_hostname(EeCertOtp, [{dns_id, Name}]) of
		true -> io:fwrite("ok\n"), halt(0);
		false -> io:fwrite("name checking failed"), halt(1)
	end.

Example test case of X.509 certificate.

-----BEGIN CERTIFICATE-----
MIIDOzCCAiOgAwIBAgICECEwDQYJKoZIhvcNAQELBQAwYzELMAkGA1UEBhMCVVMx
ITAfBgNVBAoTGFRoZSBHbyBEYWRkeSBHcm91cCwgSW5jLjExMC8GA1UECxMoR28g
RGFkZHkgQ2xhc3MgMiBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0xNjAyMDcx
NzI0MDBaFw0yNDEyMTAwNTA3MTRaMA4xDDAKBgNVBAMTAy4uYTCCASAwDQYJKoZI
hvcNAQEBBQADggENADCCAQgCggEBAKt8ORTxpKElswQ8QtFdXPcB642LllZ/wVmD
HDxvxeubnIb/eBMW8gl4FYyeJoXoGV7YWpKgBiC+LO7rCdsGMhzlYKyhx61LgXYD
M57TkFsTmjk1DoBQUuODX0V4kgf8Td2Gv6DETre/0eEMrNkCTDq6W+d6UHhMor/+
rsK986kqNxDFweyc/2BhlF6yXFpl5BFnXEwEOwRTw0Zjx6jABnOLYAfxZu4oGiSW
T0Zb2Z5vbrq0kuDQ5t2zjFvEUj9pxkoR/QgWsKwA6nRPg2L3BdnLG98dvIZqFyEn
tsKWQ3Z6x8TP/4GSyGfbjoVAmYQJQ5DAoo2MCk8kcEqjZwc5T5kCAQOjUDBOMAwG
A1UdEwEB/wQCMAAwHQYDVR0OBBYEFCTNEK+XDJUYJ45yRL+l+zNeru17MB8GA1Ud
IwQYMBaAFJcswpcBl9XDsKD5xI6eZbKBX8qrMA0GCSqGSIb3DQEBCwUAA4IBAQCp
1nnkQrCtd31yT8cPcvnE83pS8eesGSrdblkzjcvgV56SG9+/boi6/U6u0rbGDqmx
HzmVtg6YsAz4km56sgdPBeVAh4Rm3TjzeFuhCw1YjqSrvL3MVZIMNdbY5Rs2SpxT
szPc5J139UWS5VQpZrwoPOqD7ny72ve1QbNZbkERiY2mC5fJ7/pdPMzms4d/+GOf
G+YMoIf9xYIUexKImHI1OfiLG3PcBd04qolgp/p4k0O3DaF8HBsK4IukWuFGhTUB
zWvN5QOVLYdrVEpF4tvrSb6EKzNp3VQ2ucweKGdF7hNs9Wv1KotnxnScolavfBCW
1XWBeXZVmw1bennWAlfr
-----END CERTIFICATE-----

Examples of input strings: a, .a, ..a, etc.
Example of execution:

erlc test_dn.erl # compile
erl -noshell -run test_dn verify -ee <certFile> -n <inputStr> -s init stop # run

Expected behavior
It's the same as the description.

Affected versions
Testing environment: Erl 22. Not sure about other versions.

Additional context
Note that this behavior appears in matching CN. So, the certificate in the test case does not have the SAN part in certificate extensions. The matching CN as hostname rule is in the RFC 6125.

From the functionality of CN in a certificate, there is no rule that the name string and input string should be fully qualified domain names (FQDN). Such strings like ..a should be acceptable to be stored in an X.509 certificate.

[IngelaAndin]

Th verification is based on verifying the Server Name Indication extension (that will fall back to the Host argument of connect) with appropriate certificate extension (that fallbacks to the common name).

From description of SNI (Server Name Indication) RFC 6066:

"HostName" contains the fully qualified DNS hostname of the server,
as understood by the client. The hostname is represented as a byte
string using ASCII encoding without a trailing dot.

So prefix should be ok!

I tried you example on current OTP release (OTP-26) and (OTP-22.0) the result is:

 > public_key:pkix_verify_hostname(C, [{dns_id, "..a"}]).
true

[x509-name-testing]

Hi there, Thanks for your reply.

I tried you example on current OTP release (OTP-26) and (OTP-22.0) the result is:

 > public_key:pkix_verify_hostname(C, [{dns_id, "..a"}]).
true

Following your reply, do you think the the [CN]='..a' can match [input]='a' or [input]='.a' should be acceptable of OTP? E.g.:

> public_key:pkix_verify_hostname(C, [{dns_id, ".a"}]). % subject CN='..a' in C
true

---

BTW, based on my understanding, the RFC 6066 (TLS Extensions) is designed for TLS, which is an application of PKI and PKIX. The more direct reference should be RFC 5280 (X.509 PKI) and RFC 6125(PKIX).
RFC 6125:

   If a subjectAltName extension of type dNSName is present, that MUST
   be used as the identity.  Otherwise, the (most specific) Common Name
   field in the Subject field of the certificate MUST be used.

And RFC 5280 Section 7.1:

   Conforming implementations MUST use the LDAP StringPrep profile
   (including insignificant space handling), as specified in [RFC4518],
   as the basis for comparison of distinguished name attributes encoded
   in either PrintableString or UTF8String.

There should be no ignoring the prefix "." rule.

---

In summary, in my personal view, OTP should first follow the rules from PKIX. Because there is no rule that the CN should only support FQDN, some strings like ..a should also be meaningful in CN. But OTP ignoring dot can cause name confusion. I'd like to hear the comments from the developing team, and I'd like to have a discussion. Looking forward to your reply.

[IngelaAndin]

Oh sorry, now I see you are complaining

public_key:pkix_verify_hostname(C, [{dns_id, ".a"}]).

returns true not that

public_key:pkix_verify_hostname(C, [{dns_id, "..a"}]).

does not return true. It is true that public_key should follow PKIX spec. I work a lot with TLS so that is why its my default perspective. I will have to look into this an come back to you.

[IngelaAndin]

@x509-name-testing Yes I think it is a bug, and #8023 should solve it.