OpenSSL SSL_connect Connection was reset in connection to 127.0.0.1:664 Marked for [closure] Failed HTTPS connection

[ganeshkamath89]

I have 2 Curl Verbose logs. Both logs are with the same libCurl client (7.74). The differences are as follows:

1. In the Success case (with Curl - new server), I am getting back information from the Server. The Server application was recently modified to accomodate a fix but it will be in production several years from now.
2. In the Failure case (with Curl), Application is exiting after a Client Hello. There is no Server Hello in the wireshark logs. Unfortunately I cannot share the code or Wireshark log used with our application (sorry). However I can share a very similar code:

Our Curl based client side code looks similar to: https://github.com/Openwsman/openwsman/blob/master/src/lib/wsman-curl-client-transport.c

And WinHTTP based client side code looks similar to: https://github.com/Openwsman/openwsman/blob/master/src/lib/wsman-win-client-transport.c

Note: I can share specific sections of Wireshark log if required, but I don't know if that will help.

We used to support our client with WinHTTP in the past. With our WinHTTP based client application the server application is working with current production server application as well as the one that will be in production in the future. We switched over to Curl because Curl supports OpenSSL with TLS 1.3 support.

When we run HTTP based query it does not have any problem - client works with old and new server application. But in HTTPS I am facing this issue.

I have captured Verbose log just in case someone knows how to analyze these logs and help us resolve our problem. We want to change our client code so that it too works for the currently deployed server application.

I would appreciate any clue as to how to resolve this issue.

My environment:
OS: Windows 10
Curl Version: 7.74 (libcurl)
Mode: Build from source using VS project 2017
TLS Backend: OpenSSL 1.1.1i
Curl usage: As part of Client application

Successlog (with WinHTTP):

* STATE: INIT => CONNECT handle 0x37d0358; line 1654 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* family0 == v4, family1 == v6
*   Trying 127.0.0.1:664...
* STATE: CONNECT => WAITCONNECT handle 0x37d0358; line 1715 (connection #0)
* Connected to 127.0.0.1 (127.0.0.1) port 664 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x37d0358; line 1848 (connection #0)
* Marked for [keep alive]: HTTP default
* ALPN, offering http/1.1
* STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x37d0358; line 1866 (connection #0)
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=IN; ST=KA; O=PostOffice; OU=Post Delivery; CN=*.postoffice.com
*  start date: May 27 04:00:56 2020 GMT
*  expire date: May 25 04:00:56 2030 GMT
*  issuer: O=PostOffice; [email protected]; L=Bangalore; ST=KA; C=IN; CN=*.postoffice.com; OU=Post Delivery
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* STATE: PROTOCONNECT => DO handle 0x37d0358; line 1885 (connection #0)
> POST /postcard HTTP/1.1
Host: 127.0.0.1:664
Accept: */*
Content-Type: application/soap+xml;charset=UTF-8
User-Agent: Openwsman
Content-Length: 885

* upload completely sent off: 885 out of 885 bytes
* STATE: DO => DO_DONE handle 0x37d0358; line 1940 (connection #0)
* STATE: DO_DONE => PERFORM handle 0x37d0358; line 2061 (connection #0)
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 401 Unauthorized
< Content-Length: 0
< Connection: Keep-Alive
< Content-Type: application/soap+xml;charset=UTF-8
< WWW-Authenticate: Digest realm="PostOffice", nonce="5bf1156647e8eb42", algorithm="MD5", opaque="661d9eae"
<
* STATE: PERFORM => DONE handle 0x37d0358; line 2251 (connection #0)
* multi_done
* Connection #0 to host 127.0.0.1 left intact
* Expire cleared (transfer 0x37d0358)
* STATE: INIT => CONNECT handle 0x37d0358; line 1654 (connection #-5000)
* Found bundle for host 127.0.0.1: 0x3784b80 [serially]
* Re-using existing connection! (#0) with host 127.0.0.1
* Connected to 127.0.0.1 (127.0.0.1) port 664 (#0)
* STATE: CONNECT => DO handle 0x37d0358; line 1708 (connection #0)
* Server auth using Digest with user 'PostMan'
> POST /postcard HTTP/1.1
Host: 127.0.0.1:664
Authorization: Digest username="PostMan", realm="PostOffice", nonce="5bf1156647e8eb42", uri="/postcard", response="2af51f2f1272d36eca7457e222da5e2c", opaque="661d9eae", algorithm=MD5
Accept: */*
Content-Type: application/soap+xml;charset=UTF-8
User-Agent: Openwsman
Content-Length: 885

* upload completely sent off: 885 out of 885 bytes
* STATE: DO => DO_DONE handle 0x37d0358; line 1940 (connection #0)
* STATE: DO_DONE => PERFORM handle 0x37d0358; line 2061 (connection #0)
* Mark bundle as not supporting multiuse
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 200 OK
< Content-Length: 1239
< Connection: Keep-Alive
< Content-Type: application/soap+xml;charset=UTF-8
<
* STATE: PERFORM => DONE handle 0x37d0358; line 2251 (connection #0)
* multi_done
* Connection #0 to host 127.0.0.1 left intact
* Expire cleared (transfer 0x37d0358)
* STATE: INIT => CONNECT handle 0x37d0358; line 1654 (connection #-5000)
* Found bundle for host 127.0.0.1: 0x3784b80 [serially]
* Re-using existing connection! (#0) with host 127.0.0.1
* Connected to 127.0.0.1 (127.0.0.1) port 664 (#0)
* STATE: CONNECT => DO handle 0x37d0358; line 1708 (connection #0)
* Server auth using Digest with user 'PostMan'
> POST /postcard HTTP/1.1
Host: 127.0.0.1:664
Authorization: Digest username="PostMan", realm="PostOffice", nonce="5bf1156647e8eb42", uri="/postcard", response="2af51f2f1272d36eca7457e222da5e2c", opaque="661d9eae", algorithm=MD5
Accept: */*
Content-Type: application/soap+xml;charset=UTF-8
User-Agent: Openwsman
Content-Length: 947

* upload completely sent off: 947 out of 947 bytes
* STATE: DO => DO_DONE handle 0x37d0358; line 1940 (connection #0)
* STATE: DO_DONE => PERFORM handle 0x37d0358; line 2061 (connection #0)
* Mark bundle as not supporting multiuse
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 200 OK
< Content-Length: 2188
< Connection: Keep-Alive
< Content-Type: application/soap+xml;charset=UTF-8
<
* STATE: PERFORM => DONE handle 0x37d0358; line 2251 (connection #0)
* multi_done
* Connection #0 to host 127.0.0.1 left intact
* Expire cleared (transfer 0x37d0358)
* STATE: INIT => CONNECT handle 0x37d0358; line 1654 (connection #-5000)
* Found bundle for host 127.0.0.1: 0x3784b80 [serially]
* Re-using existing connection! (#0) with host 127.0.0.1
* Connected to 127.0.0.1 (127.0.0.1) port 664 (#0)
* STATE: CONNECT => DO handle 0x37d0358; line 1708 (connection #0)
* Server auth using Digest with user 'PostMan'
> POST /postcard HTTP/1.1
Host: 127.0.0.1:664
Authorization: Digest username="PostMan", realm="PostOffice", nonce="5bf1156647e8eb42", uri="/postcard", response="2af51f2f1272d36eca7457e222da5e2c", opaque="661d9eae", algorithm=MD5
Accept: */*
Content-Type: application/soap+xml;charset=UTF-8
User-Agent: Openwsman
Content-Length: 1022

* upload completely sent off: 1022 out of 1022 bytes
* STATE: DO => DO_DONE handle 0x37d0358; line 1940 (connection #0)
* STATE: DO_DONE => PERFORM handle 0x37d0358; line 2061 (connection #0)
* Mark bundle as not supporting multiuse
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 200 OK
< Content-Length: 3765
< Connection: Keep-Alive
< Content-Type: application/soap+xml;charset=UTF-8
<
* STATE: PERFORM => DONE handle 0x37d0358; line 2251 (connection #0)
* multi_done
* Connection #0 to host 127.0.0.1 left intact
* Expire cleared (transfer 0x37d0358)

* STATE: INIT => CONNECT handle 0x37d0358; line 1654 (connection #-5000)
* Found bundle for host 127.0.0.1: 0x3784b80 [serially]
* Re-using existing connection! (#0) with host 127.0.0.1
* Connected to 127.0.0.1 (127.0.0.1) port 664 (#0)
* STATE: CONNECT => DO handle 0x37d0358; line 1708 (connection #0)
* Server auth using Digest with user 'PostMan'
> POST /postcard HTTP/1.1
Host: 127.0.0.1:664
Authorization: Digest username="PostMan", realm="PostOffice", nonce="5bf1156647e8eb42", uri="/postcard", response="2af51f2f1272d36eca7457e222da5e2c", opaque="661d9eae", algorithm=MD5
Accept: */*
Content-Type: application/soap+xml;charset=UTF-8
User-Agent: Openwsman
Content-Length: 2053

* upload completely sent off: 2053 out of 2053 bytes
* STATE: DO => DO_DONE handle 0x37d0358; line 1940 (connection #0)
* STATE: DO_DONE => PERFORM handle 0x37d0358; line 2061 (connection #0)
* Mark bundle as not supporting multiuse
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 200 OK
< Content-Length: 3764
< Connection: Keep-Alive
< Content-Type: application/soap+xml;charset=UTF-8
<
* STATE: PERFORM => DONE handle 0x37d0358; line 2251 (connection #0)
* multi_done
* Connection #0 to host 127.0.0.1 left intact
* Expire cleared (transfer 0x37d0358)
* STATE: INIT => CONNECT handle 0x37d0358; line 1654 (connection #-5000)
* Found bundle for host 127.0.0.1: 0x3784b80 [serially]
* Re-using existing connection! (#0) with host 127.0.0.1
* Connected to 127.0.0.1 (127.0.0.1) port 664 (#0)
* STATE: CONNECT => DO handle 0x37d0358; line 1708 (connection #0)
* Server auth using Digest with user 'PostMan'
> POST /postcard HTTP/1.1
Host: 127.0.0.1:664
Authorization: Digest username="PostMan", realm="PostOffice", nonce="5bf1156647e8eb42", uri="/postcard", response="2af51f2f1272d36eca7457e222da5e2c", opaque="661d9eae", algorithm=MD5
Accept: */*
Content-Type: application/soap+xml;charset=UTF-8
User-Agent: Openwsman
Content-Length: 2041

* upload completely sent off: 2041 out of 2041 bytes
* STATE: DO => DO_DONE handle 0x37d0358; line 1940 (connection #0)
* STATE: DO_DONE => PERFORM handle 0x37d0358; line 2061 (connection #0)
* Mark bundle as not supporting multiuse
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 200 OK
< Content-Length: 2179
< Connection: Keep-Alive
< Content-Type: application/soap+xml;charset=UTF-8
<
* STATE: PERFORM => DONE handle 0x37d0358; line 2251 (connection #0)
* multi_done
* Connection #0 to host 127.0.0.1 left intact
* Expire cleared (transfer 0x37d0358)

Failure Log (with Curl):

* STATE: INIT => CONNECT handle 0x2b92918; line 1654 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* family0 == v4, family1 == v6
*   Trying 127.0.0.1:664...
* STATE: CONNECT => WAITCONNECT handle 0x2b92918; line 1715 (connection #0)
* Connected to 127.0.0.1 (127.0.0.1) port 664 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x2b92918; line 1848 (connection #0)
* Marked for [keep alive]: HTTP default
* ALPN, offering http/1.1
* STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x2b92918; line 1866 (connection #0)
* OpenSSL SSL_connect: Connection was reset in connection to 127.0.0.1:664
* Marked for [closure]: Failed HTTPS connection
* multi_done
* The cache now contains 0 members
* Closing connection 0
* Expire cleared (transfer 0x2b92918)
Error: Connection Failed : SSL connection error

Few things I have tried:

1. Hardcoding "CURLOPT_SSLVERSION" to 3,4,5,6 - not helping
2. Hardcoding "CURLOPT_SSL_VERIFYPEER" & "CURLOPT_SSL_VERIFYPEER" to 0
3. Tried setting to TLSv1 cipher suite using:
   curl_easy_setopt(curl, CURLOPT_SSL_CIPHER_LIST, "TLSv1"); - Same error seen
4. As the failing Server implements cipher suite "AES128-SHA", tried setting to "AES128-SHA" cipher suite using:
   curl_easy_setopt(curl, CURLOPT_SSL_CIPHER_LIST, "AES128-SHA"); - Same error seen

[jay]

Note: I can share specific sections of Wireshark log if required, but I don't know if that will help.

What does it show after the client hello? Does the server send an alert or anything at all?

[ganeshkamath89]

Hi @jay

Here is the screenshot of Wireshark capture of server reset response.

Both screenshots are for failure case itself:

Failure case 1: Client and Server on the same system and using IP 127.0.01 for connection.

Failure case 2: Client and Server on different systems.
Client IP (System with Curl): 10.252.37.10
Server IP: 10.138.152.190

Failure Client hello wireshark log (Current server - Curl client):

Frame 9: 561 bytes on wire (4488 bits), 561 bytes captured (4488 bits) on interface \Device\NPF_Loopback, id 2
Null/Loopback
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 62349 (62349), Dst Port: oob-ws-https (664), Seq: 1, Ack: 1, Len: 517
Transport Layer Security
    TLSv1 Record Layer: Handshake Protocol: Client Hello
        Content Type: Handshake (22)
        Version: TLS 1.0 (0x0301)
        Length: 512
        Handshake Protocol: Client Hello
            Handshake Type: Client Hello (1)
            Length: 508
            Version: TLS 1.2 (0x0303)
            Random: f6fa9910ecc8cc484a5d0fe607a44d5aece1388189cd22bcbb73b2b2bb9404d9
                GMT Unix Time: Apr 22, 2101 18:35:20.000000000 India Standard Time
                Random Bytes: ecc8cc484a5d0fe607a44d5aece1388189cd22bcbb73b2b2bb9404d9
            Session ID Length: 32
            Session ID: 837b1c5017abd1769c33a2427edd669dc3d4dae1086e711d27bcde154c954829
            Cipher Suites Length: 62
            Cipher Suites (31 suites)
                Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302)
                Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303)
                Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)
                Cipher Suite: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
                Cipher Suite: TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xccaa)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x006b)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (0x0067)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
                Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
                Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
                Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)
                Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)
                Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
                Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
                Cipher Suite: TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff)
            Compression Methods Length: 1
            Compression Methods (1 method)
                Compression Method: null (0)
            Extensions Length: 373
            Extension: ec_point_formats (len=4)
                Type: ec_point_formats (11)
                Length: 4
                EC point formats Length: 3
                Elliptic curves point formats (3)
                    EC point format: uncompressed (0)
                    EC point format: ansiX962_compressed_prime (1)
                    EC point format: ansiX962_compressed_char2 (2)
            Extension: supported_groups (len=12)
                Type: supported_groups (10)
                Length: 12
                Supported Groups List Length: 10
                Supported Groups (5 groups)
                    Supported Group: x25519 (0x001d)
                    Supported Group: secp256r1 (0x0017)
                    Supported Group: x448 (0x001e)
                    Supported Group: secp521r1 (0x0019)
                    Supported Group: secp384r1 (0x0018)
            Extension: next_protocol_negotiation (len=0)
                Type: next_protocol_negotiation (13172)
                Length: 0
            Extension: application_layer_protocol_negotiation (len=11)
                Type: application_layer_protocol_negotiation (16)
                Length: 11
                ALPN Extension Length: 9
                ALPN Protocol
                    ALPN string length: 8
                    ALPN Next Protocol: http/1.1
            Extension: encrypt_then_mac (len=0)
                Type: encrypt_then_mac (22)
                Length: 0
            Extension: extended_master_secret (len=0)
                Type: extended_master_secret (23)
                Length: 0
            Extension: post_handshake_auth (len=0)
                Type: post_handshake_auth (49)
                Length: 0
            Extension: signature_algorithms (len=48)
                Type: signature_algorithms (13)
                Length: 48
                Signature Hash Algorithms Length: 46
                Signature Hash Algorithms (23 algorithms)
                    Signature Algorithm: ecdsa_secp256r1_sha256 (0x0403)
                    Signature Algorithm: ecdsa_secp384r1_sha384 (0x0503)
                    Signature Algorithm: ecdsa_secp521r1_sha512 (0x0603)
                    Signature Algorithm: ed25519 (0x0807)
                    Signature Algorithm: ed448 (0x0808)
                    Signature Algorithm: rsa_pss_pss_sha256 (0x0809)
                    Signature Algorithm: rsa_pss_pss_sha384 (0x080a)
                    Signature Algorithm: rsa_pss_pss_sha512 (0x080b)
                    Signature Algorithm: rsa_pss_rsae_sha256 (0x0804)
                    Signature Algorithm: rsa_pss_rsae_sha384 (0x0805)
                    Signature Algorithm: rsa_pss_rsae_sha512 (0x0806)
                    Signature Algorithm: rsa_pkcs1_sha256 (0x0401)
                    Signature Algorithm: rsa_pkcs1_sha384 (0x0501)
                    Signature Algorithm: rsa_pkcs1_sha512 (0x0601)
                    Signature Algorithm: SHA224 ECDSA (0x0303)
                    Signature Algorithm: ecdsa_sha1 (0x0203)
                    Signature Algorithm: SHA224 RSA (0x0301)
                    Signature Algorithm: rsa_pkcs1_sha1 (0x0201)
                    Signature Algorithm: SHA224 DSA (0x0302)
                    Signature Algorithm: SHA1 DSA (0x0202)
                    Signature Algorithm: SHA256 DSA (0x0402)
                    Signature Algorithm: SHA384 DSA (0x0502)
                    Signature Algorithm: SHA512 DSA (0x0602)
            Extension: supported_versions (len=9)
                Type: supported_versions (43)
                Length: 9
                Supported Versions length: 8
                Supported Version: TLS 1.3 (0x0304)
                Supported Version: TLS 1.2 (0x0303)
                Supported Version: TLS 1.1 (0x0302)
                Supported Version: TLS 1.0 (0x0301)
            Extension: psk_key_exchange_modes (len=2)
                Type: psk_key_exchange_modes (45)
                Length: 2
                PSK Key Exchange Modes Length: 1
                PSK Key Exchange Mode: PSK with (EC)DHE key establishment (psk_dhe_ke) (1)
            Extension: key_share (len=38)
                Type: key_share (51)
                Length: 38
                Key Share extension
                    Client Key Share Length: 36
                    Key Share Entry: Group: x25519, Key Exchange length: 32
                        Group: x25519 (29)
                        Key Exchange Length: 32
                        Key Exchange: 87eec2022b15dd01a173b3a7c04c455049e3d5de9a7612d208ddadba614fe332
            Extension: padding (len=201)
                Type: padding (21)
                Length: 201
                Padding Data: 000000000000000000000000000000000000000000000000000000000000000000000000…

Success Client Hello (future server - Curl client):

Transport Layer Security
    TLSv1.3 Record Layer: Handshake Protocol: Client Hello
        Content Type: Handshake (22)
        Version: TLS 1.0 (0x0301)
        Length: 512
        Handshake Protocol: Client Hello
            Handshake Type: Client Hello (1)
            Length: 508
            Version: TLS 1.2 (0x0303)
            Random: 24f51c30e088cc5baf8febc649228db968d20f8447f72aec11bfb5b4e383f0c2
            Session ID Length: 32
            Session ID: 0daf1b814101e830f7592590280c93744027564faec26cefb47a46ce5ad6856f
            Cipher Suites Length: 62
            Cipher Suites (31 suites)
                Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302)
                Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303)
                Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)
                Cipher Suite: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
                Cipher Suite: TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xccaa)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x006b)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (0x0067)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
                Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
                Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
                Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)
                Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)
                Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
                Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
                Cipher Suite: TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff)
            Compression Methods Length: 1
            Compression Methods (1 method)
                Compression Method: null (0)
            Extensions Length: 373
            Extension: ec_point_formats (len=4)
                Type: ec_point_formats (11)
                Length: 4
                EC point formats Length: 3
                Elliptic curves point formats (3)
                    EC point format: uncompressed (0)
                    EC point format: ansiX962_compressed_prime (1)
                    EC point format: ansiX962_compressed_char2 (2)
            Extension: supported_groups (len=12)
                Type: supported_groups (10)
                Length: 12
                Supported Groups List Length: 10
                Supported Groups (5 groups)
                    Supported Group: x25519 (0x001d)
                    Supported Group: secp256r1 (0x0017)
                    Supported Group: x448 (0x001e)
                    Supported Group: secp521r1 (0x0019)
                    Supported Group: secp384r1 (0x0018)
            Extension: next_protocol_negotiation (len=0)
                Type: next_protocol_negotiation (13172)
                Length: 0
            Extension: application_layer_protocol_negotiation (len=11)
                Type: application_layer_protocol_negotiation (16)
                Length: 11
                ALPN Extension Length: 9
                ALPN Protocol
                    ALPN string length: 8
                    ALPN Next Protocol: http/1.1
            Extension: encrypt_then_mac (len=0)
                Type: encrypt_then_mac (22)
                Length: 0
            Extension: extended_master_secret (len=0)
                Type: extended_master_secret (23)
                Length: 0
            Extension: post_handshake_auth (len=0)
                Type: post_handshake_auth (49)
                Length: 0
            Extension: signature_algorithms (len=48)
                Type: signature_algorithms (13)
                Length: 48
                Signature Hash Algorithms Length: 46
                Signature Hash Algorithms (23 algorithms)
                    Signature Algorithm: ecdsa_secp256r1_sha256 (0x0403)
                    Signature Algorithm: ecdsa_secp384r1_sha384 (0x0503)
                    Signature Algorithm: ecdsa_secp521r1_sha512 (0x0603)
                    Signature Algorithm: ed25519 (0x0807)
                    Signature Algorithm: ed448 (0x0808)
                    Signature Algorithm: rsa_pss_pss_sha256 (0x0809)
                    Signature Algorithm: rsa_pss_pss_sha384 (0x080a)
                    Signature Algorithm: rsa_pss_pss_sha512 (0x080b)
                    Signature Algorithm: rsa_pss_rsae_sha256 (0x0804)
                    Signature Algorithm: rsa_pss_rsae_sha384 (0x0805)
                    Signature Algorithm: rsa_pss_rsae_sha512 (0x0806)
                    Signature Algorithm: rsa_pkcs1_sha256 (0x0401)
                    Signature Algorithm: rsa_pkcs1_sha384 (0x0501)
                    Signature Algorithm: rsa_pkcs1_sha512 (0x0601)
                    Signature Algorithm: SHA224 ECDSA (0x0303)
                    Signature Algorithm: ecdsa_sha1 (0x0203)
                    Signature Algorithm: SHA224 RSA (0x0301)
                    Signature Algorithm: rsa_pkcs1_sha1 (0x0201)
                    Signature Algorithm: SHA224 DSA (0x0302)
                    Signature Algorithm: SHA1 DSA (0x0202)
                    Signature Algorithm: SHA256 DSA (0x0402)
                    Signature Algorithm: SHA384 DSA (0x0502)
                    Signature Algorithm: SHA512 DSA (0x0602)
            Extension: supported_versions (len=9)
                Type: supported_versions (43)
                Length: 9
                Supported Versions length: 8
                Supported Version: TLS 1.3 (0x0304)
                Supported Version: TLS 1.2 (0x0303)
                Supported Version: TLS 1.1 (0x0302)
                Supported Version: TLS 1.0 (0x0301)
            Extension: psk_key_exchange_modes (len=2)
                Type: psk_key_exchange_modes (45)
                Length: 2
                PSK Key Exchange Modes Length: 1
                PSK Key Exchange Mode: PSK with (EC)DHE key establishment (psk_dhe_ke) (1)
            Extension: key_share (len=38)
                Type: key_share (51)
                Length: 38
                Key Share extension
                    Client Key Share Length: 36
                    Key Share Entry: Group: x25519, Key Exchange length: 32
                        Group: x25519 (29)
                        Key Exchange Length: 32
                        Key Exchange: 488667b4edfe6c856df78cd5350302fece6a986964e46048ed8406ad52644835
            Extension: padding (len=201)
                Type: padding (21)
                Length: 201
                Padding Data: 000000000000000000000000000000000000000000000000000000000000000000000000…

Success Client Hello Wireshark log (WinHTTP client):

Frame 4: 202 bytes on wire (1616 bits), 202 bytes captured (1616 bits) on interface \Device\NPF_Loopback, id 2
Null/Loopback
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 62331 (62331), Dst Port: oob-ws-https (664), Seq: 1, Ack: 1, Len: 158
Transport Layer Security
    TLSv1.2 Record Layer: Handshake Protocol: Client Hello
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 153
        Handshake Protocol: Client Hello
            Handshake Type: Client Hello (1)
            Length: 149
            Version: TLS 1.2 (0x0303)
            Random: 6001e363f0a9635b2dd60fdabfda3f08f0a3005314fa654b3c32a3b358e97bb2
                GMT Unix Time: Jan 16, 2021 00:18:03.000000000 India Standard Time
                Random Bytes: f0a9635b2dd60fdabfda3f08f0a3005314fa654b3c32a3b358e97bb2
            Session ID Length: 0
            Cipher Suites Length: 38
            Cipher Suites (19 suites)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
                Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
                Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
                Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)
                Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)
                Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
                Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
                Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)
            Compression Methods Length: 1
            Compression Methods (1 method)
                Compression Method: null (0)
            Extensions Length: 70
            Extension: status_request (len=5)
                Type: status_request (5)
                Length: 5
                Certificate Status Type: OCSP (1)
                Responder ID list Length: 0
                Request Extensions Length: 0
            Extension: supported_groups (len=8)
                Type: supported_groups (10)
                Length: 8
                Supported Groups List Length: 6
                Supported Groups (3 groups)
                    Supported Group: x25519 (0x001d)
                    Supported Group: secp256r1 (0x0017)
                    Supported Group: secp384r1 (0x0018)
            Extension: ec_point_formats (len=2)
                Type: ec_point_formats (11)
                Length: 2
                EC point formats Length: 1
                Elliptic curves point formats (1)
                    EC point format: uncompressed (0)
            Extension: signature_algorithms (len=26)
                Type: signature_algorithms (13)
                Length: 26
                Signature Hash Algorithms Length: 24
                Signature Hash Algorithms (12 algorithms)
                    Signature Algorithm: rsa_pss_rsae_sha256 (0x0804)
                    Signature Algorithm: rsa_pss_rsae_sha384 (0x0805)
                    Signature Algorithm: rsa_pss_rsae_sha512 (0x0806)
                    Signature Algorithm: rsa_pkcs1_sha256 (0x0401)
                    Signature Algorithm: rsa_pkcs1_sha384 (0x0501)
                    Signature Algorithm: rsa_pkcs1_sha1 (0x0201)
                    Signature Algorithm: ecdsa_secp256r1_sha256 (0x0403)
                    Signature Algorithm: ecdsa_secp384r1_sha384 (0x0503)
                    Signature Algorithm: ecdsa_sha1 (0x0203)
                    Signature Algorithm: SHA1 DSA (0x0202)
                    Signature Algorithm: rsa_pkcs1_sha512 (0x0601)
                    Signature Algorithm: ecdsa_secp521r1_sha512 (0x0603)
            Extension: session_ticket (len=0)
                Type: session_ticket (35)
                Length: 0
                Data (0 bytes)
            Extension: extended_master_secret (len=0)
                Type: extended_master_secret (23)
                Length: 0
            Extension: renegotiation_info (len=1)
                Type: renegotiation_info (65281)
                Length: 1
                Renegotiation Info extension
                    Renegotiation info extension length: 0

[ganeshkamath89]

Hi @jay. With this change I am getting the error message:
SSL certificate problem: unable to get local issuer certificate

Note: We want to support TLS 1.3 in our application, that's why we moved to libCurl.

[ganeshkamath89]

Hi @jay. Is there any option to skip SSL verification with libcurl?

[ganeshkamath89]

Hi @jay.

I generated a self signed certificate and installed it locally. Now I am getting success response with current Server for TLS 1.2.

But it is still failing when I change the MAX TLS version to 1.3.

[jay]

SSL certificate problem: unable to get local issuer certificate

SSL verification is enabled so you would need to provide the certificate. The server has responded to the TLS 1.2 ClientHello but not the TLS 1.3 ClientHello. It's possible the server does not support TLS 1.3, though I thought the TLS 1.3 ClientHello was supposed to be compatible with TLS 1.2 servers. I would find out what fixes were made to the server and if it supports TLS 1.3. If it does you may have to go through the ClientHello 1.3 and make it more like 1.2, if that's even possible.

Another thing you can do is try connecting to the server using the curl tool and not your application just to see what happens.
curl -v -k https://127.0.0.1:664
curl -v -k --tls-max 1.2 https://127.0.0.1:664

[ganeshkamath89]

Hi @jay. Thank you very much. I will try this and write back to you after some time by running Curl binary.

If you had not suggested these changes, we might not have found a solution.

Really appreciate your help.

[jay]

This is likely not a curl issue. Compare that ClientHello against one of the successful ones. The only thing unusual that I notice is no SNI (server_name) extension is sent.

[ganeshkamath89]

Hi @jay
Thank you, I will try this.

Even today I am seeing that if I replace just the WinHTTP file with Curl file, my HTTPS connection is succeeding. I not saying there is an issue with Curl, I would actually like to know what I am doing wrong while using Curl, so that i can fix it. These files are very close to what we use.

File using WinHTTP: https://github.com/Openwsman/openwsman/blob/master/src/lib/wsman-win-client-transport.c
File using Curl: https://github.com/Openwsman/openwsman/blob/master/src/lib/wsman-curl-client-transport.c

[ganeshkamath89]

Hi @jay, How do I set the SNI using curl opt header?

[ganeshkamath89]

Hi @jay. I tried the following Curl Opt command: https://curl.se/libcurl/c/CURLOPT_RESOLVE.html. But I am still getting the same error.

[jay]

Even today I am seeing that if I replace just the WinHTTP file with Curl file, my HTTPS connection is succeeding. I not saying there is an issue with Curl, I would actually like to know what I am doing wrong while using Curl, so that i can fix it. These files are very close to what we use.

There's not enough information here to know that you're doing anything wrong. It may be the server. If you have some success and some failures with the same client using libcurl then compare clienthello. If you cannot connect at all to the server when your client is using libcurl then that is a different question.

How do I set the SNI using curl opt header?

There is no option to set SNI, it is set to whatever the hostname is if there is one. That is not the problem as you have ClientHello successes without it.

[ganeshkamath89]

If you cannot connect at all to the server when your client is using libcurl then that is a different question.

Hi @jay. I am not able to connect to the current server whent he client is using libcurl. What additional inputs can I provide?

[ganeshkamath89]

Hi @jay
From client Hello, i am just capturing extensions look for success and failure cases from previously shared packets. do you think if I add the same extensions in failure case, it will enable a successful Curl based connection. If so, how to enable these extensions?

One of the main differences I can see is that the success extensions are shorter in length. But I don't know which extension is the key differentiator that will enable a connection using Curl.

Is there some table I can refer while using libcurl code to pick and choose which extentions to enable in Client Hello corresponding to curlopt flag, so that I can make a custom Client Hello with just the required set of extensions?

Success Extensions (using WinHTTP for connection):

Handshake Protocol: Client Hello
    Handshake Type: Client Hello (1)
    Length: 149
    Version: TLS 1.2 (0x0303)
    Random: 6001e363f0a9635b2dd60fdabfda3f08f0a3005314fa654b3c32a3b358e97bb2
    Session ID Length: 0
    Cipher Suites Length: 38
    Cipher Suites (19 suites)
    Compression Methods Length: 1
    Compression Methods (1 method)
    Extensions Length: 70
    Extension: status_request (len=5)
    Extension: supported_groups (len=8)
    Extension: ec_point_formats (len=2)
    Extension: signature_algorithms (len=26)
    Extension: session_ticket (len=0)
    Extension: extended_master_secret (len=0)
    Extension: renegotiation_info (len=1)

Failure extensions (Using Curl for connection):

Handshake Protocol: Client Hello
    Handshake Type: Client Hello (1)
    Length: 508
    Version: TLS 1.2 (0x0303)
    Random: f6fa9910ecc8cc484a5d0fe607a44d5aece1388189cd22bcbb73b2b2bb9404d9
    Session ID Length: 32
    Session ID: 837b1c5017abd1769c33a2427edd669dc3d4dae1086e711d27bcde154c954829
    Cipher Suites Length: 62
    Cipher Suites (31 suites)
    Compression Methods Length: 1
    Compression Methods (1 method)
    Extensions Length: 373
    Extension: ec_point_formats (len=4)
    Extension: supported_groups (len=12)
    Extension: next_protocol_negotiation (len=0)
    Extension: application_layer_protocol_negotiation (len=11)
    Extension: encrypt_then_mac (len=0)
    Extension: extended_master_secret (len=0)
    Extension: post_handshake_auth (len=0)
    Extension: signature_algorithms (len=48)
    Extension: supported_versions (len=9)
    Extension: psk_key_exchange_modes (len=2)
    Extension: key_share (len=38)
    Extension: padding (len=201)

[jay]

There is no indication that this is the correct course to take. Please refer to my earlier response to your comment regarding client hello.

[ganeshkamath89]

Hi @jay. Thanks. We have concluded that the cipher "AES128-SHA" only supports TLS 1.2. For TLS 1.3 we have asked the server implementation team to add new ciphers.

[jay]

AES128-SHA is openssl's name for the TLS ciphersuite TLS_RSA_WITH_AES_128_CBC_SHA which is the mandatory cipher suite for TLS 1.2. It is included by default when the minimum allowed TLS version in curl is less than TLS 1.3 unless you override the cipher list. You can confirm this by looking at your failed curl w/ openssl TLS 1.3 ClientHello (which should be compatible with TLS 1.2) and it will show that cipher in the list.

In most cases it is not correct to override the cipher list and I suspect support for additional ciphers may not solve your problem. Probably there is some issue with the server understanding the ClientHello. However in those cases usually the server would reply with a TLS alert handshake failure, and in your case it just hangs up. Again I suggest find out how they fixed the server because it could save you time investigating how to change the ClientHello. This assumes the server supports TLS 1.3 which is another thing you would need to find out about.

[tgoeg]

I don't know if this is related, but I see a very similar issue with a very similar problem in the client hello:
The Random field in the unsuccessful requests always have a very low starting string, which gets interpreted as a (wrong) "GMT Unix Time".
This feels like some kind of bug in openssl?

[jay]

My understanding of this is TLS 1.2 RFC says to use the first 4 bytes of the client random as the date/time gmt and the rest 28 as random bytes. I just checked the TLS 1.3 RFC and it says different, just 32 random bytes: 32 bytes generated by a secure random number generator.. So though your packet capture software may offer some interpretation of the first 4 bytes as date/time if the actual negotiated version was TLS 1.3 and not TLS 1.2 then the first 4 bytes don't necessarily mean anything. (This is complicated by the fact that TLS 1.3 hellos are sent as handshake version TLS 1.2, to look like TLS 1.2 in case TLS 1.3 isn't supported, therefore they may actually send the first 4 bytes as date/time regardless).

For example, Wireshark will show the ClientHello random first 4 bytes as the date/time even if the ClientHello supports TLS 1.3, but if TLS 1.3 is then actually negotiated from that hello then it will hide that parsing because it's no longer valid (according to the RFC).

/cc @Lekensteyn

[tgoeg]

Thanks for that last paragraph, that probably explains why I sometimes see parsed timestamps (when the session does not get successfully setup) and sometimes not. Sorry, no, the length is different in my case as well (28 vs 32 bytes), so my conclusion was wrong. Still, the referenced issue could be related:
There's an issue over at openssl/openssl#17140 that might be related, maybe this of interest here as well.

[tgoeg]

Might be a red herring as it seems, sorry for the noise.