Ryfujk

Struggling with a PKI implementation in my lab (ADCS 2012 R2) and cannot for the life of me figure out where I have gone wrong.

Got all the way to the point of being able to generate SSL/TLS certs for an IIS server and make the binding. Also used the NARTAC tool to shut down SSLv2/3 + enable TLS 1.0-1.2. Using testssl.sh I can verify that SSL is disabled and can get a handshake for TLS 1.0 and 1.1, but every time the client hello for TLS 1.2 is received the server sends a TCP Reset. The system logs show the pair of SCHANNEL 36874 and 36888 error IDs that correspond with the resets. Error 36874 suggests that the client hello is presenting cipher suites that aren't supported by the server.

This is where I hit the brick wall, and my searching the interwebs is not yielding a winner. I can all but guarantee I've overlooked something glaringly obvious. However, the blinders of frustration are preventing me from seeing it.

Thoughts?

Client hello details:

Frame 136: 377 bytes on wire (3016 bits), 377 bytes captured (3016 bits) on interface 0 Ethernet II, Src: CiscoInc_07:be:7f
(fc:5b:39:07:be:7f), Dst: Vmware_01:02:14 (00:50:56:01:02:14) Internet
Protocol Version 4, Src: 192.168.6.75, Dst: 10.22.163.219 Transmission
Control Protocol, Src Port: 35836 (35836), Dst Port: 443 (443), Seq:
1, Ack: 1, Len: 311 Secure Sockets Layer
SSL Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 306
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 302
Version: TLS 1.2 (0x0303)
Random
Session ID Length: 0
Cipher Suites Length: 176
Cipher Suites (88 suites)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
Cipher Suite: TLS_DH_DSS_WITH_AES_256_GCM_SHA384 (0x00a5)
Cipher Suite: TLS_DHE_DSS_WITH_AES_256_GCM_SHA384 (0x00a3)
Cipher Suite: TLS_DH_RSA_WITH_AES_256_GCM_SHA384 (0x00a1)
Cipher Suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f)
Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x006b)
Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 (0x006a)
Cipher Suite: TLS_DH_RSA_WITH_AES_256_CBC_SHA256 (0x0069)
Cipher Suite: TLS_DH_DSS_WITH_AES_256_CBC_SHA256 (0x0068)
Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA (0x0038)
Cipher Suite: TLS_DH_RSA_WITH_AES_256_CBC_SHA (0x0037)
Cipher Suite: TLS_DH_DSS_WITH_AES_256_CBC_SHA (0x0036)
Cipher Suite: TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA (0x0088)
Cipher Suite: TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA (0x0087)
Cipher Suite: TLS_DH_RSA_WITH_CAMELLIA_256_CBC_SHA (0x0086)
Cipher Suite: TLS_DH_DSS_WITH_CAMELLIA_256_CBC_SHA (0x0085)
Cipher Suite: TLS_GOSTR341001_WITH_28147_CNT_IMIT (0x0081)
Cipher Suite: TLS_GOSTR341094_WITH_28147_CNT_IMIT (0x0080)
Cipher Suite: TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 (0xc032)
Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02e)
Cipher Suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 (0xc02a)
Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 (0xc026)
Cipher Suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA (0xc00f)
Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA (0xc005)
Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
Cipher Suite: TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (0x0084)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
Cipher Suite: TLS_DH_DSS_WITH_AES_128_GCM_SHA256 (0x00a4)
Cipher Suite: TLS_DHE_DSS_WITH_AES_128_GCM_SHA256 (0x00a2)
Cipher Suite: TLS_DH_RSA_WITH_AES_128_GCM_SHA256 (0x00a0)
Cipher Suite: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (0x0067)
Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 (0x0040)
Cipher Suite: TLS_DH_RSA_WITH_AES_128_CBC_SHA256 (0x003f)
Cipher Suite: TLS_DH_DSS_WITH_AES_128_CBC_SHA256 (0x003e)
Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA (0x0032)
Cipher Suite: TLS_DH_RSA_WITH_AES_128_CBC_SHA (0x0031)
Cipher Suite: TLS_DH_DSS_WITH_AES_128_CBC_SHA (0x0030)
Cipher Suite: TLS_DHE_RSA_WITH_SEED_CBC_SHA (0x009a)
Cipher Suite: TLS_DHE_DSS_WITH_SEED_CBC_SHA (0x0099)
Cipher Suite: TLS_DH_RSA_WITH_SEED_CBC_SHA (0x0098)
Cipher Suite: TLS_DH_DSS_WITH_SEED_CBC_SHA (0x0097)
Cipher Suite: TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA (0x0045)
Cipher Suite: TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA (0x0044)
Cipher Suite: TLS_DH_RSA_WITH_CAMELLIA_128_CBC_SHA (0x0043)
Cipher Suite: TLS_DH_DSS_WITH_CAMELLIA_128_CBC_SHA (0x0042)
Cipher Suite: TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 (0xc031)
Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02d)
Cipher Suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 (0xc029)
Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 (0xc025)
Cipher Suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA (0xc00e)
Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA (0xc004)
Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
Cipher Suite: TLS_RSA_WITH_SEED_CBC_SHA (0x0096)
Cipher Suite: TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (0x0041)
Cipher Suite: TLS_RSA_WITH_IDEA_CBC_SHA (0x0007)
Cipher Suite: TLS_ECDHE_RSA_WITH_RC4_128_SHA (0xc011)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_RC4_128_SHA (0xc007)
Cipher Suite: TLS_ECDH_RSA_WITH_RC4_128_SHA (0xc00c)
Cipher Suite: TLS_ECDH_ECDSA_WITH_RC4_128_SHA (0xc002)
Cipher Suite: TLS_RSA_WITH_RC4_128_SHA (0x0005)
Cipher Suite: TLS_RSA_WITH_RC4_128_MD5 (0x0004)
Cipher Suite: TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (0xc012)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA (0xc008)
Cipher Suite: TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (0x0016)
Cipher Suite: TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA (0x0013)
Cipher Suite: TLS_DH_RSA_WITH_3DES_EDE_CBC_SHA (0x0010)
Cipher Suite: TLS_DH_DSS_WITH_3DES_EDE_CBC_SHA (0x000d)
Cipher Suite: TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA (0xc00d)
Cipher Suite: TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA (0xc003)
Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)
Cipher Suite: TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff)
Compression Methods Length: 1
Compression Methods (1 method)
Extensions Length: 85
Extension: ec_point_formats
Extension: elliptic_curves
Extension: SessionTicket TLS
Extension: signature_algorithms
Type: signature_algorithms (0x000d)
Length: 32
Signature Hash Algorithms Length: 30
Signature Hash Algorithms (15 algorithms)
Extension: Heartbeat

And the TCP reset:

137 2.086290 10.22.163.219 192.168.6.75 TCP 54 443 → 35836 [RST, ACK]
Seq=1 Ack=312 Win=0 Len=0 Frame 137: 54 bytes on wire (432 bits), 54
bytes captured (432 bits) on interface 0 Ethernet II, Src:
Vmware_01:02:14 (00:50:56:01:02:14), Dst: IETF-VRRP-VRID_19
(00:00:5e:00:01:19) Internet Protocol Version 4, Src: 10.22.163.219,
Dst: 192.168.6.75 Transmission Control Protocol, Src Port: 443 (443),
Dst Port: 35836 (35836), Seq: 1, Ack: 312, Len: 0
Source Port: 443
Destination Port: 35836
[Stream index: 3]
[TCP Segment Len: 0]
Sequence number: 1 (relative sequence number)
Acknowledgment number: 312 (relative ack number)
Header Length: 20 bytes
Flags: 0x014 (RST, ACK)
Window size value: 0
[Calculated window size: 0]
[Window size scaling factor: 256]
Checksum: 0x74ff [validation disabled]
Urgent pointer: 0
[SEQ/ACK analysis]

SCHANNEL 36874

Log Name: System Source: Schannel Date: 5/4/2016
9:48:36 AM Event ID: 36874 Task Category: None Level:

Error Keywords: User: SYSTEM Computer:

OCSP.corp.contoso.com Description: An TLS 1.2 connection request was
received from a remote client application, but none of the cipher
suites supported by the client application are supported by the
server. The SSL connection request has failed.

SCHANNEL 36888:

Log Name: System Source: Schannel Date: 5/4/2016
9:48:36 AM Event ID: 36888 Task Category: None Level:

Error Keywords: User: SYSTEM Computer:

OCSP.corp.contoso.com Description: A fatal alert was generated and
sent to the remote endpoint. This may result in termination of the
connection. The TLS protocol defined fatal error code is 40. The
Windows SChannel error state is 1205.

EDIT 1

Found that I can successfully get a non-domain joined 2012 R2 IIS server to respond to a TLS 1.2 hello packet. So my issue appears to be related to being a domain joined system. Still haven't solved that part of the equation.

If the issue is a domain joined computer not being able to connect (and I assume the connection client is Windows itself, IE, or another component that makes use of the native SChannel in Windows), then I would suspect that you don't have TLS 1.2 enabled on the client. And being a domain joined computer, this would mean you have a GPO Administrative Templates that's configuring the protocol and cipher suites and pushing them to the client.

Couple things I'd recommend for tracing.

I assume you're mention of the "Nartac tool' means IISCrypto. Run this up on both the client and server to see what protocols and ciphers are enable - ensure there's overlap.

Also, any changes to these items requires a reboot. Yes, this may be stating the obvious, but it needed to be said.

Next, get Fiddler. This tool is my best friend. You can use it to trace http and https traffic from a client (it inserts itself as a middle man between client and server, acting as a proxy for the client, decrypts https traffic, etc). The important part is that it also traces the https connect (and reveals cipher suits, etc). Run it on your client and see what it says.

My only other thought is that the AD Cert Authority you've setup, the root cert certificate hasn't made it into the Trusted Root Certification Authorities certificate store on the client computers. Check this too.

Another possible cause of RST could be signature_algorithm extension being missing (in which case defaults to SHA1) or not having a pair that matches the server cert signature. So if you upgrade to a sha512 RSA signed cert and the client only offers SHA1 and SHA256 the server will RST your client hello.

Not the case here but if anyone find this question, this may help.