Schannel: fix endless loop in TlsCryptographSchannel::transmit()

We've encountered that under some circumstances DecryptMessage may
return undocumented SEC_E_DECRYPT_FAILURE status. It's probably a bug
in Schannel on Win 11, as there is no such problem on Win 10.

In such cases, since we didn't handle this specifically, we can end
up in an infinite loop in TlsCryptographSchannel::transmit() when
`intermediateBuffer` is not empty and `bytesRead` is 0.

To fix this, we just add SEC_E_DECRYPT_FAILURE status handling and
will disconnect from the host and emit the error respectively.

Pick-to: 6.8
Change-Id: I340669a967be420c74cb01296629f94c4720c958
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This commit is contained in:
Vladimir Belyavsky 2024-11-21 15:11:08 +03:00
parent 001a988120
commit ed0eb386da

View File

@ -670,6 +670,8 @@ QString schannelErrorToString(qint32 status)
return QSslSocket::tr("Unexpected or badly-formatted message received");
case SEC_E_ENCRYPT_FAILURE:
return QSslSocket::tr("The data could not be encrypted");
case SEC_E_DECRYPT_FAILURE:
return QSslSocket::tr("The data could not be decrypted");
case SEC_E_ALGORITHM_MISMATCH:
return QSslSocket::tr("No cipher suites in common");
case SEC_E_UNKNOWN_CREDENTIALS:
@ -1985,6 +1987,13 @@ void TlsCryptographSchannel::transmit()
setErrorAndEmit(d, QAbstractSocket::SslInternalError,
schannelErrorToString(status));
break;
} else if (status == SEC_E_DECRYPT_FAILURE) {
// It's not documented as a possible return value for DecryptMessage,
// but we see that this may happen - supposed to be a bug in Schannel (with TLS 1.3?)
shutdown = true; // skips sending the shutdown alert
disconnectFromHost();
setErrorAndEmit(d, QAbstractSocket::SslInternalError, schannelErrorToString(status));
break;
} else if (status == SEC_I_CONTEXT_EXPIRED) {
// 'remote' has initiated a shutdown
disconnectFromHost();