Ignore warning alert returns from servername callback in TLSv1.3

If a servername callback returns SSL_TLSEXT_ERR_ALERT_WARNING, this
results in a fatal error in TLSv1.3 since alert levels are implicit
in the alert type and neither close_notify nor user_canceled make
sense in this context. OpenSSL chose to ignore this, so we need to
follow suit.

Found via a broken servername callback in p5-IO-Socket-SSL which
returns a Boolean instead of SSL_TLSEXT_ERR_*. This happened to
have worked before TLSv1.3 since warning alerts are often ignored.

This "fixes" sni.t and sni-verify.t in p5-IO-Socket-SSL.

ok beck jsing

1 files changed, 7 insertions, 3 deletions

diff --git a/lib/libssl/tls13_legacy.c b/lib/libssl/tls13_legacy.c
index beb89524027..0360f8159c8 100644
--- a/lib/libssl/tls13_legacy.c
+++ b/lib/libssl/tls13_legacy.c
@@ -1,4 +1,4 @@
-/*	$OpenBSD: tls13_legacy.c,v 1.26 2021/07/01 17:53:39 jsing Exp $ */
+/*	$OpenBSD: tls13_legacy.c,v 1.27 2021/08/30 16:50:23 tb Exp $ */
 /*
  * Copyright (c) 2018, 2019 Joel Sing <jsing@openbsd.org>
  *
@@ -515,8 +515,12 @@ tls13_legacy_servername_process(struct tls13_ctx *ctx, uint8_t *alert)
 	ret = ssl_ctx->internal->tlsext_servername_callback(s, &legacy_alert,
 	    ssl_ctx->internal->tlsext_servername_arg);
 
-	if (ret == SSL_TLSEXT_ERR_ALERT_FATAL ||
-	    ret == SSL_TLSEXT_ERR_ALERT_WARNING) {
+	/*
+	 * Ignore SSL_TLSEXT_ERR_ALERT_WARNING returns to match OpenSSL's
+	 * behavior: the only warning alerts in TLSv1.3 are close_notify and
+	 * user_canceled, neither of which should be returned by the callback.
+	 */
+	if (ret == SSL_TLSEXT_ERR_ALERT_FATAL) {
 		if (legacy_alert >= 0 && legacy_alert <= 255)
 			*alert = legacy_alert;
 		return 0;