fix Tx watchdog trigger and freeze in dwqe(4)

dwqe(4) interfaces may trigger the "dwqe_watchdog" function and stop
transmitting packets under heavy Tx load. In this situation, ifconfig(8)
will permanently show OACTIVE until the interface is restarted (ifconfig
down/up).

The problem seems to be triggered by writing to the Tx ring tail pointer
register multiple times during dwqe_start(). Updating the Tx ring tail
pointer only after all Tx descriptors have been updated seems to fix it.

The fix is based on the eqos(4) driver in NetBSD. Verified on an Intel
Elkhart Lake machine but might affect other platforms with DesignWare
Ethernet Quality-of-Service Controller version 4.

Testing by bluhm@, ok kettenis@

1 files changed, 9 insertions, 5 deletions

diff --git a/sys/dev/ic/dwqe.c b/sys/dev/ic/dwqe.c
index f853fcc109f..e515fd03374 100644
--- a/sys/dev/ic/dwqe.c
+++ b/sys/dev/ic/dwqe.c
@@ -1,4 +1,4 @@
-/*	$OpenBSD: dwqe.c,v 1.15 2023/11/10 15:51:20 bluhm Exp $	*/
+/*	$OpenBSD: dwqe.c,v 1.16 2023/12/28 14:30:28 uwe Exp $	*/
 /*
  * Copyright (c) 2008, 2019 Mark Kettenis <kettenis@openbsd.org>
  * Copyright (c) 2017, 2022 Patrick Wildt <patrick@blueri.se>
@@ -338,11 +338,18 @@ dwqe_start(struct ifqueue *ifq)
 #endif
 	}
 
-	if (sc->sc_tx_prod != idx) {
+	if (used > 0) {
 		sc->sc_tx_prod = idx;
 
 		/* Set a timeout in case the chip goes out to lunch. */
 		ifp->if_timer = 5;
+
+		/*
+		 * Start the transmit process after the last in-use Tx
+		 * descriptor's OWN bit has been updated.
+		 */
+		dwqe_write(sc, GMAC_CHAN_TX_END_ADDR(0), DWQE_DMA_DVA(sc->sc_txring) +
+		    idx * sizeof(struct dwqe_desc));
 	}
 }
 
@@ -1055,9 +1062,6 @@ dwqe_encap(struct dwqe_softc *sc, struct mbuf *m, int *idx, int *used)
 	bus_dmamap_sync(sc->sc_dmat, DWQE_DMA_MAP(sc->sc_txring),
 	    *idx * sizeof(*txd), sizeof(*txd), BUS_DMASYNC_PREWRITE);
 
-	dwqe_write(sc, GMAC_CHAN_TX_END_ADDR(0), DWQE_DMA_DVA(sc->sc_txring) +
-	    frag * sizeof(*txd));
-
 	KASSERT(sc->sc_txbuf[cur].tb_m == NULL);
 	sc->sc_txbuf[*idx].tb_map = sc->sc_txbuf[cur].tb_map;
 	sc->sc_txbuf[cur].tb_map = map;