add support for bypassing iommu translation

managing the translation table entries (TTEs) on an iommu is not
free, and is in fact extremely expensive on some platforms. the
flip side of this is that forcing dma through TTEs does provide
some safety and can help during the development of drivers. however,
this has been less true in recent years than it used to be and is
less of a concern now, especially considering the performance
differences on some platforms.

devices have to create dmamaps with BUS_DMA_64BIT to bypass the
iommu because the memory window presented to hardware with direct
access to memory is at an extremely high address. there's no 32bit
bypass access to memory, it has to go through TTEs otherwise.

on an m4000 there are several orders of magnitude performance
difference between a driver with BUS_DMA_64BIT set and one without
it.

hilariously, sun used a pci bridge on a whole generation of machines
that had broken support for dma addresses over 40 bits (or around
there), so devices behind those pci bridges need to have their
dmamap_creates intercepted and any potential BUS_DMA_64BIT flags
cleared on the way to the iommu drivers. this affects at least v215,
v245, and v445, and probably u25s and u45s. it probably explains
why all their onboard nics and disk controllers feel super slow,
and why there was a meme at sun that bcopy was cheaper than dma
when moving packets on and off a nic.

ok kettenis@ deraadt@

1 files changed, 6 insertions, 5 deletions

diff --git a/sys/arch/sparc64/dev/iommureg.h b/sys/arch/sparc64/dev/iommureg.h
index db1b0671dbe..b70b6fc4cc4 100644
--- a/sys/arch/sparc64/dev/iommureg.h
+++ b/sys/arch/sparc64/dev/iommureg.h
@@ -1,4 +1,4 @@
-/*	$OpenBSD: iommureg.h,v 1.17 2012/08/17 20:46:50 kettenis Exp $	*/
+/*	$OpenBSD: iommureg.h,v 1.18 2019/06/25 22:30:56 dlg Exp $	*/
 /*	$NetBSD: iommureg.h,v 1.6 2001/07/20 00:07:13 eeh Exp $	*/
 
 /*
@@ -90,10 +90,11 @@ struct iommu_strbuf {
 #define IOMMUCR_DE		0x000000000000000002LL	/* Diag enable */
 #define IOMMUCR_EN		0x000000000000000001LL	/* Enable IOMMU */
 
-#define IOMMUCR_FIRE_SE		0x000000000000000400LL	/* Snoop enable */
-#define IOMMUCR_FIRE_CM_EN	0x000000000000000300LL  /* Cache mode enable */
-#define IOMMUCR_FIRE_BE		0x000000000000000002LL	/* Bypass enable */
-#define IOMMUCR_FIRE_TE		0x000000000000000001LL	/* Translation enabled */
+#define IOMMUCR_FIRE_PD		0x000000000000001000UL	/* Process disable */
+#define IOMMUCR_FIRE_SE		0x000000000000000400UL	/* Snoop enable */
+#define IOMMUCR_FIRE_CM_EN	0x000000000000000300UL  /* Cache mode enable */
+#define IOMMUCR_FIRE_BE		0x000000000000000002UL	/* Bypass enable */
+#define IOMMUCR_FIRE_TE		0x000000000000000001UL	/* Translation enabled */
 
 /*
  * IOMMU stuff