kernel: introduce per-CPU panic(9) message buffers

Add a 512-byte buffer (ci_panicbuf) to each cpu_info struct on each
platform for use by panic(9).  The first panic on a given CPU writes
its message to this buffer.  Subsequent panics on a given CPU print
the panic message to the console but do not modify the buffer.  This
aids debugging in two cases:

- If 2+ CPUs panic simultaneously there is no risk of garbled messages
  in the panic buffer.

- If a CPU panics and then the operator causes a second panic while
  using ddb(4), the operator can still recall the first failure on
  a particular CPU.

Misc. changes to support this bigger change:

- Set panicstr atomically to identify the first CPU to reach panic().

- Tweak db_show_panic_cmd() to print all panic messages across all
  CPUs.  Prefix the first panic with an asterisk ('*').

- Prefer db_printf() to printf() during a panic if we have it.
  Apparently it disturbs less global state.

- On amd64, tweak fault() to write the local panic buffer.  This needs
  more work.

Prompted by bluhm@ and deraadt@.  Mostly written by deraadt@.
Discussed with bluhm@, deraadt@ and kettenis@.

Borne from a discussion on tech@ about making panic(9) more MP-safe:

https://marc.info/?l=openbsd-tech&m=162086462316143&w=2

ok kettenis@, visa@, bluhm@, deraadt@

1 files changed, 14 insertions, 8 deletions

diff --git a/sys/ddb/db_command.c b/sys/ddb/db_command.c
index 04356436b5c..57f0b7c1257 100644
--- a/sys/ddb/db_command.c
+++ b/sys/ddb/db_command.c
@@ -1,4 +1,4 @@
-/*	$OpenBSD: db_command.c,v 1.90 2020/10/26 18:53:20 deraadt Exp $	*/
+/*	$OpenBSD: db_command.c,v 1.91 2021/06/02 00:39:25 cheloha Exp $	*/
 /*	$NetBSD: db_command.c,v 1.20 1996/03/30 22:30:05 christos Exp $	*/
 
 /*
@@ -468,14 +468,20 @@ db_nfsnode_print_cmd(db_expr_t addr, int have_addr, db_expr_t count,
 void
 db_show_panic_cmd(db_expr_t addr, int have_addr, db_expr_t count, char *modif)
 {
-	if (panicstr)
-		db_printf("%s\n", panicstr);
-	else if (faultstr) {
-		db_printf("kernel page fault\n");
-		db_printf("%s\n", faultstr);
-		db_stack_trace_print(addr, have_addr, 1, modif, db_printf);
+	struct cpu_info *ci;
+	char *prefix;
+	CPU_INFO_ITERATOR cii;
+	int panicked = 0;
+
+	CPU_INFO_FOREACH(cii, ci) {
+		if (ci->ci_panicbuf[0] != '\0') {
+			prefix = (panicstr == ci->ci_panicbuf) ? "*" : " ";
+			db_printf("%scpu%d: %s\n",
+			    prefix, CPU_INFO_UNIT(ci), ci->ci_panicbuf);
+			panicked = 1;
+		}
 	}
-	else
+	if (!panicked)
 		db_printf("the kernel did not panic\n");	/* yet */
 }