Start the SMR thread when all CPUs are ready for scheduling. This

prevents the appearance of a "smr: dispatch took N seconds" message
during boot when there is an early smr_call(). Such a call can happen
with mfii(4). The initial dispatch cannot make progress until
smr_grace_wait() can visit all CPUs.

This fix is essentially a hack. It makes use of the fact that there
is no hard guarantee on how quickly the callback of smr_call() gets
invoked. It is assumed that the SMR call backlog does not grow large
during boot.

An alternative fix is to make smr_grace_wait() skip secondary CPUs
until they have been started. However, this could break if the spinup
logic of secondary CPUs was changed.

Delayed SMR dispatch reported and fix tested by Hrvoje Popovski
Discussed with and OK kettenis@, claudio@

1 files changed, 3 insertions, 5 deletions

diff --git a/sys/kern/kern_smr.c b/sys/kern/kern_smr.c
index 51165b0c3c8..a5605ed0f64 100644
--- a/sys/kern/kern_smr.c
+++ b/sys/kern/kern_smr.c
@@ -1,4 +1,4 @@
-/*	$OpenBSD: kern_smr.c,v 1.6 2019/12/30 23:56:26 jsg Exp $	*/
+/*	$OpenBSD: kern_smr.c,v 1.7 2020/02/25 16:55:33 visa Exp $	*/
 
 /*
  * Copyright (c) 2019 Visa Hankala
@@ -31,7 +31,6 @@
 
 #define SMR_PAUSE	100		/* pause between rounds in msec */
 
-void	smr_create_thread(void *);
 void	smr_dispatch(struct schedstate_percpu *);
 void	smr_grace_wait(void);
 void	smr_thread(void *);
@@ -66,13 +65,12 @@ smr_startup(void)
 {
 	SIMPLEQ_INIT(&smr_deferred);
 	WITNESS_INIT(&smr_lock_obj, &smr_lock_type);
-	kthread_create_deferred(smr_create_thread, NULL);
+	timeout_set(&smr_wakeup_tmo, smr_wakeup, NULL);
 }
 
 void
-smr_create_thread(void *arg)
+smr_startup_thread(void)
 {
-	timeout_set(&smr_wakeup_tmo, smr_wakeup, NULL);
 	if (kthread_create(smr_thread, NULL, NULL, "smr") != 0)
 		panic("could not create smr thread");
 }