Part 2 of file descriptor race and deadlock corrections.

Adjust design of file descriptor table to eliminate races
with both opening and closing of file descriptor entries
and eliminates one class of deadlocks. One nice side effect
of this change in design should be better performance for
applications that open and close many file descriptors due
to reduced fd_table_lock contention and fd entry reuse.

- Add entry states to manage use of entry and eliminate
some closing races. fd entries are not deallocated upon
close() now.
- Call _thread_fd_table_init with one of five discreet
modes to properly initialize an entry and manage the
state transition to open.
- When closing an entry hold the entry spinlock locked
across the state transition and the _thread_sys_close
call to close another race.
- Introduce a new lock type FD_RDWR_CLOSE that transitions
either a closed entry or an open entry into closing state
and then waits for a RDWR lock so that the lock queue can
unwind normally. All subsequent fd lock attempts for that
entry are rejected with EBADF until the fd is fully closed,
or reopened by dup2(). Once a thread holds the FD_RDWR_LOCK
it is safe to close() it or dup2() on it.
- When a thread creates a new fd there is a window of time
when another thread could attempt to use the fd before the
creating thread has initialized the entry for it. This can
result in improper status_flags for the entry, so record
the entries init mode, detect when this has happened and
correct the status_flags when needed.

reviewed by marc@ & brad@, tested by several, okay brad@

1 files changed, 2 insertions, 1 deletions

diff --git a/lib/libc/include/thread_private.h b/lib/libc/include/thread_private.h
index 5fbbf592a41..c5acc4b5e43 100644
--- a/lib/libc/include/thread_private.h
+++ b/lib/libc/include/thread_private.h
@@ -1,4 +1,4 @@
-/* $OpenBSD: thread_private.h,v 1.18 2006/02/22 07:16:31 otto Exp $ */
+/* $OpenBSD: thread_private.h,v 1.19 2006/09/26 14:18:28 kurt Exp $ */
 
 /* PUBLIC DOMAIN: No Rights Reserved. Marco S Hyman <marc@snafu.org> */
 
@@ -90,6 +90,7 @@ extern void *__THREAD_NAME(serv_mutex);
 #define FD_READ		0x1
 #define FD_WRITE	0x2
 #define FD_RDWR		(FD_READ | FD_WRITE)
+#define FD_RDWR_CLOSE	(FD_RDWR | 0x4)
 
 struct timespec;
 int	_thread_fd_lock(int, int, struct timespec *);