Notes on the OpenBSD threaded C library (-lc_r) ================================================ Sources The main bulk of this library came from: . FreeBSD's libc_r (John Birrell) - The scheduler and locking code in the uthread directory. - Some of the manual pages in the man directory. - . MIT pthreads (Chris Provenzano) - The test code in the TEST directory I'm mainly tracking changes in FreeBSD's libc_r and integrating them as I can. The major changes are outlined at the end of this file. Standards This implementation has also been mindful of: . Posix Threads[1] . Single Unix Specification[2] Conformance Only the absolutely required re-entrant functions have been added to the C library interface headers (in /usr/src/include). The conformance goals used were: minimal implementation; strict conformance with standard; provide reasonable utility when standards are spineless (e.g., our asynchronous cancel is eager even though POSIX says async cancels may be acted upon at any time -- ie never.) The widely used, but non-standard, gethostbyname_r() has been half-heartily added (but not prototyped in a header - see source for details). Please contact me if you have pointers to standards/comments about this function. Re-entrant functions added for the standards have also been made available to libc (i.e. without the need to define _POSIX_THREADS). Change strategy in libc The approach taken in making the libc functions re-entrant was to develop some macros that handle: . file locking . monitors (thread-shared data structure locking) . per-thread private data structure allocation These macros were used to avoid copious amounts of #ifdef statements[3]. In the non-threaded libc, file locking and the monitors are no-ops; the per-thread private data structures were previously declared static and the macros maintain this. In the threaded libc_r, file locking is as per the FreeBSD file descriptor locking, and the monitors are pthread_mutex operations. Per-thread private data structures use the pthread_specific functions to dynamically allocate memory on their first use, initialising them from the static (and hidden) per-compilation-unit data structures. In this way, each thread appears to have its very own private libc state. Errno Unlike the FreeBSD and MIT pthreads package, errno is not a macro like __error(); instead it is part of the per-thread context: saved and restored like a register. This has several advantages over the errno-as-a-macro method: - The syscall/cerror code does not have to be re-written - Libraries compiled without thread support will still work - Easier to port old packages that use errno, but don't include to get the macro - No need to go through all sources and find where errno is used as a field name or formal parameter name. The overhead of saving and restoring an integer was considered too tiny to worry about in comparison to the huge penalty hit of handling a signal and restoring the rest of a thread's context. It has been pointed out that this technique will not work in a mutiprocessor environment, and this is quite true. However the following reasons are quite persuasive: - OpenBSD does not do MP (yet) - This (FreeBSD uthread) implementation will not work in an MP environment anyway because _thread_run is not a cpu-local variable. - There is a lot of coupling between binary ports, the 'old' libc and other standard libraries. It is predicted that it will be a huge headache to come up with a scheme that when someone tries to upgrade, everything won't break in a spectacular fashion, leaving bits of crippled ports, executables and shattered libraries everywhere. Compiler support The in-tree gcc had its config/openbsd.h modified to support a `-pthread' switch. Using this switch defines _POSIX_THREADS for cpp, and replaces the normal -lc linker option with -lc_r. The objective-C component was also made aware of posix threads, via the configure switch --enable-threads=posix. {This has not been well tested though.} Debugger support The in-tree gdb has been augmented to recognise when a executable linked with -lc_r is being debugged. All the documented gdb threads commands will then work. (Except you can't resume/step execution in a thread other than the 'current' one.) To get detailed state information, issue: (gdb) call _thread_dump_info() This will show what mutexes, condvars or file locks each process is waiting on. If you use _flockfile_debug() instead of flockfile() in your programs, source code references will appear in here too. (see ) Changes to FreeBSD uthreads Although this implementation tracks the FreeBSD libc_r tree to a large extent, the following significant changes have been made to John Birrell's uthreads implementation that have yet to appear (or may never appear) in FreeBSD's implementation: - architecture independent code generalised and moved out of #ifdef's[3] (now support for sparc, mips, m68k, powerpc added to i386, alpha) - the SIGINFO handler generates much more friendly/useful output - an implementation of poll() {probably bogus, but X11 seems to work} - an implementation of pthread_cancel() - an implementation of pthread_[gs]etscope() {bogus} - removal of the freebsd's support for init(8) {i.e., pid == 1} Caveats This library is not 100% standards compliant - yet. It is certainly moving in that direction though. There are still some unresolved isseues, and the interested reader is directed towards the TODO file. Standard disclaimer This software is made available by the author to the public for free and "as is". All users of this free software are solely and entirely responsible for their own choice and use of this software for their own purposes. By using this software, each user agrees that the author shall not be liable for damages of any kind in relation to its use or performance. Some parts of this software bear their own copyright which is different to the above disclaimer. References [1] P1003.1c/D10 IEEE Draft Standard for Information Technology-- Portable Operating System Interface (POSIX) -- Part 1: System Application Program Interface (API) -- Ammendment 2: Threads Extension [C Language]. IEEE Standards, September 1994. [2] T912, The Single UNIX(R) Specification, Version 2. The Open Group, February 1997. http://www.opengroup.org/pubs/catalog/t912.htm [3] #ifdef Considered Harmful, or Portability Experience with C News. H. Spencer and G. Collyer, Proc. of the Summer 1992 USENIX Conference, San Antionio, Texas, 1992. pp. 185-198 Acknowledgements Lots of kudos to Chris Provenzano et al. for the original MIT-pthreads implementation that I still read to get ideas; and also to John Birrell and the other FreeBSD developers who consistently write high quality code. The help rendered from the OpenBSD developers was greatly appreciated - especially from Todd Fries. David Leonard $OpenBSD: NOTES,v 1.3 1999/01/17 23:43:47 d Exp $