fix deadlock with xcb_take_socket/return_socket v3

To prevent different threads from stealing the socket from each other the
caller of "xcb_take_socket" must hold a lock that is also acquired in
"return_socket". Unfortunately xcb tries to prevent calling return_socket
from multiple threads and this can lead to a deadlock situation.

A simple example:
- X11 has taken the socket
- Thread A has locked the display.
- Thread B does xcb_no_operation() and thus ends up in libX11's return_socket(),
  waiting for the display lock.
- Thread A calls e.g. xcb_no_operation(), too, ends up in return_socket() and
  because socket_moving == 1, ends up waiting for thread B
=> Deadlock

This patch allows calling return_socket from different threads at the same time
an so resolves the deadlock situation.

Partially fixes: https://bugs.freedesktop.org/show_bug.cgi?id=20708

v2: fixes additional pthread_cond_wait dependencies,
    rework comments and patch description

v3: separate pthread_cond_wait dependencies and unrelated whitespace
    change into their own patch, use unsigned for socket_seq

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Uli Schlachter <psychon@znc.in>

1 files changed, 4 insertions, 3 deletions

diff --git a/src/xcbint.h b/src/xcbint.h
index f9e5a52..7f9ab28 100644
--- a/src/xcbint.h
+++ b/src/xcbint.h
@@ -79,14 +79,15 @@ void *_xcb_map_remove(_xcb_map *q, unsigned int key);
 
 /* xcb_out.c */
 
+typedef void (*xcb_return_socket_func_t)(void *closure);
+
 typedef struct _xcb_out {
     pthread_cond_t cond;
     int writing;
 
-    pthread_cond_t socket_cond;
-    void (*return_socket)(void *closure);
+    xcb_return_socket_func_t return_socket;
     void *socket_closure;
-    int socket_moving;
+    unsigned int socket_seq;
 
     char queue[XCB_QUEUE_BUFFER_SIZE];
     int queue_len;