integer overflow in XvQueryPortAttributes() [CVE-2013-1989 1/3]

The num_attributes & text_size members of the reply are both CARD32s
and need to be bounds checked before multiplying & adding them together
to come up with the total size to allocate, to avoid integer overflow
leading to underallocation and writing data from the network past the
end of the allocated buffer.

Reported-by: Ilja Van Sprundel <ivansprundel@ioactive.com>
Signed-off-by: Alan Coopersmith <alan.coopersmith@oracle.com>

1 files changed, 8 insertions, 2 deletions

diff --git a/src/Xv.c b/src/Xv.c
index 5be1d95..3cbad35 100644
--- a/src/Xv.c
+++ b/src/Xv.c
@@ -851,9 +851,15 @@ XvQueryPortAttributes(Display *dpy, XvPortID port, int *num)
   }
 
   if(rep.num_attributes) {
-      int size = (rep.num_attributes * sizeof(XvAttribute)) + rep.text_size;
+      unsigned long size;
+      /* limit each part to no more than one half the max size */
+      if ((rep.num_attributes < ((INT_MAX / 2) / sizeof(XvAttribute))) &&
+	  (rep.text_size < (INT_MAX / 2))) {
+	  size = (rep.num_attributes * sizeof(XvAttribute)) + rep.text_size;
+	  ret = Xmalloc(size);
+      }
 
-      if((ret = Xmalloc(size))) {
+      if (ret != NULL) {
 	  char* marker = (char*)(&ret[rep.num_attributes]);
 	  xvAttributeInfo Info;
 	  int i;