f{read,write} take a number of items and an item size as arguments, and

return the number of items read of written.

When you intend to return the number of bytes actually processed, it is
wise to pass 1 as the item size and the size as the number of items.

But in *some* places, the OpenSSL does the opposite, and has extra logic
to change a successful return of 1 (item processed) into the real size.

And, guess why it does that? Because of old VMS, for they (used to) have a
substandard stdio implementation.

Note that this change causes the return values of BIO_dump_fp() and
BIO_dump_indent_fp() to no longer be useless (actual number of callback calls),
but actual bytes output. Given the irrelevance of the return value before,
it is unlikely that anything depends upon it (and if something does, it
probably has other problems in need for a fix...)

ok tedu@ beck@ jsing@

2 files changed, 7 insertions, 14 deletions

diff --git a/lib/libcrypto/bio/b_dump.c b/lib/libcrypto/bio/b_dump.c
index 91979bd7555..0943e9006df 100644
--- a/lib/libcrypto/bio/b_dump.c
+++ b/lib/libcrypto/bio/b_dump.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: b_dump.c,v 1.19 2014/07/11 08:44:47 jsing Exp $ */
+/* $OpenBSD: b_dump.c,v 1.20 2014/11/11 19:26:12 miod Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -149,7 +149,7 @@ BIO_dump_indent_cb(int (*cb)(const void *data, size_t len, void *u),
 static int
 write_fp(const void *data, size_t len, void *fp)
 {
-	return fwrite(data, len, 1, fp);
+	return fwrite(data, 1, len, fp);
 }
 
 int
diff --git a/lib/libcrypto/bio/bss_file.c b/lib/libcrypto/bio/bss_file.c
index 4fa3fb60627..c710076fea2 100644
--- a/lib/libcrypto/bio/bss_file.c
+++ b/lib/libcrypto/bio/bss_file.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: bss_file.c,v 1.30 2014/07/11 08:44:47 jsing Exp $ */
+/* $OpenBSD: bss_file.c,v 1.31 2014/11/11 19:26:12 miod Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -185,8 +185,8 @@ file_read(BIO *b, char *out, int outl)
 {
 	int ret = 0;
 
-	if (b->init && (out != NULL)) {
-		ret = fread(out, 1,(int)outl,(FILE *)b->ptr);
+	if (b->init && out != NULL) {
+		ret = fread(out, 1, outl, (FILE *)b->ptr);
 		if (ret == 0 && ferror((FILE *)b->ptr)) {
 			SYSerr(SYS_F_FREAD, errno);
 			BIOerr(BIO_F_FILE_READ, ERR_R_SYS_LIB);
@@ -201,15 +201,8 @@ file_write(BIO *b, const char *in, int inl)
 {
 	int ret = 0;
 
-	if (b->init && (in != NULL)) {
-		ret = fwrite(in,(int)inl, 1,(FILE *)b->ptr);
-		if (ret)
-			ret = inl;
-		/* ret=fwrite(in,1,(int)inl,(FILE *)b->ptr); */
-		/* according to Tim Hudson <tjh@cryptsoft.com>, the commented
-		 * out version above can cause 'inl' write calls under
-		 * some stupid stdio implementations (VMS) */
-	}
+	if (b->init && in != NULL)
+		ret = fwrite(in, 1, inl, (FILE *)b->ptr);
 	return (ret);
 }