libio: Multiple fixes for open_{w}memstram (BZ#18241 and BZ#20181)

This patches fixes multiples issues on open_{w}memstream reported on both
BZ#18241 and BZ#20181:

  - failed fseek does not set errno.
  - negative offset in fseek fails even when resulting position is
    a valid one.
  - a flush after write if the current write position is not at the
    end of the stream currupt data.

The main fix is on seek operation for memstream (_IO_{w}str_seekoff), where
both _IO_read_ptr and _IO_read_end pointer are updated if a write operation
has occured (similar to default file operations).  Also, to calculate the
offset on both read and write pointers, a temporary value is instead of
updating the argument supplied value.  Negative offset are valid if resulting
internal pointer is within the range of _IO_{read,write}_base and
_IO_{read,write}_end.

Also POSIX states that a null or wide null shall be appended to the current
buffer iff a write moves the position to a value larger than the current
lenght.  Current implementation appends a null or wide null regardless
of this condition.  This patch fixes it by removing the 'else' condition
on _IO_{w}mem_sync.

Checked on x86_64.

	[BZ #18241]
	[BZ #20181]
	* libio/Makefile (test): Add tst-memstream3 and tst-wmemstream3.
	* libio/memstream.c (_IO_mem_sync): Only append a null byte if
	write position is at the end the buffer.
	* libio/wmemstream.c (_IO_wmem_sync): Likewise.
	* libio/strops.c (_IO_str_switch_to_get_mode): New function.
	(_IO_str_seekoff): Set correct offset from negative displacement and
	set EINVAL for invalid ones.
	* libio/wstrops.c (enlarge_userbuf): Use correct function to calculate
	buffer length.
	(_IO_wstr_switch_to_get_mode): New function.
	(_IO_wstr_seekoff): Set correct offset from negative displacement and
	set EINVAL for invalid ones.
	* libio/tst-memstream3.c: New file.
	* libio/tst-wmemstream3.c: Likewise.
	* manual/examples/memstrm.c: Remove warning when priting size_t.

1 files changed, 20 insertions, 0 deletions

diff --git a/ChangeLog b/ChangeLog
index ec667708be..4b6a1740cc 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,23 @@
+2016-09-30  Adhemerval Zanella  <adhemerval.zanella@linaro.org>
+
+	[BZ #18241]
+	[BZ #20181]
+	* libio/Makefile (test): Add tst-memstream3 and tst-wmemstream3.
+	* libio/memstream.c (_IO_mem_sync): Only append a null byte if
+	write position is at the end the buffer.
+	* libio/wmemstream.c (_IO_wmem_sync): Likewise.
+	* libio/strops.c (_IO_str_switch_to_get_mode): New function.
+	(_IO_str_seekoff): Set correct offset from negative displacement and
+	set EINVAL for invalid ones.
+	* libio/wstrops.c (enlarge_userbuf): Use correct function to calculate
+	buffer length.
+	(_IO_wstr_switch_to_get_mode): New function.
+	(_IO_wstr_seekoff): Set correct offset from negative displacement and
+	set EINVAL for invalid ones.
+	* libio/tst-memstream3.c: New file.
+	* libio/tst-wmemstream3.c: Likewise.
+	* manual/examples/memstrm.c: Remove warning when priting size_t.
+
 2016-09-30  Joseph Myers  <joseph@codesourcery.com>
 
 	* sysdeps/ieee754/dbl-64/dla.h [__FP_FAST_FMA] (DLA_FMS): Define