malloc: Rewrite dumped heap for compatibility in __malloc_set_state

This will allow us to change many aspects of the malloc implementation
while preserving compatibility with existing Emacs binaries.

As a result, existing Emacs binaries will have a larger RSS, and Emacs
needs a few more milliseconds to start.  This overhead is specific
to Emacs (and will go away once Emacs switches to its internal malloc).

The new checks to make free and realloc compatible with the dumped heap
are confined to the mmap paths, which are already quite slow due to the
munmap overhead.

This commit weakens some security checks, but only for heap pointers
in the dumped main arena.  By default, this area is empty, so those
checks are as effective as before.

1 files changed, 14 insertions, 0 deletions

diff --git a/ChangeLog b/ChangeLog
index 78ac7fabf3..1f8db94e4a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,17 @@
+2016-05-13  Florian Weimer  <fweimer@redhat.com>
+
+	* malloc/malloc.c (dumped_main_arena_start)
+	(dumped_main_arena_end): New variables.
+	(DUMPED_MAIN_ARENA_CHUNK): New macro.
+	(do_check_chunk): Skip dumped fake mmapped chunks.
+	(munmap_chunk): Likewise.
+	(__libc_free): Do not adjust statistics for fake mmapped chunks.
+	(__libc_realloc): Adjust hardening check.  Always copy dumped fake
+	mmapped chunks.
+	* malloc/hooks.c (__malloc_set_state): Do not update main_arena.
+	Mark dumped chunks as using mmap.  Update dumped_main_arena_start
+	and dumped_main_arena_end to cover the dumped heap.
+
 2016-05-12  Joseph Myers  <joseph@codesourcery.com>
 
 	* conform/data/time.h-data (getdate_err): Do not expect for