aboutsummaryrefslogtreecommitdiff
path: root/sysdeps
AgeCommit message (Collapse)Author
2016-05-08Move sysdeps/x86_64/cacheinfo.c to sysdeps/x86H.J. Lu
Move sysdeps/x86_64/cacheinfo.c to sysdeps/x86. No code changes on x86 and x86_64. * sysdeps/i386/cacheinfo.c: Include <sysdeps/x86/cacheinfo.c> instead of <sysdeps/x86_64/cacheinfo.c>. * sysdeps/x86_64/cacheinfo.c: Moved to ... * sysdeps/x86/cacheinfo.c: Here.
2016-05-04Revert "aio: fix newp->running data race"Samuel Thibault
This reverts commit fd67a9cf7b733da082e4b6a5f25c19ea7921b4cd.
2016-05-04aio: fix newp->running data raceSamuel Thibault
* sysdeps/pthread/aio_misc.c (__aio_enqueue_request): Do not write `running` field of `newp` when a thread was started to process it, since that thread will not take `__aio_requests_mutex`, and the field already has the proper value actually.
2016-05-04powerpc: Fix operand prefixesGabriel F. T. Gomes
The file sysdeps/powerpc/sysdeps.h defines aliases for condition register operands. E.g.: 'cr7' means condition register 7. On the one hand, this increases readability, as it makes it easier for readers to know whether the operand is a condition register, a general purpose register or an immediate. On the other hand, this permits that condition registers be written as if they were general purpose, and vice-versa, thus reducing the readability of the code. This commit removes some of these unintentional misuses. The changes have no effect on the final code. Checked with objdump.
2016-05-04CVE-2016-1234: glob: Do not copy d_name field of struct dirent [BZ #19779]Florian Weimer
Instead, we store the data we need from the return value of readdir in an object of the new type struct readdir_result. This type is independent of the layout of struct dirent.
2016-05-03powerpc: Add missing insn in swapcontext [BZ #20004]Paul E. Murphy
A missing instruction was discovered in the compat version of swapcontext while running the GCC test suite.
2016-05-02powerpc: Fix clone CLONE_VM compareAdhemerval Zanella
This patch fixes the clone CLONE_VM change from 0cb313f (BZ#19957) where the commit changed the register that contains the save flags argument to compare with (from r28 to r29). This patch changes back to correct register. Tested on powerpc32 (thanks to Tulio Magno Quites Machado Filho). * sysdeps/unix/sysv/linux/powerpc/powerpc32/clone.S (__clone): Fix flags CLONE_VM compare.
2016-04-30m68k: use large PIC model for gcrt1.oAndreas Schwab
2016-04-30m68k: avoid local labels in symbol tableAndreas Schwab
2016-04-29Fix clone (CLONE_VM) pid/tid reset (BZ#19957)Adhemerval Zanella
As discussed in libc-alpha [1] current clone with CLONE_VM (without CLONE_THREAD set) will reset the pthread pid/tid fields to -1. The issue is since memory is shared between the parent and child it will clobber parent's cached pid/tid leading to internal inconsistencies if the value is not restored. And even it is restored it may lead to racy conditions when between set/restore a thread might invoke pthread function that validate the pthread with INVALID_TD_P/INVALID_NOT_TERMINATED_TD_P and thus get wrong results. As stated in BZ19957, previously reports of this behaviour was close with EWONTFIX due the fact usage of clone outside glibc is tricky since glibc requires consistent internal pthread, while using clone directly may not provide it. However since now posix_spawn uses clone (CLONE_VM) to fixes various issues related to previous vfork usage this issue requires fixing. The vfork implementation also does something similar, but instead it negates and restores only the *pid* field and functions that might access its value know to handle such case (getpid, raise and pthread ones that uses INVALID_TD_P/INVALID_NOT_TERMINATED_TD_P macros that check only *tid* field). Also vfork does not call __clone directly, instead calling either __NR_vfork or __NR_clone directly. So this patch removes this clone behavior by avoiding setting the pthread pid/tid field for CLONE_VM. There is no need to check for CLONE_THREAD, since the minimum supported kernel in all architecture implies that CLONE_VM must be used with CLONE_THREAD, otherwise clone returns EINVAL. Instead of current approach of: int clone(int (*fn)(void *), void *child_stack, int flags, ...) [...] if (flags & CLONE_THREAD) goto do_syscall; pid_t new_value; if (flags & CLONE_VM) new_value = -1; else new_value = getpid (); THREAD_SETMEM (THREAD_SELF, pid, new_value); THREAD_SETMEM (THREAD_SELF, tid, new_value); do_syscall: [...] The new approach uses: int clone(int (*fn)(void *), void *child_stack, int flags, ...) [...] if (flags & CLONE_VM) goto do_syscall; pid_t new_value = getpid (); THREAD_SETMEM (THREAD_SELF, pid, new_value); THREAD_SETMEM (THREAD_SELF, tid, new_value); do_syscall: [...] It also removes the linux tst-getpid2.c test which expects the previous behavior and instead add another clone test. Tested on x86_64, i686, x32, powerpc64le, aarch64, armhf, s390, and s390x. I also did limited check on mips32 and sparc64 (using the new added test). I also got reviews from both m68k, hppa, and tile. So I presume for these architecture the patch works. The fixes for alpha, microblaze, sh, ia64, and nio2 have not been tested. [1] https://sourceware.org/ml/libc-alpha/2016-04/msg00307.html * sysdeps/unix/sysv/linux/Makefile [$(subdir) == nptl] (test): Remove tst-getpid2. (test): Add tst-clone2. * sysdeps/unix/sysv/linux/tst-clone2.c: New file. * sysdeps/unix/sysv/linux/aarch64/clone.S (__clone): Do not change pid/tid fields for CLONE_VM. * sysdeps/unix/sysv/linux/arm/clone.S: Likewise. * sysdeps/unix/sysv/linux/i386/clone.S: Likewise. * sysdeps/unix/sysv/linux/mips/clone.S: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc32/clone.S: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S: Likewise. * sysdeps/unix/sysv/linux/s390/s390-32/clone.S: Likewise. * sysdeps/unix/sysv/linux/s390/s390-64/clone.S: Likewise. * sysdeps/unix/sysv/linux/sparc/sparc32/clone.S: Likewise. * sysdeps/unix/sysv/linux/sparc/sparc64/clone.S: Likewise. * sysdeps/unix/sysv/linux/x86_64/clone.S: Likewise. * sysdeps/unix/sysv/linux/tst-getpid2.c: Remove file.
2016-04-29powerpc: Zero pad using memset in strncpy/stpncpyGabriel F. T. Gomes
Call __memset_power8 to pad, with zeros, the remaining bytes in the dest string on __strncpy_power8 and __stpncpy_power8. This improves performance when n is larger than the input string, giving ~30% gain for larger strings without impacting much shorter strings.
2016-04-29CVE-2016-3706: getaddrinfo: stack overflow in hostent conversion [BZ #20010]Florian Weimer
When converting a struct hostent response to struct gaih_addrtuple, the gethosts macro (which is called from gaih_inet) used alloca, without malloc fallback for large responses. This commit changes this code to use calloc unconditionally. This commit also consolidated a second hostent-to-gaih_addrtuple conversion loop (in gaih_inet) to use the new conversion function.
2016-04-27Add missing iucv related defines.Stefan Liebler
this patch adds the missing SOL_IUCV socket level definition and socket options SO_IPRMDATA_MSG, SO_MSGLIMIT, SO_MSGSIZE which can be used with get/setsockopt(). SCM_IUCV_TRGCLS is needed to send/receive ancillary data with send/recvmsg(). The defines are copied from kernel-source: include/net/iucv/af_iucv.h include/linux/socket.h
2016-04-25powerpc: Add optimized strcspn for P8Paul E. Murphy
A few minor adjustments to the P8 strspn gives us an almost equally optimized P8 strcspn.
2016-04-25Fix stdlib/tst-makecontext regression for Nios IIChung-Lin Tang
2016-04-22powerpc: strcasestr optmization for power8Rajalakshmi Srinivasaraghavan
This patch optimizes strcasestr function for power >= 8 systems. The average improvement of this optimization is ~40% and compares 16 bytes at a time using vector instructions. This patch is tested on powerpc64 and powerpc64le.
2016-04-19Fix gprof timingSamuel Thibault
* sysdeps/mach/hurd/profil.c (__profile_frequency): Return tick frequency instead of tick length in us.
2016-04-19hurd: fix profiling short-living processesSamuel Thibault
* sysdeps/mach/hurd/profil.c (update_waiter): Initialize profil_reply_port. (profile_waiter): Do not initialize profil_reply_port.
2016-04-15powerpc: Optimization for strlen for POWER8.Carlos Eduardo Seo
This implementation takes advantage of vectorization to improve performance of the loop over the current strlen implementation for POWER7.
2016-04-15Detect Intel Goldmont and Airmont processorsH.J. Lu
Updated from the model numbers of Goldmont and Airmont processors in Intel64 And IA-32 Processor Architectures Software Developer's Manual Volume 3 Revision 058. * sysdeps/x86/cpu-features.c (init_cpu_features): Detect Intel Goldmont and Airmont processors.
2016-04-14Fix pread consolidation on ports that require argument alignmentAdhemerval Zanella
This patch fixes the __ALIGNMENT_{ARG,COUNT} definition for ports that define __ASSUME_ALIGNED_REGISTER_PAIRS by including the kernel-features.h (where it is defined if the case). This was shown on arm with failing cases: FAIL: debug/tst-chk1 FAIL: debug/tst-chk2 FAIL: debug/tst-chk3 FAIL: debug/tst-chk4 FAIL: debug/tst-chk5 FAIL: debug/tst-chk6 FAIL: debug/tst-lfschk1 FAIL: debug/tst-lfschk2 FAIL: debug/tst-lfschk3 FAIL: debug/tst-lfschk4 FAIL: debug/tst-lfschk5 FAIL: debug/tst-lfschk6 FAIL: posix/tst-preadwrite FAIL: posix/tst-preadwrite64 The patches fixes it. Tested on armhf. * sysdeps/unix/sysv/linux/sysdep.h: Include kernel-features.h.
2016-04-14malloc: Remove unused definitions of thread_atfork, thread_atfork_staticFlorian Weimer
2016-04-14malloc: Run fork handler as late as possible [BZ #19431]Florian Weimer
Previously, a thread M invoking fork would acquire locks in this order: (M1) malloc arena locks (in the registered fork handler) (M2) libio list lock A thread F invoking flush (NULL) would acquire locks in this order: (F1) libio list lock (F2) individual _IO_FILE locks A thread G running getdelim would use this order: (G1) _IO_FILE lock (G2) malloc arena lock After executing (M1), (F1), (G1), none of the threads can make progress. This commit changes the fork lock order to: (M'1) libio list lock (M'2) malloc arena locks It explicitly encodes the lock order in the implementations of fork, and does not rely on the registration order, thus avoiding the deadlock.
2016-04-14Remove union wait [BZ #19613]Florian Weimer
The overloading approach in the W* macros was incompatible with integer expressions of a type different from int. Applications using union wait and these macros will have to migrate to the POSIX-specified int status type.
2016-04-13Register extra test objectsAndreas Schwab
This makes sure that the extra test objects are compiled with the correct MODULE_NAME and dependencies are tracked.
2016-04-12X86-64: Use non-temporal store in memcpy on large dataH.J. Lu
The large memcpy micro benchmark in glibc shows that there is a regression with large data on Haswell machine. non-temporal store in memcpy on large data can improve performance significantly. This patch adds a threshold to use non temporal store which is 6 times of shared cache size. When size is above the threshold, non temporal store will be used, but avoid non-temporal store if there is overlap between destination and source since destination may be in cache when source is loaded. For size below 8 vector register width, we load all data into registers and store them together. Only forward and backward loops, which move 4 vector registers at a time, are used to support overlapping addresses. For forward loop, we load the last 4 vector register width of data and the first vector register width of data into vector registers before the loop and store them after the loop. For backward loop, we load the first 4 vector register width of data and the last vector register width of data into vector registers before the loop and store them after the loop. [BZ #19928] * sysdeps/x86_64/cacheinfo.c (__x86_shared_non_temporal_threshold): New. (init_cacheinfo): Set __x86_shared_non_temporal_threshold to 6 times of shared cache size. * sysdeps/x86_64/multiarch/memmove-avx-unaligned-erms.S (VMOVNT): New. * sysdeps/x86_64/multiarch/memmove-avx512-unaligned-erms.S (VMOVNT): Likewise. * sysdeps/x86_64/multiarch/memmove-sse2-unaligned-erms.S (VMOVNT): Likewise. (VMOVU): Changed to movups for smaller code sizes. (VMOVA): Changed to movaps for smaller code sizes. * sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: Update comments. (PREFETCH): New. (PREFETCH_SIZE): Likewise. (PREFETCHED_LOAD_SIZE): Likewise. (PREFETCH_ONE_SET): Likewise. Rewrite to use forward and backward loops, which move 4 vector registers at a time, to support overlapping addresses and use non temporal store if size is above the threshold and there is no overlap between destination and source.
2016-04-12VDSO support for MIPSMatthew Fortune
This patch adds support for using the implementations of gettimeofday() and clock_gettime() provided by the kernel in the VDSO. The VDSO will always provide clock_gettime() as CLOCK_{REALTIME,MONOTONIC}_COARSE can be implemented regardless of platform. CLOCK_{REALTIME,MONOTONIC}, along with gettimeofday(), are only implemented on platforms which make use of either the CP0 count or GIC as their clocksource. On other platforms, the VDSO does not provide the __vdso_gettimeofday symbol, as it is never useful. The VDSO functions return ENOSYS when they encounter an unsupported request, in which case glibc should fall back to the standard syscall. Tested with upstream kernel 4.5 and QEMU emulating Malta. ./vdsotest gettimeofday bench gettimeofday: syscall: 1021 nsec/call gettimeofday: libc: 262 nsec/call gettimeofday: vdso: 174 nsec/call * sysdeps/unix/sysv/linux/mips/Makefile (sysdep_routines): Include dl-vdso. * sysdeps/unix/sysv/linux/mips/Versions: Add __vdso_clock_gettime. * sysdeps/unix/sysv/linux/mips/init-first.c: New file. * sysdeps/unix/sysv/linux/mips/libc-vdso.h: New file. * sysdeps/unix/sysv/linux/mips/mips32/sysdep.h: (INTERNAL_VSYSCALL_CALL): Define to be compatible with MIPS definitions of INTERNAL_SYSCALL_{ERROR_P,ERRNO}. (HAVE_CLOCK_GETTIME_VSYSCALL): Define. (HAVE_GETTIMEOFDAY_VSYSCALL): Define. * sysdeps/unix/sysv/linux/mips/mips64/n32/sysdep.h: Likewise. * sysdeps/unix/sysv/linux/mips/mips64/n64/sysdep.h: Likewise.
2016-04-11Consolidate pwrite/pwrite64 implementationsAdhemerval Zanella
This patch consolidates all the pwrite/pwrite64 implementation for Linux in only one (sysdeps/unix/sysv/linux/pwrite{64}.c). It also removes the syscall from the auto-generation using assembly macros. For pwrite{64} offset argument placement the new SYSCALL_LL{64} macro is used. For pwrite ports that do not define __NR_pwrite will use __NR_pwrite64 and for pwrite64 ports that dot define __NR_pwrite64 will use __NR_pwrite for the syscall. Checked on x86_64, x32, i386, aarch64, and ppc64le. * sysdeps/unix/sysv/linux/arm/pwrite.c: Remove file. * sysdeps/unix/sysv/linux/arm/pwrite64.c: Likewise. * sysdeps/unix/sysv/linux/generic/wordsize-32/pwrite.c: Likewise. * sysdeps/unix/sysv/linux/generic/wordsize-32/pwrite64.c: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc32/pwrite.c: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc32/pwrite64.c: Likewise. * sysdeps/unix/sysv/linux/wordsize-64/pwrite64.c: Likewise. * sysdeps/unix/sysv/linux/wordsize-64/syscalls.list (prite): Remove syscalls generation. * sysdeps/unix/sysv/linux/powerpc/powerpc32/sysdep.h [__NR_pwrite64] (__NR_write): Remove define. * sysdeps/unix/sysv/linux/powerpc/powerpc64/sysdep.h [__NR_pwrite64] (__NR_write): Remove define. * sysdeps/unix/sysv/linux/pwrite.c [__NR_pwrite64] (__NR_pwrite): Remove define. (__libc_pwrite): Use SYSCALL_LL macro on offset argument. * sysdeps/unix/sysv/linux/pwrite64.c [__NR_pwrite64] (__NR_pwrite): Remove define. (__libc_pwrite64): Use SYSCALL_LL64 macro on offset argument. * sysdeps/unix/sysv/linux/sh/pwrite.c: Rewrite using default Linux implementation as base. * sysdeps/unix/sysv/linux/sh/pwrite64.c: Likewise. * sysdeps/unix/sysv/linux/mips/pwrite.c: Likewise. * sysdeps/unix/sysv/linux/mips/pwrite64.c: Likewise.
2016-04-11Consolidate pread/pread64 implementationsAdhemerval Zanella
This patch consolidates all the pread/pread64 implementation for Linux in only one (sysdeps/unix/sysv/linux/pread.c). It also removes the syscall from the auto-generation using assembly macros. For pread{64} offset argument placement the new SYSCALL_LL{64} macro is used. For pread ports that do not define __NR_pread will use __NR_pread64 and for pread64 ports that dot define __NR_pread64 will use __NR_pread for the syscall. Checked on x86_64, x32, i386, aarch64, and ppc64le. * sysdeps/unix/sysv/linux/arm/pread.c: Remove file. * sysdeps/unix/sysv/linux/arm/pread64.c: Likewise. * sysdeps/unix/sysv/linux/generic/wordsize-32/pread.c: Likewise. * sysdeps/unix/sysv/linux/generic/wordsize-32/pread64.c: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc32/pread.c: Likewise, * sysdeps/unix/sysv/linux/powerpc/powerpc32/pread64.c: Likewise. * sysdeps/unix/sysv/linux/wordsize-64/pread64.c: Likewise. * sysdeps/unix/sysv/linux/wordsize-64/syscalls.list (pread): Remove syscall generation. * sysdeps/unix/sysv/linux/powerpc/powerpc32/sysdep.h [__NR_pread64] (__NR_pread): Remove define. * sysdeps/unix/sysv/linux/powerpc/powerpc64/sysdep.h: [__NR_pread64] (__NR_pread): Likewise. * sysdeps/unix/sysv/linux/pread.c [__NR_pread64] (__NR_pread): Remove define. (__libc_pread): Use SYSCALL_LL macro on offset argument. * sysdeps/unix/sysv/linux/pread64.c [__NR_pread64] (__NR_pread): Remove define. (__libc_pread64): Use SYSCALL_LL64 macro on offset argument. * sysdeps/unix/sysv/linux/sh/pread.c: Rewrite using default Linux implementation as base. * sysdeps/unix/sysv/linux/sh/pread64.c: Likewise. * sysdeps/unix/sysv/linux/mips/pread.c: Likewise. * sysdeps/unix/sysv/linux/mips/pread64.c: Likewise.
2016-04-11Consolidate off_t/off64_t syscall argument passingAdhemerval Zanella
This patch add three new macros (SYSCALL_LL, SYSCALL_LL64, and __ASSUME_WORDSIZE64_ILP32) to use along with off_t and off64_t argument syscalls. The rationale for this change is: 1. Remove multiple implementations for the same syscall for different architectures (for instance, pread have 6 different implementations). 2. Also remove the requirement to use syscall wrappers for cancellable entrypoints. The macro usage should be used along __ALIGNMENT_ARG to follow ABI constrains for architecture where it applies. For instance, pread can be rewritten as: return SYSCALL_CANCEL (pread, fd, buf, count, __ALIGNMENT_ARG SYSCALL_LL (offset)); Another macro, SYSCALL_LL64, is provided for off64_t. The macro __ASSUME_WORDSIZE64_ILP32 is used by the ABI to define is uses 64-bit register even if ABI is ILP32 (for instance x32 and mips64-n32). The changes itself are not currently used in any implementation, so no code change is expected. * sysdeps/unix/sysv/linux/generic/sysdep.h (__ALIGNMENT_ARG): Move definition. (__ALIGNMENT_COUNT): Likewise. * sysdeps/unix/sysv/linux/sysdep.h (__ALIGNMENT_ARG): To here. (__ALIGNMENT_COUNT): Likewise. (SYSCALL_LL): New define. (SYSCALL_LL64): Likewise. * sysdeps/unix/sysv/linux/mips/kernel-features.h: [_MIPS_SIM == _ABIO32] (__ASSUME_WORDSIZE64_ILP32): Define. * sysdeps/unix/sysv/linux/x86_64/kernel-features.h: [ILP32] (__ASUME_WORDSIZE64_ILP32): Likewise.
2016-04-11Define __ASSUME_ALIGNED_REGISTER_PAIRS for missing portsAdhemerval Zanella
This patch defines __ASSUME_ALIGNED_REGISTER_PAIRS for the missing ports that require 64-bit value (e.g., long long) to be aligned to an even register pair in argument passing. No code change is expected, tested with builds for powerpc32, mips-o32, and armhf. * sysdeps/unix/sysv/linux/arm/kernel-features.h (__ASSUME_ALIGNED_REGISTER_PAIRS): Define. * sysdeps/unix/sysv/linux/mips/kernel-features.h [_MIPS_SIM == _ABIO32] (__ASSUME_ALIGNED_REGISTER_PAIRS): Likewise. * sysdeps/unix/sysv/linux/powerpc/kernel-features.h [!__powerpc64__] (__ASSUME_ALIGNED_REGISTER_PAIRS): Likewise.
2016-04-11Fix build with HAVE_AUX_VECTORSamuel Thibault
* sysdeps/unix/sysv/linux/ldsodefs.h (HAVE_AUX_VECTOR): Define before including <ldsodefs.h>. * sysdeps/nacl/ldsodefs.h (HAVE_AUX_VECTOR): Likewise.
2016-04-10Fix crash on getauxval call without HAVE_AUX_VECTORSamuel Thibault
* sysdeps/generic/ldsodefs.h (struct rtld_global_ro) [!HAVE_AUX_VECTOR]: Do not define _dl_auxv field. * misc/getauxval.c (__getauxval) [!HAVE_AUX_VECTOR]: Do not go through GLRO(dl_auxv) list.
2016-04-09When disabling SSE, make sure -fpmath is not set to use SSE eitherKhem Raj
This fixes errors when we inject sse options through CFLAGS and now that we have -Werror turned on by default this warning turns into an error on x86: $ gcc -m32 -march=core2 -mtune=core2 -msse3 -mfpmath=sse -x c /dev/null -S -mno-sse -mno-mmx /dev/null:1:0: warning: SSE instruction set disabled, using 387 arithmetics Where as: $ gcc -m32 -march=core2 -mtune=core2 -msse3 -mfpmath=sse -x c /dev/null -S -mno-sse -mno-mmx -mfpmath=387 Generates no warnings.
2016-04-09configure: fix `test ==` usageMike Frysinger
POSIX defines the = operator, but not ==. Fix the few places where we incorrectly used ==.
2016-04-08Fix termios.h XCASE namespace (bug 19925).Joseph Myers
bits/termios.h (various versions under sysdeps/unix/sysv/linux) defines XCASE if defined __USE_MISC || defined __USE_XOPEN. This macro was removed in the 2001 edition of POSIX, and is not otherwise reserved, so should not be defined for 2001 and later versions of POSIX. This patch fixes the conditions accordingly (leaving the macro defined for __USE_MISC, so still in the default namespace). Tested for x86_64 and x86 (testsuite, and that installed shared libraries are unchanged by the patch). [BZ #19925] * sysdeps/unix/sysv/linux/alpha/bits/termios.h (XCASE): Do not define if [!__USE_MISC && __USE_XOPEN2K]. * sysdeps/unix/sysv/linux/bits/termios.h (XCASE): Likewise. * sysdeps/unix/sysv/linux/mips/bits/termios.h (XCASE): Likewise. * sysdeps/unix/sysv/linux/powerpc/bits/termios.h (XCASE): Likewise. * sysdeps/unix/sysv/linux/sparc/bits/termios.h (XCASE): Likewise. * conform/Makefile (test-xfail-XOPEN2K/termios.h/conform): Remove variable. (test-xfail-XOPEN2K8/termios.h/conform): Likewise.
2016-04-07powerpc: Add optimized P8 strspnPaul E. Murphy
This utilizes vectors and bitmasks. For small needle, large haystack, the performance improvement is upto 8x. For short strings (0-4B), the cost of computing the bitmask dominates, and is a tad slower.
2016-04-06X86-64: Prepare memmove-vec-unaligned-erms.SH.J. Lu
Prepare memmove-vec-unaligned-erms.S to make the SSE2 version as the default memcpy, mempcpy and memmove. * sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S (MEMCPY_SYMBOL): New. (MEMPCPY_SYMBOL): Likewise. (MEMMOVE_CHK_SYMBOL): Likewise. Replace MEMMOVE_SYMBOL with MEMMOVE_CHK_SYMBOL on __mempcpy_chk symbols. Replace MEMMOVE_SYMBOL with MEMPCPY_SYMBOL on __mempcpy symbols. Provide alias for __memcpy_chk in libc.a. Provide alias for memcpy in libc.a and ld.so.
2016-04-06X86-64: Prepare memset-vec-unaligned-erms.SH.J. Lu
Prepare memset-vec-unaligned-erms.S to make the SSE2 version as the default memset. * sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S (MEMSET_CHK_SYMBOL): New. Define if not defined. (__bzero): Check VEC_SIZE == 16 instead of USE_MULTIARCH. Disabled fro now. Replace MEMSET_SYMBOL with MEMSET_CHK_SYMBOL on __memset_chk symbols. Properly check USE_MULTIARCH on __memset symbols.
2016-04-05Force 32-bit displacement in memset-vec-unaligned-erms.SH.J. Lu
* sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: Force 32-bit displacement to avoid long nop between instructions.
2016-04-05Add a comment in memset-sse2-unaligned-erms.SH.J. Lu
* sysdeps/x86_64/multiarch/memset-sse2-unaligned-erms.S: Add a comment on VMOVU and VMOVA.
2016-04-03Don't put SSE2/AVX/AVX512 memmove/memset in ld.soH.J. Lu
Since memmove and memset in ld.so don't use IFUNC, don't put SSE2, AVX and AVX512 memmove and memset in ld.so. * sysdeps/x86_64/multiarch/memmove-avx-unaligned-erms.S: Skip if not in libc. * sysdeps/x86_64/multiarch/memmove-avx512-unaligned-erms.S: Likewise. * sysdeps/x86_64/multiarch/memset-avx2-unaligned-erms.S: Likewise. * sysdeps/x86_64/multiarch/memset-avx512-unaligned-erms.S: Likewise.
2016-04-03Fix memmove-vec-unaligned-erms.SH.J. Lu
__mempcpy_erms and __memmove_erms can't be placed between __memmove_chk and __memmove it breaks __memmove_chk. Don't check source == destination first since it is less common. * sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: (__mempcpy_erms, __memmove_erms): Moved before __mempcpy_chk with unaligned_erms. (__memmove_erms): Skip if source == destination. (__memmove_unaligned_erms): Don't check source == destination first.
2016-04-01Remove Fast_Copy_Backward from Intel Core processorsH.J. Lu
Intel Core i3, i5 and i7 processors have fast unaligned copy and copy backward is ignored. Remove Fast_Copy_Backward from Intel Core processors to avoid confusion. * sysdeps/x86/cpu-features.c (init_cpu_features): Don't set bit_arch_Fast_Copy_Backward for Intel Core proessors.
2016-04-01Remove powerpc64 strspn, strcspn, and strpbrk implementationAdhemerval Zanella
This patch removes the powerpc64 optimized strspn, strcspn, and strpbrk assembly implementation now that the default C one implements the same strategy. On internal glibc benchtests current implementations shows similar performance with -O2. Tested on powerpc64le (POWER8). * sysdeps/powerpc/powerpc64/strcspn.S: Remove file. * sysdeps/powerpc/powerpc64/strpbrk.S: Remove file. * sysdeps/powerpc/powerpc64/strspn.S: Remove file.
2016-04-01Improve generic strcspn performanceWilco Dijkstra
Improve strcspn performance using a much faster algorithm. It is kept simple so it works well on most targets. It is generally at least 10 times faster than the existing implementation on bench-strcspn on a few AArch64 implementations, and for some tests 100 times as fast (repeatedly calling strchr on a small string is extremely slow...). In fact the string/bits/string2.h inlines make no longer sense, as GCC already uses strlen if reject is an empty string, strchrnul is 5 times as fast as __strcspn_c1, while __strcspn_c2 and __strcspn_c3 are slower than the strcspn main loop for large strings (though reject length 2-4 could be special cased in the future to gain even more performance). Tested on x86_64, i686, and aarch64. * string/Version (libc): Add GLIBC_2.24. * string/strcspn.c (strcspn): Rewrite function. * string/bits/string2.h (strcspn): Use __builtin_strcspn. (__strcspn_c1): Remove inline function. (__strcspn_c2): Likewise. (__strcspn_c3): Likewise. * string/string-inline.c [SHLIB_COMPAT(libc, GLIBC_2_1_1, GLIBC_2_24)] (__strcspn_c1): Add compatibility symbol. [SHLIB_COMPAT(libc, GLIBC_2_1_1, GLIBC_2_24)] (__strcspn_c2): Likewise. [SHLIB_COMPAT(libc, GLIBC_2_1_1, GLIBC_2_24)] (__strcspn_c3): Likewise. * sysdeps/i386/string-inlines.c: Include generic string-inlines.c.
2016-04-01S390: Use ahi instead of aghi in 32bit _dl_runtime_resolve.Stefan Liebler
This patch uses ahi instead of aghi in 32bit _dl_runtime_resolve to adjust the stack pointer. This is no functional change, but a cosmetic one. ChangeLog: * sysdeps/s390/s390-32/dl-trampoline.h (_dl_runtime_resolve): Use ahi instead of aghi to adjust stack pointer.
2016-03-31Increase internal precision of ldbl-128ibm decimal printf [BZ #19853]Paul E. Murphy
When the signs differ, the precision of the conversion sometimes drops below 106 bits. This strategy is identical to the hexadecimal variant. I've refactored tst-sprintf3 to enable testing a value with more than 30 significant digits in order to demonstrate this failure and its solution. Additionally, this implicitly fixes a typo in the shift quantities when subtracting from the high mantissa to compute the difference.
2016-03-31Add x86-64 memset with unaligned store and rep stosbH.J. Lu
Implement x86-64 memset with unaligned store and rep movsb. Support 16-byte, 32-byte and 64-byte vector register sizes. A single file provides 2 implementations of memset, one with rep stosb and the other without rep stosb. They share the same codes when size is between 2 times of vector register size and REP_STOSB_THRESHOLD which defaults to 2KB. Key features: 1. Use overlapping store to avoid branch. 2. For size <= 4 times of vector register size, fully unroll the loop. 3. For size > 4 times of vector register size, store 4 times of vector register size at a time. [BZ #19881] * sysdeps/x86_64/multiarch/Makefile (sysdep_routines): Add memset-sse2-unaligned-erms, memset-avx2-unaligned-erms and memset-avx512-unaligned-erms. * sysdeps/x86_64/multiarch/ifunc-impl-list.c (__libc_ifunc_impl_list): Test __memset_chk_sse2_unaligned, __memset_chk_sse2_unaligned_erms, __memset_chk_avx2_unaligned, __memset_chk_avx2_unaligned_erms, __memset_chk_avx512_unaligned, __memset_chk_avx512_unaligned_erms, __memset_sse2_unaligned, __memset_sse2_unaligned_erms, __memset_erms, __memset_avx2_unaligned, __memset_avx2_unaligned_erms, __memset_avx512_unaligned_erms and __memset_avx512_unaligned. * sysdeps/x86_64/multiarch/memset-avx2-unaligned-erms.S: New file. * sysdeps/x86_64/multiarch/memset-avx512-unaligned-erms.S: Likewise. * sysdeps/x86_64/multiarch/memset-sse2-unaligned-erms.S: Likewise. * sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: Likewise.
2016-03-31Add x86-64 memmove with unaligned load/store and rep movsbH.J. Lu
Implement x86-64 memmove with unaligned load/store and rep movsb. Support 16-byte, 32-byte and 64-byte vector register sizes. When size <= 8 times of vector register size, there is no check for address overlap bewteen source and destination. Since overhead for overlap check is small when size > 8 times of vector register size, memcpy is an alias of memmove. A single file provides 2 implementations of memmove, one with rep movsb and the other without rep movsb. They share the same codes when size is between 2 times of vector register size and REP_MOVSB_THRESHOLD which is 2KB for 16-byte vector register size and scaled up by large vector register size. Key features: 1. Use overlapping load and store to avoid branch. 2. For size <= 8 times of vector register size, load all sources into registers and store them together. 3. If there is no address overlap bewteen source and destination, copy from both ends with 4 times of vector register size at a time. 4. If address of destination > address of source, backward copy 8 times of vector register size at a time. 5. Otherwise, forward copy 8 times of vector register size at a time. 6. Use rep movsb only for forward copy. Avoid slow backward rep movsb by fallbacking to backward copy 8 times of vector register size at a time. 7. Skip when address of destination == address of source. [BZ #19776] * sysdeps/x86_64/multiarch/Makefile (sysdep_routines): Add memmove-sse2-unaligned-erms, memmove-avx-unaligned-erms and memmove-avx512-unaligned-erms. * sysdeps/x86_64/multiarch/ifunc-impl-list.c (__libc_ifunc_impl_list): Test __memmove_chk_avx512_unaligned_2, __memmove_chk_avx512_unaligned_erms, __memmove_chk_avx_unaligned_2, __memmove_chk_avx_unaligned_erms, __memmove_chk_sse2_unaligned_2, __memmove_chk_sse2_unaligned_erms, __memmove_avx_unaligned_2, __memmove_avx_unaligned_erms, __memmove_avx512_unaligned_2, __memmove_avx512_unaligned_erms, __memmove_erms, __memmove_sse2_unaligned_2, __memmove_sse2_unaligned_erms, __memcpy_chk_avx512_unaligned_2, __memcpy_chk_avx512_unaligned_erms, __memcpy_chk_avx_unaligned_2, __memcpy_chk_avx_unaligned_erms, __memcpy_chk_sse2_unaligned_2, __memcpy_chk_sse2_unaligned_erms, __memcpy_avx_unaligned_2, __memcpy_avx_unaligned_erms, __memcpy_avx512_unaligned_2, __memcpy_avx512_unaligned_erms, __memcpy_sse2_unaligned_2, __memcpy_sse2_unaligned_erms, __memcpy_erms, __mempcpy_chk_avx512_unaligned_2, __mempcpy_chk_avx512_unaligned_erms, __mempcpy_chk_avx_unaligned_2, __mempcpy_chk_avx_unaligned_erms, __mempcpy_chk_sse2_unaligned_2, __mempcpy_chk_sse2_unaligned_erms, __mempcpy_avx512_unaligned_2, __mempcpy_avx512_unaligned_erms, __mempcpy_avx_unaligned_2, __mempcpy_avx_unaligned_erms, __mempcpy_sse2_unaligned_2, __mempcpy_sse2_unaligned_erms and __mempcpy_erms. * sysdeps/x86_64/multiarch/memmove-avx-unaligned-erms.S: New file. * sysdeps/x86_64/multiarch/memmove-avx512-unaligned-erms.S: Likwise. * sysdeps/x86_64/multiarch/memmove-sse2-unaligned-erms.S: Likwise. * sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: Likwise.
generated by cgit v1.2.3-70-g09d2 (git 2.48.1) at 2025-03-26 23:40:34 +0000