| Age | Commit message (Collapse) | Author |
|---|
|
This patch, an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00195.html>, makes it
possible for .test-result files for individual tests to contain XPASS
and XFAIL rather than PASS and FAIL in cases where failure is
expected. This replaces the marking of two individual tests with "-"
to cause them to be expected at makefile level to fail;
evaluate-test.sh will ensure it exits with status 0 for an expected
failure.
Tested x86_64.
* scripts/evaluate-test.sh: Take new argument indicating whether
failure is expected.
* Makeconfig (evaluate-test): Pass argument to evaluate-test.sh
indicating whether failure is expected.
* conform/Makefile (test-xfail-run-conformtest): New variable.
($(objpfx)run-conformtest.out): Don't expect to fail at makefile
level.
* posix/Makefile (test-xfail-annexc): New variable.
($(objpfx)annexc.out): Don't expect to fail at makefile level.
|
|
In <https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html> I
noted it was necessary to add includes of Makeconfig early in various
subdirectory makefiles for the tests-special variable settings added
by that patch to be conditional on configuration information. No-one
commented on the general question there of whether Makeconfig should
always be included immediately after the definition of subdir.
This patch implements that early inclusion of Makeconfig in each
directory (which is a lot easier than consistent placement of includes
of Rules). Includes are added if needed, or moved up if already
present. Subdirectory "all:" targets are removed, since Makeconfig
provides one.
There is potential for further cleanups I haven't done. Rules and
Makerules have code such as
ifneq "$(findstring env,$(origin headers))" ""
headers :=
endif
to override to empty any value of various variables that came from the
environment. I think there is a case for Makeconfig setting all the
subdirectory variables (other than subdir) to empty to ensure no
outside value is going to take effect if a subdirectory fails to
define a variable. (A list of such variables, possibly out of date
and incomplete, is in manual/maint.texi.) Rules and Makerules would
give errors if Makeconfig hadn't already been included, instead of
including it themselves. The special code to override values coming
from the environment would then be obsolete and could be removed.
Tested x86_64, including that installed binaries are identical before
and after the patch.
* argp/Makefile: Include Makeconfig immediately after defining
subdir.
* assert/Makefile: Likewise.
* benchtests/Makefile: Likewise.
* catgets/Makefile: Likewise.
* conform/Makefile: Likewise.
* crypt/Makefile: Likewise.
* csu/Makefile: Likewise.
(all): Remove target.
* ctype/Makefile: Include Makeconfig immediately after defining
subdir.
* debug/Makefile: Likewise.
* dirent/Makefile: Likewise.
* dlfcn/Makefile: Likewise.
* gmon/Makefile: Likewise.
* gnulib/Makefile: Likewise.
* grp/Makefile: Likewise.
* gshadow/Makefile: Likewise.
* hesiod/Makefile: Likewise.
* hurd/Makefile: Likewise.
(all): Remove target.
* iconvdata/Makefile: Include Makeconfig immediately after
defining subdir.
* inet/Makefile: Likewise.
* intl/Makefile: Likewise.
* io/Makefile: Likewise.
* libio/Makefile: Likewise.
(all): Remove target.
* locale/Makefile: Include Makeconfig immediately after defining
subdir.
* login/Makefile: Likewise.
* mach/Makefile: Likewise.
(all): Remove target.
* malloc/Makefile: Include Makeconfig immediately after defining
subdir.
(all): Remove target.
* manual/Makefile: Include Makeconfig immediately after defining
subdir.
* math/Makefile: Likewise.
* misc/Makefile: Likewise.
* nis/Makefile: Likewise.
* nss/Makefile: Likewise.
* po/Makefile: Likewise.
(all): Remove target.
* posix/Makefile: Include Makeconfig immediately after defining
subdir.
* pwd/Makefile: Likewise.
* resolv/Makefile: Likewise.
* resource/Makefile: Likewise.
* rt/Makefile: Likewise.
* setjmp/Makefile: Likewise.
* shadow/Makefile: Likewise.
* signal/Makefile: Likewise.
* socket/Makefile: Likewise.
* soft-fp/Makefile: Likewise.
* stdio-common/Makefile: Likewise.
* stdlib/Makefile: Likewise.
* streams/Makefile: Likewise.
* string/Makefile: Likewise.
* sunrpc/Makefile: Likewise.
(all): Remove target.
* sysvipc/Makefile: Include Makeconfig immediately after defining
subdir.
* termios/Makefile: Likewise.
* time/Makefile: Likewise.
* timezone/Makefile: Likewise.
(all): Remove target.
* wcsmbs/Makefile: Include Makeconfig immediately after defining
subdir.
* wctype/Makefile: Likewise.
libidn/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
localedata/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
(all): Remove target.
nptl/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
nptl_db/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
|
|
This patch, an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00194.html> now
proposed for inclusion in glibc, extends the generation of PASS and
FAIL status in .test-result files for individual tests to cover tests
with their own custom makefile rules. This is just adding
$(evaluate-test) calls to all such rules, since tests with multiple
commands were previously split into separate tests.
Note that the tests the makefiles expect to fail (posix/annexc and
conformtest) currently get FAIL listed in the .test-result file,
rather than XFAIL; a subsequent patch will introduce a better XFAIL
mechanism.
Tested x86_64.
* Makefile ($(objpfx)c++-types-check.out): Use $(evaluate-test).
($(objpfx)check-local-headers.out): Likewise.
($(objpfx)begin-end-check.out): Likewise.
* Makerules (check-abi-%.out): Likewise.
* catgets/Makefile ($(objpfx)test1.cat): Likewise.
($(objpfx)test2.cat): Likewise.
($(objpfx)de/libc.cat): Likewise.
($(objpfx)test-gencat.out): Likewise.
* conform/Makefile ($(objpfx)run-conformtest.out): Likewise.
* elf/Makefile ($(objpfx)order-cmp.out): Likewise.
($(objpfx)noload-mem): Likewise.
($(objpfx)tst-pathopt.out): Likewise.
($(objpfx)tst-rtld-load-self.out): Likewise.
($(objpfx)tst-array1-cmp.out): Likewise.
($(objpfx)tst-array1-static-cmp.out): Likewise.
($(objpfx)tst-array2-cmp.out): Likewise.
($(objpfx)tst-array3-cmp.out): Likewise.
($(objpfx)tst-array4-cmp.out): Likewise.
($(objpfx)tst-array5-cmp.out): Likewise.
($(objpfx)tst-array5-static-cmp.out): Likewise.
($(objpfx)check-textrel.out): Likewise.
($(objpfx)check-execstack.out): Likewise.
($(objpfx)check-localplt.out): Likewise.
($(objpfx)order2-cmp.out): Likewise.
($(objpfx)tst-leaks1-mem): Likewise.
($(objpfx)tst-leaks1-static-mem): Likewise.
($(objpfx)tst-initorder-cmp.out): Likewise.
($(objpfx)tst-initorder2-cmp.out): Likewise.
($(objpfx)tst-unused-dep.out): Likewise.
($(objpfx)tst-unused-dep-cmp.out): Likewise.
* grp/Makefile ($(objpfx)tst_fgetgrent.out): Likewise.
* iconv/Makefile (test-iconvconfig): Likewise.
* iconvdata/Makefile ($(objpfx)mtrace-tst-loading): Likewise.
($(objpfx)iconv-test.out): Likewise.
($(objpfx)tst-tables.out): Likewise.
* intl/Makefile ($(objpfx)mtrace-tst-gettext): Likewise.
($(objpfx)tst-gettext.out): Likewise.
($(objpfx)tst-translit.out): Likewise.
($(objpfx)tst-gettext2.out): Likewise.
($(objpfx)tst-gettext4.out): Likewise.
($(objpfx)tst-gettext6.out): Likewise.
* io/Makefile ($(objpfx)ftwtest.out): Likewise.
* libio/Makefile ($(objpfx)test-freopen.out): Likewise.
($(objpfx)tst-fopenloc-cmp.out): Likewise.
($(objpfx)tst-fopenloc-mem.out): Likewise.
* malloc/Makefile ($(objpfx)tst-mtrace.out): Likewise.
* misc/Makefile ($(objpfx)tst-error1-mem): Likewise.
* posix/Makefile ($(objpfx)globtest.out): Likewise.
($(objpfx)wordexp-tst.out): Likewise.
($(objpfx)annexc.out): Likewise.
($(objpfx)tst-fnmatch-mem): Likewise.
($(objpfx)bug-regex2-mem): Likewise.
($(objpfx)bug-regex14-mem): Likewise.
($(objpfx)bug-regex21-mem): Likewise.
($(objpfx)bug-regex31-mem): Likewise.
($(objpfx)tst-vfork3-mem): Likewise.
($(objpfx)tst-rxspencer-no-utf8-mem): Likewise.
($(objpfx)tst-pcre-mem): Likewise.
($(objpfx)tst-boost-mem): Likewise.
($(objpfx)tst-getconf.out): Likewise.
($(objpfx)bug-ga2-mem): Likewise.
($(objpfx)bug-glob2-mem): Likewise.
* resolv/Makefile ($(objpfx)mtrace-tst-leaks): Likewise.
($(objpfx)mtrace-tst-leaks2): Likewise.
* stdio-common/Makefile ($(objpfx)tst-unbputc.out): Likewise.
($(objpfx)tst-printf.out): Likewise.
($(objpfx)tst-setvbuf1.out): Likewise.
($(objpfx)tst-setvbuf1-cmp.out): Likewise.
* stdlib/Makefile ($(objpfx)isomac.out): Likewise.
($(objpfx)tst-fmtmsg.out): Likewise.
* string/Makefile ($(objpfx)tst-svc-cmp.out): Likewise.
* sysdeps/x86/Makefile ($(objpfx)tst-xmmymm.out): Likewise.
localedata:
* Makefile ($(objpfx)sort-test.out): Use $(evaluate-test).
($(objpfx)tst-fmon.out): Likewise.
($(objpfx)tst-numeric.out): Likewise.
($(objpfx)tst-locale.out): Likewise.
($(objpfx)tst-rpmatch.out): Likewise.
($(objpfx)tst-trans.out): Likewise.
($(objpfx)tst-mbswcs.out): Likewise.
($(objpfx)tst-ctype.out): Likewise.
($(objpfx)tst-wctype.out): Likewise.
($(objpfx)tst-langinfo.out): Likewise.
($(objpfx)mtrace-tst-leaks): Likewise.
nptl:
* Makefile ($(objpfx)tst-stack3-mem): Use $(evaluate-test).
($(objpfx)tst-tls6.out): Likewise.
($(objpfx)tst-cleanup0.out): Likewise.
($(objpfx)tst-cleanup0-cmp.out): Likewise.
($(objpfx)tst-cancel-wrappers.out): Likewise.
($(objpfx)tst-oddstacklimit.out): Likewise.
nptl_db:
* Makefile ($(objpfx)db-symbols.out): Use
$(evaluate-test).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* conform/Makefile: Run run-conformtest.sh using $(BASH).
|
|
|
|
|
|
|
|
|
|
|
|
|
