[perl #55504] Failing test t/op/spawnw.t

[Todurov @ Gmail . Com]

# New Ticket Created by ***@gmail.com
# Please include the string: [perl #55504]
# in the subject line of all future correspondence about this issue.
# <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=55504 >


---
osname= linux
osvers= 2.6.15.7
arch= x86_64-linux-gnu-thread-multi
cc= cc
---
Flags:
category=core
severity=medium
ack=no
---
Trying Parrot for the first time ever (so I don't know much) I got this failing test:

t/codingstd/trailing_space...................ok
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
t/op/spawnw.t 7 1792 7 7 100.00% 1-7
29 tests and 597 subtests skipped.
Failed 1/582 test scripts, 99.83% okay. 7/11173 subtests failed, 99.94% okay.
make: *** [test] Error 255

The environment is
Ubuntu
x86_64
gcc (GCC) 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)
2.6.22-14-server
glibc-2.6-1

---
Summary of my parrot 0.6.2 (r28192) configuration:
configdate='Mon Jun 9 05:39:52 2008 GMT'
Platform:
osname=linux, archname=x86_64-linux-gnu-thread-multi
jitcapable=0, jitarchname=nojit,
jitosname=linux, jitcpuarch=amd64
execcapable=0
perl=/usr/bin/perl
Compiler:
cc='cc', ccflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DHASATTRIBUTE_CONST -DHASATTRIBUTE_DEPRECATED -DHASATTRIBUTE_MALLOC -DHASATTRIBUTE_NONNULL -DHASATTRIBUTE_NORETURN -DHASATTRIBUTE_PURE -DHASATTRIBUTE_UNUSED -DHASATTRIBUTE_WARN_UNUSED_RESULT -falign-functions=16 -fvisibility=hidden -maccumulate-outgoing-args -W -Wall -Waggregate-return -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wdisabled-optimization -Wendif-labels -Wextra -Wformat -Wformat-extra-args -Wformat-nonliteral -Wformat-security -Wformat-y2k -Wimplicit -Wimport -Winit-self -Winline -Winvalid-pch -Wmissing-braces -Wmissing-field-initializers -Wno-missing-format-attribute -Wmissing-include-dirs -Wpacked -Wparentheses -Wpointer-arith -Wreturn-type -Wsequence-point -Wno-shadow -Wsign-compare -Wstrict-aliasing -Wstrict-aliasing=2 -Wswitch -Wswitch-default -Wtrigraphs -Wundef -Wunknown-pragmas -Wno-unused
-Wvariadic-macros -Wwrite-strings -Wbad-function-cast -Wc++-compat -Wdeclaration-after-statement -Wimplicit-function-declaration -Wimplicit-int -Wmain -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wnonnull -DHAS_GETTEXT',
Linker and Libraries:
ld='cc', ldflags=' -L/usr/local/lib',
cc_ldflags='',
libs='-ldl -lm -lpthread -lcrypt -lrt '
Dynamic Linking:
share_ext='.so', ld_share_flags='-shared -L/usr/local/lib -fPIC',
load_ext='.so', ld_load_flags='-shared -L/usr/local/lib -fPIC'
Types:
iv=long, intvalsize=8, intsize=4, opcode_t=long, opcode_t_size=8,
ptrsize=8, ptr_alignment=1 byteorder=12345678,
nv=double, numvalsize=8, doublesize=8

---
Environment:
HOME =/home/elmo
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH =/home/elmo/bin:/home/elmo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/var/lib/gems/1.8/gems/mongrel-1.1.4/bin/
SHELL =/bin/bash

[Elmo Todurov]

Actually, I think I nailed the bug. Somehow having weird either LANG or LC_ALL or both set up breaks things.
Note that Estonian alphabet isn't in the same order as English, so having those variables set breaks other things as well (like configure scripts).

$ LC_ALL= LANG= prove --verbose t/op/spawnw.t
t/op/spawnw....1..7
ok 1 - exit code: 0
ok 2 - exit code: 123
ok 3 - exit code: 3
ok 4 - exit code: 0
ok 5 - exit code: 123
ok 6 - exit code: 3
ok 7 - grow argv buffer
ok
All tests successful.
Files=1, Tests=7, 0 wallclock secs ( 0.17 cusr + 0.05 csys = 0.22 CPU)


$ LANG= prove --verbose t/op/spawnw.t
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = "et_EE.UTF-8",
LANG = ""
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
t/op/spawnw....perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = "et_EE.UTF-8",
LANG = ""
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = "et_EE.UTF-8",
LANG = ""
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
1..7

# Failed test 'exit code: 0'
# in t/op/spawnw.t at line 49.
not ok 1 - exit code: 0
# got: 'perl: warning: Setting locale failed.
# perl: warning: Please check that your locale settings:
# LANGUAGE = (unset),
# LC_ALL = "et_EE.UTF-8",
# LANG = ""
# are supported and installed on your system.
# perl: warning: Falling back to the standard locale ("C").
# return code: 0
# '
# expected: 'return code: 0
# '
not ok 2 - exit code: 123

# Failed test 'exit code: 123'
# in t/op/spawnw.t at line 62.
# got: 'perl: warning: Setting locale failed.
# perl: warning: Please check that your locale settings:
# LANGUAGE = (unset),
# LC_ALL = "et_EE.UTF-8",
# LANG = ""
# are supported and installed on your system.
# perl: warning: Falling back to the standard locale ("C").
# return code: 123
# '
# expected: 'return code: 123
# '
not ok 3 - exit code: 3

# Failed test 'exit code: 3'
# in t/op/spawnw.t at line 75.
# got: 'perl: warning: Setting locale failed.
# perl: warning: Please check that your locale settings:
# LANGUAGE = (unset),
# LC_ALL = "et_EE.UTF-8",
# LANG = ""
# are supported and installed on your system.
# perl: warning: Falling back to the standard locale ("C").
# return code: 3
# '
# expected: 'return code: 3
# '
not ok 4 - exit code: 0

# Failed test 'exit code: 0'
# in t/op/spawnw.t at line 90.
# got: 'perl: warning: Setting locale failed.
# perl: warning: Please check that your locale settings:
# LANGUAGE = (unset),
# LC_ALL = "et_EE.UTF-8",
# LANG = ""
# are supported and installed on your system.
# perl: warning: Falling back to the standard locale ("C").
# return code: 0
# '
# expected: 'return code: 0
# '
not ok 5 - exit code: 123

# Failed test 'exit code: 123'
# in t/op/spawnw.t at line 107.
# got: 'perl: warning: Setting locale failed.
# perl: warning: Please check that your locale settings:
# LANGUAGE = (unset),
# LC_ALL = "et_EE.UTF-8",
# LANG = ""
# are supported and installed on your system.
# perl: warning: Falling back to the standard locale ("C").
# return code: 123
# '
# expected: 'return code: 123
# '
not ok 6 - exit code: 3

# Failed test 'exit code: 3'
# in t/op/spawnw.t at line 124.
# got: 'perl: warning: Setting locale failed.
# perl: warning: Please check that your locale settings:
# LANGUAGE = (unset),
# LC_ALL = "et_EE.UTF-8",
# LANG = ""
# are supported and installed on your system.
# perl: warning: Falling back to the standard locale ("C").
# return code: 3
# '
# expected: 'return code: 3
# '
not ok 7 - grow argv buffer

# Failed test 'grow argv buffer'
# in t/op/spawnw.t at line 141.
# got: 'perl: warning: Setting locale failed.
# perl: warning: Please check that your locale settings:
# LANGUAGE = (unset),
# LC_ALL = "et_EE.UTF-8",
# LANG = ""
# are supported and installed on your system.
# perl: warning: Falling back to the standard locale ("C").
# return code: 10
# '
# expected: 'return code: 10
# '
# Looks like you failed 7 tests of 7.
dubious
Test returned status 7 (wstat 1792, 0x700)
DIED. FAILED tests 1-7
Failed 7/7 tests, 0.00% okay
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
t/op/spawnw.t 7 1792 7 7 100.00% 1-7

[NotFound]

(Looks like the second message in this thread was not forwarded to the list).

There are several problems related to LANG. As this ticket shows, perl
warns when it has no support for the locale specified, and the
warnings interferes with the output expected by the tests.

Also several Configure checks for programs can fail because of LANG.
For example, in my system bison is not detected when --maintainer is
used.

Prefixing the commands with LANG=C is a solution in systems with
sh-like shells, but it will break in others.

[Chromatic]

Post by NotFound
(Looks like the second message in this thread was not forwarded to the list).
There are several problems related to LANG. As this ticket shows, perl
warns when it has no support for the locale specified, and the
warnings interferes with the output expected by the tests.
Also several Configure checks for programs can fail because of LANG.
For example, in my system bison is not detected when --maintainer is
used.
Prefixing the commands with LANG=C is a solution in systems with
sh-like shells, but it will break in others.

Could Configure.pl warn when there's no LANG? How portable is that?

-- c

[NotFound]

Post by Chromatic
Could Configure.pl warn when there's no LANG? How portable is that?

Actually Configure.pl warns at his start when the value of LANG is not
supported by perl, being itself a perl program. But I don't know if
this condition can be checked by the program to give a more meaningful
message.

The no LANG is not a problem, in that case the usual behavior is to
silently fall back to the default C or POSIX locale, and perl does not
warn in that case.

So there are two unrelated problems:

- A locale unsupported by the perl config that generates warnings that
interferes with perl tests. This one possibly can be detected and
warned in Configure.pl

- A locale supported by perl that interferes with the output expected
by the test for presence and versions of external programs.


About the second:

It takes me some time to locate the yacc detection problem: the first
line of the output with C locale is:

bison (GNU Bison) 2.3

And with es_ES.UTF-8 is:

bison (GNU bison) 2.3

And the test that fails is:
$stdout =~ /Bison .*? (\d+) \. (\d+) (\w)?/x

A 'b' instead of a 'B', that's all.


I tried a simple solution that may have lesser impact in other OS than
prefixing the commands executed: adding
$ENV{LANG}='C';
before
$conf->runsteps or exit(1);
in Configure.pl

This solves the problem with bison and my locale, and I think will
solve all related problems, but I don't know if it can impact
negatively other steps.


About the first problem, I've found the following in perldoc perllocale:

Perl's moaning about locale problems can be silenced by setting the
environment variable PERL_BADLANG to a zero value, for example "0".
This method really just sweeps the problem under the carpet: you tell
Perl to shut up even when Perl sees that something is wrong. Do not
be surprised if later something locale-dependent misbehaves.

Supposing that all expected test outputs are locale independent,
setting this variable in make test can solve current problems without
risk of creating new ones.

[James Keenan via RT]

Can someone evaluate where we stand with respect to the issues in this RT?

Thank you very much.

kid51

[James Keenan via RT]

Post by James Keenan via RT
Can someone evaluate where we stand with respect to the issues in this RT?
Thank you very much.
kid51

Still hoping for feedback on these issues.

[NotFound]

On Sun, Nov 23, 2008 at 10:41 PM, James Keenan via RT

Post by James Keenan via RT

Post by James Keenan via RT
Can someone evaluate where we stand with respect to the issues in this RT?
Thank you very much.
kid51

Still hoping for feedback on these issues.

Several parts of the configure process now set LANG to C before doing
some check and restore it after. I didn't fixed yet the bison part
expecting a more generic approach under the assumtion that the few
people that use --maintainer can set LANG themselves before
configuring.

Don't have any notice of someone having the bad lang problem trying
the PERL_BADLANG approach. We can just set this value in Configure.pl
and in the Makefile and check smolder results for possible problems.

[James Keenan via RT]

Although I cannot be sure that the issues which were cited in the
original post to this RT have been cleared up, I note that there has
been no follow-up discussion in eight months.

So I'm going to resolve this ticket and encourage any new LANG-related
failures to be reported in new tickets in our Trac system
(https://trac.parrot.org/parrot/newticket).

Thank you very much.
kid51