This Week on perl5-porters - 25-31 August 2008

This Week on perl5-porters - 25-31 August 2008
Topics of Interest

Profiling parallel tests
Wrong RE match in 5.10.0
dprofpp enhanced
Regaining 'O's
Proposed pragma/module ensure
Moving lib/ modules to ext/?
unless(...) terser than if(!...)
Blead changes to ExtUtils::CBuilder
Probably unwanted behaviour of lexical filehandles

TODO of the week

Split "linker" from "compiler"

Patches of Interest

Safer environment iteration
Stop lib/ExtUtils/t/Embed.t from generating libperl.a
Add open |- and open -| to perlopentut
Clarify documentation on exists regarding autovivification
autodie 1.991 patch

Watching the smoke signals

Smoke [5.11.0] 34226 FAIL(F) MSWin32 WinXP/.Net SP3 (x86/2 cpu)

New and old bugs from RT

Class::Struct accessor overrides not called from constructor (#29230)
Repeated spaces on shebang line stops option parsing (#30660)
Loss of stack elements with a do block inside a return (#38809)
Removing deprecated features for 5.10? (#41480)
Subclassing CGI::Pretty dies in new (#41572)
Backwards logic in perluniintro (5.10.0) (#58218)
Speed lost on /^(foo|bar|baz)$/ match (#58280)
Unicode::UCD::charinfo() does not work on 21 Han codepoints (#58428)
Unicode::UCD::casefold() does not work as documented, nor probably as intended (#58430)
perl -d:DProf handles shift of @_ with & function wrong (#58446)
Crash on exit with fork done in do FILE on Win32 (#58468)
Assigning *a = \&a drops a reference (#58480)
Perl5 Bug Summary

In Brief

Last week's summary
About this summary

This Week on perl5-porters - 25-31 August 2008

"I must admit that I find Perl's Unicode support completely baffling. I've read the documentation multiple times, and while I can now predict when it won't work, I still have no idea how to get it to do what I want. It's kind of frustrating; I don't know if I'm just dim, or if I'm missing some magic bit of documentation, or if the support is just really confusing." -- Russ Allbery, shooting fish in a barrel.

Topics of Interest

Profiling parallel tests

On the 17th of August, Nicholas Clark kicked off a thread about applying Test::Harness's version 3 goodness to the task of running the core test suite, which offers a satisfying halving of the time taken.

One thing that surprised him was how much CPU the harness itself used: if that could be reduced, things would go even faster. Andy Armstrong thought that O(N�) algorithms might be extracting their dues from large numbers of test files.

Over the following couple of days, Aristotle Pagaltzis pondered various strategies for rewriting the hot spots. Steve Hay was dismayed to discover that no parallelism was available on the Windows platform, since the technique requires being able perform a select system call on a file handle, and Windows doesn't like doing things that way.

Wrong RE match in 5.10.0

Gisle Aas mentioned that a slightly impaired regular expression used to dump core on 5.8.8. These days, with an iterative engine under the hood of 5.10, it no longer dumps core... unfortunately, it returns the wrong result. Bram was able to simplify the test case to highlight what was happening.

Abigail showed that the problem was in fact that * (0 or many times) is implemented as "no more than 32766 times". This is horribly reminiscent of CP/M, and Dave Mitchell thought that it would be really nice if this limit could die a quick death. Nicholas Clark was unsure as to how that could be brought about.

In the meantime, it would be better if hitting this limit caused the engine to die, rather than merely warn (which you don't even hear about if warnings are switched off).

dprofpp enhanced

With all the cool kids playing around with Devel::NYTProf, it was good to see someone lavish some tender loving care on Devel::DProf. Which is what Daniel Pfeiffer did by adding some new functionality to dprofpp. No-one picked the patch up.

Regaining 'O's

H.Merijn Brand was saddened by smoke tests failing, simply because a UTF-8 locale was in use and Pod::Man was getting needlessly confused and thus marring an otherwise perfect run.

After a moment of confusion, Russ Allbery realised that the problem probably lay with Pod::Simple, and all that was needed was to add an =encoding directive to the POD test file. After some Unicode guidance from Juerd Waalboer, Russ saw the light and prepared a patch that solved the problems and looked sane at the same time.

Russ was a little scared by what the Correct Use of Unicode entailed in a program and thought it was incredibly complex. Juerd explained that there were three main techniques for dealing with Unicode, and showed where PHP, Perl 5 and Perl 6 fitted in the grand scheme.

Proposed pragma/module ensure

Chris Hall has written a pragmatic module currently named ensure that attempts to check for undefined subroutine. The denizens of comp.lang.perl.modules advised him to get in touch with the porters (on this list) for guidance and feedback on the idea. Unfortunately he received none.

Moving lib/ modules to ext/?

Nicholas Clark explained that he had not got very far in the process of moving core modules from /lib to /ext, and noted that it would probably take off once the move to git was complete.

When Jerry D. Hedden pointed out that non-XS modules get built incorrectly, Nicholas wondered if it were as simple as looking for XS files in the build directory and in the process discovered a sub-optimal File::Find construct that had been in the installperl Makefile target all the way back to change #18.

Nicholas went ahead and committed the fix that Jerry wrote to deal with the above problem, and in doing so, he came to the conclusion that it would probably fix a bug that no-one had reported yet.

unless(...) terser than if(!...)

Nicholas noticed that an unless($x) uses one less op than if(!$x), and so he asked whether it was possible to optimise the latter into the former (or, more precisely, where one would go about doing it).

Vincent Pit returned a day later with a proof of concept patch which did just that. As usual with Vincent's patches, after explaining the difficulty in addressing all that needs to be taken care of, one is surprised by how little C code is needed to solve the problem.

Graham Barr warned that overloading can cause unless to behave in a most curious manner, but Yitzchak Scott-Thoennes thought that it wouldn't be an issue in practice.

Blead changes to ExtUtils::CBuilder

Ken Williams wondered about the reasoning for a line that played around with name of the linker and compiler executables used to compile external C code. Reini Urban explained that Cygwin's implementation of EU::CB is derived from ExtUtils::CBuilder::Platform::Unix, and the patch is to undo the brain damage in the parent.

Ken was not in favour of adding a hack to undo a mistake, and wanted to fix the problem upstream. Nicholas Clark summarised the problem nicely, explaining that traditionally on Unix, the same front-end program can be used for the tasks of compiling and linking. Other platforms require one to use the linker program directly, and the perl configuration doesn't really capture this distinction nicely.

Probably unwanted behaviour of lexical filehandles

Dr. Ruud ran into two types of behaviour when opening a input pipe, depending on whether a package variable or a lexical variable was used. Dave Mitchell explained that the results are due to the fact that package variables aren't cleaned up during global destruction (unless you ask for it nicely), whereas lexical variables are always destructed.

TODO of the week

Split "linker" from "compiler"

There is an implicit historical assumption from around Perl5.000alpha something, that $cc is also the correct command for linking object files together to make an executable. This may be true on Unix, but it's not true on other platforms, and there are a maze of work arounds in other places (such as Makefile.SH) to cope with this.

Ideally, we should create a new variable to hold the name of the executable linker program, probe for it in Configure, and centralise all the special case logic there or in hints files.

A small bikeshed issue remains - what to call it, given that $ld is already taken (arguably for the wrong thing now, but on SunOS 4.1 it is the command for creating dynamically-loadable modules) and $link could be confused with the Unix command line executable of the same name, which does something completely different. Andy Dougherty makes the counter argument "In parrot, I tried to call the command used to link object files and libraries into an executable link, since that's what my vaguely-remembered DOS and VMS experience suggested. I don't think any real confusion has ensued, so it's probably a reasonable name for perl5 to use."

"Alas, I've always worried that introducing it would make things worse, since now the module building utilities would have to look for $Config{link} and institute a fall-back plan if it weren't found." Although I can see that as confusing, given that $Config{d_link} is true when (hard) links are available.

Patches of Interest

Safer environment iteration

Milosz Tanski wrote a patch to avoid race conditions when iterating %ENV in embedded interpreters in multi-threaded applications. Rafa�l Garcia-Suarez applied the patch.

Stop lib/ExtUtils/t/Embed.t from generating libperl.a

Jerry D. Hedden observed that a test file produced a bogus libperl.a file on Cygwin that would up being installed by accident. So he patched things to ensure that the test cleans up properly after itself. Reini Urban wondered how he had not managed to notice this in the past, and promised to take the code for a spin.

Add open |- and open -| to perlopentut

Shlomi Fish added some documentation for the magic files -| and |- that are used to open input and output pipes. Tom Christiansen wrote a eloquent response to the question that someone raised regarding singly- and doubly-quoted strings, arguing convincingly that one should always use double quoted strings, unless there was a very good reason not to.

One of the most important reasons is merely the visual cue: it's easier to see a double quote ("), rather than a single quote ('), where certain fonts may make it very difficult to discriminate between it and a back-tick.

Much discussion followed. No-one commented that at around 1000 lines long, perlopentut is no longer a tutorial, but more of a Compleat Reference, and thus probably needs less, rather than more, prose.

Clarify documentation on exists regarding autovivification

Moritz Lenz made a laudable effort at making the documentation clearer by removing a sentence, rather than adding one. And this was applied.

autodie 1.991 patch

Paul Fenwick announced the latest version of his autodie work. Nicholas Clark admitted that he was unlikely to review anything major until 5.8.9 goes out the door.

Watching the smoke signals

Smoke [5.11.0] 34226 FAIL(F) MSWin32 WinXP/.Net SP3 (x86/2 cpu)

Steve Hay's Windows smokes started going bad with a failure in op/local.t. This highlighted a problem with shell quote characters and test.pl's runperl function. Nicholas Clark was uncertain as to what it was that caused the failure to arise, but change #34228 got things back into line again.

New and old bugs from RT

Class::Struct accessor overrides not called from constructor (#29230)

Ren�e B�cker wrote a patch to address the problem which appears to have fallen through the cracks.

Repeated spaces on shebang line stops option parsing (#30660)

Ren�e also noticed that a shebang line containing extra spaces (such as "-s -w") will fail to notice the subsequent switches (-w in this case). After a couple of tweaks from H.Merijn Brand and Nicholas Clark, the code was adjusted to deal with this problem.

Loss of stack elements with a do block inside a return (#38809)

returning undef. He had proposed a patch some time ago, but after studying the emitted opcodes, came up with an improved patch to address the problem. And then came back the next day with an improved, improved patch, and some improved tests to go along with it.

Removing deprecated features for 5.10? (#41480)

Ren�e noted that Nicholas had noted that using arrays and hashes as references had been deprecated as far back as 5.8. It so happens that the warning for undesirable syntax is present in blead (which will one day become 5.12). The unanswered question is whether or not removing the code required that deals with it would simplify matters or not.

Subclassing CGI::Pretty dies in new (#41572)

Ren�e also had a patch for this. Nicholas forwarded it to Lincoln Stein, and would fold it back into the core once he got word back from Lincoln.

Backwards logic in perluniintro (5.10.0) (#58218)

Brad Baxter noted that the documentation that shows how to deal with the detection of invalid data for a given encoding had its logic backwards. Dr. Ruud suggested a couple of code variants that were an improvement on the existing logic, one of which Rafa�l Garcia-Suarez applied to the documentation.

Speed lost on /^(foo|bar|baz)$/ match (#58280)

Michael G. Schwern, following up on scurrilous Internet rumours, discovered that captured alternations are in fact slower in 5.10 than they were in 5.8. Steve Peters and Tels confirmed the finding. No word as to why this is the case.

Unicode::UCD::charinfo() does not work on 21 Han codepoints (#58428)

Karl Williamson discovered Unicode problems that he traced to magic numbers causing upper bounds to hide more recent Unicode additions. Ren�e created a patch to fix the problem. Karl also came back with a solution that worked with all Unicode database versions from 4.1 and beyond, and imagined that it would work for older versions as well.

Unicode::UCD::casefold() does not work as documented, nor probably as intended (#58430)

Similarly, Karl found a number of problems relating to the implementation for Unicode case folding, and saw that it was such a mess that it couldn't work at all. No-one stepped up to try and improve matters.

perl -d:DProf handles shift of @_ with & function wrong (#58446)

A person, whom some people call Tim, reported that Devel::DProf failed to deal correctly with @_ being aliased via magic &sub passing.

Crash on exit with fork done in do FILE on Win32 (#58468)

Alex Davies encountered a problem with a do 'file' after a program has forked on the Windows platform. No-one reported that they had confirmed the problem, much less propose a correction, even though Alex had gone to the trouble of figuring out why perl was crashing on a null pointer.

Assigning *a = \&a drops a reference (#58480)

Nicholas Clark noted that recent changes that had shifted code around now caused the assignment of something to one's own typeglob to leak a reference. He thought he knew how to fix the matter, and called for some TODO tests, if anyone was in the mood for volunteering.

Zefram thought that the code Nicholas had used to demonstrate the problem was incorrect.

Perl5 Bug Summary

In Brief

Nicholas had been observing an occasional assertion failure in S_sv_del_backref. After struggling for a while to reproduce it at all, Dave Mitchell nailed it with a tweak to a macro in sv.c.

Rick McGee reported running into grief attempting to marry 5.10 and FreeBSD 7.0. Alas, neither Nicholas Clark, nor Scott T. Hildreth nor Reini Urban were able to reproduce the problem, although Nicholas suspected it was probably a compiler issue.

Jerry D. Hedden announced that the TODO related to threads::shared is done, and so the item could be removed (if in fact the work he did on threads::shared was indeed what perltodo was referring to).

Vincent Pit ran into a precedence nit in Test::Builder (or more specifically !$x and $y, 'foo') which meant that a test that should have failed didn't.

Last week's summary

There wasn't one. I enjoyed a lazy summer doing home renovations, camping, hiking, swimming, drinking, eating and using computers as little as possible.

About this summary

If you found this summary useful, please consider contributing to the Perl Foundation or attending a YAPC to help support the development of Perl.

This Week on perl5-porters - 25-31 August 2008

Topics of Interest

Profiling parallel tests

Wrong RE match in 5.10.0

`dprofpp` enhanced

Regaining 'O's

Proposed pragma/module `ensure`

Moving lib/ modules to ext/?

`unless(...)` terser than `if(!...)`

Blead changes to `ExtUtils::CBuilder`

Probably unwanted behaviour of lexical filehandles

TODO of the week

Split "linker" from "compiler"

Patches of Interest

Safer environment iteration

Stop lib/ExtUtils/t/Embed.t from generating libperl.a

Add open `|-` and open `-|` to `perlopentut`

Clarify documentation on `exists` regarding autovivification

`autodie` 1.991 patch

Watching the smoke signals

Smoke [5.11.0] 34226 FAIL(F) MSWin32 WinXP/.Net SP3 (x86/2 cpu)

New and old bugs from RT

`Class::Struct` accessor overrides not called from constructor (#29230)

Repeated spaces on shebang line stops option parsing (#30660)

Loss of stack elements with a do block inside a return (#38809)

Removing deprecated features for 5.10? (#41480)

Subclassing `CGI::Pretty` dies in `new` (#41572)

Backwards logic in `perluniintro` (5.10.0) (#58218)

Speed lost on `/^(foo|bar|baz)$/` match (#58280)

`Unicode::UCD::charinfo()` does not work on 21 Han codepoints (#58428)

`Unicode::UCD::casefold()` does not work as documented, nor probably as intended (#58430)

`perl -d:DProf` handles shift of `@_` with & function wrong (#58446)

Crash on exit with fork done in do FILE on Win32 (#58468)

Assigning `*a = \&a` drops a reference (#58480)

Perl5 Bug Summary

In Brief

Last week's summary

About this summary