test-more / test2-harness Goto Github PK

View Code? Open in Web Editor NEW

23.0 23.0 26.0 63.49 MB

Alternative to Test::Harness

License: Other

Perl 99.41% C 0.01% Shell 0.01% Raku 0.55% Vim Script 0.01% C++ 0.01%

test2-harness's Introduction

NAME

Test2 - Framework for writing test tools that all work together.

DESCRIPTION

Test2 is a new testing framework produced by forking Test::Builder, completely refactoring it, adding many new features and capabilities.

WHAT IS NEW?

Easier to test new testing tools.

From the beginning Test2 was built with introspection capabilities. With Test::Builder it was difficult at best to capture test tool output for verification. Test2 Makes it easy with Test2::API::intercept().
Better diagnostics capabilities.

Test2 uses an Test2::API::Context object to track filename, line number, and tool details. This object greatly simplifies tracking for where errors should be reported.
Event driven.

Test2 based tools produce events which get passed through a processing system before being output by a formatter. This event system allows for rich plugin and extension support.
More complete API.

Test::Builder only provided a handful of methods for generating lines of TAP. Test2 took inventory of everything people were doing with Test::Builder that required hacking it up. Test2 made public API functions for nearly all the desired functionality people didn't previously have.
Support for output other than TAP.

Test::Builder assumed everything would end up as TAP. Test2 makes no such assumption. Test2 provides ways for you to specify alternative and custom formatters.
Subtest implementation is more sane.

The Test::Builder implementation of subtests was certifiably insane. Test2 uses a stacked event hub system that greatly improves how subtests are implemented.
Support for threading/forking.

Test2 support for forking and threading can be turned on using Test2::IPC. Once turned on threading and forking operate sanely and work as one would expect.

GETTING STARTED

If you are interested in writing tests using new tools then you should look at Test2::Suite. Test2::Suite is a separate cpan distribution that contains many tools implemented on Test2.

If you are interested in writing new tools you should take a look at Test2::API first.

NAMESPACE LAYOUT

This describes the namespace layout for the Test2 ecosystem. Not all the namespaces listed here are part of the Test2 distribution, some are implemented in Test2::Suite.

Test2::Tools::

This namespace is for sets of tools. Modules in this namespace should export tools like ok() and is(). Most things written for Test2 should go here. Modules in this namespace MUST NOT export subs from other tools. See the "Test2::Bundle::" namespace if you want to do that.

Test2::Plugin::

This namespace is for plugins. Plugins are modules that change or enhance the behavior of Test2. An example of a plugin is a module that sets the encoding to utf8 globally. Another example is a module that causes a bail-out event after the first test failure.

Test2::Bundle::

This namespace is for bundles of tools and plugins. Loading one of these may load multiple tools and plugins. Modules in this namespace should not implement tools directly. In general modules in this namespace should load tools and plugins, then re-export things into the consumers namespace.

Test2::Require::

This namespace is for modules that cause a test to be skipped when conditions do not allow it to run. Examples would be modules that skip the test on older perls, or when non-essential modules have not been installed.

Test2::Formatter::

Formatters live under this namespace. Test2::Formatter::TAP is the only formatter currently. It is acceptable for third party distributions to create new formatters under this namespace.

Test2::Event::

Events live under this namespace. It is considered acceptable for third party distributions to add new event types in this namespace.

Test2::Hub::

Hub subclasses (and some hub utility objects) live under this namespace. It is perfectly reasonable for third party distributions to add new hub subclasses in this namespace.

Test2::IPC::

The IPC subsystem lives in this namespace. There are not many good reasons to add anything to this namespace, with exception of IPC drivers.

Test2::IPC::Driver::

IPC drivers live in this namespace. It is fine to create new IPC drivers and to put them in this namespace.

Test2::Util::

This namespace is for general utilities used by testing tools. Please be considerate when adding new modules to this namespace.

Test2::API::

This is for Test2 API and related packages.

Test2::

The Test2:: namespace is intended for extensions and frameworks. Tools, Plugins, etc should not go directly into this namespace. However extensions that are used to build tools and plugins may go here.

In short: If the module exports anything that should be run directly by a test script it should probably NOT go directly into Test2::XXX.

CONTACTING US

Many Test2 developers and users lurk on irc://irc.perl.org/#perl-qa and irc://irc.perl.org/#toolchain. We also have a slack team that can be joined by anyone with an @cpan.org email address https://perl-test2.slack.com/ If you do not have an @cpan.org email you can ask for a slack invite by emailing Chad Granum [email protected].

SOURCE

The source code repository for Test2 can be found at https://github.com/Test-More/test-more/.

MAINTAINERS

Chad Granum [email protected]

AUTHORS

Chad Granum [email protected]

COPYRIGHT

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See https://dev.perl.org/licenses/

test2-harness's People

Contributors

Stargazers

Watchers

test2-harness's Issues

Tests fail for older perls (< 5.16) (0.001030)

The following tests fail with 5.8.9 and 5.10.1:

( STDERR )  job  1    Got exception as expected, but exception is falsy (undef, '', or 0)... at t/App/Yath.t line 41.
( STDERR )  job  1    Got exception as expected, but exception is falsy (undef, '', or 0)... at t/App/Yath.t line 48.
[  FAIL  ]  job  1  +~load_command
[  FAIL  ]  job  1    + Exception if the command is not valid
(  DIAG  )  job  1    | Failed test 'Exception if the command is not valid'
(  DIAG  )  job  1    | at t/App/Yath.t line 41.
(  DIAG  )  job  1    | +-----+----+------------------------------------------------------------+
(  DIAG  )  job  1    | | GOT | OP | CHECK                                                      |
(  DIAG  )  job  1    | +-----+----+------------------------------------------------------------+
(  DIAG  )  job  1    | |     | eq | yath command 'a_fake_command' not found. (did you forget t |
(  DIAG  )  job  1    | |     |    | o install App::Yath::Command::a_fake_command?)\n           |
(  DIAG  )  job  1    | +-----+----+------------------------------------------------------------+
[  FAIL  ]  job  1    + Exception is propogated if command dies on compile
(  DIAG  )  job  1    | Failed test 'Exception is propogated if command dies on compile'
(  DIAG  )  job  1    | at t/App/Yath.t line 48.
(  DIAG  )  job  1    | +-----+----+-------------------------------------+
(  DIAG  )  job  1    | | GOT | OP | CHECK                               |
(  DIAG  )  job  1    | +-----+----+-------------------------------------+
(  DIAG  )  job  1    | |     | =~ | (?-xism:This command is broken! at) |
(  DIAG  )  job  1    | +-----+----+-------------------------------------+
            job  1    ^
(  DIAG  )  job  1    Failed test 'load_command'
(  DIAG  )  job  1    at t/App/Yath.t line 51.
(  DIAG  )  job  1    Seeded srand with seed '20171101' from local date.
(  DIAG  )  job  1    Looks like you failed 1 test of 6.
( FAILED )  job  1    t/App/Yath.t
( REASON )  job  1    Test script returned error (Code: 1)
( REASON )  job  1    Assertion failures were encountered (Count: 1)
( REASON )  job  1    Subtest failures were encountered (Count: 1)
...
( STDERR )  job 51    Can't locate object method "autoflush" via package "IO::Handle" at /usr/home/eserte/.cpan/build/2017110121/Test2-Harness-0.001030-2/lib/Test2/Formatter/Stream.pm line 83.
( STDERR )  job 51    BEGIN failed--compilation aborted at /usr/perl5.8.9/lib/5.8.9/Test/More.pm line 22.
( STDERR )  job 51    Compilation failed in require at /usr/perl5.8.9/lib/5.8.9/ok.pm line 5.
( STDERR )  job 51    BEGIN failed--compilation aborted at /usr/perl5.8.9/lib/5.8.9/ok.pm line 5.
( STDERR )  job 51    Compilation failed in require at t/Test2/Harness/Util/DepTracer.t line 4.
( STDERR )  job 51    BEGIN failed--compilation aborted at t/Test2/Harness/Util/DepTracer.t line 4.
( FAILED )  job 51    t/Test2/Harness/Util/DepTracer.t
( REASON )  job 51    Test script returned error (Code: 255)
( REASON )  job 51    No plan was declared, and no assertions were made.
...
The following test jobs failed:
  [1] t/App/Yath.t
  [51] t/Test2/Harness/Util/DepTracer.t

The t/App/Yath.t failure happens also with 5.12.4.

With perl 5.14.x:

# Failed test 'Did not track from version import'
# at t/Test2/Harness/Util/DepTracer.t line 46.
# +--------------+-----+---------+------------------+
# | PATH         | GOT | OP      | CHECK            |
# +--------------+-----+---------+------------------+
# | {feature.pm} | 1   | !exists | <DOES NOT EXIST> |
# +--------------+-----+---------+------------------+
# Seeded srand with seed '20171101' from local date.
# Looks like you failed 1 test of 9.
t/Test2/Harness/Util/DepTracer.t ....................... 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests

Test2::Harness shows attach/detach events

Test2::Workflow uses Test2::AsyncSubtest, and the attach/detach events are showing up in Test2::Harness.

Can't locate object method "pfile_name"

❯ perl -v
This is perl 5, version 27, subversion 4 (v5.27.4 (v5.27.3-100-g07d51b5358)) built for darwin-2level

❯ perl -MTest2::Harness -E 'say Test2::Harness->VERSION'
0.001015

❯ yath run
Can't locate object method "pfile_name" via package "App::Yath::Command::run" at /Users/skaji/env/plenv/versions/09.16-v5.27.3-100/lib/perl5/site_perl/5.27.4/App/Yath/Command/run.pm line 41.

❯ yath reload
Can't locate object method "pfile_name" via package "App::Yath::Command::reload" at /Users/skaji/env/plenv/versions/09.16-v5.27.3-100/lib/perl5/site_perl/5.27.4/App/Yath/Command/reload.pm line 47.

yath always includes the EventStream renderer unless -q is set

Yath should not automatically include the EventStream render if you specify an alternative renderer.

Use of uninitialized value $ROOT_PID in numeric ne (!=)

Discover this issue while running our test suite using yath

( STDERR )  job 1407    ----- START -----
Use of uninitialized value $ROOT_PID in numeric ne (!=) at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test2/Formatter/Stream.pm line 81.
new_root called from child process! at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test2/Formatter/Stream.pm line 81.
	Test2::Formatter::Stream::new_root("Test2::Formatter::Stream") called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test2/API/Stack.pm line 31
	Test2::API::Stack::new_hub(Test2::API::Stack=ARRAY(0x1571fb8)) called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test2/API/Stack.pm line 47
	Test2::API::Stack::top(Test2::API::Stack=ARRAY(0x1571fb8)) called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test2/API.pm line 325
	Test2::API::context("level", 2, "fudge", 1, "stack", Test2::API::Stack=ARRAY(0x1571fb8), "hub", undef, ...) called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test/Builder.pm line 158
	Test::Builder::ctx(Test::Builder=HASH(0x13287c0)) called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test/Builder.pm line 410
	Test::Builder::reset(Test::Builder=HASH(0x13287c0), "singleton", 1) called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test/Builder.pm line 111
	Test::Builder::__ANON__() called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test2/API/Instance.pm line 319
	Test2::API::Instance::load(Test2::API::Instance=HASH(0x1403ec8)) called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test2/API.pm line 168
	Test2::API::test2_load() called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test/Builder/Module.pm line 78
	Test::Builder::Module::import("Test::Builder::Module") called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test/More.pm line 22
	Test::More::BEGIN() called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Test/More.pm line 22

Heisenbug in t/use_harness.t

It has failed at least once in Travis for no obvious reason. Rerunning the build with no changes caused it to pass. Unfortunately that also seems to have replaced the failing build log with the one that passed.

yath -S not quite right

yath -j9 -Iperl -S "-MDevel::Cover=+ignore,\.t$,+ignore,test\.pl$,+ignore,^t/" t/

Can't open perl script "+ignore,\.t$,+ignore,test\.pl$,+ignore,^t/": No such file or directory

Test failure with StrawberryPerl 5.26.0.x

Output from 'C:\STRAWB~2\c\bin\gmake.exe test':

"C:\strawberry\perl\bin\perl.exe" "-Iblib\lib" "-Iblib\arch" test.pl
No such signal: SIGUSR1 at lib/Test2/Harness/Util/Debug.pm line 27.
C:\STRAWB~2\cpan\build\2017110121\.yath.rc: No such file or directory at lib/App/Yath/Util.pm line 44.
BEGIN failed--compilation aborted at ./scripts/yath line 4.
yath exited with 5121..1
not ok 1 - Passed tests when run by yath
gmake: *** [Makefile:1017: test_dynamic] Error 255

Test passes under prove but fails with yath

As of 0b037dca3c34d195225998f71baa14d649b30142 in the Test-Class-Moose repo, running t/process_name.t under prove passes, but fails with yath. If I insert a # HARNESS-NO-STREAM directive then it will pass under yath too. When running yath -v I get a number of warnings of the form Use of uninitialized value in print at /home/autarch/.perlbrew/libs/perl-5.26.1@dev/lib/perl5/Test2/Formatter/Test2.pm line 270. The warnings only happen when the no stream directive is enabled.

todo subtest fails confuse event stream end detection

Example test script:

use Test::More;
$TODO++;
subtest a => sub { fail; pass; fail; pass; };
done_testing;

Shell output:

marketequalizer:~# yath test.pl
[LAUNCH]  _  Started process with test.pl
[PARSER]  +  Process has exited but the event stream does not appear complete. Waiting 60 seconds...
[PARSER]  +  Event received, timeout reset.
[PASSED]  =  test.pl passed

=== ALL TESTS SUCCEEDED ===

Can't correctly preload Devel::Assert::Global with yath

I use assertions from Devel::Assert and want to enable them during tests.

If I run something like

perl -Ilocal/lib/perl5 -Ilib -MDevel::Assert::Global t/path/to/test.t

assertions works as expectable, tests failed with message Assertions failed

But if i run something like

yath -l -Ilocal/lib/perl5 -It -PDevel::Assert::Global t/path/to/test.t

assertions don't work, tests pass.

Maybe I incorrectly understand preload feature of Yath or doing something else wrong?
Is there a way to achieve my goal to run test2 tests with yath and enabled assertions?

Can leave the terminal in a weird state

If I hit ctrl-C at the wrong time while the output is happening, it can leave my terminal in a state where I can't see anything that I type. I've seen this with other apps that use the ANSI formatting codes too.

I think one possible fix is to add a DESTROY in the EventStream formatter that unconditionally resets the terminal (however one does that).

Need a way to specify terminal width when yath times is run from a script

Accidentally opened in Test-More/Test2-Suite#161

We've got code reporting yath times via email. The code is invoking yath times and is presumably not able to transmit terminal width. When it does so, the width of the boxes is short. Ideally this would be controllable from command line but possibly the fix is to default the width to something larger? At the moment, it appears to default to 72 as the terminal width.

yath fails with local $, = "|"

This causes Text::CSV_XS to fail on "yath test"

use Test::More;
local $, = "|";
ok (1, "hi");
done_testing;

$ prove t.pl
t.pl .. ok
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.04 usr  0.01 sys +  0.12 cusr  0.01 csys =  0.18 CPU)
Result: PASS

$ yath t.pl

** Defaulting to the 'test' command **

( FAILED )  job  1    t.pl
<JOB POLL>  job  1    ----- START -----
JSON decode error: garbage after JSON object, at character offset 748 (before "|\n") at /pro/lib/perl5/site_perl/5.26.0/Test2/Harness/Util/JSON.pm line 35, <$fh> line 4.
{"stamp":1521473038.800453,"facet_data":{"meta":{"Test::Builder":{"actual_ok":1,"type":"","ok":1,"reason":"","name":"hi"}},"trace":{"cid":"C3","nested":0,"uuid":"8BA29BBA-2B89-11E8-8A72-1BECC1EFA43E","hid":"32602~0~2","pid":32602,"frame":["main","t.pl",3,"Test::More::ok"],"tid":0,"huuid":"8BA22A18-2B89-11E8-8A72-1BECC1EFA43E","buffered":0},"control":{},"assert":{"no_debug":1,"details":"hi","pass":1},"hubs":[{"hid":"32602~0~2","pid":32602,"tid":0,"buffered":0,"details":"Test2::Hub","uuid":"8BA22A18-2B89-11E8-8A72-1BECC1EFA43E","nested":0,"ipc":0}],"about":{"package":"Test2::Event::Ok","uuid":"8BA29D2C-2B89-11E8-8A72-1BECC1EFA43E"}},"stream_id":1,"assert_count":1,"event_id":"8BA29D2C-2B89-11E8-8A72-1BECC1EFA43E","times":[0.01,0.01,0.0,0.0]}|

<JOB POLL>  job  1    ------ END ------
< REASON >  job  1    Errors were encountered (Count: 1)
< REASON >  job  1    ----- START -----
No plan was declared, and no assertions were made.
< REASON >  job  1    ------ END ------

================================================================================

Run ID: 8B72200C-2B89-11E8-BCEA-1AECC1EFA43E

The following test jobs failed:
  [8B887D3E-2B89-11E8-BCEA-1AECC1EFA43E] 1: t.pl


-0.48310s on wallclock (0.22 usr 0.02 sys + 0.16 cusr 0.03 csys = 0.43 CPU)

FWIW local $, set to "*" or "," also causes trouble

honor HARNESS_PERL_SWITCHES

People are used to using this with prove for things like Devel::Cover, we should honor it.

the [SKIP!!] non-verbose line doesn't give reason

Example! With prove:

~$ prove skip-all.pl
skip-all.pl .. skipped: meh
Files=1, Tests=0,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.05 cusr  0.01 csys =  0.09 CPU)
Result: NOTESTS

With yath:

~$ yath skip-all.pl
[SKIP!!]  =  skip-all.pl

=== ALL TESTS SUCCEEDED ===

Also, note the distinction between NOTESTS and ALL TESTS SUCCEEDED.

puts lib and blib directories in at-INC after already installed libs

My tests were failing because they were using features from my module that had been added in git, but the version installed was older. Consider:

use Test2::V0;

ok 1;

diag "INC:$_" for @INC;

done_testing;

Output:

crux% yath --lib t/foo.t 

** Defaulting to the 'test' command **

(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5/darwin-2level
(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5
(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5/darwin-2level
(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5
(  DIAG  )  job  1    INC:/Users/ollisg/perl5/perlbrew/perls/perl-5.26.0/lib/site_perl/5.26.0/darwin-2level
(  DIAG  )  job  1    INC:/Users/ollisg/perl5/perlbrew/perls/perl-5.26.0/lib/site_perl/5.26.0
(  DIAG  )  job  1    INC:/Users/ollisg/perl5/perlbrew/perls/perl-5.26.0/lib/5.26.0/darwin-2level
(  DIAG  )  job  1    INC:/Users/ollisg/perl5/perlbrew/perls/perl-5.26.0/lib/5.26.0
(  DIAG  )  job  1    INC:.
(  DIAG  )  job  1    INC:/Users/ollisg/dev/Alien-Build/Alien-Build-1.15_01/lib
(  DIAG  )  job  1    INC:/Users/ollisg/dev/Alien-Build/Alien-Build-1.15_01/blib/lib
(  DIAG  )  job  1    INC:/Users/ollisg/dev/Alien-Build/Alien-Build-1.15_01/blib/arch
(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5
( PASSED )  job  1    t/foo.t

================================================================================

Run ID: 1505482457

All tests were successful!

0.25390s on wallclock (0.12 usr 0.04 sys + 0.15 cusr 0.03 csys = 0.34 CPU)

crux% yath --blib t/foo.t

** Defaulting to the 'test' command **

(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5/darwin-2level
(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5
(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5/darwin-2level
(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5
(  DIAG  )  job  1    INC:/Users/ollisg/perl5/perlbrew/perls/perl-5.26.0/lib/site_perl/5.26.0/darwin-2level
(  DIAG  )  job  1    INC:/Users/ollisg/perl5/perlbrew/perls/perl-5.26.0/lib/site_perl/5.26.0
(  DIAG  )  job  1    INC:/Users/ollisg/perl5/perlbrew/perls/perl-5.26.0/lib/5.26.0/darwin-2level
(  DIAG  )  job  1    INC:/Users/ollisg/perl5/perlbrew/perls/perl-5.26.0/lib/5.26.0
(  DIAG  )  job  1    INC:.
(  DIAG  )  job  1    INC:/Users/ollisg/dev/Alien-Build/Alien-Build-1.15_01/lib
(  DIAG  )  job  1    INC:/Users/ollisg/dev/Alien-Build/Alien-Build-1.15_01/blib/lib
(  DIAG  )  job  1    INC:/Users/ollisg/dev/Alien-Build/Alien-Build-1.15_01/blib/arch
(  DIAG  )  job  1    INC:/Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5
( PASSED )  job  1    t/foo.t

================================================================================

Run ID: 1505482463

All tests were successful!

0.27038s on wallclock (0.12 usr 0.03 sys + 0.15 cusr 0.04 csys = 0.34 CPU)

crux%

yath injects relative lib and blib directories

I'm using Test2-Harness-0.001002.

yath injects relative lib and blib directories, while prove injects absolute lib and blib directories.
Because now Perl thinks the relative . in @INC is a vulnerability, so we may want to use absolute paths for yath too.

What do you think?

(FYI I found this difference in this test).

example

test.t

use strict;
use Test::More tests => 1;
diag $ENV{PERL5LIB};
pass;

prove

❯ prove -lb test.t
test.t .. # /Users/skaji/lib:/Users/skaji/blib/lib:/Users/skaji/blib/arch
test.t .. ok

yath

❯ yath test.t
(  DIAG  )  job 01    lib:blib/lib:blib/arch
( PASSED )  job 01    test.t

Preserve prove -f behavior for STDIN

Right now this code behaves differently under yath.

#!perl
use Test::More tests => 1;
ok(!-f STDIN, "stdin");

It doesn't have to (at least under UNIX)
this will preserve the behavior and then you don't need to open a temp file for STDIN

open(STDIN, '<', '/dev/null');

`yath stop` output too verbose

It seems like yath stop outputs a lot to the console that we've already seen (maybe the whole log?). If you've left the persistent runner alive for a while, this can equal lots of time printing to the console (last time for me was tens of seconds). Perhaps it could just point to where the log is stored?

Verbose should disable qvf

App/Yath/Util.t test fails on macOS

macOS has a weird approach of symlinking /var to /private/var. This causes the following test failure:

[  FAIL  ]  job 17  + Got config for foo command
(  DIAG  )  job 17    Failed test 'Got config for foo command'
(  DIAG  )  job 17    at t/App/Yath/Util.t line 100.
(  DIAG  )  job 17    +------+-----------------------------+----+-----------------------------+
(  DIAG  )  job 17    | PATH | GOT                         | OP | CHECK                       |
(  DIAG  )  job 17    +------+-----------------------------+----+-----------------------------+
(  DIAG  )  job 17    | [8]  | /private/var/folders/y0/y7f | eq | /var/folders/y0/y7fvcmnx747 |
(  DIAG  )  job 17    |      | vcmnx7477qf4wfnqgltfc0000gn |    | 7qf4wfnqgltfc0000gn/T/yath- |
(  DIAG  )  job 17    |      | /T/yath-test-31674-65W54aer |    | test-31674-65W54aer/50FACA5 |
(  DIAG  )  job 17    |      | /50FACA5E-4692-11E8-A3E0-86 |    | E-4692-11E8-A3E0-8674A99856 |
(  DIAG  )  job 17    |      | 74A998566E/tmp/PpKI_KLHqz/x |    | 6E/tmp/PpKI_KLHqz/x/y/z     |
(  DIAG  )  job 17    |      | /y/z                        |    |                             |
(  DIAG  )  job 17    +------+-----------------------------+----+-----------------------------+
(  DIAG  )  job 17    Seeded srand with seed '20180422' from local date.
(  DIAG  )  job 17    Looks like you failed 1 test of 15.
( FAILED )  job 17    t/App/Yath/Util.t
< REASON >  job 17    Test script returned error (Code: 1)
< REASON >  job 17    Assertion failures were encountered (Count: 1)

Feature request: use a symlink last to point to the last run yath

When running yath in parallel, at the end it leaves a .jsonl.bz2 file behind with a pretty long name,
would be cool to use a symlink last to the last run one
so replay could then be simpler

yath replay test-logs/last 1234

or even

yath last-replay 1234

EventStream color scheme does not work well on light backgrounds

I'm not sure if this can be auto-detected. If not, I think the easiest solution for now is to simply offer some sort of --color-scheme lightbg flag that doesn't use things like light gray and light yellow.

yath replay -f

It would be nice if yath replay could be made to tail an existing output file and continue reading it with the assumption that if it gets parial data, it should back up and attempt to read more data (a completed json blob) from the file before continuing.

Essentially I want something like this. I think the critical feature yath lacks is the willingness to wait for more output and not halt at EOF?

bzcat job.jsonl.bz2 | yath replay -f

No utf8 support for output

If a test prints utf8 the output gets scrambled. Use of Test2::Plugin::UTF8 doesn't help.

Test suite may hang

I observe a hanging Test2-Harness test suite on various systems (CentOS6, Debian, FreeBSD). It seems that all Test2-Harness versions are affected (seen with 0.000001, 0.000003, 0.001021 .. 0.001028).

The last log lines of such a hanging test suite on a CentOS6 system look like this:

( PASSED )  job 27849-63    t/failure_cases.t
(INTERNAL)     open3: close(9) failed: Bad file descriptor at /home/cpansand/.cpan/build/2017103115/Test2-Harness-0.001028-0/lib/Test2/Harness/Job/Runner/Open3.pm line 74.
( FAILED )  job 27849-67    t2/ending.t
( REASON )  job 27849-67    Test script returned error (Signal: 15)
( REASON )  job 27849-67    No plan was declared, and no assertions were made.
(INTERNAL)     Parent has vanished, exiting. at /home/cpansand/.cpan/build/2017103115/Test2-Harness-0.001028-0/lib/Test2/Harness/Run/Runner/ProcMan/Persist.pm line 121, <$fh> line 133.

On a FreeBSD9 system it looks like this:

( SKIPPED)  job 92229-41    t/Test2/Harness/Run/Runner.t  -  Not done, come back!
( SKIPPED)  job 92229-42    t/Test2/Harness/Run/Runner/Persist.t  -  TODO
( SKIPPED)  job 92229-43    t/Test2/Harness/Run/Runner/ProcMan.t  -  TODO
( SKIPPED)  job 92229-44    t/Test2/Harness/Run/Runner/ProcMan/Persist.t  -  TODO
( SKIPPED)  job 92229-45    t/Test2/Harness/Run/Runner/ProcMan/Scheduler.t  -  TODO
( SKIPPED)  job 92229-46    t/Test2/Harness/Run/Runner/ProcMan/Scheduler/Fair.t  -  TODO
( PASSED )  job 92229-63    t/failure_cases.t
( SKIPPED)  job 92229-47    t/Test2/Harness/Run/Runner/ProcMan/Scheduler/Finite.t  -  TODO
(INTERNAL)     open3: close(8) failed: Ungültiger Dateideskriptor at /usr/home/eserte/.cpan/build/2017103118/Test2-Harness-0.001028-1/lib/Test2/Harness/Job/Runner/Open3.pm line 74.
(INTERNAL)     Parent has vanished, exiting. at /usr/home/eserte/.cpan/build/2017103118/Test2-Harness-0.001028-1/lib/Test2/Harness/Run/Runner/ProcMan/Persist.pm line 121, <$fh> line 97.

Consider switching away from JSON

JSON is nice, but JSON encoders simply fail if they encounter an object that lacks the TO_JSON method. This is a problem for event meta-data, and for events that may have arbitrary objects, such as Test2::Event::Exception (Exceptions can be blessed!)

Test2::Harness should really use a serializer that understands perl. Events already work with Storable, that is how the IPC system does it.

I think Test2::Harness should switch to a formatter that sends events over STDOUT using Storable. It will also need to be sure to prevent any other output from hitting the real STDOUT.

Here is an example script that shows how to use storable over a filehandle like this, the Storable docs are not very complete....

use strict;
use warnings;

use Storable qw/nstore_fd fd_retrieve/;
use Time::HiRes qw/sleep/;
use IO::Handle;

my ($rh, $wh);
pipe($rh, $wh) or die "No pipe!";

my $pid = fork;
die "Could not fork" unless defined $pid;

if ($pid) {
    close($rh);
    sender($wh);
}
else {
    close($wh);
    reader($rh);
}

sub sender {
    my $wh = shift;

    for (1 .. 100) {
        print "Sending: $_\n";
        nstore_fd({num => $_}, $wh) or die "oops";
        $wh->flush;
        sleep 0.02;
    }   

    exit 0;
}

sub reader {
    my $rh = shift;

    while(1) {
        if(my $hashref = fd_retrieve($rh)) {
            print "Got " . $hashref->{num} . "\n";
            exit 0 if $hashref->{num} == 100;
        }   
    }   
}

t2/subtests_buffered.t fails in replay

When replaying the test log of yath itself, the t2/subtests_buffered.t test fails with a parsing error (missing closing brace on the buffered subtest)

This is probably an error with the log, but may require some futzing about with the parser/watcher in replay mode.

Fails on test without trailing ;

Consider the test:

use Test2::V0;

ok 1;

done_testing

This has always worked with prove, with yath:

yath -v t/foo.t

** Defaulting to the 'test' command **

( LAUNCH )  job  1    t/foo.t
(  NOTE  )  job  1    Seeded srand with seed '20170915' from local date.
[  PASS  ]  job  1  + <UNNAMED ASSERTION>
( STDERR )  job  1    App::Yath::Command::spawn is not intended to be instanciated at /Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5/App/Yath/Command/spawn.pm line 28, <$fh> line 5.
( STDERR )  job  1    	App::Yath::Command::spawn::init(App::Yath::Command::spawn=HASH(0x7fae32e4dc80)) called at /Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5/Test2/Harness/Util/HashBase.pm line 141
( STDERR )  job  1    	Test2::Harness::Util::HashBase::_new("App::Yath::Command::spawn", "args", ARRAY(0x7fae31002e50)) called at /Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5/App/Yath.pm line 36
( STDERR )  job  1    	App::Yath::run_command("App::Yath", "App::Yath::Command::spawn", "spawn", ARRAY(0x7fae31002e50)) called at /Users/ollisg/.perlbrew/libs/perl-5.26.0@dev/lib/perl5/App/Yath.pm line 24
( STDERR )  job  1    	App::Yath::__ANON__() called at t/foo.t line 5
(  DIAG  )  job  1    Tests were run but no plan was declared and done_testing() was not seen.
(  DIAG  )  job  1    Looks like your test exited with 255 after test #1.
( FAILED )  job  1    t/foo.t
( REASON )  job  1    Test script returned error (Code: 255)
( REASON )  job  1    No plan was declared

================================================================================

Run ID: 1505481396

The following test jobs failed:
  [1] t/foo.t

0.28497s on wallclock (0.12 usr 0.03 sys + 0.17 cusr 0.04 csys = 0.36 CPU)

Adding a ; to the end of the test works around this.

yath runs command as test if file of same name exists

For instance:

yath times ...

This normally runs the times command. However if there is a file called 'times' it will try to run that as a test.

yath times ... > times

ooops.

Stderr output ends up interleaved with test output in confusing way

Here's an example:

[LAUNCH]  _  t/basic.t
[PARSER]  +  Test2::Harness::Parser::EventStream
[STDERR]  +  Error response:
[ NOTE ]  +  Seeded srand with seed '20160525' from local date.
[STDERR]  +  
[ENCODE]  +  utf8
[STDERR]  +  404 Not Found
[  OK  ]  +  made a new WebService::PivotalTracker object
[STDERR]  +  Client-Date: Wed, 25 May 2016 16:23:11 GMT
[  OK  ]  +  WebService::PivotalTracker::Client=HASH->isa('WebService::PivotalTracker::Client')
[STDERR]  +  
[ DIAG ]  +  Tests were run but no plan was declared and done_testing() was not seen.
[STDERR]  +  
[ DIAG ]  +  Looks like your test exited with 2 after test #2.
[STDERR]  +  For the request:
[STDERR]  +  
[STDERR]  +  GET https://www.pivotaltracker.com/services/v5//stories/120292647
[STDERR]  +  Content-Type: application/json
[STDERR]  +  X-TrackerToken: e99a18c428cb38d5f260853678922e03
[STDERR]  +  
[ PLAN ]  =  No plan was ever set.
[FAILED]  =  t/basic.t

=== FAILURE SUMMARY ===
 * t/basic.t

All of the STDERR output is a single die message. It'd be nice to see it all together.

I have no idea how to fix this, however ;)

provide a way to provide only the top XXX yath times.

We have a test suite with 4000 test files. Right now we're running yath times and then stripping all but the top 100 longest tests. It might be generally useful to make this a command line argument to pass this to yath times.

yath times --max-tests 100

and/or

yath times --min-time 4 (seconds)

--qvf should not hide internal messages

Tests hang on perl 5.8.9 on Linux

Failure appears to be when looking for .yath.rc, it keeps recursing forever up a directory, and another, and another, and so on. I attach a log that I cut off after a few goes around the loop, but well after it had started to try to recurse into /'s parent directory.
test2-harness-on-5.8.9-output.log

This is testing against a fairly "vanilla" 5.8.9, with the only non-core modules in it being those required for running CPAN::Reporter. Given that on the same machine the tests pass on 5.10.1, I assume that you depend on a later version of some core module.

yath warns of "Filehandle STDOUT reopened as $fh only for input"

% yath test --version

Yath version: 0.001062

Other version info:
+----------------+----------+
| COMPONENT      | VERSION  |
+----------------+----------+
| perl           | v5.22.1  |
| App::Yath      | 0.001062 |
| Test2::Harness | 0.001062 |
| Test2::Suite   | 0.000111 |
| Test2::API     | 1.302133 |
| Test::Builder  | 1.302133 |
+----------------+----------+

With the following code:

use Test2::V0;

{
    package Foo;

    sub new { bless { fh => \*STDOUT }, shift() }
    sub DESTROY { shift->{fh}->close }
}

ok( Foo->new );
done_testing;

yath gives the following warning:

% yath test tst.t
( STDERR )  job  1    ----- START -----
Filehandle STDOUT reopened as $fh only for input at /proj/axaf/ots/pkgs/perl-5.22/x86_64-linux_debian-9/lib/site_perl/5.22.1/Test2/Harness/Util.pm line 70.
( STDERR )  job  1    ------ END ------
( PASSED )  job  1    tst.t

================================================================================

Run ID: D88D9ED6-2D4E-11E8-A18B-6E1FDEDFE4DF

All tests were successful!


0.13295s on wallclock (0.37 usr 0.06 sys + 0.50 cusr 0.09 csys = 1.02 CPU)

no documentation in yath executable

% perldoc yath
No documentation found for "yath".

I realize that there is useful information in App::Yath, but that includes information about the app API, which is not relevant to the command line, and the executable is usually the first place at least some people are going to look.

Add timeout option

Add option where harness will kill a test that has produced no output by the specified timeout. This way the harness will move on if a test enters an infinite loop.

`perldoc yath` or `man yath` doesn't show documentation

Mark@travis:~$ man yath
No manual entry for yath
Mark@travis:~$ perldoc yath
No documentation found for "yath".

I think this is because the yath script hasn't got any pod in it.

preload does not set environment properly

xxx.t:

use Test2::Bundle::Extended;
is($ENV{HARNESS_JOBS}, 9, "Set HARNESS_JOBS");
done_testing;

Works fine with just -j9

$ yath -j9 -v xxx.t
[LAUNCH]  _  xxx.t
[PARSER]  +  Test2::Harness::Parser::EventStream
[ NOTE ]  +  Seeded srand with seed '20160713' from local date.
[ENCODE]  +  utf8
[  OK  ]  +  Set HARNESS_JOBS
[ PLAN ]  +  Plan is 1 assertions
[PASSED]  =  xxx.t

Breaks with preload:

$ yath -j9 -LCarp -v xxx.t
[LAUNCH]  _  xxx.t
[STDOUT]  +  # Seeded srand with seed '20160713' from local date.                                                                                                               
[PARSER]  +  Test2::Harness::Parser::TAP
[NOT OK]  +  Set HARNESS_JOBS
[ DIAG ]  +  Failed test 'Set HARNESS_JOBS'
[ DIAG ]  +  at xxx.t line 3.
[ PLAN ]  +  Plan is 1 assertions
[ DIAG ]  +  +---------+-------+
[ DIAG ]  +  | GOT     | CHECK |
[ DIAG ]  +  +---------+-------+
[ DIAG ]  +  | <UNDEF> | 9     |
[ DIAG ]  +  +---------+-------+
[FAILED]  =  xxx.t

=== FAILURE SUMMARY ===
 * xxx.t

Can't locate object method "send_ev2" via package "Test2::API::Context"

The test suite fails, probably if not test latest Test2 is installed:

PERL_DL_NONLAZY=1 "/opt/perl-5.20.3/bin/perl" "-Iblib/lib" "-Iblib/arch" test.pl
( STDERR )  job  1    Can't locate object method "send_ev2" via package "Test2::API::Context" at /home/cpansand/.cpan/build/2018031121/Test2-Harness-0.001058-0/lib/Test2/Plugin/MemUsage.pm line 31.
( STDERR )  job  1    END failed--call queue aborted.
( FAILED )  job  1    t/App/Yath.t
< REASON >  job  1    Test script returned error (Code: 255)
( STDERR )  job  2    Can't locate object method "send_ev2" via package "Test2::API::Context" at /home/cpansand/.cpan/build/2018031121/Test2-Harness-0.001058-0/lib/Test2/Plugin/MemUsage.pm line 31.
( STDERR )  job  2    END failed--call queue aborted.
( FAILED )  job  2    t/App/Yath/Command.t
< REASON >  job  2    Test script returned error (Code: 255)
( STDERR )  job  3    Can't locate object method "send_ev2" via package "Test2::API::Context" at /home/cpansand/.cpan/build/2018031121/Test2-Harness-0.001058-0/lib/Test2/Plugin/MemUsage.pm line 31.
( STDERR )  job  3    END failed--call queue aborted at t/App/Yath/Command/failed.t line 394.
( SKIPPED)  job  3    t/App/Yath/Command/failed.t  -  TODO
< REASON >  job  3    Test script returned error (Code: 255)
... (etc) ...

test.pl hangs on Windows

because

perl -Ilib ./scripts/yath test --default-search ./t --default-search ./t2

hangs too without any output.
Test2-Harness 0.001032. Windows XP. Strawberry perl 5.24.0, 5.16.0.

perl 5.8.1: "remove_tree" is not exported by the File::Path module

❯ perl -v

This is perl, v5.8.1 built for darwin-2level
(with 1 registered patch, see perl -V for more detail)

Copyright 1987-2003, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using `man perl' or `perldoc perl'.  If you have access to the
Internet, point your browser at http://www.perl.com/, the Perl Home Page.

❯ perl -MFile::Path -le 'print File::Path->VERSION'
1.06

❯ yath

** Defaulting to the 'test' command **

"remove_tree" is not exported by the File::Path module
Can't continue after import errors at /Users/skaji/env/plenv/versions/5.8.1/lib/perl5/site_perl/5.8.1/Test2/Harness/Feeder/Job.pm line 11.
BEGIN failed--compilation aborted at /Users/skaji/env/plenv/versions/5.8.1/lib/perl5/site_perl/5.8.1/Test2/Harness/Feeder/Job.pm line 11.
Compilation failed in require at /Users/skaji/env/plenv/versions/5.8.1/lib/perl5/site_perl/5.8.1/Test2/Harness/Feeder/Run.pm line 11.
BEGIN failed--compilation aborted at /Users/skaji/env/plenv/versions/5.8.1/lib/perl5/site_perl/5.8.1/Test2/Harness/Feeder/Run.pm line 11.
Compilation failed in require at /Users/skaji/env/plenv/versions/5.8.1/lib/perl5/site_perl/5.8.1/App/Yath/Command/test.pm line 8.
BEGIN failed--compilation aborted at /Users/skaji/env/plenv/versions/5.8.1/lib/perl5/site_perl/5.8.1/App/Yath/Command/test.pm line 8.
Compilation failed in require at /Users/skaji/env/plenv/versions/5.8.1/lib/perl5/site_perl/5.8.1/App/Yath.pm line 93.
BEGIN failed--compilation aborted at /Users/skaji/env/plenv/versions/5.8.1/bin/yath line 4.

isolation tests never run in persistent mode.

"Use of uninitialized value in scalar assignment" with perl 5.8.5

When I ran yath with perl 5.8.5, I got the following warnings

> yath -l t/00_use.t
Use of uninitialized value in scalar assignment at /Users/skaji/env/plenv/versions/5.8.5/lib/perl5/site_perl/5.8.5/Test2/Harness/Runner.pm line 59.
Use of uninitialized value in scalar assignment at /Users/skaji/env/plenv/versions/5.8.5/lib/perl5/site_perl/5.8.5/Test2/Harness/Runner.pm line 220.
Use of uninitialized value in scalar assignment at /Users/skaji/env/plenv/versions/5.8.5/lib/perl5/site_perl/5.8.5/Test2/Harness/Runner.pm line 220.
Use of uninitialized value in scalar assignment at /Users/skaji/env/plenv/versions/5.8.5/lib/perl5/site_perl/5.8.5/Test2/Harness/Runner.pm line 220.
Use of uninitialized value in scalar assignment at /Users/skaji/env/plenv/versions/5.8.5/lib/perl5/site_perl/5.8.5/Test2/Harness/Runner.pm line 220.
Use of uninitialized value in scalar assignment at /Users/skaji/env/plenv/versions/5.8.5/lib/perl5/site_perl/5.8.5/Test2/Harness/Runner.pm line 220.
Use of uninitialized value in scalar assignment at /Users/skaji/env/plenv/versions/5.8.5/lib/perl5/site_perl/5.8.5/Test2/Harness/Runner.pm line 220.
Use of uninitialized value in scalar assignment at /Users/skaji/env/plenv/versions/5.8.5/lib/perl5/site_perl/5.8.5/Test2/Harness/Runner.pm line 220.
Use of uninitialized value in scalar assignment at /Users/skaji/env/plenv/versions/5.8.5/lib/perl5/site_perl/5.8.5/Test2/Harness/Runner.pm line 220.
[PASSED]  =  t/00_use.t passed

=== ALL TESTS SUCCEEDED ===

https://github.com/Test-More/Test2-Harness/blob/master/lib/Test2/Harness/Runner.pm#L59
https://github.com/Test-More/Test2-Harness/blob/master/lib/Test2/Harness/Runner.pm#L220

I confirmed that these warnings did not occur with perl 5.10+.

> perl -w -e 'print "perl $]\n"; local $ENV{T2_FORMATTER} = $ENV{T2_FORMATTER};'
perl 5.008005
Use of uninitialized value in scalar assignment at -e line 1.

> perl -w -e 'print "perl $]\n"; local $ENV{T2_FORMATTER} = $ENV{T2_FORMATTER};'
perl 5.010001

It would be nice that yath suppresses these warnings.

0.000002 sometimes fails to install

I tried twice. My first attempt: https://gist.github.com/oalders/0a4bc4e98e0997f2c988db4bba752858

On the second attempt I was successful.

--no-(b)lib doesn't work?

#!perl

use Test::More tests => 1;

diag explain \@INC;

pass ();

prove:

$>prove -wv t/foo.t                   
t/foo.t .. 
1..1
# [
#   '/usr/local/cpanel',
#   '/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/x86_64-linux-64int',
#   '/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib',
#   '/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/5.26.0/x86_64-linux-64int',
#   '/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/5.26.0',
#   '/opt/cpanel/perl5/526/site_lib/x86_64-linux-64int',
#   '/opt/cpanel/perl5/526/site_lib'
# ]

ok 1
ok
All tests successful.

yath with requisite options.

$>yath test --no-blib --no-lib t/foo.t
(  DIAG  )  job  1    [
(  DIAG  )  job  1      '/usr/local/cpanel/lib',
(  DIAG  )  job  1      '/usr/local/cpanel/blib/lib',
(  DIAG  )  job  1      '/usr/local/cpanel/blib/arch',
(  DIAG  )  job  1      '/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/x86_64-linux-64int',
(  DIAG  )  job  1      '/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib',
(  DIAG  )  job  1      '/usr/local/cpanel',
(  DIAG  )  job  1      '/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/5.26.0/x86_64-linux-64int',
(  DIAG  )  job  1      '/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/5.26.0',
(  DIAG  )  job  1      '/opt/cpanel/perl5/526/site_lib/x86_64-linux-64int',
(  DIAG  )  job  1      '/opt/cpanel/perl5/526/site_lib',
(  DIAG  )  job  1      '.'
(  DIAG  )  job  1    ]
( PASSED )  job  1    t/foo.t

Request: --timer option

I sometimes use prove's --timer option to print elapsed time after each tests. It is useful.
I hope yath has that option too.

Windows: C:\projects\.yath.rc: No such file or directory

https://ci.appveyor.com/project/skaji/cpm/build/1.0.52

yath test t xt
C:\projects\.yath.rc: No such file or directory at C:/strawberry/perl/site/lib/App/Yath/Util.pm line 47.
Command exited with code 1

If you have no intentions to support Windows, please let me know; I will not create issues for Windows.

yath won't run if perl is in #! twice

$ cat ~/tmp/test.t
#!/perl/blah/perl

use Test::More;

diag "Perl is $^X";
pass;

done_testing;

$ yath ~/tmp/test.t
[LAUNCH]  _  /Users/schwern/tmp/test.t
[STDERR]  +  Can't open perl script "/blah/perl": No such file or directory
[ PLAN ]  =  No plan was ever set.
[FAILED]  =  /Users/schwern/tmp/test.t

=== FAILURE SUMMARY ===
 * /Users/schwern/tmp/test.t

I noticed this while trying yath with Regexp::Common. It has #!/opt/perl/bin/perl. I'm wondering why yath is even trying to open the #! perl.