Comments (8)
Ok, I think I have one for you. It's called Test::Cmd -- although it isn't a core routine. It is in CPAN though and works down to perl 5.6. Written by Neil Bowers and with over 6000 testers. You can try out the attached test script, which has 36 tests of bc.
By default it will attempt to locate bc in ../bin/bc relative to the bin in which it is executing-- which is expected to be PerlPowerTools/t. But you can put the path to bc on the command line and it'll use that.
If you think this is worth adding, I'd like to create a branch called test_scripts (or maybe issue_140 if you prefer), I'll then write whatever test scripts I can, commit them to that branch, and then push it to the repo for review and possible merge into master.
from perlpowertools.
As far as I can tell, Test::Cmd doesn't do what I want. I'd like something where I can send one line of input to a command then get it's output, and then send another line, and get it's output. Think about testing things like bc or ed where you want to see the state of their buffers after every input.
The answer is probably something with IPC::Open3, but I'd like a better interface to it so it makes sense in our tests.
from perlpowertools.
If I understand you correctly, I can do just that with Test::Cmd. For example:
my $test_obj = Test::Cmd->new( prog => $Script, interpreter => 'perl', workdir => '' );
$test_obj->run( stdin => <<"__data" );
v=3
++v
__data
is( $test_obj->stdout, "3\n4\n" );
I could have set up the 36 tests of bc
expressions that I included with the file I sent you as a single test like this, but a failure anywhere would result in a very messy report.
My script produces this output:
C:\Sandbox\PerlPowerTools> perl ..\bc.t bin\bc
1..36
ok 1 - negation: -1
ok 2 - variable assignment: var=12
ok 3 - prefix increment: v=3; ++v
ok 4 - prefix increment: v=3; --v
ok 5 - postfix increment: v=3; v++; v
ok 6 - postfix increment: v=3; v--; v
ok 7 - postfix increment: v=3; v+=5; v
ok 8 - postfix decrement: v=5; v-=3; v
ok 9 - addition: 1+2
ok 10 - subtraction: 5-3
ok 11 - multiplication: 3*5
ok 12 - division: 15/3
ok 13 - remainder zero: 15%3
ok 14 - remainder four: 29%5
ok 15 - exponentiation: 2^15
ok 16 - expr without parens: 3*2+5
ok 17 - expr with parens: 3*(2+5)
ok 18 - sqrt: sqrt(16)
ok 19 - boolean not: !0
ok 20 - boolean not: !1
ok 21 - boolean and: 1 && 1
ok 22 - boolean and: 1 && 0
ok 23 - boolean and: 0 && 1
ok 24 - boolean and: 0 && 0
ok 25 - boolean or: 1 || 1
ok 26 - boolean or: 1 || 0
ok 27 - boolean or: 0 || 1
ok 28 - boolean or: 0 || 0
ok 29 - equals (true): 9 == 9
ok 30 - equals (false): 9 == 8
ok 31 - not equals (true): 9 != 8
ok 32 - not equals (false): 9 != 9
ok 33 - less than: 5 < 7
ok 34 - less than or equal: 5 <= 5
ok 35 - greater than: 7 > 5
ok 36 - greater than or equal: 7 >= 7
What my script doesn't do is catch a problem arising from some kind of state error, since every execution is independent of the previous one. But those errors are highly subject to the order of input. I was striving for basic functionality testing first.
It takes 3.8 seconds to run that set of tests on my desktop computer. Sure, it might be faster if you could set up a pipeline between the input generator and bc
and the tester, but is it worth the hassle? Will it be reliable across platforms?
Another way to approach this is to do what I did in the units script. I modified it to allow it to be require'd in the test script, and provided a test( )
function that essentially does what run( )
would do except it gets input from parameters instead of stdin, and it delivers output as data instead of printing it. That could probably be done with bc
as well.
You should give bc.t
a try. I think it's a solution that's "good enough" -- and certainly better than no tests at all!
P.S. As a point of comparison, the 28 tests in units.t take 1.47 seconds to execute on my machine vs 3.8 seconds for the 36 tests in bc.t.
from perlpowertools.
While I'm here I may as well save you the trouble of trying bc.t yourself. There's not much to it besides the table that defines the tests, Here's what that looks like:
my @test_table = (
[ '-1', '-1', 'negation' ],
[ 'var=12', '12', 'variable assignment' ],
[ 'v=3; ++v', "3\n4", 'prefix increment' ],
[ 'v=3; --v', "3\n2", 'prefix increment' ],
[ 'v=3; v++; v', "3\n3\n4", 'postfix increment' ],
[ 'v=3; v--; v', "3\n3\n2", 'postfix increment' ],
[ 'v=3; v+=5; v', "3\n8\n8", 'postfix increment' ],
[ 'v=5; v-=3; v', "5\n2\n2", 'postfix decrement' ],
[ '1+2', '3', 'addition' ],
[ '5-3', '2', 'subtraction' ],
[ '3*5', '15', 'multiplication' ],
[ '15/3', '5', 'division' ],
[ '15%3', '0', 'remainder zero' ],
[ '29%5', '4', 'remainder four' ],
[ '2^15', '32768', 'exponentiation' ],
[ '3*2+5', '11', 'expr without parens' ],
[ '3*(2+5)', '21', 'expr with parens' ],
[ 'sqrt(16)', '4', 'sqrt' ],
[ '!0', '1', 'boolean not' ],
[ '!1', '0', 'boolean not' ],
[ '1 && 1', '1', 'boolean and' ],
[ '1 && 0', '0', 'boolean and' ],
[ '0 && 1', '0', 'boolean and' ],
[ '0 && 0', '0', 'boolean and' ],
[ '1 || 1', '1', 'boolean or' ],
[ '1 || 0', '1', 'boolean or' ],
[ '0 || 1', '1', 'boolean or' ],
[ '0 || 0', '0', 'boolean or' ],
[ '9 == 9', '1', 'equals (true)' ],
[ '9 == 8', '0', 'equals (false)' ],
[ '9 != 8', '1', 'not equals (true)' ],
[ '9 != 9', '0', 'not equals (false)' ],
[ '5 < 7', '1', 'less than' ],
[ '5 <= 5', '1', 'less than or equal' ],
[ '7 > 5', '1', 'greater than' ],
[ '7 >= 7', '1', 'greater than or equal' ],
);
The rest is a foreach loop and an invocation of Test::Cmd run() as described in my previous message.
from perlpowertools.
You haven't done what I'm imagining with Test::Cmd: you gave it all the input at once and got the final output. This issue is about not doing that. I want to give one line of input, get the output, give another line of input. This way you know if a particular line of input did what it should do without having to do a lot of ad-hoc work to match up input with output.
The tests for bc are fine, but they aren't what I'm talking about here. But, we don't really need Test::Cmd for that. All of that can be done with IPC::Open3 just as easily, and if we can avoid a dependency we should. Installing things in the GitHub Actions already takes too long.
I looked at the tests you had for t/units.t and I'm about to merge a major refactor of it so it's table-oriented like you just saw. There are some other modulino tricks I added too. It's fine that you added the test
method to units, but that works because units is doing something very simple. It's not appropriate for most other programs.
from perlpowertools.
Besides what I just wrote, you should add those tests to bc. That's good work, just for a different issue. :)
from perlpowertools.
Ah, ok that helps. I wasn't against a solution using IPC::Open3. I'd love to see a working example. Maybe I can use it instead of Test::Cmd.
I do get your point about minimizing dependencies in the CI environment.
A drawback to using Test::Cmd is the process overhead. Even though it might only take 3 seconds to run 30 or so tests, we have 120 scripts to test and so that adds up. A faster solution would be better.
I fiddled with Open3 a bit but so far a working example has eluded me. Maybe I'll have better luck tomorrow.
from perlpowertools.
After working through this issue in multiple ways, I've come to the conclusion that it's not possible to get this working on Windows. If it were, we'd have a CPAN module to do it. If it can't work on Windows, it's not something we can do here.
from perlpowertools.
Related Issues (20)
- bc: test binary assignment HOT 3
- bc: test scale, ibase and obase variables and their effects
- bc: define is broken HOT 1
- bc: test the -b option (which uses Math::BigFloat)
- bc: test if HOT 3
- cp: convert to something better HOT 1
- Failed test 'blib/script/robots compiles' HOT 3
- bc: sqrt testing fails because of additional precision HOT 1
- bc: Perl error messages are not always in english HOT 1
- rm: some CPAN Testers complain about re-defining exit HOT 1
- find: find.t adds a local::lib location without checking that it exists HOT 1
- install fails for symbolic -m argument HOT 5
- Refactor mod() out of chmod, install, and mkdir but keep them standalone programs
- RT #98905: factor and primes are limited to 2^32 HOT 1
- pr: -w option is ignored
- Implement restricted mode for ed
- grep -F: regex error when pattern includes '/' HOT 1
- Unquoted $^X in date.t HOT 1
- Update packer and PerlPowerTools.exe HOT 3
- bc statements have stopped working HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from perlpowertools.