Comments (27)
@Moniya there are no tests for step0 (I may add some at some point just for completeness). Can you post the output that you get from running make test^ruby^step1
? What OS are you on?
from mal.
@kanaka I'm on OSX Lion (10.7.5). I experience the same issue running other steps, too. Here's the output when I run make test^ruby^step1
:
Testing test^bash^step1, step file: bash/step1_read_print.sh, test file: tests/step1_read_print.mal
Running: ../runtest.py ../tests/step1_read_print.mal -- bash ../bash/step1_read_print.sh
Started with:
../bash/types.sh: line 8: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
../bash/types.sh: line 8: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
Testing read of nil/true/false
TEST: nil -> ['','nil'] -> SUCCESS
TEST: true -> ['','true'] -> SUCCESS
TEST: false -> ['','false'] -> SUCCESS
Testing read of numbers
TEST: 1 -> ['','1'] -> SUCCESS
TEST: 7 -> ['','7'] -> SUCCESS
TEST: 7 -> ['','7'] -> SUCCESS
Testing read of symbols
TEST: + -> ['','+'] -> SUCCESS
TEST: abc -> ['','abc'] -> SUCCESS
TEST: abc -> ['','abc'] -> SUCCESS
TEST: abc5 -> ['','abc5'] -> SUCCESS
TEST: abc-def -> ['','abc-def'] -> SUCCESS
Testing read of strings
TEST: "abc" -> ['','"abc"']
Completes with the following after a long pause:
TEST: "abc" -> ['','"abc"'] -> FAIL (line 34):
Expected : '"abc"\r\n"abc"'
Got : None
TEST: "abc" -> ['','"abc"']Traceback (most recent call last):
File "../runtest.py", line 173, in <module>
r.writeline(form)
File "../runtest.py", line 84, in writeline
self.stdin.write(str + "\n")
IOError: [Errno 5] Input/output error
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/anaconda/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "../runtest.py", line 92, in cleanup
os.killpg(self.p.pid, signal.SIGTERM)
OSError: [Errno 1] Operation not permitted
Error in sys.exitfunc:
Traceback (most recent call last):
File "/anaconda/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "../runtest.py", line 92, in cleanup
os.killpg(self.p.pid, signal.SIGTERM)
OSError: [Errno 1] Operation not permitted
make: *** [test^bash^step1] Error 1
from mal.
@kanaka I'm excited about your project and can't wait to get on board. Well done. Thanks ;)
from mal.
@Moniya Okay, looks like a couple of different problems that are probably related to OS X (I don't have an easy OS X test system so it's not surprising really).
- something weird is going on with the makefile rules such that maybe the default rule is being run instead of the one you specify. What version of make do you have?
make --version
- there is a non-fatal complaint at cleanup for the test harness (runtest.py).
Let's see if we can't at least get you a workaround. Try this:
cd ruby
../runtest.py ../tests/step1_read_print.mal -- ruby ../ruby/step1_read_print.rb
You'll probably see some errors when the test tries to cleanup (and it may leave some orphan child processes around) but maybe that will at least allow you to run tests.
from mal.
@kanaka My make is GNU Make 3.81
. Tried the workaround. It did work around well. Thanks. :) A shame I'm not good with Make, otherwise I'd gladly contribute a fix.
As an aside, I tried the workaround on the Racket implementation (step 4) several times. It ran well, but sometimes test failures were reported, sometimes not. And the number of failures reported on test runs with failures varied. Is that normal? I don't see the issue with the Ruby implementation.
I have Racket 6.1
though, which I see is more recent than the version here.
from mal.
@Moniya I don't see that variation on my system (Ubuntu 14.04) with Racket v5.3.6. I just ran it 10 times in a row with no variation. If you're only seeing that with Racket then it is possible there is something odd with newer Racket. If you see it with other languages, then it's more likely that the use of pseudo-terminals (pty) in runtest.py is being affected by some difference of OS X.
from mal.
Maybe it has to do with your shell using ^
as a special character. ZSH does that for example. Try make "test^ruby^step1"
.
from mal.
@def- Thanks! Using double quotes solved the issue. Also, running the original command on bash gave no issues. You are right. Zsh is the culprit here. I remember having problems running git commands that contained carets in Zsh.
from mal.
@kanaka I switched to Racket v5.3.6 and still ran into the same issue. I'll see if I can look into it. I'm not seeing the issue with other languages I've tried: Ruby, Python.
Oh well, PHP is acting up, too. Same issue. The issue is not Racket specific.
from mal.
@def- good catch. I didn't realize that was a special character in zsh.
@Moniya it's possible that it's a performance related issue. Racket may be sending output faster than ruby or python and that may be what's triggering the issue for the harness. That's just a wild guess though. I'd be interested in output from a failing case vs a passing case (just the part of the test that failed vs not).
from mal.
@kanaka I'll soon post something consistent about the failures that I find interesting and might be helpful in solving the issue. I need to find power for my laptop as the battery has run out .
from mal.
@kanaka Output for two test cases when they succeed and when they fail:
TEST: (count (list 1 2 3)) -> ['','3'] -> SUCCESS
TEST: (count (list 1 2 3)) -> ['','3'] -> FAIL (line 16):
Expected : '(count (list 1 2 3))\r\n3'
Got : '(count (list 1 2 3))\r\n(count (list 1 2 3))\r\n3'
TEST: (if (> (count (list 1 2 3)) 3) "yes" "no") -> ['','"no"'] -> SUCCESS
TEST: (if (> (count (list 1 2 3)) 3) "yes" "no") -> ['','"no"'] -> FAIL (line 22):
Expected : '(if (> (count (list 1 2 3)) 3) "yes" "no")\r\n"no"'
Got : '(if (> (count (list 1 2 3)) 3) "yes" "no")\r\n(if (> (count (list 1 2 3)) 3) "yes" "no")\r\n"no"'
All the failures exhibit this pattern: the tested expression being duplicated in the result. Looks like a race issue. Also, a lot more tests were failing when my battery was low and the system culled performance than when it was operating normally.
As earlier stated, the PHP implementation also gave the same issue.
from mal.
@Moniya mind testing again with upstream changes to runtest? There is a chance some recent changes may have fixed this. Thanks.
from mal.
@kanaka The issue still occurs, but much less frequently (like 1 in 20 tries), and the number of failing test cases are fewer too, just 1 or 2 failures.
from mal.
Umm... It's kinda frequent now, but failing cases just 1 or 2.
from mal.
So it was infrequent and then suddenly became more frequent? Anything change in the meantime?
Are you still seeing this with the racket and php implementations only? Does it tend to be the same test cases that fail when it does fail?
from mal.
I think the failures got less frequent because I just got power on my laptop (MBA 1.7 GHz) and things got a little faster (power supply is often unavailable where I live).
Yes the Racket and PHP implementations only.
Each case fails or passes randomly. I illustrated the nature of the failures in an earlier comment. #30 (comment)
from mal.
I'm experiencing a memory-related issue running the Racket implementation in Ubuntu 14.10 on Virtual Box. Should I start another issue or post it here?
from mal.
Another issue. Is it perhaps leaving orphan processes around (not cleaning them up properly)?
from mal.
O well... That memory issue has stopped happening after yet another restart.
Maybe you can ignore that.
I swear I'm not doing anything crazy with my computer. :)
from mal.
P.S: These issues are not in any way hindering my progress on the project. I just deem it useful to bring them to your notice.
from mal.
FWIW, I also see the error that moniya reports where the input text is duplicated once every 20 tests or so. The issue is random. Running the same test over and over will result in the condition occurring on different test cases. Occasionally, the test runs with no errors.
This is on Mac OS X 10.10.2 with the Swift implementation I'm working on. I'm building against mal b020aa3. I've not done anything to try to track down the source of the problem.
Here's an example from the test for step4:
...
Testing vector equality
TEST: (= [] (list)) -> ['','true'] -> SUCCESS
TEST: (= (list 1 2) [1 2]) -> ['','true'] -> SUCCESS
TEST: (= (list 1) []) -> ['','false'] -> SUCCESS
TEST: (= [] [1]) -> ['','false'] -> FAIL (line 374):
Expected : '(= [] [1])\r\nfalse'
Got : '(= [] [1])\r\n(= [] [1])\r\nfalse'
TEST: (= 0 []) -> ['','false'] -> FAIL (line 376):
Expected : '(= 0 [])\r\nfalse'
Got : '(= 0 [])\r\n(= 0 [])\r\nfalse'
...
from mal.
@keith-rollin @Moniya I've pushed a hack which I think should workaround this issue. I now check for the expected output and I also check for the case where the input form is echo'd twice in the output. I'm going to close the issue because I'm pretty confident it will work around it, but can you two confirm that it has resolved it for you? If not, I'll re-open it.
from mal.
It worked. Well done. 👍
from mal.
@Moniya yay!
from mal.
The ^ issue should be fixed by #128
from mal.
I think all the problems reflected in this issue have been resolved so I'm closing it.
from mal.
Related Issues (20)
- what is the minimum requirement for the base language? HOT 2
- Verilog: is there a good reason mal has not been implemented? HOT 1
- VHDL: defmacro! doesn't modify existing functions HOT 1
- make "test^js^step0" fails -- Fatal error in , line 0 -- Check failed: result.second. HOT 1
- PDFs?
- Somebody should solve Rosetta Code challenges with mal
- Mal in Coq? HOT 2
- Apparently in step 3 'env' is a class or structure type, not an object
- Questions on different design from lisp: Mal symbol do not use property lists?
- Lots of unanswered merge requests HOT 1
- Why is quasiquote so complicated? HOT 3
- What with `` gensym ''
- lexicial versus dynamic
- Floats don't work in the python implementation. HOT 2
- Is #mal IRC channel still active? people seems tends to use discord now XD HOT 1
- Should `eval` use the top-level environment? HOT 1
- How is `cond` supposed to work? HOT 2
- Remaining impls to combine eval-ast/macroexpand into eval HOT 4
- common-lisp fails to build HOT 1
- Fix self-hosted test failures HOT 24
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mal.