gsinclair / whitestone Goto Github PK

Given the code

Mt tl1.description, /Overview of number systems/

I ran the test and got the error

Expect a String and a Regexp (any order)

The error is helpful if you genuinely misuse the Mt test, but in this case I thought I was giving it a string and a regex. It must be that tl1.description is nil. So a more helpful error message would be

Expect a String and a Regexp (any order); got [nil, Regexp]

Another thing to consider is this: should Mt accept nil and a regex, and fail the test (because no match was found) rather than error the test (because the wrong argument types were given)? This could be argued either way and requires further thought.

This is more of a question than and "issue". I'm curious about a design decision you made with Whitestone, b/c I have considered the same approach with my Assay project: Each assertion made in a test creates a new object, an instance of an assertion class. I am wondering why you choose to do this over just using a method call?

In Assay I created classes for each assertion too, but I put the assertion logic in class methods so I didn't have to instantiate an object for each assertion. I sort of prefer the idea of the way you did it, and had originally gone that route myself, but in the end wasn't sure it was worth the speed and memory footprint. For example, I created a benchmark comparing the two:

require 'benchmark'

class X
  def self.call(a,b)
    a == b
  end
  def initialize(a,b)
    @a, @b = a, b
  end
  def call
    @a == @b
  end
end

n = 1000000
Benchmark.bm do |x|
  x.report("method") { n.times{ X.call(1,1) } }
  x.report("object") { n.times{ X.new(1,1).call } }
end

The results:

       user     system      total        real
method  0.280000   0.000000   0.280000 (  0.280966)
object  1.200000   0.000000   1.200000 (  1.200855)

It's about 4 times slower, but maybe it doesn't matter since it can churn through a million of these in just over a second. On the other hand the memory footprint, est. about 100 bytes per object, a million assertion would cost 100MB. That pretty sizable but then how many test suites are really going to have a million assertions? Moreover these are all throw away objects, so they come and go very quickly.

So I am still considering moving back to the object design. But haven't been able to make up my mind either way for sure. I thought it would be good to ask your thoughts on the matter seeing that Whitestone took this approach.

Use an --order flag to specify what you want to see (report and errors), in what order, and if you want a pause in between.

whitestone --order re            # report then errors; default
whitestone --order er            # other way around
whitestone --order r             # report only (similarly 'e': errors only)
whitestone --order rpe           # report, pause, error

Given the assertion

N! tl1.description

it failed with the message

Condition expected NOT to be nil

That's fine, except that I didn't give it a condition, I gave it a value. So the message would read better:

Condition or value expected NOT to be nil

Maybe it could even be sensitive to what was passed in -- a condition or a value -- and tailor the failure message to that.

(A "condition" is something passed in braces, like N { foo.bar })

I've never seen this before, but it's very reproducible on my shiny new iMac in both Terminal and iTerm. Running whitestone | less -R in a project directory looks OK to start with, but any input you give less (e.g. Space, Down, even q) is not acted upon until you hit Enter. This is not how less is supposed to behave! Clearly the control characters are having an effect.

I'll have to try it again with the latest code (not that I think there have been any changes) on Cygwin.

Interestingly, running whitestone > test.log; less test.log solves the problem. So if less has access to the whole file, there's no issue.

Having looked at the raw Whitestone output, I think there are a lot of ANSI codes. Sure, the colouring in Whitestone is pretty complicated, but I think the codes could be simplified a bit -- that's a job for Col. Also, each line ends with \e[0m, which I think is correct, but git diff, with its coloured output, only uses \e[m -- that is, no zero. Further research warranted.

Summary: I've only seen this happen on the Mac, but other software emits coloured text without causing less any problems, so there is certainly an issue here to be fixed.