bensu / doo Goto Github PK

Hi @bensu. When I try to use this with a project generated from lein new mies using clojurescript 1.7.48 I get the following error:

$ lein doo phantom test once                                                                                                                            WARNING: Use of undeclared Var cljs.core/vswap! at line 5 test/reagent_raw/runner.cljs
;; ======================================================================
;; Testing with Phantom:
Hello world!

Testing reagent-raw.core-test
TypeError: undefined is not an object (evaluating 'cljs.core.vswap_BANG_.call')

  /var/folders/k6/nmwrydfd25l3zxq7f0rjsc_r0000gn/T/phantom3186709797736901614.js:34 in onError

Subprocess failed

I'm just doing a basic cljs test and using a runner as given in the readme. This failure also occurs with slimer. If I change to clojurescript "0.0-3308", I have no issue. My project.clj looks like this:

(defproject reagent-raw "0.1.0-SNAPSHOT"
  :description "FIXME: write this!"
  :url "http://example.com/FIXME"
  :dependencies [[org.clojure/clojure "1.7.0"]
                 [org.clojure/clojurescript "1.7.48" :classifier "aot"
                  :exclusion [org.clojure/data.json]]
                 [org.clojure/data.json "0.2.6" :classifier "aot"]]
  :jvm-opts ^:replace ["-Xmx1g" "-server"]
  :plugins [[lein-npm "0.6.1"]
            [lein-cljsbuild "1.0.5"]
            [lein-doo "0.1.4-SNAPSHOT"]]
  :npm {:dependencies [[source-map-support "0.3.2"]]}
  :cljsbuild {:builds
              [{:id "test"
                :source-paths ["src" "test"]
                :compiler {:output-to "out/testable.js"
                           :main 'reagent-raw.runner
                           :optimizations :none}}]}
  :source-paths ["src" "target/classes"]
  :clean-targets ["out" "release"]

  :target-path "target")

Let me know if I can provide anything more to recreate. I'd love to get this working on latest clojurescript. Thanks for providing a library that keeps us from copying and pasting cljs.test runners

Having some trouble using 0.1.6 with unoptimised CLJS builds - and the issue seems to be http://dev.clojure.org/jira/browse/CLJS-1444.

Until that is sorted out, such builds would need to be run from the :output-to directory.

Although this has its own problems, e.g. for commands only installed locally for the project, I can't think of another solution. On the flip-side this has advantages too (ie. support for non-global commands outside the project)

I have several macros used by tests, and any time I change one of them, I have to stop the lein doo process and restart it, then (usually) touch my test file so doo recompiles the JS and runs the updated code. If I don't touch the test file, doo seems to use the original macro definition instead of the updated one.

There have been quite a few changes since the last SNAPSHOT release. Would it be possible to release a new version?

An issue with having long running snapshots is that when code changes in a SNAPSHOT it can break running tests. Normally you would deal with that by pinning to specific snapshot versions, like 0.1.5-20150918.... However I don't think it is possible (it's certainly not easy), to pin a particular lein-doo plugin SNAPSHOT and make it use a particular pinned version of the doo library it depends on, as lein-doo just specifies a dependency on 0.1.6-SNAPSHOT.

Would it make more sense to release alpha or beta versions when you don't want to release full versions?

There are artificial restrictions on :output-dir for :none for Slimer and Phantom.

As discussed with @danielcompton, Karma builds with :none now need an :asset-path base prefix. It is a regression probably caused by 6244a0d and should be fixed.

I wasn't able to find a way how to pass additional arguments to existings runners. For example if we specify :doo {paths {:phantom "phantomjs --web-security=false" }}} it will fail because doo will check if executable exists first and with additional arguments it will fail to do so.

One way could be adding additional arguments to doo settings or maybe it would be a better just to rely on karma and give user a way to specify a custom karma.config.js, so it will be possible to configure a lot of things there.

Currently I'm using the following workaround:
project.clj

:doo {:paths {:phantom "tests/phantom.sh"}}

tests/phantom.sh

#!/bin/bash
phantomjs --web-security=false --local-to-remote-url-access=true --ignore-ssl-errors=true $@

It would be nice to have some special syntax to support running tests with multiple js-env.

e.g.

lein doo phantom,slimer test

clojure.lang.ExceptionInfo: Only :as and :refer options supported in :require / :require-macros; offending spec: [doo.runner :refer-marcos [doo-all-tests doo-tests]] at line 2 test/graphql-tlc/runner.cljs {:file #object[java.io.File 0x2ca93dee "test/graphql-tlc/runner.cljs"], :line 2, :column 1, :tag :cljs/analysis-error}

My runner looks like so:

Jonathans-MacBook-Pro:graphql-type-lang-compiler jonathan$ cat test/graphql-tlc/runner.cljs 

(ns graphql-tlc.runner
  (:require [cljs.test :as test]
            [doo.runner :refer-marcos [doo-all-tests doo-tests]]
            [graphql-tlc.test.core]))

(doo-tests 'graphql-tlc.test.core)

This is a placeholder to fix the failing tests

lein test test.doo.core

lein test :only test.doo.core/integration

FAIL in (integration) (core.clj:101)
We can compile a cljs project
expected: (let [compiler-opts' (merge compiler-opts {})] (cljs/build srcs compiler-opts') (->> [:phantom :chrome :firefox] (mapv (fn [env] (-> env (doo/run-script compiler-opts' doo-opts) doo-ok?))) (every? true?)))
  actual: false

lein test :only test.doo.core/integration

FAIL in (integration) (core.clj:101)
We can compile a cljs project
expected: (let [compiler-opts' (merge compiler-opts {:optimizations :whitespace})] (cljs/build srcs compiler-opts') (->> [:rhino :phantom :chrome :firefox] (mapv (fn [env] (-> env (doo/run-script compiler-opts' doo-opts) doo-ok?))) (every? true?)))
  actual: false

lein test :only test.doo.core/integration

FAIL in (integration) (core.clj:101)
We can compile a cljs project
expected: (let [compiler-opts' (merge compiler-opts {:optimizations :advanced})] (cljs/build srcs compiler-opts') (->> [:phantom :rhino :chrome :firefox] (mapv (fn [env] (-> env (doo/run-script compiler-opts' doo-opts) doo-ok?))) (every? true?)))
  actual: false

Ran 5 tests containing 57 assertions.
3 failures, 0 errors.

Tests failed. cd library && lein test returned exit code 1

As you demonstrate in https://github.com/bensu/doo#setting-up-environments, there are simple checks to test whether underlying runners are installed.

For a user-friendlier experience, perhaps these exact checks should be performed by the application, so that friendlier error messages (or even installation instructions) can be shown.

Motivation for this comes from when a default runner is bundled downstream (eg. from a lein template martinklepsch/tenzing#28), and the end user may not be familiar with what the command means.

Haven't tried to understand the error yet, or looked at your changes, but have verified that still working after #13 and broken on master.

Here's the stracktrace:

             clojure.lang.ExceptionInfo: java.lang.IllegalArgumentException: No value supplied for key: null
    data
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: No value supplied for key: null
     java.lang.IllegalArgumentException: No value supplied for key: null
                                         ...
                       clojure.core/hash-map            core.clj:  371
                                         ...
                          clojure.core/apply            core.clj:  630
               clojure.java.shell/parse-args           shell.clj:   50
                       clojure.java.shell/sh           shell.clj:  112
                                         ...
                          clojure.core/apply            core.clj:  630
                         doo.core/run-script            core.clj:  150

Am just calling doo like so: (doo.core/run-script :rhino) "/super/long/boot/temp/path.js"

Is it possible to get stack traces for errors from Karma?

At the moment they look like this:

Chrome 46.0.2490 (Mac OS X 10.11.0) bigco.base.util-test fail FAILED
    Fail (fail) (cljs/test.js:417:14)
    Expected (= 1 2)
    Actual: (not (= 1 2))

I'm not sure if this is related or not, but when I open the Karma Debug page, I get a bunch of warnings in the doo terminal about 404 for source map files, e.g.:

19 10 2015 08:24:30.989:WARN [web-server]: 404: /base/out/cljs_time/format.js.map

@bensu would it be possible to release 0.1.5? I understand that it's under a SNAPSHOT because the behaviour is subject to change, but it does raise the risk of breaking existing code, e.g. #36. The Karma test runner could still be marked as experimental and subject to breaking changes, but at least it would be a stable point for those that need to use it.

I know that I can specify a particular SNAPSHOT version to pin dependencies, but I don't know how widely spread this knowledge is and if others are doing it too.

Just a thought.

Since paths are defined as strings (which include the separator) a cross-platform team can't properly specify a path.

It would be convenient to be able to run only specific tests, rather than whole namespaces. This would be particularly useful with downstream tooling eg. rerunning failing tests first on save, or only the tests that call the given function.

Expect the method would take vars, ie. (doo-test-vars #'ns/test-1 #'ns/test-2), as I don't believe something like resolve exists in Clojurescript.

Mimicking run-all-tests.

I can take care of it, it should be so easy to do 👍

When trying 0.1.1-SNAPSHOT on a different project, the runner script no longer gets run... https://gist.github.com/lvh/351566fe6a6ca2ed86e0

I can't figure out what's special about it; any idea where to start? The problem does go away when switching to the old version of doo, though...

Despite the workaround for https://github.com/laurentj/slimerjs/issues/223 of using window.callPhantom, I'm not seeing any test results for the slimer option.

Have tried this with two projects, and in both cases only the slimer option is silent.

Any ideas?

Hi,

I'm trying to use rhino with doo 0.1.6-SNAPSHOT. I installed nodejs and karma but when running lein doo rhino test I get the following output:

ERROR: doo was not loaded from the compiled script.

Make sure you start your tests using doo-tests or doo-all-tests
and that you include that file in your build

;; ======================================================================
;; Testing with :

Though, I do include and use (doo-tests 'foo.runner).

Any idea on what I am doing wrong?

I'd like to be able to add karma-junit-reporter to my karma.conf.js file. I can imagine in the long run that lots of people will all want to install their own Karma plugins. There are a few ways to accomodate this:

• Add a command to allow people to generate a karma.conf.js file to disk and then they can add their plugins, configuration, e.t.c. to that static file. The problem with this approach is regenerating it when things change, and it getting out of date.
• Allow extra arguments to be passed on the command line. Some Karma plugins can be activated by this method, though they probably still need some config provided
• Allow people to provide extra config which gets merged into the map used for creating karma.conf.js. This would be somewhat similar to how Leiningen merges profiles into the main project config.

This presupposes that the Karma plugins will be able to understand the output of doo tests, I'm not sure if that's the case.

Relates to #34.

The cljs.test/report methods are not very readable due to Karma's logging facilities.

Would it be at all possible for doo to make client-server integration testing more convenient, or is that out of scope?

It looks like the way to do it in lein is to have an alias where you spin up a ring server, run your tests, and spin the ring server back down:
https://groups.google.com/d/msg/clojurescript/rArkEBCRods/Bqo4IE5ehNUJ

Anyhow this is a marvelous library and I am looking forward to using it in all of my ClojureScript projects!

Is it possible to leave the browser up while running Karma tests? It would be really nice to not have the Chrome browser popping up and down when tests are auto running.

Interested in adding this? Thinking a QUnit style "refresh and read it" to start with, but this could be extended to drive Karma or a custom reporter to gather those stats back in the console.

Currently node is very restricted to a small set of options. Cleaning up the runner script can remove most of those limitations.

There are three ways to specify :main:

1. :main 'app.core
2. :main app.core
3. :main "app.core"

We should support all of them. As noticed by @teaforthecat we currently don't support 2. which is Figwheel's preferred method.

1. Is important since it is used in scripts for the compiler, and beginners will probably paste those to project.clj.

Given the benefits both for compilation and evaluation times in a tight feedback loop, it would be great to support this compilation mode simply out of the box.

Solutions have been offered at https://github.com/mike-thompson-day8/cljsbuild-none-test-seed

Hello @bensu and again thanks for this refactoring!
First of all, my conf:

:plugins [[lein-cljsbuild "1.1.0"]
              [lein-figwheel "0.4.1" :exclusions [cider/cider-nrepl]]
              [lein-doo "0.1.6-SNAPSHOT"]]

  :clean-targets ^{:protect false} ["resources/public/js/compiled" "target" "out"]
  :hooks [leiningen.cljsbuild]
  :source-paths ["src/clj"]

  :cljsbuild {:builds [{:id "dev"
                        :source-paths ["src/cljs" "test/cljs"]
                        :figwheel {:on-jsload "launcher.test/run"
                                   :css-dirs ["resources/public/styles"]}
                        :compiler {:main "cljs-browser-repl.core"  ;; https://github.com/bensu/doo/wiki/Working-with-Figwheel
                                   :output-to "resources/public/js/compiled/cljs-browser-repl.js"
                                   :output-dir "resources/public/js/compiled/out"
                                   :asset-path "js/compiled/out"
                                   :optimizations :none
                                   :source-map-timestamp true}}
                       {:id "test"
                        :source-paths ["src/cljs" "test/cljs"]
                        :compiler {:main launcher.runner
                                   :output-to "resources/private/test/compiled/cljs-browser-repl.js"
                                   :pretty-print false}}
...

When I launch lein figwheel dev I receive a big:

Caused by: clojure.lang.ExceptionInfo: No such namespace: doo.runner, could not locate doo/runner.cljs, doo/runner.cljc, or Closure namespace "doo.runner"
 at clojure.core$ex_info.invoke (core.clj:4593)

clojure.lang.ExceptionInfo: No such namespace: doo.runner, could not locate doo/runner.cljs, doo/runner.cljc, or Closure namespace "doo.runner" {:tag :cljs/analysis-error}
    at clojure.core$ex_info.invoke(core.clj:4593)
    at cljs.analyzer$error.invoke(analyzer.cljc:562)
    at cljs.analyzer$error.invoke(analyzer.cljc:560)
    at cljs.analyzer$analyze_deps.invoke(analyzer.cljc:1668)
    at cljs.analyzer$ns_side_effects.invoke(analyzer.cljc:2504)
    at cljs.analyzer$analyze_STAR_$fn__2094.invoke(analyzer.cljc:2585)
    at clojure.lang.PersistentVector.reduce(PersistentVector.java:333)
    at clojure.core$reduce.invoke(core.clj:6518)
    at cljs.analyzer$analyze_STAR_.invoke(analyzer.cljc:2585)
    at cljs.analyzer$analyze.invoke(analyzer.cljc:2600)
    at cljs.analyzer$analyze_file$fn__2145.invoke(analyzer.cljc:2844)
    at cljs.analyzer$analyze_file.invoke(analyzer.cljc:2839)
    at figwheel_sidecar.repl$analyze_build.invoke(repl.clj:242)
    at figwheel_sidecar.repl$analyze_builds.invoke(repl.clj:247)
    at figwheel_sidecar.repl$run_autobuilder_helper.invoke(repl.clj:295)
    at figwheel_sidecar.repl$start_autobuild.invoke(repl.clj:366)
    at figwheel_sidecar.repl$run_autobuilder.invoke(repl.clj:576)
    at user$eval12982.invoke(form-init8065958100227564854.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:6782)
    at clojure.lang.Compiler.eval(Compiler.java:6772)
    at clojure.lang.Compiler.load(Compiler.java:7227)
    at clojure.lang.Compiler.loadFile(Compiler.java:7165)
    at clojure.main$load_script.invoke(main.clj:275)
    at clojure.main$init_opt.invoke(main.clj:280)
    at clojure.main$initialize.invoke(main.clj:308)
    at clojure.main$null_opt.invoke(main.clj:343)
    at clojure.main$main.doInvoke(main.clj:421)
    at clojure.lang.RestFn.invoke(RestFn.java:421)
    at clojure.lang.Var.invoke(Var.java:383)
    at clojure.lang.AFn.applyToHelper(AFn.java:156)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.main.main(main.java:37)

Any workaround?

It would be nice to have JUnit test reporting as this is the lingua franca of test result reporting. This can then be used in a number of CI tools for more detailed test reporting, beyond just pass/fail.

Something like https://github.com/karma-runner/karma-junit-reporter could be good for test reporting, although it only supports Karma. Perhaps it would be better to create a reporter that can be used for any testing tool, relying on cljs.test report to get details.

Hi @bensu,

I wanted to ask you if there is a way to know which runner is currently executed in order to pass it to my tests. For instance to execute node-specific configuration as test fixture.

If this is not exposed, I could work on doing it and submit a PR ;)

Since there is usually one test build per project, users should be able to call it with:

lein doo phantom

as long as they define it under :doo :default.

I checked out and ran the example like this:

git clone [email protected]:bensu/doo
cd doo/example
lein doo phantom test

I got this error:

ERROR: doo was not loaded from the compiled script.

Make sure you start your tests using doo-tests or doo-all-tests
and that you include that file in your build

I get this error in exactly the same way on my own project trying to get doo running, which is why I thought I'd give the example a try. Is there something I can do to resolve this?

I'm on Windows 10 using Git Bash.

I was a bit desperate, not finding much information about how to run tests with cljs.test. So I used the friendly cemerick.cljs.test until I came across your plugin and was quite happy. However, a setup very similar to my cemerick.cljs.test setup before gives me this error:

;; ======================================================================
;; Testing with Slimer:
Warning: Error unmounting:
Error: Invariant Violation: unmountComponentAtNode(...): Target container is not a DOM element.
[JavaScript Error: "Error: Invariant Violation: _registerComponent(...):
Target container is not a DOM element." {file: "…/resources/public/js/test/test-main.js" line: 20314}

It seems like there's some problem with React, even though the one test I have now uses only plain clojure.core functions.

As I said, testing with cemerick.cljs.test worked. And I could run the tests directly in my application with run-tests, which rules out incompatibilities between cljs.test and cemerick.cljs.test as the cause.

The relevant section of project.clj looks like this:

:cljsbuild
{:builds
 [{:id "test"
    :source-paths ["src/cljs" "test/cljs"]
    :compiler {:output-to "resources/public/js/test/test-main.js"
               :output-dir "resources/public/js/test"
               :main 'theatralia.t-runner
               :optimizations :whitespace}]}

Can you help?

Right now, you have to manually require namespaces before you can pass them to the doo-tests macro. It seems like the doo-tests macro could produce a (require ~@namespaces) forms?

It would be good to have CI to run tests against doo to prevent regressions. One way to approach it would be to run tests which are multiple projects * all optimisations * all runners * Windows and Linux. Travis has a good test matrix capability to support this.

First off, thanks a lot for doo; it's a fantastic tool :) While it works really well for development (since it watches and automatically runs the build), is there a way to get it to just run once, i.e. for CI?

Perhaps my googling was just weak, but was unable to find a good shim to use Rhino with {:optimizations :none, :main 'some.ns} style builds, or to alter the builds to "just work" with Rhino.

Came up with a very hacky modification (paste at the top) to the runner script to get this working:

global = this;

this.console = {log: print};

var base = new java.io.File(arguments[0]).getParent();

this.document = {};

this.document.getElementsByTagName = function () {
  return [{src: (base + '/output.out/goog/base.js')}];
}

this.document.write = function (source) {
  var src = source.match(/^<script src="([^"]+)"/)
  if (src) {
    var src = src[1];
    if ('/' !== src[0]) src = base + '/' + src;
    load(src);
  } else {
    var code = source.substring(8, source.length - 9);
    eval(code);
  }
};

this.CLOSURE_IMPORT_SCRIPT = function(src) { load(src); };

Note that {:output-dir ...} from the CLJS compilation options is used (know you were interested in possible uses for other params).

... obviously hope there is a better way however!

Would you be open to supporting non lein-cljsbuild workflows that use :doo or :cljsbuild in project.clj? Considering that cljs.test comes with clojurescript, this library could be a go to for the bare metal workflow dnolen is encouraging with mies. Depending on if we use :doo or :cljsbuild, we would have to detect the presence of lein-cljsbuild. As for reimplementing cljsbuild fns, config seems trivial to implement if we assume the latest builds format. subproject is a little involved but not much inlining.
I'm happy to take a stab if desired

When running tests on doo, it'd be ideal to have a way to inject some Javascript libraries. For instance, Jayq expects you to have linked to the jQuery library from your page, so we'd need to inject it into the tests.

I'm using phantom, but expect this is something desirable for others as well.

It would make it easier to extend them, for example by improving the error messages on what namespaces were properly loaded.

I have a then.clj file in the test/project directory (as opposed to src/project), but when I require it I get

java.io.FileNotFoundException: Could not locate project/then__init.class or project/then.clj on classpath.

I know the test directory is getting included in the build because it finds the .cljs files. Is there something special I need to do to get .clj files outside src to be found by the ClojureScript compiler?

hi!

this is my first time opening a github issue, please excuse me if i'm doing any of this wrong.

my build on travis-ci just started failing with the message in this issue's subject line. full traceback at https://s3.amazonaws.com/archive.travis-ci.org/jobs/82362310/log.txt . i'm unable to reproduce this locally, but that's probably because i'm not doing a good enough job of blowing away my currently downloaded libraries or something, i imagine that travis is running the most recent version of doo while i'm running an older version.

i see the string "(update" occur in 9c60f64 , and this just started happening for me, so i'm betting that this behavior was introduced in that commit.

i wish i could give you easy repro steps - is this not happening on your machine?

thanks!

See laurentj/slimerjs#278

CI environments like CircleCI cache m2 dependencies to speed up build time and decrease load on third party Maven repos (like Clojars). They will typically do the following:

1. Fetch all dependencies
2. Cache the dependencies
3. Run the tests

When using doo, it adds more dependencies to the project at test time, after the dependencies have been cached. There are a few possible options here to avoid this:

1. Create some kind of command like lein doo deps which would run the project with the doo deps added, triggering leiningen to download the missing dependencies
2. Add [doo "0.1.5-SNAPSHOT"] to the project dependencies to make sure the deps are included. I'm not familiar enough with doo to know if that is going to cause other problems
3. Make lein deps download the doo dynamic dependencies as well. I really don't know if this is possible or not.

2 seems like the good short term option. It would also be good to document this limitation so others know how to work around it.