spekframework / spek Goto Github PK

I tried to configure sample maven project with spek tests runnable from IDEA and during maven build but had no luck.

What I tried:

1. configured sample maven project with kotlin JUnit test similar to https://github.com/JetBrains/kotlin-examples maven/mixed-code-hello-world. Tests runs fine both with IDEA (ultimate 14.0.2) and maven test (from command line).
2. added spek dependency as described at http://jetbrains.github.io/spek/download.html version 0.1-SNAPSHOT was used. Btw: versions like 0.127 or 0.145 mentioned on the page do not exist - should be like 0.1.127 or 0.1.145
3. added spek test

import kotlin.test.assertEquals
import org.jetbrains.spek.api.Spek

class BlaherSpecs: Spek() {{

    given("Let's test Blaher") {
        val blaher = Blaher()
        on("Blaher blah") {
            val blah = blaher.blah()
            it("should be Blah!") {
                assertEquals("Blah1!", blah)
            }
        }
    }
}}

it didn't compile because maven dependency had <type>pom</type> - compliled after commenting that out.

1. Tried to run it with IDEA runner - got

/bin/bash /Users/admin/Projects/kotlin/src/test/java/SpecSample.kt
/Users/admin/Projects/kotlin/src/test/java/SpecSample.kt: line 1: import: command not found
/Users/admin/Projects/kotlin/src/test/java/SpecSample.kt: line 2: import: command not found
/Users/admin/Projects/kotlin/src/test/java/SpecSample.kt: line 4: syntax error near unexpected token `('
/Users/admin/Projects/kotlin/src/test/java/SpecSample.kt: line 4: `class BlaherSpecs: Spek() {{'

When building with maven it looks like SpecSample gets compiled

[info] Kotlin Compiler version 0.10.372
[info] Compiling Kotlin sources from [src/test/java]
[info] Classes directory is /Users/admin/Projects/kotlin/target/test-classes
[debug] Removed target directory from compiler classpath (/Users/admin/Projects/kotlin/target/test-classes)
[info] Classpath: src/test/java:/Users/admin/Projects/kotlin/target/classes:/Users/admin/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib/0.10.195/kotlin-stdlib-0.10.195.jar:/Users/admin/.m2/repository/org/jetbrains/kotlin/kotlin-runtime/0.10.195/kotlin-runtime-0.10.195.jar:/Users/admin/.m2/repository/junit/junit/4.10/junit-4.10.jar:/Users/admin/.m2/repository/org/hamcrest/hamcrest-core/1.1/hamcrest-core-1.1.jar:/Users/admin/.m2/repository/org/jetbrains/spek/spek/0.1-SNAPSHOT/spek-0.1-SNAPSHOT.jar
[info] Classes directory is /Users/admin/Projects/kotlin/target/test-classes
[debug] Scanning for kotlin annotations in /Users/admin/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib/0.10.195/kotlin-stdlib-0.10.195.jar
[debug] Scanning for kotlin annotations in /Users/admin/.m2/repository/org/jetbrains/kotlin/kotlin-runtime/0.10.195/kotlin-runtime-0.10.195.jar
[debug] Scanning for kotlin annotations in /Users/admin/.m2/repository/junit/junit/4.10/junit-4.10.jar
[debug] Scanning for kotlin annotations in /Users/admin/.m2/repository/org/hamcrest/hamcrest-core/1.1/hamcrest-core-1.1.jar
[debug] Scanning for kotlin annotations in /Users/admin/.m2/repository/org/jetbrains/spek/spek/0.1-SNAPSHOT/spek-0.1-SNAPSHOT.jar
[info] Using kotlin annotations from /Users/admin/.m2/repository/org/jetbrains/kotlin/kotlin-jdk-annotations/0.1-SNAPSHOT/kotlin-jdk-annotations-0.1-SNAPSHOT.jar
[debug] Using Kotlin home directory <no_path>
[debug] Configuring the compilation environment
[debug] Output:
/Users/admin/Projects/kotlin/target/test-classes/BlaherSpecs.class
Sources:
/Users/admin/Projects/kotlin/src/test/java/SpecSample.kt
[debug] Output:
/Users/admin/Projects/kotlin/target/test-classes/BlaherSpecs$1.class
Sources:
/Users/admin/Projects/kotlin/src/test/java/SpecSample.kt
[debug] Output:
/Users/admin/Projects/kotlin/target/test-classes/BlaherSpecs$1$1.class
Sources:
/Users/admin/Projects/kotlin/src/test/java/SpecSample.kt
[debug] Output:
/Users/admin/Projects/kotlin/target/test-classes/BlaherSpecs$1$1$1.class
Sources:
/Users/admin/Projects/kotlin/src/test/java/SpecSample.kt

but SpecSample test does not run - it should fail, but build succed.

I want to setup spek project with maven and IDEA so that

1. I can run spek tests from IDEA - it works with this project spek-samples, so I assume it can be done. spek-samples is on gradle and spek dependency is compile project(':spek-core') though.
2. Spek tests get run on maven build - I found no description or working sample how to achive that.

Sample project available at https://github.com/nhekfqn/kotlin-spek-maven-sample

If you help me to configure sample project, I will publish working version on github so that others can use it.

Thanks in advance.

Thats gonna be nice to allow running pre or post tasks before or after each on, e.g.

         given("aaa") {
              beforeOn { setupTheTest() }
              afterOn { tearDownTheTest() }

              on("aaa") { it("should be called after beforeOn closure") }
              on("bbb") { it("should be called afterOn and beforeOn prior to this") }
         }

I don't like having to do { init { } } instead of {{ }}, have you guys any ideas about how you might be able to land on a clean syntax again?

I've setup a test repo for this issue here.

When I run Spek suites in IntelliJ IDEA, the reports looks awesome:

Suite class's name is visible alongside with it's package. The sings are aligned in tree. Just great.

But when I run tests in Gradle (./gradlew test) the report (build/reports/tests/index.html) is not so good:

• Everything is put inside default-package, which is not true.
• given / describe / on blocks appear in one list.

I understand that reports are not supposed to support custom tree structures, but wouldn't it be better if describe goes to package name and all ons being grouped by them?

Also, XML reports should be tweaked too to improve reporting on CI...

Somehow it is possible to end up with run configurations that throw up this error:

java.lang.NoSuchMethodError: kotlin.properties.Delegates.lazy(Lkotlin/jvm/functions/Function0;)Lkotlin/properties/ReadOnlyProperty;

This sure looks like a classloader problem. Was able to slink out of this a few times by deleting them then recreating them.

I created a spek test somewhere in the androidTest package. How do I include it in the test APK? Or how to run it at all? The website states that you can "just run it" in IntelliJ, how do I "just run it" in Android Studio without getting an "Empty test suite" error?

Basically I'm looking for a quick Android tutorial for Spek.

I noticed current config requires Java 1.7. Why? Unless we have any prerequisites for that, let's consider 1.6 that my expand applicability of the library

Is there an equivalent for the following JUnit?

@RunWith(SpringJUnit4ClassRunner::class)
@ContextConfiguration(classes = arrayOf(Application::class), loader = SpringApplicationContextLoader::class)
@WebIntegrationTest

Expected: on runs before afterOn
Actual: afterOn runs before on

Sample:

given("") {
    beforeOn { print("1") }
        on("") {
            print("2")
            it("") {
                print("3")
            }
    }
    afterOn { print("4") }
}

output:
1423

Is it possible to write tests where specific exception is expected, like using @Test(expected = UnhandledException::class) from JUnit ?
Thanks!

Currently, Spek doesn't play well with Robolectric 😞

The reason is that Robolectric requires a BlockJUnit4ClassRunner, but Spek requires its own JUnitClassRunner. The exception thrown is:

java.lang.Exception: No runnable methods
    at org.junit.runners.BlockJUnit4ClassRunner.validateInstanceMethods(BlockJUnit4ClassRunner.java:191)
    at org.junit.runners.BlockJUnit4ClassRunner.collectInitializationErrors(BlockJUnit4ClassRunner.java:128)
    at org.junit.runners.ParentRunner.validate(ParentRunner.java:416)
    at org.junit.runners.ParentRunner.<init>(ParentRunner.java:84)
    at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:65)
    at org.robolectric.RobolectricTestRunner.<init>(RobolectricTestRunner.java:78)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
    at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
    at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
    at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

Would it be possible to support framework like Robolectric?

Since Kotlin M12 calling the constructor should be done using the init keyword.

It is not indicated in the doc : http://jetbrains.github.io/spek/docs.html

Guys,

You think we can set a date for a 0.1 release of this?

Currently we have 2 main approaches to define a test:

     class Speks : ConsoleTest() {{
         given("...") { ... }
     }}  //note double }} and {{

The next way is JUnit compatible:

     class Speks : JUnitSpek() {{
         given("...") { ... }
     }}

There are following questions:

• do we like the syntax in general?
• what is the best name for base class?

In v1.0.0, given and on are both aliases to describe.
They should be aliases that prefix the description with their keyword.

given(description, block) should translate to given("Given " + description, block)
on(description, block) should translate to on("On " + description, block)

Currently, when you use given("a calculator") {}, the IDEA test runner shows a test group that is named "a calculator" while it should be named "Given a calculator".

The IDE support for showing test results in a given > on > it tree is great, it would be cool if double-clicking on a failing line here took you to the failing code block in question.

Please build new version for the Kotlin M11

Windows 10, Chrome 49.0.2623.110 m

Hi there,
we trying to figure out how to test async code with Spek. Does Spek offer any facilities to test async code? With Quick we use waitUntil and toEventually.

Hi,

having looked at JUnitSpek and ConsoleSpek, I found they are almost identical.
the only difference is the "RunWith(javaClass<JSpec<*>>())" in the JUnitSpek.

I was thinking, we can merge them and have one parent Spek class only, which is of course annotated with "RunWith(javaClass<JSpec<*>>())".

By this approach, there will be one type of Specks only, But user can decide to whether run them with JUnit or Console Runner.

what do you think ?

Ability to run the same specification, with different input values.

Most of the time you write specifications for a single class. Normally you would create a nullable local variable of that type and instantiate the said class in a beforeEach clause. With subjects we lessen the boilerplate, each test (it) will get their own instance of the subject.

class CalculatorSpec: SubjectSpek<SampleCalculator>({
    subject { SampleCalculator() }

    it("should return the result of adding the first number to the second number") {
        val sum = subject.sum(2, 4)
        assertEquals(6, sum)
    }

    it("should return the result of subtracting the second number from the first number") {
        val subtract = subject.subtract(4, 2)
        assertEquals(2, subtract)
    }
})

In order to see a better coverage report,
A developer wants to see the execution results in HTML format.

Could you upload spek to the central maven repository as well?
The problem is that if I build an app tested with spek, then I will not be able to upload to central because from central repository all artifacts depending on external repositories are rejected. see [1]

Thank you!

1: http://blog.sonatype.com/2010/03/why-external-repos-are-being-phased-out-of-central/

Currently only support folders containing .class files. Need to add (fix) .jar support.

When using JUnit, I can tell IntelliJ to run a single test by right-clicking on it in the test results window or by right clicking on the test method name in the text area. With Spek tests, I am unable to do either of these things and it appears I can only run entire test classes (not individual given, it or on blocks). When all of the tests are passing this is fine. However, when I have a failing test that I want to debug this means I have to either comment out all other tests in the class or do some other kind of code-level test ignoring in order for my break points to not be hit when other (passing) tests in the suite are run.

Sadly, the inability to run single tests in IntelliJ has driven me away from Spek and back to JUnit for the time being. I'll watch this issue though as I would like to go back at some point. :)

Does the opposite:

shouldNotBeNull(agentCommission)

Yields this result:

  java.lang.AssertionError: . Expected value to be null, but was: <Payout@326de728>.
    at kotlin.test.DefaultAsserter.fail(TestJVM.kt:79)
    at kotlin.test.Asserter$DefaultImpls.assertTrue(Test.kt:112)
    at kotlin.test.DefaultAsserter.assertTrue(TestJVM.kt:73)
    at kotlin.test.Asserter$DefaultImpls.assertNull(Test.kt:149)

Opposite of what is expressed.

What is the main project model that is used for the project?

Currently I see .gradle and .idea project. What should I open in IDEA to start?

There is a dependency in the POM against spek-core and a unspecified version on jetbrains repo.

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.spek</groupId>
  <artifactId>spek</artifactId>
  <version>0.1-SNAPSHOT</version>
  <dependencies>
    <dependency>
      <groupId>spek</groupId>
      <artifactId>spek-core</artifactId>
      <version>unspecified</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>spek</groupId>
      <artifactId>spek-tests</artifactId>
      <version>unspecified</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
</project>

Error:Conflict with dependency 'org.jetbrains.kotlin:kotlin-stdlib'. Resolved versions for app (1.0.1) and test app (1.0.0) differ. See http://g.co/androidstudio/app-test-app-conflict for details.

I saw some people talking about excluding the runtime from the test import. But since it's not in the docs, and I just spent ~ half an hour trying to add this one dependency to an Android project, going to go back to JUnit instead.

Why are releases not made on this project?

I just updated to the latest version of spek 1.0.9 and now the Spek constructor wants a body.

I have an Android library in Kotlin and I want to start to bulid my test clases with Spek and probably with Mockito to mock a Context. I'm trying all day long but it not works.

Can Spek works with Android?

https://jetbrains.github.io/spek/download

<repositories>
    <repository>
      <id>jebrains-all</id>    <-- HERE
      <url>http://repository.jetbrains.com/all</url>
    </repository>
  </repositories>

Sometimes you are just checking the static state of something and not really performing an action. For example, something simple like my API exports a constant for use by users and I want to just verify its value.

In English, I basically want to say something like this:

given("MY_CONSTANT")
{
     it("Should have the value 5") { shouldEqual(5, MY_CONSTANT) }
}

But I am forced to add an on in there, which isn't really needed since nothing is being done. You basically end up saying something unncessary like

given("MY_CONSTANT")
{
     on("inspection") { it("Sould have the value 5") { shouldEqual(5, MY_CONSTANT) } }
}

Sometimes it's convenient to reuse specs - like testing a subclass (e.g Calculator and AdvancedCalculator).

DSL

class CalculatorSpec: SubjectSpek<Calculator>({
    ...
});

class AdvancedCalculatorSpec: SubjectSpek<AdvancedCalculator>({
    includeSubjectSpek(CalculatorSpec::class) // or an extension fun itBehavesLike(CalculatorSpec::class)
});

In KSpec we something like this:

class CalculatorSpec: KSpec() {
    override fun spec() {
        describe(Calculator::class) {
            itBehavesLike(calculator())
        }
    }

    companion object {
        fun calculator() = sharedExample<Calculator> {
            describe("add") {
                it("1 + 1 = 2") {
                    assertThat(subject.add(1, 1), equalTo(2))
                }
            }
            ...
        }
    }
}

class AdvancedCalculatorSpec: KSpec() {
    override fun spec() {
        describe(AdvancedCalculator::class) {
            itBehavesLike(CalculatorSpec.calculator())
        }
    }
}

Depends on #87.

Do we have one?

Tested on 0.1-SNAPSHOT (0.1-20140926.162946-6)

When there is an exception thrown inside "on" clause, IntelliJ shows me "Tests were not started".
Furthermore, other given clauses don't run at all.

Expected behavior:

• exception in one clause shouldn't affect the others
• IntelliJ test runner should mark only failed clauses as failed

A custom JUnit 5 engine needs to be implemented, we should preserve the current behavior as much as possible.

Any chance we'll be seeing a maven drop that includes 1ac8fe2?

(I realise I could use the snapshot version, but I'd rather keep my kotlin dependency on a stable release)

Since spek has no release tags, users have to open http://repository.jetbrains.com/all/org/jetbrains/spek/spek/ to find the latest version of spek. It it has tags, users can know about version of spek without opening the maven repo of JetBrains.

Actual: All Ons executes before Its.
Expected: Its executes after his Ons.
Kotlin M8 and Spek 0.127

class SpekSpecs: Spek() {{
    given("given1") {
        var log = ""
        on("on1") {
            log += "on1"
            it("it1 goes after on1") {
                log += "it1"
                shouldEqual("on1it1",log)
            }
        }

        on("on2") {
            log += "on2"
            it("it1 goes after on1 and it2 goes after on2") {
                log += "it2"
                shouldEqual("on1it1on2it2",log)
            }
        }
    }
}}

Correct me if I'm wrong, but it seems like Intellij doesn't support Spek as a first-class testing framework.

Ctrl+Shift+T -> Create New Test... -> Testing library

Spek is not on the list.

• The open bracket in val calculator = Calculator() { should be in the next line.
• Also there is an extra bracket in val sum = calculator.sum(2, 3) {.

When I'm selecting "Run Tests in <package_name> with Coverage" in Android Studio.
The test coverage report works as long as I don't run any tests written with Spek.
Including Spek tests the report is showing 0 % coverage everywhere.

I fear loosing coverage reports is a K.O. criteria for using Spek for me. Do you know how to fix that?

I've changed TimingSpecification test so that it fails when running with IDEA runner, than tried to run

./gradlew test

command from root of the project. Build succed. It looks like tests are not run while building project with gradle.

We need to enumerate all classes in classpath, getting +2mb for jar size is too much. Could use something like this: https://github.com/ddopson/java-class-enumerator

User should be able to add "given", "on" and "it" without implementation. this is useful when he/she wants to create the scenarios only.

given("a fact, i need to do something about it")   //No action

given("a fact") {
     on("an event, there will be some results, but they are not clear yet")  //No action
}

given("a fact") {
      on("an event") {
          it("should result to ..., but is not clear yet") //No action
      }
}

An Ignore annotation (@ignore in JUnit) or something similar would be very beneficial to skip
temporary some of the tests.

It must be operate on:
- whole spec instance
- a specific "given"
- a specific "On"
- a specific "It"