Giter Club home page Giter Club logo

clairvoyance's Introduction

Clairvoyance

Clairvoyance is an extension to Specs2, a Scala test library. Clairvoyance provides a few extensions to capture what is happening in your tests, and then produce business and tester friendly documentation.

It's a pastiche of Yatspec, a Java testing library written by my colleague Dan Bodart. It addresses the deficiencies we experienced with Fit and Concordion.

Build Status Coverage Status codecov.io Join the chat at https://gitter.im/rhyskeepence/clairvoyance

Example

class LoggingExample extends ClairvoyantSpec {
  "The coordinator" should {
    "invoke the Doomsday Device on the 21st of December 2012" in new context {
      givenTheDateIs("21/12/2012")
      whenTheCoordinatorRuns
      theDoomsdayDevice should beUnleashed
    }
  }

  trait context extends ClairvoyantContext {
    // test set up and fixtures
  }
}

It breaks down like this:

  • Create a Spec which extends ClairvoyantSpec
  • Write the spec in the mutable spec style (for various historical reasons)
  • Create a context which extends ClairvoyantContext
  • InterestingGivens can be added with statements such as interestingGivens += ("Current date" -> "21/12/2012")
  • The Scala code within the spec method is interpreted into a text specification, to encourage readability.

The full source to this example is here.

Here is the output of this spec. Example output

Get This Party Started

Add this to your SBT build:

// for Scala 2.10 and above:
libraryDependencies += "com.github.rhyskeepence" %% "clairvoyance-specs2" % <latest-version>

// for scala 2.9.x:
libraryDependencies += "com.github.rhyskeepence" %% "clairvoyance" % "27"

resolvers += "releases" at "http://oss.sonatype.org/content/repositories/releases"

Or in Maven:

<dependency>
    <groupId>com.github.rhyskeepence</groupId>
    <artifactId>clairvoyance-specs2_2.10</artifactId>
    <version>${latest-version}</version>
    <scope>test</scope>
</dependency>
...
<repository>
    <id>sonatype-releases</id>
    <name>sonatype releases</name>
    <snapshots>
        <enabled>false</enabled>
    </snapshots>
    <url>http://oss.sonatype.org/content/repositories/releases</url>
</repository>

Interesting Givens

These are inputs into your test, which may not be specified in the spec, but should be logged to the output:

  interestingGivens += ("Current date" -> "21/12/2012")

or

  ("Current date" -> "21/12/2012").isInteresting

Captured Inputs And Outputs

These are the inputs or outputs to your system, which may not be practical to assert upon, but should be logged.

Perhaps you are using a stub rather than communicating with a third party in your spec:

class StubGizmometer extends Gizmometer {
}

To capture inputs and outputs, just add the ProducesCapturedInputsAndOutputs trait and call captureValue:

class StubGizmometer extends Gizmometer with ProducesCapturedInputsAndOutputs {
  def scan(brain: Brain) {
    captureValue("Brain" -> brain)
  }
}

and in your context, register the stub so that clairvoyant knows about it:

trait context extends ClairvoyantContext {
    val gizmometer = new StubGizmometer
    override def capturedInputsAndOutputs = Seq(gizmometer)
}

The value that is captured can be of any type. XML is formatted nicely, and HTML can be inlined into the output by capturing a value in a Html type:

captureValue("Inline HTML" -> Html(<em>Hello!</em>))

All other values are displayed as their String representation, unless a custom renderer has been defined.

Custom Rendering of Interesting Givens & Captured Inputs And Outputs

When you capture a value or an interesting given, it will be rendered to the screen. XML and Strings are formatted nicely by default, but you may wish to capture your own domain objects and have them presented in readable format.

A full example is here: [clairvoyance/specs2/examples/CustomRenderingExample.scala] (https://github.com/rhyskeepence/clairvoyance/blob/master/specs2/src/test/scala/clairvoyance/specs2/examples/CustomRenderingExample.scala)

The juicy bits are shown below:

class CustomRenderingExample extends ClairvoyantSpec with CustomRendering {

  def customRendering = {
    case Brain(iq) => "a Brain with an IQ of %d".format(iq)
  }
}

customRendering is a partial function, which will be run before the default rendering.

And behold, custom rendering of Brains:

Custom Rendering of Brains

Sequence Diagrams

If your spec describes interactions between many systems, it can be nice to generate a sequence diagram automatically from CapturedInputsAndOutputs. Just add the SequenceDiagram trait to your context, ie:

trait context extends ClairvoyantContext with SequenceDiagram {
  override def capturedInputsAndOutputs = Seq(system_x, system_y)
}

The name of the captured values should be in the following formats in order to appear on the diagram:

captureValue("SOMETHING from X to Y" -> ...) or

captureValue("SOMETHING from X" -> ...) or

captureValue("SOMETHING to Y" -> ...)

In the last two cases, the default actor will be used, which can be set using this statement in the context: override def defaultSequenceDiagramActor = "Name of my component"

An example can be found here, which produces the following output:

Sequence Diagram

Alternatively, a graph can be produced:

trait context extends ClairvoyantContext with Graph

Markdown

Markdown is supported in specification descriptions, to whet your appetite see this example.

OTHER COOL STUFF!!!

TODO

  • Scenario tables

Contributing

clairvoyance's People

Contributors

ajluker avatar channingwalton avatar franckrasolo avatar gitter-badger avatar raymanoz avatar rhyskeepence avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

clairvoyance's Issues

Make ScalaTest reports optionally more concise

Reports can rapidly become cluttered by a combination of:

  • too many tests / examples
  • overly verbose literate specifications
  • too many captured values (interactions between components)

The above items are more likely to exist in software projects that have been in development for quite some time and that used ScalaTest extensively for integration and acceptance testing prior to integrating Clairvoyance.

It should be possible to selectively skip either specifications, interactions, or both for any given test, example, or specification.

HTML reporting for ScalaTest hangs for some multiline string literals

Whenever a ScalaTest spec contains a multiline string literal that is not immediately followed by a call to stripMargin, the HTML reporter hangs indefinitely, as shown by the following stack trace from a thread dump:

"ScalaTest-dispatcher" #11 prio=5 os_prio=31 tid=0x00007fd3a215b800 nid=0x6603 runnable [0x0000000135c5b000]
java.lang.Thread.State: RUNNABLE
at java.util.regex.Pattern$Curly.match0(Pattern.java:4268)
at java.util.regex.Pattern$Curly.match(Pattern.java:4234)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4658)
at java.util.regex.Pattern$Loop.match(Pattern.java:4785)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4717)
...
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4658)
at java.util.regex.Pattern$Loop.matchInit(Pattern.java:4804)
at java.util.regex.Pattern$Prolog.match(Pattern.java:4741)
at java.util.regex.Pattern$Slice.match(Pattern.java:3972)
at java.util.regex.Pattern$Start.match(Pattern.java:3461)
at java.util.regex.Matcher.search(Matcher.java:1248)
at java.util.regex.Matcher.find(Matcher.java:637)
at scala.util.matching.Regex.findFirstMatchIn(Regex.scala:250)
at clairvoyance.export.NaiveMultilineMethodNameJoiner$.clairvoyance$export$NaiveMultilineMethodNameJoiner$$doStripLine(NaiveMultilineMethodNameJoiner.scala:20)
at clairvoyance.export.NaiveMultilineMethodNameJoiner$$anonfun$join$1.apply(NaiveMultilineMethodNameJoiner.scala:5)
at clairvoyance.export.NaiveMultilineMethodNameJoiner$$anonfun$join$1.apply(NaiveMultilineMethodNameJoiner.scala:4)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at clairvoyance.export.NaiveMultilineMethodNameJoiner$.join(NaiveMultilineMethodNameJoiner.scala:4)
at clairvoyance.export.FromSource$.getCodeFrom(FromSource.scala:29)
at clairvoyance.scalatest.export.ScalaTestHtmlFormat.getCodeFrom(ScalaTestHtmlFormat.scala:119)
at clairvoyance.scalatest.export.ScalaTestHtmlFormat.clairvoyance$scalatest$export$ScalaTestHtmlFormat$$renderFragmentForBody(ScalaTestHtmlFormat.scala:107)

ScalaTestHtmlFormat dies when rendering multi-line described ScalaTest tests

If you have a test that looks like this:

test(
  """Summary
    |-------
    |
    |Do some stuff
  """.stripMargin) {
  givenYourSetup()
  itShouldFail()
}

The ScalaTestHtmlFormat dies with:

Reporter completed abruptly with an exception on invocation of the dispose method.
java.lang.IndexOutOfBoundsException: -1
    at scala.collection.immutable.Vector.checkRangeConvert(Vector.scala:137)
    at scala.collection.immutable.Vector.apply(Vector.scala:127)
    at clairvoyance.export.FromSource$.getCodeFrom(FromSource.scala:28)
    at clairvoyance.scalatest.export.ScalaTestHtmlFormat.clairvoyance$scalatest$export$ScalaTestHtmlFormat$$renderFragmentForBody(ScalaTestHtmlFormat.scala:109)
    at clairvoyance.scalatest.export.ScalaTestHtmlFormat$$anonfun$printBody$1.apply(ScalaTestHtmlFormat.scala:79)
    at clairvoyance.scalatest.export.ScalaTestHtmlFormat$$anonfun$printBody$1.apply(ScalaTestHtmlFormat.scala:77)
...

It looks like there is a bug in FromSource. Line 27 tries to get the line that is the name of the test, but this will only work (as it is currently implemented) if the test name has no new line chars.

Make the dependency on specs2 more explicit throughout

The project's README file and the base package name already clearly state the dependency on specs2. However, given that we agree that it would be useful to the wider community to be able to use Clairvoyance with ScalaTest, I suggest doing the following:

1. rename the artifact from clairvoyance to clairvoyance-specs2 (related to #4)
2. rename org.specs2.clairvoyance to clairvoyance
3. abstract away clairvoyance.export
4. push down specs2 implementations to clairvoyance.specs2 and clairvoyance.specs2.export

InterestingGivens is accumulating values between tests, in ScalaTest

If I write a test that has more than 1 test in it, each test adds values to the interesting given and they accumulate between the tests such that each test's interesting givens shows it's values and the values from all the tests before it.

This is due to ClairvoyanceContext being mixed in at the class level in the ScalaTest flavour of Clairvoyance tests.

This can be fixed by mixing in OneInstancePerTest, but I feel that even without mixing this trait into your tests, InterestingGivens should be cleared down between each test.

Any chance this project will be updated?

Through sheer coincidence I stumbled upon this project and I think there's a lot of good stuff in here! I especially like the idea of generated sequence diagrams. Any chance you'll find some time to update this for current specs2 and Scala versions?

Get Travis to build Clairvoyance

Currently, Clairvoyance is built manually from the command line.

This issue is about getting Clairvoyance continuously built by Travis.

I will raise a separate issue for publishing build artifacts, which Travis does not yet support out of the box.

Specs2 methods are not completely extracted when they contain an equals sign

With a test method of the following:
"test name blah" in new context {
saveTrade(tradeId = "ABCDEFGHIJ", book = "EXOCMS", etc etc)
etc etc
}

the test method isn't extracted correctly. I've narrowed it to the following statement in FromSource.scala:

content(lineNumber).trim().matches(".+ =\s+[^\{]+")

I guess this may be required to parse scalatest files, but I'd like to just remove this block and do a new release for specs2 users. The longer term fix might be to have separate Specs2 and ScalaTest source file readers.

Publish Clairvoyance JARs to Sonatype

Just like #3, though independently of it, this issue is about extending the SBT build to explicitly publish the Clairvoyance JAR files to Sonatype.

This mechanism will eventually be used to publish both clairvoyance-specs2 and clairvoyance-scalatest build artifacts.

TeamCity output contains spurious character

When running the build using TeamCity you no longer see test results. There is also a warning in the build log:

teamcity[testStarted name='XXX'}]

[Step 1/1] Property value not found
Valid property list format is (name( )=( )'escaped_value'( )) where escape symbol is "|"

This is because there is a spurious curly brace on line 55:

println(s"##teamcity[$messageName $formattedAttributes}]")

Add ScalaTest support

Once #5 is done, we can focus on adding support for ScalaTest so that users are not forced to write tests using specs2. This would require doing the following:

1. create ScalaTest implementations in clairvoyance.scalatest and clairvoyance.scalatest.export
2. cross-build and publish the artifact named clairvoyance-scalatest

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.