jgleitz / testfiles Goto Github PK

View Code? Open in Web Editor NEW

7.0 3.0 1.0 2.22 MB

Manage test files and directories neatly when testing with Spek or Kotest!

License: MIT License

Kotlin 100.00%

kotlin spek2 spekframework testing kotest kotest-plugin

testfiles's Introduction

testfiles

A test helper to easily create files and directories for testing purposes.

Organises test files according to the test structure
Cleans up test files, but leaves them in place if the test fails, so you can examine them
Generates random file names that are consistent each test run
Supports Spek and Kotest

Dependencies

Framework	Link	Dependency
Spek		`de.joshuagleitze:spek-testfiles:<version>`
Kotest		`de.joshuagleitze:kotest-files:<version>`

Usage

Spek

Create a testFiles instance for your Spek by using the testFiles() function:

import de.joshuagleitze.testfiles.DeletionMode.*
import de.joshuagleitze.testfiles.spek.testFiles
import org.spekframework.spek2.Spek
import org.spekframework.spek2.style.specification.describe

object ExampleSpek: Spek({
	val testFiles = testFiles()

	describe("using test files") {
		it("generates file names") {
			testFiles.createFile()
			testFiles.createDirectory()
		}

		it("cleans up files") {
			testFiles.createFile("irrelevant", delete = Always)
			testFiles.createFile("default mode", delete = IfSuccessful)
			testFiles.createFile("output", delete = Never)
		}
	}
})

Kotest

Use the global testFiles property:

import de.joshuagleitze.testfiles.DeletionMode.*
import de.joshuagleitze.testfiles.kotest.testFiles
import io.kotest.core.spec.style.DescribeSpec

class ExampleSpek: DescribeSpec({
	describe("using test files") {
		it("generates file names") {
			testFiles.createFile()
			testFiles.createDirectory()
		}

		it("cleans up files") {
			testFiles.createFile("irrelevant", delete = Always)
			testFiles.createFile("default mode", delete = IfSuccessful)
			testFiles.createFile("output", delete = Never)
		}
	}
})

Test File Directory

testfiles will automatically pick the most appropriate folder to create the files in. Concretely, it uses this logic ( see DefaultTestFiles.determineTestFilesRootDirectory):

Case	Test File Directory	Use Case
`<pwd>/build` exists	`<pwd>/build/test-outputs`	Gradle users
`<pwd>/target` exists	`<pwd>/target/test-outputs`	Maven users
`<pwd>/test-outputs` exists	`<pwd>/test-outputs`	Other users wishing to have the files in the project directory
else	`<tmpdir>/spek-test-outputs`	Fallback

The files in the test directory will be organized into directories according to the test structure. For example, the “output” file from the example above would be created at <test file directory>/[ExampleSpek]/[using test files]/[cleans up files]/output.

Deletion Mode

Per default, testiles will delete created files and directories if the current test or group was successful and retain them if the test or group failed. This allows you to examine test output after tests fails, but does not clutter your test output folder.

The behaviour can be changed by passing the correct DeletionMode to createFile or createDirectory:

`DeletionMode`	behaviour
`Always`	Always delete the created file after the test or group has run.
`IfSuccessful`	Delete the file if the test or group ran through successfully; retain the file if the test or group failed. The default value.
`Never`	always retain the file after the test run.

Retained files will be deleted once the group or test they were created for is executed again.

File Names

If no file or directory name is provided, testfiles will create one. Names are constructed following the pattern test-<number> where <number> is a randomly generated, non-negative integer. The random generator is seeded with the current test directory path to make sure the same test will always get the same file names every execution.

For example, the first generated file from the example above might have the path <test file directory>/[ExampleSpek]/[using test files]/[generates file names]/test-162363182.

If a file name is provided, the name must not be wrapped in angle brackets.

testfiles's People

Contributors

Stargazers

Watchers

Forkers

berkcanerken

testfiles's Issues

Escape Bad Characters in Test Names

Test names can contain all types of characters, however, not all characters are allowed as directory names. spek-testfiles should gracefully handle such cases by escaping offending characters.

Dependency Dashboard

This issue provides visibility into Renovate updates and their statuses. Learn more

This repository currently has no open or pending branches.

Check this box to trigger a request for Renovate to run again on this repository

testfiles fails to be auto-registered in newer Kotest versions

testfiles extension doesn't seem to be auto-registered in newer Kotest versions.

Background

My setup (I think) is quite simple:

build.gradle.kts:

dependencies {
    testImplementation("io.kotest:kotest-runner-junit5::5.6.2")
    testImplementation("de.joshuagleitze:kotest-files:2.0.0")
}

how I'm using it in an example test:

class Test : FreeSpec({

    "Some" - {
        "nested" - {
            "test" {
                val directory = testFiles.createDirectory()
            }
        }
    }
})

However, this failed for me on any subsequent build:

java.nio.file.FileAlreadyExistsException: /home/andrzej/work/xyz/build/test-outputs/test-176899501
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:94)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397)
	at java.base/java.nio.file.Files.createDirectory(Files.java:700)
	at de.joshuagleitze.testfiles.DefaultTestFiles.createDirectory(DefaultTestFiles.kt:31)

Note that the directory is created under root test-outputs and not under test scope like described in README.

The issue

I attempted to debug this and found that Kotest initializes @AutoScan extensions in io.kotest:kotest-framework-engine-jvm, io.kotest.engine.config.applyConfigFromAutoScan.kt:16. However, to my surprise the scanning code doesn't discover testfiles' de.joshuagleitze.testfiles.kotest.KotestTestFilesAdapter object which I believe to be the testfiles' extension for Kotest. Because of that it doesn't register it as a Kotest extension, making testfiles run with ROOT_SCOPE context.

I don't have any environment variables set, Java properties set or Kotest-project-wide configuration that would be related to Kotest configuration.

Suspected reason

I think the reason are annotations:

Kotest during extension scanning uses ClassGraph which uses classfile bytecode for understanding which classes are matches for a given query. It uses the following query:
```
classgraph().scan().use { result ->
    result.getClassesWithAnnotation(AutoScan::class.java.name)
        ...
}
```
The AutoScan here is io.kotest.core.annotation.AutoScan
testfiles' KotestTestFilesAdapter is annotated with io.kotest.core.spec.AutoScan (note the different package), which in newer Kotest versions is a type alias: typealias AutoScan = io.kotest.core.annotation.AutoScan

I think the scanner, due to the different annotation package, doesn't consider testfiles' KotestTestFilesAdapter as a match.

A different problem

I though: hey, doesn't matter. I can use Kotest's project-wide configuration to manually register testfiles as its extension.

This doesn't work however, as testfiles' KotestTestFilesAdapter is internal and therefore I cannot reference it from my code.

Possible solution

Migrate to updated Kotest annotation.
Possibly make KotestTestFilesAdapter public so users can have fine-grained control over registration if they want to.

Returned Paths Should Always be Absolute

spek-testfiles should only ever hand out absolute paths. This avoids strange situations where test file paths cannot be resolved.