Giter Club home page Giter Club logo

atam4j's Introduction

atam4j

Acceptance Tests As Monitors

  • A simple library that allows you to run junit tests as monitoring checks, using the dropwizard framework
  • Runs tests on a schedule and exposes results via a restful api that can be called from your monitoring setup (e.g. nagios, zabbix, icinga, pingdom etc etc)

Latest Build Status

Build Status

Maven Dependency

<dependency>    
   <groupId>me.atam</groupId>    
   <artifactId>atam4j</artifactId>    
   <version>{atam4j.version}</version>    
</dependency>

where atam4j.version is the version number you want to use. The latest version is as shown below
Maven Central

Using atam4j

  1. Include the atam4j maven dependency along with the following required dependencies
<dependency>
    <groupId>io.dropwizard</groupId>
    <artifactId>dropwizard-core</artifactId>
    <version>{dw.version}</version>
</dependency>

where {dw.version} should be replaced by the version number as defined by the dw.version property in the main project pom

  1. Write Junit based tests in the usual manner. You can choose to either add a @Monitor annotation to your test classes and let atam4j detect them or simply supply an array of classes to the builder. This is explained further, below.

  2. Instantiate Atam4j in your main dropwizard application class, as shown below

If specifying an explicit array of Test classes:

new Atam4j.Atam4jBuilder(jerseyEnvironment)     
    .withTestClasses(HelloWorldTest.class) 
    .build()      
    .initialise();

If using @Monitor annotations to auto-detect test classes:

new Atam4j.Atam4jBuilder(jerseyEnvironment)      
    .build()      
    .initialise();     

where jerseyEnvironment is an object of type io.dropwizard.jersey.setup.JerseyEnvironment

  1. Run your app and observe the status of the acceptance tests reported by the /tests endpoint.

Refer to atam4j-sample-app for a complete working example.

Additional usage documentation

Refer to our Wiki

Developing atam4j

Build and Release

Building Locally

mvn clean install

Running Tests using Maven CLI

mvn clean test

Release - Manual

Prerequisite

Only core committers can release atam4j to maven central. You need Sonatype Nexus OSS account info for atam.me.

Steps

  1. Set version of the release
mvn versions:set -DnewVersion=${versionNumber}
  1. Commit the version back to git and push to remote
git commit -a -m "Prepare release v${versionNumber}"    
git push
  1. Tag code
git tag -a v${versionNumber} -m 'version ${versionNumber}'     
git push origin v${versionNumber}
  1. Create release in Github - https://github.com/atam4j/atam4j/releases

  2. Deploy to maven central

mvn clean deploy

Artifact repo

Snapshots

https://oss.sonatype.org/content/repositories/snapshots/me/atam/atam4j/

Releases

Look for non-snapshot version https://oss.sonatype.org/content/groups/public/me/atam/atam4j/

Overview of modules

acceptance-tests

A module which tests atam4j from end to end by starting an application that imports the core-library and runs dummy-tests.

atam4j

The core library that runs tests and exposes the results via an api.

atam4j-annotations

Module containing the atam4j annotations only. If you wish to annotate your tests, this is the only dependency required.

dummy-tests

Contains dummy tests to verify that atam4j reports correctly on the status of tests that we know in advance will pass or fail. These are not run during the build.

test-utils

Contains some common code shared by tests.

atam4j's People

Contributors

anuragkapur avatar gitter-badger avatar jorgepcdv avatar phillbarber avatar tjcutajar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

atam4j's Issues

Make test interval configurable

As a user of atam4j, I want the test interval configurable so that I can run them as often as I see fit for my use-case instead of being forced to settle with a 10min default interval.

Add tests for /tests/{category}

We seem to be missing tests that verify the above endpoint.

On a related note - I like this new feature! I can see it adding a lot of value.

Atam4j not shutting down gracefully

This is having the knock on effect of messing up the logging during our build.

We need to ensure that atam4 shuts down when dropwizard shutsdown

Release process - authentication failures

Continuous delivery script is broken right now.

Example: https://travis-ci.org/atam4j/atam4j/builds/525427686

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare (default-cli) on project atam4j-parent: Unable to commit files
[ERROR] Provider message:
[ERROR] The git-push command failed.
[ERROR] Command output:
[ERROR] Warning: Permanently added the RSA host key for IP address '192.30.253.112' to the list of known hosts.
[ERROR] Permission denied (publickey).
[ERROR] fatal: Could not read from remote repository.
[ERROR] 
[ERROR] Please make sure you have the correct access rights
[ERROR] and the repository exists.
[ERROR] -> [Help 1]

Sample App

As a developer, I want a sample application available which uses atam4j library available as reference to help me see how the library can be used.

Guava dependency issue causing error

The upgrade to dropwizard 1.3.9 has somehow caused a dependency issue when atam4j is imported in other applications.

When the service is started you receive the following error:

java.lang.NoClassDefFoundError: com/google/common/base/MoreObjects

Convert to library

As a developer, I want atam4j to be converted to a java library that can be easily included into existing projects that want to leverage the library to use acceptance tests as monitors.

Additionally, I want a separate Sample application using the atam4j library and demonstrating its use.

One off test run

Hi Guys,

Im an avid user of atam4j (good work btw guys).

One thing I would like to do with my acceptance test dropwizard app is to be able to run a dropwizard command from the command line that runs all of the acceptance tests once, and then exits with a non zero code if any of the tests have failed.

Requests to /tests/{category} return 500 on startup

0:0:0:0:0:0:0:1 - - [11/Mar/2016:18:22:25 +0000] "GET /tests/critical HTTP/1.1" 500 110 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36" 4 ERROR [2016-03-11 18:22:25,665] io.dropwizard.jersey.errors.LoggingExceptionMapper: Error handling a request: 4d8ffc6e3449f089 ! java.lang.NullPointerException: null ! at me.atam.atam4j.resources.TestStatusResource.getTestStatusForACategory(TestStatusResource.java:42) ~[atam4j-0.51.jar:na] ! at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[na:na] ! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_66] ! at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66] ! at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[jersey-server-2.21.jar:na]

Groovy + Spock tests - NPE

Scenario:
A groovy + spoc test class, throws an Error in the setup phase, before individual test methods get a chance to run.

This leads to testFailure method of TestRunListener being called before testStarted is called. The code in the TestRunListener class expects testStarted to be called before testFailure instead. If this assumption is violated the testFailure method throws a NPE as the relevant IndividualTestResult object is not found in the inProgressTestresults Map.

Ref: https://github.com/atam4j/atam4j/blob/7a2137b41f71bbb7410a594f908ac25e460fb9ff/atam4j/src/main/java/me/atam/atam4j/TestRunListener.java

Implement tests

Following classes have no test coverage. This needs to be addressed!

  • AcceptanceTestsRunnerTask
  • AcceptanceTestsHealthCheckManager
  • AcceptanceTestsHealthCheck

Report on what tests ran

This is a tricky one.

Currently we get the message "All is ok" or "Number of failures N", when the tests have run. There's no way to determine which tests ran.

Do we add this to the healthcheck message? Do we expose this on a different end-point?

Possible NullPointerException

This may not actually be an atam4j problem, but I'm seeing an occasional one of these log messages:

INFO  [2016-04-06 15:03:13,888] [acceptance-tests-runner] me.atam.atam4j.logging.LoggingListener:  Test Test mechanism failed: Test mechanism
! java.lang.NullPointerException: null
! at me.atam.atam4j.TestRunListener.testFailure(TestRunListener.java:45) ~[user-cred-svc-acceptance-tests.jar:1.361.0]

Automate release to maven central

  • Commit to master should deploy a new version to maven central automatically
  • Commit to non master branches should run tests but not release to maven central
  • Manual release should still be possible

Package in-addition to of Individual test class names

As a user, I want the option to use atam4j by simply passing a package name containing my test classes instead of passing individual test class names so that adding new tests to the suite is easier and seamless and does not need modification to the test runner each time.

Severity levels for tests

As a user, I want to be able to define severity level to be associated with a test failure, so that the test reports can classify failures as per severity.

Example:

@Monitor(severity="critical")
@Test
public void myCriticalTest() throws Exception {
    // my test code
}

@Monitor(severity="major")
@Test
public void myMajorTest() throws Exception {
    // my test code
}

Release procedure

As an atam4j developer, I want the release process documented, so that I know how to release a new version using a standard and consistent way.

Document (in the project readme) the process of releasing a version. High level list of steps may be:

PS: It is ok, if all of this is manual for now. If this is all quick to script, then do it, else can be done later. Think MVP! :)

Document use of ConfigLoader

to load environment specific config files so that developers can run their test suites agains different environments using different configuration.

Add support for Prometheus

Expose test status in a prometheus metrics endpoint so it can be easily integrated with the monitoring framework.

GET /tests

Replace the current healthcheck with a resource on the above uri that returns a 200 if all tests passed in the last run or a 500 if one of them failed.
The body of the response will contain a list of each test with the test's name and status.... e.g.

[
{"testClass":"me.atam.atam4j.dummytests.PassingTest","testName":"testThatPasses","passed":true},
{"testClass":"me.atam.atam4j.dummytests.FailingTest","testName":"testThatFails","passed":false}
],"allTestsPassed":true}

Maven central integration

As a developer, I want successful releases to be uploaded to maven central so that the atam4j library is available for use to other developers.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.