Giter Club home page Giter Club logo

opengossip's Introduction

OpenGossip

Tools for metrics analysis and fault detection. It's currently experimental only. Have a look at the [wiki] (https://github.com/adrien-mogenet/opengossip/wiki) for current work's output.

Introduction

There are sevenral kind of metrics that you would want to track:

  • Metrics that should always be evolving and never be constant
  • Metrics that are highly related to each other
  • Metrics that should be perfectly predictable

Of course, putting this or that metric into this or that model depends on your platform and programs that are running on it. OpenGossip has been designed as a toolbox that you might customize to fit your own needs.

Requirements

OpenGossip is mainly written in Python so far, and Octave is used when prototyping different algorithms. For the best use of the current source code, you might want to install:

  • python >= 2.7
  • numpy
  • pypy >= 1.9
  • numpypy
  • gnuplot >= 4.2
  • octave >= 3.7

Pypy and numpypy are both not strictly required, but are speeding up the execution (at least x2 faster).

How does it work ?

OpenGossip is gathering several models to play with for metrics analysis. It extracts features for further analysis. For example, when dealing with time series, it will probably work with such features (among many others):

  • mean
  • standard deviation
  • slope
  • entropy

OpenGossip is learning a lot from these characteristics and will be able to classify your data as normal or as anomaly. Currently, learning model(s) is/are not yet established, and the "perfect" model is still pending... Sometimes naive threshold-heuristics will get the best F-Score.

Current results

Here is the graphic representing a load average (based on 1 last minute) : load-average

And now, how to automatically detect spikes ? Using classification algorithms mentionned above (here a hierarchical clustering technic) OpenGossip is able to detect abrupt changes, strange behaviors or related events. anomalies

Here is a Multivariate Gaussian implementation, in Clojure (using Incanter): multivariate

How much can I trust OpenGossip calculus ?

It is still in development, and even at an early stage. It currently mostly uses naive implementation (sometimes very high complexity) and is continuously developed to determine best way to find anomalies among your metrics.

opengossip's People

Contributors

adrien-mogenet avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.