Giter Club home page Giter Club logo

evt's Introduction

EVT

使用极端值理论(Extreme Value Theory)实现阈值动态自动化设置

介绍

我们的工作建立在2017 KDD "Anomaly Detection in Streams with Extreme Value Theory"论文的基础上,做了如下改进:

  • 引入矩估计算法,加速计算。该算法比极大似然估计快100多倍
  • 提出了更高层,更抽象的基于预测残差的算法框架,而DSPOT算法是我们提出框架的一种具体算法
  • 我们强调了数据漂移对系统带来的影响,提出了批量更新的算法,有效应对数据漂移

应用

  • 异常探测问题中,经常需要设置阈值,例如:内存的使用率大于90%时,判定为异常。这里阈值90%是人为设定的,需要用户有足够的使用经验,而且这种设定方式随机性很大,比如设置为89%或者91%似乎也是合理的。
  • 现实应用中,每条KPI都需要手动设置不同的阈值,这是一项十分复杂和庞大的工作,如果我们能够只设定概率值q而无需设定阈值,那么会免除巨大的工作量。 应用实例
  • 使用我们的方法只需定义异常事件发生的概率,而无需设置成百上千的阈值,以不变应万变 应用实例
  • 使用示例: 这里给出一个应对数据漂移的算法运行结果示意图,上下黄色虚线分别对应算法自动设置的上下阈值。 应用实例

evt's People

Contributors

dawnsonli avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.