Giter Club home page Giter Club logo

iofs's Introduction

IOFS -- The I/O file system - A FUSE file system developed for I/O monitoring

Old README

The code also contains a Grafana reporting submodule that can be easily integrated into any FUSE module. You may use the code in iofs-monitor.h and elasticsearch.c to integrate it into your own FUSE module.

For more information on how it works, how to use it and how to extend it see the documentation.

iofs's People

Contributors

lquenti avatar juliankunkel avatar

Stargazers

Tim T. avatar  avatar

Watchers

Marcel Hellkamp avatar Peter Chronz avatar  avatar Maik Srba avatar

iofs's Issues

InfluxDB auth is currently inaccessible

The influxdb-username and influxdb-password config parameter are currently only accessible via configuration.
Furthermore, the docker influx setup is configured to not use authentification at all.
Lastly,

All Related TODOs

  • Refrence all possible CLI paramters in the documentation, remove from repo root
  • Add both parameters to argp
  • Create a docker setup that does use authentication
  • Test the setup with iofs
  • Add the docker documentation to iofs book

IOFS requires blackheaps models for realistic overhead; blackheap needs realistic overhead

This is a pretty complex problem to solve elegantly.

The Problem

iofs is planned to (also) be used with blackheap. Blackheap uses blackbox methodology to provide prediction models to classify I/O requests solely based on their access times. iofs supports loading those models created by blackheap with the --classificationfile parameter.

Of course, this assumes that the latencies initially measured by blackheap are the same ones when running iofs. This is not the case, since it takes some time to classify the I/O requests. Thus, one the one hand, we require iofs to already have classifications to get the realistic overhead. However, on the other hand, we also need to create the classifications on a mounted iofs. Thus, we have a circular dependency.

Why the Trivial Solution won't work

The most obvious solution would be to just create a constant array of dummy classifications that gets evaluated against no matter whether actual classifications are provided or not. Unfortunately, this is not possible since we accept a any amount of models. For example, using the constantlinear model provided by blackheap creates twice the amount of models than the simpler linear model. See the blackheap docs for more.

Ideas

I have two idea, both of which are suboptimal at best.

Idea 1: Provide multiple dummy models, let the user choose

Create dummy CSV files with the amount of models that would be created for each model type used with blackheap. The actual parameters of those models are obviously irrelevant; it only matters that the amount of models (i.e. the number of iterations needed) are correct.

This should be fine, although not very user friendly.

Idea 2: 2 runs, 3 mounts

Just run it twice

  1. Mount iofs with no model
  2. Create a wrong model with blackheap
  3. Remount iofs with the wrong model
  4. Create a correct model with blackheap
  5. Remount iofs again with the correct model

This is the most reliant and secure way since the first model will be more correct than random data. But whether it is actually superior to the first idea is unknown. I dont think so, but I havent tested it yet.

The obvious disadvantage is how long it takes, as we have to create two models.

The current state

  • The blackheap and iofs documentation notes that this is an open issue
  • The advice is to use dummy models contained in the repository
  • It is mentioned that one can also just run it twice with pointing to this repo for further explaination

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.