Giter Club home page Giter Club logo

clip's Introduction

clip

clip (the command line illustration processor) is an open-source command line program and software library for creating charts and other data-driven illustrations.

Introduction

In essence, clip is an automated drawing program; it reads a text file containing a description of the chart or diagram and produces an image from it. This is best explained by example, so here is how to draw a simple line chart using clip:

class: plot;

limit-x: 0 7200;
limit-y: 0 100;

axes {
  font: "Roboto";
  position: left bottom;
  label-placement-x: linear-interval(900 900 7000);
  label-format-x: datetime("%H:%M:%S");
}

lines {
  data: "test/testdata/timeseries.csv";
  stroke-width: 0.8pt;
  color: #06c;
}

The input file from above (example.clp) can be processed with clip using the following command:

$ clip --export output.svg example.clp

This is the resulting SVG file (output.svg):

Note that this example is only intended to give you an idea of what the syntax looks like; for an in-depth description of all parameters, please refer to the documentation. More examples can be found on the examples page.

Documentation

You can find the full documentation and more examples at clip-lang.org.

Installation

Compile from source

To build clip, you need an up-to-date C++ compiler, cmake, fmtlib, libharfbuzz, libfreetype and cairo. Then run:

$ cmake .
$ make

To install the clip binary into your system, run make install:

$ make install

To run the test suite, run make test:

$ make test

For detailed installation instructions, have a look at the Installation page.

FAQ

I'm getting build errors when compiling the code

clip is written in C++17 and requires a reasonably modern C++ compiler and standard library. In most cases where the code doesn't build it's due to some problem with the local build environment. Still, we always appreciate bug reports via Github Issues so that we can improve our build system.

The test suite fails on my machine because text placement is slightly offset

The test suite requires that you have the original Microsoft Arial TTF files installed on your machine. To verify that this is the case, run fc-match 'Arial,Helvetica,Helvetica Neue:style=Regular,Roman' and check that it returns the correct 'arial.ttf' file.

What happened to the project name?

The project was started in 2011 and was initially called "FnordMetric". The first version from 8 years ago also included facilities for storing and transforming data in addition to the charting code. Over time, the data processing parts were removed, leaving only the plotting code. However, as a consequence, most of the search queries for the project name would return outdated information, resulting in a generally confusing and stale-feeling situation. The best solution seemed to be to rename the project and so it was renamed to "clip".

Acknowledgements

License

clip -- The command line illustration processor
https://clip-lang.org

Copyright (c) 2020, Paul Asmuth, Laura Schlimmer
All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

clip's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

clip's Issues

FnordMetric as Rack app

Its just a suggestion/idea. Can FnordMetric be transformed to start from rackup file (using thin server/daemon)?
I think it would be easier and more convenient.

No config for client timezone

Right now I see that the time is always GMT time. It's hard for me to convert time every time I look at the chart because my local time is GMT + 7. Is there a way to configure the timezone on client browser?

Active Users view when tab is hidden

When you hide the active users tab and you don't have any other tabs specified yet, you still see the layout for the active users tab. This isn't a big deal or anything. It's just a little ugly. I'd be glad to fix it but, I need to learn the JavaScript first. Just thought I'd post on here so if someone else wanted to make a quick fix,t hey could. :-)

Pies: showing the distribution as in bars

Hey,

if i have a gauge that has a number of keys (e.g. 'app version breakdown'), i'd like to go about making a pie chart of this, with percentage as a portion of the distribution.

I think this is slightly simpler to visually parse than a bar chart....

is that feasible today? is it easy to hack in?

thanks!

udp access

can you add udp access so the connection is non-blocking?

Toplists, Pies, etc - only show a subset of a gauge

Hey,

i want to enable support for certain gauges that are prone to collect a lot of data, but really i only want the top ten or so, and then an 'others' roll up group.

I'm happy to hack this in, but i'd like some pointers as to the best place to do it. It seems that the pie_widget.rb in lib/fonordmetric is a good place to start, as it's a presenter for the data... but i am concerned about what the JS is trying to do there.

thoughts?

Viewing Dashboard after following installation example raises exception

ruby -v
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin10.7.4]

gem -v
1.8.11

NoMethodError - undefined method `dashboards' for nil:NilClass:
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/haml/app.haml:26:in `evaluate_source'
/Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:209:in `instance_eval'
/Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:209:in `evaluate_source'
/Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:144:in `cached_evaluate'
/Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:127:in `evaluate'
/Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/haml.rb:24:in `evaluate'
/Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:76:in `render'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:625:in `render'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:522:in `haml'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric/app.rb:53:in `block in <class:App>'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:1212:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:1212:in `block in compile!'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:772:in `[]'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:772:in `block (3 levels) in route!'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:788:in `route_eval'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:772:in `block (2 levels) in route!'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:821:in `block in process_route'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:819:in `catch'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:819:in `process_route'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:771:in `block in route!'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:770:in `each'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:770:in `route!'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:886:in `dispatch!'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:706:in `block in call!'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:871:in `block in invoke'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:871:in `catch'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:871:in `invoke'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:706:in `call!'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:692:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/xss_header.rb:22:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/path_traversal.rb:16:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/json_csrf.rb:17:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/base.rb:47:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/xss_header.rb:22:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-1.3.5/lib/rack/nulllogger.rb:9:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-1.3.5/lib/rack/head.rb:9:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/showexceptions.rb:21:in `call'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:84:in `block in pre_process'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:82:in `catch'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:82:in `pre_process'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:57:in `process'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:42:in `receive_data'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run_machine'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric.rb:43:in `start_em'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric.rb:94:in `run'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric/standalone.rb:5:in `block in <top (required)>'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:634:in `call'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:634:in `block in execute'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:629:in `each'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:629:in `execute'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:595:in `block in invoke_with_call_chain'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:588:in `invoke_with_call_chain'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:581:in `invoke'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2041:in `invoke_task'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2019:in `block (2 levels) in top_level'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2019:in `each'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2019:in `block in top_level'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2058:in `standard_exception_handling'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2013:in `top_level'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric/standalone.rb:31:in `<top (required)>'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:55:in `require'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:55:in `require'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric.rb:121:in `standalone'
test.rb:22:in `<main>'

Dashboards with spaces in the name throw exceptions

If I have a dashboard named "Passive Users", I get this beauty when I try to access it:

   KeyError - key not found: "PassiveUsers":
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/fnordmetric-0.5.3/lib/fnordmetric/app.rb:111:in `fetch'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/fnordmetric-0.5.3/lib/fnordmetric/app.rb:111:in `block in <class:App>'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:1212:in `call'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:1212:in `block in compile!'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:772:in `[]'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:772:in `block (3 levels) in route!'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:788:in `route_eval'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:772:in `block (2 levels) in route!'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:821:in `block in process_route'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:819:in `catch'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:819:in `process_route'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:771:in `block in route!'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:770:in `each'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:770:in `route!'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:886:in `dispatch!'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:706:in `block in call!'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:871:in `block in invoke'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:871:in `catch'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:871:in `invoke'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:706:in `call!'
    /Users/dan/.rvm/gems/ruby-1.9.2-p0@rails3tutorial/gems/sinatra-1.3.1/lib/sinatra/base.rb:692:in `call'

I spent some time poking around, and it looks like the API call works fine, but the JS doesn't. I can't quite find where renderDashboard is getting called in the JS to track it down further, but I'm guessing (this is completely unfounded...) that whatever is setting / reading the hash eats spaces.

Race condition in submittion events

The example code from the Readme for submitting events has a race condition.

redis.lpush("fnordmetric-queue", uuid) 
redis.set("fnordmetric-event-#{my_uuid}", event)
redis.expire("fnordmetric-event-#{my_uuid}", 60)

I assume the receiving side does a (blocking) lPop. Following order of events can happen:

  1. Client does lPush on the queue
  2. Server does blPop on the queue
  3. Server does Get on the key (fails, not found)
  4. Client does Set on the key
  5. Client does Expire on the key

To prevent the race condition the code needs to be re-ordered to

redis.set("fnordmetric-event-#{my_uuid}", event)
redis.expire("fnordmetric-event-#{my_uuid}", 60)
redis.lpush("fnordmetric-queue", uuid) 

to guarantee that any Get on the key that was Pop'd from the queue will succeed unless the key already expired.
Alternatively one could pipe the commands on the client so they are executed in one atomic step in Redis which will also save roundtrip times.
Additionally you can combine the Set and Expire into a single SETEX call.

splitting up the dsl block

i love the way you have Fnordmetric.namespace :x do .. end - but is there a way to split this up to different ruby files?

it'd be cool to just include app/metrics/*.rb or something, and that would allow me to break it up by 'tab' in the app. Right now my metrics files run to 100s - 1000s of lines, and this makes debugging and such a bit nastier than it has to.

thanks!

Fix Simple Example

The simple example in the README raises this exception: undefined method map for nil:NilClass (NoMethodError)

The problem seems to be that a :series option is not given to the timeseries_gauge. Here is a working version (taken from doc/minimal_example.rb).

require "fnordmetric"

FnordMetric.namespace :simple do
  timeseries_gauge :unicorns_seen,
    :tick => 30.seconds,
    :calculate => :average,
    :series => [:num_unicorns]

  event :unicorn_seen do
    incr :unicorns_seen, :num_unicorns, rand(5)
  end
end

FnordMetric.standalone

Separate API for adding events

You can now add events by directly using redis, but isn't it nicer to use a simple API to add events?
It would also eliminate code duplication in InboundStream and App.

I've implemented a simple API in this commit: 637827f

Off by 1

What is happening is, when event A is triggered it doesn't increment the guage instantly. It only increments when the same or any other event is triggered. So its always off by 1. Is this the intended behaviour?

Add limit option to toplist

It would be nice to specify a limit on toplists, rather than displaying all of them.

e.g.

  widget("Top Users",
    :title => "This Week's Top Users",
    :gauges => :user_pageviews_per_week,
    :type => :toplist,
    :width => 50,
    :autoupdate => 1,
    :limit => 5  # <- limits to top 5, rather than all
  )

Tracking User Events

The doc says: We can select user and follow them step by step but i cant get it working.
Is there a way to track user events. What I did is:

  • I set users 'A' and 'B' through 3 special events ( '_pageview', '_set_name' and '_set_picture')
  • I post the event 'attack' with user session 'A' (on the client i see: event 'attack' belongs to user 'A') (post data: { '_type' => 'attack', '_session' => 'abc' })
  • Then I post the event 'rob' with user session 'B' ((post data: { '_type' => 'rob', '_session' => 'def' })
    After that I see all the posted events before become 'rob' with user 'B'.
    Did I do something wrong or it is not supported to do it yet?

fnordmetric 1.0.0: Running on heroku, websockets vs polling.

fnordmetric really is awesome, and looks great on my local machine. However, I have trouble deploying it to heroku (and even though I haven't tried that yet, probably to other servers where it would run behind an nginx proxy.)

Apparently fnordmetric 1.0.0 uses websockets as a transport, where 0.7.x polled the server. If I understand this issue correctly imanel/websocket-rack#8 running websockets on heroku just does not work.

Is there a simple way to fall back to polling instead of websockets? Some configuration option one can set somewhere? A working project

I am aware that #73 already deals with running on heroku. The suggested soultion works, but with fnordmetric 0.7.x versions only.

Syntax tweak

Hi,
is there any chance a change allowing that would get merged ?

dashboard('Overview') do

  widget "Unicorn-Sightings per 10s", {
    :type => :timeline,
    :gauges => :unicorns_seen_per_10s,  
    :include_current => true,
    :autoupdate => 2,
    :width => 60
  }

  widget "Numbers", {
    :type => :numbers,
    :gauges => [:unicorns_seen_per_10s, :unicorns_seen_per_min],
    :autoupdate => 2
  }

end

I find this syntax way more intuitive than the current one, what do you think ?
The argument to the widget method becomes its title and obviously the argument to the dashboard method becomes its title too.

I already have the code written and working.

Running on Heroku

I'm having a hard time trying to run Fnordmetric in Heroku.
Basically, what happens is that Heroku won't allow to start the EM worker on the same dyno.
Is there a way to manually run the Fnordmetric worker using a rake command or something similar?

Namespaces not handling events

I noticed that only the first registered namespace will accept events.

I currently have two namespaces configured. Only the first will accept incoming events. If I swap them then one stops working and the other starts.

Is this an issue or not a feature yet?

Need garbage collection for timeline data

Looks like you just add data to this timeline key, but never clean old elements.

  def announce_to_timeline(event)
    timeline_key = key_prefix(:timeline)
    @redis.zadd(timeline_key, event[:_time], event[:_eid])
  end

as result this key size is about 50% of database.

This even bigger problem then you have small ttl for events data.

    :event_data_ttl => 60*5,

most element in timeline keys just point to nowhere, because events already deleted.

Also, and may be I'm wrong here, you dont even need to collect this data, when you have activated

  hide_active_users

In this case, timeline data is not used at all.

Timeline widget data points not in line with axis

I have a 'daily' timeline widget. When I scroll a 'tick' forward, the data points dont match the axis anymore:

  • it's Jan 9,
  • theres data for today, not tomorrow
  • i can see the data point for today above jan 11
  • when i hover over jan 9, the tooltip for jan 9 opens, the data point over jan 11 is highlighted
  • when i hover over jan 11, nothing happens

screenshot

How does one track events by user?

Thanks for your work on this. This is awesome!

I am trying to track what an user does on our site. So I can click on a user in the active users tab and see all the urls they have visited on the site. fnordmetric seems to support this, but am not able to configure it to do this. is there a special event like _pageview that I need to add for each user for this? If this is possible adding it to the user docs would be great.

thanks

using websocket

I really like this project and I think WebSocket could benefit greatly to it, did you already gave it any thoughts ?
When I saw realtime in the description I was sure websockets were used xD

weird rendering issue with distributions/bars

  {
    "count": 592,
    "values": [
        [
            ["xyz", 64.0],
            ["abc", 40.0]
        ],
        [
            ["def", 27.0],
            ["jkl", 24.0]
        ],
        .....
}

why is it returning the arrays like this? should it not be one array, flatter?

IRC?

Is there any type of IRC for this project? I've emailed Paul Asmuth but, with no response. I'm REALLY interested in the project and would love to commit but I really want to get a feel for the direction of the project as understood by the current, main contributors. I feel the easiest way might be an IRC channel.

Maybe there could be a channel on freenode.net? Just a thought.

License issue

Hi,

FnordMetric is a nice application, but it has a license issue: highcharts license. It contains non-commercial license which is known to be not compatible with OpenSource licenses, including the MIT license.

There are some ways to fix this issue:

  • use another JS lib for charting (probably not easy, but if you want to go in this direction, sooner is better)
  • remove the highcharts from your repository and add to the README how to download it from its official website (with explanations of why you do that)
  • contact the highcarts author...

I hope it helps.

High Rate of Lost Events

I don't entirely follow how events get lost (would love an answer to that, actually), but below is a log snippet that's pretty consistent with what I'm seeing. Also, data only seems to actually file into the web interface (though accompanied by scads of lost events) when I restart the ruby server. Any ideas?

[11-12-28 21:22:31] events_received: , events_processed: 7533, queue_length: 0
[11-12-28 21:22:33] oops, lost an event :(
[11-12-28 21:22:34] events_received: , events_processed: 7534, queue_length: 0
[11-12-28 21:22:36] oops, lost an event :(
[11-12-28 21:22:37] events_received: , events_processed: 7535, queue_length: 0
[11-12-28 21:22:37] oops, lost an event :(
[11-12-28 21:22:38] oops, lost an event :(
[11-12-28 21:22:40] events_received: , events_processed: 7537, queue_length: 0
[11-12-28 21:22:41] oops, lost an event :(
[11-12-28 21:22:41] oops, lost an event :(
[11-12-28 21:22:41] oops, lost an event :(
[11-12-28 21:22:41] oops, lost an event :(
[11-12-28 21:22:41] oops, lost an event :(
[11-12-28 21:22:42] oops, lost an event :(
[11-12-28 21:22:43] events_received: , events_processed: 7543, queue_length: 0
[11-12-28 21:22:45] oops, lost an event :(
[11-12-28 21:22:46] events_received: , events_processed: 7544, queue_length: 0
[11-12-28 21:22:50] events_received: , events_processed: 7544, queue_length: 0
[11-12-28 21:22:52] oops, lost an event :(
[11-12-28 21:22:53] events_received: , events_processed: 7545, queue_length: 0
[11-12-28 21:22:55] oops, lost an event :(
[11-12-28 21:22:56] events_received: , events_processed: 7546, queue_length: 0
[11-12-28 21:22:59] events_received: , events_processed: 7546, queue_length: 0
[11-12-28 21:23:02] events_received: , events_processed: 7546, queue_length: 0

Can't install on MacOSX Lion

Hi,

(I'm new to ruby) I'm having the following error when running the simple example:

/Library/Ruby/Site/1.8/rubygems/custom_require.rb:59:in `gem_original_require': /Library/Ruby/Gems/1.8/gems/fnordmetric-0.5.5/lib/fnordmetric.rb:67: syntax error, unexpected tIDENTIFIER, expecting tAMPER (SyntaxError)
          Thin::Server.start(*opts[:web_interface], app)
                                                       ^
    from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:59:in `require'
    from my_stats_app.rb:1

My ruby version is ruby 1.8.7 (2010-01-10 patchlevel 249) [universal-darwin11.0] and using the latest fnordmetric revision.

Thanks for creating this cool project.

Can't install on debian 5.0

I just tried installing the gem on a debian 5.0 machine.


# gem install fnordmetric
Building native extensions.  This could take a while...
ERROR:  Error installing fnordmetric:
    ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb install fnordmetric
extconf.rb:1:in `require': no such file to load -- mkmf (LoadError)
    from extconf.rb:1


Gem files will remain installed in /var/lib/gems/1.8/gems/bson_ext-1.5.2 for inspection.
Results logged to /var/lib/gems/1.8/gems/bson_ext-1.5.2/ext/cbson/gem_make.out

I'm not a Ruby user so you'll have to tell me how to get more meaningful information if needed.
I tried installing the gems mkmf-lite and mkmfmf but that didn't help.

Reactor crashes when removing/renaming gauges

I had a gauge called pagevisits_per_user, which I later renamed to pagevisits_per_human.

However, the renaming, or removing of gauges with data leads to this error:

[12-09-27 16:21:48] [ERROR] reactor crashed: undefined method `field_values_at' for nil:NilClass

Any ideas how to fix this?

Edit: Seems this issue disappears when the next lot of events come in. Guess starting up the server isn't clearing obsolete data, only new events does?

Viewing Dashboard after following installation example raises exception

NoMethodError - undefined method dashboards' for nil:NilClass: /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/haml/app.haml:26:inevaluate_source'
/Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:209:in instance_eval' /Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:209:inevaluate_source'
/Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:144:in cached_evaluate' /Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:127:inevaluate'
/Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/haml.rb:24:in evaluate' /Users/john/.rvm/gems/ruby-1.9.2-p290@global/gems/tilt-1.3.3/lib/tilt/template.rb:76:inrender'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:625:in render' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:522:inhaml'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric/app.rb:53:in block in <class:App>' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:1212:incall'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:1212:in block in compile!' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:772:in[]'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:772:in block (3 levels) in route!' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:788:inroute_eval'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:772:in block (2 levels) in route!' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:821:inblock in process_route'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:819:in catch' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:819:inprocess_route'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:771:in block in route!' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:770:ineach'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:770:in route!' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:886:indispatch!'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:706:in block in call!' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:871:inblock in invoke'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:871:in catch' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:871:ininvoke'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:706:in call!' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/base.rb:692:incall'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/xss_header.rb:22:in call' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/path_traversal.rb:16:incall'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/json_csrf.rb:17:in call' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/base.rb:47:incall'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-protection-1.1.4/lib/rack/protection/xss_header.rb:22:in call' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-1.3.5/lib/rack/nulllogger.rb:9:incall'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/rack-1.3.5/lib/rack/head.rb:9:in call' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/sinatra-1.3.1/lib/sinatra/showexceptions.rb:21:incall'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:84:in block in pre_process' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:82:incatch'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:82:in pre_process' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:57:inprocess'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/thin-1.2.11/lib/thin/connection.rb:42:in receive_data' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:inrun_machine'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in run' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric.rb:43:instart_em'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric.rb:94:in run' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric/standalone.rb:5:inblock in <top (required)>'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:634:in call' /Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:634:inblock in execute'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:629:in each' /Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:629:inexecute'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:595:in block in invoke_with_call_chain' /Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:inmon_synchronize'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:588:in invoke_with_call_chain' /Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:581:ininvoke'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2041:in invoke_task' /Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2019:inblock (2 levels) in top_level'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2019:in each' /Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2019:inblock in top_level'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2058:in standard_exception_handling' /Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/rake.rb:2013:intop_level'
/Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric/standalone.rb:31:in <top (required)>' /Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:55:inrequire'
/Users/john/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:55:in require' /Users/john/.rvm/gems/ruby-1.9.2-p290/gems/fnordmetric-0.5.2/lib/fnordmetric.rb:121:instandalone'
test.rb:22:in `

'

Timezone settings

Do I need to configure the timezone?
At the event feed, the timestamps are shown in my timezone (GMT+8).
But in the history, all the events are at 08:00:00.

intern vs to_sym

Any specific reason you're using intern rather than just to_sym ?

Distinction between events_received & events_processed

Can you please provide some details on the distinction in the log output of events_received & events_processed?

What constitutes an event_received, and what constitutes an event_processed? e.g.

[11-12-30 09:46:43] events_received: 2, events_processed: 141, queue_length: 0

Thanks.

Data generator

I am new to fnordmetric.

When I started to work on it (2 day ago) it was quite hard to grasp how it really works.

I wrote some live data generator: https://gist.github.com/1579235

I will extend it now to cover all the features which are available in doc/ulm_stats.rb

It would be nice if such generator would be included in the project / would be mentioned in the Readme as it would make it easy (for fnordmetric noobies like me ;)):

  • to check all the features
  • start with development
  • prepare benchmarks?

. Please, let me know what do you think about it.

Make FnordMetric::Web a true rack app

I'd like to do thin -C fnord.yml fnord.rb to start the server daemonized. The current implementation isn't rackup friendly, so in production, I'm having to start a screen process and run the ruby file in that, which isn't very friendly.

Unique gauges need to take an option when incrementing gauge

How does unique determine when it should be uniq?

I think it needs to take a key called uniq_by. In my case, the key would be the user id of the unique page visit.

gauge :uniq_pagevisits_per_day, :tick => 1.day, :unique => true
event :pageview do
  incr :uniq_pagevisits_per_day, :uniq_by => data[:user_id]
end

Queue Length

Hi,

If I have a lot of events fnordmetric can't keep up and the queue length keeps growing until eventually Redis is claiming all memory on the box. (queue length of 8 million). I'm not sure if the queue is the only reason to blame for Redis taking up so much memory, but maybe also data stored on the various metrics? (I'm not too sure of how everything is stored).

5.4G Redis, here is last log from fnordmetric:

events_received: 25300317, events_processed: 17397513, queue_length: 7901866

historical data

Hey,

i see how awesome fnordmetric is for ongoing, current data, but is there any plan/suggested process for importing historical data?

Thanks!

Add possibility to mount FnordMetric via Rails routes

In resque, there is a feature of mounting Sinatra app directly to Rails routes.

mount Resque::Server.new, :at => "/resque"

In my opinion, it's very useful and it will be possible to mount FnordMetric web frontend to Rails app with one line.

@paulasmuth, if you agree with me, I will implement this feature in pull request.

Document memory tweaks

As it causes a lot of confusion, I think we should document how to set TTLs properly, optimize defaults and provide some redis usage benchmarks for various gauges and traffic schemas.

For example:
If somebody has higher traffic, he can reduce events TTL and sacrifice user history length but reduce Redis memory usage.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.