etsy / oculus Goto Github PK

Redis::CommandError at /search ERR wrong number of arguments for 'mget' command

After following the setup/installation I wind up getting this message in the web interface and thin log. It appears that an empty array is being passed to the redis function mget, however I am not sure exactly why. Any help would be much appreciated. Ruby version 1.9.3-p448 (rvm)

First off, awesome project! As the documentation mentions, using DTW is computationally expensive. Would the project benefit from using other timeseries indexing methodologies such as Symbolic Aggregate Approximation? In this case, you can transform and discretize time-series data to string representations and perform string-based indexed search.

Let me know what your thoughts are. I'd be happy to submit a PR to implement this.

Cheers!

While I understand the need to keep updating your application, being unable to compile against the current version of Elasticsearch is a frustrating limitation.

Is there any plans in the near future to make Oculus compatible with Elasticsearch 0.90.1 and above?

As far as I can tell from reading the source, the requirement for multiple ES servers in distinct clusters stems from the way the importer works. It deletes the old index, creates a new one, populates it, and finally adjusts the setting in redis pointing to which server is active. I'm guessing this is done so that searches can still be performed while an import is happening. Perhaps this is also an attempt to segregate resource usage, so that indexing load does not impact searching and vice versa.

None of these requirements necessitates a separate ES cluster. Oculus is only using a single index, so it's no big deal to rotate between index names. Better still, you can use ES's index alias feature to point to the current index inside ES. For example, perhaps the importer could name the index it's creating metrics.YYYY.MM.DD.HH.MM.SS. When it finishes, it can atomically switch the "metrics" alias to point to the new index. The searcher then just refers to "metrics" as if it were the name of an actual index. This scheme also allows storing multiple historical indices, allowing one to search against historical fluctuations.

If isolation of indexing and searching is required, just use elasticsearch's routing parameters. One can ensure that searching and indexing always happen on different nodes. If complete isolation is required, then scrap my naming scheme above, alternate between metrics.0 and metrics.1, and add routing rules to ensure that the shards for those indices are stored on separate nodes.

Requiring multiple ES servers adds a significant barrier to entry. I need to either run multiple ES instances on one host (probably requiring me to retool my puppet manifest) or spin up multiple hosts. This wastes resources, since searches are probably going to be fairly rare so at least one ES cluster will be mostly idle.

Ultimately, given ES's flexibility in sharding and routing, I can't think of any case when an application would truly need to use multiple clusters.

The rubberband ES library which Oculus uses is now deprecated, change Oculus to use officially supported ES ruby library

Oculus looks pretty abandoned, but are there plans to support Elasticsearch 1.x? 0.90.x has been deprecated since October 2014.

Oculus currently searches for "normal" similarity. It'd be nice to be able to search for graphs doing the opposite, ie going down where the searched-for graph goes up.

We have gone through your Project and we would like to use this for our project.

Currently we are using Elastic search version 5.1.1 which faraway from the given one. Hence we are unable to use this on our project.

Will you able to support on this??

Awaiting for your valuable feedback.

Hi there,

In oculus' configuration file, there is a part for 'Skyline':
skyline:
host: ""
port: 6379
listener_port: 2024
metric_prefix: "metrics"

I am not sure if I understand them right:
'port' : this one is for redis server running for skyline
'listener_port': this is port for skyline horizon which receives data from carbon-relay
'metrick_prefix' : this one should use the same one as "FULL_NAMESPACE = " in skyline's settings.py configuration file

Are they all right ?

And there is a 'index' part in the elasticsearch part of the conf file with default value 'mini', shall I just keep it? Or change it to other value?

When I attempt to search for a metric by name, I get this error:

NoMethodError at /search
undefined method `fingerprint' for nil:NilClass
/opt/oculus/helpers/elasticsearch.rb in get_fingerprint
312. @client.get(name).fingerprint
/opt/oculus/oculusweb.rb in block in class:Oculusweb
312. @fingerprint = @elasticsearch_helper.get_fingerprint(@formatted_query)

Could this be a bug, or a configuration issue?

When I search via Drawn Query, the result is "Your search didn't match any saved collections."

At the moment, you have to type in a full metric name to search for it. Let's have a way to browse the metrics we have a-la graphite.

Is there any plans in the near future to make oculus compatible with elasticsearch 0.90.x?

Hi there,
when I tried to open the link behind the data points path in skyline which redirects me to oculus, I always get the IndexMissingException. Is that caused by the cron job which pulls data from skyline to oculus?
This is my configuration file config/config.yml:
root@ip-10-118-150-210:/opt/oculus/config# cat config.yml
results_explain: 0
elasticsearch:
servers:
- "http://127.0.0.1:9201"
- "http://127.0.0.1:9202"
index: "mini" #same as "MINI_NAMESPACE" in skyline settings.py ?
timeout: 30
phrase_slop: 20
scorers:
dtw:
radius: 5
scale_points: 25
euclidian:
scale_points: 25
skyline:
host: "ec2-5-42-4-244.compute-1.amazonaws.com"
port: 6379
listener_port: 1500 #skyline app http server listen port ?
metric_prefix: "metrics" # same as "FULL_NAMESPACE" in skyline settings.py ?
redis:
host: "127.0.0.1"
port: 6379

I've wanted to add DTW support to https://github.com/obfuscurity/descartes for some time now. It would be nice to leverage Oculus' search via API. For example, say I have some raw data from a Graphite and want to search for any similar activity.