tnn1t1s / riemann-elastic Goto Github PK

Hi @tnn1t1s , thanks for riemann-elastic!

I've wired it into my Riemann instance and I'm able to use es-index to forward Riemann events to ES from (streams) and they show up in the ES index. However, when i try to forward events from (expired), i don't see those show up in ES.

Here is what works:

...
(streams
  (elastic/es-index "riemann-elastic" :index "myindex")
...
)

But this doesn't work:

...
  (expired
    (elastic/es-index "riemann-elastic" :index "myindex")
    ...
  )
...

Is there any reason why calling es-index from (expired) shouldn't work?

Any hints or clues greatly appreciated...

Write documentation and provide a sample riemann.config

Since ES2.x dots are not allowed in field names:

MapperParsingException[Field name [facter.operatingsystem] cannot contain '.']

The natural way to handle this would IMHO be to coerce foo.bar = "baz" into {"foo": {"bar" : "baz"}}.
The problem is I'm not very good at clojure, so I'm kindly asking if this is a feature which would seem interesting and if somebody would be willing to point me into the right direction.

What scares me most is the need to handle this recursively. But isn't list all about recursion?

Hello,

Can you recommend how to do basic authentication using this library? To connect to default Elasticsearch instances there typically a default login.

ex.
curl -su elastic:changeme 'http://localhost:9200/_search?q=*'

New to Riemann and Closure, just not 100% certain where to place the auth bits.

Thank you.

Ron

I've noticed that if ES can't keep up with the data stream, we just start to "drop" events until ES catches up. If think that if the ES eb/bulk-with-index function is blocking, it could run in a loop which exerts backpressure onto the incoming events. The buffer for those events could drop them not as FIFO but instead trying to preserve an even distribution of values from across the time range.

This would hopefully give better information for understanding a system anomaly, by attempting to keep some data points where there would've been none.