Pack provides many operations helping to manage your Elasticsearch indices and snapshots.

Copy the example configuration in elasticsearch.yaml.example to /opt/stackstorm/configs/elasticsearch.yaml and edit as required.

It must contain:

• host - Host where Elasticsearch is running
• port - Elasticsearch port - default 9200
• query_window - Rolling window size in seconds. Default 30s
• query_string - Query string
• cooldown_multiplier - Multiple of query window to cooldown after successful hit. Default 2
• count_threshold - Minimum number of hits before emitting trigger. Default 5
• index - Index to query - e.g. 'logstash*'

You can also use dynamic values from the datastore. See the docs for more info.

These actions operate similar to curator for Elasticsearch.

Actions invocation parameters will be described further. But for more detailed description what each action actually does please also refer to the curator docs, it is more in-depth.

These parameters include general options such as elasticsearch host, port etc.

These parameters filter indices or snapshots when a command is being applied.

Search actions perform a specified query in Elasticsearch. There are two search actions available: search.q and search.body. The first one takes a query string (given in lucene syntax), while the former allows to perform more sophisticated searches using Elasticsearch query DSL.

Both search actions use the same common parameters as curator based actions.

This action is enhanced with index selection parameters to simplify indices matching.

Performing curator operations on indices or snapshots at least one filtering parameter must be specified. This's a generic rule applied to all of curator actions except. However show actions can be invoked without any filtering parameters, in this case show actions will display full list of indices or snapshots.

Now let's have a look at a few invocation examples.

• Show indices older than 2 days:

st2 run elasticsearch.indices.show host=elk older_than=2 timestring=%Y.%m.%d

Shows this on my node:

{
    "result": null,
    "exit_code": 0,
    "stderr": "",
    "stdout": "logstash-2015.05.02
logstash-2015.05.04
"
}

• Delete all indices matching ^logstash.*:

st2 run elastic.indices.delete host=elk prefix=logstash

• Create a snapshot of indices based on time range criteria:

st2 run elasticsearch.indices.snapshot host=elk repository=my_backup newer_than=20 older_than=10 timestring=%Y.%m.%d

This command will create a snapshot of indices newer than 20 days and older than 10 days. Notice that filtering parameters of snapshot command apply to indices not to snapshots. That's why it's important not to mess it up. For example, the timestring parameter when created by curator with default options has a different time scheme.

• Delete specific snapshots:

st2 run elasticsearch.snapshots.delete host=elk repository=my_backup snapshot=curator-20150506155615,curator-20150506155619

• Delete all snapshots:

st2 run elasticsearch.snapshots.delete host=elk repository=my_backup all_indices=true

On successful search (total hits > 0) query actions exit with return code == 0, if no documents have been found return code == 1. In all other case such as execution exceptions return code is 99.

Let's look at a few examples:

• Run query using DSL syntax:

st2 run elasticsearch.search.body host=elk body='{"query":{"match_all":{}}}' pretty=true

• Run query using URI syntax where q is a Lucene string:

st2 run elasticsearch.search.q host=elk q='message:my_log_event' prefix=logstash

2016-01-08 pixelrebel [email protected]

• The pack seems to work with elasticsearch v2.1.1 (and curator v3.4.0). So the pack now allows elasticsearch > v1.0.0 and < v2.2.0. Please update python module elasticsearch-curator==3.4.0 before upgrading this pack.
• Updated pack version to 0.2.0.

• Author:: StackStorm (st2-dev) ([email protected])
• Author:: Denis Baryshev ([email protected])