neogenix / daikon Goto Github PK

While Create, and Delete Index have been created, they need better error handling... should be done before 1.0 is tagged.

Currently the configuration is :

[default]
host = localhost

I'd like to be able to do :

[default]
host = [ localhost, localhost2 ]

I need to pick between apache/gpl/etc...

Index.py uses old style Exception handling, it needs to be fix0red.

More pythonish changes.

Rename es_index, es_cluster, and es_node to index, cluster, node

We should be able to search a cluster, and then spit the output to JSON files ! This should of course include exception, and error handling.

Config documentation needed...

Move json to anyjson, and add tests to ensure that it's installed.

Make Index Status more generic to be reused in index list, and cluster status

Rivers should be configurable from the daikon tool, and we should be able to check status, create, delete, etc... This should include exception and error handling of course.

Make Node Status more generic to be reused in node list, and cluster status

We should spit some more information out when we use the --verbose flag... it should also have a configuration variable to define this.

sys.exit() should only be called sparingly and usually not with 0.

Functions that need to report a non-zero status, should raise exceptions. The main function should catch these exceptions and translate them into an exit code which main would return.

Then in the if name == 'main' portion it would change to :
exit = main()
if exit:
sys.exit(exit)

Index list should be able to be spat out to a file, including information from replicas, status, etc... Should perhaps include a verbose command to include more information from the original ES request.

I need to have this app actually log to syslog, and use proper logging handlers. Should also have log files, configuration around log files, and the ability to spit to syslog, perhaps configuration around that too.

Targeting Version 1.1

Update node.py to use the new magical Exception Handling methods.

We should be able to show the stats view of a Node, and this should include error and exception handling.

need to check the version for making sure that the right stuff is supported. Should be probably handled by config.

We should be able to specify multiple indexes on the command line for delete and create, perhaps like this :

daikon index create
daikon index create --cluster
daikon index create --replicas --shards

Create some packages... setup.py, and other python doodads.

We should be able to list all functioning nodes in a cluster, and their IP's. This should include error, and exception handling.

Display health status of an index. Should also have exception, and Error Handling.

We should be able to specify the host, and port option to each argument, this should include error and exception handling.

Basic Documentation Is Needed... Github Pages, or ReadTheDocs!

This should show basic cluster state information, it should also include error and exception handling.

Validity Checks for IP and Hostname, for all --host, cluster configuration, and node status.

[root@kale-testbox ~]# daikon node list
SUCCESS: Fetching Node List :

 Nodes:
     Node: eLDk7SiKTfG425N21SPV0g
     Node: r71N-indT0SkZ1yFHWmhng
     Node: UKBJWxSLR8m6u1vR98T3_g
     Node: tb2Ve4dHTyCHC-RJ5c2t9g
     Node: lLDlsSmwQXWsiGJKWWOUwA
     Node: 8TvL_m--TrOW8JHjjcq2sQ

[root@kale-testbox ~]# daikon node status eLDk7SiKTfG425N21SPV0g --extended
ERROR: Fetching Node Status : "eLDk7SiKTfG425N21SPV0g" - <urlopen error [Errno -2] Name or service not known>

We should be able to do something like this to export an index :

daikon index export
daikon index export --cluster
daikon index export --file --format json

We should be able to do something like this to import an index :

daikon index import --file --format json
daikon index import --cluster --file --format json

This should include error, and exception handling.

Update cluster.py with new Exception Handling Magic

We should be able to use the --delay with node shutdown.

Improve on the index list code to include things like replicas, shards, status, and formatting of the output to show in a table.

Apache 2 license file needs to be updated with full license, and headers need to be inserted.

Make Cluster Status more generic to use node status, and index status.

Fix the messages in es_config.py as at the moment they're using the old format.

Right now index list uses both, but cluster index list uses state.

Unify VERSION variable so that you only have to call it and update once.

We should be able to shut down a Node, and this should include error and exception handling.

We need to be able to show the cluster health, should show basic information, also should have error and exception handling.

We should be able to shut a cluster down completely, this should also include error, and exception handling.

[kale@kale-testbox~]$ daikon node status 10.35.69.119 --extended
SUCCESS: Fetching Index Status : "10.35.69.119"

Status:
Node Status:
Cluster: logstash
ID: nzAYwPTmRxyLjpFnhd694g
Name: Cecilia Reyes
Index Status:
Size:
Traceback (most recent call last):
File "/usr/bin/daikon", line 9, in
load_entry_point('daikon==1.02', 'console_scripts', 'daikon')()
File "/usr/lib/python2.6/site-packages/daikon-1.02-py2.6.egg/daikon/daikon.py", line 186, in main
es_config.config['port'], es_args.extended)
File "/usr/lib/python2.6/site-packages/daikon-1.02-py2.6.egg/daikon/modules/es_node.py", line 31, in node_status
print '\t\t\t Size:', data_result[u'nodes'][node][u'indices'][u'store'][u'size']
KeyError: u'store'

Remove sys.exit(0) from es_index.py...

Change import, and rename es_config.

Will allow to show closed indexes, and then use that to fetch information about the closed indexes from other calls. Will require more than one call.

Update /modules/ to move the files to root, and then update assorted imports.

Error catching needs updating.

Implementation for Create Index, and Delete Index

We should be able to open and close indexes from daikon.

We should be able to list all indexes on a cluster.