Giter Club home page Giter Club logo

zk_shell's Introduction

zk-shell

Build Status Coverage Status PyPI version Requirements Status Python Versions Code Climate

Table of Contents

tl;dr

A powerful & scriptable shell for Apache ZooKeeper

Installing

As Dockerfile:

::
$ docker build . -f Dockerfile -t zk-shell:1.3.3

From PyPI:

$ pip install zk-shell

Or running from the source:

# Kazoo is needed
$ pip install kazoo

$ git clone https://github.com/rgs1/zk_shell.git
$ cd zk_shell
$ export ZKSHELL_SRC=1; bin/zk-shell
Welcome to zk-shell (0.99.04)
(DISCONNECTED) />

You can also build a self-contained PEX file:

$ pip install pex

$ pex -v -e zk_shell.cli -o zk-shell.pex .

More info about PEX here.

Usage

Docker Version

$ docker run -it zk-shell:1.3.3

and use the connect command to connect to your zookeeper instance

$ zk-shell localhost:2181
(CONNECTED) /> ls
zookeeper
(CONNECTED) /> create foo 'bar'
(CONNECTED) /> get foo
bar
(CONNECTED) /> cd foo
(CONNECTED) /foo> create ish 'barish'
(CONNECTED) /foo> cd ..
(CONNECTED) /> ls foo
ish
(CONNECTED) /> create temp- 'temp' true true
(CONNECTED) /> ls
zookeeper foo temp-0000000001
(CONNECTED) /> rmr foo
(CONNECTED) />
(CONNECTED) /> tree
.
├── zookeeper
│   ├── config
│   ├── quota

Line editing and command history is supported via readline (if readline is available). There's also autocomplete for most commands and their parameters.

Individual files can be copied between the local filesystem and ZooKeeper. Recursively copying from the filesystem to ZooKeeper is supported as well, but not the other way around since znodes can have content and children.

(CONNECTED) /> cp file:///etc/passwd zk://localhost:2181/passwd
(CONNECTED) /> get passwd
(...)
unbound:x:992:991:Unbound DNS resolver:/etc/unbound:/sbin/nologin
haldaemon:x:68:68:HAL daemon:/:/sbin/nologin

Copying between one ZooKeeper cluster to another is supported, too:

(CONNECTED) /> cp zk://localhost:2181/passwd zk://othercluster:2183/mypasswd

Copying between a ZooKeeper cluster and JSON files is supported as well:

(CONNECTED) /> cp zk://localhost:2181/something json://!tmp!backup.json/ true true

Mirroring paths to between clusters or JSON files is also supported. Mirroring replaces the destination path with the content and structure of the source path.

(CONNECTED) /> create /source/znode1/znode11 'Hello' false false true
(CONNECTED) /> create /source/znode2 'Hello' false false true
(CONNECTED) /> create /target/znode1/znode12 'Hello' false false true
(CONNECTED) /> create /target/znode3 'Hello' false false true
(CONNECTED) /> tree
.
├── target
│   ├── znode3
│   ├── znode1
│   │   ├── znode12
├── source
│   ├── znode2
│   ├── znode1
│   │   ├── znode11
├── zookeeper
│   ├── config
│   ├── quota
(CONNECTED) /> mirror /source /target
Are you sure you want to replace /target with /source? [y/n]:
y
Mirroring took 0.04 secs
(CONNECTED) /> tree
.
├── target
│   ├── znode2
│   ├── znode1
│   │   ├── znode11
├── source
│   ├── znode2
│   ├── znode1
│   │   ├── znode11
├── zookeeper
│   ├── config
│   ├── quota
(CONNECTED) /> create /target/znode4 'Hello' false false true
(CONNECTED) /> mirror /source /target false false true
Mirroring took 0.03 secs
(CONNECTED) />

Debugging watches can be done with the watch command. It allows monitoring all the child watches that, recursively, fire under :

(CONNECTED) /> watch start /
(CONNECTED) /> create /foo 'test'
(CONNECTED) /> create /bar/foo 'test'
(CONNECTED) /> rm /bar/foo
(CONNECTED) /> watch stats /

Watches Stats

/foo: 1
/bar: 2
/: 1
(CONNECTED) /> watch stop /

Searching for paths or znodes which match a given text can be done via find:

(CONNECTED) /> find / foo
/foo2
/fooish/wayland
/fooish/xorg
/copy/foo

Or a case-insensitive match using ifind:

(CONNECTED) /> ifind / foo
/foo2
/FOOish/wayland
/fooish/xorg
/copy/Foo

Grepping for content in znodes can be done via grep:

(CONNECTED) /> grep / unbound true
/passwd: unbound:x:992:991:Unbound DNS resolver:/etc/unbound:/sbin/nologin
/copy/passwd: unbound:x:992:991:Unbound DNS resolver:/etc/unbound:/sbin/nologin

Or via igrep for a case-insensitive version.

Non-interactive mode can be used passing commands via --run-once:

$ zk-shell --run-once "create /foo 'bar'" localhost
$ zk-shell --run-once "get /foo" localhost
bar

Or piping commands through stdin:

$ echo "get /foo" | zk-shell --run-from-stdin localhost
bar

It's also possible to connect using an SSH tunnel, by specifying a host to use:

$ zk-shell --tunnel ssh-host zk-host

Dependencies

  • Python 2.7, 3.3, 3.4, 3.5 or 3.6
  • Kazoo >= 2.2

Testing and Development

Please see CONTRIBUTING.rst.

zk_shell's People

Contributors

abhilash1in avatar alge0 avatar alxn avatar azat avatar clvf avatar fcuny avatar justinrsweeney avatar msabramo avatar rgs1 avatar sellers avatar sowen avatar strajansebastian avatar svoutil avatar yasumoto avatar yriveiro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zk_shell's Issues

issue with tree command output

  1. The tree command shows the tree from right to left instead of left to right
  2. Using non-interactive mode e.g.
    export ZKSHELL_SRC=1; bin/zk-shell --run-once "tree /" localhost:2181 | less
    I get "'ascii' codec can't encode characters in position 0-2: ordinal not in range(128)" (followed by 'usage' help for the tree command)
    If in the non-interactive invocation above I dont pipe the output (ie. no |less), just let it go to stdout then the tree output does appear (no such ascii codec error).

documentation of cp command incorrect

The "README.md" file states:

Individual files can be copied between the local filesystem and ZooKeeper. Recursively copying from the filesystem to ZooKeeper is supported as well, but not the other way around since znodes can have content and children.
(CONNECTED) /> cp /etc/passwd zk://localhost:2181/passwd

This does not work for me.
I tried

(CONNECTED) /> cp file:///etc/passwd zk://localhost:2181/passwd

which worked for me. (notice "file://")

When running the command "cp help" this also is not mentioned.

Add verbose mode

It'd be nice if there was a verbose mode that showed the commands in kazoo being executed (be neat to see exactly what runs/is sent to zookeeper for each corresponding zk_shell command somehow).

How does the script connect to encrypted zk

When I connect to an encrypted zk, I get the following error:

ERROR:kazoo.client:Unhandled exception in connection loop
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/kazoo/protocol/connection.py", line 602, in _connect_attempt
response = self._read_socket(read_timeout)
File "/usr/local/lib/python3.9/dist-packages/kazoo/protocol/connection.py", line 453, in _read_socket
return self._read_response(header, buffer, offset)
File "/usr/local/lib/python3.9/dist-packages/kazoo/protocol/connection.py", line 391, in _read_response
callback_exception = EXCEPTIONSheader.err
File "/usr/local/lib/python3.9/dist-packages/kazoo/exceptions.py", line 54, in _invalid_error_code
raise RuntimeError('Invalid error code')
RuntimeError: Invalid error code
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 892, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.9/dist-packages/kazoo/protocol/connection.py", line 512, in zk_loop
if retry(self._connect_loop, retry) is STOP_CONNECTING:
File "/usr/local/lib/python3.9/dist-packages/kazoo/retry.py", line 126, in call
return func(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/kazoo/protocol/connection.py", line 552, in _connect_loop
status = self._connect_attempt(host, hostip, port, retry)
File "/usr/local/lib/python3.9/dist-packages/kazoo/protocol/connection.py", line 602, in _connect_attempt
response = self._read_socket(read_timeout)
File "/usr/local/lib/python3.9/dist-packages/kazoo/protocol/connection.py", line 453, in _read_socket
return self._read_response(header, buffer, offset)
File "/usr/local/lib/python3.9/dist-packages/kazoo/protocol/connection.py", line 391, in _read_response
callback_exception = EXCEPTIONSheader.err
File "/usr/local/lib/python3.9/dist-packages/kazoo/exceptions.py", line 54, in _invalid_error_code
raise RuntimeError('Invalid error code')
RuntimeError: Invalid error code

Python2 an Python3 issues with cp and mirror

Context

Debian Stretch, running python 2.7.13 and python 3.5.3 with zk-shell (1.2.3) installed via pip.

Issue 1

When doing a (recursive) cp or using mirror it stack traces. For example:

cp /src/foo /dst/bar true false false true 0

zk-shell fails with

Traceback (most recent call last):
  File "/usr/local/bin/zk-shell", line 22, in <module>
    CLI()()
  File "/usr/local/lib/python2.7/dist-packages/zk_shell/cli.py", line 165, in __call__
    shell.run(intro if first else None)
  File "/usr/local/lib/python2.7/dist-packages/xcmd/xcmd.py", line 423, in run
    self.cmdloop()
  File "/usr/lib/python2.7/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
    return func(arg)
  File "/usr/local/lib/python2.7/dist-packages/xcmd/xcmd.py", line 199, in wrapper
    return func(args[0], params)
  File "/usr/local/lib/python2.7/dist-packages/zk_shell/shell.py", line 625, in do_mirror
    self.copy(params, True, True, 0, True)
  File "/usr/local/lib/python2.7/dist-packages/zk_shell/shell.py", line 675, in copy
    src.copy(dst, recursive, max_items, mirror)
  File "/usr/local/lib/python2.7/dist-packages/zk_shell/copy_util.py", line 219, in copy
    self.do_copy(dst, opname)
  File "/usr/local/lib/python2.7/dist-packages/zk_shell/copy_util.py", line 241, in do_copy
    dst.write_path(self.read_path())
  File "/usr/local/lib/python2.7/dist-packages/zk_shell/copy_util.py", line 328, in write_path
    self.client.create(self.path, path_value.value, acl=acl, makepath=True)
  File "/usr/local/lib/python2.7/dist-packages/zk_shell/xclient.py", line 171, in create
    return super(XClient, self).create(path, value, acl, ephemeral, sequence, makepath)
  File "/usr/local/lib/python2.7/dist-packages/kazoo/client.py", line 943, in create
    sequence=sequence, makepath=makepath, include_data=include_data
TypeError: create_async() got an unexpected keyword argument 'include_data'

Issue 2

Upgrading to python3 solves the above issue but then the following problem appears when you use mirror since it prompts you for yes/no using raw_input.

Are you sure you want to replace /foo/ with /bar/? [y/n]: Traceback (most recent call last):
  File "/usr/local/bin/zk-shell", line 22, in <module>
    CLI()()
  File "/usr/local/lib/python3.5/dist-packages/zk_shell/cli.py", line 165, in __call__
    shell.run(intro if first else None)
  File "/usr/local/lib/python3.5/dist-packages/xcmd/xcmd.py", line 423, in run
    self.cmdloop()
  File "/usr/lib/python3.5/cmd.py", line 138, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib/python3.5/cmd.py", line 217, in onecmd
    return func(arg)
  File "/usr/local/lib/python3.5/dist-packages/xcmd/xcmd.py", line 199, in wrapper
    return func(args[0], params)
  File "/usr/local/lib/python3.5/dist-packages/zk_shell/shell.py", line 626, in do_mirror
    if params.skip_prompt or self.prompt_yes_no(question):
  File "/usr/local/lib/python3.5/dist-packages/xcmd/xcmd.py", line 366, in prompt_yes_no
    return strtobool(raw_input().lower())
NameError: name 'raw_input' is not defined

Expected result

It should just work out of the box :)

The README says zk-shell is compatible with both Python 2.7 and 3 so not quite sure what is going on, perhaps the first error is related to some dependency, I haven't looked into the code just yet.

Just writing this issue as a for your information.

igrep not support utf-8

  1. Create in zoo entry with utf-8 text (russian language for example)

image

check via zk-web - OK

  1. get value by zk-shell
    zk-shell --run-once "get /test/tester" 10.12.47.107:2181
    тестовая строка

zk-shell return correct value - OK

  1. igrep value
    zk-shell --run-once "igrep / 'тестовая строка' true" 10.12.47.107:2181

NOK - return empty result

summary command throws exception if object in root deleted

I am seeing the following exception, which I think happens if an object is deleted from / after summary starts running

Traceback (most recent call last):
File "bin/zk-shell", line 22, in
CLI()()
File "./zk_shell/cli.py", line 145, in call
rc = 0 if shell.onecmd(params.run_once) == None else 1
File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
return func(arg)
File "./zk_shell/shell.py", line 99, in wrapper
return func(_args, *_kwargs)
File "./zk_shell/xcmd.py", line 186, in wrapper
return func(args[0], params)
File "./zk_shell/shell.py", line 148, in wrapper
return func(self, params)
File "./zk_shell/shell.py", line 921, in do_summary
time.ctime(stat.created).ljust(32),
AttributeError: 'NoneType' object has no attribute 'created'

To reproduce

  • Standalone zookeeper server with one client connected which continuously does on an node in / :
    create
    sleep
    delete
    sleep
  • a shell script to loop and invoke zkshell --run-once summary
    Exception happens pretty quickly for me, <20 seconds in most cases

It looks like if a node is deleted, StatMap.get() returns a tuple which can contain None for the ZnodeStat and the code in shell.py does not handle this

sessions in chkzk is -1

I am seeing -1 for 'sessions' row of chkzk
I assume this should be the number of client connections?

This is consistent, against many clusters. As it happens they are all running 3.4.6

Client runs on Python version 2.7.6 on Ubuntu 14.04
zkshell 1.0.06 and also 1.1.1

intermittent issue with child_count

When I run zkshell with the parameters: --run-once "child_count ."
I sometimes get the following exception

Traceback (most recent call last):
File "/OMITTED/zk_shell-master/bin/zk-shell", line 22, in
CLI()()
File "./zk_shell/cli.py", line 129, in call
rc = 0 if shell.onecmd(params.run_once) == None else 1
File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
return func(arg)
File "./zk_shell/shell.py", line 95, in wrapper
return func(_args, *_kwargs)
File "./zk_shell/xcmd.py", line 148, in wrapper
return func(_args, *_kwargs)
File "./zk_shell/xcmd.py", line 160, in wrapper
return func(args[0], params)
File "./zk_shell/shell.py", line 138, in wrapper
return func(self, params)
File "./zk_shell/shell.py", line 732, in do_child_count
self.show_output("%s: %d", child, self._zk.child_count(child))
File "./zk_shell/xclient.py", line 251, in child_count
count = stat.numChildren
AttributeError: 'NoneType' object has no attribute 'numChildren'

This is after some of the top-level directories were correctly summarized.
In the times when the command completes okay (no exception), the count of children on the node which seems to be causing the problem is 0.
I am sorry but I am not able to share the zookeeper data-directory on which this problem occurs.

I am running under Python 2.7.3

ImportError: No module named 'colors'

when run:

bin/zk-shell

Traceback (most recent call last):
File "bin/zk-shell", line 13, in
from zk_shell.cli import CLI
File "./zk_shell/cli.py", line 11, in
from .shell import Shell
File "./zk_shell/shell.py", line 26, in
from colors import green, red
ImportError: No module named 'colors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "bin/zk-shell", line 18, in
from zk_shell.cli import CLI
File "./zk_shell/cli.py", line 11, in
from .shell import Shell
File "./zk_shell/shell.py", line 26, in
from colors import green, red
ImportError: No module named 'colors'

Python version is 3.3.0

python

Python 3.3.0 (default, Nov 17 2015, 23:35:39)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.

Who Can help me ?Thanks!

Provide option for overwriting existing files

There should exist a procedure which allows for the overwriting of files when attempting to copy files to ZK. Currently, if the file node already exists, an exception is raised, and the execution halted:

Copy failed; it may have partially completed. To return to a stable state, either fix the issue and re-run the command or manually revert.
Failure reason:
znode /path/to/file.yml exists

I can see two possible solutions to this:

  1. Rather than raising an exception and exiting, an option for user input could be given:
    znode /path/to/file.yml exists -- overwrite? [Y/n]: 
  1. Offering a dedicated command for overwriting existing files, so that you could choose between cp and say, update when sending your file.

zk_shell crash in python 3.7, async it's a reserved keyword

I'm trying to run zk-shell in python3.7 and I'm hitting this error:

Traceback (most recent call last):
  File "/usr/local/bin/zk-shell", line 13, in <module>
    from zk_shell.cli import CLI
  File "/usr/local/lib/python3.7/site-packages/zk_shell/cli.py", line 128
    async = False if params.sync_connect or not interactive else True
          ^
SyntaxError: invalid syntax

I think is related to the fact that in python 3.7 async is a reserved keyword.

ZOOKEEPER SERVER WARN: EndOfStreamException: Unable to read additional data from client, it probably closed the socket

CMD : zk-shell "$ZOO_SERVERS" --run-once 'mirror / json://!zookeeper-backup.json/'
Description : The above command executes successfully but after the execution Zookeeper server is throwing End of stream exception. it seems that zk-shell is closing the connection abruptly
Exception info:
2020-10-16 13:37:42,606 [myid:2] - WARN [NIOWorkerThread-1:NIOServerCnxn@364] - Unexpected exception
zoo2 | EndOfStreamException: Unable to read additional data from client, it probably closed the socket: address = /192.168.48.6:51942, session = 0x20022c059cb000a
zoo2 | at org.apache.zookeeper.server.NIOServerCnxn.handleFailedRead(NIOServerCnxn.java:163)
zoo2 | at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:326)
zoo2 | at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:522)
zoo2 | at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:154)
zoo2 | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
zoo2 | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
zoo2 | at java.base/java.lang.Thread.run(Unknown Source)

Traceback when starting zk-shell

guss-mbp:zkshell gus$ git clone https://github.com/rgs1/zk_shell.git
Cloning into 'zk_shell'...
remote: Counting objects: 1409, done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 1409 (delta 2), reused 0 (delta 0)
Receiving objects: 100% (1409/1409), 329.28 KiB | 0 bytes/s, done.
Resolving deltas: 100% (960/960), done.
Checking connectivity... done.
guss-mbp:zkshell gus$ cd zk_shell/
guss-mbp:zk_shell gus$ ls
CHANGES.md  CONTRIBUTING.md LICENSE     MANIFEST.in README.md   TODO        bin     setup.py    zk_shell
guss-mbp:zk_shell gus$ cd bin
guss-mbp:bin gus$ ./zk-shell localhost:2181
Traceback (most recent call last):
  File "./zk-shell", line 18, in <module>
    from zk_shell.cli import CLI
  File "../zk_shell/cli.py", line 14, in <module>
    from .shell import Shell
  File "../zk_shell/shell.py", line 21, in <module>
    from kazoo.exceptions import (
ImportError: No module named kazoo.exceptions

Are there setup steps required before running this? System Requirements? or is something missing from the current version. If so, can you tag the last working version for folks to use?

Support passing byte strings as values?

How would the maintainers feel about adding a flag to create and set to pass in byte strings (and a flag to get to display the raw byte string)? Happy to send a PR if this is ok.

Dependencies question

Out of the dependencies listed
ansicolors
kazoo>=2.0
nose
tabulate

Can you say which are needed only for testing and which are needed for zkshell itself?

Thanks

feedback/comments on command formats

So first off, I like this tool better than zkcli as it has more of the things I need when Im faced with performing zookeeper "surgery".

However I find the command line formatting very awkward with the appending of true, true to indicate things like recursive and async.

Would it be possible to move to cmd2 and utilize the @options fields? That way rather than needing a rmr subcommand you could just add a -rf option to rm. Ditto for things a -r for cp etc.

I think that would make the tool much better match common cmdline behaviors. You could define your own items such as --async or --ephemeral for additional items needed.

get crashes every time.

(CONNECTED) /> get consumers/test_group1/offsets/metric_thrift/0
Traceback (most recent call last):
  File "/usr/bin/zk-shell", line 22, in <module>
    CLI()()
  File "/usr/lib/python2.6/site-packages/zk_shell/cli.py", line 151, in __call__
    shell.run(intro if first else None)
  File "/usr/lib/python2.6/site-packages/zk_shell/xcmd.py", line 250, in run
    self.cmdloop()
  File "/usr/lib64/python2.6/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib64/python2.6/cmd.py", line 219, in onecmd
    return func(arg)
  File "/usr/lib/python2.6/site-packages/zk_shell/shell.py", line 80, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/zk_shell/xcmd.py", line 142, in wrapper
    return func(args[0], params)
  File "/usr/lib/python2.6/site-packages/zk_shell/shell.py", line 123, in wrapper
    return func(self, params)
  File "/usr/lib/python2.6/site-packages/zk_shell/shell.py", line 914, in do_get
    value, _ = self._zk.get(params.path, **kwargs)
  File "/usr/lib/python2.6/site-packages/zk_shell/xclient.py", line 167, in get
    value = value.decode(encoding="utf-8")
TypeError: decode() takes no keyword arguments

zookeeper-shell.sh get correct result

get /consumers/test_group1/offsets/metric_thrift/0
2977408
cZxid = 0x2449
ctime = Fri Jul 31 10:56:42 CST 2015
mZxid = 0x2558
mtime = Fri Jul 31 11:04:39 CST 2015
pZxid = 0x2449
cversion = 0
dataVersion = 6
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 7
numChildren = 0

zkshell reading from stdin

Is reading commands from supposed to work?

e.g. when running "zkshell host:port < file"
e.g. if the file contents is:
mntr
summary
exit

The output I see is:
Welcome to zk-shell (1.1.1)
(CONNECTING) /> Not connected and no host given.
(CONNECTING) /> Not connected.
(CONNECTING) />

Incase this is not supposed to work then FWIW I think reading from stdin, or enhancing --run-once to be able to read from a file would greatly increase the usefulness of zkshell

Tab completion breaks if there are argument ahead of the tab completed item

get_acls, get, and probably others support arguments after the path. There are times you are doing multiple operations and want to keep the additional arguments, but change the path. Typically this is done by moving the cursor back to the path, leaving the additional arguments in place. Doing this, however, results in odd tab completion results related to the values present in the later arguments.

--help is too terse

When using --help it would be nice to have:

  • info on the format of the hosts string (i.e.: how is auth encoded?)
  • general differences between interactive and non-interactive mode
  • pointers on how to get further info

The README is generally decent, so reusing some of that would be ok.

Something questions about zk_shell data restoring

Hello, I have the following questions, I hope to be answered:

  1. Zk_shell supports copying ZooKeeper cluster to json file, but how to restore from json file to original ZooKeeper cluster?
  2. Does zk_shell support copying data at a specified time?

Thank you for reading, and I am looking forward to your reply.

Import error

Here is an error I'm getting when trying to run it:

Traceback (most recent call last):
  File "bin/zk-shell", line 9, in <module>
    from zk_shell.cli import CLI
  File "./zk_shell/cli.py", line 13, in <module>
    from .shell import Shell
  File "./zk_shell/shell.py", line 39, in <module>
    from kazoo.exceptions import (
ImportError: No module named kazoo.exceptions

copy function broken

zk-shell throws exception and bails out when attempting cp file:///path/to/file /path/to/node

This line is the problem.

return PathValue("".join(os.path.join(fph.readlines())))

It should be recombining the lines of the file into a string, but instead is doubly wrong in joining the file lines as a path, then string joining that single object.

Confused, I did a bit of digging, and it appears to have been first introduced during a find/replace here: c262164#diff-5db5e6e1c9feecfe171c1b1e3ebf4392R416
That broke it first, but the fix here didn't fix it either: d5ddaab

zkshell exit code

running "zkshell --run-once summary localhost:21813"
gives a 0 exit code, even if no zookeeper is running on that port and the output is:
Failed to connect: Connection time-out
Not connected.

Python 2.7.6, Ubunto 14.04

Replace twitter-commons dependencies with paramiko

Currently, zk-shell has a dependency with twitter-commons, which creates the SSH tunnels to access the Zookeepers clusters. The problem is that twitter-commons requieres a lot more dependencies that doesn´t even use to create the tunnels, making zk-shell a lot more "overloaded" that it should be.

The use of a specific SSH library for creating tunnels called Paramiko (https://github.com/paramiko/paramiko) should solve this problem. The difference between these two is that Paramiko does not have a simple and direct function-call like twitter-commons has to create these tunnels

lhost, lport = TunnelHelper.create_tunnel(rhost, rport, self._tunnel)

Thus, the use of Paramiko to achieve the same functionality will requiere a new dependency within shell.py. This will be a new python source-code called something like tunnel.py
or paramiko_tunnel.py; it will have a class called TunnelHelper that encapsulates all the functionalities of Paramiko for the creation and maintenance of tunnels. This implementation should behave the same way as it is now, so the function create_tunnel shall return the same values as the one of twitter-commons.

Another needed feature in the current implementation is the use of some unit tests. In this new implementation I will try to include some unit tests to increase the test coverage of the program.

du command hangs

The scenario:

zookeeper 3.4.6 running in standalone mode
a (java) client connects to it and in a loop performs:
create /foo
sleep 100millis
delete /foo
sleep 100millis

now to zkshell, here is what I do to reproduce the problem:

! /bin/sh

while /bin/true
do
zkshell --run-once du localhost:2181
done

I sometimes get the correct result

Other times I get the below exception, this occurs quite frequently:
Exception in thread Thread-3 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 763, in run
File "/home/laden/.pex/install/kazoo-2.2.1-py2.py3-none-any.whl.8c0daadbce7c0827d2ff5c5d7ef717aad64a60a8/kazoo-2.2.1-py2.py3-none-any.whl/kazoo/protocol/connection.py", line 477, in zk_loop
File "/usr/lib/python2.7/threading.py", line 585, in set
File "/usr/lib/python2.7/threading.py", line 406, in notifyAll
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

And finally, after a while, zkshell/du hangs. This occurs least often and sometimes take a few mins
Not sure its helpful to mention but when this occurs, control-C does not seem to work to get out of it

stack trace after using --tunnel

Hey @franckcuny ,

I am getting this when using --tunnel after exiting zk-shell (via Ctrl-C, Ctrl-D or exit):

$ zk-shell --tunnel opsnest1 sdzookeeper.smf1.twitter.com
Welcome to zk-shell (1.0.08)
(CONNECTING) /> 
(CONNECTED) /> exit

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python2.7/site-packages/twitter/common/net/tunnel.py", line 40, in safe_kill
    po.wait()
  File "/usr/lib64/python2.7/subprocess.py", line 1376, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib64/python2.7/subprocess.py", line 476, in _eintr_retry_call
    return func(*args)
  File "/usr/lib/python2.7/site-packages/zk_shell/cli.py", line 88, in sigusr_handler
    raise StateTransition()
StateTransition
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python2.7/site-packages/twitter/common/net/tunnel.py", line 40, in safe_kill
    po.wait()
  File "/usr/lib64/python2.7/subprocess.py", line 1376, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib64/python2.7/subprocess.py", line 476, in _eintr_retry_call
    return func(*args)
  File "/usr/lib/python2.7/site-packages/zk_shell/cli.py", line 88, in sigusr_handler
    raise StateTransition()
zk_shell.cli.StateTransition

Any ideas?

cc: @Yasumoto

`dump --help` makes zk-shell quit

I am using zk-shell of version 1.0.04 on CentOS 7 machine. When I connected to the host and entered dump --help, zk-shell printed out the help message and quit itself. Is this expected behavior?

Copy sets acls before creating child znodes

Recursive cp sets acls before creating child znodes and so creation of the child znodes fails if the acls disallow creates.

Repo:
Welcome to zk-shell (0.97.3)
(CONNECTING) />
(CONNECTED) /> create /foo 'Hello'
(CONNECTED) /> create /foo/z 'Hello'
(CONNECTED) /> set_acls /foo world:anyone:r username_password:foo:foo:cdrwa
(CONNECTED) /> cp /foo /bar true true
Copy failed; it may have partially completed. To return to a stable state, either fix the issue and re-run the command or manually revert.
Failure reason:
Permission denied: Could not create znode /bar/z.
(CONNECTED) />

cp x zk:///some/path hangs forever

The (missing) destination for this cp command isn't rejected:

> cp x zk:///some/path

Proxy.from_string() should error out when given zk:///, since it's missing the host.

Readonly doesn't work?

It works fine by all other commands, but with --readonly, I can still rm nodes.

Is it a bug or I didn't get the right way using readonly?

Control+w deletes entire argument, not just path segment

This is one of the things I think the built in zkcli does right - if you hit control w it will delete the previous znode, not the entire string. This differing behavior is extremely annoying when mucking with a lot of znodes in a common path (even more annoying with the rm/rmr issues)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.