kushalp / serfclient-py Goto Github PK

View Code? Open in Web Editor NEW

22.0 5.0 14.0 106 KB

Python client for the Serf orchestration tool

License: MIT License

Python 100.00%

serfclient-py's Introduction

serfclient

The Python interface to Serf, the decentralised solution for service discovery and orchestration.

Installation

serfclient requires a running Serf agent. See Serf's agent documentation for instructions.

To install serfclient, run the following command:

$ pip install serfclient

or alternatively (you really should be using pip though):

$ easy_install serfclient

or from source:

$ python setup.py install

Getting Started

from contextlib import closing
from serfclient.client import SerfClient

with closing(SerfClient()) as client:
    client.event('foo', 'bar')

Stream usage:

from contextlib import closing
from serfclient.client import SerfClient

with closing(SerfClient(timeout=None)) as client:
    for response in client.stream('*').body:
        print(response)

Development

serfclient requires a running Serf agent. See Serf's agent documentation for instructions.

You can run the tests using the following commands:

$ serf agent --tag foo=bar  # start serf agent
$ python setup.py test

serfclient-py's People

Contributors

Stargazers

Watchers

Forkers

loadavg-io paha derpston rmca msabramo charleswhchan thieman mignev kulik0v olalonde agamdua thedrow sdil mastha123

serfclient-py's Issues

Use a contextmanager as part of tests to close connections

Use a contextmanager to make sure we don't hold connections open in our tests: https://docs.python.org/2/library/contextlib.html#contextlib.closing

Places that we can do this:

Add stream/stop support

Could you please add support for streaming events, so it's possible to watch for e.g. member updates?

Make it easier to consume any serf environment config

There are two environment variables that Serf understands at the moment and there may be more in the future. Two of these SERF_RPC_ADDR and SERF_RPC_AUTH can be found here: https://github.com/hashicorp/serf/blob/06076d51e967471ba32b2e2a32c097981bff4670/command/rpc.go

Create a class EnvironmentConfig that parses these values and provides sane defaults if they're not set.

Strange address from client.members()

I am using serf 0.6.4 and serfclient 0.5.0. Following the instructions, I tried to use client.members(), but the address format is some what cryptic. How should I interpret the 'Addr' field?

If I use serf members directly from CLI, it shows the:

node-9         192.168.xx.yyy:7946  alive
...

Whereas if I use client.members():

{  
   'Members':[  
      {  
         'Status':'alive',
         'DelegateCur':4,
         'Addr':'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xc4\xa8?\x91',
         'Tags':{  

         },
         'ProtocolMax':2,
         'DelegateMin':2,
         'ProtocolMin':1,
         'ProtocolCur':2,
         'Port':7946,
         'DelegateMax':4,
         'Name':'node-9'
      },
      {  
         'Status':'alive',
         'DelegateCur':4,
         'Addr':'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\n\x05\x02\x0f',
         'Tags':{  

         },
         'ProtocolMax':2,
         'DelegateMin':2,
         'ProtocolMin':1,
         'ProtocolCur':2,
         'Port':7946,
         'DelegateMax':4,
         'Name':'node-1'
      },
      {  
         'Status':'alive',
         'DelegateCur':4,
         'Addr':'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xc0\xb8?\x86',
         'Tags':{  

         },
         'ProtocolMax':2,
         'DelegateMin':2,
         'ProtocolMin':1,
         'ProtocolCur':2,
         'Port':7946,
         'DelegateMax':4,
         'Name':'node-3'
      },
      {  
         'Status':'alive',
         'DelegateCur':4,
         'Addr':'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xc1\xa8?{',
         'Tags':{  

         },
         'ProtocolMax':2,
         'DelegateMin':2,
         'ProtocolMin':1,
         'ProtocolCur':2,
         'Port':7946,
         'DelegateMax':4,
         'Name':'node-5'
      }
   ]
}

The expected output format as defined by Hashicorp:
https://www.serfdom.io/docs/agent/rpc.html

    {"Members": [
        {
        "Name": "TestNode"
        "Addr": [127, 0, 0, 1],
        "Port": 5000,
        "Tags": {
            "role": "test"
        },
        "Status": "alive",
        "ProtocolMin": 0,
        "ProtocolMax": 3,
        "ProtocolCur": 2,
        "DelegateMin": 0,
        "DelegateMax": 1,
        "DelegateCur": 1,
        },
        ...]
    }

It appears the array elements are not unpacked correctly.

Does the library support Qery?

Looking at the library to use to talk to serf. I notice there is a client.event() function. Do you support serf query as well as events?

Thanks

New release to PyPI

It would be nice to have a new version on PyPI with recent additions like support for auth tokens and the EnvironmentConfig class (issue #16, PR #17).

Support v0.7 of Serf

Version 0.7 was released late in 2015: https://github.com/hashicorp/serf/releases/tag/v0.7.0

Figure out what's required to support it.

Error with `serf_client.members()`: `ValueError: invalid length of packed IP address string`

I run serf with cmd:

serf agent -bind=192.168.15.129

My code:

from serfclient.client import SerfClient


if __name__ == '__main__':
    serf_client = SerfClient()
    print(serf_client.stats())
    print(serf_client.members())

And traceback:

SerfResult<head={b'Error': b'', b'Seq': 1},body={b'runtime': {b'cpu_count': b'2', b'max_procs': b'2', b'arch': b'amd64', b'goroutines': b'30', b'os': b'linux', b'version': b'go1.5.2'}, b'serf': {b'left': b'0', b'encrypted': b'false', b'failed': b'0', b'query_queue': b'0', b'member_time': b'2', b'members': b'2', b'event_queue': b'0', b'query_time': b'1', b'event_time': b'1', b'intent_queue': b'0'}, b'event_handlers': {}, b'agent': {b'name': b'binh-lab-2'}, b'tags': {}}>


Traceback (most recent call last):
  File "/home/vcteam/.pycharm_helpers/pydev/pydevd.py", line 1531, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/vcteam/.pycharm_helpers/pydev/pydevd.py", line 938, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/vcteam/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/vcteam/plugins/learn/learn_serfclient.py", line 7, in <module>
    print(serf_client.members())
  File "/usr/local/lib/python3.4/dist-packages/serfclient/client.py", line 50, in members
    return self.connection.call('members')
  File "/usr/local/lib/python3.4/dist-packages/serfclient/connection.py", line 86, in call
    for message in unpacker:
  File "msgpack/_unpacker.pyx", line 469, in msgpack._unpacker.Unpacker.__next__ (msgpack/_unpacker.cpp:469)
  File "msgpack/_unpacker.pyx", line 400, in msgpack._unpacker.Unpacker._unpack (msgpack/_unpacker.cpp:400)
  File "/usr/local/lib/python3.4/dist-packages/serfclient/connection.py", line 152, in _decode_addr_key
    ip_addr = socket.inet_ntop(socket.AF_INET6, obj_dict[key])
ValueError: invalid length of packed IP address string

Test with newer versions of serf

As of creation, the latest version of serf available is 0.8.1. We should test with this version to make sure that we can support it.

mock not included in Python version<3.3, should add as requirement to setup.py

Was trying to run python setup.py test with Python 2.7. Since mock is not included in Python 2.7 (it is included in Python 3.3+), command fails.

Need to update test_requires in setup.py.

Payload should be optional

Although the comments say the payload is optional (https://github.com/KushalP/serfclient-py/blob/master/serfclient/client.py#L17) the code requires it. Not all events need a payload.

Potential resource leak?

We have a function that we call to get a list of serf members that are alive:

def get_members(tags=None):
        client = SerfClient()
        raw_members = client.members(status='alive', tags=tags)
        ...

During my review, I notice SerfClient() doesn't have a close() function call. Since SerfClient holds a reference to SerfConnection, and that in turn holds a reference to socket -- won't this result in a resource leak if we don't cleanup the client/connection/socket properly?

Basically I am looking for a way to release the SerfClient() resource properly. Am I missing anything?