Giter Club home page Giter Club logo

rb's Introduction

rb test

logo

rb - the redis blaster.

The fastest way to talk to many redis nodes. Can do routing as well as blindly blasting commands to many nodes. How does it work?

For full documentation see rb.rtfd.org

Quickstart

Set up a cluster:

from rb import Cluster

cluster = Cluster({
    0: {'port': 6379},
    1: {'port': 6380},
    2: {'port': 6381},
    3: {'port': 6382},
}, host_defaults={
    'host': '127.0.0.1',
})

Automatic routing:

results = []
with cluster.map() as client:
    for key in range(100):
        client.get(key).then(lambda x: results.append(int(x or 0)))

print('Sum: %s' % sum(results))

Fanout:

with cluster.fanout(hosts=[0, 1, 2, 3]) as client:
    infos = client.info()

Fanout to all:

with cluster.fanout(hosts='all') as client:
    client.flushdb()

rb's People

Contributors

asottile-sentry avatar byk avatar cclauss avatar dcramer avatar evanpurkhiser avatar fengsp avatar getsentry-bot avatar mattrobenolt avatar mitsuhiko avatar tijko avatar tkaemming avatar wedamija avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rb's Issues

Fix simple typo: shold -> should

Issue Type

[x] Bug (Typo)

Steps to Replicate

  1. Examine rb/clients.py.
  2. Search for shold.

Expected Behaviour

  1. Should read should.

Semi-automated issue generated by
https://github.com/timgates42/meticulous/blob/master/docs/NOTE.md

To avoid wasting CI processing resources a branch with the fix has been
prepared but a pull request has not yet been created. A pull request fixing
the issue can be prepared from the link below, feel free to create it or
request @timgates42 create the PR. Alternatively if the fix is undesired please
close the issue with a small comment about the reasoning.

https://github.com/timgates42/rb/pull/new/bugfix_typo_should

Thanks.

Why MGET is not supported?

Personally I would say MGET is a big deal, pipelines works but maybe it is not as fast as MGET, is it possible that we could support it ?

The idea is simple, when we get one MGET call, we classify keys according to their redis nodes and send the MGET command to the corresponding node, which happens in parallel.

Mostly we use MGET a lot, maybe it is special enough to be supported as an API ?

Please public sdist on PyPI

Could you please publish sdist on PyPI?
It is preferred to build FreeBSD port (databases/py-rb) from sdist from PyPI.
Thanks!

Keyword arguments passed to `execute_command` cause error.

I've seen this with operations that accept a score_cast_func keyword argument, such as zrange and it's variants.

For example:

>>> with cluster.all() as client:
...   client.zrange('key', 0, -1)
... 
Traceback (most recent call last):
  File "<console>", line 2, in <module>
  File "/usr/local/lib/python2.7/site-packages/redis/client.py", line 1648, in zrange
    return self.execute_command(*pieces, **options)
TypeError: execute_command() got an unexpected keyword argument 'score_cast_func'

It looks like StrictRedis.zrange invokes FanoutClient.execute_command with withscores and score_cast_func keyword arguments, but execute_command doesn't accept any keyword arguments. It looks this would also affect MappingClient.execute_command but I didn't test that directly. RoutingClient is not affected.

Poll usage: Hangup on pending write and Buffer Limits

Currently our way to invoke poll is pretty hacky and it might cause us problems in two situations:

  • If we write to the buffer limit we might have to read first to get more. I don't see how that is in general possible with redis as redis cannot start processing unless it got the entire pipeline, but i need to verify that this is not an issue with writing to different redises
  • I think we might not support a hangup while writing. Need to verify this.

Add full usage docs

i.e. things like fanout aren't fully documented and its a bit tricky to find source classes in the code

sentry.exceptions.InvalidConfiguration: 'unix_socket_path'

I'm upgrading from an on premise sentry 7.6 installation and blocked by this error. I made a patch similar to this: getsentry/sentry#2322, changing 'path' with 'unix_socket_path' on sentry but trough above exception.

I cannot change the redis server because is being in used for another app so, could you please guide me on how can I implement this change? Thanks in advance!

When a redis server drops an already established connection, it fails loudly

This can be tested by setting CONFIG SET timeout 1 in Redis, then running:

from time import sleep
from rb import Cluster

cluster = Cluster({0:{'host': '127.0.0.1', 'port': 6379}})

with cluster.map() as c:
    c.get('foo')

sleep(2)

with cluster.map() as c:
    c.get('foo')

Yells with redis.exceptions.ConnectionError: Error while trying to write requests to redis..

The problem is the first poll on the socket is a close event before any data has been written to the socket.

We should detect this and attempt to reconnect.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.