Redis AUTH Support about twemproxy HOT 14 OPEN

Twemproxy can only support redis commands that takes "key" as an argument. This is the case because sharding layer of twemproxy routes redis commands based on consistent hashed values on "key".

Since AUTH command does not take key as an argument we cannot support this in twemproxy.

Is there a reason why you use AUTH in your setup? Can you make it work without Redis AUTH?

from twemproxy.

Functionally, yes it would work. However, it is in use for a PHP session backend (slightly modified phpredis) which handles the PHP sessions for thousands of different clients. Thus, if someone were to open a direct connection from within PHP to the Redis nodes they would have access to other clients' session data. Thus, the authentication (compiled into the modified phpredis module) prohibits that.

Thus functionally, yes, but it isn't something that can be allowed for non-technical reasons.

I suppose I could write an auth proxy ....

from twemproxy.

That would cause a problem if the node it hashed to went down.

from twemproxy.

@therealbill I believe the way redis authentication works is that once a connection to a redis node has been established, the client sends the "AUTH password" command to the redis node before sending any other commands. A failure to do so would mean that the redis server configured with requirepass directive would reject any unautenticated connections.

An auth proxy will not solve the issue, because connections made my twemproxy to backend redis nodes would still be unauthenticated.

The right way to solve this problem IMHO, would be to implemented authentication on the client side so that you have an architecture where clients talking to twemproxy / redis nodes are trusted and hence don't required authentication.

thoughts?

from twemproxy.

The with proxy idea would essentially handle just the auth and do a direct proxy for all non-auth commands. Less than ideal but potentially workable. It would be ok if the twemcache to redis were unauthenticated in that case because the systems would be isolated from customer reachable nodes.

Ultimately, since the concern is potential unauthorized clients and authorized clients connecting from the same servers there has to be something sitting between the nodes and redis, or redis itself (or the proxy) needs to authenticate with the client.

Another possible route is to locally modify twemproxy to reject any commands other than get, del, and setex - the only commands our custom phpredis sends. That could potentially ameliorate the concerns with unauthorized connections have visibility to the keys and thus the session data from other customers. Though that might still be a significant patch we would have to maintain. At that point I'd have to consider writing a "smaller" version of twemproxy in Python that either handles the authentication from clients and proxies authenticated connections, or simply proxies a subset of commands and does the node key balance and routing operations.

Or the third option of hacking auth into twemproxy myself. Might be an excuse to dust off my C chops.

from twemproxy.

@therealbill I like the first option of "python proxy" only for commands that need auth. I believe this design is clean.

The second option would require hacking of twemproxy. You can easily achieve this by modifying (commenting out) some of the code between https://github.com/twitter/twemproxy/blob/master/src/proto/nc_redis.c#L396 and https://github.com/twitter/twemproxy/blob/master/src/proto/nc_redis.c#L805

Btw, let me know if rackspace is using twemproxy. If so, I will add you guys to the users list (https://github.com/twitter/twemproxy/blob/master/README.md#users)

I will close this issue for now. Let me know once you finish implementing the solution and how you ended up solving your auth issue.

from twemproxy.

Would you accept a pull request that adds auth support into twemproxy? In our deployment, we generally like to restrict the hosts that can connect to a particular system and using auth is a lightweight way of doing that (similar to putting a password on a mysql server).

from twemproxy.

Absolutely! How do you intend to implement AUTH support?

from twemproxy.

I think. just sending auth command to all connected redis is easy. but Handling irregular cases are hard, for example, one server's setting is different, or and when it failed, I think it can cause many problems which is hard to solve.

from twemproxy.

So, you are saying that when a connection is established to a server, you first send out an AUTH command on that server and only if that succeeds do you use that connection

from twemproxy.

@manjuraj I have an idea about this. it is using my former patch about reconnection.
It used "set" operation to check. if I change this to use "auth" command. I might think I will work well.
Of course, there is another problem, I should implement extra code for first connection.
reconnection code is only for reconnection. but I think it is not difficult. I will try. and tell you. Thank you.

from twemproxy.

thanks @charsyam! sorry I haven't reviewed your "liveness" code yet. but it is on my plate though

from twemproxy.

@manjraj, thanks for the go-ahead! We'll dig into this and get back to you, assuming charsyam@ doesn't beat us to it. Thanks for the effort on this software.

from twemproxy.

@manjuraj, @ejc3 Hi, guys, I sent a pull request to support redis auth.
and It doesn't need "liveness code"

#81

from twemproxy.