redis-rb / redis-client Goto Github PK

View Code? Open in Web Editor NEW

120.0 120.0 54.0 687 KB

Simple low level client for Redis 6+

License: MIT License

Ruby 83.70% Shell 0.35% C 15.94%

redis-client's People

Contributors

Stargazers

Watchers

redis-client's Issues

ArgumentError(unknown keyword: :scheme)

Hi there!

I've update redis gem to 5.0.6 and now I get this error :

[2023-04-28T02:49:46.180956 #66989 #] ERROR [] - concerto.puma: There was an exception - ArgumentError(unknown keyword: :scheme)
[2023-04-28T02:49:46.182350 #66989 #] ERROR [] - concerto.puma: /Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-client-0.14.1/lib/redis_client/config.rb:21:in `initialize'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-client-0.14.1/lib/redis_client/config.rb:184:in `initialize'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:143:in `new'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:143:in `config'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-5.0.6/lib/redis/client.rb:23:in `config'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-5.0.6/lib/redis.rb:157:in `initialize_client'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-5.0.6/lib/redis.rb:73:in `initialize'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-store-1.9.2/lib/redis/store.rb:17:in `initialize'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-store-1.9.2/lib/redis/store/factory.rb:26:in `new'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-store-1.9.2/lib/redis/store/factory.rb:26:in `create'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-store-1.9.2/lib/redis/store/factory.rb:9:in `create'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-rack-2.1.4/lib/redis/rack/connection.rb:39:in `store'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-rack-2.1.4/lib/redis/rack/connection.rb:24:in `with'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-rack-2.1.4/lib/rack/session/redis.rb:82:in `with'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-rack-2.1.4/lib/rack/session/redis.rb:88:in `get_session_with_fallback'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-rack-2.1.4/lib/rack/session/redis.rb:38:in `block in find_session'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-rack-2.1.4/lib/rack/session/redis.rb:70:in `with_lock'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/redis-rack-2.1.4/lib/rack/session/redis.rb:37:in `find_session'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rack-2.2.7/lib/rack/session/abstract/id.rb:314:in `load_session'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actionpack-7.0.4.3/lib/action_dispatch/middleware/session/abstract_store.rb:45:in `block in load_session'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actionpack-7.0.4.3/lib/action_dispatch/middleware/session/abstract_store.rb:53:in `stale_session_check!'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actionpack-7.0.4.3/lib/action_dispatch/middleware/session/abstract_store.rb:45:in `load_session'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actionpack-7.0.4.3/lib/action_dispatch/request/session.rb:263:in `load!'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actionpack-7.0.4.3/lib/action_dispatch/request/session.rb:246:in `load_for_read!'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actionpack-7.0.4.3/lib/action_dispatch/request/session.rb:111:in `[]'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/warden-1.2.9/lib/warden/session_serializer.rb:31:in `fetch'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/warden-1.2.9/lib/warden/proxy.rb:224:in `user'
/Users/nicolas/PROJECTS/CONCERTO/concerto/app/channels/application_cable/connection.rb:15:in `block in find_verified_user'
/Users/nicolas/PROJECTS/CONCERTO/concerto/app/channels/application_cable/connection.rb:14:in `catch'
/Users/nicolas/PROJECTS/CONCERTO/concerto/app/channels/application_cable/connection.rb:14:in `find_verified_user'
/Users/nicolas/PROJECTS/CONCERTO/concerto/app/channels/application_cable/connection.rb:8:in `connect'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actioncable-7.0.4.3/lib/action_cable/connection/base.rb:173:in `handle_open'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sentry-rails-5.9.0/lib/sentry/rails/action_cable.rb:58:in `block in handle_open'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sentry-rails-5.9.0/lib/sentry/rails/action_cable.rb:23:in `block in capture'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sentry-ruby-5.9.0/lib/sentry/hub.rb:59:in `with_scope'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sentry-ruby-5.9.0/lib/sentry-ruby.rb:365:in `with_scope'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sentry-rails-5.9.0/lib/sentry/rails/action_cable.rb:15:in `capture'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sentry-rails-5.9.0/lib/sentry/rails/action_cable.rb:57:in `handle_open'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actioncable-7.0.4.3/lib/action_cable/server/worker.rb:59:in `block in invoke'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.0.4.3/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.0.4.3/lib/active_support/tagged_logging.rb:99:in `block in tagged'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.0.4.3/lib/active_support/tagged_logging.rb:37:in `tagged'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.0.4.3/lib/active_support/tagged_logging.rb:99:in `tagged'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actioncable-7.0.4.3/lib/action_cable/connection/tagged_logger_proxy.rb:24:in `tag'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actioncable-7.0.4.3/lib/action_cable/server/worker/active_record_connection_management.rb:16:in `with_database_connections'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.0.4.3/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actioncable-7.0.4.3/lib/action_cable/engine.rb:71:in `block (4 levels) in <class:Engine>'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.0.4.3/lib/active_support/execution_wrapper.rb:92:in `wrap'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actioncable-7.0.4.3/lib/action_cable/engine.rb:66:in `block (3 levels) in <class:Engine>'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.0.4.3/lib/active_support/callbacks.rb:127:in `instance_exec'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.0.4.3/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.0.4.3/lib/active_support/callbacks.rb:138:in `run_callbacks'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actioncable-7.0.4.3/lib/action_cable/server/worker.rb:42:in `work'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actioncable-7.0.4.3/lib/action_cable/server/worker.rb:58:in `invoke'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/actioncable-7.0.4.3/lib/action_cable/server/worker.rb:53:in `block in async_invoke'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:352:in `run_task'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:343:in `block (3 levels) in create_worker'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `loop'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `block (2 levels) in create_worker'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in `catch'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in `block in create_worker'
/Users/nicolas/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/logging-c6fd5e23f07d/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Let the test suite work with "external" redis server

I am trying to package redis-client for Fedora, because it seems to be used by redis. In this attempt, I'd like to run the test suite with "Fedora" distributed version of Redis. Nevertheless, this seems to be pain ATM, because the whole test suite is built around the idea that Redis is downloaded and build.

redis-client/test/support/redis_builder.rb

Line 35 in 7897ffe

FileUtils.mkdir_p(@tmp_dir)

Also, I wonder why toxiproxy, while it is IMHO similar case, is handled completely differently, via install-toxiproxy

command_builder should should allow SADD to pass an array as an argument

Using 0.11.2 when using the SADD command, if I pass an array as an argument I get this error: Unsupported command argument type: Array because he argument validation in command_builder does not allow this.

Fix is trivial. PR to follow.

Command Builder does not handle Big Decimal.

When passing in a BigDecimal to the command,I get a TypeError "Unsupported command argument type: BigDecimal" error.

Not sure if its a big deal as I can cast it to a float or an int before passing it in, but I have attempted a fix in #108

Thanks!

Support Sidekiq required feature set

As discussed in redis/redis-rb#1070 (comment), I'm trying to convert Sidekiq to use redis-client, so far the missing features I found:

url: connection param.
id connection param (CLIENT SETNAME)
helper for SCAN family of commands
~~Redis::Namespace equivalent (this one is debatable, could potentially be implemented as some kind of middleware that has COMMAND knowledge)~~
reconnect_attempts, Sidekiq sets it to 1. However if we are to implement this, we should have COMMAND knowledge to only retry readonly tagged commands.
Empty pipelines. Sidekiq test suite allowed to noticed empty pipeline were crashing.
The sidekiq test suite has some Redis sentinel configuration, we should support that as well (#8).

Hiredis "Unknown Timeout"

Hi,

Since upgrading to redis-client from the oldest one, I am starting to get Redis::TimeoutError: Unknown Error while using hiredis-client.

I saw in the documentation:

Contrary to the redis gem, redis-client doesn't protect against concurrent access. To use redis-client in concurrent environments, you MUST use a connection pool, or have one client per Thread or Fiber.

My rails app is running in multi-threaded Puma app, the question - is the comment about concurrency is relevant to hiredis-client as well?

Thanks

SystemStackError: stack level too deep

Error when hgetall a large object.

pry(main)> (1..(60_000)).each { |i| $redis_client.call("hset", "test", i, i) };
pry(main)> $redis_client.with { |r| r.call("hgetall", "test") }.size
=> 60000
pry(main)> (1..(70_000)).each { |i| $redis_client.call("hset", "test", i, i) };
pry(main)> $redis_client.with { |r| r.call("hgetall", "test") }.size
SystemStackError: stack level too deep
from /Users/jjwang/.rbenv/versions/2.6.8/lib/ruby/gems/2.6.0/gems/redis-client-0.5.1/lib/redis_client/ruby_connection/resp3.rb:151:in `parse_map'

Is there a script to refresh the test certificates?

It would be useful to have some script to refresh the test certificates. It would help the package in Fedora to prevent the test failures during the package lifetime.

ERR Protocol error: too big inline request (Redis::CommandError)

Linked issue: redis/redis-rb#1185

Redis 5.3.3
Ruby 3.1.3
Rails 7.0.4.2
redis gem 5.0.6
resque gem 2.4.0

Just upgraded the redis/resque gems and their dependencies and am getting this intermittent error ERROR -- : ERR Protocol error: too big inline request (Redis::CommandError)

The backtrace is not super helpful

E, [2023-03-14T14:41:13.543261 #44489] ERROR -- : ERR Protocol error: too big inline request (Redis::CommandError)
.....shared/bundle/ruby/3.1.0/gems/redis-5.0.6/lib/redis/subscribe.rb:52:in subscription' .....shared/bundle/ruby/3.1.0/gems/redis-5.0.6/lib/redis/subscribe.rb:17:in subscribe'
.....shared/bundle/ruby/3.1.0/gems/redis-5.0.6/lib/redis.rb:188:in _subscription' .....shared/bundle/ruby/3.1.0/gems/redis-5.0.6/lib/redis/commands/pubsub.rb:17:in subscribe'
.....shared/bundle/ruby/3.1.0/gems/actioncable-7.0.4.2/lib/action_cable/subscription_adapter/redis.rb:83:in block in listen' .....shared/bundle/ruby/3.1.0/gems/redis-client-0.12.1/lib/redis_client.rb:645:in ensure_connected'
.....shared/bundle/ruby/3.1.0/gems/redis-5.0.6/lib/redis/client.rb:104:in disable_reconnection' .....shared/bundle/ruby/3.1.0/gems/redis-5.0.6/lib/redis.rb:79:in without_reconnect'
.....shared/bundle/ruby/3.1.0/gems/actioncable-7.0.4.2/lib/action_cable/subscription_adapter/redis.rb:80:in listen' .....shared/bundle/ruby/3.1.0/gems/actioncable-7.0.4.2/lib/action_cable/subscription_adapter/redis.rb:154:in block in ensure_listener_running'

seems to indicate that action_cable is involved, but we use it a lot and having trouble tracking it down. I can't reproduce it and it appears on calls that don't trigger any actioncable actions.

The error is generated in redis, which has not changed, but looking at the gem difference, the error being raised in subscribe.rb is new. Am I looking at something that was happening all along, but was being swallowed by earlier versions of the redis gem?

Looking at the c extension code to try to figure out if there's a path that could be taken that doesn't prepend with "*" so it's treated as inline, but not having much luck.

Allow to run the test suite without Toxiproxy

Trying to package redis-client for Fedora, there is no Toxiproxy available in Fedora and there is more reasons why it is not really feasible to have it "somehow" available just for the tests. While it seems that currently most of the test cases are directed towards Toxiproxy, they could IMHO run directly against Redis. E.g. this test fails without Toxyproxy available:

$ ruby -Ilib:test -e 'Dir.glob "./test/redis_client/middlewares_test.rb", &method(:require)' -- -n /test_failing_call_instrumentation/
starting redis... started with pid=1798
Waiting for redis (port 16380)... ready.
Running test suite with driver: RedisClient::RubyConnection
Run options: -n /test_failing_call_instrumentation/ --seed 33075

# Running:

E

Finished in 0.003621s, 276.1378 runs/s, 0.0000 assertions/s.

  1) Error:
RedisClient::MiddlewaresTest#test_failing_call_instrumentation:
RedisClient::CannotConnectError: Connection refused - connect(2) for 127.0.0.1:16379
    /usr/share/ruby/socket.rb:1217:in `__connect_nonblock'
    /usr/share/ruby/socket.rb:1217:in `connect_nonblock'
    /usr/share/ruby/socket.rb:60:in `connect_internal'
    /usr/share/ruby/socket.rb:141:in `connect'
    /usr/share/ruby/socket.rb:645:in `block in tcp'
    /usr/share/ruby/socket.rb:231:in `each'
    /usr/share/ruby/socket.rb:231:in `foreach'
    /usr/share/ruby/socket.rb:635:in `tcp'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/lib/redis_client/ruby_connection.rb:49:in `initialize'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/lib/redis_client.rb:664:in `new'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/lib/redis_client.rb:664:in `block in connect'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/lib/redis_client/middlewares.rb:12:in `connect'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/test/redis_client/middlewares_test.rb:106:in `connect'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/lib/redis_client.rb:663:in `connect'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/lib/redis_client.rb:657:in `raw_connection'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/lib/redis_client.rb:624:in `ensure_connected'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/lib/redis_client.rb:208:in `call'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/test/support/client_test_helper.rb:38:in `setup'
    /builddir/build/BUILD/redis-client-0.12.1/usr/share/gems/gems/redis-client-0.12.1/test/redis_client/middlewares_test.rb:15:in `setup'

1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

But just simple change of ports such as:

$ sed -i '/REDIS.*79/ s/79/80/' test/support/servers.rb

Let the test case succeed:

$ ruby -Ilib:test -e 'Dir.glob "./test/redis_client/middlewares_test.rb", &method(:require)' -- -n /test_failing_call_instrumentation/
starting redis... started with pid=1819
Waiting for redis (port 16380)... ready.
Running test suite with driver: RedisClient::RubyConnection
Run options: -n /test_failing_call_instrumentation/ --seed 19990

# Running:

.

Finished in 0.004641s, 215.4873 runs/s, 646.4620 assertions/s.

1 runs, 3 assertions, 0 failures, 0 errors, 0 skips

So I wonder, would it be possible to run most of the parts of the test suite without Toxiproxy and use Toxiproxy only when necessary?

Attempt Reconnect on Redis::CommandError (Elasticache failover)

Hello!

I'm digging into some process crashes after an Elasticache failover. It appears as if a READONLY You can't write against a read only replica. (Redis::CommandError) error is returned after the failover. I believe this is because redis-client is still connected to the old primary node instead of the new primary node.

Is there a way to reset connection after a command error? Looking here, it looks like only ConnectionError and ProtocolError is handled, which I'm sure is for a very good reason.

Apologies if my assumptions are wrong here and I'm misreading the code. Thanks for the library!

`hiredis` bindings

While redis-client is noticeably faster than the redis gem with the default "ruby driver", the "hiredis driver" still has a noticeable performance edge on large responses and I doubt we can't do much about it.

Ultimately parsing line protocols in Ruby involve a lot of string allocations and copying, which really shows on benchmarks.

I think it would make sense for redis-client to have a native extension to bind against hiredis when installed on MRI platform.

Just a reminder: maybe remove Gemfile.lock from git

sorry about posting an issue, as there is no discussions tab available;

I'v seen many gems don't check Gemfile.lock into git, I suppose this gem probably should do the same.

Glad to see your work on this gem 👍

sscan without block

I have this code in Sidekiq:

       procs = conn.sscan_each("processes").to_a

This worked in 0.4.0 and broke in the latest version. Recall that I need code which works with both redis-rb and redis-client.

ArgumentError: can't issue an empty redis command
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.5.0/lib/redis_client/command_builder.rb:44:in `generate!'
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.5.0/lib/redis_client.rb:245:in `sscan'
    /Users/mperham/src/sidekiq/lib/sidekiq/api.rb:972:in `each'
    /Users/mperham/src/sidekiq/lib/sidekiq/api.rb:972:in `to_a'
    /Users/mperham/src/sidekiq/lib/sidekiq/api.rb:972:in `block in each'
    /Users/mperham/src/sidekiq/lib/sidekiq.rb:156:in `block in redis'

Question about redis/redis-rb vs redis-rb/redis-client

@byroot I'm confused as to what is happening with redis/redis-rb and this repo. You've been very active in the redis/redis-rb repo implementing these clients as the real clients in the redis gem. I'm wondering if it makes sense to completely switch to redis-client and redis-cluster-client from redis-rb and redis-clustering. Can you provide some guidance on the differences between the two and the future of the Ruby clients? Thanks for all your work!

Using Action Cable with redis-client

Hi,

I'm trying to use redis-client instead of redis with Action Cable but I'm getting the erro "Error loading the 'redis' Action Cable pubsub".

It's seems that Action Cable doesn't work with the default "redis" adaptar for Action Cable.

I have to use redis for Action Cable and redis-client for sidekiq or anyone is running redis-client for both?

ps: Sorry if here is not the right place for this kind of issue

Should RedisClient::TimeoutError descend from Timeout::Error?

Seems weird to raise network timeout errors which are not catchable by a standard rescue Timeout::Error. Does it make sense to extend Ruby's timeout error to gain compatibility? Or maybe just raise Timeout::Error directly? Is there a benefit to defining your own class vs reusing stdlib errors?

Password issue when refreshing Sentinels

Hi,
I have Redis Sentinels running with Docker compose and a password is required to authenticate with those sentinels.
I have noticed that when redis-client is refreshing the sentinels, the password does not persist.
I end up with sentinels that are missing the command AUTH which I believe is the reason why I am getting in my logs the error message "NOAUTH HELLO must be called with the client already authenticated".
Is this behavior expected ? Am I missing something ?
Thanks

PS: The issue started happening from version v0.14.0 and the introduction of the refresh_sentinels method.

Ruby 2.6 incompatible syntax in pooled.rb crashes Ruby when used with TracePoint

In the debug gem, there's a usage of TracePoint#instruction_sequence:

And when using it in Ruby 2.6 and encounters invalid syntax like the ones in pooled.rb, it crashes Ruby:

/opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/bundler/gems/debug-2cb44483c681/lib/debug/session.rb:129: [BUG] Segmentation fault at 0x0000000000000008
ruby 2.6.10p210 (2022-04-12 revision 67958) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0035 p:---- s:0157 e:000156 CFUNC  :instruction_sequence
c:0034 p:0016 s:0153 e:000151 BLOCK  /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/bundler/gems/debug-2cb44483c681/lib/debug/session.rb:129 [FINISH]
c:0033 p:---- s:0148 e:000147 CFUNC  :class_eval
c:0032 p:0047 s:0141 e:000140 BLOCK  /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/redis-client-0.7.4/lib/redis_client/pooled.rb:56 [FINISH]
c:0031 p:---- s:0137 e:000136 CFUNC  :each
c:0030 p:0084 s:0133 e:000132 CLASS  /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/redis-client-0.7.4/lib/redis_client/pooled.rb:55
c:0029 p:0007 s:0128 e:000127 CLASS  /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/redis-client-0.7.4/lib/redis_client/pooled.rb:6
c:0028 p:0014 s:0125 e:000124 TOP    /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/redis-client-0.7.4/lib/redis_client/pooled.rb:5 [FINISH]
c:0027 p:---- s:0122 e:000121 CFUNC  :require
c:0026 p:0049 s:0117 e:000116 TOP    /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/redis-client-0.7.4/lib/redis_client.rb:691 [FINISH]
c:0025 p:---- s:0114 e:000113 CFUNC  :require
c:0024 p:0006 s:0109 e:000108 TOP    /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/redis-client-0.7.4/lib/redis-client.rb:3 [FINISH]
c:0023 p:---- s:0106 e:000105 CFUNC  :require
c:0022 p:0006 s:0101 e:000100 TOP    /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/redis-5.0.3/lib/redis/client.rb:3 [FINISH]
c:0021 p:---- s:0098 e:000097 CFUNC  :require
c:0020 p:0042 s:0093 e:000092 TOP    /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/redis-5.0.3/lib/redis.rb:204 [FINISH]
c:0019 p:---- s:0090 e:000089 CFUNC  :require
c:0018 p:0006 s:0085 e:000084 TOP    /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/redis-namespace-1.9.0/lib/redis/namespace.rb:1 [FINISH]
c:0017 p:---- s:0082 e:000081 CFUNC  :require
c:0016 p:0013 s:0077 e:000076 TOP    /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/resque-2.4.0/lib/resque.rb:2 [FINISH]
c:0015 p:---- s:0074 e:000073 CFUNC  :require
c:0014 p:0044 s:0069 e:000068 TOP    /home/runner/work/sentry-ruby/sentry-ruby/sentry-resque/spec/spec_helper.rb:5 [FINISH]
c:0013 p:---- s:0066 e:000065 CFUNC  :require
c:0012 p:0006 s:0061 e:000060 TOP    /home/runner/work/sentry-ruby/sentry-ruby/sentry-resque/spec/sentry/resque/configuration_spec.rb:1 [FINISH]
c:0011 p:---- s:0058 e:000057 CFUNC  :load
c:0010 p:0008 s:0053 e:000052 METHOD /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/rspec-core-3.11.0/lib/rspec/core/configuration.rb:2115
c:0009 p:0023 s:0044 e:000043 BLOCK  /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/rspec-core-3.11.0/lib/rspec/core/configuration.rb:[161](https://github.com/getsentry/sentry-ruby/runs/8225389799?check_suite_focus=true#step:6:162)5 [FINISH]
c:0008 p:---- s:0039 e:000038 CFUNC  :each
c:0007 p:0024 s:0035 e:000034 METHOD /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/rspec-core-3.11.0/lib/rspec/core/configuration.rb:1613
c:0006 p:0039 s:0031 e:000030 METHOD /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:102
c:0005 p:0008 s:0025 e:000024 METHOD /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:86
c:0004 p:0072 s:0019 e:000018 METHOD /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:71
c:0003 p:0020 s:0011 e:000010 METHOD /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:45
c:0002 p:0021 s:0006 e:000005 EVAL   /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/gems/2.6.0/gems/rspec-core-3.11.0/exe/rspec:4 [FINISH]
c:0001 p:0000 s:0003 E:001e20 (none) [FINISH]

Reproduction Scripts

Here's a high-level reproduction script:

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  ruby "~> 2.6.6"

  gem "debug"
  gem "redis-client"
end

We can also reproduce the error without any library:

TracePoint.new(:script_compiled) do |tp|
  puts tp.instruction_sequence
end.enable

class Foo
  begin
    method = "foo"
    class_eval <<~RUBY, __FILE__, __LINE__ + 1
      def #{method}(...)
        with { |r| r.#{method}(...) }
      end
    RUBY
  rescue SyntaxError
  end
end

Solution

I know it's actually an issue of Ruby 2.6, but it's already EOL and I don't see we can patch it for this.

So the other option is to prevent invalid code being loaded in pooled.rb. That'll avoid the issue:

TracePoint.new(:script_compiled) do |tp|
  puts tp.instruction_sequence
end.enable

class Foo
  # doesn't crash Ruby!
  if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new("2.7.0")
    method = "foo"
    class_eval <<~RUBY, __FILE__, __LINE__ + 1
      def #{method}(...)
        with { |r| r.#{method}(...) }
      end
    RUBY
  end
end

Given that redis-client is a very popular gem, making it work along with Ruby 2.6 and the debug gem is worth the change imo. WDYT? @casperisfine

Provide URL getter

It's really useful to be able to see a URL of the configured Redis client. For instance, the footer of the Sidekiq Web UI shows the active Redis URL. Rather than Sidekiq piecing it together, would it make sense to have a RedisClient::Config#url method?

Add sentinel support

Ref: https://redis.io/docs/reference/sentinel-clients/

Should redis-client provide connection pooling by default?

Some Redis clients like redis-py are pooled by default.

It may make sense to do the same.

redis-client does not correctly auth without a username

I tried using redis-client on heroku, and ran into some issues.

Heroku uses redis URLs that do not have a username. They look like rediss://:password@host:6379

According to the redis docs the implicit username is default.

However, when I call RedisClient.config(url: 'rediss://:password@host:6379') I get the username set to password, and not default. This causes the subsequent redis auth to fail.

I assume this is because of

redis-client/lib/redis_client/config.rb

Line 125 in 81460ca

kwargs[:username] ||= uri.user && uri.password

I don't know the history there, but it seems like uri.user.empty? ? 'default' : uri.user might make more sense. (I'm not sure what the preferred style is there, and if it needs to be defensive for nil)

I (and future searchers) can work around this by manually specifying the URL. In my Heroku setup, I use:

RedisClient.config(url: ENV.fetch("REDIS_URL"), ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE }, username: 'default' )

Add common convenience methods

Similar to the https://github.com/redis/redis-rb gem, it would be very helpful to add common (or all) Redis keyword calls, so instead of redis.call('SET', 'foo', 'bar'), we could call redis.set('foo', 'bar').

Unhandled EOFError

This error may be raised by fill_buffer

Since the Redis gem relies on RedisClient, I'd expect to have the same error raised when using Redis, but it appears that the Redis gem handles this. I was wondering if this is a bug in the RedisClient or something undocumented which libraries using RedisClient need to handled

redis-client-0.12.1/lib/redis_client/ruby_connection/buffered_io.rb:145 in block in fill_buffer
redis-client-0.12.1/lib/redis_client/ruby_connection/buffered_io.rb:122 in loop
redis-client-0.12.1/lib/redis_client/ruby_connection/buffered_io.rb:122 in fill_buffer
redis-client-0.12.1/lib/redis_client/ruby_connection/buffered_io.rb:114 in ensure_remaining
redis-client-0.12.1/lib/redis_client/ruby_connection/buffered_io.rb:85 in getbyte
redis-client-0.12.1/lib/redis_client/ruby_connection/resp3.rb:113 in parse
redis-client-0.12.1/lib/redis_client/ruby_connection/resp3.rb:50 in load
redis-client-0.12.1/lib/redis_client/ruby_connection.rb:123 in read
redis-client-0.12.1/lib/redis_client/connection_mixin.rb:18 in call
redis-client-0.12.1/lib/redis_client.rb:210 in block (2 levels) in call
redis-client-0.12.1/lib/redis_client/middlewares.rb:16 in call
redis-client-0.12.1/lib/redis_client.rb:209 in block in call
redis-client-0.12.1/lib/redis_client.rb:626 in ensure_connected
redis-client-0.12.1/lib/redis_client.rb:208 in call
redlock-2.0.1/lib/redlock/client.rb:172 in block (2 levels) in lock
connection_pool-2.3.0/lib/connection_pool.rb:65 in block (2 levels) in with
connection_pool-2.3.0/lib/connection_pool.rb:64 in handle_interrupt
connection_pool-2.3.0/lib/connection_pool.rb:64 in block in with
connection_pool-2.3.0/lib/connection_pool.rb:61 in handle_interrupt
connection_pool-2.3.0/lib/connection_pool.rb:61 in with
connection_pool-2.3.0/lib/connection_pool/wrapper.rb:14 in with
redlock-2.0.1/lib/redlock/client.rb:171 in block in lock

Monitoring and observability

The next big issue I can think of for adoption will be metrics and observability. There are lots of gems and APMs which patch into the Redis client in order to monitor usage and track app performance.

Here's AppSignal's hook:

https://github.com/appsignal/appsignal-ruby/blob/main/lib/appsignal/hooks/redis.rb

Here's Skylight's:

https://github.com/skylightio/skylight-ruby/blob/master/lib/skylight/probes/redis.rb

Is compatibility possible or worthwhile? Do you have or want to provide a public hook point for APMs?

Use more Rubocop defaults

There are many instances of Enabled: false in https://github.com/redis-rb/redis-client/blob/master/.rubocop.yml. Why even use Rubocop if all of the most important cops are disabled?

Ruby 2.7 issue

/Users/mperham/.gem/ruby/2.7.5/gems/redis-client-0.8.0/lib/redis_client/decorator.rb:26: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
/Users/mperham/.gem/ruby/2.7.5/gems/redis-client-0.8.0/lib/redis_client.rb:254: warning: The called method `blocking_call' is defined here

not an issue, just a question

it still still very much discouraged to use

Hi, this line of changelog confuses me a little( two still), was that mis-typed?

RedisClient::ConnectionError with pubsub

Hello,

I just switched from redis-rb to redis-client and I see some connection errors on one of my production instance.

I use the code :

$conn = RedisClient.new(reconnect_attempts: 1).pubsub and $conn.call('publish', event, payload.to_json).

After some time I see some RedisClient::ConnectionError Not connected or Broken pipe errors raised by

/app/shared/bundle/ruby/3.0.0/gems/hiredis-client-0.14.1/lib/redis_client/hiredis_connection.rb:94:in `_write'
/app/shared/bundle/ruby/3.0.0/gems/hiredis-client-0.14.1/lib/redis_client/hiredis_connection.rb:94:in `write'
/app/shared/bundle/ruby/3.0.0/gems/redis-client-0.14.1/lib/redis_client.rb:421:in `call'
/app/releases/20230522090020/lib/pubsub_singleton.rb:69:in `publish'

Is the reconnect_attempts also used after a broken connection ? Am I missing a parameter that used to make redis-rb work and not redis-client?

Thanks for your help and for the good work!

Allow subclasses of String as values.

I have encountered this error using Redis 5.x, which uses redis-client. Which I think could pop up in several compatibility issues (redis/redis-rb#1142).

 Failure/Error: redis.setex key, TTL, value
 
 TypeError:
   Unsupported command argument type: ActiveSupport::SafeBuffer

I would expect a subclass of String to be ok, to pass as a value. But the redis-client gem that redis uses in 5.x is very strict.
Instead of

case value
when String
end

It does

method = DUMP_TYPES.fetch(object.class) do
  raise TypeError, "Unsupported command argument type: #{object.class}"
end

Consider whether the type checking is too strict.

Support for ActiveSupport::Duration in command_builder.rb

I am not sure, if there is any specific reason causing it to be excluded. I was updating Sidekiq version fro 6.0 to 7.0 and came across this issue.

Sidekiq.redis do |r|
  r.pipelined do |p|
    p.rpush('dummy_key, 'dummy_file_path')
    p.expire('dummy_key', 1.day)
  end
end

# raised this exception
TypeError: Unsupported command argument type: ActiveSupport::Duration

Can you please help me out here with better insights? Much appreciated 👍🏻

Instrumentation Middleware introduced too late for desired error reporting

I'm excited to see the introduction of the Instrumentation and Middlewares API for this library! While updating the New Relic Ruby agent's Redis 5 instrumentation, I had an opportunity to test it out. (If you're curious, see: newrelic/newrelic-ruby-agent#1611)

Registering the middleware and writing the instrumentation for each method was a breeze, though it took me a while to notice its addition to the README. The methods chosen are the same methods the agent used for redis library instrumentation for years. The client object's accessibility from these methods also maintains parity with earlier versions, though accessing information like database name needed to be adjusted to client.db instead of just db.

The only issue I came up against was with error reporting. Where the middleware gets registered and instrumented leaves a much larger gap between earlier Redis operations than where the methods were located in version 4.8.0. In New Relic, this looked like the errors still getting reported on the overall request, but the individual segment where the error was raised (ex. the #get method call) wouldn't have the error.

We could still capture segment-level errors using our traditional method chaining/module prepending strategy.

This caused us to ultimately use the Middleware API only for #call_pipelined, because we couldn't get access to the command values through any other means.

Unfortunately, I don't have any solutions to offer at this time. Since my work on Redis instrumentation has wrapped up for now, I wanted to drop this here in case others have ideas or similar experiences, and to remind me of where these thoughts left off when I am able to circle back.

Thank you for your efforts maintaining this library and making instrumentation more accessible!

Bug: hiredis driver conflict with the hiredis gem

I noticed this when working on the semian adapter.

Repro (linux only):

require 'redis'
require 'redis-client'

Redis.new(host: "redis", driver: :hiredis).ping
RedisClient.new(host: "redis", driver: :hiredis).call("ping")

From what I gather, we end up with redisContext->fd = 0, which later causes Errno::ENOTSOCK

Provide invalid command in error

RedisClient::CommandError: ERR syntax error
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.4.0/lib/redis_client/connection_mixin.rb:9:in `call'
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.4.0/lib/redis_client.rb:177:in `block (2 levels) in call'
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.4.0/lib/redis_client/middlewares.rb:8:in `call'
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.4.0/lib/redis_client.rb:176:in `block in call'
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.4.0/lib/redis_client.rb:506:in `ensure_connected'
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.4.0/lib/redis_client.rb:175:in `call'
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.4.0/lib/redis_client.rb:475:in `scan_list'
    /Users/mperham/.gem/ruby/3.0.3/gems/redis-client-0.4.0/lib/redis_client.rb:224:in `scan'
    /Users/mperham/src/ent/lib/sidekiq-ent/limiter/status.rb:25:in `each'

I'd like to see the actual command that generated this error, similar to a compiler emitting the line of code which generated an error. Since you are processing the response, you should have the request context. Can you include the line that was sent to Redis?

`hiredis-client` failed to compile on FreeBSD

OS: FreeBSD 13.1-RELEASE amd64.
Ruby: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-freebsd13.1].

current directory: /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/hiredis-client-0.11.2/ext/redis_client/hiredis
make DESTDIR\=
compiling hiredis_connection.c
linking shared-object redis_client/hiredis_connection.so
ld: error: /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/hiredis-client-0.11.2/ext/redis_client/hiredis/export.clang:1: unexpected
EOF
>>> _Init_hiredis_connection
>>> ^
clang: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1

Full log

Installing hiredis-client 0.11.2 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/hiredis-client-0.11.2/ext/redis_client/hiredis
/root/.rbenv/versions/3.1.2/bin/ruby -I /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0 -r ./siteconf20221122-34326-szp23j.rb extconf.rb
checking for rb_hash_new_capa() in ruby.h... no
checking for openssl/ssl.h... yes
cc -std=c99 -pedantic -c -O3 -fPIC  -DHIREDIS_TEST_SSL -I/usr/local/opt/openssl/include -Wall -W -Wstrict-prototypes -Wwrite-strings
-Wno-missing-field-initializers -g -ggdb alloc.c
cc -std=c99 -pedantic -c -O3 -fPIC  -DHIREDIS_TEST_SSL -I/usr/local/opt/openssl/include -Wall -W -Wstrict-prototypes -Wwrite-strings
-Wno-missing-field-initializers -g -ggdb net.c
cc -std=c99 -pedantic -c -O3 -fPIC  -DHIREDIS_TEST_SSL -I/usr/local/opt/openssl/include -Wall -W -Wstrict-prototypes -Wwrite-strings
-Wno-missing-field-initializers -g -ggdb hiredis.c
cc -std=c99 -pedantic -c -O3 -fPIC  -DHIREDIS_TEST_SSL -I/usr/local/opt/openssl/include -Wall -W -Wstrict-prototypes -Wwrite-strings
-Wno-missing-field-initializers -g -ggdb sds.c
cc -std=c99 -pedantic -c -O3 -fPIC  -DHIREDIS_TEST_SSL -I/usr/local/opt/openssl/include -Wall -W -Wstrict-prototypes -Wwrite-strings
-Wno-missing-field-initializers -g -ggdb async.c
cc -std=c99 -pedantic -c -O3 -fPIC  -DHIREDIS_TEST_SSL -I/usr/local/opt/openssl/include -Wall -W -Wstrict-prototypes -Wwrite-strings
-Wno-missing-field-initializers -g -ggdb read.c
cc -std=c99 -pedantic -c -O3 -fPIC  -DHIREDIS_TEST_SSL -I/usr/local/opt/openssl/include -Wall -W -Wstrict-prototypes -Wwrite-strings
-Wno-missing-field-initializers -g -ggdb sockcompat.c
ar rcs libhiredis.a alloc.o net.o hiredis.o sds.o async.o read.o sockcompat.o
cc -std=c99 -pedantic -c -O3 -fPIC  -DHIREDIS_TEST_SSL -I/usr/local/opt/openssl/include -Wall -W -Wstrict-prototypes -Wwrite-strings
-Wno-missing-field-initializers -g -ggdb ssl.c
ar rcs libhiredis_ssl.a ssl.o
creating Makefile

current directory: /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/hiredis-client-0.11.2/ext/redis_client/hiredis
make DESTDIR\= clean

current directory: /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/hiredis-client-0.11.2/ext/redis_client/hiredis
make DESTDIR\=
compiling hiredis_connection.c
linking shared-object redis_client/hiredis_connection.so
ld: error: /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/hiredis-client-0.11.2/ext/redis_client/hiredis/export.clang:1: unexpected
EOF
>>> _Init_hiredis_connection
>>> ^
clang: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1

Stop.
make: stopped in /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/hiredis-client-0.11.2/ext/redis_client/hiredis

make failed, exit code 1

Gem files will remain installed in /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/hiredis-client-0.11.2 for inspection.
Results logged to /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/extensions/x86_64-freebsd-13/3.1.0/hiredis-client-0.11.2/gem_make.out

  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:95:in `run'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:44:in `block in make'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:36:in `each'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:36:in `make'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/ext_conf_builder.rb:63:in `block in build'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/tempfile.rb:317:in `open'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/ext_conf_builder.rb:26:in `build'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:161:in `build_extension'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:195:in `block in build_extensions'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:192:in `each'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:192:in `build_extensions'
  /root/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/installer.rb:853:in `build_extensions'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/rubygems_gem_installer.rb:72:in `build_extensions'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/rubygems_gem_installer.rb:28:in `install'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/source/rubygems.rb:207:in `install'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/installer/gem_installer.rb:54:in `install'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/installer/gem_installer.rb:16:in `install_from_spec'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/installer/parallel_installer.rb:186:in `do_install'
/root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/installer/parallel_installer.rb:177:in `block in
worker_pool'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/worker.rb:62:in `apply_func'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/worker.rb:57:in `block in process_queue'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/worker.rb:54:in `loop'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/worker.rb:54:in `process_queue'
  /root/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/bundler-2.3.26/lib/bundler/worker.rb:91:in `block (2 levels) in create_threads'

An error occurred while installing hiredis-client (0.11.2), and Bundler cannot continue.

In Gemfile:
  hiredis-client

irb(main):001:0> RbConfig::CONFIG['CC']
=> "clang"

# clang -v
FreeBSD clang version 13.0.0 ([email protected]:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
Target: x86_64-unknown-freebsd13.1
Thread model: posix
InstalledDir: /usr/bin

Broken Pipe and EOFErrors

We recently switched from Sidekiq 6.5.8 to 7.1 (where there was a change to use redis-client) and have noticed a significant increase in errors, all coming from RedisClient. All errors follow one of three forms:

RedisClient::ConnectionError: EOFError
/app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection.rb:101:in `rescue in read': EOFError (RedisClient::ConnectionError)

RedisClient::ConnectionError: Broken pipe
/app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection.rb:88:in `rescue in write_multi': Broken pipe (RedisClient::ConnectionError)

RedisClient::ConnectionError: Broken pipe
/app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection.rb:76:in `rescue in write': Broken pipe (RedisClient::ConnectionError)
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection.rb:73:in `write'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/connection_mixin.rb:30:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:211:in `block (2 levels) in call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/middlewares.rb:16:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/contrib/redis/trace_middleware.rb:23:in `block in call'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/trace_operation.rb:192:in `block in measure'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/span_operation.rb:150:in `measure'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/trace_operation.rb:192:in `measure'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/tracer.rb:380:in `start_span'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/tracer.rb:160:in `block in trace'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/context.rb:45:in `activate!'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/tracer.rb:159:in `trace'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing.rb:18:in `trace'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/contrib/redis/trace_middleware.rb:13:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:210:in `block in call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:626:in `ensure_connected'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:209:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/decorator.rb:26:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/scripting.rb:21:in `block in call'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:107:in `block (2 levels) in with'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:106:in `handle_interrupt'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:106:in `block in with'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:103:in `handle_interrupt'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:103:in `with'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/scripting.rb:20:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/senate.rb:148:in `update_leader'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/senate.rb:124:in `election'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/senate.rb:103:in `cycle'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.1.0/lib/sidekiq/component.rb:10:in `watchdog'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.1.0/lib/sidekiq/component.rb:19:in `block in safe_thread'
/app/vendor/ruby-3.2.1/lib/ruby/3.2.0/openssl/buffering.rb:415:in `syswrite_nonblock': Broken pipe (Errno::EPIPE)
	from /app/vendor/ruby-3.2.1/lib/ruby/3.2.0/openssl/buffering.rb:415:in `write_nonblock'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection/buffered_io.rb:64:in `block in write'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection/buffered_io.rb:63:in `loop'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection/buffered_io.rb:63:in `write'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection.rb:74:in `write'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/connection_mixin.rb:30:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:211:in `block (2 levels) in call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/middlewares.rb:16:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/contrib/redis/trace_middleware.rb:23:in `block in call'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/trace_operation.rb:192:in `block in measure'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/span_operation.rb:150:in `measure'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/trace_operation.rb:192:in `measure'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/tracer.rb:380:in `start_span'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/tracer.rb:160:in `block in trace'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/context.rb:45:in `activate!'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/tracer.rb:159:in `trace'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing.rb:18:in `trace'
	from /app/vendor/bundle/ruby/3.2.0/gems/ddtrace-1.11.1/lib/datadog/tracing/contrib/redis/trace_middleware.rb:13:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:210:in `block in call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:626:in `ensure_connected'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client.rb:209:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/redis-client-0.14.1/lib/redis_client/decorator.rb:26:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/scripting.rb:21:in `block in call'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:107:in `block (2 levels) in with'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:106:in `handle_interrupt'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:106:in `block in with'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:103:in `handle_interrupt'
	from /app/vendor/bundle/ruby/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:103:in `with'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/scripting.rb:20:in `call'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/senate.rb:148:in `update_leader'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/senate.rb:124:in `election'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-ent-7.1.0/lib/sidekiq-ent/senate.rb:103:in `cycle'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.1.0/lib/sidekiq/component.rb:10:in `watchdog'
	from /app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.1.0/lib/sidekiq/component.rb:19:in `block in safe_thread'

#116 seemed possibly related, but it looks like Sidekiq already defaults to having reconnect_attempts set to 1 so maybe a dead end there.

I've struggled to figure out what the actual cause is. Any ideas what direction to head on this?

Sidekiq stuck in redis-client buffered_io

Like #92, I'm seeing enough people raise a similar issues that I think there's a subtle bug somewhere. The bug results in a Sidekiq process hanging, with all threads blocked on the same backtrace as #92.

I don't have any more information at this time but I will add more info as I get it.

RedisClient::ConnectionError Broken pipe

We exchanged our old redis setup:

@pool = ConnectionPool.new(size: 5, timeout: 5) { Redis.new(url: Settings.redis.url, ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE }) }

with redis-client this morning:

redis_config = RedisClient.config(url: Settings.redis.url, ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE })
    @pool = redis_config.new_pool(timeout: 5, size: 5)

Most things work fine, but we got a few RedisClient::ConnectionError Broken pipe errors since upgrading. Any pointers on how to debug this, and what the reason could be?

About error handling of pipelining

Hello,

Since pipelining is a bulk processing, probably, there might be a use case to want to handle errors by caller-side. I'm not sure yet either RedisClient#pipelined method should raise the first error or not.

In case of redis-cluster-client, the client should handle errors of MOVED and ASK redirection in pipeline. Of course it is a matter of the cluster client, so it can be implemented with overwriting or a dedicated method. But I'd say that it would be better to have the same behavior between RedisClient and RedisClient::Cluster.

What are your thoughts on this?

cli = RedisClient.config.new_client
replies = cli.pipelined do |pi|
  # pi.call('something')
end

# Case: 1
errors = replies.select { |reply| reply.is_a?(RedisClient::Error) }
raise MyError, errors.map(&:message) unless errors.empty?

# Case: 2
replies.each_with_index do |reply, i|
  if reply.is_a?(RedisClient::Error)
    # do something
  else
    # do something
  end
end

$ (printf "PING\r\nPANG\r\nPING\r\nQUIT\r\n"; sleep 0.1) | nc 127.0.0.1 6379
+PONG
-ERR unknown command 'PANG', with args beginning with:
+PONG
+OK

irb(main):001:0> cli = RedisClient.config.new_client
=> #<RedisClient redis://localhost:6379/0>

irb(main):005:1* cli.pipelined do |pi|
irb(main):006:1*   pi.call('PING')
irb(main):007:1*   pi.call('PANG')
irb(main):008:1*   pi.call('PING')
irb(main):009:1*   pi.call('QUIT')
irb(main):010:0> end
/home/kasuga/vcs/redis-client/lib/redis_client/connection_mixin.rb:34:in `call_pipelined': ERR unknown command 'PANG', with args beginning with:  (RedisClient::CommandError)
        from /home/kasuga/vcs/redis-client/lib/redis_client.rb:287:in `block (2 levels) in pipelined'
        from /home/kasuga/vcs/redis-client/lib/redis_client/middlewares.rb:8:in `call'
        from /home/kasuga/vcs/redis-client/lib/redis_client.rb:286:in `block in pipelined'
        from /home/kasuga/vcs/redis-client/lib/redis_client.rb:520:in `ensure_connected'
        from /home/kasuga/vcs/redis-client/lib/redis_client.rb:284:in `pipelined'
        from (irb):5:in `<main>'
        from bin/console:8:in `<main>'

redis-client/lib/redis_client/connection_mixin.rb

Lines 16 to 38 in dcd7f3d

 def call_pipelined(commands, timeouts) 

 exception = nil 

 size = commands.size 

 results = Array.new(commands.size) 

 write_multi(commands) 

 size.times do |index| 

 timeout = timeouts && timeouts[index] 

 result = read(timeout) 

 if result.is_a?(CommandError) 

 result._set_command(commands[index]) 

 exception ||= result 

 end 

 results[index] = result 

 end 

 if exception 

 raise exception 

 else 

 results 

 end 

 end

Investigate SSL hostname verification

We currently run the SSL tests with hostname verification disabled:

redis-client/test/support/redis_server_helper.rb

Line 40 in 9ffd076

 verify_hostname: false, # TODO: See if we could actually verify the hostname with our CI and dev setup 

We should have a look at how we could enable it on local and CI, to make sure it works in production.

https://github.com/redis/redis-rb does have some test for it, so it should be doable.

Mocking redis client in tests

Hello!

Is there a possibility or existing gem (I haven't found it so far) which allows to mock real interaction with redis in tests when using redis-client? Analogous to fakeredis or mok_redis for redis-rb.

Thanks in advance.

Improve URL handling in config.rb

https://github.com/redis-rb/redis-client/blob/master/lib/redis_client/config.rb#L169

Why is Integer being called on db_path, which is a string? It results in the error in the title of this issue. In my case it is localhost.

EDIT: it appears that adding redis:// to the beginning of the URL fixes the issue. It would be nice if better error handling were provided in config.rb. For example, adding the appropriate prefix on localhost would be a nice addition or raising a specific "Malformed URL provided".

Installing hiredis-client on macos with M1 fails

Trying to install this as part of a bundle fails

Ruby: ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [arm64-darwin21]
OS: macos 12.6.1 / Apple M1 Pro

...
Using hashery 2.1.2
Using hirb 0.7.3
Fetching hiredis-client 0.10.0
Installing hiredis-client 0.10.0 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
...

checking for rb_hash_new_capa() in ruby.h... no
cc -std=c99 -pedantic -c -O3 -fPIC  "-I/opt/homebrew/opt/[email protected]/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb alloc.c
cc -std=c99 -pedantic -c -O3 -fPIC  "-I/opt/homebrew/opt/[email protected]/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb net.c
cc -std=c99 -pedantic -c -O3 -fPIC  "-I/opt/homebrew/opt/[email protected]/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb hiredis.c
cc -std=c99 -pedantic -c -O3 -fPIC  "-I/opt/homebrew/opt/[email protected]/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb sds.c
cc -std=c99 -pedantic -c -O3 -fPIC  "-I/opt/homebrew/opt/[email protected]/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb async.c
cc -std=c99 -pedantic -c -O3 -fPIC  "-I/opt/homebrew/opt/[email protected]/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb read.c
cc -std=c99 -pedantic -c -O3 -fPIC  "-I/opt/homebrew/opt/[email protected]/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb sockcompat.c
ar rcs libhiredis.a alloc.o net.o hiredis.o sds.o async.o read.o sockcompat.o
cc -std=c99 -pedantic -c -O3 -fPIC  "-I/opt/homebrew/opt/[email protected]/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb ssl.c
ar rcs libhiredis_ssl.a ssl.o
creating Makefile
..
make "DESTDIR="
compiling hiredis_connection.c
linking shared-object redis_client/hiredis_connection.bundle
Undefined symbols for architecture arm64:
  "_ruby_abi_version", referenced from:
     -exported_symbol[s_list] command line option
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [hiredis_connection.bundle] Error 1

make failed, exit code 2

..

An error occurred while installing hiredis-client (0.10.0), and Bundler cannot continue.
Make sure that `gem install hiredis-client -v '0.10.0' --source 'https://rubygems.org/'` succeeds before bundling.

unknown keyword: :thread_safe After Upgrading to Rails 6.1

Hello,

I've just upgraded my app to Rails 6.1 and in the process of going through and dealing with various deprecated methods I've run into one I can't resolve.

When I run bundle exec rspec I'm getting an ArgumentError: unknown keyword: :thread_safe

From some quick googling I found out that in Rails 6.1 they removed the after_intialize method as that was now deprecated.

Remove deprecated ActiveSupport::LoggerThreadSafeLevel#after_initialize

rails/rails@e84af03

And since I'm getting this error on an intialize method in redis-client I figured that's where the problem lies.

Any thoughts on what I can look into to solve this, or if perhaps I'm misunderstanding something and am on the wrong track, would be much appreciated.

Is there way to add some commands to connection prelude?

Hello,

I'm implementing a cluster client with RedisClient. The cluster client needs to call the READONLY command at connection prelude if we use the scale reading feature of cluster mode. Is there way to add some commands to the prelude without method overriding?

redis-client/lib/redis_client/config.rb

Lines 95 to 107 in f85e2a8

 def build_connection_prelude 

 prelude = [] 

 prelude << if @password 

 ["HELLO", "3", "AUTH", @username, @password] 

 else 

 ["HELLO", "3"] 

 end 

 if @db && @db != 0 

 prelude << ["SELECT", @db.to_s] 

 end 

 prelude.freeze 

 end

redis-client/lib/redis_client.rb

Lines 551 to 582 in f85e2a8

 def connect 

 connection = config.driver.new( 

 config, 

 connect_timeout: connect_timeout, 

 read_timeout: read_timeout, 

 write_timeout: write_timeout, 

 ) 

 prelude = config.connection_prelude.dup 

 if id 

 prelude << ["CLIENT", "SETNAME", id.to_s] 

 end 

 # The connection prelude is deliberately not sent to Middlewares 

 if config.sentinel? 

 prelude << ["ROLE"] 

 role, = connection.call_pipelined(prelude, nil).last 

 config.check_role!(role) 

 else 

 connection.call_pipelined(prelude, nil) 

 end 

 connection 

 rescue CommandError => error 

 if error.message.include?("ERR unknown command `HELLO`") 

 raise UnsupportedServer, 

 "Your Redis server version is too old. redis-client requires Redis 6+. (#{config.server_url})" 

 else 

 raise 

 end 

 end

Running into RedisClient::ConnectionError: stream closed in another thread

Since I upgraded redis-client to 14.1 from 12.2 I have been seeing a large number of the above error. I am running this app on Heroku and using Heroku Redis. It is primarily used by redlock-rb gem which is there in the stacktrace.

Error is coming from: https://github.com/redis-rb/redis-client/blob/v0.14.1/lib/redis_client/ruby_connection.rb#L92

Stacktrace:

2023-04-27T05:10:28Z 21 TID-10v8 ERROR: /app/vendor/bundle/ruby/3.1.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection.rb:101:in `rescue in read'
/app/vendor/bundle/ruby/3.1.0/gems/redis-client-0.14.1/lib/redis_client/ruby_connection.rb:92:in `read'
/app/vendor/bundle/ruby/3.1.0/gems/redis-client-0.14.1/lib/redis_client/connection_mixin.rb:31:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/redis-client-0.14.1/lib/redis_client.rb:211:in `block (2 levels) in call'
/app/vendor/bundle/ruby/3.1.0/gems/redis-client-0.14.1/lib/redis_client/middlewares.rb:16:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/redis-client-0.14.1/lib/redis_client.rb:210:in `block in call'
/app/vendor/bundle/ruby/3.1.0/gems/redis-client-0.14.1/lib/redis_client.rb:626:in `ensure_connected'
/app/vendor/bundle/ruby/3.1.0/gems/redis-client-0.14.1/lib/redis_client.rb:209:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:172:in `block (2 levels) in lock'
/app/vendor/bundle/ruby/3.1.0/gems/redis-client-0.14.1/lib/redis_client.rb:182:in `with'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:171:in `block in lock'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:216:in `recover_from_script_flush'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:170:in `lock'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:266:in `block (2 levels) in lock_instances'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:266:in `select'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:266:in `block in lock_instances'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:319:in `timed'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:265:in `lock_instances'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:240:in `block in try_lock_instances'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:236:in `times'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:236:in `try_lock_instances'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:69:in `lock'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:104:in `lock!'
/app/app/lib/lock.rb:4:in `lock'
/app/app/models/intent.rb:132:in `deliver_notification'
/app/app/models/intent.rb:128:in `notify'
/app/app/workers/shopify/webhooks/inventory_levels/update_worker.rb:25:in `block in notify_intent'
/app/app/lib/lock.rb:5:in `block in lock'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:106:in `block in lock!'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:81:in `lock'
/app/vendor/bundle/ruby/3.1.0/gems/redlock-2.0.1/lib/redlock/client.rb:104:in `lock!'
/app/app/lib/lock.rb:4:in `lock'
/app/app/workers/shopify/webhooks/inventory_levels/update_worker.rb:23:in `notify_intent'
/app/app/workers/shopify/webhooks/inventory_levels/update_worker.rb:15:in `block in handle_webhook'
/app/app/workers/shopify/webhooks/inventory_levels/update_worker.rb:15:in `each'
/app/app/workers/shopify/webhooks/inventory_levels/update_worker.rb:15:in `handle_webhook'
/app/app/workers/shopify/webhooks/webhook_worker.rb:13:in `block in perform'
/app/vendor/bundle/ruby/3.1.0/gems/shopify_app-21.4.0/lib/shopify_app/session/session_storage.rb:14:in `block in with_shopify_session'
/app/vendor/bundle/ruby/3.1.0/gems/shopify_api-12.4.0/lib/shopify_api/auth/session.rb:81:in `temp'
/app/vendor/bundle/ruby/3.1.0/gems/sorbet-runtime-0.5.10676/lib/types/private/methods/call_validation.rb:256:in `bind_call'
/app/vendor/bundle/ruby/3.1.0/gems/sorbet-runtime-0.5.10676/lib/types/private/methods/call_validation.rb:256:in `validate_call'
/app/vendor/bundle/ruby/3.1.0/gems/sorbet-runtime-0.5.10676/lib/types/private/methods/call_validation.rb:177:in `block in create_validator_slow'
/app/vendor/bundle/ruby/3.1.0/gems/shopify_app-21.4.0/lib/shopify_app/session/session_storage.rb:13:in `with_shopify_session'
/app/app/workers/shopify/webhooks/webhook_worker.rb:12:in `perform'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/processor.rb:21:in `block (2 levels) in process'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/chain.rb:98:in `block in invoke'
/app/vendor/bundle/ruby/3.1.0/gems/bugsnag-6.25.2/lib/bugsnag/integrations/shoryuken.rb:29:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/chain.rb:100:in `block in invoke'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/server/active_record.rb:6:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/chain.rb:100:in `block in invoke'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/server/auto_extend_visibility.rb:10:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/chain.rb:100:in `block in invoke'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/server/auto_delete.rb:6:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/chain.rb:100:in `block in invoke'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/server/exponential_backoff_retry.rb:8:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/chain.rb:100:in `block in invoke'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/server/timing.rb:12:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/chain.rb:100:in `block in invoke'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/middleware/chain.rb:103:in `invoke'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/processor.rb:20:in `block in process'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/logging.rb:20:in `with_context'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/processor.rb:19:in `process'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/processor.rb:8:in `process'
/app/vendor/bundle/ruby/3.1.0/gems/shoryuken-5.3.2/lib/shoryuken/manager.rb:100:in `block in assign'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:24:in `block in execute'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:48:in `block in synchronize'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:48:in `synchronize'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:48:in `synchronize'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:22:in `execute'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/promise.rb:564:in `block in realize'
/app/vendor/bundle/ruby/3.1.0/gems/newrelic_rpm-8.16.0/lib/new_relic/agent/tracer.rb:423:in `block (2 levels) in thread_block_with_current_transaction'
/app/vendor/bundle/ruby/3.1.0/gems/newrelic_rpm-8.16.0/lib/new_relic/agent/tracer.rb:356:in `capture_segment_error'
/app/vendor/bundle/ruby/3.1.0/gems/newrelic_rpm-8.16.0/lib/new_relic/agent/tracer.rb:422:in `block in thread_block_with_current_transaction'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:352:in `run_task'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:343:in `block (3 levels) in create_worker'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `loop'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `block (2 levels) in create_worker'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in `catch'
/app/vendor/bundle/ruby/3.1.0/gems/concurrent-ruby-1.2.0/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in `block in create_worker'

ArgumentError: unknown keyword: :raw

Treat READONLY errors as connection errors

These error are very likely to be indicative of some kind of failover happening, and reconnecting has a good chance to solve them.

Fix: redis/redis-rb#543

cc @yosiat

Can not pass in nil for HMSET

It raises an error:

[4] pry(main)> RedisCache.current.hmset('foo123', 'time', Time.current.to_s, 'foo', nil)
TypeError: Unsupported command argument type: NilClass
from /usr/local/bundle/ruby/3.2.0/gems/redis-client-0.12.0/lib/redis_client/command_builder.rb:37:in `block in generate'

Previously:

Redis.current.hmset('foo123', 'time', Time.current.to_s, 'foo', nil)
=> "OK"


Redis.current.hget('foo123', 'foo')
=> ""

	def call_pipelined(commands, timeouts)
	exception = nil

	size = commands.size
	results = Array.new(commands.size)
	write_multi(commands)

	size.times do \|index\|
	timeout = timeouts && timeouts[index]
	result = read(timeout)
	if result.is_a?(CommandError)
	result._set_command(commands[index])
	exception \|\|= result
	end
	results[index] = result
	end

	if exception
	raise exception
	else
	results
	end
	end

	def build_connection_prelude
	prelude = []
	prelude << if @password
	["HELLO", "3", "AUTH", @username, @password]
	else
	["HELLO", "3"]
	end

	if @db && @db != 0
	prelude << ["SELECT", @db.to_s]
	end
	prelude.freeze
	end

	def connect
	connection = config.driver.new(
	config,
	connect_timeout: connect_timeout,
	read_timeout: read_timeout,
	write_timeout: write_timeout,
	)

	prelude = config.connection_prelude.dup

	if id
	prelude << ["CLIENT", "SETNAME", id.to_s]
	end

	# The connection prelude is deliberately not sent to Middlewares
	if config.sentinel?
	prelude << ["ROLE"]
	role, = connection.call_pipelined(prelude, nil).last
	config.check_role!(role)
	else
	connection.call_pipelined(prelude, nil)
	end

	connection
	rescue CommandError => error
	if error.message.include?("ERR unknown command `HELLO`")
	raise UnsupportedServer,
	"Your Redis server version is too old. redis-client requires Redis 6+. (#{config.server_url})"
	else
	raise
	end
	end

redis-rb / redis-client Goto Github PK

redis-client's People

Contributors

Stargazers

Watchers

Forkers

redis-client's Issues

Reproduction Scripts

Solution

Recommend Projects

Recommend Topics

Recommend Org