Giter Club home page Giter Club logo

csredis's Introduction

Features

  • CSRedisClient and RedisHelper Keep all method names consistent with redis-cli

  • Support geo type commands (redis-server 3.2 or above is required)

  • Support Redis Cluster redis-trib.rb

  • Support Redis Sentinel and master-slave

  • Supports stream type commands (requires redis-server 5.0 and above)

Package Name NuGet Downloads
CSRedisCore nuget stats
Caching.CSRedis nuget stats IDistributedCache

dotnet add package CSRedisCore

Single machine redis

var csredis = new CSRedis.CSRedisClient("127.0.0.1:6379,password=123,defaultDatabase=13,prefix=my_");
Parameter Default Explain
user <Empty> Redis server user (redis 6.0+)
password <Empty> Redis server password
defaultDatabase 0 Redis server database
asyncPipeline false The asynchronous method automatically uses pipeline, and the 10W concurrent time is 450ms (welcome to feedback)
poolsize 50 Connection pool size
idleTimeout 20000 Idle time of elements in the connection pool (MS), suitable for connecting to remote redis server
connectTimeout 5000 Connection timeout (MS)
syncTimeout 10000 Send / receive timeout (MS)
preheat 5 Preheat connections, receive values such as preheat = 5 preheat 5 connections
autoDispose true Follow system exit event to release automatically
ssl false Enable encrypted transmission
testcluster true 是否尝试集群模式,阿里云、腾讯云集群需要设置此选项为 false
tryit 0 Execution error, retry attempts
name <Empty> Connection name, use client list command to view
prefix <Empty> key前辍,所有方法都会附带此前辍,csredis.Set(prefix + "key", 111);

IPv6: [fe80::b164:55b3:4b4f:7ce6%15]:6379

Redis Sentinel

var csredis = new CSRedis.CSRedisClient("mymaster,password=123,prefix=my_", 
  new [] { "192.169.1.10:26379", "192.169.1.11:26379", "192.169.1.12:26379" });

Read only: new CSRedisClient("mymaster,password=123", new [] { Sentinels }, false)

Redis Cluster

假设你已经配置好 redis-trib 集群,定义一个【普通模式】的 CSRedisClient 对象,它会根据 redis-server 返回的 MOVED | ASK 错误记录slot,自动增加节点 Nodes 属性。

127.0.0.1:6379,password=123,defaultDatabase=0,poolsize=50,prefix=

其他节点在运行过程中自动增加,确保每个节点密码一致。

警告:本模式与【分区模式】同时使用时,切记不可设置“prefix=key前辍”(或者全部设置成一样),否则会导致 keySlot 计算结果与服务端不匹配,无法记录 slotCache。

注意:官方集群不支持多 keys 的命令、【管道】、Eval(脚本)等众多杀手级功能。

IDistributedCache

dotnet add package Caching.CSRedis

RedisHelper.Initialization(csredis);
services.AddSingleton<IDistributedCache>(new Microsoft.Extensions.Caching.Redis.CSRedisCache(RedisHelper.Instance));

Note: CSRedisClient is singleton, RedisHelper static class is recommended

RedisHelper.Set("test1", "123123", 60);
RedisHelper.Get("test1");
//The method name is the same as the command of redis cli

Operate on multiple databases

var connectionString = "127.0.0.1:6379,password=123,poolsize=10";
var redis = new CSRedisClient[14]; //Singleton
for (var a = 0; a< redis.Length; a++) 
  redis[a] = new CSRedisClient(connectionString + ",defaultDatabase=" + a);

redis[1].Get("test1");

Multiple RedisHelper

public abstract class MyHelper1 : RedisHelper<MyHelper1> {}
public abstract class MyHelper2 : RedisHelper<MyHelper2> {}

MyHelper1.Initialization(new CSRedisClient("...."));
MyHelper2.Initialization(new CSRedisClient("...."));

Subscribe/Publish

//Native subscribe
RedisHelper.Subscribe(
  ("chan1", msg => Console.WriteLine(msg.Body)),
  ("chan2", msg => Console.WriteLine(msg.Body)));

RedisHelper.PSubscribe(new[] { "test*", "*test001", "test*002" }, msg => {
  Console.WriteLine($"PSUB   {msg.MessageId}:{msg.Body}    {msg.Pattern}: chan:{msg.Channel}");
});

//模式订阅已经解决的难题:
//1、分区的节点匹配规则,导致通配符最大可能匹配全部节点,所以全部节点都要订阅
//2、本组 "test*", "*test001", "test*002" 订阅全部节点时,需要解决同一条消息不可执行多次

RedisHelper.Publish("chan1", "123123123");

参考资料:【由浅至深】redis 实现发布订阅的几种方式

CacheShell

//不加缓存的时候,要从数据库查询
var t1 = Test.Select.WhereId(1).ToOne();

//一般的缓存代码,如不封装还挺繁琐的
var cacheValue = RedisHelper.Get("test1");
if (!string.IsNullOrEmpty(cacheValue)) {
	try {
		return JsonConvert.DeserializeObject(cacheValue);
	} catch {
		//出错时删除key
		RedisHelper.Remove("test1");
		throw;
	}
}
var t1 = Test.Select.WhereId(1).ToOne();
RedisHelper.Set("test1", JsonConvert.SerializeObject(t1), 10); //缓存10秒

//使用缓存壳效果同上,以下示例使用 string 和 hash 缓存数据
var t1 = RedisHelper.CacheShell("test1", 10, () => Test.Select.WhereId(1).ToOne());
var t2 = RedisHelper.CacheShell("test", "1", 10, () => Test.Select.WhereId(1).ToOne());
var t3 = RedisHelper.CacheShell("test", new [] { "1", "2" }, 10, notCacheFields => new [] {
  ("1", Test.Select.WhereId(1).ToOne()),
  ("2", Test.Select.WhereId(2).ToOne())
});

Pipeline

使用管道模式,打包多条命令一起执行,从而提高性能。

var ret1 = RedisHelper.StartPipe(p => p.Set("a", "1").Get("a"));

Benchmark

100,000 operations

StackExchange.Redis StringSet:7882ms
CSRedisCore Set:6101ms
-------------------
StackExchange.Redis StringGet:7729ms
CSRedisCore Get:5762ms
-------------------
StackExchange.Redis StringSetAsync:8094ms
CSRedisCore SetAsync:6315ms
-------------------
StackExchange.Redis StringGetAsync:7986ms
CSRedisClient GetAsync:4931ms
CSRedisCore GetAsync:5960ms
-------------------
CSRedisCore SetAsync(Task.WaitAll):559ms
StackExchange.Redis StringSetAsync (concurrent Task.WaitAll):172ms
-------------------
CSRedisCore GetAsync(Task.WaitAll):435ms
StackExchange.Redis StringGetAsync (concurrent Task.WaitAll):176ms

💕 Donation (捐赠)

感谢你的打赏

Thank

Original open source project: https://github.com/ctstone/csredis

csredis's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

csredis's Issues

关于并发时连接数问题

我这边测试程序用了 100 个并发,1秒发起一次,redis 连接池配置为 500 。不到一会连接池满,就会报 CSRedis.ConnectionPool.GetConnection 连接池获取超时(10秒)
使用的方式是用 RedisHelper.

希望增加 GetIfNotExist 方法

public static T GetIfNotExist<T>(string Key, Func<T> Func, int ExpireSecond = 3600) where T : class
{
    var t = Get<T>(Key) ?? Func();
    Set(Key, t, ExpireSecond);
    return t;
}

作为缓存库使用的话用的非常多....

centos中,报Too many open files in system错误。

错误日志如下:
System.Net.Sockets.SocketException (23): Too many open files in system
at CSRedis.CSRedisClient.GetConnectionAndExecute[T](ConnectionPool pool, Func2 handle) at CSRedis.CSRedisClient.ExecuteScalar[T](String key, Func3 hander)
at CSRedis.CSRedisClient.Get(String key)
at RedisHelper.Get(String key)

服务端强制关闭一个现有链接。

class Program
{
static CSRedis.CSRedisClient client = new CSRedis.CSRedisClient("ssssssss,defaultDatabase=50,poolsize=50,ssl=false,writeBuffer=10240,prefix=tt");

    static void Main(string[] args)
    {
        RedisHelper.Initialization(client,
                                     value => Newtonsoft.Json.JsonConvert.SerializeObject(value),
                                     deserialize: (data, type) => Newtonsoft.Json.JsonConvert.DeserializeObject(data, type));


        for (int i = 0; i < 1000; i++)
        {
            Thread t = new Thread(new ThreadStart(Pro));
            t.Start();
        }


        Console.ReadKey();
    }

    static void Pro()
    {
        for (int i = 0; i < 4; i++)
        {
            RedisHelper.Instance.Get("sdfsdfsdfsdf");
        }
    }

}

模拟1000个并发来测试。总是会提示 链接远程关闭。

HashSetExpire使用异常

我使用这个方法去缓存大概400条数据时候,显示function or expression too complex near '[' " 这个错误,我的数据格式类似这种:[“1”,“json字符串”],麻烦跟踪下,谢谢!

ZRangeByScor返回Score的值

能不能实现ZRangeByScor把Score的值也一同返回,用来统计7日、30日访问量等需要用到
数据如下(例):
ArticleViewCount:20180930
文章编号 访问量
1 20
2 192

使用RedisHelper.CacheShellAsync方法后,会无限等待,不返回

代码如下:
var list = await RedisHelper.CacheShellAsync(_key_bank_list, 60 * 60 * 24 * 7, () => bankdal.Banks());
其中bankdal.Banks()是一个异步方法,使用的是Dapper的QueryAsync方法读取数据库。
使用Postman进行提交测试,单步跟踪,到CacheShellAsync之后,从数据库中读取完数据,就直接跳到Startup.cs中注入CSRedis位置:

 RedisHelper.Initialization(csredis,
              value => Newtonsoft.Json.JsonConvert.SerializeObject(value),
              deserialize: (data, type) => Newtonsoft.Json.JsonConvert.DeserializeObject(data, type));

再之后就看不到其他的执行,而Postman还在等待接口返回数据,一直处在“loading”状态;如果Redis中有数据的话,就不会有这问题。应该是异步写Redis出的问题,麻烦修复一下,谢谢。

System.IO.EndOfStreamException: Unexpected end of stream; expected type 'MultiBulk'

调用RedisHelper.Keys频繁抛出异常.

System.IO.EndOfStreamException: Unexpected end of stream; expected type 'MultiBulk'
   at CSRedis.Internal.IO.RedisReader.ExpectType(RedisMessage expectedType)
   at CSRedis.Internal.Commands.RedisArray.Generic`1.Parse(RedisReader reader)
   at CSRedis.Internal.RedisConnector.Call[T](RedisCommand`1 command)
   at CSRedis.RedisClient.Write[T](RedisCommand`1 command)
   at CSRedis.RedisClient.Keys(String pattern)
   at CSRedis.CSRedisClient.Keys(String pattern)
   at RedisHelper.Keys(String pattern)
   at NbCRM.Account.TokenHelper.GetXTokenFromRedis(String plainToken) in C:\Projects\NbCRM\NbCRM.Account\src\NbCRM.Account\Helper\TokenHelper.cs:line 75
   at NbCRM.Account.Impl.UserServiceImpl.GetUserPath(Querys request, ServerCallContext context) in C:\Projects\NbCRM\NbCRM.Account\src\NbCRM.Account\Impl\UserServiceImpl.cs:line 750
   at Grpc.Core.Internal.UnaryServerCallHandler`2.<>c__DisplayClass4_2.<<HandleCall>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---

如何释放一个链接?

CSRedis.CSRedisClient Redis = RedisBase.RedisSentinelX;
var s1 = Redis.Increment(key1, numbers);
var s2 = Redis.Increment(key2, fees);

进行操作之后,如何释放Redis链接
来防止get connection error

关于重连后订阅恢复的问题

目前测了一下用CSRedisClient订阅之后,模拟断一下网然后恢复(禁用网卡),重新连上之后,就无法收到原先订阅的Channel了。
而且发现网络断一下之后,所有的API调用都会巨卡,感觉是每个命令都达到了超时时间。版本是2.6.9。

BRPop 在同一个进程中只能链接30-50个?【已解决 线程池的问题】

阿里云的 redis 服务。
写了一个测试应用,BRPop 超时时间 60秒,100个线程,全部阻塞结束后需要 135-144秒。
多个实例也是如此。

同一个机器多开测试应用仍然是 135-144秒完成,也就是说,每个进程都可以同时并发 30-50个 BRPop 链接。
多个客户机同时测试结果也是如此。

请问大佬,这个限制是如何产生的?有没有办法规避?

链接池链接维护。

连接池中链接一旦创建之后在一定时间不用之后会销毁链接码?
在线上项目中发现以下问题:
ERR max number of clients reached
重启以下服务就可以了。
然后观察redis的连接数。在服务运行期间connected_clients一直在不断的增加。

ERR invalid password

` var csredis = new CSRedis.CSRedisClient("ip,password=xxxx");
RedisHelper.Initialization(csredis,
value => Newtonsoft.Json.JsonConvert.SerializeObject(value),
deserialize: (data, type) => Newtonsoft.Json.JsonConvert.DeserializeObject(data, type));

        RedisHelper.Set("test1", "123123");`

我用
image
同样的配置是可以连接上的。

关于 Publish 方法 {msgid}|

///


/// 用于将信息发送到指定分区节点的频道
///

/// 频道名
/// 消息文本
///
public long Publish(string channel, string message) {
var msgid = HIncrBy("csredisclient:Publish:msgid", channel, 1);
return ExecuteScalar(channel, (c, k) => c.Value.Publish(channel, $"{msgid}|{message}"));
}

大神,发布消息为什么加个{msgid}|

普通连接,但是pool.ClusterKey一直是127.0.0.1

var csredis = new CSRedis.CSRedisClient("192.168.1.48:7001,defaultDatabase=15,poolsize=50");
var r1 = csredis.Set("aaa", DateTime.Now.ToString());
报异常CSRedis.RedisClientException:“Connection was not opened”

image

大佬readme里面写的有瑕疵

如果确定一定以及肯定非要有切换数据库的需求,请看以下代码:

......
下面的这段代码:
for (var a = 0; a< redis.Length; a++) redis[a] = new CSRedisClient(connectionString + "; defualtDatabase=" + a);
应该写成:
for (var a = 0; a< redis.Length; a++) redis[a] = new CSRedisClient(connectionString + ",defaultDatabase=" + a);

程序运行中与 redis-server 断开连接后,异步连接池获取逻辑有bug

CosmoKey 说:

感谢快速迭代。上午题的问题已经解决。
但是又发现新的问题了。(捂脸。。。我都不好意思在博客下留言了

50个线程跑(应该不过分吧?)
在redis重启后,直接卡主不动了。

30个重启一次还可以正常使用,但是停止或多次后也出现锁住的情况。

 for (int i = 0; i < 50; i++)
        {
            new Thread(() => {
                while (true) {
                    Task.Run(async () => {

                        string aaa = "aaa undefined";
                        try {
                            aaa = await RedisHelper.CacheShellAsync("keyAsync", "1", 1000,
                                async () => await Task.FromResult("Async Cache" + DateTime.Now.ToLongTimeString())); ;
                        } catch (Exception ex) {
                            Console.WriteLine(ex.Message);
                        }
                        Console.WriteLine(DateTime.Now.ToLongTimeString() + " : " + aaa);
                    }).Wait();                    
                    Thread.CurrentThread.Join(500);
                }

            }).Start();
        }

仅测了一次30并发,重启后没有恢复。

支持继续更新,别搞静态方法了,ioc注入吧

core中使用一段时间不管web,再操作时总会报redis连接异常,不知道什么原因
linux下没发现这问题,iis里的很多这种错误

Unable to write data to the transport connection: 你的主机中的软件中止了一个已建立的连接。. System.IO.IOException: Unable to write data to the transport connection: 你的主机中的软件中止了一个已建立的连接。. ---> System.Net.Sockets.SocketException: 你的主机中的软件中止了一个已建立的连接。 at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size) --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.IO.BufferedStream.ReadByteSlow() at System.IO.BufferedStream.ReadByte() at CSRedis.Internal.IO.RedisReader.ReadType() at CSRedis.Internal.IO.RedisReader.ExpectType(RedisMessage expectedType) at CSRedis.Internal.IO.RedisReader.ReadStatus(Boolean checkType) at CSRedis.Internal.Commands.RedisStatus.Parse(RedisReader reader) at CSRedis.Internal.RedisConnector.Call[T](RedisCommand1 command)
at CSRedis.RedisClient.Write[T](RedisCommand1 command)

与 redis-server 断开连接后,无法捕捉异步方法的异常

问题发现人:CosmoKey


感谢分享,已在使用。

线上有个需求,需要对Cache做熔断处理,在做测试期间发现:在网络不通的情况下(redis启动晚于客户端),使用RedisHelper.CacheShellAsync方法出现异常(System.IO.IOException: The operation is not allowed on non-connected sockets.)并无法被拦截。

测试代码:
try
{
await RedisHelper.CacheShellAsync("key", "1", 1000,
async () => await Task.FromResult("Cache" + DateTime.Now.ToLongTimeString()));
}
catch (Exception e)
{
}
希望能有更好的修复方案。

修复csredis获取redis sentinel的问题

在使用csredis获取sentinel时产生运行时异常,调查问题最后发现是获取sentinel的s-down-time配置参数存在问题。在sentinel集群中并非每个sentinel都能获取到这个参数,获取不到就抛出异常了。

获取s-down-time的代码在Types.cs文件中,RedisSentinelInfo类的构造函数:
public RedisSentinelInfo(SerializationInfo info, StreamingContext context): base(info, context)
SDownTime = info.GetInt64("s-down-time");
由于info中不存在s-down-time,在此出现了异常。

 从SerializationInfo中安全获取某个值的方法:
    /// <summary>
    /// Get a value from an instance of the SerializationInfo
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="info"></param>
    /// <param name="key"></param>
    /// <returns></returns>
    private T GetSerializationItemValue<T>(SerializationInfo info, string key)
    {
        foreach (SerializationEntry entry in info)
        {
            if (entry.Name == key)
            {
                return (T)Convert.ChangeType(entry.Value, typeof(T), System.Globalization.CultureInfo.InvariantCulture);
            }
        }
        return default(T);
    }

如果对应的值不存在,则返回指定类型的默认值。

因此修改获取s-down-time的方法为:
var s_down_time = GetSerializationItemValue(info, "s-down-time");

SDownTime = s_down_time == 0 ? -1 : s_down_time;

3.0及3.0以后的版本,报错。状态不可用,等待后台检查程序恢复方可使用。Unexpected end of stream;

System.Exception: 【127.0.0.1:6379/0】状态不可用,等待后台检查程序恢复方可使用。Unexpected end of stream; expected type 'Status'
at SafeObjectPool.ObjectPool1.getFree(Boolean checkAvailable) at SafeObjectPool.ObjectPool1.Get(Nullable1 timeout) at CSRedis.CSRedisClient.GetAndExecute[T](RedisClientPool pool, Func2 handler, Int32 jump)
at CSRedis.CSRedisClient.ExecuteScalar[T](String key, Func`3 hander)
at CSRedis.CSRedisClient.Get(String key)
at RedisHelper.Get(String key)

3.0以前的版本没问题。

修改缓存的时候怎么才能使过期时间不变化呢?

比如,我现在设置key=aaa,value=bbb,600秒后过期,但是在过了320秒后,我要修改这个数据,key=aaa,value=abc,但是过期时间不改变(现在应该是280)。这个该如何操作呢?set方法有个默认值,如果不传递的话,会导致过期时间变更为-1

Unexpected end of stream; expected type 'Bulk'

在实际使用中会经常性抛出Unexpected end of stream; expected type 'Bulk'异常和System.NotSupportedException异常,在压测时会周期性抛连接被重置异常,造成整个redis一直不可用,过一会又自己好了,使用的是单例模式,连接64个。

怎样切换数据库

你好,按照实例在Startup中配置:

           var csredis= new CSRedis.CSRedisClient("172.16.8.222:6379,password=,defaultDatabase=11,poolsize=10,ssl=false,writeBuffer=10240");
            RedisHelper.Initialization(csredis,
              value => Newtonsoft.Json.JsonConvert.SerializeObject(value),
              deserialize: (data, type) => Newtonsoft.Json.JsonConvert.DeserializeObject(data, type));
            
            //注入Redis缓存
            services.AddSingleton<IDistributedCache>(new Microsoft.Extensions.Caching.Redis.CSRedisCache(RedisHelper.Instance));

之后,我要操作的数据,分别在同一个Redis服务器上的不同的db中,要在代码中切换数据库,请问应该如何操作?谢谢

HSet操作返回值错误

redis Hset :如果字段是哈希表中的一个新建字段,并且值设置成功,返回 1 。 如果哈希表中域字段已经存在且旧值已被新值覆盖,返回 0
更新的时候,程序里也和1比较了。

这里可能是为了性能,而忽略了

CSRedisCache来源

//注册mvc分布式缓存
services.AddSingleton(new Microsoft.Extensions.Caching.Redis.CSRedisCache(RedisHelper.Instance));
这个 CSRedisCache指的是什么,从哪里来的呢,求解一下,谢谢!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.