Giter Club home page Giter Club logo

zio-redis's Introduction

ZIO Redis

Development CI Badge Sonatype Releases Sonatype Snapshots javadoc ZIO Redis

Introduction

ZIO Redis is a ZIO-native Redis client. It aims to provide a type-safe and performant API for accessing Redis instances.

Installation

To use ZIO Redis, add the following line to your build.sbt:

libraryDependencies += "dev.zio" %% "zio-redis" % "0.2.0"

Example

To execute our ZIO Redis effect, we should provide the RedisExecutor layer to that effect. To create this layer we should also provide the following layers:

  • RedisConfig — Using default one, will connect to the localhost:6379 Redis instance.
  • BinaryCodec — In this example, we are going to use the built-in ProtobufCodec codec from zio-schema project.

To run this example we should put following dependencies in our build.sbt file:

libraryDependencies ++= Seq(
  "dev.zio" %% "zio-redis" % "0.2.0",
  "dev.zio" %% "zio-schema-protobuf" % "0.4.9"
)
import zio._
import zio.redis._
import zio.schema._
import zio.schema.codec._

object ZIORedisExample extends ZIOAppDefault {
  
  object ProtobufCodecSupplier extends CodecSupplier {
    def get[A: Schema]: BinaryCodec[A] = ProtobufCodec.protobufCodec
  }
  
  val myApp: ZIO[Redis, RedisError, Unit] = 
    for {
      redis <- ZIO.service[Redis]
      _     <- redis.set("myKey", 8L, Some(1.minutes))
      v     <- redis.get("myKey").returning[Long]
      _     <- Console.printLine(s"Value of myKey: $v").orDie
      _     <- redis.hSet("myHash", ("k1", 6), ("k2", 2))
      _     <- redis.rPush("myList", 1, 2, 3, 4)
      _     <- redis.sAdd("mySet", "a", "b", "a", "c")
    } yield ()

  override def run = 
    myApp.provide(Redis.local, ZLayer.succeed[CodecSupplier](ProtobufCodecSupplier))
}

Testing

To test you can use the embedded redis instance by adding to your build:

libraryDependencies := "dev.zio" %% "zio-redis-embedded" % "0.2.0"

Then you can supply EmbeddedRedis.layer.orDie as your RedisConfig and you're good to go!

import zio._
import zio.redis._
import zio.redis.embedded.EmbeddedRedis
import zio.schema.{DeriveSchema, Schema}
import zio.schema.codec.{BinaryCodec, ProtobufCodec}
import zio.test._
import zio.test.Assertion._
import java.util.UUID

object EmbeddedRedisSpec extends ZIOSpecDefault {
  object ProtobufCodecSupplier extends CodecSupplier {
    def get[A: Schema]: BinaryCodec[A] = ProtobufCodec.protobufCodec
  }
  
  final case class Item private (id: UUID, name: String, quantity: Int)
  object Item {
    implicit val itemSchema: Schema[Item] = DeriveSchema.gen[Item]
  }
  
  def spec = suite("EmbeddedRedis should")(
    test("set and get values") {
      for {
        redis <- ZIO.service[Redis]
        item   = Item(UUID.randomUUID, "foo", 2)
        _     <- redis.set(s"item.${item.id.toString}", item)
        found <- redis.get(s"item.${item.id.toString}").returning[Item]
      } yield assert(found)(isSome(equalTo(item)))
    }
  ).provideShared(
    EmbeddedRedis.layer,
    ZLayer.succeed[CodecSupplier](ProtobufCodecSupplier),
    Redis.singleNode
  ) @@ TestAspect.silentLogging
}

Resources

  • ZIO Redis by Dejan Mijic — Redis is one of the most commonly used in-memory data structure stores. In this talk, Dejan will introduce ZIO Redis, a purely functional, strongly typed client library backed by ZIO, with excellent performance and extensive support for nearly all of Redis' features. He will explain the library design using the bottom-up approach - from communication protocol to public APIs. Finally, he will wrap the talk by demonstrating the client's usage and discussing its performance characteristics.

Documentation

Learn more on the ZIO Redis homepage!

Contributing

For the general guidelines, see ZIO contributor's guide.

Code of Conduct

See the Code of Conduct

Support

Come chat with us on Badge-Discord.

License

License

zio-redis's People

Contributors

adamgfraser avatar aleksandarskrbic avatar anatolysergeev avatar anovakovic01 avatar drmarjanovic avatar gagandeepkalra avatar geoffjohn11 avatar github-actions[bot] avatar hcwilhelm avatar helanto avatar ioleo avatar jgoday avatar justinhj avatar jxnu-liguobin avatar khajavi avatar landlockedsurfer avatar m-kalai avatar mberndt123 avatar mijicd avatar mschuwalow avatar mvelimir avatar paulpdaniels avatar quelgar avatar renovate[bot] avatar rfmejia avatar scala-steward avatar sloanesturz avatar sunlightleaf avatar victornguen avatar zio-assistant[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zio-redis's Issues

Fix BITFIELD command

The current bitField implementation has the following problems:

  1. it doesn't allow mixed order of different commands like in the example below (order matters).
  2. it cannot parse current output because it is an array of integers and if there is overflow failure, it will have NULL multistring output ($-1) as an element of an output array.

Example:

BITFIELD key INCRBY u4 0 1 OVERFLOW FAIL INCRBY u4 0 9223372036854775807 INCRBY u4 0 1 INCRBY u4 0 1
*4
:10
$-1
:11
:12

Implement cluster API

Extracted from this comment authored by @regis-leray.

Commands

  • CLUSTER ASK When a cluster client receives an -ASK redirect, command is sent to the target node followed by the command which was redirected.
  • CLUSTER ADDSLOTS slot [slot ...] Assign new hash slots to receiving node
  • CLUSTER BUMPEPOCH Advance the cluster config epoch
  • CLUSTER COUNT-FAILURE-REPORTS node-id Return the number of failure reports active for a given node
  • CLUSTER COUNTKEYSINSLOT slot Return the number of local keys in the specified hash slot
  • CLUSTER DELSLOTS slot [slot ...] Set hash slots as unbound in receiving node
  • CLUSTER FAILOVER [FORCE|TAKEOVER] Forces a replica to perform a manual failover of its master.
  • CLUSTER FLUSHSLOTS Delete a node's own slots information
  • CLUSTER FORGET node-id Remove a node from the nodes table
  • CLUSTER GETKEYSINSLOT slot count Return local key names in the specified hash slot
  • CLUSTER INFO Provides info about Redis Cluster node state
  • CLUSTER KEYSLOT key Returns the hash slot of the specified ke- [ ]
  • CLUSTER MEET ip port Force a node cluster to handshake with another node
  • CLUSTER MYID Return the node id
  • CLUSTER NODES Get Cluster config for the node
  • CLUSTER REPLICATE node-id Reconfigure a node as a replica of the specified master node
  • CLUSTER RESET [HARD|SOFT] Reset a Redis Cluster node
  • CLUSTER SAVECONFIG Forces the node to save cluster state on disk
  • CLUSTER SET-CONFIG-EPOCH config-epoch Set the configuration epoch in a new node
  • CLUSTER SETSLOT slot IMPORTING|MIGRATING|STABLE|NODE [node-id] Bind a hash slot to a specific node
  • CLUSTER SLAVES node-id List replica nodes of the specified master node
  • CLUSTER REPLICAS node-id List replica nodes of the specified master node
  • CLUSTER SLOTS Get array of Cluster slot to node mappings- [ ]
  • READONLY Enables read queries for a connection to a cluster replica node
  • READWRITE Disables read queries for a connection to a cluster replica node

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Benchmark keys API

Measure the performance of the most frequently used commands from the keys API. Please make sure to benchmark each command using the following client libraries:

Since the ticket is "open" in terms of scenarios you can benchmark, make sure to announce your work in the comments so that we can avoid any duplicated work.

Tips

  • Add all benchmarks to the benchmarks project.
  • Please aim for 10-15 warmup and 10-15 benchmarking iterations. Make sure to submit the results as part of the pull request description.
  • Feel free to tackle "low-hanging fruit" optimizations but prefer opening an issue for larger ones.
  • Make sure to link the ticket in the pull request description.

Add missing strings commands

Strings API currently doesn't support:

  • BITFIELD
  • BITOP
  • BITPOS
  • SET

Please raise your hand for the ones you're taking :).

Benchmark protocol codecs

Measure the performance of protocol codecs using JMH.

Tips

  • Add all benchmarks to benchmarks project.
  • Please aim for 10-15 warmup and 10-15 benchmarking iterations. Make sure to submit the results as part of the pull request description.
  • Feel free to tackle "low-hanging fruit" optimizations but prefer opening an issue for larger ones.
  • Make sure to link the ticket in the pull request description.

Benchmark strings API

Measure the performance of the most frequently used commands from the strings API. Please make sure to benchmark each command using the following client libraries:

Since the ticket is "open" in terms of scenarios you can benchmark, make sure to announce your work in the comments so that we can avoid any duplicated work.

Tips

  • Add all benchmarks to the benchmarks project.
  • Please aim for 10-15 warmup and 10-15 benchmarking iterations. Make sure to submit the results as part of the pull request description.
  • Feel free to tackle "low-hanging fruit" optimizations but prefer opening an issue for larger ones.
  • Make sure to link the ticket in the pull request description.

Add missing lists commands

Keys API currently doesn't support BLPOP, BRPOP and LINSERT. To be able to represent them, our DSL needs to be adjusted to support non-empty lists that aren't in last positions (as well as having multiple non-empty lists).

cc @jdegoes

Collect the list of missing commands

Redis API has been expanded since the library was designed. The purpose of this issue is to come with the list of missing commands, and decide which ones we are going to support.

Fix DUMP command

Dump response can't be parsed. Find the root cause and "unignore" the existing test.

Move RESP serialisation into RedisExecutor service

I think some changes to Interpreter are needed.

The problem is that the current method of receiving responses isn't reliable, and it can't support pipelining. The unsafeReceive method just keeps reading until it only partially fills its buffer, assuming that means it has now read the entire response. There's a few ways this can go wrong, but the easiest way to make it fail when using localhost is to SET a key with a 1015 byte value. If you GET that with zio-redis, the read will very likely hang because the response happens to fit exactly in the buffer.

I can't see a way to fix this with the current RedisExecutor API, which accepts and returns RESP serialized strings. The problem with this is you can't reliably tell where a response ends without actually deserializing.

I propose to fix this by moving RESP serialization/deserialization into the executor. This is generic serialization that doesn't depend on which command is being sent. The RESP serialization can be represented with something like this:

sealed trait RespValue

object RespValue {
  final case class SimpleString(value: String) extends RespValue

  final case class Error(value: String) extends RespValue

  final case class Integer(value: Long) extends RespValue

  final case class BulkString(value: Chunk[Byte]) extends RespValue

  final case class Array(values: List[RespValue]) extends  RespValue
  
  case object NullString extends RespValue
}

and the executor service could be:

trait Service {

      def execute(command: RespValue.Array): IO[RedisError, RespValue]

    }

The receive then keeps reading bytes until the deserialization produces a complete response value, which can then be provided to the promise, and then moves on to reading the next response (I'm thinking the deserializaition can be implemented as ZTransducer). This will also allow support for pipelining, where the responses will not remotely align with the read buffer.

This will require changes to the Input serialization and Output deserialization as it exists today, but it should make them a little simpler and separate concerns nicely. For example, DurationSecondsOutput can just deal with converting the integer value to a duration, and will no longer need to know the RESP details like the string starting with : and parsing the rest to an int.

Thoughts?

Benchmark sets API

Measure the performance of the most frequently used commands from the sets API. Please make sure to benchmark each command using the following client libraries:

Since the ticket is "open" in terms of scenarios you can benchmark, make sure to announce your work in the comments so that we can avoid any duplicated work.

Tips

  • Add all benchmarks to the benchmarks project.
  • Please aim for 10-15 warmup and 10-15 benchmarking iterations. Make sure to submit the results as part of the pull request description.
  • Feel free to tackle "low-hanging fruit" optimizations but prefer opening an issue for larger ones.
  • Make sure to link the ticket in the pull request description.

Implement remaining connection commands

Extracted from this comment authored by @regis-leray.

Commands

  • CLIENT CACHING YES|NO Instruct the server about tracking or not keys in the next request
  • CLIENT ID Returns the client ID for the current connection
  • CLIENT KILL [ip:port] [ID client-id] [TYPE normal|master|slave|pubsub] [USER username] [ADDR ip:port] [SKIPME yes/no] Kill the connection of a client
  • CLIENT LIST [TYPE normal|master|replica|pubsub] Get the list of client connections
  • CLIENT GETNAME Get the current connection name
  • CLIENT GETREDIR Get tracking notifications redirection client ID if any
  • CLIENT PAUSE timeout Stop processing commands from clients for some time
  • CLIENT REPLY ON|OFF|SKIP Instruct the server whether to reply to commands
  • CLIENT SETNAME connection-name Set the current connection name
  • CLIENT TRACKING ON|OFF [REDIRECT client-id] [PREFIX prefix [PREFIX prefix ...]] [BCAST] [OPTIN] [OPTOUT] [NOLOOP] Enable or disable server assisted client side caching support
  • CLIENT UNBLOCK client-id [TIMEOUT|ERROR] Unblock a client blocked in a blocking command from a different connection
  • HELLO protover [AUTH username password] [SETNAME clientname] switch Redis protocol
  • QUIT Close the connection

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Publish microsite

Once the implementation is ready for preliminary release, make sure microsite is ready as well. Use the templates provided by g8 template.

IMPORTANT: Don't forget to create a new SSH key and gh-pages branch.

Benchmark list API

Measure the performance of the most frequently used commands from the list API. Please make sure to benchmark each command using the following client libraries:

Since the ticket is "open" in terms of scenarios you can benchmark, make sure to announce your work in the comments so that we can avoid any duplicated work.

Tips

  • Add all benchmarks to the benchmarks project.
  • Please aim for 10-15 warmup and 10-15 benchmarking iterations. Make sure to submit the results as part of the pull request description.
  • Feel free to tackle "low-hanging fruit" optimizations but prefer opening an issue for larger ones.
  • Make sure to link the ticket in the pull request description.

Implement scripting API

Extracted from this comment authored by @regis-leray.

Commands

  • EVAL script numkeys key [key ...] arg [arg ...] Execute a Lua script server side
  • EVALSHA sha1 numkeys key [key ...] arg [arg ...] Execute a Lua script server side
  • SCRIPT DEBUG YES|SYNC|NO Set the debug mode for executed scripts.
  • SCRIPT EXISTS sha1 [sha1 ...] Check existence of scripts in the script cache.
  • SCRIPT FLUSH Remove all the scripts from the script cache.
  • SCRIPT KILL Kill the script currently in execution.
  • SCRIPT LOAD script Load the specified Lua script into the script cache.

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Implement transactions API

Extracted from this comment authored by @regis-leray.

Commands

  • DISCARD Discard all commands issued after MULTI
  • EXEC Execute all commands issued after MULTI
  • MULTI Mark the start of a transaction block
  • UNWATCH Forget about all watched keys
  • WATCH key [key ...] Watch the given keys to determine execution of the MULTI/EXEC block

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Implement remaining string commands

Extracted from this comment authored by @regis-leray.

Commands

  • STRALGO LCS algo-specific-argument [algo-specific-argument ...] Run algorithms (currently LCS) against strings

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Test input encoding

Write a suite that checks command encoding. Since there are 50+ possible inputs, the work can be parallelized and shipped in multiple pull requests.

Benchmark the implementation

Benchmark the performance of the API using JMH. Benchmarks should include the other client libraries (e.g. redis4cats and laserdisc-io).

Ensure all command outputs are proper

Some commands can return (nil) in case of missing values. Instead of treating that as an error, proper optional responses should be specified (see #19 for more details).

Implement remaining list commands

Extracted from this comment authored by @regis-leray.

Commands

  • LPOS key element [RANK rank] [COUNT num-matches] [MAXLEN len] Return the index of matching elements on a list
  • SORT key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination] Sort the elements in a list, set or sorted set

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Implement remaining set commands

Extracted from this comment authored by @regis-leray.

Commands

  • SINTERSTORE destination key [key ...] Intersect multiple sets and store the resulting set in a key

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Implement streams API

Extracted from this comment authored by @regis-leray.

Commands

  • XINFO [CONSUMERS key groupname] [GROUPS key] [STREAM key] [HELP] Get information on streams and consumer groups
  • XADD key ID field value [field value ...] Appends a new entry to a stream
  • XTRIM key MAXLEN [] count Trims the stream to (approximately if '' is passed) a certain size
  • XDEL key ID [ID ...] Removes the specified entries from the stream. Returns the number of items actually deleted, that may be different from the number of IDs passed in case certain IDs do not exist.
  • XRANGE key start end [COUNT count] Return a range of elements in a stream, with IDs matching the specified IDs interval
  • XREVRANGE key end start [COUNT count] Return a range of elements in a stream, with IDs matching the specified IDs interval, in reverse order (from greater to smaller IDs) compared to XRANGE
  • XLEN key Return the number of entires in a stream
  • XREAD [COUNT count] [BLOCK milliseconds] STREAMS key [key ...] id [id ...] Return never seen elements in multiple streams, with IDs greater than the ones reported by the caller for each stream. Can block.
  • XGROUP [CREATE key groupname id-or-$] [SETID key groupname id-or-$] [DESTROY key groupname] [DELCONSUMER key groupname consumername] Create, destroy, and manage consumer groups.
  • XREADGROUP GROUP group consumer [COUNT count] [BLOCK milliseconds] [NOACK] STREAMS key [key ...] ID [ID ...] Return new entries from a stream using a consumer group, or access the history of the pending entries for a given consumer. Can block.
  • XACK key group ID [ID ...] Marks a pending message as correctly processed, effectively removing it from the pending entries list of the consumer group. Return value of the command is the number of messages successfully acknowledged, that is, the IDs we were actually able to resolve in the PEL.
  • XCLAIM key group consumer min-idle-time ID [ID ...] [IDLE ms] [TIME ms-unix-time] [RETRYCOUNT count] [FORCE] [JUSTID] Changes (or acquires) ownership of a message in a consumer group, as if the message was delivered to the specified consumer.
  • XPENDING key group [start end count] [consumer] Return information and entries from a stream consumer group pending entries list, that are messages fetched but never acknowledged.

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Release the library

Once the implementation is ready, align the release job with the rest of the organization, i.e. enable snapshots and stable artifact publishing.

Benchmark sorted sets API

Measure the performance of the most frequently used commands from the sorted sets API. Please make sure to benchmark each command using the following client libraries:

Since the ticket is "open" in terms of scenarios you can benchmark, make sure to announce your work in the comments so that we can avoid any duplicated work.

Tips

  • Add all benchmarks to the benchmarks project.
  • Please aim for 10-15 warmup and 10-15 benchmarking iterations. Make sure to submit the results as part of the pull request description.
  • Feel free to tackle "low-hanging fruit" optimizations but prefer opening an issue for larger ones.
  • Make sure to link the ticket in the pull request description.

Implement server API

Extracted from this comment authored by @regis-leray.

Commands

  • ACL LOAD Reload the ACLs from the configured ACL file
  • ACL SAVE Save the current ACL rules in the configured ACL file
  • ACL LIST List the current ACL rules in ACL config file format
  • ACL USERS List the username of all the configured ACL rules
  • ACL GETUSER username Get the rules for a specific ACL user
  • ACL SETUSER username [rule [rule ...]] Modify or create the rules for a specific ACL user
  • ACL DELUSER username [username ...] Remove the specified ACL users and the associated rules
  • ACL CAT [categoryname] List the ACL categories or the commands inside a category
  • ACL GENPASS [bits] Generate a pseudorandom secure password to use for ACL users
  • ACL WHOAMI Return the name of the user associated to the current connection
  • ACL LOG [count or RESET] List latest events denied because of ACLs in place
  • ACL HELP Show helpful text about the different subcommands
  • BGREWRITEAOF Asynchronously rewrite the append-only file
  • BGSAVE [SCHEDULE] Asynchronously save the dataset to disk
  • COMMAND Get array of Redis command details
  • COMMAND COUNT Get total number of Redis commands
  • COMMAND GETKEYS Extract keys given a full Redis command
  • COMMAND INFO command-name [command-name ...] Get array of specific Redis command details
  • CONFIG GET parameter Get the value of a configuration parameter
  • CONFIG REWRITE Rewrite the configuration file with the in memory configuration
  • CONFIG SET parameter value Set a configuration parameter to the given value
  • CONFIG RESETSTAT Reset the stats returned by INFO
  • DBSIZE Return the number of keys in the selected database
  • DEBUG OBJECT key Get debugging information about a key
  • DEBUG SEGFAULT Make the server crash
  • FLUSHALL [ASYNC] Remove all keys from all databases
  • FLUSHDB [ASYNC] Remove all keys from the current database
  • INFO [section] Get information and statistics about the server
  • LOLWUT [VERSION version] Display some computer art and the Redis version
  • LASTSAVE Get the UNIX time stamp of the last successful save to disk
  • MEMORY DOCTOR Outputs memory problems report
  • MEMORY HELP Show helpful text about the different subcommands
  • MEMORY MALLOC-STATS Show allocator internal stats
  • MEMORY PURGE Ask the allocator to release memory
  • MEMORY STATS Show memory usage details
  • MEMORY USAGE key [SAMPLES count] Estimate the memory usage of a key
  • MODULE LIST List all modules loaded by the server
  • MODULE LOAD path [ arg [arg ...]] Load a module
  • MODULE UNLOAD name Unload a module
  • MONITOR Listen for all requests received by the server in real time
  • ROLE Return the role of the instance in the context of replication
  • SHUTDOWN [NOSAVE|SAVE] Synchronously save the dataset to disk and then shut down the server
  • SLAVEOF host port Make the server a replica of another instance, or promote it as master. Deprecated starting with Redis 5. Use REPLICAOF instead.
  • REPLICAOF host port Make the server a replica of another instance, or promote it as master.
  • SLOWLOG subcommand [argument] Manages the Redis slow queries log
  • SWAPDB index1 index2 Swaps two Redis databases
  • SYNC Internal command used for replication
  • PSYNC replicationid offset Internal command used for replication
  • TIME Return the current server time- [ ] ROLE Return the role of the instance in the context of replication
  • SHUTDOWN [NOSAVE|SAVE] Synchronously save the dataset to disk and then shut down the server
  • SLAVEOF host port Make the server a replica of another instance, or promote it as master. Deprecated starting with Redis 5. Use REPLICAOF instead.
  • REPLICAOF host port Make the server a replica of another instance, or promote it as master.
  • SLOWLOG subcommand [argument] Manages the Redis slow queries log
  • SWAPDB index1 index2 Swaps two Redis databases
  • SYNC Internal command used for replication
  • PSYNC replicationid offset Internal command used for replication
  • TIME Return the current server time
  • LATENCY DOCTOR Return a human readable latency analysis report.
  • LATENCY GRAPH event Return a latency graph for the event.
  • LATENCY HISTORY event Return timestamp-latency samples for the event.
  • LATENCY LATEST Return the latest latency samples for all events.
  • LATENCY RESET [event [event ...]] Reset latency data for one or more events.
  • LATENCY HELP Show helpful text about the different subcommands.
  • OBJECT subcommand [arguments [arguments ...]] Inspect the internals of Redis objects
  • SAVE Synchronously save the dataset to disk

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Implement pub/sub API

Extracted from this comment authored by @regis-leray.

Commands

  • PSUBSCRIBE pattern [pattern ...] Listen for messages published to channels matching the given patterns
  • PUBSUB subcommand [argument [argument ...]] Inspect the state of the Pub/Sub subsystem
  • PUBLISH channel message Post a message to a channel
  • PUNSUBSCRIBE [pattern [pattern ...]] Stop listening for messages posted to channels matching the given patterns
  • SUBSCRIBE channel [channel ...] Listen for messages published to the given channels
  • UNSUBSCRIBE [channel [channel ...]] Stop listening for messages posted to the given channels

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Add transactions support

Update the interpreter to support transaction execution. Note that this change might require adjusting the public API.

Add missing sorted sets commands

Sorted sets API currently doesn't support the following:

  • ZINTERSTORE
  • ZLEXCOUNT
  • ZPOPMAX
  • ZPOPMIN
  • ZRANGEBYLEX
  • ZREVRANGEBYLEX
  • ZRANGEBYSCORE
  • ZREMRANGEBYLEX
  • ZREMRANGEBYRANK
  • ZREMRANGEBYSCORE
  • ZREVRANGE
  • ZREVRANGEBYSCORE
  • ZREVRANK
  • ZUNIONSTORE

As there are quite a few of them, please raise your hand for the ones you're tackling :).

Add missing keys commands

Keys API currently doesn't support the following:

  • MIGRATE
  • OBJECT
  • RANDOMKEY
  • RESTORE
  • SORT
  • WAIT

Determine which of these are necessary, and represent them.

Benchmark hashes API

Measure the performance of the most frequently used commands from the hashes API. Please make sure to benchmark each command using the following client libraries:

Since the ticket is "open" in terms of scenarios you can benchmark, make sure to announce your work in the comments so that we can avoid any duplicated work.

Tips

  • Add all benchmarks to the benchmarks project.
  • Please aim for 10-15 warmup and 10-15 benchmarking iterations. Make sure to submit the results as part of the pull request description.
  • Feel free to tackle "low-hanging fruit" optimizations but prefer opening an issue for larger ones.
  • Make sure to link the ticket in the pull request description.

Add logging capability

I don't think we need to go overboard with logging, but it would be useful to log when connections and reconnections are occurring, and why. Once we have Sentinel support we'd also want to log the outcomes of Sentinel discovery.

It might also help users debugging apps if there was an option to log a trace of commands and responses?

Probably zio-logging is the way to go.

Fix KEYS command

Keys response can't be parsed once there is more than 24 keys as pointed in #74. Whoever picks this ticket should fix the issue and "unignore" the existing test.

Implement remaining hash commands

Extracted from this comment authored by @regis-leray.

Commands

  • HMSET key field value [field value ...] Set multiple hash fields to multiple values

Tips

  • Define the API in zio.redis.api.
  • Command-specific options should be defined in zio.redis.options.
  • Additional inputs and outputs can be defined in zio.redis.Input and zio.redis.Output, respectively.
  • Redis API evolves, therefore the commands listed above might be slightly changed. Double check the official docs before starting with implementation.
  • Use telnet to understand command protocol details.
  • Write inputs and outputs tests in zio.redis.InputSpec and zio.redis.OutputSpec, respectively.
  • Write integration tests by expanding the zio.redis.ApiSpec.
  • If you feel that the pull request size is growing out of control, feel free to split it but make sure to link this issue in each of the related PRs.

Interpret commands

This issue is consisted of multiple steps, some of which may end up in a separate issues:

  • adjust the API to support pipelining
  • implement encoding and decoding
  • connection management strategies
  • enable integration tests on CircleCI

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.