Giter Club home page Giter Club logo

tairhash's Introduction

TairHash

Introduction 中文说明

     TairHash is a hash data structure developed based on the redis module. TairHash not only has the same rich data interface and high performance as the native hash, but also can set the expiration and version for the field. TairHash provides an active expiration mechanism, even if the field is not accessed after expiration, it can be actively deleted to release the memory.

The main features:

  • Field supports setting expire and version
  • Support efficient active expire and passivity expire for field
  • The cmd is similar to the native hash data type
  • Support redis swapdb, rename, move and copy command

Active expire

SORT_MODE:

avatar

  • Use a two-level sort index, the first level sorts the main key of tairhash, and the second level sorts the fields inside each tairhash
  • The first-level uses the smallest ttl in the second-level for sorting, so the main key is globally ordered
  • The built-in timer will periodically scan the first-level index to find out a key that has expired, and then check the secondary index of these keys to eliminate the expired fields. This is the active expire
  • Every time a write operation to tairhash, the first level index will be checked first, and at most three fields will be expired, Note these fields may not belong to the key currently being operated, so in theory, the faster you write, the faster the elimination
  • Every time you read or write a field, it will also trigger the expiration of the field itself
  • All keys and fields in the sorting index are pointer references, no memory copy, no memory expansion problem

Advantages: higher efficiency of expire elimination
Disadvantages: Because the SORT_MODE implementation relies on the unlink2 callback function (see this PR)) to release the index structure synchronously, you need to ensure that REDISMODULE_TYPE_METHOD_VERSION in your Redis is not Less than 4.

Usage: cmake with -DSORT_MODE=yes option, and recompile

SCAN_MODE:

  • Do not sort TairHash globally
  • Each TairHash will still use a sort index to sort the fields internally
  • The built-in timer will periodically use the SCAN command to find the TairHash that contains the expired field, and then check the sort index inside the TairHash to eliminate the field
  • Every time you read or write a field, it will also trigger the expiration of the field itself
  • All keys and fields in the sorting index are pointer references, no memory copy, no memory expansion problem

Advantages: can run in the low version of redis (redis 5.x and redis 6.x)
Disadvantages: low efficiency of expire elimination

Usage: cmake with -DSORT_MODE=no option, and recompile


Quick Start

127.0.0.1:6379> EXHSET k f v ex 10
(integer) 1
127.0.0.1:6379> EXHGET k f
"v"
127.0.0.1:6379> EXISTS k
(integer) 1
127.0.0.1:6379> debug sleep 10
OK
(10.00s)
127.0.0.1:6379> EXISTS k
(integer) 0
127.0.0.1:6379> EXHGET k f
(nil)
127.0.0.1:6379> EXHSET k f v px 10000
(integer) 1
127.0.0.1:6379> EXHGET k f
"v"
127.0.0.1:6379> EXISTS k
(integer) 1
127.0.0.1:6379> debug sleep 10
OK
(10.00s)
127.0.0.1:6379> EXISTS k
(integer) 0
127.0.0.1:6379> EXHGET k f
(nil)
127.0.0.1:6379> EXHSET k f v  VER 1
(integer) 1
127.0.0.1:6379> EXHSET k f v  VER 1
(integer) 0
127.0.0.1:6379> EXHSET k f v  VER 1
(error) ERR update version is stale
127.0.0.1:6379> EXHSET k f v  ABS 1
(integer) 0
127.0.0.1:6379> EXHSET k f v  ABS 2
(integer) 0
127.0.0.1:6379> EXHVER k f
(integer) 2

BUILD

mkdir build  
cd build  
cmake ../ && make -j

then the tairhash_module.so library file will be generated in the lib directory

./redis-server --loadmodule /path/to/tairhash_module.so

TEST

  1. Modify the path in the tairhash.tcl file in the tests directory to set testmodule [file your_path/tairhash_module.so]
  2. Add the path of the tairhash.tcl file in the tests directory to the all_tests of redis test_helper.tcl
  3. run ./runtest --single tairhash

Client

API

Reference

Our modules

TairHash: A redis module, similar to redis hash, but you can set expire and version for the field
TairZset: A redis module, similar to redis zset, but you can set multiple scores for each member to support multi-dimensional sorting
TairString: A redis module, similar to redis string, but you can set expire and version for the value. It also provides many very useful commands, such as cas/cad, etc.

tairhash's People

Contributors

chenyang8094 avatar alibaba-oss avatar yangbodong22011 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.