Giter Club home page Giter Club logo

ssdb-rocks's Introduction

ssdb-rocks's People

Contributors

fanweixiao avatar ideawu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ssdb-rocks's Issues

ssdb-server and ssdb.conf

for rocksdb, ssdb-server and ssdb.conf is all you need + the directory folder of the data right?

by the way, the configuration for ssdbROCKSDB is under "leveldb"?

same configuration settings?

leveldb:
# in MB
cache_size: 500
# in KB
block_size: 32
# in MB
write_buffer_size: 64
# in MB
compaction_speed: 1000
# yes|no
compression: no

Multithread scalability issue -- BinlogQueue lock contention

I am running multiple reader/writer worker threads and seeing 15ms latency spikes. Based on various test results, I think the cause of high tail latency is contention for the lock associated with the BinlogQueue object (acquired when a Transaction object is created in functions like SSDB::set()). An SSDB instance has a single BinlogQueue object, so the lock acts as a global lock. If a worker thread is de-scheduled when it holds the lock, no other threads can make progress until this thread is re-scheduled, finishes its transaction, and releases the lock.

I am wondering about the purpose of BinlogQueue (which uses RocksDB's WriteBatch object). Each "batch" contains only a single Put() or Delete() operation, followed by a Put() associated with the BinlogQueue log (why?), and then db->Write() is called. My questions are:

  1. What is the purpose of the BinlogQueue log?
    I see there is an option to set no_log = true for the BinlogQueue. When is this recommended?

  2. Given that a transaction seems to only contain a single operation, is a lock operation in the Transaction constructor necessary? If yes, why?

Thanks.

why use lua set value is much slower than php?

lua:
location /ssdb1 {
default_type 'text/plain';
content_by_lua '
local ssdb = require "resty.ssdb"
local db = ssdb:new()

            db:set_timeout(1000) -- 1 sec

            local ok, err = db:connect("127.0.0.1", 8888)
            if not ok then
                ngx.say("failed to connect: ", err)
                return
            end                

            for i=1,100000,1
            do
                db:init_pipeline()
                db:set("key"..tostring(i), i)
                local results, err = db:commit_pipeline()
            end
        ';

}
spend about 42 seconds

php:

pconnect('127.0.0.1', 8888); $time_start = microtime_float(); //保存数据 for($i = 0; $i < 100000; $i++){ $redis->sadd("key$i",$i); } spend about 24 seconds

util compilation issue

compilation environment:
CentOS release 6.6 and Ubuntu 14.04.4 LTS

/root/ali/ssdb-rocks/src/util does not compile and gives long list of error:
`g++ -std=c++0x -DNDEBUG -D__STDC_FORMAT_MACROS -Wall -O2 -Wno-sign-compare -I "/root/ali/ssdb-rocks/deps/rocksdb-master/include" -I "/root/ali/ssdb-rocks/deps/jemalloc-3.3.1/include" -c log.cpp
In file included from /opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/stringfwd.h:40:0,
from /opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/string:39,
from ./strings.h:5,
from /root/ali/ssdb-rocks/deps/jemalloc-3.3.1/include/jemalloc/jemalloc.h:8,
from ../version.h:7,
from ../include.h:24,
from log.h:4,
from log.cpp:1:
/opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/memoryfwd.h:63:3: error: template with C linkage
template
^
/opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/memoryfwd.h:66:3: error: template specialization with C linkage
template<>

/opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/memoryfwd.h:70:3: error: template with C linkage
template<typename, typename>
^
In file included from /opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/string:39:0,
from ./strings.h:5,
from /root/ali/ssdb-rocks/deps/jemalloc-3.3.1/include/jemalloc/jemalloc.h:8,
from ../version.h:7,
from ../include.h:24,
from log.h:4,
from log.cpp:1:
/opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/stringfwd.h:52:3: error: template with C linkage
template
.
.
[snip]
.
.
^
./strings.h:235:32: error: declaration of C function ‘uint64_t big_endian(uint64_t)’ conflicts with
uint64_t big_endian(uint64_t v){
^
./strings.h:225:11: error: previous declaration ‘uint16_t big_endian(uint16_t)’ here
uint16_t big_endian(uint16_t v){
^
make[1]: *** [log.o] Error 1
make[1]: Leaving directory /root/ali/ssdb-rocks/src/util' make: *** [all] Error 2

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.