Giter Club home page Giter Club logo

beanseye's People

Contributors

davies avatar hurricane1026 avatar menghan avatar mxlxm avatar youngsofun avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

beanseye's Issues

It has two configuration file under conf directory, which one should I use?

Hi,
we are using beansdb and beanseye. I found that there are 2 file under conf dir. Which one should I use when stating the server? example.ini or example.yaml? What's the difference between them?Could you tell me the detail of configuration options about the two kinds of configration file, so that I could set the configuration options correctly .

have some questions about yaml conf file, I have to ask you again.

Hi,
I'm sorry to ask you again. I have ask some questions about the yaml file . But there are also some questions about the yaml conf file . About the example.yaml file :

  1. I saw that N=3 in the example.yaml file ,so there are 3 beansdb node in the cluster, but why Servers option only set one server(localhost:7900 and 127.0.0.1:7900 are the same server) only?
    2)The bucket in every beansdb which is in cluster have 16 numbers, Why only set '0 1 2' or ' A -1 B'?
    Why not set other buckets in the beansdb which is in the cluster?Could not
  2. I have two beansdb and one beanseye(IP is 192.168.1.1) ,I want to let the two beansdb (which ip are 192.168.1.2 and 192.168.1.3 ) in one cluster by the beanseye , and make them backup for each other . Is the following options in the yaml file right?
    Servers :
    192.168.1.2: 7900
    192.168.1.3: 7900
    I don't know how to set the buckets . How to set if I must set the buckets for each beansdb in the cluster ?
  3. the yaml file have options :
    threads : 8
    listen : 0.0.0.0
    readonly : false
    what are the options? How to set them ?

At last , thanks for your reply .

Could you help me to use some config options about yaml config file?

Hi, I have know that the beanseye is using yaml config file after your reply , and thanks very much. We will use beanseye and beansdb in our project. So I have to ask you some questions about config options.I need your help . The yaml has some options , could you tell me the detail :

1 .servers option :
servers:

localhost:7900 0 1 2
127.0.0.1:7900 A -1 B 
 I konw that 'localhost:7900' and '127.0.0.1:7900' but what's means about the '0 1 2' and 'A -1 B'?How to set the options?

2 . how to set following options :

threads: 8
n: 3
w: 2
r: 1
buckets: 16
slow: 200
listen: 0.0.0.0
proxies:

localhost:7905 readonly: false

编译beanseye 出错:make: *** [dep] 错误 2

[root@localhost beanseye]# make
go get github.com/douban/goyaml

github.com/douban/goyaml

src/github.com/douban/goyaml/decode.go: 在函数‘_cgo_bec77eaae85a_Cfunc_event_alias’中:
src/github.com/douban/goyaml/decode.go:36: 错误:expected specifier-qualifier-list before ‘<’ token
make: *** [dep] 错误 2

怎么手动指定bucket分布

代码里面好像默认直接写死了用16个bucket,bucket的分布是根据key的hash来确定,有办法手动指定bucket分布不?

如果现在有4台beansdb,N=2, W=3。然后其中一台机器挂掉了(这台机器数据全部丢失),如何能恢复这台机器?我知道应该是可以通过其他的机器恢复数据,不知道有没有现成的方法,还是直接把其他所有的数据都同步一份过来?

还有如果现在用了4台beansdb,想再加一台beansdb,现有在使用一致性hash基础上,该如何做?

在beansdb机器数目发生改变的时候,是通过什么方法来保证数据的备份数(即N是配置文件里面设置的数目)?

run beanseye error: panic: runtime error: index out of range

  1. error info :
    eyes@eyes-H55M-S2:~/beanseye$ sudo ./bin/proxy -conf conf/conf.yaml -basepath /home/eyes/beanseye
    panic: runtime error: index out of range

goroutine 19 [running]:
runtime.panic(0x6de3c0, 0xc74117)
/usr/lib/go/src/pkg/runtime/panic.c:266 +0xb6
memcache.(*ManualScheduler).try_reward(0xc2100d5080)
/home/eyes/beanseye/src/memcache/schedule.go:244 +0x6ab
memcache.func·009()
/home/eyes/beanseye/src/memcache/schedule.go:208 +0x33
created by memcache.NewManualScheduler
/home/eyes/beanseye/src/memcache/schedule.go:211 +0x6d0

goroutine 1 [runnable]:
syscall.Syscall(0x2, 0xc2100b42c0, 0x0, 0x0, 0x9, ...)
/usr/lib/go/src/pkg/syscall/asm_linux_amd64.s:18 +0x5
syscall.open(0x754bd0, 0xe, 0x0, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/syscall/zsyscall_linux_amd64.go:16 +0x9c
syscall.Open(0x754bd0, 0xe, 0x0, 0xc200000000, 0x30, ...)
/usr/lib/go/src/pkg/syscall/syscall_linux.go:23 +0x5e
time.readFile(0x754bd0, 0xe, 0x0, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/time/sys_unix.go:23 +0x91
time.loadZoneFile(0x734e20, 0x0, 0x754bd0, 0xe, 0x0, ...)
/usr/lib/go/src/pkg/time/zoneinfo_read.go:206 +0x16d
time.initLocal()
/usr/lib/go/src/pkg/time/zoneinfo_unix.go:59 +0x7f
sync.(_Once).Do(0xc96c28, 0x7ebeb8)
/usr/lib/go/src/pkg/sync/once.go:40 +0x9a
time.(_Location).get(0xc8db40, 0x8)
/usr/lib/go/src/pkg/time/zoneinfo.go:69 +0x60
time.Time.abs(0xecc01372e, 0x1b2a53c1, 0xc8db40, 0x444142)
/usr/lib/go/src/pkg/time/time.go:253 +0xc5
time.Time.date(0xecc01372e, 0x1b2a53c1, 0xc8db40, 0x1, 0x45a4f3, ...)
/usr/lib/go/src/pkg/time/time.go:677 +0x40
time.Time.Date(0xecc01372e, 0x1b2a53c1, 0xc8db40, 0xc2100b4280, 0xc, ...)
/usr/lib/go/src/pkg/time/time.go:293 +0x40
log.(_Logger).formatHeader(0xc21000a230, 0xc21000a260, 0xecc01372e, 0x1b2a53c1, 0xc8db40, ...)
/usr/lib/go/src/pkg/log/log.go:84 +0x1ae
log.(_Logger).Output(0xc21000a230, 0x2, 0xc2100b50c0, 0x1e, 0x0, ...)
/usr/lib/go/src/pkg/log/log.go:148 +0x201
log.Println(0x7f37d3224e98, 0x2, 0x2)
/usr/lib/go/src/pkg/log/log.go:282 +0x69
main.main()
/home/eyes/beanseye/src/proxy/proxy.go:543 +0x1441

goroutine 4 [syscall]:
os/signal.loop()
/usr/lib/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
/usr/lib/go/src/pkg/os/signal/signal_unix.go:27 +0x31

goroutine 5 [chan receive]:
cmem.func·001()
/home/eyes/beanseye/src/cmem/cmem.go:15 +0x3d
created by cmem.init·1
/home/eyes/beanseye/src/cmem/cmem.go:18 +0x48

goroutine 6 [syscall]:
runtime.goexit()
/usr/lib/go/src/pkg/runtime/proc.c:1394

goroutine 13 [runnable]:
net.(_netFD).dial(0xc210102000, 0x7f37d33b4bc0, 0x0, 0x7f37d33b4bc0, 0xc2100b0f90, ...)
/usr/lib/go/src/pkg/net/sock_posix.go:98
net.socket(0x74ec80, 0x3, 0x2, 0x1, 0x0, ...)
/usr/lib/go/src/pkg/net/sock_posix.go:91 +0x40b
net.internetSocket(0x74ec80, 0x3, 0x7f37d33b4bc0, 0x0, 0x7f37d33b4bc0, ...)
/usr/lib/go/src/pkg/net/ipsock_posix.go:136 +0x161
net.dialTCP(0x74ec80, 0x3, 0x0, 0xc2100b0f90, 0xecc01372e, ...)
/usr/lib/go/src/pkg/net/tcpsock_posix.go:155 +0xef
net.dialSingle(0x74ec80, 0x3, 0xc21004e620, 0x12, 0x0, ...)
/usr/lib/go/src/pkg/net/dial.go:225 +0x3d8
net.func·015(0xecc01372e, 0x2d08b3e3, 0xc8db40, 0x11e1a300, 0xecc01372e, ...)
/usr/lib/go/src/pkg/net/dial.go:158 +0xde
net.dial(0x74ec80, 0x3, 0x7f37d33b4b58, 0xc2100b0f90, 0x7f37d323a598, ...)
/usr/lib/go/src/pkg/net/fd_unix.go:40 +0x45
net.(_Dialer).Dial(0xc2100f3100, 0x74ec80, 0x3, 0xc21004e620, 0x12, ...)
/usr/lib/go/src/pkg/net/dial.go:165 +0x3e0
net.DialTimeout(0x74ec80, 0x3, 0xc21004e620, 0x12, 0x11e1a300, ...)
/usr/lib/go/src/pkg/net/dial.go:145 +0xaa
memcache.(_Host).createConn(0xc2100f3000, 0x7f37d323a768, 0xc21000b1c0, 0x434e00, 0xca1c80)
/home/eyes/beanseye/src/memcache/host.go:58 +0x1d0
memcache.(_Host).getConn(0xc2100f3000, 0x0, 0x0, 0x0, 0x0)
/home/eyes/beanseye/src/memcache/host.go:73 +0x12d
memcache.(_Host).execute(0xc2100f3000, 0xc2100f30c0, 0x0, 0x0, 0x0)
/home/eyes/beanseye/src/memcache/host.go:92 +0x59
memcache.(_Host).Stat(0xc2100f3000, 0x0, 0x0, 0x0, 0x2, ...)
/home/eyes/beanseye/src/memcache/host.go:197 +0xa1
main.update_stats(0xc2100b5040, 0x2, 0x2, 0x0, 0x0, ...)
/home/eyes/beanseye/src/proxy/proxy.go:185 +0x357
created by main.main
/home/eyes/beanseye/src/proxy/proxy.go:456 +0x1912

goroutine 14 [select]:
net.lookupIPDeadline(0xc210036780, 0x9, 0xecc01372e, 0x2d064ec1, 0xc8db40, ...)
/usr/lib/go/src/pkg/net/lookup.go:83 +0x326
net.resolveInternetAddr(0x74ec80, 0x3, 0xc210036780, 0xe, 0xecc01372e, ...)
/usr/lib/go/src/pkg/net/ipsock.go:285 +0x395
net.resolveAddr(0x742e80, 0x4, 0x74ec80, 0x3, 0xc210036780, ...)
/usr/lib/go/src/pkg/net/dial.go:104 +0x32a
net.(_Dialer).Dial(0xc21004f480, 0x74ec80, 0x3, 0xc210036780, 0xe, ...)
/usr/lib/go/src/pkg/net/dial.go:153 +0xa7
net.DialTimeout(0x74ec80, 0x3, 0xc210036780, 0xe, 0x11e1a300, ...)
/usr/lib/go/src/pkg/net/dial.go:145 +0xaa
memcache.(_Host).createConn(0xc21004f1c0, 0x7f37d12dc768, 0xc2100a7000, 0x434c00, 0x7f37d33b0178)
/home/eyes/beanseye/src/memcache/host.go:58 +0x1d0
memcache.(_Host).getConn(0xc21004f1c0, 0x0, 0x0, 0x0, 0x0)
/home/eyes/beanseye/src/memcache/host.go:73 +0x12d
memcache.(_Host).execute(0xc21004f1c0, 0xc21004f400, 0x0, 0x0, 0x0)
/home/eyes/beanseye/src/memcache/host.go:92 +0x59
memcache.(*Host).Stat(0xc21004f1c0, 0x0, 0x0, 0x0, 0x1, ...)
/home/eyes/beanseye/src/memcache/host.go:197 +0xa1
main.update_stats(0xc210036ea0, 0x1, 0x1, 0x0, 0x0, ...)
/home/eyes/beanseye/src/proxy/proxy.go:185 +0x357
created by main.main
/home/eyes/beanseye/src/proxy/proxy.go:460 +0x19b2

goroutine 15 [runnable]:
net.zoneToInt(0x0, 0x0, 0x10)
/usr/lib/go/src/pkg/net/ipsock.go:309
net.ipToSockaddr(0xa, 0xc96c40, 0x10, 0x10, 0x1ee4, ...)
/usr/lib/go/src/pkg/net/ipsock_posix.go:172 +0x3b0
net.(_TCPAddr).sockaddr(0xc2100ae600, 0xa, 0x0, 0x4ac513, 0xc210100000, ...)
/usr/lib/go/src/pkg/net/tcpsock_posix.go:52 +0x93
net.(_netFD).listenStream(0xc210100000, 0x7f37d33b4bc0, 0xc2100ae600, 0x80, 0x7e67d8, ...)
/usr/lib/go/src/pkg/net/sock_posix.go:143 +0x83
net.socket(0x74ec80, 0x3, 0xa, 0x1, 0x0, ...)
/usr/lib/go/src/pkg/net/sock_posix.go:78 +0x27a
net.internetSocket(0x74ec80, 0x3, 0x7f37d33b4bc0, 0xc2100ae600, 0x0, ...)
/usr/lib/go/src/pkg/net/ipsock_posix.go:136 +0x161
net.ListenTCP(0x74ec80, 0x3, 0xc2100ae600, 0xc2100ae600, 0xc2100b4001, ...)
/usr/lib/go/src/pkg/net/tcpsock_posix.go:293 +0x177
net.Listen(0x74ec80, 0x3, 0xc2100b4080, 0xc, 0x2, ...)
/usr/lib/go/src/pkg/net/dial.go:256 +0x3be
main.func·005()
/home/eyes/beanseye/src/proxy/proxy.go:470 +0x16f
created by main.main
/home/eyes/beanseye/src/proxy/proxy.go:476 +0x1ad9

goroutine 17 [runnable]:
net._Cfunc_CString(0xc210036780, 0x9, 0xc20fff55f8)
net/_obj/_cgo_defun.c:31 +0x2a
net.cgoLookupIPCNAME(0xc210036780, 0x9, 0x0, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/net/cgo_unix.go:94 +0x124
net.cgoLookupIP(0xc210036780, 0x9, 0x0, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/net/cgo_unix.go:148 +0x69
net.lookupIP(0xc210036780, 0x9, 0x0, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/net/lookup_unix.go:64 +0x63
net.func·022(0xc8d400, 0xc2100ab240, 0xc210036780, 0x9)
/usr/lib/go/src/pkg/net/lookup.go:41 +0x2d
net.(*singleflight).Do(0xc8d400, 0xc210036780, 0x9, 0x7f37d02baed0, 0x0, ...)
/usr/lib/go/src/pkg/net/singleflight.go:45 +0x1de
net.lookupIPMerge(0xc210036780, 0x9, 0x0, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/net/lookup.go:42 +0xc0
net.func·023()
/usr/lib/go/src/pkg/net/lookup.go:80 +0x36
created by net.lookupIPDeadline
/usr/lib/go/src/pkg/net/lookup.go:82 +0x239

goroutine 18 [chan receive]:
memcache.(*ManualScheduler).procFeedback(0xc2100d5080)
/home/eyes/beanseye/src/memcache/schedule.go:277 +0x58
created by memcache.NewManualScheduler
/home/eyes/beanseye/src/memcache/schedule.go:205 +0x6a0

2 my config options :

servers:

  • 192.168.1.174:7900
  • 192.168.1.175:7900
    port: 7905
    webport: 7908
    threads: 3
    n: 2
    w: 2
    r: 1
    buckets: 16
    slow: 200
    listen: 0.0.0.0
    proxies:
  • 192.168.20.166:7905
    accesslog: /log/beansproxy/beansproxy.log
    errorlog: /log/beansproxy/beansproxy_error.log
    basepath: /var/lib/beanseye
    readonly: false

panic: runtime error: index out of range

go version
go version go1.4 darwin/amd64

panic: runtime error: index out of range
goroutine 18 [running]:
memcache.(*ManualScheduler).try_reward(0xc208084d00)
/Users/kenneth/beanseye/src/memcache/schedule.go:244 +0x905
memcache.func·009()
/Users/kenneth/beanseye/src/memcache/schedule.go:208 +0x34
created by memcache.NewManualScheduler
/Users/kenneth/beanseye/src/memcache/schedule.go:211 +0x9b3

goroutine 1 [runnable]:
main.main()
/Users/kenneth/beanseye/src/proxy/proxy.go:540 +0x1538

goroutine 6 [syscall]:
os/signal.loop()
/usr/local/Cellar/go/1.4/libexec/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
/usr/local/Cellar/go/1.4/libexec/src/os/signal/signal_unix.go:27 +0x35

goroutine 7 [chan receive]:
cmem.func·001()
/Users/kenneth/beanseye/src/cmem/cmem.go:15 +0x4a
created by cmem.init·1
/Users/kenneth/beanseye/src/cmem/cmem.go:18 +0x5c

goroutine 14 [runnable]:
main.update_stats(0xc20801f940, 0x2, 0x2, 0x0, 0x0, 0x0, 0xc2080c6390, 0x2, 0x2, 0x1)
/Users/kenneth/beanseye/src/proxy/proxy.go:166 +0x97
created by main.main
/Users/kenneth/beanseye/src/proxy/proxy.go:458 +0x1ee3

goroutine 15 [runnable]:
main.update_stats(0xc20802b160, 0x1, 0x1, 0x0, 0x0, 0x0, 0xc208056330, 0x1, 0x1, 0x0)
/Users/kenneth/beanseye/src/proxy/proxy.go:166
created by main.main
/Users/kenneth/beanseye/src/proxy/proxy.go:462 +0x1fc8

goroutine 16 [runnable]:
main.func·005()
/Users/kenneth/beanseye/src/proxy/proxy.go:471 +0xb7
created by main.main
/Users/kenneth/beanseye/src/proxy/proxy.go:478 +0x21c0

goroutine 17 [runnable]:
memcache.(*ManualScheduler).procFeedback(0xc208084d00)
/Users/kenneth/beanseye/src/memcache/schedule.go:276
created by memcache.NewManualScheduler
/Users/kenneth/beanseye/src/memcache/schedule.go:205 +0x95e

AutoScheduler的数据分布问题

配置:N=3,W=2,32个节点。
现有的算法似乎会导致数据都分布在前三个节点,而后面的节点一点数据都没有,因为在初始化buckets时,每个bucket的节点列表顺序都是一样的。我测试的结果如下图所示,另外,图中除了bucket 12列标题显示warning对应的颜色,其它的bucket都显示为invalid对应的颜色。
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.