vicvinc / httpsqs Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/httpsqs
Automatically exported from code.google.com/p/httpsqs
Hi, I want to know that which are the main configuration which cause the good
performance. I am also interested to know about the acknowledge of message
consumed.
Original issue reported on code.google.com by [email protected]
on 21 May 2012 at 5:02
What steps will reproduce the problem?
1. $./httpsqs -x ~/
2. the httpsqs_settings_dataname=/home/foo//httpsqs.db
What is the expected output? What do you see instead?
httpsqs_settings_dataname=/home/foo/httpsqs.db
同时判断输入目录合法性
What version of the product are you using? On what operating system?
1.7
Original issue reported on code.google.com by [email protected]
on 1 Jul 2012 at 11:07
Attachments:
HttpsqsStatus.java line58:
protected static Pattern pattern = Pattern.compile("HTTP Simple Queue Service
v(.+?)\\s(?:.+?)\\sQueue Name: (.+?)\\sMaximum number of queues: (\\d+)\\sPut
position of queue \\((\\d+)st lap\\): (\\d+)\\sGet position of queue
\\((\\d+)st lap\\): (\\d+)\\sNumber of unread queue: (\\d+)");
since the client use (\\d+)st lap to match the first lap, it will cause regex
matching failed when the unread number goes into lap 2 or latter.
I change the pattern as follows:
protected static Pattern pattern = Pattern.compile("HTTP Simple Queue Service
v(.+?)\\s(?:.+?)\\sQueue Name: (.+?)\\sMaximum number of queues: (\\d+)\\sPut
position of queue \\((\\d+)\\w\\w lap\\): (\\d+)\\sGet position of queue
\\((\\d+)\\w\\w lap\\): (\\d+)\\sNumber of unread queue: (\\d+)");
hope this would help
Original issue reported on code.google.com by [email protected]
on 7 Jan 2011 at 8:16
What steps will reproduce the problem?
1. name=your_queue_name&opt=maxqueue&num=10
2. 5 puts => 0 get(or 1 get) => 6 puts.
What is the expected output? What do you see instead?
#expected output: return HTTPSQS_PUT_END while performing the 11th(or 12th) put
# what i saw: return HTTPSQS_PUT_OK while performing the 11th(or 12th) put
What version of the product are you using? On what operating system?
httpsqs version: httpsqs-1.3
OS: x86_64 GNU/Linux
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 18 Feb 2011 at 3:10
What steps will reproduce the problem?
1. install nginx-echo-module , add "echo_location_async
"http://192.168.1.12:1218/?name=log&opt=put&data=test";" into
nginx.conf("location" section)
2. start httpsqs
3. access the url (hosted by nginx) , then httpsqs crashed
What version of the product are you using? On what operating system?
v1.4(CentOS 5.1 x86_64)
Please provide any additional information below.
log in /var/log/messages:
Aug 20 16:45:50 node1 kernel: httpsqs[20496]: segfault at ffffffff00000010 rip
00002aaaaaad033b rsp 00007fffd41a4880 error 4
Aug 20 17:53:33 node1 kernel: httpsqs[20781]: segfault at 0000000000000021 rip
00002aaaaaacee7f rsp 00007fff688b7f90 error 6
Original issue reported on code.google.com by [email protected]
on 20 Aug 2010 at 10:21
一直没明白 tchdb,tcbdb,tcfdb,tctdb,tcadb!
看了您的源码:
TCBDB *httpsqs_db_tcbdb;
如果说, 我使用的是 TT, 我怎么选择使用哪种结构呢?
Original issue reported on code.google.com by [email protected]
on 31 Oct 2010 at 2:39
if(pid = fork())
exit(0);
else if(pid < 0)
exit(1);
pid=fork()
当返回值是-1时条件也是成立的,虽然能正常工作。但不够严
谨。
建议这样处理:
if((pid = fork()) >0)
exit(0);
else if(pid < 0)
exit(1);
Original issue reported on code.google.com by [email protected]
on 27 May 2013 at 1:29
now this queue only stores text data,
can it store the object data ?
Original issue reported on code.google.com by [email protected]
on 31 May 2010 at 9:58
文档中都是对单个队列进行操作的方法,
有没有对全部队列进行操作的相关接口呢?
Original issue reported on code.google.com by [email protected]
on 3 Aug 2011 at 4:06
修改为:
protected static Pattern pattern = Pattern.compile("HTTP Simple Queue Service
v(.+?)\\s(?:.+?)\\sQueue Name: (.+?)\\sMaximum number of queues: (\\d+)\\sPut
position of queue \\((\\d+)\\w+ lap\\): (\\d+)\\sGet position of queue
\\((\\d+)\\w+ lap\\): (\\d+)\\sNumber of unread queue: (\\d+)");
主要当put第二轮的时候\\d+nd 而原来在正则还是匹配
\\d+st改成 (\\d+)\\w+:
Queue Name: low
Maximum number of queues: 1000000
Put position of queue (2nd lap): 29068
Get position of queue (1st lap): 997815
Number of unread queue: 31253
Original issue reported on code.google.com by [email protected]
on 21 Oct 2011 at 3:27
What steps will reproduce the problem?
1. create a queue
2. set the maxqueue a little small, say: 10
3. just enqueue msgs, never dequeue, the 11st msg will overlap the 1st msg
Please provide any additional information below.
原因在于第303行判断队列已满时遗漏一种情况: (0 ==
queue_get_value && queue_put_value > maxqueue_num)。
Original issue reported on code.google.com by [email protected]
on 9 Mar 2011 at 9:04
Purpose of code changes on this branch:
When reviewing my code changes, please focus on:
After the review, I'll merge this branch into:
/trunk
Original issue reported on code.google.com by MRares
on 3 May 2010 at 9:49
How do I know what the current queue2.
Original issue reported on code.google.com by [email protected]
on 21 Dec 2010 at 3:49
更好发挥httpsqs
keepalive特性,减少请求连接数,将socket链接修改为长连接,��
�失去链接时自动重连
Original issue reported on code.google.com by [email protected]
on 13 Jun 2012 at 7:01
Attachments:
opt=status的时候, 没有Pos值
Original issue reported on code.google.com by [email protected]
on 2 Sep 2010 at 12:46
一直没明白 tchdb,tcbdb,tcfdb,tctdb,tcadb!
看了您的源码:
TCBDB *httpsqs_db_tcbdb;
如果说, 我使用的是 TT, 我怎么选择使用哪种结构呢?
Original issue reported on code.google.com by [email protected]
on 31 Oct 2010 at 2:41
Attention: Unable to open the database.
Original issue reported on code.google.com by [email protected]
on 8 Feb 2010 at 12:29
这份http://blog.s135.com/httpsqs/文档里的
"4、HTTPSQS 服务器使用文档" 小节
请麻烦更新补充以下说明:
如何创建不同的队列?
httpsqs -d -p 1218 -x /usr/local/project/httpsqs/q2
httpsqs -d -p 1218 -x /usr/local/project/httpsqs/q1
q1和q2为队列名字,对应于客户端
http://x.x.x.x:1218/?name=q2&opt=put..
里的name参数。
方便后来人。
Original issue reported on code.google.com by [email protected]
on 10 Feb 2010 at 9:04
line 76:
$body = @fread($httpsqs_socket, $len);
应该改为如下方式
if ($len < 0)
{
return false;
}
$body = '';
//修复fread大小限制问题
while (!feof($httpsqs_socket)) {
$body .= @fread($httpsqs_socket, 1024);
}
fclose($httpsqs_socket);
$result_array["pos"] = (int)$pos_value;
$result_array["data"] = $body;
return $result_array;
Original issue reported on code.google.com by [email protected]
on 18 Aug 2010 at 8:46
What steps will reproduce the problem?
1. ab -c1000 -n50000 http://ip:port/?name=x&data=somedata&opt=put
2. 看status发现数据条目数往往不对。
3. 同样,可以做下get的测试,往往也是不对的。
What is the expected output? What do you see instead?
What version of the product are you using? On what operating system?
Please provide any additional information below.
—— 1并发的测试 ———–
队列初始信息:
HTTP Simple Queue Service v1.3.1
——————————
Queue Name: x
Maximum number of queues: 1000000
Put position of queue (1st lap): 0
Get position of queue (1st lap): 0
Number of unread queue: 0
尝试1并发的50000次添加:
~/laoyc/soft/bin/ab -c1 -n50000
“http://192.168.22.31:1218/?name=x&data=1000000000&opt=put”
全部成功,查看队列状态:
HTTP Simple Queue Service v1.3.1
——————————
Queue Name: x
Maximum number of queues: 1000000
Put position of queue (1st lap): 50000
Get position of queue (1st lap): 0
Number of unread queue: 50000
尝试49999次get:
~/laoyc/soft/bin/ab -c1 -n49999
“http://192.168.22.31:1218/?name=x&data=1000000000&opt=get”
全部成功,再次查看队列状态:
HTTP Simple Queue Service v1.3.1
——————————
Queue Name: x
Maximum number of queues: 1000000
Put position of queue (1st lap): 50000
Get position of queue (1st lap): 49999
Number of unread queue: 1
正确。
在测试一次:发现正确。
————– 1000 并发的测试 ————-
HTTP Simple Queue Service v1.3.1
——————————
Queue Name: x
Maximum number of queues: 1000000
Put position of queue (1st lap): 0
Get position of queue (1st lap): 0
Number of unread queue: 0
开始测试并发1000放50000次:
~/laoyc/soft/bin/ab -c1000 -n50000
“http://192.168.22.31:1218/?name=x&data=1000000000&opt=put”
发现结果是:
HTTP Simple Queue Service v1.3.1
——————————
Queue Name: x
Maximum number of queues: 1000000
Put position of queue (1st lap): 50020
Get position of queue (1st lap): 0
Number of unread queue: 50020
看起来并发是有问题,现在不确定是ab还是httpsqs的并发处理��
�还是tc的问题。
所以这个队列目前只能应用在缓存和不重要的任务的情景。
1000并行get49999次:
~/laoyc/soft/bin/ab -c1000 -n49999
“http://192.168.22.31:1218/?name=x&data=1000000000&opt=get”
HTTP Simple Queue Service v1.3.1
——————————
Queue Name: x
Maximum number of queues: 1000000
Put position of queue (1st lap): 50020
Get position of queue (1st lap): 50020
Number of unread queue: 0
可见,get也是不准确的。
Original issue reported on code.google.com by [email protected]
on 16 Nov 2010 at 2:07
当队列信息出现 2nd
lap时候,即队列已满,重新从队列头开始覆盖。httpsqs4j
会出现无法正常创建HttpsqsStatus对象的问题。
例如:
HTTP Simple Queue Service v1.6
------------------------------
Queue Name: xoyo
Maximum number of queues: 1000000
Put position of queue (2nd lap): 4562
Get position of queue (1st lap): 900045
Number of unread queue: 104517
原因是以前是2st,现在是2nd,所以改了下正则,就可以正常��
�用了。另附上patch
Original issue reported on code.google.com by [email protected]
on 12 Jun 2011 at 7:42
Attachments:
What steps will reproduce the problem?
1. Load the queue up to maxlen + 1
2. Queue resets to 0
3. You lose maxlen messages
What is the expected output? What do you see instead?
What version of the product are you using? On what operating system?
Please provide any additional information below.
Original issue reported on code.google.com by MRares
on 29 Apr 2010 at 7:53
What steps will reproduce the problem?
1. Create a queue of length L.
2. View the contents of the specified queue pos < 1.
3. View the contents of the specified queue pos > L.
4. View the contents of the specified queue pos (pos is not numerical).
5. View the contents of the specified queue pos >=1 && pos <=L.
What is the expected output? What do you see instead?
What I saw:
HTTPSQS_ERROR when pos < 1 (Expected)
BLANK when pos > L [(Expected): HTTPSQS_ERROR]
HTTPSQS_ERROR when pos is not numerical (Expected)
Get the right value when pos >=1 && pos <=L (Expected)
What version of the product are you using? On what operating system?
HTTP Simple Queue Service v1.1
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 11 Feb 2010 at 1:45
# -*- coding: utf-8 -*-
#!/usr/bin/python
import urllib2
import urllib
import sys
def httpsqs_get(host, port,name,charset='utf-8'):
data={'charset':charset,'name' : name, 'opt' : 'get'}
url='http://'+host+':'+port+'/?'
data=urllib.urlencode(data)
getString=url+data
req = urllib2.Request(getString)
fd = urllib2.urlopen(req)
str=fd.read()
if str=='HTTPSQS_GET_END':
print 'HTTPSQS_GET_END'
else:
return str
def httpsqs_put(host, port,name,data,charset='utf-8'):
data={'charset':charset,'name' : name, 'opt' : 'put','data' : data}
url='http://'+host+':'+port+'/?'
data=urllib.urlencode(data)
getString=url+data
req = urllib2.Request(getString)
fd = urllib2.urlopen(req)
str=fd.read()
if str=='HTTPSQS_PUT_OK':
return 1
else:
return 0
#下面是调用方式
data=sfsdfasafasdfasdfadfasdfadsfsdf
srr=httpsqs_put('127.0.0.1','1218','test',data)
sss=httpsqs_get('127.0.0.1','1218','test')
保存为py文件即可
Original issue reported on code.google.com by [email protected]
on 13 Jun 2010 at 6:01
What steps will reproduce the problem?
1. 用ab压力测试httpsqs
What is the expected output? What do you see instead?
What version of the product are you using? On what operating system?
httpsqs 1.3
libevent-1.4.13-stable
tokyocabinet-1.4.45
OS:ubuntu 10.10
Please provide any additional information below.
在ab一直put,get压力测试下,我发现硬盘数据一直增大,内存
使用一直增大,如何操作或者设置能让它们降下来?
附件是内存和硬盘的部分变化。操作就是用ab
put和get压力测试。
Original issue reported on code.google.com by [email protected]
on 11 Dec 2010 at 3:17
Attachments:
输入:SS%EE
编码后为:SS%25EE
输出:SS�
输入:SS%88EE
编码后为:SS%2588EE
输出:SS�EE
当出现下面的情况会导致数据直接被截断:
输入:321%00123
编码后为:321%2500123
输出:321(后面的字符被截断了)
临时解决办法:
添加数据进队列前如果发现数据中含有%,则先将%替换为%25再
添加进队列,取出来的就是正确的数据了,似乎是因为如果��
�现了%25,会先将%25解析为%,然后又解析了一次,才导致了这
个问题。
具体的代码没仔细研究(不熟悉C),也许是 char *urldecode(char
*input_str) 函数导致的吧。
Original issue reported on code.google.com by [email protected]
on 5 May 2014 at 3:35
How do I know what the current queue. if have another key to get this messaage?
Original issue reported on code.google.com by [email protected]
on 21 Dec 2010 at 3:50
查看指定队列位置点的内容时,返回false。经检查发现变量写
错。
Original issue reported on code.google.com by [email protected]
on 16 Nov 2011 at 6:41
Attachments:
提交了一个PR,移植httpsqs到FreeBSD上:
http://www.freebsd.org/cgi/query-pr.cgi?pr=156595
问题:
1. 编译期的一个warning,wait函数被隐式定义。需要加入头文件
#include <sys/wait.h>
根据编译gcc的参数不同,有些gcc会自己include某些头文件,可�
��看不到这个warning。
2. 使用了 procfs,只是为了获取可执行文件的路径。
BSD* 下 procfs
是不推荐使用的(默认也不挂载),但也没有简单可靠地获��
�进程路径的方法。我在移植过程中把这个路径用编译时port系
统指定的安装路径替换了。
3. 不支持读配置文件。
这个给编写 rc.d
脚本(或者init.d脚本)带来了困难。因为这些脚本只控制daemo
n的启动,终止等状态,daemon的功能性配置应该与之分离。
4. 以非daemon方式运行时,键盘Ctrl-c不会kill掉子进程。
考虑一下在非daemon模式下响应SIGINT信号(更简单的做法是绑��
�atexit函数)。
Original issue reported on code.google.com by [email protected]
on 23 Apr 2011 at 8:57
What steps will reproduce the problem?
1.
2.
3.
What is the expected output? What do you see instead?
What version of the product are you using? On what operating system?
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 16 Nov 2011 at 11:25
假设在1个Web请求中,有100次入队,HTTP的话需要连接-关闭100��
�。TCP只需连接-关闭1次,发送100次数据即可
Original issue reported on code.google.com by [email protected]
on 13 Sep 2010 at 3:05
wget http://httpsqs.googlecode.com/files/tokyocabinet-1.4.47.tar.gz
tar zxvf tokyocabinet-1.4.47.tar.gz
cd tokyocabinet-1.4.47/
./configure --prefix=/usr/local/tokyocabinet-1.4.47/
执行到上一句时,出现以下错误
configure: error: bzlib.h is required
说是需要安装啥啥来着,可是我内网服务器不能上网,网络��
�供的都是YUM安装呀。
Original issue reported on code.google.com by [email protected]
on 21 Aug 2012 at 10:55
如题?
Original issue reported on code.google.com by ferock
on 28 Aug 2014 at 3:54
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.