alibaba / nginx-tfs Goto Github PK
View Code? Open in Web Editor NEWAn Asynchronous Nginx module providing a RESTful API for TFS (Taobao File System).
Home Page: http://tfs.taobao.org/
An Asynchronous Nginx module providing a RESTful API for TFS (Taobao File System).
Home Page: http://tfs.taobao.org/
您好,请教下upstream动态更改 怎么实现的?
PUT /v2/tfs/1/1234/file/file_1 HTTP/1.1 写文件报500错误,其他所有测试都过了,不知道什么原因 ,目录创建删除,文件创建删除都没有问题,就是写文件时报错500,不知道什么原因。
objs/addon/src/ngx_http_tfs_module.o: In function `ngx_http_tfs_log':
/home/makerpm/git/nginx-tfs/src/ngx_http_tfs_module.c:686: undefined reference to `ngx_log_create'
/home/makerpm/git/nginx-tfs/src/ngx_http_tfs_module.c:695: undefined reference to `ngx_log_set_levels'
collect2: ld returned 1 exit status
gmake[2]: *** [objs/nginx] Error 1
gmake[2]: Leaving directory `/home/makerpm/tar/ngx_openresty-1.7.0.1/build/nginx-1.7.0'
gmake[1]: *** [build] Error 2
gmake[1]: Leaving directory `/home/makerpm/tar/ngx_openresty-1.7.0.1/build/nginx-1.7.0'
gmake: *** [all] Error 2
ngx_log_create 被不存在了。
我找其中一个等待shutdown的进程,跟踪下结果如下,发现有Resource temporarily unavailable不知道什么原因,这个问题只有使用nginx-tfs时才会出现,版本tengine-2.1.0和tengine2.1.2上都会出现,有的进程shutdown状态持续到几个月。
strace -p 31072
Process 31072 attached - interrupt to quit
epoll_wait(27, {}, 512, 7029) = 0
epoll_wait(27, {}, 512, 1) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 2682
ioctl(2682, FIONBIO, [1]) = 0
epoll_ctl(27, EPOLL_CTL_ADD, 2682, {EPOLLIN|EPOLLOUT|EPOLLET|0x2000, {u32=495118000, u64=140699328767664}}) = 0
connect(2682, {sa_family=AF_INET, sin_port=htons(5100), sin_addr=inet_addr("172.16.0.198")}, 16) = -1 EINPROGRESS (Operation now in progress)
epoll_wait(27, {{EPOLLOUT, {u32=495118000, u64=140699328767664}}}, 512, 9999) = 1
getsockopt(2682, SOL_SOCKET, SO_ERROR, [102868521188327424], [4]) = 0
writev(2682, [{"TFSN\r\1\0\0D\0\2\0f\20\r\0\0\0\0\0gi-\3632\0\0\0002-31"..., 293}], 1) = 293
epoll_wait(27, {{EPOLLIN|EPOLLOUT, {u32=495118000, u64=140699328767664}}}, 512, 9997) = 1
readv(2682, [{"TFSN\1\0\0\0E\0\2\0f\20\r\0\0\0\0\0\253\306Hl", 24}, {"\0\311\377\377\27\0\0\0execute message failed\0\2"..., 20971520}], 2) = 25
recvfrom(2682, 0x7fff6b9ed65f, 1, 2, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
getsockopt(2682, SOL_SOCKET, SO_ERROR, [102868521188327424], [4]) = 0
writev(2682, [{"TFSN\r\1\0\0D\0\2\0g\20\r\0\0\0\0\0\23\310\36232\0\0\0001-31"..., 293}], 1) = 293
readv(2682, 0x7fff6b9ed4c0, 2) = -1 EAGAIN (Resource temporarily unavailable)
epoll_wait(27, {{EPOLLIN|EPOLLOUT, {u32=495118000, u64=140699328767664}}}, 512, 9996) = 1
readv(2682, [{"TFSN\1\0\0\0E\0\2\0g\20\r\0\0\0\0\0\253\306Hl", 24}, {"\0\311\377\377\27\0\0\0execute message failed\0\2"..., 20971520}], 2) = 25
recvfrom(2682, 0x7fff6b9ed60f, 1, 2, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
close(2682) = 0
发现tfs_pass只能配置在 "/" location下面,如果该nginx不光处理tfs,还可能有其他业务的配置,感觉让tfs走”/“ location有点不妥。
Hi,
我正在使用TFS,想使用restapi,发现没办法编译过去。
cc1: warnings being treated as errors
../nginx-tfs-master//ngx_http_tfs_module.c: 在函数‘ngx_http_tfs_tackle_log’中:
../nginx-tfs-master//ngx_http_tfs_module.c:517: 错误:隐式声明函数‘ngx_log_create’
../nginx-tfs-master//ngx_http_tfs_module.c:517: 错误:赋值时将整数赋给指针,未作类型转换
../nginx-tfs-master//ngx_http_tfs_module.c:526: 错误:隐式声明函数‘ngx_log_set_levels’
../nginx-tfs-master//ngx_http_tfs_module.c:526: 错误:返回时将整数赋给指针,未作类型转换
make[1]: *** [objs/addon/nginx-tfs-master/ngx_http_tfs_module.o] 错误 1
make[1]: Leaving directory `/root/tengine-2.1.0'
make: *** [build] 错误 2
去掉MakeFile 的-Werror 选项会出现下面的错误:
objs/addon/nginx-tfs-master/ngx_http_tfs_module.o: In function ngx_http_tfs_tackle_log': /root/tengine-2.1.0/../nginx-tfs-master//ngx_http_tfs_module.c:517: undefined reference to ngx_log_create'
/root/tengine-2.1.0/../nginx-tfs-master//ngx_http_tfs_module.c:526: undefined reference to `ngx_log_set_levels'
请问我该设置什么才能编译过去?
1.在post数据的时候如果不加suffix=这个参数会提示400错误,然后加了之后就可以提交数据了,然后我多加了个simple_name参数就又报400错误,然后看你api说明的时候这两个不加的时候也应该是可以的,加了两个也可以,现在这个算是bug吗?
2.我用post提交完数据之后返回文件名了,然后用这个文件名想用restful api的get接口搞下来,但始终无效,报404 not found,然后用后台的nameserver查看那个文件名发现是已经提交上去了的,请问这是什么原因?
/usr/local/src/nginx-tfs/src/ngx_http_tfs.c: 在函数‘ngx_http_tfs_process_upstream_request’中:
/usr/local/src/nginx-tfs/src/ngx_http_tfs.c:874: 错误:提供给函数‘c->recv_chain’的实参太少
make[1]: *** [objs/addon/src/ngx_http_tfs.o] 错误 1
make[1]: *** 正在等待未完成的任务....
cc1: warnings being treated as errors
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c: 在函数‘ngx_http_tfs_log’中:
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:684: 错误:隐式声明函数‘ngx_log_create’
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:684: 错误:赋值时将整数赋给指针,未作类型转换
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:693: 错误:隐式声明函数‘ngx_log_set_levels’
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:693: 错误:返回时将整数赋给指针,未作类型转换
make[1]: *** [objs/addon/src/ngx_http_tfs_module.o] 错误 1
make[1]: Leaving directory `/usr/local/src/nginx-1.10.3'
make: *** [build] 错误 2
编译一直报这个错,这里用的是nginx-1.10.3版本,换了其它版本以及tengine也不行
10.81.254.51 - - [18/Aug/2014:17:04:47 +0800] "GET /tfs/v1/tfs/T1Bvh8BTZ_1RCvBVdK HTTP/1.1" 200 909787 "-" "Golang rpc package" "-" 7.765 7.312
10.81.254.51 - - [18/Aug/2014:17:04:48 +0800] "GET /tfs/v1/tfs/T1Bvh8BTZ_1RCvBVdK HTTP/1.1" 200 863255 "-" "Golang rpc package" "-" 1.555 1.342
返回的大小相差不小。
nginx version: ngx_openresty/1.4.3.9
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
TLS SNI support enabled
configure arguments: --prefix=/home/server/openresty/nginx --add-module=../ngx_devel_kit-0.2.19 --add-module=../iconv-nginx-module-0.10 --add-module=../echo-nginx-module-0.50 --add-module=../xss-nginx-module-0.04 --add-module=../ngx_coolkit-0.2rc1 --add-module=../set-misc-nginx-module-0.23 --add-module=../form-input-nginx-module-0.07 --add-module=../encrypted-session-nginx-module-0.03 --add-module=../ngx_postgres-1.0rc3 --add-module=../srcache-nginx-module-0.24 --add-module=../ngx_lua-0.9.3 --add-module=../headers-more-nginx-module-0.24 --add-module=../array-var-nginx-module-0.03 --add-module=../memc-nginx-module-0.14 --add-module=../redis2-nginx-module-0.10 --add-module=../redis-nginx-module-0.3.7 --add-module=../auth-request-nginx-module-0.2 --add-module=../rds-json-nginx-module-0.13 --add-module=../rds-csv-nginx-module-0.05 --with-ld-opt=-Wl,-rpath,/home/server/openresty/luajit/lib --with-http_ssl_module --with-http_stub_status_module --add-module=/home/makerpm/git/nginx-auth-ldap --add-module=/home/makerpm/git/nginx_upstream_hash --add-module=/home/makerpm/git/ngx_http_substitutions_filter_module --add-module=/home/makerpm/git/nginx-tfs --add-module=/home/makerpm/git/nginx-upstream-fair --with-pcre=/home/makerpm/tar/pcre-8.34 --with-pcre-jit --with-openssl=/home/makerpm/tar/openssl-1.0.1h
nginx-tfs 模块是从 tenginx 里面拷贝出来的。
请教:在通过http://xxx:7500/v1/tfs 上传文件时,如何加鉴权?
比如想在客户端http POST时在head中加一个从用户系统得到的token,然后请求TFS时如何鉴权这个token?
目前的想法独立做一个上传文件的WEB应用,在这里做head token鉴权,通过后再转发到TFS上。这样的话就多了一个应用。我们nginx-tfs目前有没有类似的鉴权方法?盼复~~
如题。restful api 中有选择存储为大文件或者小文件,我们根据什么规则存储为大文件或小文件呢?这两种存储方式有什么区别呢?
类似的问题:
http://code.taobao.org/p/tfs/issue/1312/
相关信息:
$ uname -a
Linux tfs004 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
$ ./nginx -V
nginx version: ngx_openresty/1.2.7.1
built by gcc 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC)
TLS SNI support enabled
configure arguments: --prefix=/home/server/openresty-debug/nginx --with-debug --with-cc-opt=-O0 --add-module=../ngx_devel_kit-0.2.18 --add-module=../iconv-nginx-module-0.10rc7 --add-module=../echo-nginx-module-0.42 --add-module=../xss-nginx-module-0.03rc9 --add-module=../ngx_coolkit-0.2rc1 --add-module=../set-misc-nginx-module-0.22rc8 --add-module=../form-input-nginx-module-0.07 --add-module=../encrypted-session-nginx-module-0.02 --add-module=../ngx_postgres-1.0rc2 --add-module=../srcache-nginx-module-0.19 --add-module=../ngx_lua-0.7.16 --add-module=../headers-more-nginx-module-0.19 --add-module=../array-var-nginx-module-0.03rc1 --add-module=../memc-nginx-module-0.13rc3 --add-module=../redis2-nginx-module-0.09 --add-module=../redis-nginx-module-0.3.6 --add-module=../auth-request-nginx-module-0.2 --add-module=../rds-json-nginx-module-0.12rc10 --add-module=../rds-csv-nginx-module-0.05rc2 --with-ld-opt=-Wl,-rpath,/home/server/openresty-debug/luajit/lib --with-http_ssl_module --with-http_stub_status_module --add-module=/home/makerpm/git/nginx-auth-ldap --add-module=/home/makerpm/git/nginx_upstream_hash --add-module=/home/makerpm/git/ngx_http_substitutions_filter_module --add-module=/home/makerpm/git/nginx-tfs
$ git log -1
commit d1c8349
Merge: dcb7507 83d9a33
Author: Zheng Cen [email protected]
Date: Thu Jan 17 03:03:57 2013 -0800
Merge pull request #12 from monadbobo/fix_upstream
Fix upstream
(1) 使用下面的 nginx.conf 启动 nginx
nginx.conf: https://gist.github.com/harveyzh/5055847#file-nginx-conf
(2) ngx.location.capture 指令直接调用 nginx-tfs 模块会一直 block
测试命令:
$ curl -i 127.0.0.1/test_block
^C
相关配置片段:
location /test_block {
content_by_lua '
res = ngx.location.capture("/v1/tfs/T17yCTByJT1RCvBVdK")
if res.status == ngx.HTTP_OK then
ngx.print(res.body)
else
ngx.exit(res.status)
end
';
}
debug log: https://gist.github.com/harveyzh/5055847#file-error-log-block
block时的 debug log 停在
2013/02/28 18:41:06 [debug] 29836#0: *6 http copy filter: 0 "/v1/tfs/T17yCTByJT1RCvBVdK?"
2013/02/28 18:41:06 [debug] 29836#0: *6 http finalize request: 0, "/v1/tfs/T17yCTByJT1RCvBVdK?" a:0, c:2
2013/02/28 18:41:06 [debug] 29836#0: *6 lua run post subrequest handler, rc:0 c:2
2013/02/28 18:41:06 [debug] 29836#0: *6 lua restoring write event handler
2013/02/28 18:41:06 [debug] 29836#0: *6 http wake parent request: "/test_block?"
2013/02/28 18:41:06 [debug] 29836#0: keepalive dummy handler
2013/02/28 18:41:06 [debug] 29836#0: timer delta: 1
2013/02/28 18:41:06 [debug] 29836#0: posted events 0000000000000000
2013/02/28 18:41:06 [debug] 29836#0: worker cycle
2013/02/28 18:41:06 [debug] 29836#0: epoll timer: -1
(3) ngx.location.capture 指令间接通过 proxy_pass 调用 nginx-tfs 模块则 ok
测试命令:
$ curl -i 127.0.0.1/test_ok
HTTP/1.1 200 OK
Server: ngx_openresty
Date: Thu, 28 Feb 2013 10:59:44 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Connection: keep-alive
test!
相关配置片段:
location /tfs/ {
proxy_pass http://127.0.0.1/v1/tfs/;
}
location /test_ok {
content_by_lua '
res = ngx.location.capture("/tfs/T17yCTByJT1RCvBVdK")
if res.status == ngx.HTTP_OK then
ngx.print(res.body)
else
ngx.exit(res.status)
end
';
}
debug log: https://gist.github.com/harveyzh/5055847#file-error-log-ok
问题描述:
通过nginx-tfs上传大于2m的文件到TFS失败,nginx-tfs返回500。而使用TFS client可以正常写入大于2m的文件。在dataserver的日志发现, dataserver 解析nginx-tfs传来的数据报,取到数据包长度为负值。会议nginx-tfs存在bug。
nginx-tfs 相关的配置如下:
## TFS
tfs_send_timeout 3s;
tfs_read_timeout 3s;
tfs_connect_timeout 3s;
tfs_body_buffer_size 10M;
tfs_block_cache_zone size=256M;
tfs_keepalive max_cached=128 bucket_count=16;
nginx的debug日志如下:
2013/10/09 16:15:20 [info] 19460#0: 8 meta segment: block_id: 0, fileid: 0, seq_id: 0, suffix: 0, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs..com"
2013/10/09 16:15:20 [info] 19460#0: *8 get block info from ns while connecting server, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**.com"
2013/10/09 16:15:20 [debug] 19460#0: *8 connecting name server, addr: 10.33.56.125:8108
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs finalize state name server, 0 while reading response header from tfs, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.__.com"
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs process next peer is data server, addr: 10.33.56.198:8010 while reading response header from tfs, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**__.com"
2013/10/09 16:15:20 [debug] 19460#0: *8 connecting data server, addr: 10.33.56.198:8010
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs finalize state data server, 0 while reading response header from tfs, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs..com"
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs process next peer is data server, addr: 10.33.56.198:8010 while reading response header from tfs, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**.com"
2013/10/09 16:15:20 [info] 19460#0: 8 write segment index 0, block id: 2404301, file id: 2354, offset: 0, length: 2097152, crc: 2021805896 while connecting server, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.__.com"
2013/10/09 16:15:20 [debug] 19460#0: *8 connecting data server, addr: 10.33.56.198:8010
2013/10/09 16:15:20 [error] 19460#0: *8 readv() failed (104: Connection reset by peer) while reading response header from tfs, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**__.com"
2013/10/09 16:15:20 [error] 19460#0: *8 recv chain error while reading response header from tfs, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs._.com"
TFS 使用2.2.13 版本。dataserver的报错日志为:
2013-10-09 18:07:22] ERROR getPacketInfo (base_packet_streamer.cpp:86) [139896236799744] stream error: 69fc7285<>4d534654,4e534654, dataLen: 432408354
[2013-10-09 18:07:22] ERROR reply (base_packet.cpp:221) [139896073172736] post packet failure, server: 10.33.56.23:55566, pcode:1
[2013-10-09 18:07:22] ERROR callback (dataservice.cpp:854) [139896073172736] write data fail. filenumber: 14152088283100828036, blockid: 2399407, fileid: 2915, version: 2914, leaseid: 1198248338, role: master
[2013-10-09 17:28:43] ERROR getPacketInfo (base_packet_streamer.cpp:86) [139896236799744] stream error: ebe940fc<>4d534654,4e534654, dataLen: -1052153523
[2013-10-09 17:28:43] ERROR reply (base_packet.cpp:221) [139896167581440] post packet failure, server: 10.81.102.121:54715, pcode:1
[2013-10-09 17:28:43] ERROR callback (dataservice.cpp:854) [139896167581440] write data fail. filenumber: 14152088283100823092, blockid: 2407850, fileid: 1621, version: 1620, leaseid: 1197348515, role: master
Hi, I noticed that there are APIs for creating and manipulating directories, but missing docs describing how to store a file under dir, So, could you give me some hints for this? thanks.
执行创建文件(create_file) POST /v2/appkey/appid/uid/file/file_name HTTP/1.1接口可以正确返回201状态,但是在执行写文件(write_file) PUT /v2/appkey/appid/uid/file/file_name HTTP/1.1接口时,nginx err.log中出现如下信息:
2017/12/06 17:50:14 [error] 34442#0: *1257 error, cluster id conflict: 1(ns) <> 54(rcs) while reading response header from tfs, client: 192.168.10.57, server: localhost, request: "PUT /v2/tappkey00001/1/21312/file/file1?offset=213213213&size=2436 HTTP/1.1", host: "192.168.10.128:7500" 2017/12/06 17:50:14 [error] 34442#0: *1257 http tfs process name server request failed while reading response header from tfs, client: 192.168.10.57, server: localhost, request: "PUT /v2/tappkey00001/1/21312/file/file1?offset=213213213&size=2436 HTTP/1.1", host: "192.168.10.128:7500"
请大神帮忙解惑!小弟不胜感激!
tfs递归创建文件时怎么第一次操作是创建目录而不是文件?
想创建/cc/cc/cc文件,却创建/cc/cc/cc目录。
[root@tfsdatasvr1 ~]# curl -i -XPOST http://tfs.test.com/v2/tappkey00001/1/1030/file/cc/cc/cc?recursive=1
HTTP/1.1 201 Created
Server: Tengine/2.1.2
Date: Mon, 27 Feb 2017 03:06:59 GMT
Content-Length: 0
Connection: keep-alive
[root@tfsdatasvr1 ~]# curl http://tfs.test.com/v2/tappkey00001/1/1030/dir/cc
[
{
"NAME": "cc",
"PID": 43489476,
"ID": 43489477,
"SIZE": 0,
"IS_FILE": false,
"CREATE_TIME": "Mon, 27 Feb 2017 02:47:12 GMT",
"MODIFY_TIME": "Mon, 27 Feb 2017 02:47:12 GMT",
"VER_NO": 0
}
]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.