Giter Club home page Giter Club logo

saber's Introduction

Saber

Latest Version PHPUnit for Saber Php Version Swoole Version Saber License

简介

HTTP军刀(呆毛王), Swoole人性化组件库之PHP高性能HTTP客户端, 基于Swoole原生协程, 支持多种风格操作, 底层提供高性能解决方案, 让开发者专注于功能开发, 从传统同步阻塞且配置繁琐的Curl中解放.

English Document

  • 基于Swoole协程Client开发
  • 人性化使用风格, ajax.js/axios.js/requests.py用户福音, 同时支持PSR风格操作
  • 浏览器级别完备的Cookie管理机制, 完美适配爬虫/API代理应用
  • 请求/响应/异常拦截器
  • 多请求并发, 并发重定向优化
  • 连接池, 自动化复用长连接
  • 通道池(Chan): 最大连接数限制+无阻塞
  • HTTPS连接, CA证书自动化支持
  • HTTP/Socks5 Proxy支持
  • WebSocket连接支持
  • 毫秒级超时定时器
  • 自动化 编码请求/解析响应 数据
  • 响应报文自动编码转换
  • 异步超大文件上传/下载, 断点重传
  • 自动重试机制
  • 单次并发数控制
  • 多模式/超细粒度异常处理机制
  • (=)浏览器级别缓存机制
  • (=)随机UA生成器


安装

最好的安装方法是通过 Composer 包管理器 :

composer require swlib/saber

依赖

  • PHP71 or later
  • Swoole 2.1.2 or later
  • Swoole 4 is the best


协程调度

Swoole底层实现协程调度, 业务层无需感知, 开发者可以无感知的用同步的代码编写方式达到异步IO的效果和超高性能,避免了传统异步回调所带来的离散的代码逻辑和陷入多层回调中导致代码无法维护.

需要在onRequet, onReceive, onConnect等事件回调函数中使用, 或是使用go关键字包裹 (swoole.use_shortname默认开启).

go(function () {
    echo SaberGM::get('http://httpbin.org/get');
})

目录


例子

静态方法

数据自动打包: 传入的data会自动转换成content-type所指定的类型格式

默认为x-www-form-urlencoded, 也支持json等其它格式

SaberGM := Saber Global Manager, 如果觉得类名有点长, 可以使用class_alias自己取别名, 推荐服务中使用生成实例的方式使用, 而把SaberGM作为快捷方式.

SaberGM::get('http://httpbin.org/get');
SaberGM::delete('http://httpbin.org/delete');
SaberGM::post('http://httpbin.org/post', ['foo' => 'bar']);
SaberGM::put('http://httpbin.org/put', ['foo' => 'bar']);
SaberGM::patch('http://httpbin.org/patch', ['foo' => 'bar']);

生成实例

适用API代理服务

$saber = Saber::create([
    'base_uri' => 'http://httpbin.org',
    'headers' => [
        'Accept-Language' => 'en,zh-CN;q=0.9,zh;q=0.8',
        'Content-Type' => ContentType::JSON,
        'DNT' => '1',
        'User-Agent' => null
    ]
]);
echo $saber->get('/get');
echo $saber->delete('/delete');
echo $saber->post('/post', ['foo' => 'bar']);
echo $saber->patch('/patch', ['foo' => 'bar']);
echo $saber->put('/put', ['foo' => 'bar']);

生成会话

Session会自动保存cookie信息, 其实现是浏览器级别完备

$session = Saber::session([
    'base_uri' => 'http://httpbin.org',
    'redirect' => 0
]);
$session->get('/cookies/set?foo=bar&k=v&apple=banana');
$session->get('/cookies/delete?k');
echo $session->get('/cookies')->body;

并发请求

注意: 此处使用了并发重定向优化方案, 多个重定向总是依旧并发的而不会退化为队列的单个请求

$responses = SaberGM::requests([
    ['uri' => 'http://github.com/'],
    ['uri' => 'http://github.com/'],
    ['uri' => 'https://github.com/']
]);
echo "multi-requests [ {$responses->success_num} ok, {$responses->error_num} error ]:\n" ."consuming-time: {$responses->time}s\n";

// multi-requests [ 3 ok, 0 error ]:
// consuming-time: 0.79090881347656s
// 别名机制可以省略参数书写参数名
$saber = Saber::create(['base_uri' => 'http://httpbin.org']);
echo $saber->requests([
    ['get','/get'],
    ['post','/post'],
    ['patch','/patch'],
    ['put','/put'],
    ['delete','/delete']
]);

数据解析

目前支持json,xml,html,url-query四种格式的数据快速解析

[$json, $xml, $html] = SaberGM::list([
    'uri' => [
        'http://httpbin.org/get',
        'http://www.w3school.com.cn/example/xmle/note.xml',
        'http://httpbin.org/html'
    ]
]);
var_dump($json->getParsedJsonArray());
var_dump($json->getParsedJsonObject());
var_dump($xml->getParsedXmlArray());
var_dump($xml->getParsedXmlObject(true));
var_dump($html->getParsedDomObject()->getElementsByTagName('h1')->item(0)->textContent);

网络代理

支持HTTP和SOCKS5代理

$uri = 'http://myip.ipip.net/';
echo SaberGM::get($uri, ['proxy' => 'http://127.0.0.1:1087'])->body;
echo SaberGM::get($uri, ['proxy' => 'socks5://127.0.0.1:1086'])->body;

文件上传

底层自动协程调度, 可支持异步发送超大文件, 断点续传

同时上传三个文件(三种参数风格string| array |object)

$file1 = __DIR__ . '/black.png';
$file2 = [
    'path' => __DIR__ . '/black.png',
    'name' => 'white.png',
    'type' => ContentType::MAP['png'],
    'offset' => null, //re-upload from break
    'size' => null //upload a part of the file
];
$file3 = new SwUploadFile(
    __DIR__ . '/black.png',
    'white.png',
    ContentType::MAP['png']
);

echo SaberGM::post('http://httpbin.org/post', null, [
        'files' => [
            'image1' => $file1,
            'image2' => $file2,
            'image3' => $file3
        ]
    ]
);

超大文件下载

Download收到数据后会直接异步写入到磁盘, 而不是在内存中对HttpBody进行拼接. 因此download仅使用小量内存, 就可以完成超大文件的下载. 且支持断点续传, 通过设置offset参数来进行断点下载.

异步下载Saber壁纸

$download_dir = '/tmp/saber.jpg';
$response = SaberGM::download(
    'https://ws1.sinaimg.cn/large/006DQdzWly1fsr8jt2botj31hc0wxqfs.jpg',
    $download_dir
);
if ($response->success) {
    exec('open ' . $download_dir);
}

自动重试

在爬虫项目中, 请求失败自动重试是非常常见的需求, 比如会话过期后重新登录.

Saber内置了此功能, 并可使用拦截器来强化它.

如未设置retry_time而设置了retry拦截器, 则retry_time会置为1, 如retry拦截器的回调方法返回了false, 无论retry_time是多少, 都会在返回false时终止重试.

$uri = 'http://eu.httpbin.org/basic-auth/foo/bar';
$res = SaberGM::get(
    $uri, [
        'exception_report' => 0,
        'retry_time' => 3,
        'retry' => function (Saber\Request $request) {
            echo "retry...\n";
            $request->withBasicAuth('foo', 'bar'); //发现失败后添加验证信息
            if ('i don not want to retry again') {
                return false; // shutdown
            }
        }
    ]
);
echo $res;

缓存机制

有时候HTTP资源并不会总是变更, 我们可以学习浏览器缓存不会变动的资源, 来加快请求效率, 由Saber自动化地完成且不必自己维护缓存逻辑(CURD或文件读写), 协程的调度使得其不论如何都不会阻塞服务器, Saber没有使用中间件机制因为它和Swoole是强相关的, 但是缓存可以使用 内存/文件/数据库 等多种方式, 所以虽然它尚未实现, 但它将会列入Saber的后续路线图中.

PSR风格

$bufferStream = new BufferStream();
$bufferStream->write(json_encode(['foo' => 'bar']));
$response = SaberGM::psr()
    ->withMethod('POST')
    ->withUri(new Uri('http://httpbin.org/post?foo=bar'))
    ->withQueryParams(['foo' => 'option is higher-level than uri'])
    ->withHeader('content-type', ContentType::JSON)
    ->withBody($bufferStream)
    ->exec()->recv();
echo $response->getBody();

WebSocket

可以通过websocketFrame数据帧的__toString方法直接打印返回数据字符串

$websocket = SaberGM::websocket('ws://127.0.0.1:9999');
while (true) {
    echo $websocket->recv(1) . "\n";
    $websocket->push("hello");
    co::sleep(1);
}

极限压力测试

测试机器为最低配MacBookPro, 请求服务器为本地echo服务器

0.9秒完成6666个请求, 成功率100%.

co::set(['max_coroutine' => 8191]);
go(function () {
    $requests = [];
    for ($i = 6666; $i--;) {
        $requests[] = ['uri' => 'http://127.0.0.1'];
    }
    $res = SaberGM::requests($requests);
    echo "use {$res->time}s\n";
    echo "success: $res->success_num, error: $res->error_num";
});
// on MacOS
// use 0.91531705856323s
// success: 6666, error: 0

列式请求集

在实际项目中, 经常会存在使用URL列表来配置请求的情况, 因此提供了list方法来方便使用:

echo SaberGM::list([
    'uri' => [
        'https://www.qq.com/',
        'https://www.baidu.com/',
        'https://www.swoole.com/',
        'http://httpbin.org/'
    ]
]);

单次并发控制

在实际爬虫项目中, 我们往往要限制单次并发请求数量以防被服务器防火墙屏蔽, 而一个max_co参数就可以轻松地解决这个问题, max_co会将请求根据上限量分批将请求压入队列并执行收包.

// max_co is the max number of concurrency request once, it's very useful to prevent server-waf limit.
$requests = array_fill(0, 10, ['uri' => 'https://www.qq.com/']);
echo SaberGM::requests($requests, ['max_co' => 5])->time."\n";
echo SaberGM::requests($requests, ['max_co' => 1])->time."\n";

高性能无极限协程连接池

在常驻内存的服务器中使用时, 一定要手动开启连接池选项:

$swoole = Saber::create([
    'base_uri' => 'https://www.swoole.com/',
    'use_pool' => true
]);

在通过该实例使用时, 就会启用连接池特性, 即底层与www.swoole.com网站的连接客户端将会用一个全局连接池存取, 避免了每次使用创建/连接的开销.

无限连接池

在参数为true时, 该网站的连接池容量是无限的, 一般情况下没有问题, 且无限容量的连接池性能更好.

定容连接池

但如果你使用其作为爬虫代理服务, 遭遇大量请求时, 连接池中的客户端数量就会不可控制地快速上升, 甚至超出你所请求的源网站的最大允许连接数, 这时候你就需要将use_pool设置为一个理想数值(int), 此时, 底层会使用Channel作为连接池, 在连接池创建的客户端超出数量且不够取用时, 挂起需要取用客户端的协程, 并等待正在使用客户端的协程归还客户端, 协程等待和切换几乎没有多大的性能消耗, 是一种非常先进的解决方式.

动态变容

需要注意的是, 连接池是绑定服务器IP+端口的, 即如果你有多个实例面向的是同一个服务器IP+端口, 他们之间使用的连接池也是同一个.

所以你在重复创建服务器IP+端口的实例时, 新创建的实例指定的use_pool是允许覆盖之前数值的, 即连接池底层是自动变容的, 容量增加时底层会重新创建新的连接池并转移客户端, 容量减少时也会销毁在连接池内的多余的客户端.

注意事项

注册你所希望的配置

除了一定要记得配备连接池以外, 异常处理的方式也需要注意是符合你的编程习惯的, Saber默认的异常处理是最主流且严谨的抛出异常, 但Saber也支持静默地使用错误码状态位, 可能更符合很多人的口味.

SaberGM::exceptionReport(0); // 关闭抛出异常报告, 在业务代码之前注册即可全局生效
$saber->exceptionReport(0);  //也可以单独设置某个实例

同理, 你所希望的配置都可以在业务代码之前如onWorkerStart甚至是swoole_server启动之前预先配置.

SaberGM::default([
    'exception_report' => 0
    'use_pool' => true
]);

像这样配置你所期望的选项可以让你获得更好的使用体验!

注意在一次性脚本中释放连接池

go(function(){
    // your code with pool...
    saber_pool_release(); // and this script will exit
});

如果你在一次性脚本中使用的连接池, 由于协程客户端是存在池中的, 引用计数为1无法释放, 就会导致swoole一直处于事件循环中, 脚本就无法退出, 你需要手动调用saber_pool_releasesaber_exitswoole_event_exit来正常退出, 也可以使用exit强制退出当前脚本(不要在server中使用exit).


配置参数表

|符号分割多种可选值

key type introduction example remark
protocol_version string HTTP协议版本 1.1 HTTP2还在规划中
base_uri string 基础路径 http://httpbin.org 将会与uri按照rfc3986合并
uri string 资源标识符 http://httpbin.org/get | /get | get 可以使用绝对路径和相对路径
uri_query string|array 请求信息 ['foo' => 'bar'] 非字符串会自动转换
method string 请求方法 get | post | head | patch | put | delete 底层自动转换为大写
headers array 请求报头 ['DNT' => '1'] | ['accept' => ['text/html'], ['application/xml']] 字段名不区分大小写, 但会保留设定时的原始大小写规则, 底层每个字段值会根据PSR-7自动分割为数组
cookies array|string ['foo '=> 'bar'] | 'foo=bar; foz=baz' 底层自动转化为Cookies对象, 并设置其domain为当前的uri, 具有浏览器级别的完备属性.
useragent string 用户代理 curl-1.0 默认为macos平台的chrome
referer string 来源地址 https://www.google.com 默认为空
redirect int 最大重定向次数 5 默认为3, 为0时不重定向.
keep_alive bool 是否保持连接 true | false 默认为true, 重定向时会自动复用连接
content_type string 发送的内容编码类型 text/plain | Swlib\Http\ContentType::JSON 默认为application/x-www-form-urlencoded
data array | string 发送的数据 'foo=bar&dog=cat' | ['foo' => 'bar'] 会根据content_type自动编码数据
before callable | array 请求前拦截器 function(Request $request){} 具体参考拦截器一节
after callable | array 响应后拦截器 function(Response $response){} 具体参考拦截器一节
before_redirect callable | array 重定向后拦截器 function(Request $request, Response $response){} 具体参考拦截器一节
timeout float 超时时间 0.5 默认5s, 支持毫秒级超时
bind_address string 绑定地址 192.168.1.1 或 eth0 默认不设置
bind_port int 绑定端口 80 默认不设置
proxy string 代理 http://127.0.0.1:1087 | socks5://127.0.0.1:1087 支持http和socks5
ssl int 是否开启ssl连接 0=关闭 1=开启 2=自动 默认自动
cafile string ca文件 __DIR__ . '/cacert.pem' 默认自带
ssl_verify_peer bool 验证服务器端证书 false | true 默认关闭
ssl_allow_self_signed bool 允许自签名证书 true | false 默认允许
ssl_cert_file string cert 证书 __DIR__ . '/ssl.cert' 默认不设置
ssl_key_file string key 私钥 __DIR__ . '/ssl.key' 默认不设置
iconv array 指定编码转换 ['gbk', 'utf-8'] 共三个参数为from,to,use_mb, 默认自动识别
exception_report int 异常报告级别 HttpExceptionMask::E_ALL 默认汇报所有异常
exception_handle callable|array 异常自定义处理函数 function(Exception $e){} 函数返回true时可忽略错误
retry callable 自动重试拦截器 function(Request $request, Response $response){} 位于发生错误后及重试之前
retry_time int 自动重试次数 默认不重试
use_pool bool|int 连接池 true false
pool_key callable|array 连接池的key function(Request $request):string { return $key; } 默认为请求地址的host:port

配置参数别名

为了使用方便与容错, 配置项的键值具有别名机制, 建议尽量使用本名:

key alias
method 0
uri 1 | url
data 2 | body
base_uri base_url
after callback
content_type content-type | contentType
cookies cookie
headers header
redirect follow
useragent ua | user-agent
exception_report error_report | report
before_retry retry
referer ref | referrer


拦截器

拦截器是Saber的一个非常强大的特性, 它可以让你非常方便地处理各种事情, 比如打印dev日志:

SaberGM::get('http://twosee.cn/', [
    'before' => function (Saber\Request $request) {
        $uri = $request->getUri();
        echo "log: request $uri now...\n";
    },
    'after' => function (Saber\Response $response) {
        if ($response->success) {
            echo "log: success!\n";
        } else {
            echo "log: failed\n";
        }
        echo "use {$response->time}s";
    }
]);
// log: request http://twosee.cn/ now...
// log: success!
// use 0.52036285400391s

甚至连异常自定义处理函数,会话都是通过拦截器来实现的.

拦截器可以有多个, 会依照注册顺序执行, 并且你可以为拦截器命名, 只需要使用数组包裹并指定key值, 如果你要删除这个拦截器, 给它覆盖一个null值即可.

[
    'after' => [
        'interceptor_new' => function(){},
        'interceptor_old' => null
    ]
]

拦截器可以使用四种方式注册(4种PHP回调函数):

callable: function(){}
string: 'function_name'
string: 'ClassName::method_name'
array: [$object, 'method_name']


Cookies

Cookie的实现是浏览器级别完备的, 它具体参考了Chrome浏览器的实现, 并遵循其相关规则.

属性

Cookies是一堆Cookie的集合, 而每个Cookie具有以下属性:

name, value, expires, path, session, secure, httponly, hostonly

任意格式互转

并且Cookies类支持多种格式互转, 如

  • foo=bar; foz=baz; apple=banana

  • Set-Cookie: logged_in=no; domain=.github.com; path=/; expires=Tue, 06 Apr 2038 00:00:00 -0000; secure; HttpOnly

  • ['foo'=>'bar', 'foz'=>'baz']

等格式转到Cookie类, 或是Cookie类到该几种格式的序列化.

域名路径和过期时限校验

Cookie也支持域名和时限校验, 不会丢失任何信息, 如domain是github.comcookie, 不会出现在help.github.com, 除非domain不是hostonly的(.github.com通配).

如果是session-cookie(没有过期时间,浏览器关闭则过期的), expires属性会设置为当前时间, 你可以通过拦截器来对其设置具体的时间.

持久化存储

通过读取Cookies的raw属性, 可以轻松地将其持久化到数据库中, 非常适合登录类爬虫应用.

更多详情具体请参考Swlib/Http库文档和例子.



异常机制

Saber遵循将业务与错误分离的守则, 当请求任意环节失败时, 默认都将会抛出异常.

强大的是, Saber的异常处理也是多样化的, 且和PHP的原生的异常处理一样完善.

异常的命名空间位于Swlib\Http\Exception

Exception Intro scene
RequestException 请求失败 请求配置错误
ConnectException 连接失败 如无网络连接, DNS查询失败, 超时等, errno的值等于Linux errno。可使用swoole_strerror将错误码转为错误信息。
TooManyRedirectsException 重定向次数超限 重定向的次数超过了设定的限制, 抛出的异常将会打印重定向追踪信息
ClientException 客户端异常 服务器返回了4xx错误码
ServerException 服务器异常 服务器返回了5xx错误码
BadResponseException 未知的获取响应失败 服务器无响应或返回了无法识别的错误码

除一般异常方法外, 所有HTTP异常类还拥有以下方法 :

Method Intro
getRequest 获取请求实例
hasResponse 是否获得响应
getResponse 获取响应实例
getResponseBodySummary 获取响应主体的摘要内容

捕获例子

try {
    echo SaberGM::get('http://httpbin.org/redirect/10');
} catch (TooManyRedirectsException $e) {
    var_dump($e->getCode());
    var_dump($e->getMessage());
    var_dump($e->hasResponse());
    echo $e->getRedirectsTrace();
}
// int(302)
// string(28) "Too many redirects occurred!"
// bool(true)
#0 http://httpbin.org/redirect/10
#1 http://httpbin.org/relative-redirect/9
#2 http://httpbin.org/relative-redirect/8

异常报告级别控制

同时, Saber亦支持以温和的方式来对待异常, 以免使用者陷入在不稳定的网络环境下, 必须在每一步都使用try包裹代码的恐慌中:

设定errorReport级别, 它是全局生效的, 对已创建的实例不会生效.

// 启用所有异常但忽略重定向次数过多异常
SaberGM::exceptionReport(
    HttpExceptionMask::E_ALL ^ HttpExceptionMask::E_REDIRECT
);

掩码表

下面的值(数值或者符号)用于建立一个二进制位掩码,来制定要报告的错误信息。可以使用按位运算符来组合这些值或者屏蔽某些类型的错误。标志位与掩码

Mask Value Intro
E_NONE 0 忽略所有异常
E_REQUEST 1 对应RequestException
E_CONNECT 2 对应RequestException
E_REDIRECT 4 对应RequestException
E_BAD_RESPONSE 8 对应BadRException
E_CLIENT 16 对应ClientException
E_SERVER 32 对应ServerException
E_ALL 63 所有异常

异常自定义处理函数

本函数可以用你自己定义的方式来处理HTTP请求中产生的错误, 可以更加随心所欲地定义你想要捕获/忽略的异常.

注意: 除非函数返回 TRUE (或其它真值),否则异常会继续抛出而不是被自定义函数捕获.

SaberGM::exceptionHandle(function (\Exception $e) {
    echo get_class($e) . " is caught!";
    return true;
});
SaberGM::get('http://httpbin.org/redirect/10');
//output: Swlib\Http\Exception\TooManyRedirectsException is caught!


Road Map

File Upload ✔ WebSocket ✔ AutoParser✔ AutoRetry✔ BigFile Download✔ Cache ClientPool Random UA
4 (High-priority) 3 2 1 .5 .5 .5 .175

Why not Http2 ?

As the main HTTP/2 benefit is that it allows multiplexing many requests within a single connection, thus [almost] removing the limit on number of simultaneous requests - and there is no such limit when talking to your own backends. Moreover, things may even become worse when using HTTP/2 to backends, due to single TCP connection being used instead of multiple ones, so Http2 Will not be a priority. (#ref)


IDE Helper

将本项目源文件加入到IDE的 Include Path 中.

(使用composer安装,则可以包含整个vendor文件夹, PHPStorm会自动包含)

良好的注释书写使得Saber完美支持IDE自动提示, 只要在对象后书写箭头符号即可查看所有对象方法名称, 名称都十分通俗易懂, 大量方法都遵循PSR规范或是参考Guzzle项目(感谢)而实现.

对于底层Swoole相关类的IDE提示则需要引入eaglewu的swoole-ide-helper(composer在dev环境下会默认安装), 但是该项目为手动维护, 不太完整, 也可以使用swoft-ide-helper或:

Swoole官方的ide-helper.



重中之重

欢迎提交issue和PR.



附录

Saber API

由于无法在魔术方法中使用协程(__call, __callStatic), 源码中的方法都是手动定义.

为了使用方便,已为所有支持的请求方法提供了别名。

Swlib\SaberGM

public static function psr(array $options = []): Swlib\Saber\Request
public static function wait(): Swlib\Saber
public static function request(array $options = [])
public static function get(string $uri, array $options = [])
public static function delete(string $uri, array $options = [])
public static function head(string $uri, array $options = [])
public static function options(string $uri, array $options = [])
public static function post(string $uri, $data = null, array $options = [])
public static function put(string $uri, $data = null, array $options = [])
public static function patch(string $uri, $data = null, array $options = [])
public static function download(string $uri, string $dir, int $offset, array $options = [])
public static function requests(array $requests, array $default_options = []): Swlib\Saber\ResponseMap
public static function list(array $options, array $default_options = []): Swlib\Saber\ResponseMap
public static function websocket(string $uri)
public static function default(?array $options = null): array
public static function exceptionReport(?int $level = null): int
public static function exceptionHandle(callable $handle): void

Swlib\Saber

public static function create(array $options = []): self
public static function session(array $options = []): self
public static function websocket(string $uri): WebSocket
public function request(array $options)
public function get(string $uri, array $options = [])
public function delete(string $uri, array $options = [])
public function head(string $uri, array $options = [])
public function options(string $uri, array $options = [])
public function post(string $uri, $data = null, array $options = [])
public function put(string $uri, $data = null, array $options = [])
public function patch(string $uri, $data = null, array $options = [])
public function download(string $uri, string $dir, int $offset, array $options = [])
public function requests(array $requests, array $default_options = []): ResponseMap
public function list(array $options, array $default_options = []): ResponseMap
public function upgrade(?string $path = null): WebSocket
public function psr(array $options = []): Request
public function wait(): self
public function exceptionReport(?int $level = null): int
public function exceptionHandle(callable $handle): void
public static function getAliasMap(): array
public function setOptions(array $options = [], ?Swlib\Saber\Request $request = null): self
public static function getDefaultOptions(): array
public static function setDefaultOptions(array $options = [])

Swlib\Saber\Request

public function getExceptionReport(): int
public function setExceptionReport(int $level): self
public function isWaiting(): bool
public function getPool()
public function withPool($bool_or_max_size): self
public function tryToRevertClientToPool(bool $connect_failed = false)
public function getSSL(): int
public function withSSL(int $mode = 2): self
public function getCAFile(): string
public function withCAFile(string $ca_file = __DIR__ . '/cacert.pem'): self
public function getSSLCertFile(): string
public function withSSLCertFile(string $cert_file): self
public function getSSLKeyFile(): string
public function withSSLKeyFile(string $key_file): self
public function withSSLVerifyPeer(bool $verify_peer = false, ?string $ssl_host_name = ''): self
public function withSSLAllowSelfSigned(bool $allow = true): self
public function getSSLConf()
public function getKeepAlive()
public function withKeepAlive(bool $enable): self
public function withBasicAuth(?string $username = null, ?string $password = null): self
public function withXHR(bool $enable = true)
public function getProxy(): array
public function withProxy(string $host, int $port): self
public function withSocks5(string $host, int $port, ?string $username, ?string $password): self
public function withoutProxy(): self
public function getBindAddress(): ?string
public function withBindAddress(string $address): self
public function getBindPort(): ?int
public function withBindPort(int $port): self
public function getTimeout(): float
public function withTimeout(float $timeout): self
public function getRedirect(): int
public function getName()
public function withName($name): self
public function withRedirect(int $time): self
public function isInQueue(): bool
public function withInQueue(bool $enable): self
public function getRetryTime(): int
public function withRetryTime(int $time): self
public function withAutoIconv(bool $enable): self
public function withExpectCharset(string $source = 'auto', string $target = 'utf-8', bool $use_mb = false): self
public function withDownloadDir(string $dir): self
public function withDownloadOffset(int $offset): self
public function resetClient($client)
public function exec()
public function recv()
public function getRequestTarget(): string
public function withRequestTarget($requestTarget): self
public function getMethod(): string
public function withMethod($method): self
public function getUri(): Psr\Http\Message\UriInterface
public function withUri(?Psr\Http\Message\UriInterface $uri, $preserveHost = false): self
public function getCookieParams(): array
public function getCookieParam(string $name): string
public function withCookieParam(string $name, ?string $value): self
public function withCookieParams(array $cookies): self
public function getQueryParam(string $name): string
public function getQueryParams(): array
public function withQueryParam(string $name, ?string $value): self
public function withQueryParams(array $query): self
public function getParsedBody(?string $name = null)
public function withParsedBody($data): self
public function getUploadedFile(string $name): Psr\Http\Message\UploadedFileInterface
public function getUploadedFiles(): array
public function withUploadedFile(string $name, ?Psr\Http\Message\UploadedFileInterface $uploadedFile): self
public function withoutUploadedFile(string $name): self
public function withUploadedFiles(array $uploadedFiles): self
public function __toString()
public function getProtocolVersion(): string
public function withProtocolVersion($version): self
public function hasHeader($name): bool
public function getHeader($name): array
public function getHeaderLine($name): string
public function getHeaders(bool $implode = false, bool $ucwords = false): array
public function getHeadersString(bool $ucwords = true): string
public function withHeader($raw_name, $value): self
public function withHeaders(array $headers): self
public function withAddedHeaders(array $headers): self
public function withAddedHeader($raw_name, $value): self
public function withoutHeader($name): self
public function getBody(): Psr\Http\Message\StreamInterface
public function withBody(?Psr\Http\Message\StreamInterface $body): self
public function getCookies()
public function setCookie(array $options): self
public function unsetCookie(string $name, string $path = '', string $domain = ''): self
public function withInterceptor(string $name, array $interceptor)
public function withAddedInterceptor(string $name, array $functions): self
public function withoutInterceptor(string $name): self
public function callInterceptor(string $name, $arguments)
public function getSpecialMark(string $name = 'default')
public function withSpecialMark($mark, string $name = 'default'): self

Swlib\Saber\Response

public function isSuccess(): bool
public function getUri(): Psr\Http\Message\UriInterface
public function getTime(): float
public function getRedirectHeaders(): array
public function getStatusCode()
public function withStatus($code, $reasonPhrase = '')
public function getReasonPhrase()
public function __toString()
public function getProtocolVersion(): string
public function withProtocolVersion($version): self
public function hasHeader($name): bool
public function getHeader($name): array
public function getHeaderLine($name): string
public function getHeaders(bool $implode = false, bool $ucwords = false): array
public function getHeadersString(bool $ucwords = true): string
public function withHeader($raw_name, $value): self
public function withHeaders(array $headers): self
public function withAddedHeaders(array $headers): self
public function withAddedHeader($raw_name, $value): self
public function withoutHeader($name): self
public function getBody(): Psr\Http\Message\StreamInterface
public function withBody(?Psr\Http\Message\StreamInterface $body): self
public function getCookies()
public function setCookie(array $options): self
public function unsetCookie(string $name, string $path = '', string $domain = ''): self
public function getSpecialMark(string $name = 'default')
public function withSpecialMark($mark, string $name = 'default'): self
public function getParsedJsonArray(bool $reParse = false): array
public function getParsedJsonObject(bool $reParse = false): object
public function getParsedQueryArray(bool $reParse = false): array
public function getParsedXmlArray(bool $reParse = false): array
public function getParsedXmlObject(bool $reParse = false): SimpleXMLElement
public function getParsedDomObject(bool $reParse = false): DOMDocument
public function getDataRegexMatch(string $regex, $group = null, int $fill_size)
public function getDataRegexMatches(string $regex, int $flag): array
public function isExistInData(string $needle, int $offset)

Swlib\Saber\RequestQueue

public function enqueue($request)
public function getMaxConcurrency(): int
public function withMaxConcurrency(int $num = -1): self
public function recv(): Swlib\Saber\ResponseMap
public function withInterceptor(string $name, array $interceptor)
public function withAddedInterceptor(string $name, array $functions): self
public function withoutInterceptor(string $name): self
public function callInterceptor(string $name, $arguments)

Swlib\Saber\ResponseMap

public $time = 0.0;
public $status_map = [];
public $success_map = [];
public $success_num = 0;
public $error_num = 0;
public function offsetSet($index, $response)
public function __toString()

Swlib\Saber\WebSocket

public function withMock(bool $ssl): self
public function recv(float $timeout = -1)
public function push(string $data, int $opcode = 1, bool $finish = true): bool
public function close(): bool

Swlib\Saber\WebSocketFrame

public $finish = true;
public $opcode = null;
public $data = null;
public function getOpcodeDefinition()
public function getOpcode()
public function getData()
public function __toString()

saber's People

Contributors

alebedev80 avatar amuluowin avatar fdreamsu avatar ihipop avatar ruesin avatar shenzhe avatar simplekalvin avatar sy-records avatar twose avatar wudi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

saber's Issues

并发请求 requests 的例子,真的是并发吗?从时间来看,太慢了呀。

以下 10 个请求 串发和并发的例子,为啥并发时间还是这么慢啊?

$responses = SaberGM::requests([
        ['uri' => 'https://www.qq.com'],
        ['uri' => 'https://www.baidu.com'],
        ['uri' => 'https://news.qq.com'],
        ['uri' => 'https://baike.baidu.com'],
        ['uri' => 'https://new.qq.com/ch/finance/'],
        ['uri' => 'https://zhidao.baidu.com'],
        ['uri' => 'https://new.qq.com/ch/auto/'],
        ['uri' => 'https://pan.baidu.com'],
        ['uri' => 'https://sports.qq.com'],
        ['uri' => 'https://www.hao123.com']
    ]);
    $t1 = microtime(true);
    SaberGM::get('https://www.qq.com');
    SaberGM::get('https://www.baidu.com');
    SaberGM::get('https://news.qq.com');
    SaberGM::get('https://baike.baidu.com');
    SaberGM::get('https://new.qq.com/ch/finance/');
    SaberGM::get('https://zhidao.baidu.com');
    SaberGM::get('https://new.qq.com/ch/auto/');
    SaberGM::get('https://pan.baidu.com');
    SaberGM::get('https://sports.qq.com');
    SaberGM::get('https://www.hao123.com');
    $t2 = microtime(true);
    $runt = $t2-$t1;
    $result = "multi-requests [ {$responses->success_num} ok, {$responses->error_num} error ]:\n" ."consuming-time: {$responses->time}s\nrunt: $runt";
    // multi-requests [ 10 ok, 0 error ]:
    // consuming-time: 0.95005702972412s
    // runt: 1.6822638511658

PSR Request的exec()方法是否应该实现在saber上比较妥当?

PSR的RequestInterface没有规定实现exec方法,所以我设计组装一个http客户端无关的request的时候,我肯定不能绑定和客户端强相关的exec方法到Request上,因为每个客户端的异常类型、处理逻辑都不相同。

我设计一个composer组件,在组装请求部分,返回了个psr对象,本意是guzzle或者saber等支持PSR标准的HTTP客户端都可以按psr标准把这个对象代表的请求发送出去,现在Guzzle可以做到($guzzleClient->send($PSRrequest))而saber因为把PSR相关的处理逻辑绑定到他自定义的Request上,导致这样的设计没法实施。

似乎retry=3次无用???

        $responses = Swlib\SaberGM::requests($urls, [
            'exception_report' => 0,
            'retry_time' => 3,
            'retry' => function (Swlib\Saber\Request $request) {
                $uri = $request->getUri();
                echo date('Y-m-d H:i:s') . " DNS hijack check {$uri} re-requesting...\n";
            },
        ]);

[root@localhost urlcheckerclient]# php check_dns.php
2018-11-28 06:01:15 DNS hijack check http://www.mantoub.com/ re-requesting...
2018-11-28 06:01:15 DNS hijack check http://www.mantoub.com/ re-requesting...
2018-11-28 06:01:18 DNS hijack check http://www.mantoub.com/ re-requesting...
PHP Warning: DOMDocument::loadHTML(): Empty string supplied as input in /vagrant_data/urlcheckerclient/vendor/swlib/util/src/DataParser.php on line 160
2018-11-28 06:01:24 DNS hijack check http://www.mantoub.com/ re-requesting...
2018-11-28 06:01:34 DNS hijack check http://www.mantoub.com/ re-requesting...
2018-11-28 06:01:40 DNS hijack check http://www.mantoub.com/ re-requesting...
PHP Warning: DOMDocument::loadHTML(): Empty string supplied as input in /vagrant_data/urlcheckerclient/vendor/swlib/util/src/DataParser.php on line 160

按照 redeme 中的连接 WebSocket 报错

(base) MBP:blive chen$ php artisan test
[2019-07-03 19:49:48 @20747.0]	ERROR	(PHP Fatal Error: 10001):
Swoole\Coroutine\Http\Client::upgrade: API must be called in the coroutine
Stack trace:
#0  Swoole\Coroutine\Http\Client->upgrade() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/WebSocket.php:40]
#1  Swlib\Saber\WebSocket->__construct() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/Saber.php:308]
#2  Swlib\Saber->upgrade() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/Saber.php:134]
#3  Swlib\Saber::websocket() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/SaberGM.php:101]
#4  Swlib\SaberGM::websocket() called at [/Users/chen/wwwroot/blive/app/Console/Commands/test.php:43]
#5  App\Console\Commands\test->handle()
#6  call_user_func_array() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:32]
#7  Illuminate\Container\BoundMethod::Illuminate\Container\{closure}() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:90]
#8  Illuminate\Container\BoundMethod::callBoundMethod() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:34]
#9  Illuminate\Container\BoundMethod::call() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/Container.php:576]
#10 Illuminate\Container\Container->call() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Console/Command.php:183]
#11 Illuminate\Console\Command->execute() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Command/Command.php:255]
#12 Symfony\Component\Console\Command\Command->run() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Console/Command.php:170]
#13 Illuminate\Console\Command->run() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Application.php:921]
#14 Symfony\Component\Console\Application->doRunCommand() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Application.php:273]
#15 Symfony\Component\Console\Application->doRun() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Application.php:149]
#16 Symfony\Component\Console\Application->run() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Console/Application.php:90]
#17 Illuminate\Console\Application->run() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php:133]
#18 Illuminate\Foundation\Console\Kernel->handle() called at [/Users/chen/wwwroot/blive/artisan:37]
(base) MBP:blive chen$ 

use_pool为数字时报错

测试脚本:

go(function () {
    $pool = Saber::create([
        'base_uri' => 'http://www.qq.com',
        'use_pool' => 10
    ]);
    var_dump($pool->get('/')->success);
});

错误信息:

Fatal error: Uncaught InvalidArgumentException: Host should not be empty! in /web/wwwroot/test/vendor/swlib/saber/src/Request.php:118
Stack trace:
#0 /web/wwwroot/test/vendor/swlib/saber/src/Request.php(152): Swlib\Saber\Request->getConnectionTarget()
#1 /web/wwwroot/test/vendor/swlib/saber/src/Saber.php(446): Swlib\Saber\Request->withPool(10)
#2 /web/wwwroot/test/vendor/swlib/saber/src/Saber.php(378): Swlib\Saber::transOptionsToRequest(Array, Object(Swlib\Saber\Request))
#3 /web/wwwroot/test/vendor/swlib/saber/src/Saber.php(145): Swlib\Saber->setOptions(Array, Object(Swlib\Saber\Request))
#4 /web/wwwroot/test/vendor/swlib/saber/src/Saber.php(108): Swlib\Saber->__construct(Array)
#5 /web/wwwroot/test/test.php(14): Swlib\Saber::create(Array)
#6 {main}
  thrown in /web/wwwroot/test/vendor/swlib/saber/src/Request.php on line 118

use_pool不为数字时则正常,简单看了一下是因为Request.php里面处理不一样:

        if (is_numeric($this->use_pool)) {
            // limit max num
            ClientPool::getInstance()->setMaxEx($this->getConnectionTarget(), $bool_or_max_size);
        }

拦截器回调写法tips

如果使用这种写法:array: [$object, 'method_name'], 在配置拦截器时应该这样:'before' => [[$object, 'method_name']]

添加Client的setting自定义配置吧

目前在用saber做clickhouse的包,由于ck的并发能力较低,适合大批量数据一次写入,默认的socket_buffer_size不够用了,或者在swoole底层可以通过co::set设置全局的配置,然后单个客户端的配置可以覆盖全局配置?

Does Websocket client support wss:// (Websocket Secure) ?

Web socket Demo https://html5demos.com/web-socket/

\Swoole\Runtime::enableCoroutine(true);
go(function(){
    $websocket = \Swlib\SaberGM::websocket('wss://remy-ws.glitch.me:443');
    while (true) {
        echo $websocket->recv(-1) . "\n";
        $websocket->push("hello");
        co::sleep(1);
    }
});
$ php src/client2.php 
PHP Warning:  Swoole\Coroutine\Http\Client::recv(): you should not use recv without defer. in /home/ihipop/Dev/workspace/***/vendor/swlib/saber/src/WebSocket.php on line 65

PHP Warning:  Swoole\Coroutine\Http\Client::push(): websocket handshake failed, cannot push data. in /home/ihipop/Dev/workspace/***/vendor/swlib/saber/src/WebSocket.php on line 72

Q1:

Does the coroutine http client of swoole support wss:// (Websocket Secure)?

Q2:

Should $websocke->client->setDefer(); be call automaticly at \Swlib\Saber\WebSocket::__construct

设置Host头无效

测试代码:

<?php
require('vendor/autoload.php');

go(function() {
	$saber = Swlib\Saber::create([
		'base_uri' => 'http://127.0.0.1',
		'headers' => [
			'Host' => 'example.com',
			'X-Test' => 'test'
		]
	]);

	echo $saber->get('/host.php');
	
	echo "\n-------------------\n";
	
	$saber = Swlib\Saber::create([
		'base_uri' => 'http://127.0.0.1'
	]);
	$saber->setOptions([
		'headers' => [
			'Host' => 'example.com',
			'X-Test' => 'test'
		]
	]);

	echo $saber->get('/host.php');
});

服务器输出内容是:(打印了$_SERVER):

HTTP/1.1 200 OK
Server: nginx
Date: Wed, 03 Oct 2018 02:57:14 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Content-Encoding: gzip

Array
(
    [TEMP] => /tmp
    [TMPDIR] => /tmp
    [TMP] => /tmp
    [PATH] => /usr/local/bin:/usr/bin:/bin
    [USER] => www
    [HOME] => /home/www
    [HTTP_X_TEST] => test
    [HTTP_CONTENT_TYPE] => application/x-www-form-urlencoded
    [HTTP_USER_AGENT] => Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
    [HTTP_ACCEPT_ENCODING] => gzip
    [HTTP_ACCEPT] => text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
    [HTTP_HOST] => 127.0.0.1
    [HTTP_CONNECTION] => keep-alive
    [REDIRECT_STATUS] => 200
    [SERVER_NAME] => _
    [SERVER_PORT] => 80
    [SERVER_ADDR] => 127.0.0.1
    [REMOTE_PORT] => 46126
    [REMOTE_ADDR] => 127.0.0.1
    [SERVER_SOFTWARE] => nginx/1.12.2
    [GATEWAY_INTERFACE] => CGI/1.1
    [REQUEST_SCHEME] => http
    [SERVER_PROTOCOL] => HTTP/1.1
    [DOCUMENT_ROOT] => /web/wwwroot/default
    [DOCUMENT_URI] => /host.php
    [REQUEST_URI] => /host.php
    [SCRIPT_NAME] => /host.php
    [CONTENT_LENGTH] =>
    [CONTENT_TYPE] => application/x-www-form-urlencoded
    [REQUEST_METHOD] => GET
    [QUERY_STRING] =>
    [SCRIPT_FILENAME] => /web/wwwroot/default/host.php
    [FCGI_ROLE] => RESPONDER
    [PHP_SELF] => /host.php
    [REQUEST_TIME_FLOAT] => 1538535434.4496
    [REQUEST_TIME] => 1538535434
)

-------------------
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 03 Oct 2018 02:57:14 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Content-Encoding: gzip

Array
(
    [TEMP] => /tmp
    [TMPDIR] => /tmp
    [TMP] => /tmp
    [PATH] => /usr/local/bin:/usr/bin:/bin
    [USER] => www
    [HOME] => /home/www
    [HTTP_X_TEST] => test
    [HTTP_CONTENT_TYPE] => application/x-www-form-urlencoded
    [HTTP_USER_AGENT] => Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
    [HTTP_ACCEPT_ENCODING] => gzip
    [HTTP_ACCEPT] => text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
    [HTTP_HOST] => 127.0.0.1
    [HTTP_CONNECTION] => keep-alive
    [REDIRECT_STATUS] => 200
    [SERVER_NAME] => _
    [SERVER_PORT] => 80
    [SERVER_ADDR] => 127.0.0.1
    [REMOTE_PORT] => 46128
    [REMOTE_ADDR] => 127.0.0.1
    [SERVER_SOFTWARE] => nginx/1.12.2
    [GATEWAY_INTERFACE] => CGI/1.1
    [REQUEST_SCHEME] => http
    [SERVER_PROTOCOL] => HTTP/1.1
    [DOCUMENT_ROOT] => /web/wwwroot/default
    [DOCUMENT_URI] => /host.php
    [REQUEST_URI] => /host.php
    [SCRIPT_NAME] => /host.php
    [CONTENT_LENGTH] =>
    [CONTENT_TYPE] => application/x-www-form-urlencoded
    [REQUEST_METHOD] => GET
    [QUERY_STRING] =>
    [SCRIPT_FILENAME] => /web/wwwroot/default/host.php
    [FCGI_ROLE] => RESPONDER
    [PHP_SELF] => /host.php
    [REQUEST_TIME_FLOAT] => 1538535434.4521
    [REQUEST_TIME] => 1538535434
)

可以看到,X-Test头是成功了的,Host头修改无效

Swoole\Buffer is deprecated

swoole 4.4.0 版本,使用saber报错Class Swoole\Buffer is deprecated, it will be removed in v4.5.0。
file:vendor/swlib/http/src/SwooleBuffer.php line 19
请问适配4.4以上版本的saber什么时候会有呢

发现bug

/src/Request.php633行,应为-3,项目中写成了3,部分代码:

    public function recv()
    {
        retry_recv:
        if (self::STATUS_WAITING !== $this->_status) {
            throw new \BadMethodCallException('You can\'t recv because client is not in waiting stat.');
        }
        $this->client->recv($this->getTimeout());
        $this->_status = self::STATUS_NONE;
        $this->_time = microtime(true) - $this->_start_time;

        $is_report = $this->getExceptionReport() & HttpExceptionMask::E_CONNECT;
        $statusCode = $this->client->statusCode;
        $errCode = $this->client->errCode;
        if ($statusCode < 0 || $errCode !== 0) {
            if ($is_report) {
                if ($statusCode === -1) {
                    $message = 'Connect timeout! the server is not listening on the port or the network is missing!';
                } elseif ($statusCode === -2) {
                    $timeout = $this->getTimeout();
                    $message = "Request timeout! the server hasn't responded over the timeout setting({$timeout}s)!";
                } elseif ($statusCode === 3) { //这里写错啦!
                    $message = 'Connection is forcibly cut off by the remote server';
                } else {
                    $message = "Linux Code {$errCode}: " . swoole_strerror($errCode);
                }
                $exception = new ConnectException($this, $statusCode, $message);
                $ret = $this->callInterceptor('exception', $exception);
                if (!$ret) {
                    $this->tryToRevertClientToPool(true);
                    throw $exception;
                }
            } else {
                // Exception is no longer triggered after an exception is ignored
                $this->setExceptionReport(HttpExceptionMask::E_NONE);
            }
        }

如何添加握手的参数呢?

require_once './vendor/autoload.php';
go(function () {
    $websocket = \Swlib\SaberGM::websocket('127.0.0.1:9093'); //这里想给后面加上token和uid咋加呢?
 
});

我在一个请求中发生了无法返回并阻塞的问题.

代码是这样的

 $chan = new \Swoole\Coroutine\Channel(1);

 go(function () use($chan) {
     $uri     = 'http://10.40.32.1:8800';
     $content = \Swlib\SaberGM::post($uri);
     $chan->push($content);
 });

 $res = [];
 $chanNum = 1;
 while ($chanNum > 0) {
     $item = $chan->pop();
     $res[] = $item;
     $chanNum--;
 }
 print_r($res);

但是我通过断点,看到阻塞发生在了
swlib/saber/src/Request.php $this->client->recv($this->getTimeout()); 这一行代码.

我通过tcpdump 抓包发现

17:34:06.867207 IP 4f86ed14dc50.40110 > bogon.8800: Flags [S], seq 623124665, win 29200, options [mss 1460,sackOK,TS val 2069769 ecr 0,nop,wscale 7], length 0
17:34:06.868292 IP bogon.8800 > 4f86ed14dc50.40110: Flags [S.], seq 452901758, ack 623124666, win 65535, options [mss 1460,wscale 2,eol], length 0
17:34:06.868325 IP 4f86ed14dc50.40110 > bogon.8800: Flags [.], ack 1, win 229, length 0
17:34:06.869648 IP 4f86ed14dc50.40110 > bogon.8800: Flags [F.], seq 1, ack 1, win 229, length 0
17:34:06.869798 IP bogon.8800 > 4f86ed14dc50.40110: Flags [.], ack 2, win 65535, length 0
17:34:06.872652 IP bogon.8800 > 4f86ed14dc50.40110: Flags [F.], seq 1, ack 2, win 65535, length 0
17:34:06.872667 IP 4f86ed14dc50.40110 > bogon.8800: Flags [.], ack 2, win 229, length 0

在tcp建立连接之后,客户端主动断开连接FIN. 并没有把链接交到应用层http.服务器web server都没有访问日志,这是什么原因呢? 难道是swoole底层问题?

HTTP短连接连接池?

请求在swoole作为网关分发请求到业务服务的场景下,如何使用saber来避免高并发场景下出现大量TIME_WAIT的现象?

连接池的方式我不是很理解,如果被请求方的webserver不支持HTTP KeepAlive的话,依然是短连接,那连接池里面的实例再每次发起新的请求时不是还要重新建立TCP吗?

1

HTTP代理是时候怎么使用密码

stringToJsonArray不支持解析被urlencode后的数据

swlib\util\src\DataParser.php 第94行

 public static function stringToJsonArray(string $var): array
    {
        //无法解析被urlencode后的数据
        var_dump($var);//%7b%22jlunit%22%3a%22%22%2c%22jsunit%22%3a%22%22%2c%22sgunit%22%3a%22%22%2c%22serialno%22%3a%2244b16204daaf41c3bed192681d70e5e5%22%2c%22errorcode%22%3a%220%22%2c%22errormessage%22%3a%22%22%7d
        return ($var = json_decode($var, true)) === false ? [] : $var;
    }

大神,怎么带着cookie去访问?

[
'before' => function (Swlib\Saber\Request $request) {
echo "\n=====================设置Cookies====================\n";
},
'after' => function (Swlib\Saber\Response $response) {
echo "\n=====================获取Cookies====================\n";
},
]

我的想法是在以上拦截器里写,但不知如何写这块的代码。看文档说要去看Saber\Http,但没找到相关代码和文档

大神代码出现 Segmentation fault 不知如何定位

go(function () use ($host, $port, $poolSize, $queueKey) {
    $pool = new DotCloud\RedisPool($host, $port, $poolSize);
    while (true) {
        $redis = $pool->get();
        if ($redis->lLen($queueKey) > 0) {
            $data = $redis->rPop($queueKey);
            if (empty($data)) {
                $pool->put($redis);
                continue;
            }
            // ......
            // redis 里取出数据[ ['uri'=> ''], ['uri'=> ''] ],使用requests并发请求
            $responses = Swlib\SaberGM::requests($urls);
            // ......
        }
        $pool->put($redis);
    }
});
# php --ri swoole

swoole

swoole support => enabled
Version => 4.2.9
Author => Swoole Group[email: [email protected]]
coroutine => enabled
debug => enabled
trace_log => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.0.2k-fips  26 Jan 2017
http2 => 1.31.1
pcre => enabled
zlib => enabled
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.aio_thread_num => 2 => 2
swoole.display_errors => On => On
swoole.use_namespace => On => On
swoole.use_shortname => On => On
swoole.fast_serialize => Off => Off
swoole.unixsock_buffer_size => 8388608 => 8388608
# cat /var/log/messages | grep swoole
Dec  3 11:06:01 izwz99ilp8qh025s8ak1y9z kernel: php[12589]: segfault at 8 ip 00007fdd0544409b sp 00007fdd04f9fed0 error 4 in swoole.so[7fdd053c9000+132000]
Dec  3 13:40:32 izwz99ilp8qh025s8ak1y9z kernel: php[16214]: segfault at 8 ip 00007fd42a84409b sp 00007fd42a39fed0 error 4 in swoole.so[7fd42a7c9000+132000]

# uname -a
Linux izwz99ilp8qh025s8ak1y9z 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

# php -v
PHP 7.1.24 (cli) (built: Nov  7 2018 18:08:20) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.1.24, Copyright (c) 1999-2018, by Zend Technologies

# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC)

Websocket upgrade failed by [Success].

带了get参数去升级为websocket失败, 使用js来连接是成功的

$websocket = \Swlib\SaberGM::websocket('ws://127.0.0.1:2023/CloseOrderSercive');

        while (true) {
            echo $websocket->recv(1) . "\n";
            $websocket->push("hello");
            co::sleep(1);
        }

长连接服务端主动断开连接的时候,客户端没有检测keep_liveness然后继续去请求了

PHP Fatal error:  Uncaught Swlib\Http\Exception\ConnectException: Connection is forcibly cut off by the remote server in /home/ihipop/Dev/workspace/composer/saber/src/Request.php:641
Stack trace:
#0 /home/ihipop/Dev/workspace/composer/taobao-top/src/client/Adapter/SaberAdapter.php(39): Swlib\Saber\Request->recv()
#1 {main}
  thrown in /home/ihipop/Dev/workspace/composer/saber/src/Request.php on line 641

v1.0.6
use pool=>true

重现代码(平均250次请求左右可以重现):

<?php
\Swoole\Runtime::enableCoroutine();
require __DIR__ . '/vendor/autoload.php';
go(function () {
    $saber = Swlib\Saber::create([
        'use_pool'   => true,
        'keep_alive' => true,
    ]);
    $i     = 1;
    while (true) {
       try{
           //        $result = $saber->psr()->withMethod('GET')->withUri(new \Swlib\Http\Uri('http://gw.api.taobao.com/router/rest'))->exec()->recv();
           $result = $saber->get('http://gw.api.taobao.com/router/rest');

           echo $result->statusCode .'=>'.$i. "\n";
       }catch (\Throwable $e){
           echo $e->__toString();
           break;
       }finally{
           $i++;
       }

    }
    echo $i;
});

如下的写法不是协程请求。。。。求指点

$this->saber = Saber::create([
            'base_uri' => $this->baseurl,
            'use_pool' => true
        ]);
// 调用部分
// 循环从队列中取连接 请求 我的理解 这必然是协程的  使用了 连接池不是这么用的吗?
            while ($this->pageQueue->count()) {
                $res = $this->saber->get($this->pageQueue->dequeue());
            }

Issue when trying to connect to https address?

I keep getting this error on the latest version of this library (with latest swoole)

$res = $saber->get(
                    $uri,
                    [
                        'timeout' => 60,
                        'redirect' => false
                    ]
                );

HTTP -1 Unknown: Connect timeout! the server is not listening on the port or the network is missing!

SaberGM::requests -> You can't enqueue a waiting request when using the max concurrency control

require __DIR__ . '/../vendor/autoload.php';
go(function () {
	$tidList = [
		'4824528997',
		'5309603784',
		'6042765344',
	];
	foreach ($tidList as $tid) {
		$requests[] = [
			'uri' => 'https://tieba.baidu.com/p/' . $tid . '?fid=52',
		];
	}
	$res = \Swlib\SaberGM::requests($requests, [
		'max_co' => 3,
	]);
});

RT……最小化代码……

报错是这个:
RequestQueue.php#L28
Stack trace:

#0 /mnt/hgfs/144/wwwroot/dev_resources/vendor/swlib/saber/src/RequestQueue.php(118): Swlib\Saber\RequestQueue->enqueue(Object(Swlib\Saber\Request))
#1 /mnt/hgfs/144/wwwroot/dev_resources/vendor/swlib/saber/src/Saber.php(278): Swlib\Saber\RequestQueue->recv()
#2 /mnt/hgfs/144/wwwroot/dev_resources/vendor/swlib/saber/src/SaberGM.php(90): Swlib\Saber->requests(Array, Array)
#3 /mnt/hgfs/144/wwwroot/dev_resources/php/zzz_swooleCurl.php(23): Swlib\SaberGM::requests(Array, Array)
#4 {main}
  thrown in /mnt/hgfs/144/wwwroot/dev_resources/vendor/swlib/saber/src/RequestQueue.php on line 28

如果 $tidList 只有两个, max_co 参数无论如何设定都不会产生错误
如果 $tidList 有三个,而且 max_co 大于等于3,就会产生这个报错……

目前还没有仔细去查怎么解决这个问题……不过这个似乎……可以算个bug吧………………

环境
ubuntu 16.04
php 7.1.7
swoole 4.2.13
swlib/saber v1.0.6

跑一短时间后,会出现异常

Fatal error: Uncaught Swlib\Http\Exception\ConnectException: Request timeout! the server hasn't responded over the timeout setting(5s)! in /var/www/html/swoole/vendor/swlib/saber/src/Request.php:641

使用代码

$http = new swoole_http_server("0.0.0.0", 9510);

$http->on("start", function ($server) use ($urls) {
    echo "Swoole http server is started at http://127.0.0.1:9501\n";

    
    for ($i = 0; $i < count($urls); $i++) {
        $url = $urls[$i]['url'];
        Swoole\Timer::tick(1000, function () use ($url) {
            go(function () use($url) {
                echo SaberGM::get($url, [
                ])->getBody();
            });
        });
    }
});

Swoole\Buffer::__construct(): handle 10060447 exceed 10000000 in /vendor/swlib/http/src/SwooleBuffer.php on line 19

请求了次数多了以后就出现这样的问题,已经开启连接池

文档上没说明到底多大的限制 https://wiki.swoole.com/wiki/page/246.html

hp --ri swoole

swoole

Swoole => enabled
Author => Swoole Team <[email protected]>
Version => 4.3.1
Built => Apr 12 2019 18:22:24
coroutine => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.1.0h  27 Mar 2018
http2 => enabled
pcre => enabled
zlib => enabled
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled
coroutine_postgresql => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608

host过滤问题

在docker-compose或者k8s等有内置dns的情况下,host的域名一般不会写".xxx",而是直接用服务名,这时候过滤器会报错。

echo $res->body 是乱码,什么情况?

Swlib\Http\SwooleBuffer 转字符串,就是乱码

如果响应的字节比较少,就不是乱码,比如get('http://myip.ipip.net/')
如果响应的字节多,就是乱码,比如get('http://www.myip.cn/')

不知道是不是 swoole 的问题,文档里也有人反映 https://wiki.swoole.com/wiki/page/582.html

环境
CentOS 7.3 + docker(alpine 3.8)
php 7.1.23,装了zlib
swoole 4.3.1
swlib/saber v1.0.6

731724ac8bad:~# php --ri swoole

swoole

Swoole => enabled
Author => Swoole Team [email protected]
Version => 4.3.1
Built => Mar 20 2019 12:52:44
coroutine => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => LibreSSL 2.7.4
http2 => enabled
mutex_timedlock => enabled
pthread_barrier => enabled
mysqlnd => enabled
async_redis => enabled
coroutine_postgresql => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608
731724ac8bad:~#

先post再get,竟然还保持post时的请求主体数据

先post再get,竟然还保持post时的请求主体数据,建议get默认情况下不带任何请求主体数据。

另外,getParsedJsonObject(bool $reParse = false): object 在php7.1是不支持的。建议判断版本<=7.1时定义成getParsedJsonObject(bool $reParse = false): \stdClass

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.