cloudwego / kitex Goto Github PK
View Code? Open in Web Editor NEWGo RPC framework with high-performance and strong-extensibility for building micro-services.
Home Page: https://www.cloudwego.io
License: Apache License 2.0
Go RPC framework with high-performance and strong-extensibility for building micro-services.
Home Page: https://www.cloudwego.io
License: Apache License 2.0
github.com/cloudwego/thriftgo/parser go/src/github.com/cloudwego/thriftgo/parser/AST.go:290:35: not enough arguments in call to iprot.ReadStructBegin have () want (context.Context) go/src/github.com/cloudwego/thriftgo/parser/AST.go:295:54: not enough arguments in call to iprot.ReadFieldBegin have () want (context.Context) go/src/github.com/cloudwego/thriftgo/parser/AST.go:310:24: not enough arguments in call to iprot.Skip have (thrift.TType) want (context.Context, thrift.TType) go/src/github.com/cloudwego/thriftgo/parser/AST.go:330:30: not enough arguments in call to iprot.ReadFieldEnd have () want (context.Context) go/src/github.com/cloudwego/thriftgo/parser/AST.go:334:30: not enough arguments in call to iprot.ReadStructEnd have () want (context.Context) go/src/github.com/cloudwego/thriftgo/parser/AST.go:355:31: not enough arguments in call to iprot.ReadString have () want (context.Context) go/src/github.com/cloudwego/thriftgo/parser/AST.go:364:28: not enough arguments in call to iprot.ReadI32 have () want (context.Context) go/src/github.com/cloudwego/thriftgo/parser/AST.go:374:33: not enough arguments in call to oprot.WriteStructBegin have (string) want (context.Context, string) go/src/github.com/cloudwego/thriftgo/parser/AST.go:388:31: not enough arguments in call to oprot.WriteFieldStop have () want (context.Context) go/src/github.com/cloudwego/thriftgo/parser/AST.go:391:31: not enough arguments in call to oprot.WriteStructEnd have () want (context.Context) go/src/github.com/cloudwego/thriftgo/parser/AST.go:391:31: too many errors
you can build kitex by hand.
pull the source code from main branch
go to the dictionary:kitex/tool/cmd/kitex
exec go build
you will find a binary execable file,which named kitex.
if you want use this command everywhere ,you can add this to PATH
Now,The service diccovery depend on kubernets。Are there plans to support other serivce discovery?For Example,ZooKeeper or Nacos。
下面这块代码有问题
func (handler) BidiSideStreaming(stream echo.EchoService_BidiSideStreamingServer) (err error) {
go func() {
for {
req, err := stream.Recv()
if err != nil {
return err
}
}
}()
for {
resp := &echo.Response{Msg: "world"}
if err := stream.Send(resp); err != nil {
return err
}
}
}
在server启动时候,定义errHandler未生效,无法正常输出123431
server.WithErrorHandler(func(err error) error {
fmt.Println(123431)
return err
})
method defaultCodec.Decode
could occupy ~10% cpu usage in high concurrency services.
PR #122 try to improve codec.ttHeader.decode
by reducing memory copy times and reusing small map objects.
非常感谢字节跳动开源Kitex微服务框架。
我们团队有在使用 Kitex 做微服务开发来构建程序业务。最近在知乎上也看到一个关于版权争议的问题,所以在这里请教下:
Kitex 仓库是基于 Apache 2.0 开源协议的,那放在当前仓库下的文档教程(docs目录下),是否也算是遵循 Apache 2.0 协议?比如我对文档教程做了转载等,转载有注明出处来源,这种行为算不算侵权?
We mainly focus on frame synchronization in online games and real-time battle scenarios which has high requirements for latency. After looking at the test data on the Internet, the performance of kcp is better than that of quic in weak network scenarios. I looked at grpc and rpcx. The mode based on a single long connection can support kcp, but kitex cannot.
refer to : https://github.com/xtaci/kcp-go
kitex generate proto failure
Here's the command
kitex -type protobuf -service helloworldservice -module xxx helloword.proto
--kitex_out: render *service.go failed: template: *service.go:71:26: executing "*service.go" at <$arg.Type>: can't evaluate field Type in type string
如题,netpollmux的server_handler中的rpcinfo用了对象池,然后在finishTracer函数中,会将rpcStats清零,这样后继的请求如果拿到这个rpcInfo,其statsLevel会是0,即LevelDisabled,导致不会被trace
kitex/pkg/endpoint/endpoint_test.go
Line 56 in 0c25876
I see the code & document has psm
everywhere.
What does that mean?
Just like gRPC http gateway, because some language such Java or PHP also want to join...
In documentation 'kitex/docs/guide/getting_started_cn.md'
It tells we could clone the repository https://github.com/cloudwego/examples.git
to quickly get started.
But this repository is not existed.
Hi, I use protobuf as IDL, here's the protobuf content:
syntax = "proto3";
package helloworld;
option go_package="/helloworld";
service HelloService {
rpc Hello(HelloRequest) returns(HelloResponse);
}
message HelloRequest {
}
message HelloResponse {
}
then generate the serverside code: kitex -type protobuf -service helloworldservice -module xxxxxx hello.proto
the generated project structure is:
.
├── build.sh
├── conf
│ └── kitex.yml
├── go.mod
├── go.sum
├── handler.go
├── hello.proto
├── kitex_gen
│ ├── github
│ │ └── com
│ │ └── helloworld
│ │ └── helloservice
│ │ ├── client.go
│ │ ├── helloservice.go
│ │ ├── invoker.go
│ │ └── server.go
│ └── github.com
│ └── helloworld
│ └── hello.pb.go
├── main.go
├── output
│ ├── bin
│ ├── bootstrap.sh
│ └── conf
│ └── kitex.yml
└── script
└── bootstrap.sh
12 directories, 15 files
run go mod tidy && go build -v
, it reports the error:
main.go:5:2: package xxxxxx/kitex_gen/github.com/helloworld/helloservice is not in GOROOT (/usr/local/go/src/xxxxxx/kitex_gen/github.com/helloworld/helloservice)
we could see the generated folder kitex_gen/github/com/helloworld
should be put under folder kitex_gen/github.com/helloworld
, after fixing this error, the project can be built sucess.
In the comment of registry.Info, it is said that
// ServiceName will be set in kitex by default
but it turns out that this field is not set
And the reason is that in server.go, s.opt.Svr.ServiceName is not set too.
Is this is expected?
Thanks.
We hope more developers join CloudWeGo. This is a task issue that needs your help.
Issue Description
Support more extensions of service Registration and Discovery. You can choose ZooKeeper, Consul, etcd...
Code Repository
Please create a new repo in https://github.com/kitex-contrib, the repo name is suggested registry-xxx
, eg registry-zookeeper
.
Extension API
Picked
readme 汉语好评!
文档中写明支持thrift和protobuf,我试了proto文件,提示:
Expect the IDL file to have an extension of '.thrift'. Got 'string.proto'.
目前 server 通过生成 invoker 支持了 SDK 化,但是 Invoker 本身并没有 IDL 定义的 RPC, 用起来十分不便,能不能进一步生成 Client 去调 Invoker?或者有什么更好的办法?
➜ hello kitex -service a.b.c ./src/hello.thrift
Outside of $GOPATH. Please specify a module name with the '-module' flag.
Describe the bug
write gjson raw data to set fail
data:
&gjson.Result{Type:5, Raw:"[]", Str:"", Num:0, Index:1729, Indexes:[]int(nil)}
err:
remote or network error: thrift marshal, Write failed: nextWriter of field[val_set_string_optional] error data:&gjson.Result{Type:5, Raw:"[]", Str:"", Num:0, Index:1729, Indexes:[]int(nil)}, expected type:SET, err:<nil>
目前的http的路由前缀树是基于httprouter最早的版本修改而来的,gin 同样是在httprouter的基础上不断迭代, https://github.com/gin-gonic/gin/commits/master/tree.go 从而可以看到,我原本打算基于gin版本pr一版上来,但发现 fasthttp router 性能同样优异,同时这里有个关于router 的benchmark,https://github.com/smallnest/go-web-framework-benchmark, 不知目前kitex 框架的对应场景是否有必要更新,以及如果要更新的话,更加适合基于那个版本的更新.
根据kitex-develop/docs/guide/getting_started_cn.md文档,快速搭建hello例子,按照这个文档步骤,操作
新方法分别定义一个新的请求和响应,
AddRequest和
AddResponse,并在
service Hello中增加
add 方法:
在运行的时候报异常
handler.go:20:2: package kitex-examples/hello/kitex_gen/api is not in GOROOT (/root/go/src/kitex-examples/hello/kitex_gen/api)
经对比初始例子,发现kitex生成的新handler.go中多了一条引入包的import代码,如下图:
需要升级kitex框架,升级完后,重新执行kitex命令导出接口文件,handler.go中不再有
api "kitex-examples/hello/kitex_gen/api"
升级执行以下命令:
$ go get github.com/cloudwego/kitex
$ go mod tidy
需要调整getting_started_cn.md文档里面的步骤,提前执行以上这两条命令
# github.com/cloudwego/kitex/pkg/utils
../../../pkg/mod/github.com/cloudwego/[email protected]/pkg/utils/thrift.go:54:36: not enough arguments in call to t.tProt.WriteMessageBegin
have (string, thrift.TMessageType, int32)
want (context.Context, string, thrift.TMessageType, int32)
Have thinked about upgrading thrift?
When I tried to generate project structure with cmd $ kitex -module example -service example echo.thrift
. An error comes up: Failed to find "" from $PATH: exec: "thriftgo": executable file not found in $PATH. Try $GOPATH/bin/thriftgo instead
Link in the readme
file such as https://github.com/cloudwego/kitex-benchmark is invalid.
I noticed that a warning always occurs when the service handles health check request by Consul:
[Warn] transport: http2Server.HandleStreams failed to read frame: EOF
https://github.com/cloudwego/kitex/blob/develop/pkg/remote/trans/nphttp2/grpc/http2_server.go#L342
And I founded that err
equals to a constant netpoll.ErrEOF
at that time, which should be equivalent to io.EOF
and io.ErrUnexpectedEOF
then.
In my opinion, we can add some if conditions to eliminate this redundant warning.
Hi there,
is there any chance that kitex could provide some ways to hook before processing rpc request?
I would like to incr stack size in my service for reducing stack-copy overhead, which might be larger then 10%.
in my service request come with ttheader-framed
, which could be almost 2KB, and therefore go runtime increases the stack size during reading, decoding or processing requests almost all the time. The later morestack
is invoked, the larger overhead it would take.
currently i make a pre-allocation for request in a tricky way with stats.Tracer, saving 50% of the overhead. would be nice if kitex provides a graceful method to hook before processing requests.
The features of kitex support thrift more than grpc, so We want to know:
行数:
kitex/pkg/retry/backup_retryer.go
Line 104 in 6726266
测试代码
func Test_backupRetryer_Do(t *testing.T) {
r := &backupRetryer{
enable: true,
policy: &BackupPolicy{
RetryDelayMS: 0,
StopPolicy: StopPolicy{
MaxRetryTimes: 100,
},
RetrySameNode: true,
},
cbContainer: &cbContainer{
cbCtl: nil,
cbPanel: nil,
cbStat: false,
},
retryDelay: 0,
logger: klog.DefaultLogger(),
RWMutex: sync.RWMutex{},
errMsg: "",
}
r.Do(
context.Background(),
func(ctx context.Context, retryer Retryer) (rpcinfo.RPCInfo, error) {
time.Sleep(100 * time.Millisecond)
return nil, errors.New("1")
},
mockRPCInfo{}, nil)
}
// PayloadCodec is used to marshal and unmarshal payload.
type PayloadCodec interface {
Marshal(ctx context.Context, message Message, out ByteBuffer) error
Unmarshal(ctx context.Context, message Message, in ByteBuffer) error
Name() string
}
This interface specifies how to Marshal and Unmarshal the payload, literal Serializer
may be more precise.
go mod init echo
创建 echo.thrift 如下
namespace go echo
struct Request {
1: string Msg
}
struct Response {
1: string Msg
}
service EchoService {
Response Echo(1: Request req); // pingpong method
oneway void Echo(1: Request req); // oneway method
}
3.我使用kitex 去生成代码,但是没成功
命令如下
kitex -module echo -service echoservice echo.thrift
process 'echo.thrift' failed: failed to reserve NewEchoServiceEchoArgs for $new:$EchoServiceEchoArgs: $new:$EchoServiceEchoArgs
We are always interested in finding out who is using Kitex, what attracted you to using it, how we can listen to your needs and if you are interested, help promote your organization.
Submit a comment in this issue to include the following information
Organization/Company: ByteDance
Website: https://bytedance.com
Country: China
Contact: [email protected]
Usage scenario: Using Kitex to build large scale Cloud Native applications
Status: Production
kitex/pkg/limiter/qps_limiter.go
Line 98 in b2d229b
在 line 100,和 line 106 之间如果涌入大量流量 v,v的误差可能过大
func (l *qpsLimiter) updateToken() {
var v int32
v = atomic.LoadInt32(&l.tokens) // line 100
if v < 0 {
v = atomic.LoadInt32(&l.once)
} else if v+atomic.LoadInt32(&l.once) > atomic.LoadInt32(&l.limit) {
v = atomic.LoadInt32(&l.limit)
} else {
v = v + atomic.LoadInt32(&l.once)
}
atomic.StoreInt32(&l.tokens, v) // line 106
}
改成类似如下流程或许更好:
func (l *qpsLimiter) updateToken() {
if atomic.LoadInt32(&l.limit) <= atomic.LoadInt32(&l.tokens) {
return
}
once := atomic.LoadInt32(&l.once)
limit := atomic.LoadInt32(&l.limit)
delta := limit - atomic.LoadInt32(&l.tokens)
if delta > once || delta < 0 {
delta = once
}
v := atomic.AddInt32(&l.tokens, delta)
if v < once {
atomic.StoreInt32(&l.tokens, once)
}
}
We hope mores developers join CloudWeGo. This is a task issue which need your help.
Issue Description
Write a business demo code use Kitex, which can guide users how to use Kitex.
Please submit your code to https://github.com/cloudwego/biz-demo{yourdemo}.
Picked
I think it would be more convenient to add a flag -c
to pass the config filepath.
And in the config file, we could add 'logdir' item.
ps: When deployed in K8S, passing configuration via environment variables is convenient.
In the comment of registry.Info, it is said that
// ServiceName will be set in kitex by default
but it turns out that this field is not set
And the reason is that in server.go, s.opt.Svr.ServiceName is not set too.
Is this is expected?
Thanks.
Kitex is a very useful framework
kitex/pkg/compress/compressor.go
Lines 19 to 40 in 0f58b30
我想在一个server run的时候注册多个handle,但是我们看到文档或者代码中有有效的方法,请问我这种想法是不是切合实际。
另外:
关于文档中的server sdk 页面好像有点问题,message引入可能有问题。
如题
if developers want to customize the templates, like:
So far, the templates are defined in go code under tool/internal/pkg/tpl
, developers can do this only by fork and changing code.
kitex 关注消息如何编解码(thrift、protobuf),如何高效的传输(TTHeader),也内置了熔断限流和服务治理的功能,跟dubbo比较像(kitex能够支持跨语言)。这种编码和协议对service mesh不那么友好,现在很多公司都在往service mesh的方向探索,dubbo3.0已经支持Service Mesh了,kitex会往这个方向发展吗
hi, I'm getting started with the tutorial on Windows 10. I had install kitex and thriftgo by go install
, and run kitex -module example -service example echo.thrift
in powerShell, but it's stuck, and the console has nothing output, I notice that kitex will create symbol links thrift-gen-kitex
and protoc-gen-kitex
in ~/go/bin, but I didn't see these two things in the directory, so I try to use root permissions to start powerShell and run the command, but to no avail. Hope to get your help. Thanks.
go version: 1.16.8
I tested a protofile with 2 service definitions.
service HelloService{}
service HelloServiceX{}
The generated code says only the last service is supported.
// HelloServiceXImpl implements the last service interface defined in the IDL.
type HelloServiceXImpl struct{}
...
And, I want to know whether a kitex service instance support multiple protocols or not. Maybe not, right?
go version go1.15.3 windows/amd64
使用
git clone https://github.com/cloudwego/kitex-examples.git
cd examples/hello
go run .
出现
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\connection.go:59:18: undefined: OnRequest
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\connection_impl.go:30:2: undefined: netFD
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\connection_impl.go:40:19: undefined: barrier
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\connection_impl.go:41:19: undefined: barrier
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\connection_impl.go:252:32: undefined: Conn
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\connection_impl.go:252:46: undefined: OnPrepare
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\connection_impl.go:272:38: undefined: Conn
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\connection_onevent.go:48:39: undefined: OnRequest
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\connection_onevent.go:71:40: undefined: OnPrepare
C:\Users*\go\pkg\mod\github.com\cloudwego\[email protected]\net_sock.go:41:116: undefined: netFD
C:\Users***\go\pkg\mod\github.com\cloudwego\[email protected]\net_sock.go:41:116: too many errors
echo.thrift
namespace go api
struct Request {
1: string message
}
struct Response {
1: string message
}
service Echo {
Response echo(1: Request req)
}
使用这个echo.thrift用命令kitex -module me -service echo echo.thrift生成后
在kitex_gen/api/echo.go文件中很多方法出现参数缺失或参数出现丢失context参数情况
kitex -version v0.0.6
thriftgo --version thriftgo 0.1.3
如题,公司开源的版本怎样修改8888端口呢?我本地测试需要跑几个RPC项目。
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.