ojlm / pea Goto Github PK

View Code? Open in Web Editor NEW

45.0 6.0 30.0 3.98 MB

分布式压测引擎. A distributed stress tool based on gatling

Home Page: https://asura-pro.github.io/indigo-docs/#/zh-cn/pea/

Scala 67.42% JavaScript 0.51% TypeScript 17.65% CSS 5.38% HTML 8.75% Java 0.18% Shell 0.07% Dockerfile 0.04%

indigo akka gatling distributed-stress pea

pea's Introduction

Gatling Pea

关于 Gatling

Gatling 是基于 Netty 和 Akka 技术实现的高性能压测工具.

关于 Pea

由于单独一台机器硬件资源和网络协议的限制存在, 在高负载测试中需要多台机器共同提供负载. Pea 是在以 Galting 为引擎, 在多节点场景下的压测工具. 包含以下特性:

管理和监控多个工作节点. 依赖 Zookeeper
运行过程中可以实时查看每个节点的具体执行状态
多个节点执行完后会自动收集日志, 生成统一的报告
支持原生的 Gatling 脚本, 原生的 HTTP 协议
扩展支持了 Dubbo和 Grpc 协议
以 Git 仓库管理脚本和资源文件
内置了 Scala 增量编译器, 脚本可在线快速编译
不同于其他实现, 所有这些功能都在同一进程内实现. 感谢 Gatling 作者高质量的代码
可以在实体机, 虚拟机, Docker 容器中运行

脚本示例

import io.gatling.core.Predef._
import pea.dubbo.Predef._
import pea.dubbo.api.GreetingService
import pea.gatling.PeaSimulation

class DubboGreetingSimulation extends PeaSimulation {
  override val description: String =
    """
      |Dubbo simulation example
      |""".stripMargin
  val dubboProtocol = dubbo
    .application("gatling-pea")
    .endpoint("127.0.0.1", 20880)
    .threads(10)
  val scn = scenario("dubbo")
    .exec(
      invoke(classOf[GreetingService]) { (service, _) =>
        service.sayHello("pea")
      }.check(simple { response =>
        response.value == "hi, pea"
      }).check(
        jsonPath("$").is("hi, pea")
      )
    )
  setUp(
    scn.inject(atOnceUsers(10000))
  ).protocols(dubboProtocol)
}

版本

pea	gatling	dubbo	grpc
0.6.0~	3.3.1	2.7.4.1	1.22.2

视频演示

https://www.bilibili.com/video/av73339161/

截图示例

创建任务

任务执行中的节点状态

整体报告

单个请求细节报告

pea's People

Contributors

Stargazers

Watchers

pea's Issues

请教：如何将多台机器的报告聚合成一份

你好，
请教：如何将多台机器的报告聚合成一份，因为我正在使用python开发压测平台，其他功能基本完成，但在报告聚合上遇到了瓶颈，希望可以不吝赐教！

查看报告的时候报错了，文件存在的，是什么原因呢？

{"code":"90000","msg":"Blocking access to this file: /opt/pea/pea/target/universal/pea/logs/centos-test-20200225142732080/index.html"}

脚本管理关联不上git仓库

配置了webEditorBaseUrl，页面上也找不到，请问这里面是如何把绑定的呢？后台compiler成功了，文件生成在test/simulaiton目录下，又被删除了，请问大佬这是什么原因呢？

Dubbo 的接口声明不能热编译重载

https://github.com/asura-pro/pea/blob/master/pea-dubbo/src/main/scala/asura/pea/dubbo/request/CustomReferenceConfig.java#L180 这个地方可以设置使用 ReloadableClassloader ，但是实际执行会失效, 如果移除调用 https://github.com/asura-pro/pea/blob/master/app/asura/pea/hook/ApplicationStart.scala#L166 ，就会发现 ClassNotFound。

估计会导致, Dubbo 的 Interface 不能重载, 脚本依然可以重载

webEdittorBaseUrl拉不到对应的simulations脚本

compiler报错，尝试各种版本的jar包都不行，错误都不太一样，
[debug] p.a.c.ZincCompilerInstance - [sbt]:Full compilation, no sources in previous analysis.
[debug] p.a.c.ZincCompilerInstance - [sbt]:Full compilation, no sources in previous analysis.
[debug] p.a.c.ZincCompilerInstance - [sbt]:All sources are invalidated.
[debug] p.a.c.ZincCompilerInstance - [sbt]:All sources are invalidated.
[debug] p.a.c.ZincCompilerInstance - [sbt]:Initial set of included nodes:
[debug] p.a.c.ZincCompilerInstance - [sbt]:Recompiling all sources: number of invalidated sources > 50.0% of all sources
[info] p.a.c.ZincCompilerInstance - [sbt]:Compiling 12 Scala sources and 2 Java sources to D:\pea\logs\output ...
[debug] p.a.c.ZincCompilerInstance - [sbt]:Initial set of included nodes:
[debug] p.a.c.ZincCompilerInstance - [sbt]:Recompiling all sources: number of invalidated sources > 50.0% of all sources
[info] p.a.c.ZincCompilerInstance - [sbt]:Compiling 12 Scala sources and 2 Java sources to D:\pea\logs\output ...
[debug] p.a.c.ZincCompilerInstance - [sbt]:Returning already retrieved and compiled bridge: D:\Program\sbt\bin\compiler-bridge_2.13-1.3.5.jar.
[debug] p.a.c.ZincCompilerInstance - [sbt]:Returning already retrieved and compiled bridge: D:\Program\sbt\bin\compiler-bridge_2.13-1.3.5.jar.
[error] p.a.c.ZincCompilerInstance - [sbt]:## Exception when compiling 14 sources to D:\pea\logs\output
java.lang.VerifyError: class scala.collection.mutable.LinkedHashMap overrides final method stringPrefix.()Ljava/lang/String;
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
[error] p.a.c.ZincCompilerInstance - [sbt]:## Exception when compiling 14 sources to D:\pea\logs\output
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at scala.collection.mutable.LinkedHashMap$.from(LinkedHashMap.scala:30)
at scala.collection.mutable.LinkedHashMap$.from(LinkedHashMap.scala:24)
at scala.collection.MapFactory.apply(Factory.scala:390)
at scala.collection.MapFactory.apply$(Factory.scala:390)
at scala.tools.nsc.settings.MutableSettings.allSettings$lzycompute(MutableSettings.scala:28)
at scala.tools.nsc.settings.MutableSettings.allSettings(MutableSettings.scala:28)

报告中请求数

生成的报告中请求数是用户数,怎么改成请求总数呢?

GRPC 异常

forever ... 

setUp(
    scn.inject(atOnceUsers(100))
  ).maxDuration(2 minutes).protocols(grpcProtocol)

[ERROR] from pea.grpc.action.GrpcAction in GatlingSystem-akka.actor.default-dispatcher-33 - 'grpcRequest-2167976' crashed on session Session(grpc,53,1576562221974,Map(7e99e6b
2-4672-40cd-b844-c0527fd6dcb0 -> 37560),0,KO,List(ExitOnCompleteLoopBlock(7e99e6b2-4672-40cd-b844-c0527fd6dcb0)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$1948/483463209@440b28c),
forwarding to the next one
java.util.concurrent.RejectedExecutionException: Task io.grpc.internal.SerializingExecutor@74d98b80 rejected from java.util.concurrent.ThreadPoolExecutor@7d62a635[Terminated, pool size = 0, active t
hreads = 0, queued tasks = 0, completed tasks = 4246882]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
        at io.grpc.internal.SerializingExecutor.schedule(SerializingExecutor.java:93)
        at io.grpc.internal.SerializingExecutor.execute(SerializingExecutor.java:86)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.closedInternal(ClientCallImpl.java:690)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.closed(ClientCallImpl.java:643)
        at io.grpc.internal.FailingClientStream.start(FailingClientStream.java:54)
        at io.grpc.internal.ClientCallImpl.startInternal(ClientCallImpl.java:289)
        at io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:190)
        at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1.start(CensusTracingModule.java:394)
        at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1.start(CensusStatsModule.java:695)
        at io.grpc.ForwardingClientCall.start(ForwardingClientCall.java:32)
        at io.grpc.stub.MetadataUtils$HeaderAttachingClientInterceptor$HeaderAttachingClientCall.start(MetadataUtils.java:88)
        at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:292)
        at io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:267)
        at io.grpc.stub.ClientCalls.futureUnaryCall(ClientCalls.java:177)
        at pea.grpc.GrpcDsl$CallWithMethod.$anonfun$f$2(GrpcDsl.scala:33)
        at pea.grpc.action.GrpcAction.run(GrpcAction.scala:66)
        at pea.grpc.action.GrpcAction.$anonfun$sendRequest$5(GrpcAction.scala:58)
        at pea.grpc.action.GrpcAction.$anonfun$sendRequest$5$adapted(GrpcAction.scala:45)
        at io.gatling.commons.validation.Success.map(Validation.scala:45)
        at pea.grpc.action.GrpcAction.$anonfun$sendRequest$4(GrpcAction.scala:45)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:46)
        at pea.grpc.action.GrpcAction.sendRequest(GrpcAction.scala:39)
        at io.gatling.core.action.RequestAction.$anonfun$execute$1(Action.scala:100)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:46)
        at io.gatling.core.action.RequestAction.execute(Action.scala:97)
        at io.gatling.core.action.RequestAction.execute$(Action.scala:96)
        at pea.grpc.action.GrpcAction.execute(GrpcAction.scala:20)
        at io.gatling.core.action.Action.$bang(Action.scala:38)
        at io.gatling.core.action.Action.$bang$(Action.scala:38)
        at pea.grpc.action.GrpcAction.io$gatling$core$action$ChainableAction$$super$$bang(GrpcAction.scala:20)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:63)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:61)
        at pea.grpc.action.GrpcAction.io$gatling$core$action$ExitableAction$$super$$bang(GrpcAction.scala:20)
        at io.gatling.core.action.ExitableAction.$bang(BlockExit.scala:141)
        at io.gatling.core.action.ExitableAction.$bang$(BlockExit.scala:139)
        at pea.grpc.action.GrpcAction.$bang(GrpcAction.scala:20)
        at io.gatling.core.action.InnerLoop.execute(Loop.scala:94)
        at io.gatling.core.action.Action.$bang(Action.scala:38)
        at io.gatling.core.action.Action.$bang$(Action.scala:38)
        at io.gatling.core.action.InnerLoop.io$gatling$core$action$ChainableAction$$super$$bang(Loop.scala:58)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:63)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:61)
        at io.gatling.core.action.InnerLoop.$bang(Loop.scala:58)
        at io.gatling.core.action.Loop.execute(Loop.scala:54)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:72)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:61)
        at pea.grpc.action.GrpcAction.io$gatling$core$action$ExitableAction$$super$$bang(GrpcAction.scala:20)
        at io.gatling.core.action.ExitableAction.$bang(BlockExit.scala:141)
        at io.gatling.core.action.ExitableAction.$bang$(BlockExit.scala:139)
        at pea.grpc.action.GrpcAction.$bang(GrpcAction.scala:20)
        at io.gatling.core.action.InnerLoop.$anonfun$execute$1(Loop.scala:91)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:49)
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

请问一下，如何将jar包上传到本地私服？没找到相关的配置

可以列出这个项目的技术栈等信息吗

启动的时候报错，是什么原因？

com.google.inject.CreationException: Unable to create injector, see the following errors:

No implementation for org.pac4j.play.scala.SecurityComponents was bound.
while locating org.pac4j.play.scala.SecurityComponents
for the 4th parameter of pea.app.api.GatlingApi.(GatlingApi.scala:34)
while locating pea.app.api.GatlingApi
for the 3rd parameter of router.Routes.(Routes.scala:33)
at play.api.inject.RoutesProvider$.bindingsFromConfiguration(BuiltinModule.scala:129):
Binding(class router.Routes to self) (via modules: com.google.inject.util.Modules$OverrideModule -> play.api.inject.guice.GuiceableModuleConversions$$anon$4)
No implementation for org.pac4j.play.scala.SecurityComponents was bound.
while locating org.pac4j.play.scala.SecurityComponents
for the 4th parameter of pea.app.api.HomeApi.(HomeApi.scala:33)
while locating pea.app.api.HomeApi
for the 2nd parameter of router.Routes.(Routes.scala:33)
at play.api.inject.RoutesProvider$.bindingsFromConfiguration(BuiltinModule.scala:129):
Binding(class router.Routes to self) (via modules: com.google.inject.util.Modules$OverrideModule -> play.api.inject.guice.GuiceableModuleConversions$$anon$4)
No implementation for org.pac4j.play.scala.SecurityComponents was bound.
while locating org.pac4j.play.scala.SecurityComponents
for the 4th parameter of pea.app.api.ResourceApi.(ResourceApi.scala:30)
while locating pea.app.api.ResourceApi
for the 4th parameter of router.Routes.(Routes.scala:33)
at play.api.inject.RoutesProvider$.bindingsFromConfiguration(BuiltinModule.scala:129):
Binding(class router.Routes to self) (via modules: com.google.inject.util.Modules$OverrideModule -> play.api.inject.guice.GuiceableModuleConversions$$anon$4)

3 errors
at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:543)
at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:159)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106)
at com.google.inject.Guice.createInjector(Guice.java:87)
at com.google.inject.Guice.createInjector(Guice.java:78)
at play.api.inject.guice.GuiceBuilder.injector(GuiceInjectorBuilder.scala:200)
at play.api.inject.guice.GuiceApplicationBuilder.build(GuiceApplicationBuilder.scala:155)
at play.api.inject.guice.GuiceApplicationLoader.load(GuiceApplicationLoader.scala:21)
at play.core.server.ProdServerStart$.start(ProdServerStart.scala:56)
at play.core.server.ProdServerStart$.main(ProdServerStart.scala:30)
at play.core.server.ProdServerStart.main(ProdServerStart.scala)

依赖问题：我将build.sbt 中akka相关升级为2.6.0每次拉取依赖时候会有2.5.26的包被拉取

我在build.sbt中添加了以下代码想将akka升级到2.6.0 但是每次拉取依赖时候都会有2.5.26的包被拉去下来我没有找到这个依赖在哪里配置的，如何去掉2.5.26这个依赖呢？

val akka                           = "com.typesafe.akka"                   %% "akka-actor"                      % "2.6.0"
val akkaSlf4j                      = akka.organization                     %% "akka-slf4j"                      % akka.revision
val akkaStream                     = akka.organization                     %% "akka-stream"                     % akka.revision
val akkaProtobuf                   = akka.organization                     %% "akka-protobuf"                   % akka.revision
val akkaTestKit                    = akka.organization                     %% "akka-testkit"                    % akka.revision       % "test"

libraryDependencies ++= Seq(akka, akkaSlf4j, akkaStream, akkaProtobuf, akkaTestKit)

Dubbo 接口自定义对象响应 cannot be cast to 异常

复现

https://github.com/asura-pro/pea-simulations/blob/master/ext-library/src/main/java/pea/example/ext/dubbo/api/GreetingService.java#L9

异常

HelloResponse 第一次正常, 后续的调用会报 pea.example.ext.dubbo.response.HelloResponse cannot be cast to pea.example.ext.dubbo.response.HelloResponse 这种异常.

这边每次执行会使用自定义的类加载器重载, 可能是 dubbo 生成的动态代理类缓存, 在后续的执行用了缓存的 HelloResponse class, 这时候引擎的该对象的 classloader 和 dubbo 内部的代理对象不一样啦