jgomezselles / hermes Goto Github PK

Hermes HTTP/2 traffic generator

License: MIT License

CMake 2.57% C++ 93.76% Dockerfile 1.42% Go 0.74% Smarty 0.92% Shell 0.58%

nghttp2 traffic http2 http-client cpp traffic-simulation tool testing-tools load-testing

hermes's Introduction

What is hermes?

Hermes is an http/2 traffic generator written in C++ able to send multiplexed requests over one connection at a given rate.

It also measures and calculates some statistics, validates response codes and handles user-defined traffic sequences, a.k.a. scripts.

Because of the nature of the project itself, it has a minimal footprint impact, and has proven to be reliable: traffic is generated in a steady pace, without unwanted traffic bursts. This is achieved by using boost async io_context utility and thread synchronization strategies.

It is based in the nghttp2 implementation, and provides a different functionality than h2load, also written in C++ by the nghttp2 creator, because hermes allows to use traffic scripts, traffic is injected at a pre-defined rate, and validates the results.

Hermes aims to be a low footprint and easy to use but high-performance general use tool.

How can you use it?

Hermes may be used standalone (check the releases!), from a docker container, or as part of a kubernetes deployment.

Executing hermes

You may take a look to Hermes help by just typing ./hermes -h:

hermes: C++ Traffic Generator. Usage:  hermes [options]

options:
       -r <rate>      Requests/second ( Default: 10 )

       -t <time>      Time to run traffic (s) ( Default: 60 )

       -p <period>    Print and save statistics every <period> (s) ( Default: 10 )

       -f <path>      Path with the traffic json definition ( Default: /etc/scripts/traffic.json )

       -s             Show schema for json traffic definition.

       -o <file>      Output file for statistics( Default: hermes.out )

       -h             This help.

At the end of the day a typical use is:

./hermes -r2400 -p1 -t3600

Hermes results, console and file outputs are explained here.

hermes helm chart integration

The idea is simple:

Add it to your Chart.yaml dependencies.
Reference hermes in your chart under .Values.hermes.image repository and tag.
Define a config map for your traffic definition (script) and reference it in your values with the name you gave it under .Values.hermes.script.cm

But what is inside that ConfigMap? Your traffic script definition.

And that's it!

If you have some doubts, take a look at the example-hermes helm chart, or the following examples.

Examples

This repository provides a server-mock implementation which enables you to test and taste easily how hermes works.

Container to Container example

Build and run a server-mock container by following these instructions.
Attach to a docker container running a ghcr.io/jgomezselles/hermes image, and create a script file like the one located in the helm example.
Run hermes providing the path to your new script:

./hermes -r2400 -p1 -t3600 -f <path/to/script.json>

hermes in kubernetes example

Build (but not run) a server-mock by following these instructions.
Update the example-hermes helm chart dependencies, and install the chart:

helm dep update helm/example-hermes
helm install example-hermes helm/example-hermes

Run hermes

kubectl exec <hermes-pod> -c hermes -- /hermes/hermes -p1 -t10 -r1000

The server-mock prints every received request to the console, so you can check the received requests by inspecting the logs: kubectl logs <server-mock-pod>

Developing & testing hermes

Take a look to hermes dev info here.

License

The Software implemented in this repository is distributed under MIT license, as stated in the LICENSE file. It makes use of some boost C++ library functionalities, which have their own license. Hermes core functionality is based on nghttp2, and json handling is done with rapidjson, both under MIT license. Tests are done with Googletest, which is published under BSD 3-Clause License.

hermes's People

Contributors

Stargazers

Watchers

Forkers

hombrenieve pepeludev

hermes's Issues

Upload docker images

Uploading them will help anyone to just use or develop/build.

Increase coverage

It seems a bit low right now. Let's see how further we can get.

Execute hermes via h2 requests

Having a server listening for requests which will be able to START an execution will help tests and will add a whole new functionality. In this issue, some very basic PUT will be added.

Enhance documentation

Maybe with github pages

Fix UT

Test connection_test.incorrect_initialization_wrong_host is failing. Fix it in this task.

Add Sonar badg(es)

Just go here and pick some of them when everything is set: https://sonarcloud.io/project/information?id=jgomezselles_hermes

Add configurable headers to message definition

Add prometheus metrics

Add json_reader tests

It is tested tangentially, but dedicated tests are needed.

Add coverage to sonar

Add GET request support to the server

Add schema documentation

Traffic schema documentation is a bit ugly. Find a better way to do it.

Upload script usage

Add some code scanner tool

Helm chart linting

Add script/s metric option

Consider if as an input/target, but as a metric

Add Observability docs

Improve testing for files under src/script

It has been noticed that the files under src/script folder are not really tested in a functional way, (just to get the coverage, which is a bit dangerous), so new tasks will be added.

Publish helmcharts

Add application testing

Hermes CI/CD checks shall have a way of testing the application itself. It might be done through helm chart installation and full k8s deployment.

In order for this to work, a server will be introduced, listening to requests. This is not only a way to ease the tests, but also a functionality which could give hermes a lot of power.

Make docker images lighter

Hermes base and build can have some stuff, but maybe it has so much,
We could have hermes_prod with just the app, and hermes_test for component testing or something like that, from a basic alpine(?) image.

Parallel scripts

Running several clients with their own script/traffic and execution configuration (or shared) could be benefitial.

Add header handling in script

This entails using save_from_answer, variables and add_from_saved_to.... body???

Multiple connections

Analyze and decide whether to use/configure multiple connections with a maximum number of streams to be negociated.

Add coverage to application test

Collecting coverage and merging it with unit tests will give a better view on what is actually covered by tests.

Validate responses

Hermes is only validating response codes. Include body validation as well.

Add tests to check metrics and traces

Multiple client endpoints

Is your feature request related to a problem? Please describe.
Not actually a problem, but a limitation which could provide a better testing environment.

Describe the solution you'd like
Extend the schema and the hermes logic to accept multiple clients (a section to define the server and port within a json list) instead of the current unique one.
So, you could define a traffic profile to, for example, launch an specific Request definition through an specific client endpoint (a new field should be added in the schema here to specify on which endpoint is desired to be sent), and then mix with other requests with different destinations.
The solution permits mixed-protocols testing relaying on REST API for those which are not http2 traffic servers.

Additional context
Imagine that your system under test is composed by the main process to test and then some server components mocking the data base or whatever facility provided to simulate the SUT context. Now imagine that those mocks have an smart REST API which could be used to inspect the traffic reveiced. From hermes you could also use that REST API as part of traffic validation delegating on those external agents. In this way you could make complex test flows including whatever kind of agent (servers, even clients, for different protocols, not only http2) just having a REST API for them which is based in http2 (or even http/1 with a nginx reverse proxy in fornt of them).
This feature could be backward compatible with a default definition (not a one-item-list but just accepting current schema definition or a list with 2 (or even 1) or more items).