philsupertramp / chain-smoker Goto Github PK

The examples and action should be merged as subdirectories into the main repository.

There're several options how to do that

• django app
• python app with celery or other task scheduler
• cron tasks in dedicated docker containers
• implement github action

At the end we should decide which way to go, commit to it, build the monitoring solution and release this tool.

Utilize go-reverse-proxy

• proxy incoming request to provided host
• log request-response pairs as JSON to stdout

Build parser:

• listens to stdin
• build unique tests from incoming JSON of form

{"Request": {"Payload": "", "Protocol": "HTTP/1.1", "Path": "https://postman-echo.com/get/", "Method": "GET", "Headers": {"Accept": ["application/json"], "User-Agent": ["curl/7.82.0"], "X-Forwarded-For": ["127.0.0.1"]}}, "Response": {"Status_code": 200, "Body": "{\"args\":{},\"headers\":{\"x-forwarded-proto\":\"https\",\"x-forwarded-port\":\"443\",\"host\":\"postman-echo.com\",\"x-amzn-trace-id\":\"Root=1-636b9eb1-32e9a8e376d08f1854179afc\",\"user-agent\":\"curl/7.82.0\",\"accept\":\"application/json\",\"accept-encoding\":\"gzip\"},\"url\":\"https://postman-echo.com/get/\"}", "Headers": {"Content-Length": ["281"], "Content-Type": ["application/json; charset=utf-8"], "Date": ["Wed, 09 Nov 2022 12:36:01 GMT"], "Etag": ["W/\"119-oeZf0KoFbxz7XopTDTpm9GdAPOo\""], "Set-Cookie": ["sails.sid=s%3AD8J_rCEAoSONQwJGoQO31FC9isYn6uER.zcaeLNzYKBupkRBVpv2BJkU6m0esCe7bOcq8bjqFS00; Path=/; HttpOnly"], "Vary": ["Accept-Encoding"]}}}

• store tests in directory for usage by chain-smoker

Implement on/off mechanic:

• either parser.py or proxy.go need a mechanic how it can be de-/activated

New infrastructure diagram

(dashed lines: traffic flow; regular lines: internal flow)

graph LR;
    dev -- release --> proxy[Proxy];
    traffic -.-> proxy;
    proxy -. req .-> prod[Application];
    prod -. res .-> proxy;
    proxy -- build tests --> parser[Parser];
    prod -- run tests --> test[chain-smoker];
    test -- run tests --> prod;
    parser -- fills --> test_dir[Test directory];
    test -- uses --> test_dir;

Description

To enable all features supported by PyYAML we need to use load_all instead of load_safe when loading test case configuration from YAML files.

This enables all features described in https://pyyaml.org/wiki/PyYAMLDocumentation

Required changes

https://github.com/philsupertramp/chain-smoker/blob/main/src/chain_smoker/file_loader.py#L25

By now the tool is already pretty complex.
To allow any kind of user to setup a test suite using chain-smoker, the user facing API, the test configuration, needs to be documented thoroughly.

We should first try to generate them out of the pydantic models. If this doesn't work out, it needs to be done manually!

A coffee, a glass of wine or whatever floats your boat is all you need to go through this pain, once. After the initial docs, we just need to append changes, not start from ground up.

If it ain't me doin it, I'll pay you a coffee from the most expensive specialty coffee place in your city!

Would be cool to be able to add default values for parameters of ApiClient request methods.
The idea would be to write configuration, e.g.

client:
  base_url: example.com
  kwargs:
    timeout: 15
    stream: false

Options documented in https://requests.readthedocs.io/en/latest/api/#requests.request

Then pass it into the method every time we execute a request

by enhancing the passed kwargs

**{**self.default_kwargs, **kwargs}

or

final_kwargs = self.default_kwargs.copy()
final_kwargs.update(kwargs)
**final_kwargs

Especially ContainsTest is too complex to maintain by now.

This should be re-written in a simplified way.

I mean look at that...
https://github.com/philsupertramp/chain-smoker/blob/main/src/chain_smoker/test_clients.py#L59-L126

We need support for

• payload cookies
• expected response cookies

Example:
Supposedly endpoint /step-6-of-10 requires two cookies, a key cookie and a sessionId cookie and responds with a progress cookie, then a test for this might look like this:

example_test:
  endpoint: /step-6-of-10
  method: get
  payload_cookies:
    - domain: example.com
      max_age: 2022-02-02T10:00:00+01:00:00
      name: key
      value: value
    - domain: example.com
      max_age: 2022-02-02T10:00:00+01:05:00
      key: sessionId
      value: 420
  response_cookies:
    - domain: example.com
      max_age: Session
      name: progress
      value: 60

To generate documentation, for the options available in YAML config files, out of code and to simplify parsing it we can wrap the configuration options into a data structure of nested Python dataclass objects.

Example:

@dataclass
class Config:
    type: str
    ...

Would be great to have some service that collects data.
Optionally, we should publish those results as artifacts in the action