Giter Club home page Giter Club logo

yandex-tank-api's Introduction

Yandex.Tank API

This is an HTTP server that controls Yandex.Tank execution. It allows the client to:

  • set a breakpoint before an arbitrary test stage (and reset it later)
  • launch Yandex.Tank and begin the test
  • upload files into the test working directory
  • terminate the launched Tank at an arbitrary moment
  • obtain the test status
  • download the artifacts in a safe manner, without interference to another tests

Start API server

by running yandex-tank-api-server [options...] in console

API-managed Tank

General information on Yandex.Tank installation and usage can be found in its documentation. This section covers the difference between console Tank and API-managed Tank configuration and provides more details on the sequence of the test stages.

Tank configuration

API-managed Tank is configured via configuration files only. They have the same syntax and options as the console Tank configs.

The configuration parameters are applied in the following order:

  1. common Yandex.Tank configuration files that reside in /etc/yandex-tank/
  2. Yandex.Tank API defaults in /etc/yandex-tank-api/defaults
  3. the configuration file sent by the client when launching the test
  4. Yandex.Tank API overriding configs in /etc/yandex-tank-api/override

Test stages

When the client launches a new test, a new session is created and a separate Tank worker process is spawned. After this, the test stages are executed in the following order:

  1. lock

    Attempt to acquire the tank lock. This stage fails if another test started via console is running.

  2. init

    Logging is set up and tank configuration files are read at this stage.

  3. configure

    The configure() method is called for each module. Most of the dynamic configuration is done here.

  4. prepare

    The prepare_test() method is called for each module. Heavy and time-consuming tasks (such as stpd-file generation and monitoring agent setup) are done here.

  5. start

    The start_test() method is called for each module. This should take as little time as possible. Load generation begins at this moment.

  6. poll

    Once per second the is_test_finished() method is called for each module. This stage ends when any of the modules requests to stop the test.

  7. end

    The end_test() method is called for each module. This should take as little time as possible.

  8. postprocess

    The post_process() method is called for each module. Heavy and time-consuming tasks are performed here.

  9. unlock

    The tank lock is released and the Tank worker exits.

  10. finished

 This is a virtual stage. Reaching this stage means that the Tank worker has already terminated.

The last session status is temporarily stored after tank exit. The test artifacts are stored forever and should be deleted by external means when not needed.

Pausing the test sequence

When the session is started, the client can specify the test stage before which the test will be paused (the breakpoint) . After completing the stages preceding the breakpoint, the Tank will wait until the breakpoint is moved further. You cannot move the breakpoint back.

The breakpoint can be set before any stage. One of the most frequent use cases is to set the breakpoint before the start stage to synchronize several tanks. Another is setting the breakpoint before the unlock stage to download the artifacts without interference to other tests. Third is setting the breakpoint before the init stage to upload additional files. Beware that setting the breakpoint between the init and the poll stages can lead to very exotic behaviour.

API requests

All API requests are asynchronous: we do not wait the tank process to perform requested action. HTTP code 200 is returned when no error occured while processing the request. However, this does not necessarily mean that the requested action will be successfully performed. The client should check the session status to detect Tank failures.

All handles, except for /artifact, return JSON. On errors this is a JSON object with a key 'reason'.

List of API requests

  1. POST /validate

Request body: Yandex.Tank config in .yaml format (the same as for console Tank)

Checks if config provided is valid within local defaults

Reply on success:

{
  "config": "<yaml string>", // your config
  "errors": [] // empty if valid
}

Error codes and corresponding reasons in the reply:

  • 400, 'Config is not a valid YAML.'
  1. POST /run?[test=...]&[break=...]

Request body: Yandex.Tank config in .yaml format (the same as for console Tank)

Creates a new session with an unique session ID and launches a new Tank worker.

Parameters:

  • test: Prefix of the session ID. Should be a valid directory name. Default: current datetime in the %Y%m%d%H%M%S format
  • break: the test stage before which the tank will stop and wait until the next break is set. Default: "finished"

Reply on success:

{
  "session": "20150625210015_0000000001", //ID of the launched session
  "test": "20150625210015_0000000001" //Deprecated, do not use
}

Error codes and corresponding reasons in the reply:

  • 400, 'Specified break is not a valid test stage name.'
  • 409, 'The test with this ID is already running.'
  • 409, 'The test with this ID has already finished.'
  • 503, 'Another session is already running.'
  1. GET /run?session=...&[break=...]

Sets a new break point for the running session.

Parameters:

  • session: session ID
  • break: the test stage before which the tank will stop and wait until the next break is set. Default: "finished"

Return codes and corresponding reasons:

  • 200, 'Will try to set break before [new break point]'
  • 400, 'Specified break is not a valid test stage name.'
  • 404, 'No session with this ID.'
  • 418, ... (returned when client tries to move the break point back)
  • 500, 'Session failed.'
  1. GET /stop?session=...

Terminates the current test.

Parameters:

  • session: ID of the session to terminate

Return codes and corresponding reasons:

  • 200, 'Will try to stop tank process.'
  • 404, 'No session with this ID.'
  • 409, 'This session is already stopped.'
  1. GET /status?session=...

Returns the status of the specified session. Parameters:

  • session: ID of the session.

Status examples:

{
  "status": "running",
  "stage_completed": true,
  "break": "start",
  "current_stage": "prepare",
  "test": "9f43f3104c2549b98bf74b817dc71cef",
  "failures": []
}
{
  "status": "failed", 
  "retcode": 1, 
  "stage_completed": true, 
  "break": "finished", 
  "current_stage": "finished", 
  "test": "9f43f3104c2549b98bf74b817dc71cef", 
  "failures": [
      {
          "reason": "Interrupted", 
          "stage": "prepare"
      }
  ]
}

Error code and the corresponding reason:

  • 404, 'No session with this ID.'
  1. GET /status?

Returns a JSON object where keys are known session IDs and values are the corresponding statuses.

  1. GET /artifact?session=...

Returns a JSON array of artifact filenames.

Parameters:

  • test: ID of the test

Error codes and the corresponding reasons:

  • 404, 'No test with this ID found.'
  • 404, 'Test was not performed, no artifacts.'
  1. GET /artifact?session=...&filename=...

Sends the specified artifact file to the client.

Parameters:

  • session: ID of the session
  • filename: the artifact file name

Error codes and the corresponding reasons:

  • 404, 'No session with this ID found'
  • 404, 'Test was not performed, no artifacts.'
  • 404, 'No such file'
  • 503, 'File is too large and test is running' (when the file size exceeds 128 kB and some test is running)
  1. POST /upload?session=...&filename=...

Stores the request body on the server in the tank working directory for the session under the specified filename. The session should be running.

Parameters:

  • session: ID of the session
  • filename: the name to store the file under

Error codes and the corresponding reasons:

  • 404, 'Specified session is not running'

Writing plugins

Some custom plugins might need to know if they are wokring in the console Tank or under API.

API worker process uses a subclass of a standard TankCore as a tank core. Thus, a plugin can detect execution under API by simply checking that

str(self.core.__class__)=='yandex_tank_api.worker.TankCore'

yandex-tank-api's People

Contributors

asekretenko avatar citizen-stig avatar direvius avatar tonsofattraction avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yandex-tank-api's Issues

failing to start yandex-tank-api-server

after installing pip dev branch , not able to start api server:

(appenv) root@linuxkit-025000000001:/tank-api-workdir# /appenv/bin/yandex-tank-api-server --debug
2018-03-14 10:37:18,606 INFO: Starting server
2018-03-14 10:37:18,606 [INFO] yandex_tank_api.manager Starting server
2018-03-14 10:37:18,610 INFO: Resetting current session variables
2018-03-14 10:37:18,610 [INFO] yandex_tank_api.manager Resetting current session variables
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/webserver.py", line 474, in main
    ApiServer(webserver_queue, manager_queue, test_directory, debug).serve()
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/webserver.py", line 455, in serve
    update_cb = tornado.ioloop.PeriodicCallback(self.check, 300, ioloop)
TypeError: __init__() takes exactly 3 arguments (4 given)
2018-03-14 10:37:19,615 ERROR: Webserver died unexpectedly.
2018-03-14 10:37:19,615 [ERROR] yandex_tank_api.manager Webserver died unexpectedly.
2018-03-14 10:37:19,615 ERROR: Unhandled exception in manager.run_server:
Traceback (most recent call last):
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/manager.py", line 298, in run_server
    Manager(cfg).run()
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/manager.py", line 233, in run
    self._handle_webserver_exit()
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/manager.py", line 219, in _handle_webserver_exit
    raise RuntimeError("Unexpected webserver exit")
RuntimeError: Unexpected webserver exit
2018-03-14 10:37:19,615 [ERROR] yandex_tank_api.manager Unhandled exception in manager.run_server:
Traceback (most recent call last):
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/manager.py", line 298, in run_server
    Manager(cfg).run()
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/manager.py", line 233, in run
    self._handle_webserver_exit()
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/manager.py", line 219, in _handle_webserver_exit
    raise RuntimeError("Unexpected webserver exit")
RuntimeError: Unexpected webserver exit

and this is for 0.0.11 version:

2018-03-14 15:26:29,627 INFO: Starting server
2018-03-14 15:26:29,627 [INFO] yandex_tank_api.manager Starting server
2018-03-14 15:26:29,630 INFO: Resetting current session variables
2018-03-14 15:26:29,630 [INFO] yandex_tank_api.manager Resetting current session variables
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/webserver.py", line 482, in main
    ApiServer(webserver_queue, manager_queue, test_directory, debug).serve()
  File "/appenv/local/lib/python2.7/site-packages/yandex_tank_api/webserver.py", line 463, in serve
    self.check, 300, ioloop)
TypeError: __init__() takes exactly 3 arguments (4 given)
2018-03-14 15:26:30,633 ERROR: Webserver died unexpectedly.
2018-03-14 15:26:30,633 [ERROR] yandex_tank_api.manager Webserver died unexpectedly.```

ImportError: No module named pidlockfile

$ yandex-tank-api-server
Traceback (most recent call last):
  File "/data/qa/ltbot/venv/bin/yandex-tank-api-server", line 9, in <module>
    import daemon.pidlockfile
ImportError: No module named pidlockfile
$ pip list
argparse (1.3.0)
backports.ssl-match-hostname (3.4.0.2)
certifi (2015.4.28)
docutils (0.12)
importlib (1.0.3)
ipaddr (2.1.11)
lockfile (0.10.2)
pip (7.1.0)
progressbar (2.3)
psutil (3.0.1)
pyjade (3.0.0)
python-daemon (2.0.5)
requests (2.7.0)
setuptools (18.0.1)
simplejson (3.7.3)
six (1.9.0)
TornadIO2 (0.0.4)
tornado (4.2)
wheel (0.24.0)
yandex-tank-api (0.0.7)
yandextank (1.7.13)
yatank-online (0.0.2)

Possible solution are here thread/captain#1 (comment)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.