ExtensiveAutomation enable you to create custom workflows to automate your project.
- a workflow is the combination of differents actions.
- an action is individual python code source with enriched parameters.
The architecture is composed of 3 parts:
- Server Installation
- Server is running fine
- Understand the Data Storage
- Working with actions
- Working with workflows
- Automation using the Web Interface
- Automation using the REST API
- More security
- Migration from old version
- About
The server is the main part of the ExtensiveAutomation project.
It's is running on the following tcp ports:
- tcp/8081: REST API
- tcp/8082: Websocket tunnel for app client
- tcp/8083: Websocket tunnel for agents
A user account is required, you can use the default ones or create your own account. Default user accounts and passwords:
- admin/password
- tester/password
- monitor/password
YAML files storage can be split into different workspaces.
The Common workspace is available by default, attached to the previous users.
-
Run the following command
python3 -m pip install extensiveautomation_server
-
Type the following command on your shell to start the server
extensiveautomation --start
-
Finally, check if the server is running fine.
-
Downloading the image
docker pull extensiveautomation/extensiveautomation-server:latest
-
Start the container
docker run -d -p 8081:8081 -p 8082:8082 -p 8083:8083 \ --name=extensive extensiveautomation/extensiveautomation-server
If you want to start the container with persistant tests data, go to Docker Hub page.
-
Finally, check if the server is running fine.
-
Clone this repository on your linux server
git clone https://github.com/ExtensiveAutomation/extensiveautomation-server.git cd extensiveautomation-server/ -
As precondition, install the additional python libraries with
pipcommand:python3 -m pip install -r requirements.txt
-
Start the server. On linux the server is running as daemon.
cd src/ python3 extensiveautomation.py --start -
Finally, check if the server is running fine.
You can add plugins to your extensive automation server to add more functionnalities like
make http requests, send ssh commands. By default the server comes without plugins so you need
to install them one by one according to your needs with pip commands.
Main plugins:
Checking if the REST api working fine using curl or postman.
curl -s -X POST http://127.0.0.1:8081/session/login \
-H "Content-Type: application/json" \
-d '{"login": "admin", "password": "password"}' | jq .success response:
{
"cmd": "/session/login",
"message": "Logged in",
"session_id": "MjA1OWI1OTc1MWM0NDU2NDg4MjQxMjRjNWFmN2FkNThhO",
"expires": 86400,
"user_id": 1,
"levels": ["Administrator"],
"project_id": 1,
"api_login": "admin",
"api_secret": "6977aa6a443bd3a6033ebb52557cf90d24c79857",
"client-available": false,
"version": "",
"name": ""
}All data necessary for the server is stored in a specific folder. The location of the storage can be found with the following command:
extensiveautomation --show-data-path
/<install_project>/ea/var/Data storage overview:
var/
tests/
<project_id>/
[...yaml files...]
testsresult/
<project_id>/
<result_id>/
logs/
output.log
data.dbAction is individual python code source with parameters and must be defined with YAML file.
You can create your own actions but some actions are available by default in the folder /actions
Actions must be defined with YAML file.
The default ones:
- basic/helloworld.yml
- basic/wait.yml
- cache/log.yml
- generator/random_string.yml
- generator/random_integer.yml
Additional actions are available with plugins:
This following action is available in the data storage in "/actions/basic/" folder. This basic action shows how to write python source code with parameters in YAML format.
properties:
parameters:
- name: msg
value: hello world
python: |
class HelloWorld(Action):
def definition(self):
self.info(input("msg"))
HelloWorld().execute()A workflow is the combination of differents actions and must be defined with YAML file. Parameters from actions can be easily overwritten and conditions between actions can be defined.
You can create your own workflows but some workflows are available by default in the folder /workflows
Workflows must be defined with YAML file.
The default ones:
Additional workflows are available with plugins:
This following workflow is available in the data storage in "/workflows/basic/" folder. This workflow shows how to use actions with updated parameters.
actions:
- description: HelloWorld
file: Common:actions/basic/helloworld.yml
parameters:
- name: msg
value: Hola MundoThis example describe how to write a ssh workflow to execute some commands remotely using SSH. The SSH plugin must be installed, please refer to the chapter Install plugins.
The following example show how to execute remote ssh commands.
actions:
- description: execute commands remotely using SSH
file: Common:actions/ssh/send_commands.yml
parameters:
- name: ssh-hosts
value:
- ssh-host: 10.0.0.55
ssh-login: root
ssh-password: ESI23xgx4yYukF9rsA1O
- name: ssh-commands
value: |-
echo "hello world" >> /var/log/messages
echo "hola mondu" >> /var/log/messagesIf your want to run the previous workflow with a remote agent:
- you must deploy a
sshagent - you must define the agent to use in your worflow like below
properties:
parameters:
- name: agent
value: agent01.sshAdditional examples are available in the data storage ./workflows/ssh/.
This example describe how to write a HTTP workflow to send HTTP requests. The WEB plugin must be installed, please refer to the chapter install plugins.
The following example show how to send a http request and waiting response with specific json key.
actions:
- description: Get my origin IP
file: Common:actions/http/curl.yml
parameters:
- name: curl-hosts
value: https://httpbin.org/ip
- name: response-body-json
value: |
origin -> [!CAPTURE:externalip:]
- description: Log external IP
file: Common:actions/cache/log.yml
parameters:
- name: key
value: externalipIf your want to run the previous workflow with a remote agent:
- you must deploy a
curlagent - you must define the agent to use in your worflow like below
properties:
parameters:
- name: agent
value: agent01.curlAdditional examples are available in the data storage ./workflows/http/.
This example describe how to write a Selenium workflow.
The GUI plugin must be installed, please refer to the chapter install plugins.
and the selenium agent must installed
and running.
The following example show how to open the web browser, that loads the url www.google.com and finally close-it after.
properties:
parameters:
- name: agent
value: agent01.selenium
- name: browser
value: chrome
actions:
- description: open browser
file: Common:actions/selenium/openbrowser.yml
id: 1
parameters:
- name: url
value: https://www.google.com
- description: close browser
file: Common:actions/selenium/closebrowser.yml
id: 2
parent: 1Additionals examples are available in the data storage ./workflows/selenium/.
This example describe how to write a Sikulix workflow. Sikulix is nice to simulate keyboard and mouse interactions on your system.
The GUI plugin must be installed, please refer to the chapter install plugins.
and the sikulix agent must installed
and running.
The following example show how to execute Windows + R shorcut and run a command.
properties:
parameters:
- name: agent
value: agent01.sikulix
actions:
- description: Type Windows+R on keyboard
file: Common:actions/sikulix/type_shorcut.yml
id: 1
parameters:
- name: key
value: KEY_WIN
- name: other-key
value: r
- description: "Type text: cmd"
file: Common:actions/sikulix/type_text.yml
id: 1
parent: 1
parameters:
- name: text
value: cmdThe web client is optional because you can do everything from the REST API of the server. But on some cases, it's more user friendly to use the web interface to manage:
- users
- projects
- variables
- and more...
Go to the menu Automation > Job > Add Job
Select the your action or worflow and click on the button CREATE
Go to the menu Automation > Executions and display Logs
You can do everything from the REST API, below a small overview. Take a look to the swagger.
Get the API secret for the user admin
extensiveautomation --generate-api-key admin
API key: admin
API secret: 6977aa6a443bd3a6033ebb52557cf90d24c79857Make a POST on /v1/jobs with basic auth to create a job wich will execute your actions or workflows.
Copy/Paste the following curl command:
curl -s --user admin:6977aa6a443bd3a6033ebb52557cf90d24c79857 \
-d '{"yaml-file": "/workflows/basic/helloworld.yml"}' \
-H "Content-Type: application/json" \
-X POST http://127.0.0.1:8081/v1/jobs?workspace=1 | jq .Success response:
{
"cmd": "/v1/jobs",
"message": "background",
"job-id": 2,
"execution-id": "e57aaa43-325d-468d-8cac-f1dea822ef3a"
}Make a GET on /v1/executions with basic auth to get logs generated by the job.
Copy/Paste the following curl command:
curl -s --user admin:6977aa6a443bd3a6033ebb52557cf90d24c79857 \
"http://127.0.0.1:8081/v1/executions?workspace=1&id=eab41766-c9b6-4632-8a73-42232a431051" | jq .Success response:
{
"cmd": "/v1/executions",
"execution-id": "e57aaa43-325d-468d-8cac-f1dea822ef3a",
"status": "complete",
"verdict": "pass",
"logs": "10:50:10.7241 task-started
10:50:10.7243 script-started helloworld
10:50:10.7309 script-stopped PASS 0.007
10:50:10.7375 task-stopped 0.006909608840942383",
"logs-index": 156
}Adding a reverse proxy in the front of server enables to expose only one tcp port (8080) and to have a TLS link between the client and the server. Also, the default behaviour of the QT client and toolbox is to try to connect on the tcp/8080 port with ssl (can be modifed).
If you want to install a reverse proxy, please to follow this procedure.
-
Install the example provided
scripts\reverseproxy\extensiveautomation_api.confin your apache instance. If you install the reverse proxy on a new server, don't forget to replace the 127.0.0.1 address by the ip of your extensive server.Listen 8080 <VirtualHost *:8080> SSLEngine on SSLCertificateFile /etc/pki/tls/certs/localhost.crt SSLCertificateKeyFile /etc/pki/tls/private/localhost.key LogLevel warn ErrorLog /var/log/extensiveautomation_api_error_ssl_rp.log CustomLog /var/log/extensiveautomation_api_access_ssl_rp.log combined Redirect 307 / /rest/session/login ProxyPass /rest/ http://127.0.0.1:8081/ ProxyPassReverse /rest/ http://127.0.0.1:8081/ ProxyPass /wss/client/ ws://127.0.0.1:8082 disablereuse=on ProxyPassReverse /wss/client/ ws://127.0.0.1:8082 disablereuse=on ProxyPass /wss/agent/ ws://127.0.0.1:8083 disablereuse=on ProxyPassReverse /wss/agent/ ws://127.0.0.1:8083 disablereuse=on </VirtualHost>
With this configuration in apache, the REST API is now running on the port tcp/8080 (tls).
-
Checking if the REST api working fine with curl command.
curl -X POST https://127.0.0.1:8080/rest/session/login --insecure \ -H "Content-Type: application/json" \ -d '{"login": "admin", "password": "password"}'
By default, users are authenticated locally from database (by checking hash password). This behavior can be modified by using a remote authentication server. In this mode, you always need to add users in the local database.
Follow this procedure to enable LDAP authentication:
-
Install python dependancies with the
pipcommand:python3 -m pip install ldap3
-
Configure the
settings.inifile to enable ldap authentication and other stuff[Users_Session] ; enable ldap user authentication for rest api session only ; 0=disable 1=enable ldap-authbind=1 ; remote addresses of your ldap servers ; ldaps://127.0.0.1:636 ldap-host=[ "ldap://127.0.0.1:389" ] ; username form ; uid=%%s,ou=People,dc=extensive,dc=local ldap-dn=[ "uid=%%s,ou=People,dc=extensive,dc=local" ]
-
Restart the server
cd src/ python3 extensiveautomation.py --stop python3 extensiveautomation.py --start -
Check the new user authentication method
curl -X POST http://127.0.0.1:8081/session/login \ -H "Content-Type: application/json" \ -d '{"login": "admin", "password": "password"}'
Since version 22 of the server, a major change has been introduced on the test files. All the old tests in XML can still be used but they are obsolete. We must favor the new YAML format.
XML to YAML conversion can be done with the following command. A new YML file will be created automatically after converting the XML reading.
extensiveautomation --convert-to-yamlThis project is an effort, driven in my spare time.
| Copyright | Copyright (c) 2010-2023 Denis Machard [email protected] |
| License | MIT |
| Docs | https://extensiveautomation.readthedocs.io/en/latest/ |
| Github | https://github.com/ExtensiveAutomation |
| Docker Hub | https://hub.docker.com/u/extensiveautomation |
| PyPI | https://pypi.org/project/extensiveautomation-server/ |
| Google Users | https://groups.google.com/group/extensive-automation-users |
| https://twitter.com/Extensive_Auto |
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent