elastic / beats-tester Goto Github PK

E2E tests for the Beats

PowerShell 0.52% Shell 20.80% Makefile 23.43% Groovy 25.32% Jinja 29.94%

beats-tester's Introduction

The Beats Tester

Vagrant + Ansible setup for testing the OS packages and basic e2e tests for all the Beats.

OS matrix

OS	Support
awslinux	✅
awslinux2	✅
centos7-64	✅
centos8-64	✅
debian10-64	✅
debian8-64	✅
debian9-64	✅
sles12-64	✅
ubuntu1804-64	✅
ubuntu2004-64	✅
win12-64	✅

Requirements

Python version <= 3.8
Ansible version <= 2.8.8

Those versions are defined in the requirements.txt.

Execute

First, for Linux and macOS, you need to enable some IP address range for the host-only network. To do so, create (or edit) /etc/vbox/networks.conf and add the following range there:

192.168.33.0/24

For more information, check the official documentation

Second, you need to bring the machines up:

make setup

Then you can use Ansible to run the tests. Because they involve lots of VMs and commands executed over SSH, these tests are slow (currently 15 minutes in total). However, while creating tests or checking something quickly, you can use different Ansible commands to execute only a subset.

Here are some execution examples:

Test the most recent nightly builds (excluding OS X):
```
  RUN_SETTINGS=snapshot make run
```

All tests, all platforms, a particular release:

  # Edit version contained in run-settings-released.yml.
  RUN_SETTINGS=released make run

Only a particular Beat, Packetbeat in the example:

  export ANSIBLE_EXTRA_FLAGS="--tags packetbeat"
  make run

Only a particular OS, Debian 8 amd64 in the example:

  # Instead using 'make setup' launch vagrant machines manually.
  machine=tester-debian8-64
  vagrant up $machine
  vagrant ssh-config $machine > ssh_config
  export ANSIBLE_LIMIT="$machine"
  make run

Only a set of ansible groups in batches:

  # Run tests in batches for all hosts
  make batch

  # Run tests in batches for all Linux hosts
  GROUPS='centos debian sles' make batch

  # Run tests in batches for all Debian hosts
  GROUPS=debian make batch

  # Run tests in batches for Windows hosts
  GROUPS=windows make batch

Enable Ansible debug.

  export ANSIBLE_VERBOSE="-vvv"
  make run

Executing tests on your local macOS machine. In System Preferences -> Sharing enable Remote Login (SSH). Don't forget to disable Remote Login when you are done.
```
  export ANSIBLE_EXTRA_FLAGS="--ask-sudo-pass"
  export ANSIBLE_LIMIT="localhost"
  make run
```
To run just the APM Server tests, set ANSIBLE_EXTRA_FLAGS="--tags apm-server" in the environment.

Cleanup

You need to tear down the VMs when you are finished.

make clean

beats-tester's People

Contributors

Stargazers

Watchers

Forkers

tsg ruflin andrewkroh forging2012 adriansr ph kvch elasticdog urso hutingyue cwurm mikemadden42 elwpenn v1v cachedout isabella232 mdelapenya chrsmark axw simitt belimawr

beats-tester's Issues

Add Winlogbeat Tests

With the upcoming release of Winlogbeat, we need to add tests for it on Windows.

Align operating system coverage with current support matrix

We should align our operating system coverage with the current support matrix.

It looks like we are missing the following operating systems:

Ubuntu 14.04
Windows Server 2016

Additionally these operating systems are not included in the support matrix, but they are covered by our tests.

openSUSE 42
Ubuntu 12.04

Add alerts to beats-tester master Jenkins job

Currently when a beats-tester Jenkins job fails, there is no alert for the failure. Configure email and Slack alerts.

Add CI Pipeline

The current CI pipeline works only for the master builds. So let's add support for the PRs and tags with the multibranch pipeline.

Tasks

Create JJBB
Create CI Pipeline
Create JJBB automation in the infra side.
Decommission the existing job

Add SLES12 testing

Add a SLES 12 virtual machine to the testing environment.

https://app.vagrantup.com/elastic/boxes/sles-12-x86_64

Implement OS X Testing

Currently beats-tester needs for OS X testing an Mac OS X machine with ssh enabled running the test as it access this machine through localhost:

beats-tester/hosts

Line 14 in 23c5a0c

localhost

A way should be found to have OS X as virtual machine or a similar way to not make testing dependent on a OS X machine that is set up.

Add support for older OSes

Because of the nature of beats as a whole-fleet utility, we should expand our packaging tests platforms. I would say at minimum:

And probably:

Use config files from beats-packer for tests

See issue: https://github.com/elastic/filebeat/pull/150

See comment for potential problems: https://github.com/elastic/filebeat/pull/150#issuecomment-151532590

"Start apm-server Windows service" task is failing

What

The "Start apm-server Windows service" task is failing

beats-tester/roles/test-beat/tasks/win32nt/main.yml

Lines 27 to 30 in eb09f0d

 - name: "Start {{ beat_name }} Windows service" 

 win_service: 

 name: '{{ beat_name }}' 

 state: started

Version

It uses the latest 8.0-SNAPSHOT version stored in gs://apm-ci-artifacts/snapshots/

Log

Error
Service 'apm-server (apm-server)' cannot be started due to the following error: Cannot start service apm-server on computer '.'.
Stacktrace
{
"can_pause_and_continue": false, 
"changed": false, 
"depended_by": [], 
"dependencies": [], 
"description": "", 
"desktop_interact": false, 
"display_name": "apm-server", 
"exists": true, 
"msg": "Service 'apm-server (apm-server)' cannot be started due to the following error: Cannot start service apm-server on computer '.'.", 
"name": "apm-server", 
"path": "\"C:\\Users\\Administrator\\apm-server-8.0.0-SNAPSHOT-windows-x86\\apm-server.exe\" --environment=windows_service -c \"C:\\Users\\Administrator\\apm-server-8.0.0-SNAPSHOT-windows-x86\\apm-server.yml\" --path.home \"C:\\Users\\Administrator\\apm-server-8.0.0-SNAPSHOT-windows-x86\" --path.data \"C:\\ProgramData\\apm-server\" --path.logs \"C:\\ProgramData\\apm-server\\logs\" -E logging.files.redirect_stderr=true", 
"start_mode": "delayed", 
"state": "stopped", 
"username": "LocalSystem"
}

[2021-06-15T16:11:57.866Z] TASK [test-beat : Start apm-server Windows service] ****************************
[2021-06-15T16:11:59.306Z] fatal: [tester-win12-64]: FAILED! => {"can_pause_and_continue": false, "changed": false, "depended_by": [], "dependencies": [], "description": "", "desktop_interact": false, "display_name": "apm-server", "exists": true, "msg": "Service 'apm-server (apm-server)' cannot be started due to the following error: Cannot start service apm-server on computer '.'.", "name": "apm-server", "path": "\"C:\\Users\\Administrator\\apm-server-8.0.0-SNAPSHOT-windows-x86\\apm-server.exe\" --environment=windows_service -c \"C:\\Users\\Administrator\\apm-server-8.0.0-SNAPSHOT-windows-x86\\apm-server.yml\" --path.home \"C:\\Users\\Administrator\\apm-server-8.0.0-SNAPSHOT-windows-x86\" --path.data \"C:\\ProgramData\\apm-server\" --path.logs \"C:\\ProgramData\\apm-server\\logs\" -E logging.files.redirect_stderr=true", "start_mode": "delayed", "state": "stopped", "username": "LocalSystem"}
[2021-06-15T16:11:59.306Z] 
[2021-06-15T16:11:59.306Z] TASK [test-beat : Save output to host] *****************************************
[2021-06-15T16:12:01.556Z] fatal: [tester-win12-64]: FAILED! => {"changed": false, "msg": "Path c:\\ProgramData\\apm-server\\logs\\output.json is not found"}
[2021-06-15T16:12:01.556Z]

You can see the whole log output in -> https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/pipelines/beats-tester-mbp/branches/master/runs/476/nodes/205/log/?start=0

How to reproduce it

$ GROUP= windows make run-group

beats-tester jobs are failing for beats and apm-server due to changes in output.file handling in beats

Currently beats-tester CI is failing for beats and apm-server because of an invalid output file name.

Seems to be related to changes from elastic/beats#28927

Add tests for beat.fd.* metrics

As requested in elastic/beats#7986 (comment)

Add support for RHEL 8

RHEL 8 was released this week. We should integrate this vagrant box for RHEL 8 into beats-tester.

https://app.vagrantup.com/elastic/boxes/rhel-8-x86_64

[artifactory] use artifactory vagrant boxes

Caused by #170

Change the Vagrantfile file to support the artifactory proxy but with the following considerations:

people can fetch those vagrant boxes.
people can configure a different proxy.

The above requirements will help for the cases that users of this repo do want o don't want to use our artifactory proxy.

Assert the correct license for the beats

We have accidentally packaged an OSS binary inside an Elastic artifact, we want to make changes in our build process to include the license when we call the version subcommand at the same time we should add an assert in this project to make sure the right binaries are correctly packaged.

Add tests for MacOS DMG files

Relates: elastic/beats#7481

Test failure of Auditbeat on Suse and SLES

The package dataset fails and then Auditbeat doesn't start.

@cwurm Should we just not configure those operating systems with the package dataset? Or would the dataset work on these systems if it got passed the OS family check? Ansible classifies these as being in the suse family (so not debian or redhat), and go-sysinfo doesn't classify them at all right now. I think they use Zypper, for package management, but they might also have librpm available.

https://beats-ci.elastic.co/job/elastic+beats-tester+master/382/

Add tests to ensure a Beat binary belongs to the correct distribution (Elastic or oss)

Until elastic/beats#25696, we used the help enroll command in Beats to tell Elastic and oss binaries apart.

Once the enroll feature was removed, we had to remove the checks (#196) we put in place (#105) to ensure each distribution contained the right binary.

In order to keep validating that binaries belong to the expected distribution, we need to find an alternate method to identify the distribution for a Beat binary.

[CI] worker-395930 got some issues with vagrant/virtualbox

What

Allocated worker caused this build error

Worker: worker-395930

I'll disable it for the time being

Stacktrace

[2020-10-02T10:41:59.391Z] Progress state: NS_ERROR_FAILURE
[2020-10-02T10:41:59.391Z] VBoxManage: error: Failed to create the host-only adapter
[2020-10-02T10:41:59.391Z] VBoxManage: error: VBoxNetAdpCtl: Error while adding new interface: VBoxNetAdpCtl: ioctl failed for /dev/vboxnetctl: Cannot allocate memory
[2020-10-02T10:41:59.392Z] VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component HostNetworkInterfaceWrap, interface IHostNetworkInterface
[2020-10-02T10:41:59.392Z] VBoxManage: error: Context: "RTEXITCODE handleCreate(HandlerArg*)" at line 94 of file VBoxManageHostonly.cpp
[2020-10-02T10:41:59.392Z] Makefile:36: recipe for target 'run-group' failed
[2020-10-02T10:41:59.392Z] make[1]: *** [run-group] Error 1
[2020-10-02T10:41:59.392Z] make[1]: Leaving directory '/var/lib/jenkins/workspace/Beats_beats-tester-mbp_master/src'
[2020-10-02T10:41:59.392Z] Makefile:29: recipe for target 'batch' failed
[2020-10-02T10:41:59.392Z] make: *** [batch] Error 1

[Packaging] Beats Tester should run with specific versions rather than latest

Otherwise, the chances the packaging is happening when the beats-tester runs is high and therefore files might be corrupted or their sha files don't match with the binaries.

Let's think about the packaging folder structure that can allow to test every packaging correctly

[centos-6] errors

15:48:59  ==> tester-centos6-32: Running provisioner: shell...
15:48:59      tester-centos6-32: Running: inline script
15:49:00      tester-centos6-32: Loaded plugins: fastestmirror
15:49:00      tester-centos6-32: Setting up Install Process
15:49:00      tester-centos6-32: Determining fastest mirrors
15:49:00      tester-centos6-32: Error: Cannot find a valid baseurl for repo: base
15:49:00      tester-centos6-32: YumRepo Error: All mirror URLs are not using ftp, http[s] or file.
15:49:00      tester-centos6-32:  Eg. Invalid release/repo/arch combination/
15:49:00      tester-centos6-32: removing mirrorlist with no valid mirrors: /var/cache/yum/i386/6/base/mirrorlist.txt

Setup Beats-Tester on Jenkins Slave

Beats-Tester should run on all new builds. For this it needs to be installed on a raw metal machine. This issue is to document / track changes and issues which occurred.

Add tests for filebeat

Add tests for 64-bit Windows binaries

Once Packetbeat 64-bit is building correctly we should update beats-tester to test both 32-bit and 64-bit binaries.

Assertions for Winlogbeat 7 need updated due to ECS

Field names were changed in 7.0 and the assertions need to be updated to match. It should still be able to test Winlogbeat so the assertions need to be version conditional.

TASK [test-beat : debug] *******************************************************
ok: [tester-win12-64] => {
    "wlb_event": {
        "@metadata": {
            "beat": "winlogbeat", 
            "type": "_doc", 
            "version": "8.0.0"
        }, 
        "@timestamp": "2015-01-13T17:26:10.000Z", 
        "agent": {
            "ephemeral_id": "92738272-fc44-49e2-8531-0fd57b816f6a", 
            "hostname": "vagrant-2012-r2", 
            "id": "1b5f9551-a6d1-4011-8339-3c9dfa0853d9", 
            "type": "winlogbeat", 
            "version": "8.0.0"
        }, 
        "ecs": {
            "version": "1.0.0"
        }, 
        "event": {
            "code": 4625, 
            "created": "2019-03-22T23:58:10.561Z", 
            "kind": "event"
        }, 
        "host": {
            "name": "vagrant-2012-r2"
        }, 
        "log": {
            "level": "information"
        }, 
        "message": "The EventSystem sub system is suppressing duplicate event log entries for a duration of 86400 seconds.  The suppression timeout can be controlled by a REG_DWORD value named SuppressDuplicateDuration under the following registry key: HKLM\\Software\\Microsoft\\EventSystem\\EventLog.", 
        "winlog": {
            "api": "wineventlog", 
            "channel": "Application", 
            "computer_name": "windows-n7iefg2", 
            "event_data": {
                "param1": "86400", 
                "param2": "SuppressDuplicateDuration", 
                "param3": "Software\\Microsoft\\EventSystem\\EventLog"
            }, 
            "event_id": 4625, 
            "keywords": [
                "Classic"
            ], 
            "provider_guid": "{899daace-4868-4295-afcd-9eb8fb497561}", 
            "provider_name": "Microsoft-Windows-EventSystem", 
            "record_id": 1, 
            "task": ""
        }
    }
}

TASK [test-beat : assert] ******************************************************
fatal: [tester-win12-64]: FAILED! => {
    "assertion": "'log_name' in wlb_event", 
    "changed": false, 
    "evaluated_to": false, 
    "failed": true
}

Beats tester is failing to stop metricbeat service on Windows

In this PR: #224
We removed Windows as part of the tested OS because Beats-tester was failing to stop metricbeat service and showing the following error message:

[2023-06-14T09:33:18.279Z] TASK [test-beat : Stop metricbeat Windows service] *****************************

[2023-06-14T09:34:50.468Z] fatal: [tester-win12-64]: FAILED! => {"can_pause_and_continue": false, "changed": false, "depended_by": [], "dependencies": [], "description": "", "desktop_interact": false, "display_name": "metricbeat", "exists": true, "msg": "Service 'metricbeat (metricbeat)' cannot be stopped due to the following error: Cannot stop metricbeat service on computer '.'.", "name": "metricbeat", "path": "\"C:\\Users\\Administrator\\metricbeat-8.9.0-SNAPSHOT-windows-x86_64\\metricbeat.exe\" --environment=windows_service -c \"C:\\Users\\Administrator\\metricbeat-8.9.0-SNAPSHOT-windows-x86_64\\metricbeat.yml\" --path.home \"C:\\Users\\Administrator\\metricbeat-8.9.0-SNAPSHOT-windows-x86_64\" --path.data \"C:\\ProgramData\\metricbeat\" --path.logs \"C:\\ProgramData\\metricbeat\\logs\" -E logging.files.redirect_stderr=true", "start_mode": "delayed", "state": "running", "username": "LocalSystem"}

The purpose of this agent is to reenable Windows tests and make them pass

Add tests for module enable/disable

Add tests for the Beats the support module enable and module disable commands. We test this from the Beats' system tests, but testing it here could add value since it will ensure the path.* configs are set correctly and that the permissions and file ownership are correct such that the Beat can rename the files.

See #87 (comment)

Update to latest supported ansible

ansible is currently pinned to 2.4.5 which is end of life. We should update to the latest supported ansible for security and critical bug fixes.

Run beats-tester for release branches

placeholder

Ensure recent Linux distributions are tested

Add version check

A simple test that calls *beat -version and checks that we have the expected version would be helpful.

Tests are failing for the `x-pack/packetbeat help enroll`

What

The test assertion for packetbeat help enroll doesn't contain 'Unknown help topic' is failing

This assertion was introduced a couple of years ago, see #105

Version

It uses the latest 8.0-SNAPSHOT version stored in gs://beats-ci-artifacts/snapshots/packetbeat

Log

[2021-06-15T08:54:22.828Z] TASK [test-beat : Get packetbeat help output for an X-Pack feature] ************
[2021-06-15T08:54:23.526Z] changed: [tester-awslinux]
[2021-06-15T08:54:23.526Z] changed: [tester-centos7-64]
[2021-06-15T08:54:23.526Z] changed: [tester-awslinux2]
[2021-06-15T08:54:23.526Z] changed: [tester-centos8-64]
[2021-06-15T08:54:23.526Z] 
[2021-06-15T08:54:23.526Z] TASK [test-beat : Check that packetbeat contains X-Pack features] **************
[2021-06-15T08:54:23.526Z] fatal: [tester-awslinux]: FAILED! => {
[2021-06-15T08:54:23.526Z]     "assertion": "'Unknown help topic' not in xpack_cmd_result.stderr", 
[2021-06-15T08:54:23.526Z]     "changed": false, 
[2021-06-15T08:54:23.526Z]     "evaluated_to": false, 
[2021-06-15T08:54:23.526Z]     "msg": "Assertion failed"
[2021-06-15T08:54:23.526Z] }
[2021-06-15T08:54:23.526Z] fatal: [tester-awslinux2]: FAILED! => {
[2021-06-15T08:54:23.526Z]     "assertion": "'Unknown help topic' not in xpack_cmd_result.stderr", 
[2021-06-15T08:54:23.526Z]     "changed": false, 
[2021-06-15T08:54:23.526Z]     "evaluated_to": false, 
[2021-06-15T08:54:23.526Z]     "msg": "Assertion failed"
[2021-06-15T08:54:23.526Z] }
[2021-06-15T08:54:23.526Z] fatal: [tester-centos7-64]: FAILED! => {
[2021-06-15T08:54:23.526Z]     "assertion": "'Unknown help topic' not in xpack_cmd_result.stderr", 
[2021-06-15T08:54:23.526Z]     "changed": false, 
[2021-06-15T08:54:23.526Z]     "evaluated_to": false, 
[2021-06-15T08:54:23.526Z]     "msg": "Assertion failed"
[2021-06-15T08:54:23.526Z] }
[2021-06-15T08:54:23.526Z] fatal: [tester-centos8-64]: FAILED! => {
[2021-06-15T08:54:23.526Z]     "assertion": "'Unknown help topic' not in xpack_cmd_result.stderr", 
[2021-06-15T08:54:23.526Z]     "changed": false, 
[2021-06-15T08:54:23.526Z]     "evaluated_to": false, 
[2021-06-15T08:54:23.526Z]     "msg": "Assertion failed"
[2021-06-15T08:54:23.526Z] }

You can see the whole log output in -> https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/pipelines/beats-tester-mbp/pipelines/PR-193/runs/1/nodes/93/steps/133/log/?start=0

How to reproduce it

$ GROUP=centos make run-group

Fix warnings and deprecations with latest ansible for beats-tester

I ran into these warnings & deprecations when I updated beats-tester for ansible 2.7.10.

WARNINGS

 [WARNING]: Consider using the unarchive module rather than running 'tar'.  If you need to use command because unarchive is insufficient you can add 'warn: false' to
this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.
 [WARNING]: Consider using the get_url or uri module rather than running 'wget'.  If you need to use command because get_url or uri is insufficient you can add 'warn:
false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.
 [WARNING]: Consider using the service module rather than running 'service'.  If you need to use command because service is insufficient you can add 'warn: false' to
this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.
 [WARNING]: Consider using the unarchive module rather than running 'tar'.  If you need to use command because unarchive is insufficient you can add 'warn: false' to
this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.
 [WARNING]: Consider using the get_url or uri module rather than running 'wget'.  If you need to use command because get_url or uri is insufficient you can add 'warn:
false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.
 [WARNING]: Consider using the service module rather than running 'service'.  If you need to use command because service is insufficient you can add 'warn: false' to
this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.

DEPRECATIONS

[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|version_compare` use `result is version_compare`. This feature will be removed in
version 2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

Update for ansible 2.9.x for beats-tester

Ansible 2.9.0 has been released. We need to ensure this works with beats-tester.

https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.9.html

yum issues with awslinux2 box

It looks like there's a timing issue with the awslinux2 box. The tests fail the first time the box is booted up. After the box is running, new tests work fine.

$ cat ~/bin/test_box.sh                                                                                                           master
#!/bin/sh

machine=$1
vagrant up $machine
vagrant ssh-config $machine >ssh_config
export ANSIBLE_LIMIT="$machine"
make run

# Boot up the box; tests have issues with yum being locked.
$ test_box.sh tester-awslinux2
...
...
****************************************************************************************************************
fatal: [tester-awslinux2]: FAILED! => {"changed": false, "msg": "yum lockfile is held by another process"}

PLAY RECAP **************************************************************************************************************************************************************
tester-awslinux2           : ok=6    changed=3    unreachable=0    failed=1    skipped=7    rescued=0    ignored=0

# Box is already up; tests have no issues.
$ test_box.sh tester-awslinux2
...
...
PLAY RECAP **************************************************************************************************************************************************************
tester-awslinux2           : ok=319  changed=145  unreachable=0    failed=0    skipped=189  rescued=0    ignored=0

# Destroy any boxes;  tests have yum lock issues issues again
$ make clean                                                                                                                      master
vagrant destroy -f
$ test_box.sh tester-awslinux2
...
...
fatal: [tester-awslinux2]: FAILED! => {"changed": false, "msg": "yum lockfile is held by another process"}

PLAY RECAP **************************************************************************************************************************************************************
tester-awslinux2           : ok=6    changed=3    unreachable=0    failed=1    skipped=7    rescued=0    ignored=0

Use official images from Elastic for testing

Whenever possible, we should use official images from Elastic for testing.

The current images are being pulled from various sources, and some of the images are outdated.

centos 6 32-bit - source is puppetlabs and version is 6.5
centos 6 64-bit - source is puppetlabs and version is 6.5
centos 7 64-bit - source is relativkreativ and version is unknown
debain 8 64-bit - source is debian
debain 9 64-bit - source is debian
ubuntu 16.04 64-bit - source is ubuntu
awslinux 64-bit - source is mvbcoding and version is unknown
windows server 2012 64-bit - source is s3

[artifactory] mirror vagrant boxes

What

This should fix the issue we have faced when the vagrant box has gone:

#169

Let's use artifactory to cache all the vagrant boxes we use in this project.

Follow-ups

#171 should be the next step to be done

Tests are failing for the `Wait for the output file to be created, should contain HTTP pinging`

What

The test assertion for hearbeat contains "scheme":"http" is failing

This assertion was introduced a long time ago:

beats-tester/roles/test-linux-binary/tasks/main.yml

Lines 74 to 78 in eb09f0d

 - name: Wait for the output file to be created, should contain HTTP pinging 

 wait_for: > 

  path={{workdir}}/output/heartbeat timeout=5 

  search_regex='"scheme":"http"' 

  when: beat_name == "heartbeat"

0022c19

Version

It uses the latest 8.0-SNAPSHOT version stored in gs://beats-ci-artifacts/snapshots/heartbeat

Log

Error
Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat
Stacktrace
{
"changed": false, 
"elapsed": 5, 
"msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat"
}

[2021-06-15T16:04:10.902Z] TASK [test-linux-binary : Wait for the output file to be created, should contain HTTP pinging] ***
[2021-06-15T16:04:16.070Z] fatal: [tester-awslinux]: FAILED! => {"changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat"}
[2021-06-15T16:04:16.070Z] fatal: [tester-centos7-64]: FAILED! => {"changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat"}
[2021-06-15T16:04:16.070Z] fatal: [tester-awslinux2]: FAILED! => {"changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat"}
[2021-06-15T16:04:16.070Z] fatal: [tester-centos8-64]: FAILED! => {"changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat"}

You can see the whole log output in -> https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/pipelines/beats-tester-mbp/branches/master/runs/476/nodes/77/log/?start=0

How to reproduce it

$ GROUP=centos make run-group

Remove debian6

Debian 6 uses a very old OpenSSL lib, which doesn't support TLS 1.2. Downloading snapshots requires TLS 1.2 minimum though.

Support matrix states we support debian 8+. When removing debian 6, we should at least introduce debian 8.

Add support for new OS

We have to add the new images we created to test on the CI to the beats tester.

	Beats CI	Beats tester
CentOS/RHEL 6.5+/7.x (64 bits)	x	x
CentOS/RHEL 8 (64 bits)	x	x
Ubuntu 14.04 (32 bits)	x
Ubuntu 14.04 (64 bits)	x	x
Ubuntu 16.04 (64 bits)	x	x
Ubuntu 18.04 (64 bits)	x	x
Ubuntu 20.04 (64 bits)	x
Debian 8 (64 bits)	x	x
Debian 9 (64 bits)	x	x
Debian 10 (64 bits)	x	x
Windows 7 SP1 (64bits)	x
Windows 7 SP1 (32bits)	x
Windows 8.1 (64 bits)	x
Windows 10 (64 bits)	x
Windows Server 2008/R2 (64bits)	x
Windows Server 2012/R2 (64bits)	x	x
Windows Server 2016	x
Windows Server 2019	x	x

TASK [common : Download package sha1] ******************************************
fatal: [tester-ubuntu1204-32]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Shared connection to 127.0.0.1 closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_7IOM02/ansible_module_get_url.py\", line 471, in <module>\r\n    from ansible.module_utils.urls import *\r\n  File \"/tmp/ansible_7IOM02/ansible_modlib.zip/ansible/module_utils/urls.py\", line 174, in <module>\r\nAttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_2'\r\n", "msg": "MODULE FAILURE", "rc": 0}
fatal: [tester-centos6-64]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Shared connection to 127.0.0.1 closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_87Aek_/ansible_module_get_url.py\", line 471, in <module>\r\n    from ansible.module_utils.urls import *\r\n  File \"/tmp/ansible_87Aek_/ansible_modlib.zip/ansible/module_utils/urls.py\", line 174, in <module>\r\nAttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_2'\r\n", "msg": "MODULE FAILURE", "rc": 0}
fatal: [tester-centos6-32]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Shared connection to 127.0.0.1 closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_ax2R2N/ansible_module_get_url.py\", line 471, in <module>\r\n    from ansible.module_utils.urls import *\r\n  File \"/tmp/ansible_ax2R2N/ansible_modlib.zip/ansible/module_utils/urls.py\", line 174, in <module>\r\nAttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_2'\r\n", "msg": "MODULE FAILURE", "rc": 0}
fatal: [tester-debian6-64]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Shared connection to 127.0.0.1 closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible__4rPqL/ansible_module_get_url.py\", line 471, in <module>\r\n    from ansible.module_utils.urls import *\r\n  File \"/tmp/ansible__4rPqL/ansible_modlib.zip/ansible/module_utils/urls.py\", line 174, in <module>\r\nAttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_2'\r\n", "msg": "MODULE FAILURE", "rc": 0}
fatal: [tester-centos7-64]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Shared connection to 127.0.0.1 closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_7JxfKN/ansible_module_get_url.py\", line 471, in <module>\r\n    from ansible.module_utils.urls import *\r\n  File \"/tmp/ansible_7JxfKN/ansible_modlib.zip/ansible/module_utils/urls.py\", line 174, in <module>\r\nAttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_2'\r\n", "msg": "MODULE FAILURE", "rc": 0}
	to retry, use: --limit @/var/lib/jenkins/workspace/elastic+beats-tester+master/site.retry

https://beats-ci.elastic.co/job/elastic+beats-tester+master/7/console

	- name: "Start {{ beat_name }} Windows service"
	win_service:
	name: '{{ beat_name }}'
	state: started

	- name: Wait for the output file to be created, should contain HTTP pinging
	wait_for: >
	path={{workdir}}/output/heartbeat timeout=5
	search_regex='"scheme":"http"'
	when: beat_name == "heartbeat"

elastic / beats-tester Goto Github PK

beats-tester's Introduction

The Beats Tester

OS matrix

Requirements

Execute

Cleanup

beats-tester's People

Contributors

Stargazers

Watchers

Forkers

beats-tester's Issues

Tasks

What

Version

Log

How to reproduce it

What

Stacktrace

What

Version

Log

How to reproduce it

What

Follow-ups

What

Version

Log

How to reproduce it

Recommend Projects

Recommend Topics

Recommend Org