Giter Club home page Giter Club logo

Comments (8)

JonoRicci avatar JonoRicci commented on June 12, 2024

Reproducing the error with Chef-Splunk Kitchen test with EC2-driver

I can run the chef-splunk kitchen test with dokken successfully without reproducing the error.

If I swap the dokken driver with the ec2 driver and add a very simple Inspec test I can reproduce our error in the chef-splunk cookbook. (I ran the Inspec test in the dokken driver which resulted with the same outcome).

The Inspec test:

describe service('splunk') do
  it { should be_installed }
  it { should be_enabled }
  it { should be_running }
end

The result:

  Service splunk
     ✔  is expected to be installed
     ✔  is expected to be enabled
     ×  is expected to be running
     expected that `Service splunk` is running

Investigating on the instance:

Shell output
ubuntu@ip-10-0-0-150:~$ systemctl status splunk.service
● splunk.service - Splunk
     Loaded: loaded (/etc/systemd/system/splunk.service; enabled; vendor preset: enabled)
     Active: inactive (dead)
ubuntu@ip-10-0-0-150:~$ sudo systemctl start splunk.service
Job for splunk.service failed because the service did not take the steps required by its unit configuration.
See "systemctl status splunk.service" and "journalctl -xe" for details.
ubuntu@ip-10-0-0-150:~$ systemctl status splunk.service
● splunk.service - Splunk
     Loaded: loaded (/etc/systemd/system/splunk.service; enabled; vendor preset: enabled)
     Active: failed (Result: protocol) since Thu 2020-10-15 13:33:28 UTC; 2s ago
    Process: 3266 ExecStart=/opt/splunkforwarder/bin/splunk start --answer-yes --no-prompt (code=exited, status=0/SUCCESS)

Oct 15 13:33:27 ip-10-0-0-150 systemd[1]: Starting Splunk...
Oct 15 13:33:28 ip-10-0-0-150 splunk[3266]: The splunk daemon (splunkd) is already running.
Oct 15 13:33:28 ip-10-0-0-150 systemd[1]: splunk.service: Refusing to accept PID outside of service control group, acquired through unsafe symlink chain: /opt/splunkforwarder>
Oct 15 13:33:28 ip-10-0-0-150 systemd[1]: splunk.service: Refusing to accept PID outside of service control group, acquired through unsafe symlink chain: /opt/splunkforwarder>
Oct 15 13:33:28 ip-10-0-0-150 systemd[1]: splunk.service: Failed with result 'protocol'.
Oct 15 13:33:28 ip-10-0-0-150 systemd[1]: Failed to start Splunk.

kitchen.yml

kitchen.yml
---
driver:
  name: ec2
  region: eu-west-1
  interface: public
  instance_type: t2.medium
  require_chef_omnibus: true
  subnet_filter:
    ...
  security_group_filter:
    ...
  tags:
    ...

transport:
  max_threads: 5
  connection_timeout: 10
  connection_retries: 36
  connection_retry_sleep: 10
  max_wait_until_ready: 1200

provisioner:
  name: chef_zero
  log_level: auto
  product_name: chef
  product_version: 14
  max_retries: 3
  wait_for_retry: 90
  retry_on_exit_code:
    - 35 # chef-client's reboot scheduled exit status
  chef_license: accept
  attributes:
    dev_mode: true
    splunk:
      accept_license: true
      enable_ssl: false
      ssl_options:
        enableSplunkWebSSL: 0
        httpport: 8000
        startwebserver: 1
      web_port: 8000

verifier:
  name: inspec
  sudo: true
  root_path: '/opt/verifier'

platforms:
  - name: ubuntu-2004
    driver:
      image_search:
        owner-id: "099720109477"
        name: ubuntu/images/*/ubuntu-*-20.04*
  - name: ubuntu-1804
    driver:
      image_search:
        owner-id: "099720109477"
        name: ubuntu/images/*/ubuntu-*-18.04*
  - name: ubuntu-1604
    driver:
      image_search:
        owner-id: "099720109477"
        name: ubuntu/images/*/ubuntu-*-16.04*

suites:
  - name: client
    run_list:
      - recipe[chef-splunk::default]
    attributes:
      dev_mode: true
      splunk:
        accept_license: true
    verifier:
      inspec_tests:
        - path: test/integration/default

from chef-splunk.

haidangwa avatar haidangwa commented on June 12, 2024

@JonoRicci can you show what your systemd unit file looks like? If you're calling the client recipe directly, you may not be setting up the splunk auth attributes. There is logic in the default recipe that reads the splunk admin user/pass from a data bag or from chef-vault.

from chef-splunk.

haidangwa avatar haidangwa commented on June 12, 2024

you need to have this in an encrypted data bag or chef-vault item:

vault_item = chef_vault_item(node['splunk']['data_bag'], "splunk_#{node.chef_environment}")

from chef-splunk.

jjm avatar jjm commented on June 12, 2024

Hi @haidangwa, I've created PR #186 that adds some inspec tests to chef-splunk for the client suite that shows the issue we are seeing without our wrapper cookbook.

The output of verify on ubuntu-2004 is as follows:

  System Package splunkforwarder
     ✔  should be installed
  Service splunk
     ✔  should be installed
     ✔  should be enabled
     ×  should be running
     expected that `Service splunk` is running
  Port 8089
     ✔  should be listening
     ✔  protocols should include "tcp"
  Processes splunkd
     ✔  should exist

Test Summary: 6 successful, 1 failure, 0 skipped

To me the cause seems with the starting of splunk to accept the license, if I login to the docker container and stop splunk with /opt/splunkforwarder/bin/splunk stop and service splunk start. All the tests pass

  System Package splunkforwarder
     ✔  should be installed
  Service splunk
     ✔  should be installed
     ✔  should be enabled
     ✔  should be running
  Port 8089
     ✔  should be listening
     ✔  protocols should include "tcp"
  Processes splunkd
     ✔  should exist

Test Summary: 7 successful, 0 failures, 0 skipped

Edit: Made it clearer we see these issues directly with chef-splunk and added summary to the verify output.

from chef-splunk.

haidangwa avatar haidangwa commented on June 12, 2024

@jjm Have you accepted the license? There is one way and only one way to accept the license: https://github.com/chef-cookbooks/chef-splunk#license-acceptance

from chef-splunk.

jjm avatar jjm commented on June 12, 2024

@haidangwa Yes, it's done by this line of the kitchen.yml file:

https://github.com/chef-cookbooks/chef-splunk/blob/98a95a26472f8e04cfef207bc50276154f068d71/kitchen.yml#L18

EDIT: Linked to chef license acceptance, not splunk.

from chef-splunk.

jjm avatar jjm commented on June 12, 2024

The failure mode can be seem at https://github.com/chef-cookbooks/chef-splunk/pull/186/checks?check_run_id=1260401551 too.

from chef-splunk.

ehvidal avatar ehvidal commented on June 12, 2024

Happening the same here.

In my case, kitchen converge completes without an error:

Recipe: chef-splunk::service
  * service[splunk] action restart
    - restart service service[splunk]

Running handlers:
Running handlers complete
Chef Infra Client finished, 22/44 resources updated in 36 seconds

but after that if I run kitchen verify:

  System Package splunkforwarder
     ✔  is expected to be installed
  Service splunk
     ✔  is expected to be installed
     ✔  is expected to be enabled
     ×  is expected to be running
     expected that `Service splunk` is running

Test Summary: 3 successful, 1 failure, 0 skipped

The interesting thing is what ps -aux shows me:

root@default-ubuntu-1804:/# ps -aux | grep splunk
root         839  1.6  1.0 294276 80752 ?        Sl   20:28   0:00 splunkd -p 8089 restart
root         840  0.0  0.1  87852 13584 ?        Ss   20:28   0:00 [splunkd pid=839] splunkd -p 8089 restart [process-runner]
root         965  0.0  0.0  11460  1028 pts/0    S+   20:29   0:00 grep --color=auto splunk

it seems to me that the problem is in the restart of the service. If I kill those processes and I converge again, then everything is fine:

  System Package splunkforwarder
     ✔  is expected to be installed
  Service splunk
     ✔  is expected to be installed
     ✔  is expected to be enabled
     ✔  is expected to be running

Test Summary: 4 successful, 0 failures, 0 skipped

from chef-splunk.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.