Comments (8)
Reproducing the error with Chef-Splunk Kitchen test with EC2-driver
I can run the chef-splunk kitchen test with dokken successfully without reproducing the error.
If I swap the dokken driver with the ec2 driver and add a very simple Inspec test I can reproduce our error in the chef-splunk
cookbook. (I ran the Inspec test in the dokken driver which resulted with the same outcome).
The Inspec test:
describe service('splunk') do
it { should be_installed }
it { should be_enabled }
it { should be_running }
end
The result:
Service splunk
✔ is expected to be installed
✔ is expected to be enabled
× is expected to be running
expected that `Service splunk` is running
Investigating on the instance:
Shell output
ubuntu@ip-10-0-0-150:~$ systemctl status splunk.service
● splunk.service - Splunk
Loaded: loaded (/etc/systemd/system/splunk.service; enabled; vendor preset: enabled)
Active: inactive (dead)
ubuntu@ip-10-0-0-150:~$ sudo systemctl start splunk.service
Job for splunk.service failed because the service did not take the steps required by its unit configuration.
See "systemctl status splunk.service" and "journalctl -xe" for details.
ubuntu@ip-10-0-0-150:~$ systemctl status splunk.service
● splunk.service - Splunk
Loaded: loaded (/etc/systemd/system/splunk.service; enabled; vendor preset: enabled)
Active: failed (Result: protocol) since Thu 2020-10-15 13:33:28 UTC; 2s ago
Process: 3266 ExecStart=/opt/splunkforwarder/bin/splunk start --answer-yes --no-prompt (code=exited, status=0/SUCCESS)
Oct 15 13:33:27 ip-10-0-0-150 systemd[1]: Starting Splunk...
Oct 15 13:33:28 ip-10-0-0-150 splunk[3266]: The splunk daemon (splunkd) is already running.
Oct 15 13:33:28 ip-10-0-0-150 systemd[1]: splunk.service: Refusing to accept PID outside of service control group, acquired through unsafe symlink chain: /opt/splunkforwarder>
Oct 15 13:33:28 ip-10-0-0-150 systemd[1]: splunk.service: Refusing to accept PID outside of service control group, acquired through unsafe symlink chain: /opt/splunkforwarder>
Oct 15 13:33:28 ip-10-0-0-150 systemd[1]: splunk.service: Failed with result 'protocol'.
Oct 15 13:33:28 ip-10-0-0-150 systemd[1]: Failed to start Splunk.
kitchen.yml
kitchen.yml
---
driver:
name: ec2
region: eu-west-1
interface: public
instance_type: t2.medium
require_chef_omnibus: true
subnet_filter:
...
security_group_filter:
...
tags:
...
transport:
max_threads: 5
connection_timeout: 10
connection_retries: 36
connection_retry_sleep: 10
max_wait_until_ready: 1200
provisioner:
name: chef_zero
log_level: auto
product_name: chef
product_version: 14
max_retries: 3
wait_for_retry: 90
retry_on_exit_code:
- 35 # chef-client's reboot scheduled exit status
chef_license: accept
attributes:
dev_mode: true
splunk:
accept_license: true
enable_ssl: false
ssl_options:
enableSplunkWebSSL: 0
httpport: 8000
startwebserver: 1
web_port: 8000
verifier:
name: inspec
sudo: true
root_path: '/opt/verifier'
platforms:
- name: ubuntu-2004
driver:
image_search:
owner-id: "099720109477"
name: ubuntu/images/*/ubuntu-*-20.04*
- name: ubuntu-1804
driver:
image_search:
owner-id: "099720109477"
name: ubuntu/images/*/ubuntu-*-18.04*
- name: ubuntu-1604
driver:
image_search:
owner-id: "099720109477"
name: ubuntu/images/*/ubuntu-*-16.04*
suites:
- name: client
run_list:
- recipe[chef-splunk::default]
attributes:
dev_mode: true
splunk:
accept_license: true
verifier:
inspec_tests:
- path: test/integration/default
from chef-splunk.
@JonoRicci can you show what your systemd unit file looks like? If you're calling the client recipe directly, you may not be setting up the splunk auth attributes. There is logic in the default recipe that reads the splunk admin user/pass from a data bag or from chef-vault.
from chef-splunk.
you need to have this in an encrypted data bag or chef-vault item:
vault_item = chef_vault_item(node['splunk']['data_bag'], "splunk_#{node.chef_environment}")
from chef-splunk.
Hi @haidangwa, I've created PR #186 that adds some inspec tests to chef-splunk for the client suite that shows the issue we are seeing without our wrapper cookbook.
The output of verify on ubuntu-2004 is as follows:
System Package splunkforwarder
✔ should be installed
Service splunk
✔ should be installed
✔ should be enabled
× should be running
expected that `Service splunk` is running
Port 8089
✔ should be listening
✔ protocols should include "tcp"
Processes splunkd
✔ should exist
Test Summary: 6 successful, 1 failure, 0 skipped
To me the cause seems with the starting of splunk to accept the license, if I login to the docker container and stop splunk with /opt/splunkforwarder/bin/splunk stop
and service splunk start
. All the tests pass
System Package splunkforwarder
✔ should be installed
Service splunk
✔ should be installed
✔ should be enabled
✔ should be running
Port 8089
✔ should be listening
✔ protocols should include "tcp"
Processes splunkd
✔ should exist
Test Summary: 7 successful, 0 failures, 0 skipped
Edit: Made it clearer we see these issues directly with chef-splunk and added summary to the verify output.
from chef-splunk.
@jjm Have you accepted the license? There is one way and only one way to accept the license: https://github.com/chef-cookbooks/chef-splunk#license-acceptance
from chef-splunk.
@haidangwa Yes, it's done by this line of the kitchen.yml
file:
EDIT: Linked to chef license acceptance, not splunk.
from chef-splunk.
The failure mode can be seem at https://github.com/chef-cookbooks/chef-splunk/pull/186/checks?check_run_id=1260401551 too.
from chef-splunk.
Happening the same here.
In my case, kitchen converge
completes without an error:
Recipe: chef-splunk::service
* service[splunk] action restart
- restart service service[splunk]
Running handlers:
Running handlers complete
Chef Infra Client finished, 22/44 resources updated in 36 seconds
but after that if I run kitchen verify:
System Package splunkforwarder
✔ is expected to be installed
Service splunk
✔ is expected to be installed
✔ is expected to be enabled
× is expected to be running
expected that `Service splunk` is running
Test Summary: 3 successful, 1 failure, 0 skipped
The interesting thing is what ps -aux
shows me:
root@default-ubuntu-1804:/# ps -aux | grep splunk
root 839 1.6 1.0 294276 80752 ? Sl 20:28 0:00 splunkd -p 8089 restart
root 840 0.0 0.1 87852 13584 ? Ss 20:28 0:00 [splunkd pid=839] splunkd -p 8089 restart [process-runner]
root 965 0.0 0.0 11460 1028 pts/0 S+ 20:29 0:00 grep --color=auto splunk
it seems to me that the problem is in the restart of the service. If I kill those processes and I converge again, then everything is fine:
System Package splunkforwarder
✔ is expected to be installed
Service splunk
✔ is expected to be installed
✔ is expected to be enabled
✔ is expected to be running
Test Summary: 4 successful, 0 failures, 0 skipped
from chef-splunk.
Related Issues (20)
- `splunk_auth` no longer required by `splunk_app` resource
- Error executing action `run` on resource 'execute[update-splunk-mgmt-port]' HOT 7
- Splunk forwarder installation failure HOT 6
- chef-splunk installation failure due default_description as a property field
- SHC captain logic fails when existing captain exists
- shcluster_member? is missing an argument on line 213
- splunk version conflict after upgrades can cause Splunk to be downgraded
- systemd unit file in cookbook has diverged from splunk documentation HOT 2
- GitHub actions don't test all supported OSes
- shcluster_member? is returning false when it should be true
- Implementing Windows Support for Client Only HOT 1
- splunk_login_successful? helper method fails to send auth info to splunkd
- Add SUSE support
- undefined local variable or method `node' for Chef::Provider::Service::Systemd:Class HOT 3
- Splunk restarting on every run because of mgmt port
- splunk_monitor resource fails for valid index names
- chef-vault and default recipe being invoked when only installing the forwarder
- Systemd doesn't recognize Splunkd.service on upgrade
- Dependency Dashboard
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chef-splunk.