graylog2 / graylog2-cookbook Goto Github PK
View Code? Open in Web Editor NEWChef recipes to deploy Graylog
Home Page: https://www.graylog.org
License: Apache License 2.0
Chef recipes to deploy Graylog
Home Page: https://www.graylog.org
License: Apache License 2.0
In template file graylog.server.conf.erb following line is putting a quotation (") mark around password_secret
<%= config_option 'password_secret', node.graylog2[:password_secret], enclose: '"' -%>
Its will create problem when migrating users from a different installation of graylog2 (older version). This will hinder to use the same password for the users who are in the older graylog2 installation. It seems graylog is misinterpreting the quotation mark. To avoid this problem the above mentioned line in graylog.server.conf.erb should be changed to
<%= config_option 'password_secret', node.graylog2[:password_secret]-%>
Hello,
Graylog install fails on Ubuntu 16.04 as the tzdata-java
package is no longer in repositories, according to this page.
Is it planned to support Ubuntu 16.04 at some point or should I definitly use 14.04 for my graylog server ?
Hello,
I have a tool that orchestrates the deploy of a single or multi-node graylog stack on Amazon Web Services with the provided OVA AMIs.
Randomly when doing the reconfigure command for first time in the master node it throws the following error.
2017-03-22 08:33:47,399 P7483 [INFO] [2017-03-22T08:33:44+01:00] INFO: template[/opt/graylog/sv/graylog-server/log/run] owner changed to 0
2017-03-22 08:33:47,399 P7483 [INFO] [2017-03-22T08:33:44+01:00] INFO: template[/opt/graylog/sv/graylog-server/log/run] group changed to 0
2017-03-22 08:33:47,399 P7483 [INFO] [2017-03-22T08:33:44+01:00] INFO: template[/opt/graylog/sv/graylog-server/log/run] mode changed to 755
2017-03-22 08:33:47,399 P7483 [INFO] [2017-03-22T08:33:44+01:00] INFO: template[/var/log/graylog/server/config] created file /var/log/graylog/server/config
2017-03-22 08:33:47,399 P7483 [INFO] [2017-03-22T08:33:44+01:00] INFO: template[/var/log/graylog/server/config] updated file contents /var/log/graylog/server/config
2017-03-22 08:33:47,399 P7483 [INFO] [2017-03-22T08:33:44+01:00] INFO: template[/var/log/graylog/server/config] owner changed to 0
2017-03-22 08:33:47,399 P7483 [INFO] [2017-03-22T08:33:44+01:00] INFO: template[/var/log/graylog/server/config] group changed to 0
2017-03-22 08:33:47,400 P7483 [INFO] [2017-03-22T08:33:44+01:00] INFO: link[/opt/graylog/init/graylog-server] created
2017-03-22 08:33:47,400 P7483 [INFO] [2017-03-22T08:33:44+01:00] INFO: link[/opt/graylog/service/graylog-server] created
2017-03-22 08:33:47,400 P7483 [INFO] [2017-03-22T08:33:45+01:00] INFO: ruby_block[supervise_graylog-server_sleep] called
2017-03-22 08:33:47,400 P7483 [INFO] [2017-03-22T08:33:45+01:00] INFO: execute[/opt/graylog/embedded/bin/graylog-ctl start graylog-server] ran successfully
2017-03-22 08:33:47,400 P7483 [INFO]
2017-03-22 08:33:47,400 P7483 [INFO] ================================================================================
2017-03-22 08:33:47,400 P7483 [INFO] Error executing action `run` on resource 'ruby_block[add node to server list]'
2017-03-22 08:33:47,400 P7483 [INFO] ================================================================================
2017-03-22 08:33:47,400 P7483 [INFO]
2017-03-22 08:33:47,400 P7483 [INFO] Errno::ECONNREFUSED
2017-03-22 08:33:47,400 P7483 [INFO] -------------------
2017-03-22 08:33:47,400 P7483 [INFO] Connection refused - connect(2) for "127.0.0.1" port 4001
2017-03-22 08:33:47,400 P7483 [INFO]
2017-03-22 08:33:47,400 P7483 [INFO] Cookbook Trace:
2017-03-22 08:33:47,400 P7483 [INFO] ---------------
2017-03-22 08:33:47,400 P7483 [INFO] /opt/graylog/embedded/cookbooks/graylog/libraries/registry.rb:17:in `set_master'
2017-03-22 08:33:47,400 P7483 [INFO] /opt/graylog/embedded/cookbooks/graylog/recipes/graylog-server.rb:86:in `block (2 levels) in from_file'
2017-03-22 08:33:47,400 P7483 [INFO]
2017-03-22 08:33:47,400 P7483 [INFO] Resource Declaration:
2017-03-22 08:33:47,401 P7483 [INFO] ---------------------
2017-03-22 08:33:47,401 P7483 [INFO] # In /opt/graylog/embedded/cookbooks/graylog/recipes/graylog-server.rb
2017-03-22 08:33:47,401 P7483 [INFO]
2017-03-22 08:33:47,401 P7483 [INFO] 84: ruby_block "add node to server list" do
2017-03-22 08:33:47,401 P7483 [INFO] 85: block do
2017-03-22 08:33:47,401 P7483 [INFO] 86: $registry.set_master
2017-03-22 08:33:47,401 P7483 [INFO] 87: $registry.add_gl_server(node['ipaddress'])
2017-03-22 08:33:47,401 P7483 [INFO] 88: $registry.add_es_node(node['ipaddress'])
2017-03-22 08:33:47,401 P7483 [INFO] 89: end
2017-03-22 08:33:47,401 P7483 [INFO] 90: end
2017-03-22 08:33:47,401 P7483 [INFO]
2017-03-22 08:33:47,401 P7483 [INFO] Compiled Resource:
2017-03-22 08:33:47,401 P7483 [INFO] ------------------
2017-03-22 08:33:47,401 P7483 [INFO] # Declared in /opt/graylog/embedded/cookbooks/graylog/recipes/graylog-server.rb:84:in `from_file'
2017-03-22 08:33:47,401 P7483 [INFO]
2017-03-22 08:33:47,401 P7483 [INFO] ruby_block("add node to server list") do
2017-03-22 08:33:47,401 P7483 [INFO] action [:run]
2017-03-22 08:33:47,401 P7483 [INFO] retries 0
2017-03-22 08:33:47,401 P7483 [INFO] retry_delay 2
2017-03-22 08:33:47,402 P7483 [INFO] default_guard_interpreter :default
2017-03-22 08:33:47,402 P7483 [INFO] block_name "add node to server list"
2017-03-22 08:33:47,402 P7483 [INFO] declared_type :ruby_block
2017-03-22 08:33:47,402 P7483 [INFO] cookbook_name :graylog
2017-03-22 08:33:47,402 P7483 [INFO] recipe_name "graylog-server"
2017-03-22 08:33:47,402 P7483 [INFO] block #<Proc:0x00000004dfaf08@/opt/graylog/embedded/cookbooks/graylog/recipes/graylog-server.rb:85>
2017-03-22 08:33:47,402 P7483 [INFO] end
2017-03-22 08:33:47,402 P7483 [INFO]
What I've found so far is that sometimes ETCD is not yet ready when Chef executes the 'add node to server list'. Seems both commands are executed one after the other, so if for any reason etcd delays the start, the command will fail.
With the same code, the reconfigure will fail about 2-3 times out of 10.
Would be nice if there is any way to ensure etcd is up and listening on port 4001 before trying to run the consecutive actions.
I tried to find where it's this code across some graylog projects but had no luck.
If someone can point me what project has the code I could try to code a fix.
Thanks you,
First of all, thanks for this cookbook, we're going to switch to it soon (from our own cookbook).
It would be nice to provide a Changelog/History file so we don't have to dig in the git changelogs to find the changes. Thanks :)
The cookbook should be able to create a dashboard like this:
{
"title" : "Stream Dashboard",
"description" : "Stream related Dashboard",
"creator_user_id" : "admin",
"widgets" : [ {
"description" : "All Logs (5m)",
"config" : {
"interval" : "minute",
"query" : "*",
"timerange" : {
"range" : 300,
"type" : "relative"
},
"stream_id" : "54ebad87e4b010108e5347ad"
},
"type" : "search_result_chart",
"creator_user_id" : "admin"
} ]
}
The important value is the stream_id
. We need to create the stream in one resource or reach in a value for it.
The collector recipe installs version 0.4.2 instead of 0.5.0 on Ubuntu 14.04 which isn't compatible with Graylog 2.0.
With the graylog 1.x cookbook, I was able to use chef-vault to load the various secrets instead of using an encrypted data bag. With the 2.0 cookbook, it doesn't use "Chef::Mixin::DeepMerge.deep_merge" any more.
This works ok with the attribute assignment for "password_secret" and "root_password_sha2":
graylog2-cookbook/recipes/server.rb
Lines 7 to 8 in 47e013a
But not for "rest_tls_key_password" and "web_tls_key_password" because it attempts to use the encrypted data bag item first instead of the attribute.
graylog2-cookbook/recipes/server.rb
Lines 72 to 73 in 47e013a
Is it possible to have "rest_tls_key_password" and "web_tls_key_password" function the same way as "password_secret" and "root_password_sha2", or to use some other method to support chef-vault? The other option would be to use chef-vault instead of encrypted data bags because they seem to be a better solution for storing/retrieving secrets, and can fall back to using unencrypted data bags for use in test-kitchen.
With the current version of the 2.0 cookbook, I can't find a way to converge my wrapper cookbook any more and I would like to test the 2.0.0-2.beta.3 version.
Thanks, Richard
In recipes/default.rb
the yum repo is set to be located at https://packages.graylog2.org/repo/el/stable/...
which doesn't appear to be valid.
I'm not sure whether this is an issue with the cookbook or with the repo itself however.
This should be prevented or at least optional.
nt
I see error in below gist we see when trying to upgrade an existing install to graylog 1.2:
https://gist.github.com/TanmayaA/06ec722ecedd4e8349ec
Unable to attach here due to some issue in github uploader
I am trying to upgrade Graylog2 on my servers, but unfortunately I am running into a small issue.
Previously I had setup Graylog2 with the cookbook version "graylog2", :git => "https://github.com/Graylog2/graylog2-cookbook", :tag => "2.0"
, and now trying to switch to "graylog2", "2.1.1"
.
I am getting an error when it is trying to install the .deb
package:
dpkg: error processing archive /var/chef/cache/graylog-2.1-repository_1-1_all.deb (--install):
trying to overwrite '/etc/apt/trusted.gpg.d/graylog-keyring.gpg', which is also in package graylog-2.0-repository 1-1
Errors were encountered while processing:
/var/chef/cache/graylog-2.1-repository_1-1_all.deb
I thought deleting graylog-keyring.gpg
would resolve the issue, but unfortunately it has not.
I also tried manually downloading and install this .deb
, with the same results.
I am assuming there is something silly I am missing/doing wrong here, and would love any help. I have provided some information below, more than happy to share more if needed.
$ uname -a
Linux ip-X-X-X-X 3.13.0-74-generic #118-Ubuntu SMP Thu Dec 17 22:52:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ dpkg --list | grep graylog
ii graylog-2.0-repository 1-1 all Package to install Graylog 2.0 GPG key and repository
ii graylog-server 2.0.0-5 all Graylog server
* apt_package[graylog-2.1-repository_1-1_all.deb] action install
================================================================================
Error executing action `install` on resource 'apt_package[graylog-2.1-repository_1-1_all.deb]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of dpkg -i --force-confold /var/chef/cache/graylog-2.1-repository_1-1_all.deb ----
STDOUT: (Reading database ... 85675 files and directories currently installed.)
Preparing to unpack .../graylog-2.1-repository_1-1_all.deb ...
Unpacking graylog-2.1-repository (1-1) ...
STDERR: dpkg: error processing archive /var/chef/cache/graylog-2.1-repository_1-1_all.deb (--install):
trying to overwrite '/etc/apt/trusted.gpg.d/graylog-keyring.gpg', which is also in package graylog-2.0-repository 1-1
Errors were encountered while processing:
/var/chef/cache/graylog-2.1-repository_1-1_all.deb
---- End output of dpkg -i --force-confold /var/chef/cache/graylog-2.1-repository_1-1_all.deb ----
Ran dpkg -i --force-confold /var/chef/cache/graylog-2.1-repository_1-1_all.deb returned 1
Cookbook Trace:
---------------
/var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:78:in `run_action'
/var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:106:in `block (2 levels) in converge'
/var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:106:in `each'
/var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:106:in `block in converge'
/var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:105:in `converge'
Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/graylog2/recipes/default.rb
31: package repository_file do
32: action :install
33: source "#{Chef::Config[:file_cache_path]}/#{repository_file}"
34: if platform_family?('rhel')
35: provider Chef::Provider::Package::Rpm
36: options '--force'
37: notifies :run, 'execute[yum-clean]', :immediately
38: elsif platform?('ubuntu', 'debian')
39: provider Chef::Provider::Package::Dpkg
40: options '--force-confold'
41: notifies :run, 'execute[apt-update]', :immediately
42: end
43: end
Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/graylog2/recipes/default.rb:31:in `from_file'
apt_package("graylog-2.1-repository_1-1_all.deb") do
package_name "graylog-2.1-repository_1-1_all.deb"
provider Chef::Provider::Package::Dpkg
action [:install]
retries 0
retry_delay 2
default_guard_interpreter :default
declared_type :package
cookbook_name "graylog2"
recipe_name "default"
options "--force-confold"
source "/var/chef/cache/graylog-2.1-repository_1-1_all.deb"
end
Accessing attributes via node.key.key2
will be an error in upcoming versions of chef
It will be good to have configurable NO_RETRIES parameters in the api_check provider because my installation fails every time with timeout during the first run. Currently this parameter is hardcoded:
NO_RETRIES = 300 # 5 minutes, waiting a second to retry
There is a "h" too much in the end of this line.
elasticsearch_max_time_per_index = <% if Graylog['rotation_time'] != 0 %><%= Graylog['rotation_time'] %><% else %>1<% end %>h
If i run: sudo graylog-ctl set-retention --time=1d --indices=20 --journal=5
The graylog.conf is set too;
elasticsearch_max_time_per_index = 1dh
And graylog failes to start because of the h.
3c8699b introduced an adjustment in secret preference, however secrets
is undefined if not retrieved.
Hello,
When building out a test environment to upgrade our graylog2 cluster, I noticed that the web recipe is missing. Is this on purpose?
graylog2 Version 0.92.3-1419362218
AMI used: ami-843e0e99
Error messages after running:
sudo graylog2-ctl reconfigure
Recipe: ntp::default
* apt_package[ntp] action install
================================================================================
Error executing action `install` on resource 'apt_package[ntp]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '100'
---- Begin output of apt-get -q -y install ntp=1:4.2.6.p5+dfsg-3ubuntu2.14.04.1 ----
STDOUT: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
ntp : Depends: libopts25 (>= 1:5.18) but it is not installable
STDERR: E: Unable to correct problems, you have held broken packages.
---- End output of apt-get -q -y install ntp=1:4.2.6.p5+dfsg-3ubuntu2.14.04.1 ----
Ran apt-get -q -y install ntp=1:4.2.6.p5+dfsg-3ubuntu2.14.04.1 returned 100
.
.
.
Recipe: graylog2::graylog2-server
* service[graylog2-server] action restart
================================================================================
Error executing action `restart` on resource 'service[graylog2-server]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of /opt/graylog2/embedded/bin/chpst -u root /opt/graylog2/embedded/bin/sv -w 45 restart /opt/graylog2/service/graylog2-server ----
STDOUT: timeout: run: /opt/graylog2/service/graylog2-server: (pid 1619) 74s
STDERR:
---- End output of /opt/graylog2/embedded/bin/chpst -u root /opt/graylog2/embedded/bin/sv -w 45 restart /opt/graylog2/service/graylog2-server ----
Ran /opt/graylog2/embedded/bin/chpst -u root /opt/graylog2/embedded/bin/sv -w 45 restart /opt/graylog2/service/graylog2-server returned 1
Resource Declaration:
---------------------
# In /opt/graylog2/embedded/cookbooks/runit/definitions/runit_service.rb
190: service params[:name] do
191: control_cmd = node[:runit][:sv_bin]
192: if params[:owner]
193: control_cmd = "#{node[:runit][:chpst_bin]} -u #{params[:owner]} #{control_cmd}"
194: end
195: provider Chef::Provider::Service::Simple
Compiled Resource:
------------------
# Declared in /opt/graylog2/embedded/cookbooks/runit/definitions/runit_service.rb:190:in `block in from_file'
service("graylog2-server") do
params {:directory=>"/opt/graylog2/sv", :only_if=>false, :finish_script=>false, :control=>["t"], :run_restart=>false, :active_directory=>"/opt/graylog2/service", :init_script_template=>nil, :owner=>"root", :group=>"root", :template_name=>"graylog2-server", :start_command=>"start", :stop_command=>"stop", :restart_command=>"-w 45 restart", :status_command=>"status", :options=>{:log_directory=>"/var/log/graylog2/server", :install_directory=>"/opt/graylog2", :directory=>nil, :only_if=>false, :finish_script=>false, :control=>["t"], :run_restart=>false, :active_directory=>nil, :init_script_template=>nil, :owner=>"root", :group=>"root", :template_name=>nil, :start_command=>"start", :stop_command=>"stop", :restart_command=>"-w 45 restart", :status_command=>"status", :options=>{}, :log_options=>{}, :env=>{}, :action=>:enable, :down=>false}, :log_options=>{"svlogd_size"=>209715200, "svlogd_num"=>30, "svlogd_timeout"=>86400, "svlogd_filter"=>"gzip", "svlogd_udp"=>nil, "svlogd_prefix"=>nil, "udp_log_shipping_host"=>nil, "udp_log_shipping_port"=>514, "enable"=>true, "log_directory"=>"/var/log/graylog2/server", "data_directory"=>"/var/opt/graylog2/data/spool", "node_id"=>"/var/opt/graylog2/graylog2-server-node-id", "wrapper"=>"/opt/graylog2/embedded/bin/authbind"}, :env=>{}, :action=>:enable, :down=>false, :name=>"graylog2-server"}
provider Chef::Provider::Service::Simple
action [:nothing]
supports {:restart=>true, :status=>true}
retries 0
retry_delay 2
default_guard_interpreter :default
service_name "graylog2-server"
pattern "graylog2-server"
start_command "/opt/graylog2/embedded/bin/chpst -u root /opt/graylog2/embedded/bin/sv start /opt/graylog2/service/graylog2-server"
stop_command "/opt/graylog2/embedded/bin/chpst -u root /opt/graylog2/embedded/bin/sv stop /opt/graylog2/service/graylog2-server"
status_command "/opt/graylog2/embedded/bin/chpst -u root /opt/graylog2/embedded/bin/sv status /opt/graylog2/service/graylog2-server"
restart_command "/opt/graylog2/embedded/bin/chpst -u root /opt/graylog2/embedded/bin/sv -w 45 restart /opt/graylog2/service/graylog2-server"
declared_type :service
cookbook_name :graylog2
recipe_name "graylog2-server"
end
This is a future deprecation warning currently.
In line with the recommendation here: https://docs.chef.io/ruby.html#node-set it is better to use node.default
in most cases as this is rebuilt on the node object during each chef run. Where persistence is really needed, node.normal
is more clear.
Restarting the graylog-server
service should not result in Graylog Server output appearing on STDOUT.
Ideally graylog2-cookbook/templates/default/graylog.server.log4j2.xml.erb
would be made consistent with fpm-recipes/recipes/graylog-server/files/log4j2.xml
, which (among other things) does not have a STDOUT appender.
Restarting the graylog-server
service (without doing output redirection) results in Graylog Server output appearing directly in the console.
Removal of the STDOUT appender in graylog2-cookbook/templates/default/graylog.server.log4j2.xml.erb
, removal of the STDOUT appender from the Root logger, or an attribute controlling whether STDOUT is included (but disabled by default).
sudo service graylog-server restart
)graylog2-cookbook/templates/default/graylog.server.log4j2.xml.erb
, removing <AppenderRef ref="STDOUT"/>
from the Root logger.sudo service graylog-server restart
)If I try to override attribute [:graylog2][:server][:log_file] is correctly updates in log4j.xml, however it doesn't create the necessary directory, or ensures that it has correct ownership/permissions.
Can this be handled in this cookbook?
Handling this in wrapper recipe poses a problem that for the first run neither the directory, nor the user will exist.
Ref:
graylog2-cookbook/attributes/default.rb
Line 183 in e9eae61
We were looking to provision a "stand alone" Graylog node with a few default inputs setup via Chef with the api_access recipe. In practice we have been unable to provision the "default", "server" and "api_access" recipes in a single Chef pass.
Issue:
The "api_access" recipe is expecting the API endpoints for the Graylog service to be enabled prior to the cookbook running, however the default notification policy on the service is "delayed". As this has not yet been triggered, the "api_access" recipe will timeout after several minutes of waiting for the endpoint to become available.
Expected Behavior:
Recipes are able to better synergize with one another and be installed in series on a single node in one Chef run.
Steps To Reproduce:
Add the "recipe[api_access]" to the runlist via the .kitchen.yml. Example log @ http://pastebin.com/raw/K4TT3ze3
Attempted Resolutions:
I am still working through this, but I have not come up with a great solution yet. Things we've tried:
Please, create releases (tags) when versioning in metadata.rb
It seems that the elasticsearch cookbook 3.x.x and great only supports installation of version 5.x and above of elasticsearch. Dependencies need to be changed in order to reflect this.
It would be cool to have a LWRP which installs Graylog plugins from the marketplace...
If you create two inputs using the same name only the first input will be created. There is no error in chef but you see an error in graylog2.
Sample recipe:
graylog2_inputs "syslog udp high" do
input '{ "title": "syslog_high", "type":"org.graylog2.inputs.syslog.udp.SyslogUDPInput", "global": true, "configuration": { "port": 1514, "allow_override_date": true, "bind_address": "0.0.0.0", "store_full_message": true, "recv_buffer_size": 1048576 } }'
end
graylog2_inputs "syslog udp low" do
input '{ "title": "syslog_low", "type":"org.graylog2.inputs.syslog.udp.SyslogUDPInput", "global": true, "configuration": { "port": 514, "allow_override_date": true, "bind_address": "0.0.0.0", "store_full_message": true, "recv_buffer_size": 1048576 } }'
end
graylog2_inputs "syslog tcp high" do
input '{ "title": "syslog_high", "type":"org.graylog2.inputs.syslog.tcp.SyslogTCPInput", "global": true, "configuration": { "port": 1514, "allow_override_date": true, "bind_address": "0.0.0.0", "store_full_message": true, "recv_buffer_size": 1048576 } }'
end
graylog2_inputs "syslog tcp low" do
input '{ "title": "syslog_low", "type":"org.graylog2.inputs.syslog.tcp.SyslogTCPInput", "global": true, "configuration": { "port": 514, "allow_override_date": true, "bind_address": "0.0.0.0", "store_full_message": true, "recv_buffer_size": 1048576 } }'
end
After upgrade from v 1.1.4 to 1.1.5 and reconfiguring graylog, i got the failure decribed above:
ruby_block[add node to server list] action run
================================================================================
Error executing action run
on resource 'ruby_block[add node to server list]'
================================================================================
Errno::ECONNREFUSED
-------------------
Connection refused - connect(2) for "127.0.0.1" port 4001
Cookbook Trace:
---------------
/opt/graylog/embedded/cookbooks/graylog/libraries/registry.rb:17:in `set_master'
/opt/graylog/embedded/cookbooks/graylog/recipes/graylog-server.rb:81:in `block (2 levels) in from_file'
Resource Declaration:
---------------------
# In /opt/graylog/embedded/cookbooks/graylog/recipes/graylog-server.rb
79: ruby_block "add node to server list" do
80: block do
81: $registry.set_master
82: $registry.add_gl_server(node['ipaddress'])
83: $registry.add_es_node(node['ipaddress'])
84: end
85: end
Compiled Resource:
------------------
# Declared in /opt/graylog/embedded/cookbooks/graylog/recipes/graylog-server.rb:79:in `from_file'
ruby_block("add node to server list") do
action "run"
retries 0
retry_delay 2
default_guard_interpreter :default
block_name "add node to server list"
declared_type :ruby_block
cookbook_name :graylog
recipe_name "graylog-server"
block #<Proc:0x000000042ee4e8@/opt/graylog/embedded/cookbooks/graylog/recipes/graylog-server.rb:80>
end
Running handlers:
[2015-07-31T10:00:16+02:00] ERROR: Running exception handlers
Running handlers complete
[2015-07-31T10:00:16+02:00] ERROR: Exception handlers complete
[2015-07-31T10:00:16+02:00] FATAL: Stacktrace dumped to /opt/graylog/embedded/cookbooks/cache/chef-stacktrace.out
Chef Client failed. 6 resources updated in 5.707804974 seconds
[2015-07-31T10:00:16+02:00] ERROR: ruby_block[add node to server list](graylog::graylog-server line 79) had an error: Errno::ECONNREFUSED: Connection refused - connect(2) for "127.0.0.1" port 4001
[2015-07-31T10:00:16+02:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Graylog seems running, because i get some stream alerts, but WEB-GUI can not be opened.
Hi,
is it possible that the admin_access_token
can also be stored in the encrypted data bag?
regards
dieter
It took me a while to find this problem, but when you attempt to create a dashboard, input, or stream using the rest API you will get back a 403 error telling you to go make that sort of API call to the master node instead. The 403 is not logged because line 58 causes a fault here before it reaches line 59 which would have logged the message.
https://github.com/Graylog2/graylog2-cookbook/blob/master/providers/dashboards.rb#L58-L59
I suggest re-ordering those two lines. The same problem also exists here:
https://github.com/Graylog2/graylog2-cookbook/blob/master/providers/streams.rb#L60-L61
There should also be a check to make sure that we received a 200 OK instead of an error code, and then log the error as an error level log message if we get one.
The other problem is that the cookbook is attempting to make configuration changes to a non-master node(This causes our un-handled 403 error). Maybe a new attribute called graylog2[:rest][:master_uri] should be created, or maybe some sort of check to see if we are the master node and skip all of the rest API configuration stuff if we are not the master node. Maybe something like:
https://github.com/Graylog2/graylog2-cookbook/blob/master/recipes/server.rb#L41
if node.graylog2[:ip_of_master] == node.ipaddress
I would be awesome if we could configure LDAP authentication directly from the cookbook (much like inputs & streams).
If I reboot my ubuntu 14.04 machine graylog-server does not start automatically.
Is this a known bug in this cookbook or deb package?
I would like to setup options like this:
default['graylog2']['server']['additional_options']['trusted_proxies'] = '127.0.0.1/32, 0:0:0:0:0:0:0:1/128'
default['graylog2']['server']['additional_options']['whatever'] = false
But the additional_options only supports a string which does not allow mutiple options...
In order to recreate / provision a fully functioning Graylog cluster we would need to have access to most if not all of the REST resources using LWRPs or similar - that's a lot of work to create, maintain, test, etc.
Since the configuration is persisted in mongo it might be a good idea to support restoring the entire configuration from a mongodump - something along the lines of this:
file 'mongorestore_lockfile' do
action :create_if_missing
notifies :run, 'execute[run_mongorestore]', :immediately
end
execute 'run_mongorestore' do
command 'mongorestore -foo - bar ...'
action :nothing
end
I might do a PR if your interested in having such a feature ?
Right now the web interface gets 1GB and the server no memory limit at all:
graylog 26245 5.4 13.9 3669176 563120 ? Ssl 10:49 1:06 /usr/bin/java -Djava.net.preferIPv4Stack=true -jar -Dlog4j.configuration=file:///etc/graylog/server/log4j.xml -Djava.library.path=/usr/share/graylog-server/lib/sigar /usr/share/graylog-server/graylog.jar server -f /etc/graylog/server/server.conf -np
graylog+ 26268 2.0 12.3 3422004 498516 ? Ssl 10:49 0:24 java -Xms1024m -Xmx1024m -XX:ReservedCodeCacheSize=128m -Dconfig.file=/etc/graylog/web/web.conf -Dlogger.file=/etc/graylog/web/logback.xml -Dpidfile.path=/dev/null -Dhttp.address=0.0.0.0 -Dhttp.port=9000 -Djava.net.preferIPv4Stack=true -Duser.dir=/usr/share/graylog-web -cp /usr/share/graylog-web/lib/graylog-web-interface.graylog-web-interface-1.3.2.jar:/usr/share/graylog-web/lib/org.scala-lang.scala-library-2.10.4.jar:/usr/share/graylog-web/lib/com.typesafe.play.twirl-api_2.10-1.0.2.jar:/usr/share/graylog-web/lib/org.apache.commons.commons-lang3-3.1.jar:/usr/share/graylog-web/lib/com.typesafe.play.play_2.10-2.3.10.jar:/usr/share/graylog-web/lib/com.typesafe.play.build-link-2.3.10.jar:/usr/share/graylog-web/lib/com.typesafe.play.play-exceptions-2.3.10.jar:/usr/share/graylog-web/lib/com.typesafe.play.play-iteratees_2.10-2.3.10.jar:/usr/share/graylog-web/lib/org.scala-stm.scala-stm_2.10-0.7.jar:/usr/share/graylog-web/lib/com.typesafe.config-1.2.1.jar:/usr/share/graylog-web/lib/com.typesafe.play.play-json_2.10-2.3.10.jar:/usr/share/graylog-web/lib/com.typesafe.play.play-functional_2.10-2.3.10.jar:/usr/share/graylog-web/lib/com.typesafe.play.play-datacommons_2.10-2.3.10.jar:/usr/share/graylog-web/lib/org.joda.joda-convert-1.6.jar:/usr/share/graylog-web/lib/org.scala-lang.scala-reflect-2.10.4.jar:/usr/share/graylog-web/lib/com.typesafe.netty.netty-http-pipelining-1.1.2.jar:/usr/share/graylog-web/lib/org.slf4j.jul-to-slf4j-1.7.6.jar:/usr/share/graylog-web/lib/org.slf4j.jcl-over-slf4j-1.7.6.jar:/usr/share/graylog-web/lib/ch.qos.logback.logback-core-1.1.1.jar:/usr/share/graylog-web/lib/ch.qos.logback.logback-classic-1.1.1.jar:/usr/share/graylog-web/lib/com.typesafe.akka.akka-slf4j_2.10-2.3.4.jar:/usr/share/graylog-web/lib/commons-codec.commons-codec-1.9.jar:/usr/share/graylog-web/lib/xerces.xercesImpl-2.11.0.jar:/usr/share/graylog-web/lib/xml-apis.xml-apis-1.4.01.jar:/usr/share/graylog-web/lib/javax.transaction.jta-1.1.jar:/usr/share/graylog-web/lib/com.typesafe.play.play-java_2.10-2.3.10.jar:/usr/share/graylog-web/lib/org.yaml.snakeyaml-1.13.jar:/usr/share/graylog-web/lib/javax.validation.validation-api-1.1.0.Final.jar:/usr/share/graylog-web/lib/org.springframework.spring-context-4.0.3.RELEASE.jar:/usr/share/graylog-web/lib/org.springframework.spring-aop--4.0.3.RELEASE-4.0.3.RELEASE.jar:/usr/share/graylog-web/lib/aopalliance.aopalliance-1.0.jar:/usr/share/graylog-web/lib/org.springframework.spring-beans-4.0.3.RELEASE.jar:/usr/share/graylog-web/lib/org.springframework.spring-core-4.0.3.RELEASE.jar:/usr/share/graylog-web/lib/commons-logging.commons-logging-1.1.3.jar:/usr/share/graylog-web/lib/org.springframework.spring-expression--4.0.3.RELEASE-4.0.3.RELEASE.jar:/usr/share/graylog-web/lib/org.javassist.javassist-3.19.0-GA.jar:/usr/share/graylog-web/lib/org.reflections.reflections-0.9.8.jar:/usr/share/graylog-web/lib/com.google.guava.guava-18.0.jar:/usr/share/graylog-web/lib/javassist.javassist--3.12.1.GA-3.12.1.GA.jar:/usr/share/graylog-web/lib/dom4j.dom4j-1.6.1.jar:/usr/share/graylog-web/lib/org.apache.tomcat.tomcat-servlet-api-8.0.5.jar:/usr/share/graylog-web/lib/com.typesafe.play.play-cache_2.10-2.3.10.jar:/usr/share/graylog-web/lib/net.sf.ehcache.ehcache-core-2.6.8.jar:/usr/share/graylog-web/lib/commons-beanutils.commons-beanutils-1.8.3.jar:/usr/share/graylog-web/lib/com.google.inject.guice-4.0.jar:/usr/share/graylog-web/lib/javax.inject.javax.inject-1.jar:/usr/share/graylog-web/lib/com.google.inject.extensions.guice-assistedinject-4.0.jar:/usr/share/graylog-web/lib/org.graylog2.play2-graylog2_2.10-1.2.1.jar:/usr/share/graylog-web/lib/org.graylog2.graylog2-rest-client--1.3.2-1.3.2.jar:/usr/share/graylog-web/lib/org.graylog2.graylog2-rest-models-1.3.2.jar:/usr/share/graylog-web/lib/com.fasterxml.jackson.core.jackson-annotations-2.6.2.jar:/usr/share/graylog-web/lib/joda-time.joda-time-2.8.2.jar:/usr/share/graylog-web/lib/org.hibernate.hibernate-validator-5.2.2.Final.jar:/usr/share/graylog-web/lib/org.jboss.logging.jboss-logging-3.2.1.Final.jar:/usr/share/graylog-web/lib/com.fasterxml.classmate-1.1.0.jar:/usr/share/graylog-web/li
ubuntu 30888 0.0 0.0 10464 928 pts/0 S+ 11:09 0:00 grep --color=auto java
nt
I've just realised that the permissions for the config files are all set to 0644. The server config file for example includes things like clear text SMTP credentials, which you probably don't want to be world readable.
Should I do a pull request to change everything to 640?
It might be too early to ask this question. When are you planing to support graylog version 1.1.0?
I tried to switch to time-based index limit, but Chef won't let me override an attribute to nil:
https://gist.github.com/gregkare/01c64453c566b131b31b
That's on Chef 11, I haven't tried if Chef 12 still has the same behaviour
Because the config file is written with max_size_per_index
and max_docs_per_index
the max_time_per_index
is ignored
This happens on a new Ubuntu 14.04 box:
Recipe: graylog2::default
* remote_file[/var/chef/cache/graylog2-0.92-repository-ubuntu14.04_latest.deb] action create_if_missing (up to date)
* execute[apt-update] action nothing (skipped due to action :nothing)
* apt_package[graylog2-0.92-repository-ubuntu14.04_latest.deb] action install (up to date)
Recipe: graylog2::server
* apt_package[graylog2-server] action install (up to date)
* directory[/var/run/graylog2] action create (up to date)
* service[graylog2-server] action nothing (skipped due to action :nothing)
* template[/etc/graylog2.conf] action create (up to date)
* template[/etc/default/graylog2-server] action create[2015-01-15T15:11:39+00:00] INFO: template[/etc/default/graylog2-server] backed up to /var/chef/backup/etc/default/graylog2-server.chef-20150115151139.019786
[2015-01-15T15:11:39+00:00] INFO: template[/etc/default/graylog2-server] updated file contents /etc/default/graylog2-server
- update content in file /etc/default/graylog2-server from c13a92 to 56a42c
--- /etc/default/graylog2-server 2015-01-15 14:00:52.010540644 +0000
+++ /tmp/chef-rendered-template20150115-12545-1hcjjjg 2015-01-15 15:11:39.014540644 +0000
@@ -6,5 +6,5 @@
# Program that will be used to wrap the graylog2-server command. Useful to
# support programs like authbind.
-GRAYLOG2_COMMAND_WRAPPER=""
+GRAYLOG2_COMMAND_WRAPPER="authbind"
* template[/etc/graylog2/server/log4j.xml] action create (up to date)
* template[/etc/graylog2-elasticsearch.yml] action create (up to date)
Recipe: graylog2::web
* apt_package[graylog2-web] action install (up to date)
* service[graylog2-web] action nothing (skipped due to action :nothing)
* template[/etc/graylog2/web/graylog2-web-interface.conf] action create (up to date)
* template[/etc/default/graylog2-web] action create (up to date)
* template[/etc/graylog2/web/logback.xml] action create (up to date)
Recipe: graylog2::authbind
* authbind_port[AuthBind Graylog2 port 514] action add
* file[/etc/authbind/byport/!514] action create
* Parent directory /etc/authbind/byport does not exist.
================================================================================
Error executing action `create` on resource 'file[/etc/authbind/byport/!514]'
================================================================================
Chef::Exceptions::EnclosingDirectoryDoesNotExist
------------------------------------------------
Parent directory /etc/authbind/byport does not exist.
Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/authbind/providers/port.rb
23: file "/etc/authbind/byport/#{port}" do
24: owner new_resource.user
25: group new_resource.group if new_resource.group
26: mode "0550"
27: action :create
28: end
29: end
Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/authbind/providers/port.rb:23:in `block in class_from_file'
file("/etc/authbind/byport/!514") do
action [:create]
retries 0
retry_delay 2
default_guard_interpreter :default
path "/etc/authbind/byport/!514"
backup 5
atomic_update true
declared_type :file
cookbook_name "graylog2"
owner "graylog2"
mode "0550"
end
[2015-01-15T15:11:39+00:00] INFO: Running queued delayed notifications before re-raising exception
================================================================================
Error executing action `add` on resource 'authbind_port[AuthBind Graylog2 port 514]'
================================================================================
Chef::Exceptions::EnclosingDirectoryDoesNotExist
------------------------------------------------
file[/etc/authbind/byport/!514] (/var/chef/cache/cookbooks/authbind/providers/port.rb line 23) had an error: Chef::Exceptions::EnclosingDirectoryDoesNotExist: Parent directory /etc/authbind/byport does not exist.
Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/graylog2/recipes/authbind.rb
12: authbind_port "AuthBind Graylog2 port #{node.graylog2[:authorized_ports]}" do
13: port node.graylog2[:authorized_ports]
14: user node.graylog2[:user]
15: end
16: end
Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/graylog2/recipes/authbind.rb:12:in `from_file'
authbind_port("AuthBind Graylog2 port 514") do
action :add
retries 0
retry_delay 2
default_guard_interpreter :default
declared_type :authbind_port
cookbook_name "graylog2"
recipe_name "authbind"
port 514
user "graylog2"
end
[2015-01-15T15:11:39+00:00] INFO: Running queued delayed notifications before re-raising exception
[2015-01-15T15:11:39+00:00] INFO: template[/etc/default/graylog2-server] sending restart action to service[graylog2-server] (delayed)
Recipe: graylog2::server
* service[graylog2-server] action restart[2015-01-15T15:11:43+00:00] INFO: service[graylog2-server] restarted
- restart service service[graylog2-server]
Running handlers:
[2015-01-15T15:11:43+00:00] ERROR: Running exception handlers
Running handlers complete
[2015-01-15T15:11:43+00:00] ERROR: Exception handlers complete
[2015-01-15T15:11:43+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
Chef Client failed. 8 resources updated in 27.321896223 seconds
When I tried below command it display error as "curl: (6) Couldn't resolve host 'packages.graylog2.org'"
rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-2.1-repository_latest.rpm
Retrieving https://packages.graylog2.org/repo/packages/graylog-2.1-repository_latest.rpm
curl: (6) Couldn't resolve host 'packages.graylog2.org'
error: skipping https://packages.graylog2.org/repo/packages/graylog-2.1-repository_latest.rpm - transfer failed
I have manually added repository in /etc/yum.repos.d/graylog.repo
But same error message observed
Without this feature I have been able to get all of the service to startup on amazon ec2. When enabling this feature however it hangs on the test faraday-conductivity. Any ideas what to look for? I was thinking something with proxy?
I have created a role in chef using the recommended attributes from the graylog cookbook README file.
"elasticsearch": { "version": "2.2.0", "cluster": { "name": "graylog" } },
For some reason Elasticsearch is not taking the cluster name from my role attribute. The /etc/elasticsearch/elasticsearch.yml continues to use the default "elasticsearch" cluster name.
Is it possible to define the default timezone for new users in this cookbook?
The default recipe sets up the repo but doesn't allow to set a different repo URL. It would improve the cookbook if that is possible because then you do need to workaround this if a server does not have internet access.
The workaround is to not include the graylog2::default
recipe and manually check for the correct java version and set up the repo.
The cookbook allows the use of a custom repo to download graylog packages. Line 7 & 8 in the attributes/default.rb file says, By default the cookbook installs a meta package containing the key and URL for the current Graylog repository. To disable this behavior set your own repository informations here.
However even after specifying the custom repos, the default cookbook download the same package, recipes/default.rb on line 13
Downloading of the file should be moved to between line 30 & 31 in recipes/default.rb where the file is required.
My wrapper cookbook overwrites specific values like:
default['graylog2']['elasticsearch']['max_size_per_index'] = 1073741824
default['graylog2']['elasticsearch']['rotation_strategy'] = 'size'
But the web interface still shows the wrong settings after the first chef run. I have to restart it manually in order to see the changes.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.