evertrue / zookeeper-cookbook Goto Github PK
View Code? Open in Web Editor NEWChef cookbook for installing and managing Zookeeper.
Home Page: https://supermarket.chef.io/cookbooks/zookeeper
License: Apache License 2.0
Chef cookbook for installing and managing Zookeeper.
Home Page: https://supermarket.chef.io/cookbooks/zookeeper
License: Apache License 2.0
it untars and chown's the folder on each chef-client run:
somebox01 Recipe: zookeeper::install
somebox01 * zookeeper[3.4.6] action install
somebox01 Recipe:
somebox01 * chef_gem[zookeeper] action install (up to date)
somebox01 * chef_gem[json] action install (up to date)
somebox01 * group[zookeeper] action create (up to date)
somebox01 * user[zookeeper] action create (up to date)
somebox01 * remote_file[zookeeper-3.4.6] action create (up to date)
somebox01 * directory[/opt/zookeeper] action create (up to date)
somebox01 * execute[install zookeeper] action run
somebox01 - execute tar -C /opt/zookeeper -zxf zookeeper-3.4.6.tar.gz
somebox01 chown -R zookeeper:zookeeper /opt/zookeeper
Both exhibitor and zookeeper should log to syslog (or at least support that option).
@miah wrote a kickass cookbook for Java that we should definitely use.
Thanks for your work on this handy cookbook.
Unfortunately, deployment of the cookbook fails on CentOS hosts that do not have 'patch' installed.
ERROR: Failed to build gem native extension.
/opt/chef/embedded/bin/ruby extconf.rb
Building zkc.
tar xzf zkc-3.4.5.tar.gz 2>&1
patch -p0 < patches/zkc-3.4.5-logging.patch 2>&1
sh: patch: command not found
The 'patch' package should be added as a dependency; fortunately the package is named the same across CentOS, Debian, and Ubuntu.
Thanks!
$ chef-solo ...
....
* remote_file[/var/chef/cache/zookeeper-3.4.5.tar.gz] action create
- create new file /var/chef/cache/zookeeper-3.4.5.tar.gz[2015-01-21T08:04:18+00:00] WARN: remote_file[/var/chef/cache/zookeeper-3.4.5.tar.gz] cannot be downloaded from http://mirrors.ibiblio.org/apache/zookeeper/zookeeper-3.4.5/zookeeper-3.4.5.tar.gz: 404 "Not Found"
================================================================================
Error executing action `create` on resource 'remote_file[/var/chef/cache/zookeeper-3.4.5.tar.gz]'
================================================================================
Net::HTTPServerException
------------------------
404 "Not Found"
Getting this warning...
WARN: node['build-essential']['compiletime'] has been deprecated. Please use
node['build-essential']['compile_time'] instead. I have gracefully converted the
attribute for you, but this warning and converstion will be removed in the next
major release of the build-essential cookbook.
I'm running amazon linux (centos) and while Exhibitor and Zookeeper start fine if I manually execute the exec line of /etc/init/exhibitor.conf, the recipe itself hangs at
Recipe: zookeeper::exhibitor
for 5+ minutes before reporting a successful finish. But, chef doesn't actually start it. a ps shows no java running or anything for exhibitor.
issuing "start exhibitor" seems to just hang and not do anything.
Can you tag all versions ?
There is no tag for 1.7.3
Looks like it was removed for the 2.0.0
Zookeeper's mirror URL is directly derived from the version attribute:
default[:zookeeper][:version] = "3.4.5"
default[:zookeeper][:mirror] = "http://mirrors.ibiblio.org/apache/zookeeper/zookeeper-#{default[:zookeeper][:version]}/zookeeper-#{default[:zookeeper][:version]}.tar.gz"
It seems that overriding the version field does not change the mirror URL subsequently because the latter will already have been computed with the default version value ("3.4.5" as of now). I had to override the mirror attribute in my wrapper cookbook, which implies repeating the base URL and missing out on a potential future URL change in the Zookeeper cookbook.
It would be great if all that was required to pull in a different Zookeeper version was specifying the desired version attribute. The Opscode Cookbook Style Guide suggests to _not calculate or derive values in attributes files, [but] use a recipe instead_. I believe the Zookeeper recipe already does something similar when building the version-dependent installation path (e.g., /opt/zookeeper/zoopeeker-3.4.5
), so hopefully a similar approach should work for the mirror URL too.
I am by no means a Chef expert, so let me know if I missed something. If not, please let me know if you would like to me to submit a pull request. Would be more than happy to help out on this one.
close this ISSUE please.
I was wrong.
sorry
It would be nice if this listed the supported operating systems on the chef super market.
Does it work with Ubuntu?
Does it work with CentOs?
Getting this when running build-essential:
WARN: node['build-essential']['compiletime'] has been deprecated. Please use
node['build-essential']['compile_time'] instead. I have gracefully converted the
attribute for you, but this warning and converstion will be removed in the next
major release of the build-essential cookbook.
Trying this out on an AWS instance.
I made modifications to the default.rb, mainly the settings for s3 and changed the user to use from zookeeper to ec2-user
On a side note, I had to install "patch" on the machine before the recipe was successfully installed.
Then once it reaches the end, it hangs on:
x.x.x.x * service[exhibitor] action start
x.x.x.x - start service service[exhibitor]
x.x.x.x
x.x.x.x * service[exhibitor] action stop (up to date)
x.x.x.x * service[exhibitor] action start
x.x.x.x - start service service[exhibitor]
When i check the running processes on the machine, exhibitor is not running, but the check-local-zk.py script is.
Everytime I try to start the service:
sudo start -v -n exhibitor
I get the same behaviour and the following output in /var/log/messages
.
.
.
Jan 27 21:09:36 ip-x init: exhibitor main process ended, respawning
Jan 27 21:09:36 ip-x init: exhibitor main process (29668) terminated with status 1
Jan 27 21:09:36 ip-x init: exhibitor main process ended, respawning
Jan 27 21:09:36 ip-x init: exhibitor main process (29671) terminated with status 1
Jan 27 21:09:36 ip-x init: exhibitor respawning too fast, stopped
I tried this with the latest commit:
* 8622711 (HEAD, origin/master, origin/HEAD, master) Merge pull request #50 from N3TWORK/opsworks
If I copy the exec line from the init script, and try it on the command line it works just fine.
exec sudo -u $USER bash -c "java -Dlog4j.configuration=file:///opt/exhibitor/log4j.properties -jar /opt/exhibitor/1.5.0.jar \
> --configtype s3 --defaultconfig /var/chef/cache/defaultconfig.exhibitor --hostname x.x.x.x --port 8080 --s3config config:name --s3credentials /opt/exhibitor/exhibitor.s3.properties --s3region us-east-1 "
Any help would be appreciated.
Latest version there is 1.4.6, but this repository is (from what I can tell via changes to the metadata.rb), two patches newer.
Would be helpful for both versions to be published to the Community site.
And every previous version cries
While the two projects work together, they're unnecessarily tied together in this cookbook. It would be great to be able to easily leverage the exhibitor functionality in the community hadoop cookbook's zookeeper recipes. If this were split into two repositories, this would be possible. It would also allow for things like the community mesos cookbook, which only uses the exhibitor functionality of this cookbook, to be compatible with the hadoop cookbook.
I am willing to put in the work to split this, if necessary.
@jakedavis looks like v2.5.0 never made it to the Supermarket. Would be lovely to have it in there, seeing as it’s already been bumped & tagged here.
Having the config render stuff that's currently in default.rb be in a separate recipe that's included in default.rb would simplify enable me to use my wrapper cookbook for both AMI generation (where I don't want to generate a config, just plop java and zk onto the node) and for actually deploying a cluster
I'm attempting to download the SimpleFinance/chef-zookeeper cookbook with Librarian-Chef 0.0.4. I'm using the following Cheffile:
site 'https://supermarket.getchef.com/api/v1'
cookbook 'zookeeper'
I'm running Librarian-Chef like so:
$ librarian-chef install --verbose
Librarian-Chef fails with the following stacktrack:
[Librarian] Ruby Version: 1.9.3
[Librarian] Ruby Platform: i386-mingw32
[Librarian] Rubygems Version: 1.8.28
[Librarian] Librarian Version: 0.1.2
[Librarian] Librarian Adapter: chef
[Librarian] Librarian Adapter Version: 0.0.4
[Librarian] Project: d:/dev/temp/chef-zookeeper-client
[Librarian] Specfile: Cheffile
[Librarian] Lockfile: Cheffile.lock
[Librarian] Git: C:/Program Files (x86)/Git/bin/git.EXE
[Librarian] Git Version: 1.9.4
[Librarian] Git Environment Variables:
[Librarian] (empty)
[Librarian] Pre-Cached Sources:
[Librarian] Post-Cached Sources:
[Librarian] [:site, "https://supermarket.getchef.com/api/v1", {}]
[Librarian] Resolving zookeeper (>= 0) <https://supermarket.getchef.com/api/v1>
[Librarian] Checking manifests
[Librarian] Caching https://supermarket.getchef.com/api/v1/cookbooks/zookeeper to d:/dev/temp/chef-zookeeper-client/tmp/librarian/cache/source/chef/site/2036ece20da2a6e8/zookeeper/metadata.json
[Librarian] Performing http-get for https://supermarket.getchef.com/api/v1/cookbooks/zookeeper
[Librarian] Responded with success
[Librarian] Checking zookeeper/2.4.1 <https://supermarket.getchef.com/api/v1>
[Librarian] Unpacking tmp/librarian/cache/source/chef/site/2036ece20da2a6e8/zookeeper/version-uri/ffe81f3a5c974987/package.tar.gz to tmp/librarian/scratch/0b9b7e63e0dcfd26b0adbcdce68c3ec1
d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/source/site.rb:309:in `unpack_package!': The package archive has too many children! (RuntimeError)
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/source/site.rb:224:in `cache_version_uri_unpacked!'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/source/site.rb:93:in `block in version_uri_manifest'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/source/site.rb:381:in `memo'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/source/site.rb:92:in `version_uri_manifest'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/source/site.rb:88:in `version_manifest'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/source/site.rb:54:in `version_dependencies'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/source/site.rb:462:in `fetch_dependencies'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/manifest.rb:125:in `fetch_dependencies!'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/manifest.rb:117:in `fetched_dependencies'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/manifest.rb:81:in `dependencies'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:117:in `block in check_manifest_for_cycles'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:117:in `each'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:117:in `map'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:117:in `check_manifest_for_cycles'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:64:in `block in recursive_resolve'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:154:in `block (3 levels) in resolving_dependency_map_find_manifests'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:187:in `block in scope_checking_manifest'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:223:in `scope'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:186:in `scope_checking_manifest'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:153:in `block (2 levels) in resolving_dependency_map_find_manifests'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:211:in `block in map_find'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:210:in `each'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:210:in `map_find'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:152:in `block in resolving_dependency_map_find_manifests'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:165:in `block (2 levels) in scope_resolving_dependency'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:179:in `block in scope_checking_manifests'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:223:in `scope'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:178:in `scope_checking_manifests'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:164:in `block in scope_resolving_dependency'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:223:in `scope'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:163:in `scope_resolving_dependency'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:151:in `resolving_dependency_map_find_manifests'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:62:in `recursive_resolve'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver/implementation.rb:50:in `resolve'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/resolver.rb:23:in `resolve'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/action/resolve.rb:26:in `run'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/cli.rb:169:in `resolve!'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/cli.rb:41:in `install'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/cli.rb:26:in `block (2 levels) in bin!'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/cli.rb:31:in `returning_status'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/cli.rb:26:in `block in bin!'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/cli.rb:47:in `with_environment'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-0.1.2/lib/librarian/cli.rb:26:in `bin!'
from d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/bin/librarian-chef:7:in `<top (required)>'
from d:/dev/tools/Ruby193/bin/librarian-chef:23:in `load'
from d:/dev/tools/Ruby193/bin/librarian-chef:23:in `<main>'
The important bit from all that was:
[Librarian] Unpacking tmp/librarian/cache/source/chef/site/2036ece20da2a6e8/zookeeper/version-uri/ffe81f3a5c974987/package.tar.gz to tmp/librarian/scratch/0b9b7e63e0dcfd26b0adbcdce68c3ec1
d:/dev/tools/Ruby193/lib/ruby/gems/1.9.1/gems/librarian-chef-0.0.4/lib/librarian/chef/source/site.rb:309:in 'unpack_package!': The package archive has too many children! (RuntimeError)
I've not seen this before with any other Chef cookbooks, and a similar StackOverflow question indicates that the cookbook may be at fault in some way.
So... any thoughts? Are you able to replicate this issue at all?
On a fresh box, I need to add this to an attributes file, otherwise the chef_gem 'zookeeper'
directive will fail:
default['build-essential']['compile_time'] = true
My understanding is that this will resolve earlier in the chef run than the node.override line. Does that make sense to anyone else?
I'm not 100% clear what the Right Way to deal with this is, but this was my experience and fix that works for me.
As far as i have tested the cookbook, there is no way to set the nodes list. Every node has server.1=x.x.x.x in its conf file.
Is there a way to generate myid files or to create proper conf files (with full list of nodes)?
If you attempt to start up the vagrant vm, you will get the following error.
sowens-MBP:chef-zookeeper sowen$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'opscode_ubuntu-12.04_provisionerless' could not be found. Attempting to find and install...
default: Box Provider: virtualbox
default: Box Version: >= 0
==> default: Adding box 'opscode_ubuntu-12.04_provisionerless' (v0) for provider: virtualbox
default: Downloading: https://opscode-vm.s3.amazonaws.com/vagrant/opscode_ubuntu-12.04_provisionerless.box
==> default: Successfully added box 'opscode_ubuntu-12.04_provisionerless' (v0) for 'virtualbox'!
There are errors in the configuration of this machine. Please fix
the following errors and try again:
Vagrant:
* Unknown configuration section 'omnibus'.
The solution is to install the plugin
vagrant plugin install vagrant-omnibus
I don't like how implicit everything is. attempted to clean up in #49, but it was sort of ill-conceived
Hi -
No matter how I try to use the chef-zookeeper cookbook I get the following 'undefined method'... Could someone point out what I'm doing wrong? Thanks!
I have a wrapper cookbook (cl-zookeeper) and it depends on 'java' and 'chef-zookeeper' cookbooks.
cl-zookeeper::default.rb looks like this
node.default[:java][:install_flavor] = "openjdk"
node.default[:java][:jdk_version] = "7"
node.default[:java][:set_default] = true
node.default[:zookeeper][:version] = '3.4.6'
node.default[:zookeeper][:checksum] = '01b3938547cd620dc4c93efe07c0360411f4a66962a70500b163b59014046994'
node.default[:zookeeper][:mirror] = 'http://apache.cs.utah.edu/zookeeper'
node.default[:zookeeper][:user] = 'zookeeper'
node.default[:zookeeper][:install_dir] = '/opt/zookeeper'
zookeeper '3.4.6' do
user 'zookeeper'
mirror 'http://apache.cs.utah.edu/zookeeper/'
# Undocumented but this is the sha256 checksum...
checksum '01b3938547cd620dc4c93efe07c0360411f4a66962a70500b163b59014046994'
action :install
end
NoMethodError
-------------
undefined method `new' for Zookeeper:Module
Cookbook Trace:
---------------
/var/chef/cache/cookbooks/chef-zookeeper/recipes/default.rb:22:in `from_file'
/var/chef/cache/cookbooks/cl-zookeeper/recipes/default.rb:20:in `from_file'
Relevant File Content:
----------------------
/var/chef/cache/cookbooks/chef-zookeeper/recipes/default.rb:
15: # limitations under the License.
16:
17: node.override['build-essential']['compile_time'] = true
18:
19: include_recipe 'build-essential::default'
20: include_recipe 'java::default'
21:
22>> zookeeper node[:zookeeper][:version] do
23: user node[:zookeeper][:user]
24: mirror node[:zookeeper][:mirror]
25: checksum node[:zookeeper][:checksum]
26: install_dir node[:zookeeper][:install_dir]
27: action :install
28: end
29:
30: config_path = ::File.join(node[:zookeeper][:install_dir],
31: "zookeeper-#{node[:zookeeper][:version]}",
Documenting this here for completeness, we were seeing an issue where downloading the cookbook via berkshelf from the community site resulted in a corrupted download. It downloaded the archive file, but when unpacked we just got a single ~/.berkshelf/cookbooks/zookeeper-1.5.0
instead of a directory like it should be. The file contained the text 52 comment=<hex hash string>
but I couldn't find any meaning to it. Forcing berkshelf to download from git fixed the issue.
/cc @reset @sethvargo
Hello potential committer!
We haven't had as much time as we'd like recently to maintain this cookbook. If you use it, and you want commit bit, reply here. If you want to help maintain it, also let me know. This would include tagging version, uploading to community site, ensuring backwards-compatability of the patches, etc.
thanks
The CHANGELOG appears to be woefully out-of-date. This isn't necessarily an issue on its own, except that there are also no releases/tags, so it's rather difficult to see what's happened from version to version.
Hi,
Just messing around with Chef and tried to deploy this to EC2 using knife-ec2. After some output inlcude "include_recipe "zookeeper::exhibitor"" I eventually get an error:
"Chef::Exceptions::CookbookNotFound: Cookbook zookeeper not found. If you're loading zookeeper from another cookbook, make sure you configure the dependency in your metadata"
I assume this is the include of the zookeeper file from within exhibitor.rb. All of the recipes are on the server so I'm not really sure what the problem is. Any suggestions and sorry for the noob question.
Thanks
rewrite cd4ebeb slightly to use true/false and move logic in to template
Just trying last version of cookbook on a fresh machine. Any idea?
zookeeper_config
is used to create the configuration file for Zookeeper. As with many config files it is necessary to restart the service if this file changes. The current LWRP fails to let Chef know so there is no notifications happened when updated.
I have a PR coming that will fix it so the following resource now works:
zookeeper_config "#{node[:zookeeper][:install_dir]}/zookeeper-#{node[:zookeeper][:version]}/conf/zoo.cfg" do
config node[:zookeeper][:config]
user node[:zookeeper][:user]
action :render
notifies :restart, "runit_service[zookeeper]", :delayed
end
Now that this is approaching LWRP form (#83), we should write some tests and hook it into TravisCI/Coveralls.
Why have both? Why not just use metadata.rb, so things are simple, and pull requests like #43 don't miss changing the JSON.
If it wouldn't be too much trouble, could you create tags in GitHub when you bump the version? That way those of us using tools like Berkshelf can continue to use the old version without any problems.
Right now, this cookbook only sort of works, since it doesn't render a config file, and thus Zookeeper can never startup. We need at least these parameters, but we should be able to easily make a resource which can render a Zookeeper (Java properties) file.
Some light discussion in #83.
exhibitor.property doesn't contain .override,etc to attributes in node[:exhibitor][:defaultconfig] because chef creates the instance variable @defaultconfig
before attributes are modified.
I'm currently running chef 11.10.4, with chef-zookeeper ver 1.5.1
will need to update Usage at https://github.com/SimpleFinance/chef-zookeeper/edit/master/README.md to reflect new usage.
I am running the zookeeper 2.4.1 cookbook on os x, and get this error. From other reports its something todo with bsd tar on a mac? I guess berkshelf is untaring the cookbook... lusis/chef-logstash#294
Can you guys publish the latest version to chef supermarket and preferably publish from ubuntu?
Compiling Cookbooks...
[2014-09-19T16:53:51+00:00] INFO: WindowsPackage lightweight resource already initialized -- overriding!
================================================================================
Recipe Compile Error in /tmp/kitchen/cookbooks/zookeeper/providers/PaxHeader/config.rb
================================================================================
SyntaxError
-----------
/tmp/kitchen/cookbooks/zookeeper/providers/PaxHeader/config.rb:1: syntax error, unexpected tIDENTIFIER, expecting $end
17 gid=993687553
^
Especially for the service recipe (how-to + exhibitor support).
Running it now (using a wrapper cookbook here: https://github.com/synhershko/Isengard.Cookbooks/blob/master/isengard-zookeeper/recipes/as_service.rb) I get:
================================================================================
Error executing action `enable` on resource 'runit_service[zookeeper]'
================================================================================
RuntimeError
------------
Could not locate main runit sv_bin at "/sbin/sv". Did you remember to install runit before declaring a "runit_service" resource?
Try adding the following to the top of your recipe:
include_recipe "runit"
Cookbook Trace:
---------------
/var/lib/aws/opsworks/cache.stage2/cookbooks/runit/libraries/provider_runit_service.rb:71:in `load_current_resource'
Resource Declaration:
---------------------
# In /var/lib/aws/opsworks/cache.stage2/cookbooks/zookeeper/recipes/service.rb
46: runit_service 'zookeeper' do
47: default_logger true
48: options({
49: exec: executable_path
50: })
51: action [:enable, :start]
52: end
53: when 'exhibitor'
Compiled Resource:
------------------
# Declared in /var/lib/aws/opsworks/cache.stage2/cookbooks/zookeeper/recipes/service.rb:46:in `from_file'
runit_service("zookeeper") do
provider Chef::Provider::Service::Runit
action [:enable, :start]
supports {:restart=>true, :reload=>true, :status=>true}
retries 0
retry_delay 2
service_name "zookeeper"
pattern "zookeeper"
status_command "/sbin/sv status /etc/service"
sv_bin "/sbin/sv"
sv_dir "/etc/sv"
service_dir "/etc/service"
lsb_init_dir "/etc/init.d"
options {:exec=>"/opt/zookeeper/zookeeper-3.4.6/bin/zkServer.sh"}
log true
default_logger true
restart_on_update true
run_template_name "zookeeper"
log_template_name "zookeeper"
check_script_template_name "zookeeper"
finish_script_template_name "zookeeper"
sv_templates true
service_mirror # Declared in
service("zookeeper") do
provider Chef::Provider::Service::Simple
action [:nothing]
supports {:restart=>true, :reload=>true, :status=>true}
retries 0
retry_delay 2
service_name "zookeeper"
pattern "zookeeper"
start_command "/sbin/sv start /etc/service/zookeeper"
stop_command "/sbin/sv stop /etc/service/zookeeper"
status_command "/sbin/sv status /etc/service/zookeeper"
restart_command "/sbin/sv restart /etc/service/zookeeper"
end
cookbook_name "zookeeper"
recipe_name "service"
end
It may be broken, but more lilkely I'm missing something
================================================================================
Recipe Compile Error in /mnt/chef/chef-solo-1/cookbooks/simpleinabox/recipes/default.rb
================================================================================
Errno::ETIMEDOUT
----------------
Connection timed out - connect(2)
Cookbook Trace:
---------------
/mnt/chef/chef-solo-1/cookbooks/zookeeper/libraries/exhibitor_discovery.rb:28:in `discover_zookeepers'
/mnt/chef/chef-solo-1/cookbooks/simple-kafka/recipes/default.rb:76:in `from_file'
/mnt/chef/chef-solo-1/cookbooks/simpleinabox/recipes/default.rb:25:in `from_file'
Relevant File Content:
----------------------
/mnt/chef/chef-solo-1/cookbooks/zookeeper/libraries/exhibitor_discovery.rb:
21: require 'net/http'
22: require 'uri'
23:
24:
25: def discover_zookeepers(exhibitor_host)
26: url = URI.parse(exhibitor_host) + '/exhibitor/v1/cluster/list'
27: req = Net::HTTP::Get.new(url.path)
28>> res = Net::HTTP.start(url.host, url.port) {|http|
29: http.request(req)
30: }
31: return JSON.parse(res.body)
32: end
33:
34: def zk_connect_str(zookeepers, chroot = nil)
35: # zookeepers: as returned from discover_zookeepers
36: # chroot: optional chroot
37: #
[2013-07-09T14:35:56+00:00] ERROR: Running exception handlers
[2013-07-09T14:35:56+00:00] ERROR: Exception handlers complete
[2013-07-09T14:35:56+00:00] FATAL: Stacktrace dumped to /mnt/chef/chef-stacktrace.out
[2013-07-09T14:35:56+00:00] FATAL: Errno::ETIMEDOUT: Connection timed out - connect(2)
If you run vagrant up, then you will get a stacktrace from chef
==> default: stdin: is not a tty
==> default: [2015-01-20T00:00:38+00:00] INFO: Forking chef instance to converge...
==> default: [2015-01-20T00:00:38+00:00] INFO: *** Chef 12.0.3 ***
==> default: [2015-01-20T00:00:38+00:00] INFO: Chef-client pid: 1396
==> default: [2015-01-20T00:00:39+00:00] INFO: Setting the run_list to ["recipe[zookeeper::default]"] from CLI options
==> default: [2015-01-20T00:00:39+00:00] INFO: Run List is [recipe[zookeeper::default]]
==> default: [2015-01-20T00:00:39+00:00] INFO: Run List expands to [zookeeper::default]
==> default: [2015-01-20T00:00:39+00:00] INFO: Starting Chef Run for zookeeper
==> default: [2015-01-20T00:00:39+00:00] INFO: Running start handlers
==> default: [2015-01-20T00:00:39+00:00] INFO: Start handlers complete.
==> default: [2015-01-20T00:00:39+00:00] ERROR: Running exception handlers
==> default: [2015-01-20T00:00:39+00:00] ERROR: Exception handlers complete
==> default: [2015-01-20T00:00:39+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
==> default: [2015-01-20T00:00:39+00:00] ERROR: Cookbook zookeeper not found. If you're loading zookeeper from another cookbook, make sure you configure the dependency in your metadata
==> default: [2015-01-20T00:00:39+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.
If you look at the stack trace, it shows a missing dependency
vagrant@zookeeper:/var/chef/cache$ cat chef-stacktrace.out
cat: chef-stacktrace.out: Permission denied
vagrant@zookeeper:/var/chef/cache$ sudo !!
sudo cat chef-stacktrace.out
Generated at 2015-01-20 00:00:39 +0000
Chef::Exceptions::CookbookNotFound: Cookbook zookeeper not found. If you're loading zookeeper from another cookbook, make sure you configure the dependency in your metadata
/opt/chef/embedded/apps/chef/lib/chef/cookbook/cookbook_collection.rb:38:in `block in initialize'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/ohai-8.0.1/lib/ohai/mash.rb:77:in `yield'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/ohai-8.0.1/lib/ohai/mash.rb:77:in `default'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/ohai-8.0.1/lib/ohai/mash.rb:77:in `default'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:277:in `[]'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:277:in `each_cookbook_dep'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:255:in `add_cookbook_with_deps'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:88:in `block in cookbook_order'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:86:in `each'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:86:in `cookbook_order'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:98:in `compile_libraries'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:71:in `compile'
/opt/chef/embedded/apps/chef/lib/chef/run_context.rb:92:in `load'
/opt/chef/embedded/apps/chef/lib/chef/policy_builder/expand_node_object.rb:73:in `setup_run_context'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:235:in `setup_run_context'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:397:in `run'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:261:in `block in fork_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:249:in `fork'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:249:in `fork_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:215:in `block in run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/local_mode.rb:38:in `with_server_connectivity'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:201:in `run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/solo.rb:245:in `block in interval_run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/solo.rb:234:in `loop'
/opt/chef/embedded/apps/chef/lib/chef/application/solo.rb:234:in `interval_run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/solo.rb:224:in `run_application'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:58:in `run'
/opt/chef/embedded/apps/chef/bin/chef-solo:25:in `<top (required)>'
/usr/bin/chef-solo:40:in `load'
/usr/bin/chef-solo:40:in `<main>'vagrant@zookeeper:/var/chef/cache$ ls
chef-client-running.pid chef-stacktrace.out
I'm guessing this is a simple fix, but I am new to chef.
The Zookeeper docs recommend process supervision. As a baseline, we should have a resource which basically calls zkServer.sh start
. But as a more resilient solution, we should probably have a Runit option, and possibly others depending on peoples' cup of tea.
In a chef-client run of my wrapper cookbook, 2.4.3 was causing build-essential to error out during the catalog compilation with the error below.
I'm not entirely sure why, but I found a fix.
[2014-09-03T01:28:06+00:00] INFO: Run List is [recipe[lookout-zk::default]]
18:28:06 [2014-09-03T01:28:06+00:00] INFO: Run List expands to [lookout-zk::default]
18:28:06 [2014-09-03T01:28:06+00:00] INFO: Starting Chef Run for lookout-zk-bburton-0
18:28:06 [2014-09-03T01:28:06+00:00] INFO: Running start handlers
18:28:06 [2014-09-03T01:28:06+00:00] INFO: Start handlers complete.
18:28:07 [2014-09-03T01:28:07+00:00] INFO: HTTP Request Returned 404 Object Not Found:
18:28:07 [2014-09-03T01:28:07+00:00] INFO: Loading cookbooks [[email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]]
18:28:07 [2014-09-03T01:28:07+00:00] INFO: Storing updated cookbooks/lookout-zk/recipes/default.rb in the cache.
18:28:07 [2014-09-03T01:28:07+00:00] INFO: WindowsPackage light-weight resource already initialized -- overriding!
18:28:08 [2014-09-03T01:28:07+00:00] INFO: Zookeeper light-weight provider already initialized -- overriding!
18:28:08 [2014-09-03T01:28:08+00:00] WARN: Cloning resource attributes for execute[apt-get-update] from prior resource (CHEF-3694)
18:28:08 [2014-09-03T01:28:08+00:00] WARN: Previous execute[apt-get-update]: /var/chef/cache/cookbooks/lookout-zk/recipes/default.rb:11:in `from_file'
18:28:08 [2014-09-03T01:28:08+00:00] WARN: Current execute[apt-get-update]: /var/chef/cache/cookbooks/apt/recipes/default.rb:40:in `from_file'
18:28:08 [2014-09-03T01:28:08+00:00] INFO: Processing package[autoconf] action install (build-essential::_debian line 108)
18:28:08 [2014-09-03T01:28:08+00:00] INFO: Processing package[binutils-doc] action install (build-essential::_debian line 108)
18:28:08 [2014-09-03T01:28:08+00:00] INFO: Processing package[bison] action install (build-essential::_debian line 108)
18:28:08 [2014-09-03T01:28:08+00:00] INFO: Processing package[build-essential] action install (build-essential::_debian line 108)
18:28:09 �[0m
18:28:09 ================================================================================�[0m
18:28:09 �[31mError executing action `install` on resource 'package[build-essential]'�[0m
18:28:09 ================================================================================�[0m
18:28:09
18:28:09 �[0m
18:28:09 Mixlib::ShellOut::ShellCommandFailed�[0m
18:28:09 ------------------------------------�[0m
18:28:09 Expected process to exit with [0], but received '100'
18:28:09 ---- Begin output of apt-get -q -y install build-essential=11.5ubuntu2.1 ----
18:28:09 STDOUT: Reading package lists...
18:28:09 Building dependency tree...
18:28:09 Reading state information...
18:28:09 The following extra packages will be installed:
18:28:09 binutils cpp cpp-4.6 dpkg-dev fakeroot g++ g++-4.6 gcc gcc-4.6
18:28:09 libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl
18:28:09 libc-dev-bin libc6-dev libdpkg-perl libgomp1 libmpc2 libmpfr4 libquadmath0
18:28:09 libstdc++6-4.6-dev linux-libc-dev make manpages-dev
18:28:09 Suggested packages:
18:28:09 cpp-doc gcc-4.6-locales debian-keyring g++-multilib g++-4.6-multilib
18:28:09 gcc-4.6-doc libstdc++6-4.6-dbg gcc-multilib automake1.9 libtool flex gdb
18:28:09 gcc-doc gcc-4.6-multilib libmudflap0-4.6-dev libgcc1-dbg libgomp1-dbg
18:28:09 libquadmath0-dbg libmudflap0-dbg binutils-gold glibc-doc libstdc++6-4.6-doc
18:28:09 make-doc
18:28:09 The following NEW packages will be installed:
18:28:09 binutils build-essential cpp cpp-4.6 dpkg-dev fakeroot g++ g++-4.6 gcc
18:28:09 gcc-4.6 libalgorithm-diff-perl libalgorithm-diff-xs-perl
18:28:09 libalgorithm-merge-perl libc-dev-bin libc6-dev libdpkg-perl libgomp1 libmpc2
18:28:09 libmpfr4 libquadmath0 libstdc++6-4.6-dev linux-libc-dev make manpages-dev
18:28:09 0 upgraded, 24 newly installed, 0 to remove and 0 not upgraded.
18:28:09 Need to get 3,893 kB/30.6 MB of archives.
18:28:09 After this operation, 83.9 MB of additional disk space will be used.
18:28:09 Err http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ precise-updates/main libc-dev-bin amd64 2.15-0ubuntu10.6
18:28:09 404 Not Found [IP: 91.189.91.14 80]
18:28:09 Err http://security.ubuntu.com/ubuntu/ precise-security/main libc-dev-bin amd64 2.15-0ubuntu10.6
18:28:09 404 Not Found [IP: 91.189.91.15 80]
18:28:09 Err http://security.ubuntu.com/ubuntu/ precise-security/main linux-libc-dev amd64 3.2.0-67.101
18:28:09 404 Not Found [IP: 91.189.91.15 80]
18:28:09 Err http://security.ubuntu.com/ubuntu/ precise-security/main libc6-dev amd64 2.15-0ubuntu10.6
18:28:09 404 Not Found [IP: 91.189.91.15 80]
18:28:09 STDERR: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/e/eglibc/libc-dev-bin_2.15-0ubuntu10.6_amd64.deb 404 Not Found [IP: 91.189.91.15 80]
18:28:09 Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/l/linux/linux-libc-dev_3.2.0-67.101_amd64.deb 404 Not Found [IP: 91.189.91.15 80]
18:28:09 Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/e/eglibc/libc6-dev_2.15-0ubuntu10.6_amd64.deb 404 Not Found [IP: 91.189.91.15 80]
18:28:09 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
18:28:09 ---- End output of apt-get -q -y install build-essential=11.5ubuntu2.1 ----
18:28:09 Ran apt-get -q -y install build-essential=11.5ubuntu2.1 returned 100�[0m
18:28:09
18:28:09 �[0m
18:28:09 Cookbook Trace:�[0m
18:28:09 ---------------�[0m
18:28:09 /var/chef/cache/cookbooks/build-essential/libraries/timing.rb:114:in `block in method_missing'
18:28:09 /var/chef/cache/cookbooks/build-essential/libraries/timing.rb:113:in `each'
18:28:09 /var/chef/cache/cookbooks/build-essential/libraries/timing.rb:113:in `method_missing'
18:28:09 /var/chef/cache/cookbooks/build-essential/recipes/_debian.rb:24:in `block in from_file'
18:28:09 /var/chef/cache/cookbooks/build-essential/libraries/timing.rb:104:in `instance_eval'
18:28:09 /var/chef/cache/cookbooks/build-essential/libraries/timing.rb:104:in `evaluate'
18:28:09 /var/chef/cache/cookbooks/build-essential/libraries/timing.rb:47:in `potentially_at_compile_time'
18:28:09 /var/chef/cache/cookbooks/build-essential/recipes/_debian.rb:20:in `from_file'
18:28:09 /var/chef/cache/cookbooks/build-essential/recipes/default.rb:21:in `from_file'
18:28:09 /var/chef/cache/cookbooks/zookeeper/recipes/install.rb:20:in `from_file'
18:28:09 /var/chef/cache/cookbooks/lookout-zk/recipes/install.rb:10:in `from_file'
18:28:09 /var/chef/cache/cookbooks/lookout-zk/recipes/default.rb:34:in `from_file'�[0m
I have been having trouble installing zookeeper via chef.
Error executing action install
on resource 'chef_gem[zookeeper]'
Illformed requirement [""]
I am able to install zookeeper via cli ..
/opt/chef/embedded/bin/gem install zookeeper
I did include this line in the service.rb file - include_recipe "runit"
I still see the same issue.
Appreciate any help.
Thanks
Shekar
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.