justicel / puppet-couchbase Goto Github PK

View Code? Open in Web Editor NEW

9.0 2.0 30.0 134 KB

Puppet couchbase module for auto-scaling of couchbase with puppet

Puppet 72.34% Shell 7.39% HTML 2.88% Ruby 17.39%

puppet-couchbase's Introduction

puppet-couchbase: Scalable couchbase module

This module can install couchbase on servers, create buckets and automatically maintain the cluster.

Usage

Install a couchbase server with a standard user/password and create the password-less default bucket:

class { 'couchbase':
    size     => 1024,
    user     => 'couchbase',
    password => 'password',
    version  => latest,
}

Create additional buckets (Note the user/password):

couchbase::bucket { 'memcached':
    port     => 11211,
    size     => 1024,
    user     => 'couchbase',
    password => 'password',
    type     => 'memcached',
    replica  => 1
}

Install the SDK for your language (currently supported ruby and python):

couchbase::client { 'ruby': }

Using it with an unsupported language will install the libcouchbase-devel and libcouchbase2-libevent which are required for any other SDK. So for example if you want to install your php client you should use this to install the required libs and then install the php pecl couchbase extension using another module. Don't forget to define your relationships which in the case of module example42/php could look like::

Class['Couchbase'] -> Couchbase::Client <| |> -> Class['Php'] -> Php::Pecl::Module <| |>

You can add a moxi listener on Windows machines now. It is a resource define so can be added like so:

couchbase::moxi { 'default':
  nodes => ['127.0.0.1:8091'],
}

For more details about Moxi: http://docs.couchbase.com/moxi-manual-1.8/

Notes

This module uses a puppetdb installation (it is actually required) to generate a set of server installation files. These will run on any new node added to the cluster. What this means is that transparently you can add nodes to a Couchbase server group. This could be done via auto-scaling or other methods, as long as they all are assigned to the same server group.

Due to this, the module requires the mentioned puppetdb services as well as storeconfigs, puppetlabs/concat and puppetlabs/stdlib.

Testing

You might want to get some ruby and then:

gem install bundler
bundler install
bundle exec rake test

If you want to do acceptance testing:

bundle exec rake spec_prep
BEAKER_set=ubuntu-1204-x64 BEAKER_destroy=no rake beaker

TODO

Add the ability to do cleanup of nodes from cluster
Build more tests into module to increase coverage

puppet-couchbase's People

Contributors

Stargazers

Watchers

puppet-couchbase's Issues

resource order

Hi
the installation of the server some times failed because of the order of the resources.
can you please add the order to all the resources thus it will always succeeded in the first shut.

License terms?

There is no LICENSE file, nor are any licensing terms stated in this project's sources -- only a copyright declaration and a couple of developer's names. Hats off to these two, and to the other contributors on this project.

Can the owner of this repo provide some licensing guidance? Ideally it will be something compatible with whatever license Puppetlabs uses, in case they adopt it as "supported" or "approved".

Couchbase community install not correctly detected after first run

In the latest commits, if you're using couchbase-community-server and the curl install method, the first run works as expected. Secondary runs, however, cause a failure because the yum-registered package name differs from what the package type now expects. Here's the output you'll receive:

Error: Execution of '/bin/rpm -i /opt/couchbase-server-community-3.1.3-centos6.x86_64.rpm' returned 1: package couchbase-server-3.1.3-1823.x86_64 is already installed Error: /Stage[main]/Couchbase::Install/Package[couchbase-server-community]/ensure: change from absent to present failed: Execution of '/bin/rpm -i /opt/couchbase-server-community-3.1.3-centos6.x86_64.rpm' returned 1: package couchbase-server-3.1.3-1823.x86_64 is already installed

Listing the package in yum demonstrates the discrepancy:

[root@hqintrubystageutil01 ~]# yum list couchbase*
Loaded plugins: fastestmirror, security, versionlock
Loading mirror speeds from cached hostfile
 * epel: reflector.westga.edu
Installed Packages
couchbase-server.x86_64                                                                                                 3.1.3-1823                                                                                                  installed

For reference,, this is our instantiation of the class in our profile code:


  if $password == 'default' {
    $couchbase_passwords = hiera('couchbase::passwords', {})
    $actual_password = $couchbase_passwords[$server_group]
  } else {
    $actual_password = $password
  }
  class { '::couchbase':
    size           => $size,
    user           => $user,
    password       => $actual_password,
    version        => $version,
    edition        => $edition,
    install_method => $install_method,
    server_group   => $server_group,
  }
  if ($include_default_bucket) {
    couchbase::bucket { 'default':
      port     => $default_bucket_port,
      size     => $size,
      user     => $user,
      password => $actual_password,
      type     => $default_bucket_type,
      replica  => $default_bucket_replica,
      flush    => $default_bucket_flush,
    }
  }

Incorrect autofailover options

Hi Justice,

I have found my own that the file https://github.com/justicel/puppet-couchbase/blob/master/templates/couchbase-cluster-init.sh.erb

has two options that are not working for me (couchbase 2.1.1).:

--enable-auto-failover=1
--auto-failover-timeout=30

Indeed looking at the cli help they are part from the setting-autofailover command and not the cluster-init

Trace:

Tue Apr 14 11:00:42 +0000 2015 /Stage[main]/Couchbase::Config/Exec[couchbase-init]/returns (notice): ERROR: option --enable-auto-failover not recognized
Tue Apr 14 11:00:42 +0000 2015 /Stage[main]/Couchbase::Config/Execcouchbase-init: Failed to call refresh: /usr/local/bin/couchbase-cluster-init.sh returned 2 instead of one of [0]
Tue Apr 14 11:00:42 +0000 2015 /Stage[main]/Couchbase::Config/Execcouchbase-init: /usr/local/bin/couchbase-cluster-init.sh returned 2 instead of one of [0]

Running from command line I get the same error

EDIT:

seems related with this:

https://issues.couchbase.com/browse/MB-9933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

I modified the erb commenting the last two lines and adding the changes with curl, is dirty but works until I update:

Commented as bug in

https://issues.couchbase.com/browse/MB-9933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

--enable-auto-failover=<%= @_autofailover %> \

--auto-failover-timeout=30

if [ $? -ne 0 ]; then
echo "error with $1" >&2
fi
curl -i -u <%= @user %>:<%= @password %> http://localhost:8091/settings/autoFailover -d 'enabled=true&timeout=30'

When creating buckets if bucket B name is a substring of bucket A only bucket A will be created or vice versa

Hi,
when we define buckets like:
couchbase::bucket { 'make-love':
....
}

couchbase::bucket { 'make-love-not-war':
....
}

Only one of the two will be created.

Issue with install on Amazon Linux which is OS Family of Redhat and Centos

HI Justice,

I am having issues installing coucbase server. when put runs it downloads the file. but when it tries to install it failed. but its not a RPM file it is a XML file.

Jun 21 15:16:56 puppet-agent[2838]: (/Stage[main]/Couchbase::Install/Package[openssl098e]/ensure) created
Jun 21 15:16:56 puppet-agent[2838]: (/Stage[main]/Couchbase::Install/Exec[download_couchbase]/returns) executed successfully
Jun 21 15:16:57 puppet-agent[2838]: Execution of '/bin/rpm -i /opt/couchbase-server-community-latest-centos6.x86_64.rpm' returned 1: error: /opt/couchbase-server-community-latest-centos6.x86_64.rpm: not an rpm package (or package manifest):
Jun 21 15:16:57 i puppet-agent[2838]: (/Stage[main]/Couchbase::Install/Package[couchbase-server]/ensure) change from absent to present failed: Execution of '/bin/rpm -i /opt/couchbase-server-community-latest-centos6.x86_64.rpm' returned 1: error: /opt/couchbase-server-community-latest-centos6.x86_64.rpm: not an rpm package (or package manifest):

also i tried installing the file manually and it failed:

[root@ip- opt]# rpm -i couchbase-server-community-latest-centos6.x86_64.rpm
error: couchbase-server-community-latest-centos6.x86_64.rpm: not an rpm package (or package manifest):

the file its installing is not even a rpm
[root@ opt]# ls -al couchbase-server-community-latest-centos6.x86_64.rpm
-rw-r--r-- 1 root root 243 Jun 21 15:16 couchbase-server-community-latest-centos6.x86_64.rpm
[root@ip-10-17-103-238 opt]# file couchbase-server-community-latest-centos6.x86_64.rpm
couchbase-server-community-latest-centos6.x86_64.rpm: XML document text

No longer maintained?

@justicel Is this repository no longer being maintained?

Support Windows OS

Unable to use this module due to lacking Windows support (RE: https://github.com/justicel/puppet-couchbase/blob/master/manifests/params.pp#L72)

With Couchbase 4.5 exposing everything in their API these days, a better "rewrite" of this module would be to use their REST API and minimally expose those cluster management functions for Windows nodes to benefit from (even if installation cannot be supported and has to be done manually).

Downloads not available

It seems couchbase don't allow direct downloads anymore. Is there a way this module can be adapted to this new state of affairs?

Split brain issue

If I would simultaneously start 20 nodes each applying this module with same cluster name, is there a chance that I will get split cluster issue? After going through source code it seems like nothing would stop coucbase from doing it.

Failure of ::couchbase::bucket/Exec[bucket-create-*] Logs Passwords in Plain Text

Due to the logoutput => true setting on the Exec resource in ::couchbase::bucket, a failure of the CLI logs the password for couchbase in plaintext. If the server is using tagmail, this password will also be mailed out in the failure report.

Example output from our servers (with password and user changed):

Stage[main]/Profiles::Couchbase/Couchbase::Bucket[default]/Exec[bucket-create-default]/returns (err): change from notrun to 0 2 failed: couchbase-cli bucket-create -c 127.0.0.1 -u user -p 'password_in_plaintext' --bucket=default --bucket-type=couchbase --bucket-ramsize=1024 --bucket-port=11211 --bucket-replica=1 --enable-flush=1 returned 1 instead of one of [0,2]

Error processing archive on Ubuntu 14.04

Hello,

I seem to have the following errors when using the module with default options on Ubuntu 14.04:

==> default: Error: Execution of '/usr/bin/dpkg --force-confold -i /opt/couchbase-server-community_latest-ubuntu14.04_amd64.deb' returned 1: dpkg-deb: error:/opt/couchbase-server-community_latest-ubuntu14.04_amd64.deb' is not a debian format archive
==> default: dpkg: error processing archive /opt/couchbase-server-community_latest-ubuntu14.04_amd64.deb (--install):
==> default: subprocess dpkg-deb --control returned error exit status 2
==> default: Errors were encountered while processing:
==> default: /opt/couchbase-server-community_latest-ubuntu14.04_amd64.deb
==> default:
==> default: Error: /Stage[main]/Couchbase::Install/Package[couchbase-server-community]/ensure: change from purged to present failed: Execution of '/usr/bin/dpkg --force-confold -i /opt/couchbase-server-community_latest-ubuntu14.04_amd64.deb' returned 1: dpkg-deb: error: /opt/couchbase-server-community_latest-ubuntu14.04_amd64.deb' is not a debian format archive ==> default: dpkg: error processing archive /opt/couchbase-server-community_latest-ubuntu14.04_amd64.deb (--install): ==> default: subprocess dpkg-deb --control returned error exit status 2 ==> default: Errors were encountered while processing: ==> default: /opt/couchbase-server-community_latest-ubuntu14.04_amd64.deb ==> default:

Please let me know if you need other info.

Could not find dependency Package[libcouchbase2-libevent]

Hi,

I'm trying this module with foreman so I need to comment out most parts in init.pp which runs the script but I get the following error for Ubuntu:

Could not find dependency Package[libcouchbase2-libevent] for Package[couchbase_python] at /etc/puppet/environments/common/couchbase/manifests/client/python.pp:23

What is going wrong here ?

Thanks.

Lint errors in config.pp and moxi.pp

modules/couchbase/manifests/config.pp - WARNING: top-scope variable being used without an explicit namespace on line 56
modules/couchbase/manifests/config.pp - WARNING: top-scope variable being used without an explicit namespace on line 63
modules/couchbase/manifests/config.pp - WARNING: top-scope variable being used without an explicit namespace on line 96

I can make a pull request but was unsure of intention, or how you would like this fixed.

easiest fix is probably to add server_name to config.pp:

diff --git a/manifests/config.pp b/manifests/config.pp
index 920ded8..bf005d2 100644
--- a/manifests/config.pp
+++ b/manifests/config.pp
@@ -28,6 +28,7 @@ class couchbase::config (
   $user         = $::couchbase::user,
   $password     = $::couchbase::password,
   $server_group = 'default',
+  $server_name  = $::fqdn,
   $ensure       = $::couchbase::ensure,
   $autofailover = $::couchbase::params::autofailover,

Unrecognised escape sequence '\l' in file /Users/meklund/git/puppet/modules/couchbase/manifests/moxi.pp at line 54
Unrecognised escape sequence '\m' in file /Users/meklund/git/puppet/modules/couchbase/manifests/moxi.pp at line 54
Unrecognised escape sequence '\b' in file /Users/meklund/git/puppet/modules/couchbase/manifests/moxi.pp at line 56
Unrecognised escape sequence '\m' in file /Users/meklund/git/puppet/modules/couchbase/manifests/moxi.pp at line 56
Unrecognised escape sequence '\b' in file /Users/meklund/git/puppet/modules/couchbase/manifests/moxi.pp at line 63
Unrecognised escape sequence '\m' in file /Users/meklund/git/puppet/modules/couchbase/manifests/moxi.pp at line 63

Fix is to escape the windows slashes:

@@ -51,16 +51,16 @@ define couchbase::moxi (
   }
   elsif $::kernel == 'windows' {
     $moxi_root = 'c:\moxi'
-    $moxi_log  = "${moxi_root}\log\moxi_${port}.log"
+    $moxi_log  = "${moxi_root}\\log\\moxi_${port}.log"

-    file { "${$moxi_root}\bin\moxi-server_${port}.cmd":
+    file { "${$moxi_root}\\bin\\moxi-server_${port}.cmd":
       content => template("${module_name}/moxi-win_service.erb"),
       require => Package['moxi'],
       notify  => Service["Couchbase Moxi ${bucket} ${port}"],
     }

     exec {"register-moxi-service_${port}":
-      command => "nssm install \"Couchbase Moxi ${bucket} ${port}\" ${moxi_root}\bin\moxi-server_${port}.cmd",
+      command => "nssm install \"Couchbase Moxi ${bucket} ${port}\" ${moxi_root}\\bin\\moxi-server_${port}.cmd",
       unless  => "sc query \"Couchbase Moxi ${bucket} ${port}\"",
       path    => $::path,
       require => Package['nssm','moxi'],