Giter Club home page Giter Club logo

moonshine_redis's Introduction

Moonshine Redis

A plugin for installing and managing Redis, an advanced persistent key-value store. Currently this, like Moonshine's Rails recipes in general, is Ubuntu/Debian-specific.

Instructions

  • script/plugin install git://github.com/railsmachine/moonshine_redis.git
  • Configure settings if needed. You may specify:
    • Package architecture. Default is amd64.
    • Version. Default is 2.4.17, but should work fine with 2.6+
    • enable_on_boot in case you want to disable starting the service on boot at system-level (useful if you want god to do it instead, for instance). Default is true.
    • ruby_client to optionally install the given version of the redis client library as a rubygem. This also accepts latest as a value, which will install updated versions if available each time your Moonshine manifest is applied. Default is none.
    • max_fd to adjust the maximum number of open file descriptors allotted to Redis by the shell that launches it. Servers with a high number of connections may need to specify a value here. A typical default is 1024. Versions of Redis released prior to this commit have a compiled-in upper limit of 10240.

For example, in moonshine.yml:

:redis:
  :version: 2.0.1-2
  :arch: i386
  :ruby_client: 1.0.7
  :enable_on_boot: false
  :max_fd: 4096
  • Include the recipe in your Moonshine manifest

    recipe :redis

Available configuration options in the :redis: section of moonshine.yml, their default and an explanation if needed:

  • :version: 2.4.17
  • :restart_on_change: true # should redis restart after its config files change?
  • :port: 6379
  • :bind: 0.0.0.0
  • :loglevel: notice
  • :databases: 16 # Number of databases to run with. 16 should be enough for almost all uses.
  • :saves: # how often to write the rdb file based on number of changes per time period (ie: 1 change, save every 900 seconds)
      • 'save 900 1'
      • 'save 300 10'
      • 'save 60 10000'
  • :rdbcompression: false
  • :slaves: # an array of 'slaveof IP PORT', defaults to none.
  • :masterauth: # defaults to no authentication. If you want a password, :masterauth: should be set to that password.
  • :requirepass: false
  • :maxclients: 0
  • :maxmemory: 0
  • :appendonly: false
  • :appendfsync: everysec # this is the default for redis and the recommended setting.
  • :vm_enabled: false # warning: virtual memory has been removed in 2.6 and won't be added to the config file if you have version set > 2.4
  • :vm_max_memory: # defaults to unset
  • :vm_page_size: # defaults to unset
  • :vm_pages: # defaults to unset
  • :vm_max_threads: # defaults to unset

Virtual Memory and Redis

We highly recommend setting the sysctl setting vm.overcommit_memory. Thankfully, moonshine_systcl can help with this. Just install the plugin and then add the following the config/moonshine.yml:

:sysctl:
  vm.overcommit_memory: 1

You'll also need to add recipe :sysctl to your manifest.


Unless otherwise specified, all content copyright © 2014, Rails Machine, LLC

moonshine_redis's People

Contributors

benstein avatar bryantraywick avatar ches avatar dbrm avatar ernie avatar jdhollis avatar jnewland avatar kplawver avatar leejones avatar mattrogish avatar nragaz avatar railsmachine-ops avatar skalnik avatar technicalpickles avatar wfarr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

moonshine_redis's Issues

Use vm.memory_overcommit=1 by default

From http://redis.io/topics/faq

Background saving is failing with a fork() error under Linux even if I've a lot of free RAM!

Short answer: echo 1 > /proc/sys/vm/overcommit_memory :)

And now the long one:

Redis background saving schema relies on the copy-on-write semantic of fork in modern operating systems: Redis forks (creates a child process) that is an exact copy of the parent. The child process dumps the DB on disk and finally exits. In theory the child should use as much memory as the parent being a copy, but actually thanks to the copy-on-write semantic implemented by most modern operating systems the parent and child process will share the common memory pages. A page will be duplicated only when it changes in the child or in the parent. Since in theory all the pages may change while the child process is saving, Linux can't tell in advance how much memory the child will take, so if the overcommit_memory setting is set to zero fork will fail unless there is as much free RAM as required to really duplicate all the parent memory pages, with the result that if you have a Redis dataset of 3 GB and just 2 GB of free memory it will fail.

Setting overcommit_memory to 1 says Linux to relax and perform the fork in a more optimistic allocation fashion, and this is indeed what you want for Redis.

A good source to understand how Linux Virtual Memory work and other alternatives for overcommit_memory and overcommit_ratio is this classic from Red Hat Magazine, "Understanding Virtual Memory".

We already have moonshine_sysctl for managing this, so ideally it should just work when you install moonshine_redis. At the very least, the documentation should recommend it.

Support for Redis 3.0.7

Any plans on supporting Redis 3.0.7 version? It does not work if I change the configuration to 3.0.7. In application_manifest.rb:

configure(:redis => { :version => '3.0.7', :ruby_client => :latest }
recipe :redis

Exec[install redis] fails if redis-cli isn't present

There were some recent modifications to moonshine_redis to allow for upgrades to happen. This exec will try to shutdown with redis-cli first before re-installing.

I'm not sure if this worked in the first place with new installs, or if it's something to do with the shadow_puppet version with 1.9 support.

I'm thinking these two things should be split off (shutdown redis if its installed and would be updated, then install it).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.