Giter Club home page Giter Club logo

ansible-redis's Introduction

ansible-redis

Build Status Ansible Galaxy

  • Ansible 2.4+
  • Compatible with most versions of Ubuntu/Debian and RHEL/CentOS 6.x

Contents

  1. Installation
  2. Getting Started
  3. Single Redis node
  4. Master-Slave Replication
  5. Redis Sentinel
  6. Advanced Options
  7. Verifying checksums
  8. Install from local tarball
  9. Building 32-bit binaries
  10. Role Variables

Installation

$ ansible-galaxy install davidwittman.redis

Getting started

Below are a few example playbooks and configurations for deploying a variety of Redis architectures.

This role expects to be run as root or as a user with sudo privileges.

Single Redis node

Deploying a single Redis server node is pretty trivial; just add the role to your playbook and go. Here's an example which we'll make a little more exciting by setting the bind address to 127.0.0.1:

---
- hosts: redis01.example.com
  vars:
    - redis_bind: 127.0.0.1
  roles:
    - davidwittman.redis
$ ansible-playbook -i redis01.example.com, redis.yml

Note: You may have noticed above that I just passed a hostname in as the Ansible inventory file. This is an easy way to run Ansible without first having to create an inventory file, you just need to suffix the hostname with a comma so Ansible knows what to do with it.

That's it! You'll now have a Redis server listening on 127.0.0.1 on redis01.example.com. By default, the Redis binaries are installed under /opt/redis, though this can be overridden by setting the redis_install_dir variable.

Master-Slave replication

Configuring replication in Redis is accomplished by deploying multiple nodes, and setting the redis_slaveof variable on the slave nodes, just as you would in the redis.conf. In the example that follows, we'll deploy a Redis master with three slaves.

In this example, we're going to use groups to separate the master and slave nodes. Let's start with the inventory file:

[redis-master]
redis-master.example.com

[redis-slave]
redis-slave0[1:3].example.com

And here's the playbook:

---
- name: configure the master redis server
  hosts: redis-master
  roles:
    - davidwittman.redis

- name: configure redis slaves
  hosts: redis-slave
  vars:
    - redis_slaveof: redis-master.example.com 6379
  roles:
    - davidwittman.redis

In this case, I'm assuming you have DNS records set up for redis-master.example.com, but that's not always the case. You can pretty much go crazy with whatever you need this to be set to. In many cases, I tell Ansible to use the eth1 IP address for the master. Here's a more flexible value for the sake of posterity:

redis_slaveof: "{{ hostvars['redis-master.example.com'].ansible_eth1.ipv4.address }} {{ redis_port }}"

Now you're cooking with gas! Running this playbook should have you ready to go with a Redis master and three slaves.

Redis Sentinel

Introduction

Using Master-Slave replication is great for durability and distributing reads and writes, but not so much for high availability. If the master node fails, a slave must be manually promoted to master, and connections will need to be redirected to the new master. The solution for this problem is Redis Sentinel, a distributed system which uses Redis itself to communicate and handle automatic failover in a Redis cluster.

Sentinel itself uses the same redis-server binary that Redis uses, but runs with the --sentinel flag and with a different configuration file. All of this, of course, is abstracted with this Ansible role, but it's still good to know.

Configuration

To add a Sentinel node to an existing deployment, assign this same redis role to it, and set the variable redis_sentinel to True on that particular host. This can be done in any number of ways, and for the purposes of this example I'll extend on the inventory file used above in the Master/Slave configuration:

[redis-master]
redis-master.example.com

[redis-slave]
redis-slave0[1:3].example.com

[redis-sentinel]
redis-sentinel0[1:3].example.com redis_sentinel=True

Above, we've added three more hosts in the redis-sentinel group (though this group serves no purpose within the role, it's merely an identifier), and set the redis_sentinel variable inline within the inventory file.

Now, all we need to do is set the redis_sentinel_monitors variable to define the Redis masters which Sentinel should monitor. In this case, I'm going to do this within the playbook:

- name: configure the master redis server
  hosts: redis-master
  roles:
    - davidwittman.redis

- name: configure redis slaves
  hosts: redis-slave
  vars:
    - redis_slaveof: redis-master.example.com 6379
  roles:
    - davidwittman.redis

- name: configure redis sentinel nodes
  hosts: redis-sentinel
  vars:
    - redis_sentinel_monitors:
      - name: master01
        host: redis-master.example.com
        port: 6379
  roles:
    - davidwittman.redis

This will configure the Sentinel nodes to monitor the master we created above using the identifier master01. By default, Sentinel will use a quorum of 2, which means that at least 2 Sentinels must agree that a master is down in order for a failover to take place. This value can be overridden by setting the quorum key within your monitor definition. See the Sentinel docs for more details.

Along with the variables listed above, Sentinel has a number of its own configurables just as Redis server does. These are prefixed with redis_sentinel_, and are enumerated in the Role Variables section below.

Multiple role inclusions

Should you need to execute the role several times, have a look at test/test_all.yml to see how to proceed. See here and here for context.

Advanced Options

Verifying checksums

Set the redis_verify_checksum variable to true to use the checksum verification option for get_url. Note that this will only verify checksums when Redis is downloaded from a URL, not when one is provided in a tarball with redis_tarball.

When using Ansible 2.x, this role will verify the sha1 checksum of the download against checksums defined in the redis_checksums variable in vars/main.yml. If your version is not defined in here or you wish to override the checksum with one of your own, simply set the redis_checksum variable. As in the example below, you will need to prefix the checksum with the type of hash which you are using.

- name: install redis on ansible 1.x and verify checksums
  hosts: all
  roles:
    - role: davidwittman.redis
      redis_version: 3.0.7
      redis_verify_checksum: true
      redis_checksum: "sha256:b2a791c4ea3bb7268795c45c6321ea5abcc24457178373e6a6e3be6372737f23"

Install from local tarball

If the environment your server resides in does not allow downloads (i.e. if the machine is sitting in a dmz) set the variable redis_tarball to the path of a locally downloaded Redis tarball to use instead of downloading over HTTP from redis.io.

Do not forget to set the version variable to the same version of the tarball to avoid confusion! For example:

vars:
  redis_version: 2.8.14
  redis_tarball: /path/to/redis-2.8.14.tar.gz

In this case the source archive is copied to the server over SSH rather than downloaded.

Building 32 bit binaries

To build 32-bit binaries of Redis (which can be used for memory optimization), set redis_make_32bit: true. This installs the necessary dependencies (x86 glibc) on RHEL/Debian/SuSE and sets the option '32bit' when running make.

Building with TLS support

To build Redis with TLS support (Added in version 6), set redis_make_tls: true. This requires OpenSSL development libraries (e.g. libssl-dev on Debian/Ubuntu).

Role Variables

Here is a list of all the default variables for this role, which are also available in defaults/main.yml. One of these days I'll format these into a table or something.

---
## Installation options
redis_version: 2.8.24
redis_install_dir: /opt/redis
redis_dir: /var/lib/redis/{{ redis_port }}
redis_config_file_name: "{{ redis_port }}.conf"
redis_download_url: "http://download.redis.io/releases/redis-{{ redis_version }}.tar.gz"
# Set this to true to validate redis tarball checksum against vars/main.yml
redis_verify_checksum: false
# Set this value to a local path of a tarball to use for installation instead of downloading
redis_tarball: false
# Set this to true to build 32-bit binaries of Redis
redis_make_32bit: false
# Set this to true to build redis with TLS support, available only for versions >= 6 (require OpenSSL development libraries)
redis_make_tls: false

redis_user: redis
redis_group: "{{ redis_user }}"

# The open file limit for Redis/Sentinel
redis_nofile_limit: 16384

## Role options
# Configure Redis as a service
# This creates the init scripts for Redis and ensures the process is running
# Also applies for Redis Sentinel
redis_as_service: true
# Add local facts to /etc/ansible/facts.d for Redis
redis_local_facts: true
# Service name
redis_service_name: "redis_{{ redis_port }}"

## Networking/connection options
redis_bind: false
redis_port: 6379
redis_password: false
# Slave replication options
redis_min_slaves_to_write: 0
redis_min_slaves_max_lag: 10
redis_tcp_backlog: 511
redis_tcp_keepalive: 0
# Max connected clients at a time
redis_maxclients: 10000
redis_timeout: 0
# Socket options
# Set socket_path to the desired path to the socket. E.g. /var/run/redis/{{ redis_port }}.sock
redis_socket_path: false
redis_socket_perm: 755

## Replication options
# Set slaveof just as you would in redis.conf. (e.g. "redis01 6379")
redis_slaveof: false
# Make slaves read-only. "yes" or "no"
redis_slave_read_only: "yes"
redis_slave_priority: 100
redis_repl_backlog_size: false

## Logging
redis_logfile: '""'
# Enable syslog. "yes" or "no"
redis_syslog_enabled: "yes"
redis_syslog_ident: "{{ redis_service_name }}"
# Syslog facility. Must be USER or LOCAL0-LOCAL7
redis_syslog_facility: USER

## General configuration
redis_daemonize: "yes"
redis_pidfile: /var/run/redis/{{ redis_port }}.pid
# Number of databases to allow
redis_databases: 16
redis_loglevel: notice
# Log queries slower than this many milliseconds. -1 to disable
redis_slowlog_log_slower_than: 10000
# Maximum number of slow queries to save
redis_slowlog_max_len: 128
# Redis memory limit (e.g. 4294967296, 4096mb, 4gb)
redis_maxmemory: false
redis_maxmemory_policy: noeviction
redis_rename_commands: []
redis_db_filename: dump.rdb
# How frequently to snapshot the database to disk
# e.g. "900 1" => 900 seconds if at least 1 key changed
redis_save:
  - 900 1
  - 300 10
  - 60 10000
redis_stop_writes_on_bgsave_error: "yes"
redis_rdbcompression: "yes"
redis_rdbchecksum: "yes"
redis_appendonly: "no"
redis_appendfilename: "appendonly.aof"
redis_appendfsync: "everysec"
redis_no_appendfsync_on_rewrite: "no"
redis_auto_aof_rewrite_percentage: "100"
redis_auto_aof_rewrite_min_size: "64mb"
redis_notify_keyspace_events: '""'

## Additional configuration options
# leave empty if not required. Use a block style scalar to add options, e.g.
# redis_config_additional: |
#   io-threads 4
#   io-threads-do-reads yes
redis_config_additional: ""

## Redis sentinel configs
# Set this to true on a host to configure it as a Sentinel
redis_sentinel: false
redis_sentinel_dir: /var/lib/redis/sentinel_{{ redis_sentinel_port }}
redis_sentinel_bind: 0.0.0.0
redis_sentinel_port: 26379
redis_sentinel_password: false
redis_sentinel_pidfile: /var/run/redis/sentinel_{{ redis_sentinel_port }}.pid
redis_sentinel_logfile: '""'
redis_sentinel_syslog_ident: sentinel_{{ redis_sentinel_port }}
redis_sentinel_monitors:
  - name: master01
    host: localhost
    port: 6379
    quorum: 2
    auth_pass: ant1r3z
    down_after_milliseconds: 30000
    parallel_syncs: 1
    failover_timeout: 180000
    notification_script: false
    client_reconfig_script: false
    rename_commands: []

Facts

The following facts are accessible in your inventory or tasks outside of this role.

  • {{ ansible_local.redis.bind }}
  • {{ ansible_local.redis.port }}
  • {{ ansible_local.redis.sentinel_bind }}
  • {{ ansible_local.redis.sentinel_port }}
  • {{ ansible_local.redis.sentinel_monitors }}

To disable these facts, set redis_local_facts to a false value.

ansible-redis's People

Contributors

actionjack avatar adborden avatar agapov97 avatar avra911 avatar camjay avatar carljm avatar darthwade avatar davidwittman avatar dependabot[bot] avatar gomes avatar henyxia avatar loliee avatar mattock avatar mikechau avatar mpataki avatar mtpereira avatar neofob avatar nickjj avatar oaleynik avatar rathboma avatar slapers avatar snopoke avatar splitice avatar thbar avatar thetortuga avatar viktorcollin avatar wesgarrison avatar yamikep avatar yannik avatar zephyrean avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ansible-redis's Issues

Dynamic selection of master

Hi there,

I've been doing some manual testing with Redis sentinel and its election of a new master. This all works fine as expected πŸ˜„

The issue I've got is that my inventory is currently as follows:

[redis_master]
redis01.staging

[redis_slaves]
redis02.staging
redis03.staging

If, for example, redis02.staging has previously become the master due to a reelection then re-running the playbook appears to break my setup. If redis01.staging is still the master then it works fine.

Do you think I need to use a dynamic inventory to calculate which member if redis0[1:3].staging is the master? I think this issue may be down to the redis-server config file being rewritten as part of the sentinel re-election process.

Thanks!

Add status reporting to init.d script

It would be very useful to have the ability to see the Redis service(s) status via service status.

I guess we could use:

  • Debian -> status_of_proc via /lib/lsb/init-functions
  • RedHat -> status via /etc/init.d/functions

Happy to have a crack at it but as it's non-trivial I thought I'd raise an issue to capture the requirement πŸ˜„

Missing dependencies on Debian Wheezy

I think there might be missing dependencies that prevent redis from compiling. I setup a new debian wheezy instance and redis was the first thing on my list to install, it crashed when trying to compile it which is listed below.

After installing: postgres, nodejs and ruby then redis was able to compile but only after I manually deleted the src/redis folder because the role wasn't able to recover.

GATHERING FACTS *************************************************************** 
ok: [teststack]

TASK: [DavidWittman.redis | install dependencies] ***************************** 
changed: [teststack] => (item=gcc,make)

TASK: [DavidWittman.redis | install dependencies] ***************************** 
skipping: [teststack]

TASK: [DavidWittman.redis | enable overcommit in sysctl] ********************** 
changed: [teststack]

TASK: [DavidWittman.redis | add redis user] *********************************** 
changed: [teststack]

TASK: [DavidWittman.redis | download redis] *********************************** 
changed: [teststack]

TASK: [DavidWittman.redis | extract redis tarball] **************************** 
changed: [teststack]

TASK: [DavidWittman.redis | compile redis] ************************************ 
failed: [teststack] => {"changed": true, "cmd": ["make", "-j5"], "delta": "0:00:00.841175", "end": "2014-08-02 05:49:29.803057", "rc": 2, "start": "2014-08-02 05:49:28.961882"}
stderr: lapi.c:8:20: fatal error: assert.h: No such file or directory
compilation terminated.
lcode.c:8:20: fatal error: stdlib.h: No such file or directory
compilation terminated.
linenoise.c:97:21: fatal error: termios.h: No such file or directory
compilation terminated.
net.c:34:23: fatal error: sys/types.h: No such file or directory
compilation terminated.
make[3]: *** [lapi.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: *** [net.o] Error 1
make[3]: *** Waiting for unfinished jobs....
hiredis.c:33:20: fatal error: string.h: No such file or directory
compilation terminated.
make[3]: *** [lcode.o] Error 1
make[2]: *** [lua] Error 2
make[2]: *** Waiting for unfinished jobs....
make[3]: *** [linenoise.o] Error 1
make[2]: *** [linenoise] Error 2
make[3]: *** [hiredis.o] Error 1
make[2]: *** [hiredis] Error 2
configure: error: in `/usr/local/src/redis-2.8.9/deps/jemalloc':
configure: error: C compiler cannot create executables
See `config.log' for more details
make[2]: *** [jemalloc] Error 77
make[1]: [persist-settings] Error 2 (ignored)
    CC adlist.o
    CC ae.o
    CC anet.o
    CC dict.o
anet.c:33:23: fatal error: sys/types.h: No such file or directory    CC redis.o
ae.c:33:19: fatal error: stdio.h: No such file or directory
compilation terminated.
adlist.c:32:20: fatal error: stdlib.h: No such file or directory
compilation terminated.
dict.c:38:19: fatal error: stdio.h: No such file or directory
compilation terminated.

compilation terminated.
In file included from redis.h:34:0,
                 from redis.c:30:
config.h:89:27: fatal error: linux/version.h: No such file or directory
compilation terminated.
make[1]: *** [anet.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [ae.o] Error 1
make[1]: *** [dict.o] Error 1
make[1]: *** [adlist.o] Error 1
make[1]: *** [redis.o] Error 1
make: *** [all] Error 2
stdout: cd src && make all
make[1]: Entering directory `/usr/local/src/redis-2.8.9/src'
rm -rf redis-server redis-sentinel redis-cli redis-benchmark redis-check-dump redis-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html
(cd ../deps && make distclean)
make[2]: Entering directory `/usr/local/src/redis-2.8.9/deps'
(cd hiredis && make clean) > /dev/null || true
(cd linenoise && make clean) > /dev/null || true
(cd lua && make clean) > /dev/null || true
(cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true
(rm -f .make-*)
make[2]: Leaving directory `/usr/local/src/redis-2.8.9/deps'
(rm -f .make-*)
echo STD=-std=c99 -pedantic >> .make-settings
echo WARN=-Wall >> .make-settings
echo OPT=-O2 >> .make-settings
echo MALLOC=jemalloc >> .make-settings
echo CFLAGS= >> .make-settings
echo LDFLAGS= >> .make-settings
echo REDIS_CFLAGS= >> .make-settings
echo REDIS_LDFLAGS= >> .make-settings
echo PREV_FINAL_CFLAGS=-std=c99 -pedantic -Wall -O2 -g -ggdb   -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src -DUSE_JEMALLOC -I../deps/jemalloc/include >> .make-settings
echo PREV_FINAL_LDFLAGS=  -g -ggdb -rdynamic >> .make-settings
(cd ../deps && make hiredis linenoise lua jemalloc)
make[2]: Entering directory `/usr/local/src/redis-2.8.9/deps'
(cd hiredis && make clean) > /dev/null || true
(cd linenoise && make clean) > /dev/null || true
(cd lua && make clean) > /dev/null || true
(cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true
(rm -f .make-*)
(echo "" > .make-ldflags)
(echo "" > .make-cflags)
MAKE hiredis
MAKE linenoise
cd hiredis && make static
MAKE lua
cd linenoise && make
cd lua/src && make all CFLAGS="-O2 -Wall -DLUA_ANSI " MYLDFLAGS=""
MAKE jemalloc
make[3]: Entering directory `/usr/local/src/redis-2.8.9/deps/linenoise'
cd jemalloc && ./configure --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops " LDFLAGS=""
cc  -Wall -Os -g  -c linenoise.c
make[3]: Entering directory `/usr/local/src/redis-2.8.9/deps/lua/src'
cc -O2 -Wall -DLUA_ANSI    -c -o lapi.o lapi.c
cc -O2 -Wall -DLUA_ANSI    -c -o lcode.o lcode.c
make[3]: Entering directory `/usr/local/src/redis-2.8.9/deps/hiredis'
cc -std=c99 -pedantic -c -O3 -fPIC  -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb  net.c
cc -std=c99 -pedantic -c -O3 -fPIC  -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb  hiredis.c
make[3]: Leaving directory `/usr/local/src/redis-2.8.9/deps/lua/src'
make[3]: Leaving directory `/usr/local/src/redis-2.8.9/deps/linenoise'
make[3]: Leaving directory `/usr/local/src/redis-2.8.9/deps/hiredis'
checking for xsltproc... no
checking for gcc... gcc
checking whether the C compiler works... no
make[2]: Leaving directory `/usr/local/src/redis-2.8.9/deps'
make[1]: Leaving directory `/usr/local/src/redis-2.8.9/src'

FATAL: all hosts have already failed -- aborting

Define handlers with port in name

If this role is defined multiple times with parameterized variables (as in test/test_all.yml), the handler notification can cause some unexpected behaviors due to namespace overlap. Because the handlers are all named the same (e.g. restart redis or restart sentinel), multiple role inclusions define the handler the same. When those handlers are later notified, only the handler from the first inclusion is run.

The behavior is much better documented in this issue:

ansible/ansible#4853

To fix, change the handler name to restart redis {{ redis_port }} and restart sentinel {{ redis_sentinel_port }}. We may need to bump the minimum Ansible version in response to this issue.

Override properties are not affecting the redis configuration: redis_password & redis_logfile

When adding the following properties to the vars/main.yml and including the file in the main.yml that
i.e.


vars_files:
vars/main.yml
tasks:
include: tasks/master.yml


and using the DavidWittman.redis role
i.e.


name: "Configure the master redis server."
hosts: redis-master
roles:
DavidWittman.redis
sudo: yes


The following properties in vars/main.yml do not seem to have an effect on the redis configuration file:


redis_password: somePasswordThatYouWillNeverGUessBecauseItIsSoExcruciatinglyLong
redis_logfile: '/var/log/redis.log'


Which look like the following after the Ansible run is compete.


requirepass foobared

logfile ""


Sorry the markdown is interfering with the code above but you should get the gist of it!

Failing with vagrant 1.4.3 + ansible 1.9 (dev)

Hello, i'm trying to use your role script to configure a vagrant virtual machine.

My playbook:


---
- hosts: all 
  user: vagrant
  sudo: yes 
  roles:
    - DavidWittman.redis
    - database

Running "vagrant up" i get:

Bringing machine 'payroll' up with 'virtualbox' provider...
[payroll] Importing base box 'precise32'...
[payroll] Matching MAC address for NAT networking...
[payroll] Setting the name of the VM...
[payroll] Clearing any previously set forwarded ports...
[payroll] Clearing any previously set network interfaces...
[payroll] Preparing network interfaces based on configuration...
[payroll] Forwarding ports...
[payroll] -- 22 => 2222 (adapter 1)
[payroll] Booting VM...
[payroll] Waiting for machine to boot. This may take a few minutes...
[payroll] Machine booted and ready!
[payroll] The guest additions on this VM do not match the installed version of
VirtualBox! In most cases this is fine, but in rare cases it can
prevent things such as shared folders from working properly. If you see
shared folder errors, please make sure the guest additions within the
virtual machine match the version of VirtualBox you have installed on
your host and reload your VM.

Guest Additions Version: 4.2.0
VirtualBox Version: 4.3
[payroll] Configuring and enabling network interfaces...
[payroll] Mounting shared folders...
[payroll] -- /vagrant
[payroll] Running provisioner: ansible...

PLAY [all] ******************************************************************** 

GATHERING FACTS *************************************************************** 
ok: [payroll]

TASK: [DavidWittman.redis | install dependencies] ***************************** 
changed: [payroll] => (item=gcc,make,libc6-dev)

TASK: [DavidWittman.redis | install dependencies] ***************************** 
skipping: [payroll]

TASK: [DavidWittman.redis | enable overcommit in sysctl] ********************** 
changed: [payroll]

TASK: [DavidWittman.redis | add redis user] *********************************** 
changed: [payroll]

TASK: [DavidWittman.redis | download redis] *********************************** 
changed: [payroll]

TASK: [DavidWittman.redis | upload redis] ************************************* 
skipping: [payroll]

TASK: [DavidWittman.redis | extract redis tarball] **************************** 
changed: [payroll]

TASK: [DavidWittman.redis | compile redis] ************************************ 
changed: [payroll]

TASK: [DavidWittman.redis | create redis install directory] ******************* 
ok: [payroll]

TASK: [DavidWittman.redis | create /etc/redis] ******************************** 
changed: [payroll]

TASK: [DavidWittman.redis | create /var/run/redis] **************************** 
changed: [payroll]

TASK: [DavidWittman.redis | install redis] ************************************ 
changed: [payroll]

TASK: [DavidWittman.redis | list redis binaries] ****************************** 
ok: [payroll]

TASK: [DavidWittman.redis | add redis binaries to alternatives] *************** 
failed: [payroll] => (item=redis-benchmark) => {"failed": true, "item": "redis-benchmark", "parsed": false}
SUDO-SUCCESS-ukekmrntwbssexuzjesgrgrsnvxovkqg
Traceback (most recent call last):
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.08-274944950675321/alternatives", line 1723, in <module>
    main()
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.08-274944950675321/alternatives", line 126, in main
    line_name, line_mode, line_path = line.strip().split("\t")
ValueError: need more than 1 value to unpack

failed: [payroll] => (item=redis-check-aof) => {"failed": true, "item": "redis-check-aof", "parsed": false}
SUDO-SUCCESS-herdhrmjwuehyatkopuplogtywaosaph
Traceback (most recent call last):
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.27-64895011136970/alternatives", line 1723, in <module>
    main()
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.27-64895011136970/alternatives", line 126, in main
    line_name, line_mode, line_path = line.strip().split("\t")
ValueError: need more than 1 value to unpack

failed: [payroll] => (item=redis-check-dump) => {"failed": true, "item": "redis-check-dump", "parsed": false}
SUDO-SUCCESS-fyqfxpjfoeilddbqlthcosdoejjphyhf
Traceback (most recent call last):
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.45-51532570914849/alternatives", line 1723, in <module>
    main()
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.45-51532570914849/alternatives", line 126, in main
    line_name, line_mode, line_path = line.strip().split("\t")
ValueError: need more than 1 value to unpack

failed: [payroll] => (item=redis-cli) => {"failed": true, "item": "redis-cli", "parsed": false}
SUDO-SUCCESS-htrsqivxvwgfilfataujadaqqxdrhieq
Traceback (most recent call last):
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.61-128316913027241/alternatives", line 1723, in <module>
    main()
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.61-128316913027241/alternatives", line 126, in main
    line_name, line_mode, line_path = line.strip().split("\t")
ValueError: need more than 1 value to unpack

failed: [payroll] => (item=redis-server) => {"failed": true, "item": "redis-server", "parsed": false}
SUDO-SUCCESS-uvahlwknzeuzpxrpwchvuooglmchmefi
Traceback (most recent call last):
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.79-139109903766318/alternatives", line 1723, in <module>
    main()
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1424885715.79-139109903766318/alternatives", line 126, in main
    line_name, line_mode, line_path = line.strip().split("\t")
ValueError: need more than 1 value to unpack


FATAL: all hosts have already failed -- aborting

PLAY RECAP ******************************************************************** 
           to retry, use: --limit @/home/menegazzo/playbook.retry

payroll                    : ok=12   changed=9    unreachable=0    failed=1   

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

Seems to be an issue with ansible, but i have no idea where to start.

Any tips will be welcome.

Verify checksums

Since Redis does not provide HTTPS download URLs, it's probably intelligent to verify checksums for the downloaded tarballs. Salvatore provides these checksums in the redis-hashes repository.

This will require a flag to disable verification as well.

Ansible didn't continue after enable service step

This line https://github.com/DavidWittman/ansible-redis/blob/master/tasks/server.yml#L24

fatal: [104.199.185.189]: FAILED! => {"changed": false, "failed": true, "msg": "Could not find the requested service \"'redis_6379'\": "}

I have to manually SSH into the server and run below command
systemctl enable redis_6379

I use Debian Jessie on GCP

The playbook then stop again when i'm trying to deploy sentinel.

I already set ansible_become=true in the host file
i use ansible 2.2 on OSX.

Please advise

How to execute "cluster" commands?

Hi,

A newbie question but in the config files there's no "cluster-enabled yes" setting so redis-cli> cluster commands, like "cluster nodes" gives: "ERR This instance has cluster support disabled". How should I overcome this?

Thank you,
Balazs

When install dir does contain redis version number, new version is not installed

When you change the version number of redis, like if you want to upgrade the version is not put into work.

This is due to the following task:

- name: install redis
  command: make PREFIX={{ redis_install_dir }} install
           chdir=/usr/local/src/redis-{{ redis_version }}
           creates={{ redis_install_dir }}/bin/redis-server

Due to the creates option, and because the redis_install_dir does not contain the redis version number, this new version is never installed, but just extracted and compiled.

Can you start tagging your releases?

Hello,

I'm using your role in a tool and in the near future I'm going to have it manage role dependencies since it dumps out code using those roles and I want to make sure the correct version of the roles are being used.

My issue is you're not tagging your releases (neither through github releases or standard tags) so there's no way for me to version lock your role.

Can you start tagging them? Thanks.

hangs on restarting redis

Hi,

Very nice Ansible script. Good job. Thank you, it helps a lot.

However, when I run it, it hangs on the "restart redis" task, I have to stop it manually:

NOTIFIED: [rolebooks/db/redis | restart redis] ********************************

Do you have the same behavior?

When I look at the services running on the targeted machine, it seems to be ok:

vagrant@vagrant-ubuntu-trusty-64:/$ ps aux | grep redis
root      6851  0.0  0.1   4444   648 ?        S    01:10   0:00 /bin/sh /etc/init.d/redis_6379 start
redis     6855  0.3  0.4  36740  2388 ?        Sl   01:10   0:00 /opt/redis/bin/redis-server 127.0.0.1:6379

Here is how I used the script:

- hosts: all
  sudo: yes  
  roles:
    - { 
        role: rolesbooks/db/redis,
        redis_version: "2.8.17",
        redis_bind: "127.0.0.1",
        redis_port: "6379",
        redis_logfile: "{{project_dirs.logs}}/redis.log",
        redis_databases: "1",
        redis_daemonize: "no",
        redis_maxmemory: "100mb",
        redis_maxmemory_policy: "allkeys-lru",
      }   

Thanks.

OS Permission Denied Error during enable overcommit in sysctl step

Thanks for creating this playbook! I'm having a small problem during installation, and I'm not 100% sure if it's a problem with the package or my vagrant box, or with something I'm doing.

Here's the output I'm getting when I attempt to provision:

...

TASK: [DavidWittman.redis | install dependencies] ***************************** 
ok: [default] => (item=gcc,make,libc6-dev)

TASK: [DavidWittman.redis | install dependencies] ***************************** 
skipping: [default]

TASK: [DavidWittman.redis | enable overcommit in sysctl] ********************** 
failed: [default] => {"failed": true, "parsed": false}
Traceback (most recent call last):
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1429466343.23-73074173030441/sysctl", line 1936, in <module>
    main()
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1429466343.23-73074173030441/sysctl", line 335, in main
    result = SysctlModule(module)    
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1429466343.23-73074173030441/sysctl", line 118, in __init__
    self.process()
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1429466343.23-73074173030441/sysctl", line 164, in process
    self.write_sysctl()
  File "/home/vagrant/.ansible/tmp/ansible-tmp-1429466343.23-73074173030441/sysctl", line 302, in write_sysctl
    fd, tmp_path = tempfile.mkstemp('.conf', '.ansible_m_sysctl_', os.path.dirname(self.sysctl_file))
  File "/usr/lib/python2.7/tempfile.py", line 300, in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags)
  File "/usr/lib/python2.7/tempfile.py", line 235, in _mkstemp_inner
    fd = _os.open(file, flags, 0600)
OSError: [Errno 13] Permission denied: '/etc/.ansible_m_sysctl_lh1_3d.conf'


FATAL: all hosts have already failed -- aborting

PLAY RECAP ******************************************************************** 
           to retry, use: --limit @/Users/ryanisnan/vagrant.retry

default                    : ok=7    changed=5    unreachable=0    failed=1   

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

Setting advanced values in configuration file

I would like to set values in the redis.config under "Advanced" such as hash-max-ziplist-entries.

Is there a preferred method for doing that?

Would a new role variable for a redis.config file to override the default tempting solution be an option?

Thanks,
Ben

Restart shutdown redis on previous bind IP when changed

When changing the bind address of the redis the managed redis instances, ansible hangs on waiting for the service to come back up. I think that this is because the bind address is changed in the config file, then the redis service is restarted using a command based on the new bind address.

Test

Use this awesome repo to create a redis instance bound to 127.0.0.1, then modify the playbook to bind to the machine's actual IP.

Let me know if you would like more info, or can recommend a solution that I can implement.

unable to install redis sentinel on redis master and/or slave

Hello,

I use a configuration like that.

[redis-master]
redis-1

[redis-slave]
redis-2
redis-3

[redis-sentinel]
redis-1 redis_sentinel=True
redis-2 redis_sentinel=True
redis-3 redis_sentinel=True

in this case, sentinel configuration override the master/slave configuration.
i need to play 2 times my ansible scripts (the second time i have to comment the redis-sentinel section) to have a correct setup.

README role name is incorrect?

From the README:

$ ansible-galaxy install DavidWittman.redis
---
- hosts: redis01.example.com
  vars:
    - redis_bind: 127.0.0.1
  roles:
    - redis

When I run that:

/data/src/my-app [master]* $ ansible-playbook --i deployment/hosts.test deployment/setup.yml
ERROR: cannot find role in /data/src/my-app/deployment/roles/redis or /data/src/my-app/deployment/redis or /etc/ansible/roles/redis

However, it works when I rename the role to DavidWittman.redis

Sentinels Rewrite Config File

If I run the ansible role and deploy sentinels and a normal redis host, and then later I do it again, it looks like the role rewrites the config file, erasing all of the persistence sentinels have stored there about who the slaves are, and the other sentinels, along with who the current master is.

Is there a way around that, or am I doing something wrong?
Is there a way to edit settings without resetting sentinels back to their initial state?

Add variable to set open file limit

Since we're not running redis as root, we'll need to bump up the open file limit for the redis user. This will likely need to be set in the init script (via a variable in /etc/default) or in /etc/security/limits.conf.

/var/run/redis/redis.pid --> init file needs to create /var/run/redis

since /var/run is not permanent (well, subfolders and files) i added to init script

DAEMON=/opt/redis/bin/redis-server
PIDFOLDER=/var/run/redis
PIDFILE=${PIDFOLDER}/${REDIS_PORT}.pid

REDIS_USER=redis
CONF="/etc/redis/${REDIS_PORT}.conf"
CLIEXEC=/opt/redis/bin/redis-cli

mkdir ${PIDFOLDER}
chown $REDIS_USER:root ${PIDFOLDER}

if [ -r /etc/default/redis_${REDIS_PORT} ]; then
. /etc/default/redis_${REDIS_PORT}
fi

Add systemd support

We may be able to rely on ansible_service_mgr for detecting which init system to use.

Update init scripts to include Redis bind address

If a bind address other than localhost/0.0.0.0 is used, the Redis init scripts fail to cleanly shutdown the service. We'll need to account for this by including the bind address either in the init script itself, or in the init config file in /etc/{sysconfig,default}.

Restart Fails when Password Changes

If you use the ansible role to change the auth for a server, it will rewrite the password in the config file.
To restart, though, the generated service uses the CLI interface to run commands, with the CLI interface reading from the config file we've already overwritten, so it just fails to auth to itself, since it's using the new password while the old server is still running.

This is made worse because the service just fires the CLI message and then takes it on faith that it will work, and goes into a while loop waiting for the pid file to disappear to declare success.

But, when there's an auth error and the server doesn't actually get the message about the restart, the restart service will block for all time.

How to deploy multi redis

hi:

If i want to deploy 32 instances redis, and i have to write 32 times "redis_port"

how can i combine that such as
redis_port: 2000-2031

Supporting HA Master with Slaves

Not an issue, but more of a call for assistance!

Is there a way with ansible-redis (I assume using Sentenial) to have a High Availability setup for Master, and with multiple slaves connecting to the active master.

So for example;

        +--------+       +--------+
        -   M1   -  <=>  -   S1   -
        +--------+       +--------+
      _ _ _ | _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
     /                  \                  \         
+--------+          +--------+         +--------+
-   C1   -          -   C2   -         -   C3   - 
+--------+          +--------+         +--------+

S1 is a slave of M1, if M1 were to fail, S1 would be promoted to Master.

C1, C2, C3 are all slaves of M1 or S1 (depending which is the active master)

Thanks,
Adam

Virtual Memory Config Support In Role Variables?

I have a use case that will need me to configure Redis to use virtual memory. Looking at the readme/docs I can't find support in your Ansible role for virtual memory options. Are those options unsupported, or just undocumented?

Symlink redis to /usr/bin

As far as I can tell, both redis-cli and redis-server aren't symlinked to /usr/bin during install. Is there any reasoning behind this?

Make Install Stage Skippable and Remove Source

Great module; very nicely setup Redis on my CentOS7 system. Much easier than writing it myself. However, I found myself wanting a couple features:

  1. Don't rerun install if the correct version is already installed. (I realized there are pros and cons to always rerunning the install stage, but if Redis is already installed, I'd like to skip running download/make/etc.)
  2. Remove source after installation.

My first pass looks like this for main.yml


---
- name: Check redis-server version
  command: "redis-server --version"
  register: redis_sever_version
  # May not be installed, or may be the wrong version
  # Version format is: Redis server v<version> sha=...
  # So [2][1:] is all of the second word except the leading 'v'
  changed_when: redis_sever_version.rc or redis_sever_version.stdout.split(' ')[2][1:]|version_compare(redis_version, '<')
  failed_when: False
  always_run: yes

- include: install.yml
  when: redis_sever_version.changed

- file: name=/usr/local/src/redis-{{ redis_version }} state=absent
  when: redis_sever_version.changed

- file: name=/usr/local/src/redis-{{ redis_version }}.tar.gz state=absent
  when: redis_sever_version.changed

- include: server.yml
  when: not redis_sentinel
  tags:
    - config

- include: sentinel.yml
  when: redis_sentinel
  tags:
    - config

- include: local_facts.yml

Consider using official packages

Hi,

I've started using your role (thank your for sharing your work), and I've noticed that it is compiling Redis from sources. It's great if you want to have the latest (or a specific) version of it, but not so great when you want the packaged version from the distribution.

I've noticed that the Elasticsearch role is offering the possibility to use the package from the official repository, use a downloaded package or compile from source. It's definitely more complex to handle, but very good for the user, who is able to choose how he wants things to be installed.

Thanks again for sharing this πŸ™‡

/var/run/redis_2323 directory cannot be created in systemd service

Ubuntu 16.04 service not working by default.

Problem:

ExecStartPre=-/bin/mkdir /var/run/redis

Fails on service start (after reboot) because /run is tmpfs, and var/run/redis is removed on reboot. The error is:

# systemctl restart redis_2323
Job for redis_2323.service failed because the control process exited with error code. See "systemctl status redis_2323.service" and "journalctl -xe" for details.

# journalctl -xe
mkdir[1937]: /bin/mkdir: cannot create directory '/var/run/redis': Read-only file system
chown[1940]: /bin/chown: cannot access '/var/run/redis': No such file or directory
systemd[1]: redis_2323.service: Control process exited, code=exited status=1
systemd[1]: Failed to start Advanced key-value store.
-- Subject: Unit redis_2323.service has failed
...

For workaround, I set this in the variables:

redis_pidfile: "/var/lib/redis/{{ redis_port }}.pid"

I tried this in the [Service]:
PermissionsStartOnly=true

It did not helped.

Adding reids user fails if user already exists and has a process attached to it

Hi @DavidWittman ,

When installing sentinel after already have redis installed, it fails in the "add user" step as redis is currently running and the user is attached to it.

TASK: [ansible-redis | add redis user] ****************************************
failed: [ruben-redis-sentinel] => {"failed": true, "name": "redis", "rc": 8}
msg: usermod: user redis is currently used by process 23205

FATAL: all hosts have already failed -- aborting

I guess a quick solution would be to check if the user already exists.

 name: check if redis user exists
  shell: id {{ redis_user }} 2> /dev/null | wc -l
  register: redis_user_exists

- name: add redis user
  user:
    name={{ redis_user }}
    comment="Redis"
    home={{ redis_install_dir }}
    shell=/bin/false
    system=yes
  when: redis_user_exists.stdout == "0"

Cheers and thanks for the role.

Ansible Standard Code structure

Hi, DavidWittman,
i have noticed a standard playbook pattern should be:

-- group_vars -- all
-- roles -- redis -- files
-- roles -- redis -- tasks -- main.yml
-- roles -- redis -- templates
-- hosts
-- site.yml

$ ansible-playbook -i hosts site.yml

so i'm just wondering how you organize your playbook structure ? Coz i have not seen this pattern in Ansible Docs.
Thx

No License

Hey David, could you add an appropriate license to the repo so that it can be used without legal ambiguity? Cheers!

Systemd service does not start

When the role tries to start the redis service, I get this error:

[/etc/systemd/system/redis_6379.service:10] Executable path is not absolute, ignoring: "/opt/redis/bin/redis-server" /etc/redis/6379.conf
systemd[1]: redis_6379.service lacks ExecStart setting. Refusing.

I think the problem is that we have some overquoting in redis.service.j2. I end up with this line in the output file /etc/systemd/system/redis_NNNN.service:

ExecStart="/opt/redis/bin/redis-server" /etc/redis/6379.conf

After manually modifying the file to look like this, I can start the service:

ExecStart=/opt/redis/bin/redis-server /etc/redis/6379.conf

New release

Hey thank you for your role.
Can you make a new release after c7d2afc for ansible v2 ?

service enabled not working with Ubuntu 16.04

OS: Ubuntu 16.04
Ansible: 2.2.0.0

When I run ansible-playbook -i production build.yml, this exception was happened

TASK [DavidWittman.redis : set redis to start at boot] *************************
fatal: [app]: FAILED! => {"changed": false, "failed": true, "msg": "Could not find the requested service \"'redis_6379'\": "}

then I run

$ sudo systemctl enable redis_6379
redis_6379.service is not a native service, redirecting to systemd-sysv-install
Executing /lib/systemd/systemd-sysv-install enable redis_6379

And Rerun ansible-playbook -i production build.yml, that exception was not happened.

Maybe it is an ansible bug ?

Ensure log file exists

Hi there,

If I want Redis to log to a file rather than stdout it feels like the playbook should handle the log file existing and being writing by the {{ redis_user }}. How about something like this in tasks/server.yml (and equivalent in tasks/sentinel.yml):

- name: ensure that log file exists and is writable by redis
  file: path={{ redis_logfile }}
        owner={{ redis_user }}
        group={{ redis_user }}
        mode=0600
        state=touch
  when: redis_logfile|length > 0

What d'you reckon? πŸ˜„

Unexpected setting owner of log_dir/* if logfile is given

Hi,

I've specified log file as redis_logfile: /var/log/redis.log and it caused all the remaining log files in /var/log have changed owner to redis, e.g:

-rw-r-----  1 redis    adm             0 lis 27 06:38 syslog

which essentially caused other non–root services either to crash or unable to log .

install testing version of redis

Hello there,
this is wonderful but I have noticed if I put as version the testing one which is 3.2 that is not in the standard release url it will not fetch it, so this version will be fine:
http://download.redis.io/releases/redis-3.0.7.tar.gz
But if I type 3.2.0-rc3 which is in the archive the path will be:
https://github.com/antirez/redis/archive/3.2.0-rc3.tar.gz
Could you do a patch to allow also to fetch newer versions from the archive which is here?
Let me know!

Avoid start/restart of Redis services

With the addition of the task to ensure Redis is started (in 54d67ab), there's now a scenario where the Redis services could potentially be started and then immediately restarted:

  1. Run playbook
  2. Shut down Redis services
  3. Change Redis configs in any way
  4. Run playbook again

When this happens, Redis is started at the end of the server.yml tasks, and then restarted later when the restart redis handler is notified due to the config changes.

Per some feedback from halberom on #ansible, it looks like the handlers can be flushed before ensuring the service is started, which should prevent this from happening.

   - meta: flush_handlers

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.