This main repository for Data.gov's stack deployment onto AWS Infrastructure. The responsitory is broken into the following roles all created/provisioned using Ansible and :
Included in this Repository:
- Software
- Data.gov (Wordpress)
- Catalog.data.gov (CKAN 2.3)
- Inventory.data.gov (CKAN 2.5)
- Labs.data.gov/CRM (Open311 CRM)
- Labs.data.gov/Dashboard (Project Open Data Dashboard)
- Security
- Baseline Hardening
- GSA IT Security Agents
- Fluentd (Logging)
- Ansible > 1.10
- SSH access (via keypair) to remote instances
- boto3 (for infrastructure provisioning only): https://github.com/boto/boto3
- ansible-secret.txt:
export ANSIBLE_VAULT_PASSWORD_FILE=~/ansible-secret.txt
- run all provisioning/app deployment commands from repo's
ansible
folder - for wordpress/dashboard/crm/monitoring/jekyll run the following command within the role's root folder before you provision anything:
ansible-galaxy install -r requirements.yml
- {{ inventory }} can be:
- inventories/staging/hosts
- inventories/production/hosts
- inventories/local/hosts
Moved to datagov-infrastructure
cd /catalog-deploy/ansible and us -i "inventory/../hosts" flag to run playbooks w/ ansible-playbook --help
or as ansible all -a "cmd"
to run a one-off command on all hosts (only suggested for -m ping
for query/stats/services all installation and configuration is done using playbooks)
provision vm & deploy app: ansible-playbook datagov-web.yml -i {{ inventory }} --tags="provision" --limit wordpress-web
deploy app: ansible-playbook datagov-web.yml -i {{ inventory }} --tags="deploy"
deploy rollback: ansible-playbook datagov-web.yml -i {{ inventory }} --tags="deploy-rollback"
provision vm & deploy app: ansible-playbook dashboard-web.yml -i {{ inventory }} --tags="provision" --limit dashboard-web
deploy app: ansible-playbook dashboard-web.yml -i {{ inventory }} --tags="deploy"
deploy rollback: ansible-playbook dashboard-web.yml -i {{ inventory }} --tags="deploy-rollback"
provision vm & deploy app: ansible-playbook crm-web.yml -i {{ inventory }} --tags="provision" --limit crm-web
deploy app: ansible-playbook crm-web.yml -i {{ inventory }} --tags="deploy"
deploy rollback: ansible-playbook crm-web.yml -i {{ inventory }} --tags="deploy-rollback"
provision vm - web: ansible-playbook catalog.yml -i {{ inventory }} --tags="frontend,ec2" --skip-tags="solr,db,cron" --limit catalog-web
provision vm - harvester: ansible-playbook catalog.yml -i {{ inventory }} --tags="harvester,ec2" --skip-tags="apache,solr,db,saml2,redis" --limit catalog-harvester
provision vm - solr: ansible-playbook catalog.yml -i {{ inventory }} --tags="solr,secops,trendmicro,misc" --limit solr
provision vm && deploy app: ansible-playbook inventory.yml -i {{ inventory }} --skip-tags="solr,db,deploy-rollback" --limit inventory-web
provision vm - solr: ansible-playbook inventory.yml -i {{ inventory }} --tags="solr,secops,trendmicro,misc" --limit solr
provision vm && deploy app: ansible-playbook jekyll.yml -i {{ inventory }} --limit jekyll-web
provision vm && deploy app: ansible-playbook elasticsearch.yml -i {{ inventory }}
provision vm && deploy app: ansible-playbook kibana.yml -i {{ inventory }}
provision vm && deploy app: ansible-playbook efk_nginx.yml -i {{ inventory }}
install the trendmicro agent: ansible-playbook trendmicro.yml -i {{ inventory }}
Add SecOps user: ansible-playbook secops.yml -i {{ inventory }}
ansible all -m shell -a "apt-get update && apt-get dist-upgrade" --sudo
ansible all -m shell -a "service tomcat6 restart" --sudo
ansible all -m shell -a "service ntp restart" --sudo
ansible all -m shell -a "/usr/bin/killall dhclient && dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0" --sudo
dpkg errors:
sed -i '/postdrop/d' /var/lib/dpkg/statoverride
sed -i '/ssl-cert/d' /var/lib/dpkg/statoverride
ntpd issues: apt-get remove ntp && apt-get purge ntp && apt-get autoclean && apt-get autoremove
Unable to resolve host IP: echo 127.0.0.1 $(hostname) >> /etc/hosts