This main repository for Data.gov's stack deployment onto AWS Infrastructure. The responsitory is broken into the following roles all created/provisioned using Ansible and :

Included in this Repository:

• Software
  • Data.gov (Wordpress)
  • Catalog.data.gov (CKAN 2.3)
  • Inventory.data.gov (CKAN 2.5)
  • Labs.data.gov/CRM (Open311 CRM)
  • Labs.data.gov/Dashboard (Project Open Data Dashboard)
• Security
  • Baseline Hardening
  • GSA IT Security Agents
  • Fluentd (Logging)

Milestone	Status	Target Date
Architecture		8/16/2016
Development Environment		10/1/2016
Staging Environment		10/30/2016
Production Environment		12/15/2016
System Security Plan	&	12/8/2016 (90 Day) & 12/8/2016 (1 Year ATO)
Authority to Operate kick-off meeting		12/8/2016
Scanning and Penetration Testing		12/1/2016
Remediation of scanning/pen test findings		12/5/2016
Authority to Operate Issued 90-day		12/8/2016
Infrastructure Switch Over		12/29/2016
Start of 1 year Authority to Operate		1/2/2017

• Ansible > 1.10
• SSH access (via keypair) to remote instances
• boto3 (for infrastructure provisioning only): https://github.com/boto/boto3
• ansible-secret.txt: export ANSIBLE_VAULT_PASSWORD_FILE=~/ansible-secret.txt
• run all provisioning/app deployment commands from repo's ansible folder
• for wordpress/dashboard/crm/monitoring/jekyll run the following command within the role's root folder before you provision anything: ansible-galaxy install -r requirements.yml
• {{ inventory }} can be:
  • inventories/staging/hosts
  • inventories/production/hosts
  • inventories/local/hosts

Moved to datagov-infrastructure

cd /catalog-deploy/ansible and us -i "inventory/../hosts" flag to run playbooks w/ ansible-playbook --help or as ansible all -a "cmd" to run a one-off command on all hosts (only suggested for -m ping for query/stats/services all installation and configuration is done using playbooks)

provision vm & deploy app: ansible-playbook datagov-web.yml -i {{ inventory }} --tags="provision" --limit wordpress-web

deploy app: ansible-playbook datagov-web.yml -i {{ inventory }} --tags="deploy"

deploy rollback: ansible-playbook datagov-web.yml -i {{ inventory }} --tags="deploy-rollback"

provision vm & deploy app: ansible-playbook dashboard-web.yml -i {{ inventory }} --tags="provision" --limit dashboard-web

deploy app: ansible-playbook dashboard-web.yml -i {{ inventory }} --tags="deploy"

deploy rollback: ansible-playbook dashboard-web.yml -i {{ inventory }} --tags="deploy-rollback"

provision vm & deploy app: ansible-playbook crm-web.yml -i {{ inventory }} --tags="provision" --limit crm-web

deploy app: ansible-playbook crm-web.yml -i {{ inventory }} --tags="deploy"

deploy rollback: ansible-playbook crm-web.yml -i {{ inventory }} --tags="deploy-rollback"

provision vm - web: ansible-playbook catalog.yml -i {{ inventory }} --tags="frontend,ec2" --skip-tags="solr,db,cron" --limit catalog-web

provision vm - harvester: ansible-playbook catalog.yml -i {{ inventory }} --tags="harvester,ec2" --skip-tags="apache,solr,db,saml2,redis" --limit catalog-harvester

provision vm - solr: ansible-playbook catalog.yml -i {{ inventory }} --tags="solr,secops,trendmicro,misc" --limit solr

provision vm && deploy app: ansible-playbook inventory.yml -i {{ inventory }} --skip-tags="solr,db,deploy-rollback" --limit inventory-web

provision vm - solr: ansible-playbook inventory.yml -i {{ inventory }} --tags="solr,secops,trendmicro,misc" --limit solr

provision vm && deploy app: ansible-playbook jekyll.yml -i {{ inventory }} --limit jekyll-web

provision vm && deploy app: ansible-playbook elasticsearch.yml -i {{ inventory }}

provision vm && deploy app: ansible-playbook kibana.yml -i {{ inventory }}

provision vm && deploy app: ansible-playbook efk_nginx.yml -i {{ inventory }}

install the trendmicro agent: ansible-playbook trendmicro.yml -i {{ inventory }}

Add SecOps user: ansible-playbook secops.yml -i {{ inventory }}

ansible all -m shell -a "apt-get update && apt-get dist-upgrade" --sudo

ansible all -m shell -a "service tomcat6 restart" --sudo

ansible all -m shell -a "service ntp restart" --sudo

ansible all -m shell -a "/usr/bin/killall dhclient && dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0" --sudo

dpkg errors:

sed -i '/postdrop/d' /var/lib/dpkg/statoverride

sed -i '/ssl-cert/d' /var/lib/dpkg/statoverride

ntpd issues: apt-get remove ntp && apt-get purge ntp && apt-get autoclean && apt-get autoremove

Unable to resolve host IP: echo 127.0.0.1 $(hostname) >> /etc/hosts