Giter Club home page Giter Club logo

sre-salt-prime's Introduction

sre-salt-prime

Site Reliability Engineering / DevOps SaltStack configuration files

Code of Conduct

CODE_OF_CONDUCT.md:

The Creative Commons team is committed to fostering a welcoming community. This project and all other Creative Commons open source projects are governed by our Code of Conduct. Please report unacceptable behavior to [email protected] per our reporting guidelines.

Contributing

See CONTRIBUTING.md.

Development Notes

  • Avoid insecure repository clones: This repository includes encrypted secrets. Do not run git-crypt unlock on clones that are not otherwise secured (ex. strong login password, disk encryption).
  • Avoid editing the base environment: The base environment is configured to prevent commit and push actions. Please use your development environment and pull the changes to base.
  • Sign your commits:
    • The commit signing option has been adjusted in the repository to facilitate smoother collaboration. This modification is intended to ease the contribution process. However, it is strongly encourged that staff members continue to adhere to best practices by using GPG for all signed commits, ensuring the security and integrity of the project.

    • Ensure you are using RemoteForward in your SSH configuration to forward your GnuPG agent to salt-prime (see the example configuration, under Setup, below).

    • Ensure you have configured your newly cloned repository to sign commits (see the git config command, under Setup, below).

Setup

  • SSH connection information: example local/laptop ~/.ssh/config configugration:
    Host bastion-us-east-2
        HostName bastion-us-east-2.creativecommons.org
        User ARTHUR
    
    Host salt-prime
        HostName 10.22.11.11
        ProxyJump bastion-us-east-2
        RemoteForward /run/user/4242/gnupg/S.gpg-agent /Users/ARTHUR/.gnupg/S.gpg-agent.extra
        User ARTHUR
    
    Host *
        ServerAliveCountMax 60
        ServerAliveInterval 30
        TCPKeepAlive no
    
    • Assumes remote username ARTHUR and remote uid 4242. Replace these values in your own local/laptop configuration.
    • ProxyJump allows you to ssh salt-prime from your local/laptop.
    • RemoteForward allows you to sign your commits.
  • Setup your development repository on salt-prime:
    1. Clone repository to /srv with your username. For example:
      cd /srv
      git clone [email protected]:creativecommons/sre-salt-prime.git ${USER}
    2. Setup your newly cloned repository.
      1. Configure commit signing:
        cd /srv/${USER}
        git config user.email YOUR_EMAIL
        git config user.signingkey YOUR_GPG_ID
        git config commit.gpgsign true
      2. Unlock encrypted secrets:
        cd /srv/${USER}
        git-crypt unlock
    3. Specify the environment when you test changes. For example:
      sudo salt \* state.highstate saltenv=${USER} test=True
      • use --state-verbose=True to see successes
      • use --state-output=full_id to see full detail of successes
      • use --log-level=debug --log-file-level=warning to see debug messages (without logging those debug messages, which may contain secrets, to the log file)

Goals

  • Use AWS well, but avoid technologies that create AWS lock-in (ex. Confidant)
  • Salt Prime must not contain any exclusive data (use Git)
  • Git repository must not contain any unencrypted secrets
  • Git repository commits must be signed and applied to the main branch via Pull Requests
  • A compromised minion must not be able to escalate access
    • SysAdmins must not forward their SSH agent
    • Must not reuse application passwords (ex. Prod and Dev databases must have different passwords)
    • Pillar data must be restricted by Minion ID based classification
      • The only grain which can be safely used is grains['id'] which contains the Minion ID. (FAQ Q.21)

Decisions

  • Amazon Web Services (AWS)
    • Creative Commons is already using it and staff are familiar with it
    • Features allow security (ex. screened subnets, security groups policies)
    • Features allows Infrastructure as Code
    • us-east-2
      • cost effective
      • avoid conflict/collision over region limited resources (ex. ElasticIPs)
  • Debian 11 (Bullseye), Debian 10 (Buster), and Debian 9 (Stretch)
    • Free/Open Source
    • Debian Stable
    • Creative Commons is already using it and staff are familiar with it
  • git-crypt - transparent file encryption in git
    • Free/Open Source
    • Performance: files are decrypted in the checked out repository
    • Security: automatic encryption and directory based filters minimize the chance of unencrypted secrets being pushed to GitHub
  • SaltStack
    • Free/Open Source
    • Performance
    • Creative Commons is already using it and staff are familiar with it
    • Version: 3006.7

Host Classification

Minions are added and configured from salt-prime with the following Minion ID schema: HST__POD__LOC (host/role__pod/group__location). These variables are used to determine the state and pillar data.

Show top states example command:

sudo salt \* pillar.item states saltenv=${USER}

See docs/Host_Classification.md for details.

Orchestration

See docs/Orchestration.md.

References

SaltStack

Best Practices

Frequently Referenced Documentation

Repository Documentation

Formula Repositories

License

sre-salt-prime's People

Contributors

aldenstpage avatar amandayclee avatar brylie avatar dependabot[bot] avatar kgodey avatar nox1134 avatar shafiya-heena avatar timidrobot avatar zackkrida avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sre-salt-prime's Issues

Enable use of repository for local WordPress development

Enable use of repository for local WordPress development

Add orchestration for docker or virtual machine WordPress development pod to maximize parity between production, staging, and development environments.

Known Hurdles

  • SaltStack errors on git-crypt locked pillar files

Deploy dev/staging environment for cc-licenses

Deploy initial dev/staging environment for creativecommons/cc-licenses develop branch:

  • Set up a web host EC2 instance
  • Set up a small PostgreSQL 10+ RDS instance
    • Use Security Group to limit connections to only those that originate from web host
  • Configure DNS (cclicdev.creativecommons.org)
  • Configure web host with Let's Encrypt and NGINX
    • use HTTP Basic Auth to prevent anonymous use
  • Configure web host with Gunicorn and Django using Caktus' provisioning instructions
  • Allow emails to caktusgroup.com
    • (normally staging servers only allow emails to creativecommons.org and discard email to any other domain)
  • Verify basic functionality: https://cclicdev.creativecommons.org/
  • Configure Caktus access and permissions on web host
    • add Caktus users as admins
      • include Caktus SSH keys
    • strip passwords from all admin users (!)
    • support direct connections to SSH (instead of requiring a bastion/jump-host)
    • sudo NOPASSWD
  • Distribute connection information to Caktus

This is meant to give Caktus a venue for sharing their progress with us and performing usability testing; configuration automation can be dealt with later.

Update OS of licbuttons__prod__us-east-2 by upgrade

Description

Update OS of licbuttons__prod__us-east-2 to Debian 11 Bullseye by upgrade

This process involves less downtime than the replacement method (ex. #115). It also maintains configuration data (ex. Let's Encrypt certificates) that can cause order of operation issues. However it requires more manual action and unidentified cruft/detritus may cause future issues.

Process

All shell commands should be run on the host (licbuttons__prod__us-east-2), unless otherwise stated:

  1. Verify operating system
    lsb_release -a
  2. Pause uptime monitoring
  3. Disable and update SaltStack apt source
    sudo sed -e's|^|#|' -e's|/debian/10/|/debian/11/|' -e's|buster|bullseye|' -i /etc/apt/sources.list.d/saltstack.list
  4. Remove SaltStack packages
    sudo DEBIAN_FRONTEND=noninteractive apt-get remove salt-common salt-minion -yq
  5. Remove Debian packages that were automatically installed and are no longer required
    sudo DEBIAN_FRONTEND=noninteractive apt autoremove -yq
  6. Ensure existing OS (Debian 10 Buster) is up-to-date
    1. Update Apt/Dpkg (and only show errors):
      sudo apt-get update >/dev/null
    2. Download Debian package updates and review changes:
      sudo apt-get dist-upgrade -yqd | sed -e"/Get:[0-9]/d"
    3. Install Debian package updates:
      sudo DEBIAN_FRONTEND=noninteractive apt-get dist-upgrade -yq
    4. Determine if a reboot is required:
      test -f /run/reboot-required && echo REBOOT REQUIRED
    5. (optionally, if a reboot is required) reboot immediately:
      sudo reboot
    6. (optionally) Clean-up Debian packages:
      sudo DEBIAN_FRONTEND=noninteractive apt-get autoremove -yq
      sudo DEBIAN_FRONTEND=noninteractive apt-get autoclean -yq
  7. Clean up leftover configuration files
    sudo find /etc \( -name '*.dpkg-*' -o -name '*.ucf-*' -o -name '*.merge-error' \) -exec rm -v {} +
  8. Check Debian package status
    sudo dpkg --audit
  9. Reconfigure APT source-list file
    sudo sed -e's|buster|bullseye|g' -e's|debian-security bullseye/updates|debian-security bullseye-security|' -i /etc/apt/sources.list
  10. Upgrade OS
    1. Update Apt/Dpkg (and only show errors):
      sudo apt-get update >/dev/null
    2. Perform full-upgrade:
      sudo apt-get full-upgrade
      • Postfix Configuration: No Configuration (leave the current configuration unchanged)
      • Configuring libc6:amd64: Restart services during package upgrades without asking: Yes
      • Configuration file /etc/nginx/nginx.conf: N (keep your currently-installed version)
      • Modified configuration file /usr/share/chrony/chrony.conf: keep the local version currently installed
      • Configuring openssh-server /etc/ssh/sshd_config: keep the local version currently installed
    3. Reboot immediately:
      sudo reboot
  11. Verify operating system
    lsb_release -a
  12. Enable SaltStack apt source and reinstall minion
    sudo sed -e's|^#||' -i /etc/apt/sources.list.d/saltstack.list
    sudo apt-get update >/dev/null
    sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq salt-minion
  13. On salt-prime: Verify minion is connected to master
    sudo salt licbuttons\* test.ping
  14. Clean-up Debian packages
    1. Purge removed packages
      sudo DEBIAN_FRONTEND=noninteractive apt-get purge -q $(dpkg -l | awk '/^rc/ { print $2 }')
    2. Clean-up Debian packages:
      sudo DEBIAN_FRONTEND=noninteractive apt-get autoremove -yq
      sudo DEBIAN_FRONTEND=noninteractive apt-get autoclean -yq
  15. Remove old certbot virtual environment
    sudo rm -rf /opt/certbot_virtualenv/
  16. On salt-prime: Run highstate for minion
    sudo salt licbuttons\* state.highstate saltenv=${USER} test=True
    sudo salt licbuttons\* state.highstate saltenv=${USER}
  17. Verify services
  18. Unpause uptime monitoring

Additional context

[Bug] Index Apache2 config CC Legal Tools section is missing the `language-redirects` include

Description

Index Apache2 config (states/apache2/files/index.conf) CC Legal Tools section is missing the language-redirects include:

###########################################################################
# CC Legal Tools
# Directory Aliases
Alias /status /var/www/git/cc-legal-tools-data/docs/status
Alias /rdf /var/www/git/cc-legal-tools-data/docs/rdf
Alias /publicdomain /var/www/git/cc-legal-tools-data/docs/publicdomain
Alias /licenses /var/www/git/cc-legal-tools-data/docs/licenses
Alias /cc-legal-tools /var/www/git/cc-legal-tools-data/docs/cc-legal-tools
# File Aliases
Alias /schema.rdf /var/www/git/cc-legal-tools-data/docs/rdf/schema.rdf
Alias /ns.html /var/www/git/cc-legal-tools-data/docs/rdf/ns.html
Alias /ns /var/www/git/cc-legal-tools-data/docs/rdf/ns.html
<Directory /var/www/git/cc-legal-tools-data/docs>
# Disable .htaccess (for security and performance)
AllowOverride None
# Also serve HTML files without .html extension
RewriteCond %{REQUEST_FILENAME}.html -f
RewriteRule !.*\.html$ %{REQUEST_FILENAME}.html [L]
# Redirect .../index.php to .../
RewriteCond %{REQUEST_FILENAME} "index\.php$" [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule (.*/)index\.php$ $1 [L,NC,R=301]
# Deny access to PHP files (content should be only static files)
RewriteRule .*\.php$ "-" [F,L]
# Correct mimetype for .../rdf files
RewriteRule (.*/rdf$) $1 [T=application/rdf+xml]
</Directory>

language-redirects include: creativecommons/cc-legal-tools-data: config/language-redirects

Reproduction

Go to a legacy URL (ex. https://creativecommons.org/licenses/by/4.0/legalcode.zh)

Expectation

For example, /licenses/by/4.0/legalcode.zh should redirect to /licenses/by/4.0/legalcode.zh-hans.

Resolution

  • I would be interested in resolving this bug.

Amazon EBS and NVMe on Linux instances requires manual configuration

Describe the bug

The states/pillars assume specific NVMe block devices instead of querying AWS. I thought this would be a rare, but I was wrong.

To Reproduce

Create an instance, guess at NVMe block device in pillar configuration.

Expected behavior

Only minimal manual configuration should be necessary.

Additional context

[Feature] index: allow CORS (cross-origin resource sharing) for CC Legal Tools

Problem

[#78119] Machine readable licenses are not machine readable : Creative Commons Freshdesk ticket contains:

From JavaScript, I cannot execute an XMLHttpRequest for either the HTML URL (needed to find HTTP link element) or the RDF URL. The request is denied as a cross-origin request.

I don't know, but to make the license available to machines that want to check the license details, then both the license HTML and the RDF should be served with the HTTP response header:

Access-Control-Allow-Origin: *

Cross-origin resource sharing - Wikipedia:

A wildcard same-origin policy is appropriate when a page or API response is considered completely public content and it is intended to be accessible to everyone, including any code on any site.

Description

  1. Enable headers apache2 module
  2. Set header in states/apache2/files/index.conf:
    index 9088170..d2871c3 100644
    --- states/apache2/files/index.conf
    +++ states/apache2/files/index.conf
    @@ -30,28 +30,30 @@
         <Directory /var/www/git/cc-legal-tools-data/docs>
             # Disable .htaccess (for security and performance)
             AllowOverride None
             # Also serve HTML files without .html extension
             RewriteCond %{REQUEST_FILENAME}.html -f
             RewriteRule !.*\.html$ %{REQUEST_FILENAME}.html [L]
             # Redirect .../index.php to .../
             RewriteCond %{REQUEST_FILENAME} "index\.php$" [NC]
             RewriteCond %{REQUEST_FILENAME} !-f
             RewriteRule (.*/)index\.php$ $1 [L,NC,R=301]
             # Deny access to PHP files (content should be only static files)
             RewriteRule .*\.php$ "-" [F,L]
             # Correct mimetype for .../rdf files
             RewriteRule (.*/rdf$) $1 [T=application/rdf+xml]
    +        # Enable CORS (cross-origin resource sharing)
    +        Header set Access-Control-Allow-Origin "*"
         </Directory>
         Include /var/www/git/cc-legal-tools-data/config/language-redirects
         RedirectPermanent  /licenses/mark/1.0  /publicdomain/mark/1.0
         RedirectPermanent  /licences           /licenses
     
         ###########################################################################
         # Chooser
     #    Alias /choose /var/www/git/chooser/docs
     #    <Directory /var/www/git/chooser/docs>
     #        # Disable .htaccess (for security and performance)
     #        AllowOverride None
     #        # Redirect .../index.php to .../
     #        RewriteCond %{REQUEST_FILENAME} "index\.php$" [NC]
     #        RewriteCond %{REQUEST_FILENAME} !-f

Alternatives

🤷🏻

Additional context

Implementation

  • I would be interested in implementing this feature.

Postfix should use 465 with TLS

Bug

Description

Currently, all servers within the salt-prime infrastructure are configured to utilize port 587 for the postfix service. Going forward, it is imperative to address this configuration by transitioning to the utilization of port 465.

Expectation

  • Email transport should be encrypted

Update SaltStack to 3000.3+ds-1

Problem Description

A new version of SaltStack was released.

Solution Description

Update SaltStack to 3000.3+ds-1 (or current/latest):

Additional context

Add support for multiple instances in a role

Add support for multiple instances in a role

The current Hosts Classification scheme assumes a single instance for each role.

Consider:

  • Expanding host classification to included INC (increment)
  • Assuming all multiple instances in a role will be part of an Amazon EC2 Auto Scaling group using AMIs without a Salt Minion

Update dispatch host to support long term logging

Problem Description

Update NGINX configuration for the dispatch host to support long term logging. The goal of the log format is to allow simple log term log analysis without compromising users privacy or increasing Creative Commons liability. The log format should:

  1. ISO date format
  2. Anonymized real client IP (not CDN IP)
  3. Truncate or exclude User Agent

The solution should include:

  1. NGINX log configuration
  2. logrotate configuration
  3. EBS mount configuration

Additional context

SaltStack Host Classification needs improvement

Host Classification needs improvement

Overview

This builds on Host Classification - README.md.

Like Apache2, SaltStack pillar data uses a last declared wins model. This repo uses (from least specific to most specific):

  1. LOC: location
  2. POD: pod (a synonym of group that is helpfully three characters long)
  3. HST: host

The current systems is ok. I like that it driven by the salt primary. I think it could be better articulated so that pillars fully use increasing specificity.

It should be easy, for instance to configure a production cluster and a staging cluster by using different values for POD.

I think the first step will be to decouple networking from POD so that each POD can declare which network it uses.

I also suspect that the way I've done lookups in states/orch/aws/jinja2.sls is too inflexible/fragile.

Actionables

  • Create helper script to easily illustrate what files are loaded for a given minion id or HST/POD/LOC combination.
  • Use addition of WordPress staging environment as an engine to drive refactoring/improvements
  • Maybe also add 2nd bastion as an engine to drive refactoring/improvements

Create Licenses staging host

Description

During active development, the staging host will be active (with Django app). Once production is deployed, I expect we'll remove Django and PostgreSQL.

Additional context

[Bug] Remove summit__prod__us-east-2 and related resources

Description

Remove summit__prod__us-east-2 and related resources (these do not include production services)

Removal Rationale

  • Resources were created in preparation for WordPress development that never happened
  • Future development will happen on Pantheon's managed WordPress hosting.
  • Vendor cost savings (averages $93.85USD/month per AWS Cost Management: Cost Explorer)
  • Maintenance cost savings

Update wp-plugin-creativecommons-website

Problem

We released the version 0.2.0 of wp-plugin-creativecommons-website.

Description

Since composer.json is included in the new release, we can update the ccorg-compser.json to include the new version.

Update OS of openglam__prod__us-east-2 by replacement

Description

Update OS of openglam__prod__us-east-2 to Debian 11 Bullseye by replacement (data preserved in EBS volume)

This process is simpler than the upgrade method (ex. #186). However it may cause order of operation issues (ex. due to missing Let's Encrypt certificates).

Process

All shell commands should be run on the host (openglam__prod__us-east-2), unless otherwise stated:

  1. Ensure AMI is latest
  2. Verify operating system
    lsb_release -a
  3. rename EC2 instance so that name ends in _TERMINATED
  4. terminate EC2 instance
  5. On salt-prime: run SaltStack orchestration for host (recreates EC2 instance)
  6. On salt-prime: run SaltStack highstate for host (configures EC2 instance)
    1. Run highstate as a test to preview changes:
      sudo salt openglam\* state.highstate saltenv=${USER} test=True
    2. Run highstate:
      sudo salt openglam\* state.highstate saltenv=${USER}
  7. Perform OS updates and reboot host
    1. Update Apt/Dpkg (and only show errors):
      sudo apt-get update >/dev/null
    2. Download updates and review changes:
      sudo apt-get dist-upgrade -yqd | sed -e"/Get:[0-9]/d"
    3. Install updates:
      sudo DEBIAN_FRONTEND=noninteractive apt-get dist-upgrade -yq
    4. Determine if a reboot is required:
      test -f /run/reboot-required && echo REBOOT REQUIRED
    5. (optionally, if a reboot is required) reboot immediately:
      sudo reboot
    6. (optionally) Clean-up packages:
      sudo apt-get autoremove -y && sudo apt-get autoclean
  8. Verify operating system
    lsb_release -a
  9. Verify services

Configure and deploy new ccstatic.org host

Problem Description

The existing ccstatic.org host has the following issues:

  • hosting provider support for infrastructure as code is inadquate
  • OS is not current
  • less than optimal security settings
  • less than optimal web hosting (Apache2 does too much slowing work by default)
  • log configuration does not support long term retention
    • (long term retention must have strong privacy protections)

Solution Description

Configure and deploy new ccstatic.org host that resolves the issues listed above.

license pages do not respond to Accept headers (ex. for requesting RDF)

Expected Behavior

Sending an Accept header with the desired content-type (e.g., RDF) should result in CC returning the RDF for a license, rather than the HTML:

curl -iL -H 'Accept: application/rdf+xml' https://creativecommons.org/licenses/by-nc/4.0/

Actual Behavior

The server ignores the Accept header and returns HTML.

This is a separate issue from creativecommons/creativecommons.org#542; in that case it was a separate URL that was down.

@robmyers @little-wow

[Feature] Replace stage.creativecommons.org with new hosting model

Description

Replace stage.creativecommons.org with new hosting model based on creativecommons/project_creativecommons.org

Improved Data Path

The new hosting model greatly simplifies the hosting environment and data path compared to the current legacy site. It features a roughly 85% decrease in complexity (from 7 dedicated services to 1 dedicated service).

Proposed Data Path

  1. Site host Apache2
    • CC Legal Tools content
    • Chooser content
    • Misc content (may be removed and included in WordPress instead)
    • WordPress content (default data path)

Legacy Data Path

  1. Dispatch host NGINX
  2. ccEngine host Apache2 (ccEngine content)
  3. Misc host NGINX (Misc content)
  4. Loadbalancer host HAProxy (WordPress/default content)
  5. WordPress host Varnish (WordPress/default content)
  6. WordPress host Apache2 (WordPress/default content)
  7. WordPress host PHP+FPM (WordPress/default content)

Example of issue caused by legacy data path: https://github.com/creativecommons/tech-support/issues/838 (private repo)

Additional context

[Bug] Remove sotc__prod__us-east-2 and related resources

Description

Remove deprecated/unused hosts and any configurations that are no longer needed:

  • sotc__prod__us-east-2
    • averages $81.10/month per AWS Cost Management: Cost Explorer

Rationale

  • Legacy SotC reports moved to GitHub pages: creativecommons/stateof
  • Recent SotC reports have been PDFs (not websites)
  • 2022-06-02: Removal approved in #comms channel on Internal Slack

Discard non-Creative-Commons email on staging servers

Problem Description

When staging WordPress servers import production data, it increases the chance that operational emails (ex. password resets) are sent to end-users from the staging server.

Solution Description

Add Postfix configuration to discard non-Creative-Commons email on staging servers so that they are less likely to cause problems or confusion.

Alternatives

  • Anonymize data

[Bug] Remove deprecated/unused hosts 2022Q1

Description

Remove deprecated/unused hosts and any configurations that are no longer needed:

  • ccorgwp__stagelegacy__us-east-2
  • cclicdev__stage__us-east-2
  • dispatch__stagelegacy__us-east-2

These hosts are no longer used and have been offline. Fully removing them and their configurations will make future development and management easier.

Additional Context

Remove `[www.]creativecommons.dk` web forwarding

Description

The domain creativecommons.dk and www.creativecommons.dk, currently hosted on puntum.dk, is no longer managed by Creative Commons. To streamline operations and avoid unnecessary alerts, we are removing it from creativecommons/sre-salt-prime repository. If we manage it in the future, we will use our domain provider’s web forwarding

Future Insights:

Post-approval for the domain transfer, proceed with the transfer following the Gandi domain transfer steps. This will include setting up necessary redirects and configuring pagerduty alerts to reflect the changes.

Recommended Steps:

On redirects__core_us-east-2 server

  • Disable Nginx Site Configuration
    sudo rm /etc/nginx/sites-enabled/creativecommons.dk
    sudo rm /etc/nginx/sites-enabled/www.creativecommons.dk
  • Remove Nginx Configuration Files
    sudo rm /etc/nginx/sites-available/creativecommons.dk
    sudo rm /etc/nginx/sites-available/www.creativecommons.dk
  • Update Let's Encrypt Configuration
    sudo certbot delete --cert-name creativecommons.dk
  • Update Domain Sets Configuration
    sudo nano /etc/letsencrypt/domainsets.yaml
  • Reload Nginx Configuration
    sudo systemctl reload nginx

On salt-prime server

  • Remove the domain entries from pillars/5_HST__POD/redirects__core.sls.
  • Execute highstate to apply these changes.
  • Verify that the domain has been removed from the domainset.yaml on redirects__core__us-east-2 servers.

Related Links:

Update to support Debian Buster

Task

Update repository to support both Debian 9 "stretch" and Debian 10 "buster" to allow for a gradual transition to the new stable Debian release

Debian Buster Released

Debian 10 "buster" includes numerous updated software packages (over 62%
of all packages in the previous release), such as:

  • Apache 2.4.38
  • BIND DNS Server 9.11
  • Chromium 73.0
  • Emacs 26.1
  • Firefox 60.7 (in the firefox-esr package)
  • GIMP 2.10.8
  • GNU Compiler Collection 7.4 and 8.3
  • GnuPG 2.2
  • Golang 1.11
  • Inkscape 0.92.4
  • LibreOffice 6.1
  • Linux 4.19 series
  • MariaDB 10.3
  • OpenJDK 11
  • Perl 5.28
  • PHP 7.3
  • PostgreSQL 11
  • Python 3 3.7.2
  • Ruby 2.5.1
  • Rustc 1.34
  • Samba 4.9
  • systemd 241
  • Thunderbird 60.7.2
  • Vim 8.1
  • more than 59,000 other ready-to-use software packages, built from
    nearly 29,000 source packages.

[Bug] Remove ccorg_stage_us-east-2 and related provisioned services

Description

Remove ccorg_stage_us-east-2 and related provisioned services (leave SaltStack configuration--state and pillar information--intact)

(approved by @xolotl and @possumbilities in internal Slack)

Removal Rationale

  • Creation of a staging environment was premature
  • Vendor cost savings
  • Maintenance cost savings

Default Debian Apache2 configuration assumes too many resources

Describe the bug

Default Debian Apache2 configuration assumes too many resources.

For example, MaxRequestWorkers defaults to 150, which assumes either very small processes (ex. static files) or larger EC2 instances with more memory.

Expected behavior

SaltStack should manage files and configure resources based on EC2 instance type with a more conservative default.

Additional context

You can, and should, control the MaxRequestWorkers setting so that your server does not spawn so many children that it starts swapping.

(Hardware and Operating System Issues - Apache Performance Tuning - Apache HTTP Server Version 2.4)

Configure and deploy new licensebuttons.net host

Problem Description

The existing licensebuttons.net host has the following issues:

  • hosting provider support for infrastructure as code is inadquate
  • OS is not current
  • less than optimal security settings
  • less than optimal web hosting (Apache2 does too much slowing work by default)
  • log configuration does not support long term retention
    • (long term retention must have strong privacy protections)

Solution Description

Configure and deploy new licensebuttons.net host that resolves the issues listed above.

Additional context

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.