steveazz / cicd-process Goto Github PK

Sections:

• The Goal
• Definitions
• Guidelines
• Performance Testing

##The Goal

"The ability to deliver features quickly to production with high reliability, frequency and quality"

##Definitions

• CICD: Continuous Integration / Continuous Delivery (CICD) is a process for providing new functionality to end users by releasing small low risk features frequently and reliably

• End to End Test: A functional test which interacts with all of the application's dependencies for that scenario

• Midway Test: A functional test which interacts with stubbed endpoints instead of an application's dependencies.

• Smoke Test: End to End test which validates critical functionality (show stoppers)

  Goals:

  • Validate critical functionality in less than 10 minutes
  • 100% consistent
  • Clear descriptive reporting in the event of a failure

• Regression Test : End to End test which validates all pre-existing functionality.

  Goals:

  • Validate functionality in less than 20 minutes
  • 95% consistent
  • Clear descriptive reporting in the event of the failure

• Unit Test: Test which validate a single method of functionality

  Goals:

  • 90% coverage of all code
  • 100% Consistent
  • Clear descriptive reporting for coverage, errors, and maintainability
  • Execute tests in less than 30 seconds

• Web Component: Custom and reusable UI elements in the browser

  Goals:

  • Provide consistent user interaction and look & feel
  • Provide easy to use, well documented, and fully tested components for fast development

• Component Test: Test which validates a web component in isolation and in a browser

  Goals:

  • Separate the testing of functional stories and web components
  • Validate rendering and functionality inside a browser
  • Clear descriptive reporting in the event of a failure
  • 100% all of components tested with 85% coverage of unit tests
  • Execute all tests in less than 30 seconds

• Image Diff: Automated tool which renders a site and creates screenshots

  Goals:

  • Create a report to indicate what has visually changed on your website
  • Enable developer to validate intended visual changes and detect unintended changes before released to production

• Tripwire: A signal which indicates a decision needs to be made before continuing on with the process.

Guidelines

Guidelines are general principles a team should consider as part of their process. Guidelines are not hard rules and can be overruled. Teams should consciously overrule a guideline and determine their reasons for doing so.

• master is Prod Ready

  • The master branch should always be ready for production
  • If a show stopper defect enters master, roll forward with a fix or revert the commit.

• Releases are small changes

  • Releases should be isolated to small feature changes
  • Reduces risk for a single release
  • Releases become routine
  • Allows team to respond quickly to feedback and defects

• First In First Out:

  • Features are released in the order they enter the process
  • If feature blocks the process, then it should be removed and start the process again
  • Reduces merge conflicts
  • Clarifies priority of feature testing and delivery

• Automate:

  • Automate any tasks that are routine, mundane or does not require human interaction
  • Enables team to focus on more value-add work and decisions
  • Ensures repeatability of tasks (ie: deployments)

• Keep WIP down:

  • Work in progress should be reduced to keep team members focused and prevent task switching
  • WIP should be reduced to one feature per workflow step

• Perfect is the enemy of Good Enough

  • Team should release features even if they have 'minor' defect
  • Team decides what is a 'minor' defect
  • Iterate over features to improve reliabiliy and address defects

• Iterate on Process:

  • Team should identify bottle necks, problematic tasks or other areas of concernin their process
  • Team should experiment with process improvements
  • Team should drop any process or steps which do not add value

• Clarity & Visibility:

  • Team should provide an easily accessible report which provides the current state of the project
  • Team should provide an easily accessible report which lists the history of releases and their modifications

• Release as often as possible

Performance Testing

Performance testing is a vital part of maintaining a production application. In order to ensure performance either improves or remains constant over time, the following features are suggested.

• Create Thresholds
  • Team should create thresholds for the application's performance.
  • Thresholds should be validated via reporting and monitoring of the application.
• Monitor Production
  • Production should be monitored and metrics should be tracked on each of the production deployments
  • Alerts should be provided to the team when performance of the application falls under a pre-determined threshold
• Reduce logging impact to users
  • Monitoring and reporting should impact users as little as possible
  • Leverage asynchronous or 'fire & forget' (ie: UDP network protocol) techniques for tracking performance
• Measure Perceived Performance
  • Perceived performance is the perfomance of the application as it appears to end users.
  • Track all measurable aspects of your application which impacts users (ie: rendering, visual feedback, etc.)
• Track over time
  • Performance metrics should be recorded and tracked overtime.
  • Team should review regularly how new features or changes impact performance
• Communicate with dependant teams
  • Teams who manage the application's dependencies should receive feedback when latency increases or improves
  • Historical reporting should be provided with the feedback