Tests

pm- 11367
Using the new jobRequest and previous api

• Verify if a image is not specified, the default build image is used to run the job
• Verify if a image is specified, that image is used to run the job
• Verify a single non-matrix build works
• Verify a matrix build works
• Verify a build which has includes works
• Verify a failed build's rerun works
• Verify job times out correctly

Without specifying runtime in the yml

• Verify if a image is not specified, the default build image is used to run the job
• Verify if a image is specified, that image is used to run the job
• Verify a single non-matrix build works
• Verify a matrix build works
• Verify a build which has includes works
• Verify a failed build's rerun works
• Verify job times out correctly

 runtime:
     nodePool: x86_u16_pool

For the above yml

• Verify if a image is not specified, the default build image is used to run the job
• Verify if a image is specified, that image is used to run the job
• Verify a single non-matrix build works
• Verify a matrix build works
• Verify a build which has includes works
• Verify a failed build's rerun works
• Verify job times out correctly

 runtime:     
    - nodePool: x86_u16_pool
    - nodePool: arm32_u16_pool

For the above yml

• Verify non-matrix build works. This will result in 2 jobs.
• Verify a matrix build works. It will have 2 * matrix jobs.
• Verify a build which has includes works
• Verify if a image is not specified, the default build image used to run the job is correct
• Verify if a image is specified, that image is used to run the job
• Verify timeouts are set correctly based on the cluster the job runs on.
• If one of the nodePools specified in the yml is a restricted nodePool, make sure restrictions are applied. (Example: In restricted nodePool pre_ci is not allowed. In multiple node pool even if one is restricted and build doesnot in restricted not applied ALL builds will be skipped even nodes which can run will also skipped)
• If one of the nodePools are part of windows or mac, the build should be FAILED
• Verify a failed build's rerun works
• Verify adding a sub nodePool and shared nodePool makes the build on both nodePools
• If no nodePools are specified in the yml, verify the build runs on default nodePool
• If no nodePools are specified in the yml, verify build gets SKIPPED, when there no default nodePool

cases::::

• custom and shippable node pool in same build
• check secure variable

pm - 11408

• Verify the main build section gets inherited correctly when runtime build section is not specified
• Verify runtime's build section overrides the main build section
  • Verify that for a runtime, if pre_ci section is overriden in runtime build section, the runtime build's pre_ci runs
  • Verify that for a runtime, if pre_ci_boot section is overriden in runtime build section, the runtime build's pre_ci_boot values are taken for the runtime
  • Verify that for a runtime, if ci section is overriden in runtime build section, the runtime build's ci runs
  • Verify that for a runtime, if post_ci section is overriden in runtime build section, the runtime build's post_ci runs
  • Verify that for a runtime, if on_success section is overriden in runtime build section, the on_success build's pre_ci runs
  • Verify that for a runtime, if on_failure section is overriden in runtime build section, the runtime build's on_failure runs
  • Verify that for a runtime, if always section is overriden in runtime build section, the runtime build's always runs
  • Verify that for a runtime, if cache section is overriden in runtime build section, the runtime build's cache is honored
  • Verify that for a runtime, if cache_dir_list section is overriden in runtime build section, the runtime build's cache_dir_list is honored
  • Verify that for a runtime, if push section is overriden in runtime build section, the runtime build's push is honored
• Verify build restrictions are honored for restricted nodePools, if they are overriden in the runtime section

pm- 11441

• Verify not specifying a nodePool in the allow_failures section allows failures in all the nodePools the jobs run on
• Verify specifying invalid nodePool in allow_failures section results in a failed build with appropriate message
• Verify specifying a nodePool in the allow_failures section allows failures in the jobs which run on the specified nodePool

pm - 11368

• Verify nodePools are shown in the matrix summary table if multiple nodePools are specified
• Verify nodePools are shown in the matrix summary header if jobs are running on one nodePool
• Verify job consoles page show which nodePool the job ran on

pm- 11440

• Verify not giving a nodePool in the matrix include section results the included job running on the first nodePool in the list, or a default nodePool if no nodePool is specified
• Verify a wrong nodePool given in the matrix include list will result in a failed build with appropriate error message
• Verify giving a nodePool in the matrix include section results in a job being added to be run on the specified nodePool
• Verify not giving a nodePool in the matrix exclude section results the excluded job which matches the given config
• Verify a wrong nodePool given in the matrix exclude list will result in a failed build with appropriate error message
• Verify giving a nodePool in the matrix exclude section results in a job being excluded for only builds on that nodePool

pm-11436

• Verify when a build succeeds, scripts in always section gets executed.
• Verify when a build fails, scripts in always section gets executed.
• Verify when a build is timed out, always section doesn't get executed.
• Verify when a build is canceled, always section doesn't get executed.
• Verify scripts in always section should be able to use, env's and files created during the build.
• Verify build should fail, if the scripts in always section fail.

heap - 2611

• Matic should be able to sync the github account to shippable.
• Matic should be able to enable a github project in shippable.
• Matic should also be able to run commit, PR, tag and release builds for github project.
• Matic should be able to disable/reset github's project in shippable.
• Matic should be able to add a new syncRepo for github project.
• Matic should able run runSh jobs in shippable for github project.
• Matic should be able to delete github's project resources/jobs in SPOG.
• Matic should be able to add a gitRepo resource in the yml.
• Matic should be able to reset a github's subscription.