Install codecov for coverage tracking.

The scorecard actions is failing for https://github.com/ossf-tests/ossf-scorecard-action-private-repo-tests which have 0 commits.

This is a private repo.

/usr/bin/docker run --name a95e45f949b376e24a46b610c0[8](https://github.com/ossf-tests/ossf-scorecard-action-private-repo-tests/runs/5486680969?check_suite_focus=true#step:4:8)418327d2d_764[9](https://github.com/ossf-tests/ossf-scorecard-action-private-repo-tests/runs/5486680969?check_suite_focus=true#step:4:9)fd --label 29a95e --workdir /github/workspace --rm -e INPUT_RESULTS_FILE -e INPUT_RESULTS_FORMAT -e INPUT_REPO_TOKEN -e INPUT_PUBLISH_RESULTS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_REF_NAME -e GITHUB_REF_PROTECTED -e GITHUB_REF_TYPE -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e GITHUB_STEP_SUMMARY -e RUNNER_OS -e RUNNER_ARCH -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/ossf-scorecard-action-private-repo-tests/ossf-scorecard-action-private-repo-tests":"/github/workspace" 29a95e:45f949b376e24a46b6[10](https://github.com/ossf-tests/ossf-scorecard-action-private-repo-tests/runs/5486680969?check_suite_focus=true#step:4:10)c08418327d2d
Event file: /github/workflow/event.json
Event name: push
Ref: refs/heads/main
Repository: ossf-tests/ossf-scorecard-action-private-repo-tests
Fork repository: false
Private repository: true
Publication enabled: false
Format: sarif
Policy file: /policy.yml
Default branch: refs/heads/main
2022/03/09 20:42:28 unable to get tarball tarball not found: https://api.github.com/repos/ossf-tests/ossf-scorecard-action-private-repo-tests/tarball/. Skipping...
2022/03/09 20:42:28 internal error: ListCommits:error during graphqlHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
panic: internal error: ListCommits:error during graphqlHandler.setup: internal error: githubv4.Query: Resource not accessible by integration

goroutine 1 [running]:
log.Panic(0xc0006c1aa8, 0x1, 0x1)
	log/log.go:354 +0xae
github.com/ossf/scorecard/v4/cmd.scorecardCmd(0x176e760, 0xc000402060, 0x0, 0x6)
	github.com/ossf/scorecard/v4/cmd/root.go:168 +0x7bf
github.com/spf[13](https://github.com/ossf-tests/ossf-scorecard-action-private-repo-tests/runs/5486680969?check_suite_focus=true#step:4:13)/cobra.(*Command).execute(0x[17](https://github.com/ossf-tests/ossf-scorecard-action-private-repo-tests/runs/5486680969?check_suite_focus=true#step:4:17)6e760, 0xc0000ca010, 0x6, 0x6, 0x176e760, 0xc0000ca010)
	github.com/spf13/[email protected]/command.go:860 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x176e760, 0x177ee60, 0x0, 0xc000058778)
	github.com/spf13/[email protected]/command.go:974 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/[email protected]/command.go:902
github.com/ossf/scorecard/v4/cmd.Execute()
	github.com/ossf/scorecard/v4/cmd/root.go:104 +0x31
main.main()
	github.com/ossf/scorecard/v4/main.go:[21](https://github.com/ossf-tests/ossf-scorecard-action-private-repo-tests/runs/5486680969?check_suite_focus=true#step:4:21) +0x[25](https://github.com/ossf-tests/ossf-scorecard-action-private-repo-tests/runs/5486680969?check_suite_focus=true#step:4:25)

Scope of the token

The Signed-Releases check encourage developers to sign their artifacts locally using keys they manage locally (see https://github.com/ossf/scorecard/blob/main/docs/checks.md#signed-releases). I think we may want to remove this check from the action until cosign officially announces OIDC flow + verification. We're going to be asking users to switch to keyless cosign anyway later - in accordance with SLSA.

At this point we'll be able to tell users to use cosign keyless, and maybe give some points to those that use some sort of signature.

cc @asraa @azeemshaikh38 @naveensrinivasan

wdut?

I have these settings

I have an error in github actions:
https://github.com/YetiForceCompany/YetiForceCRM/runs/4893120151?check_suite_focus=true

/usr/bin/docker run --name e3725067a919945d28eaf8acaeef53846_f8b42a --label 84217e --workdir /github/workspace --rm -e INPUT_RESULTS_FILE -e INPUT_RESULTS_FORMAT -e INPUT_REPO_TOKEN -e INPUT_PUBLISH_RESULTS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_REF_NAME -e GITHUB_REF_PROTECTED -e GITHUB_REF_TYPE -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_ARCH -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/YetiForceCRM/YetiForceCRM":"/github/workspace" 84217e:3725067a919945d28eaf8acaeef53846
Event file: /github/workflow/event.json
Event name: push
Ref: refs/heads/developer
Private repository: false
Publication enabled: true
Format: sarif
Policy file: /policy.yml
refs/heads/developer not supported with 'push' event.
Only the default branch is supported

I think the problem is on the line
^refs/heads/(main|master)

• Explain that the results will only be available once a new commit is pushed. We can have a workflow_dispatch trigger users can trigger manually the first time to see the results flow
• Explain where to find the reason why a run failed
• Move users to GITHUB_TOKEN
• support for workflow_dispatch
• Explain whether pull request trigger is supported or not
• Call out that running scorecard on forks is not officially supported
• Orgs may need to enable GitHub Advanced Security #134 (comment)
• Explain the difference of PAT permissions that are needed
• In the starter workflow and our example workflow, remove the PAT. Have a commented out line for PAT, with a link to generate the PAT using the URL + the URL to create the repo secret

I believe #71 regressed PR runs of scorecard, where we don't actually need the repo_token because we aren't pushing the results, just checking them:

https://github.com/microsoft/msquic/runs/5259593124?check_suite_focus=true

The entrypoint.sh uses env variables because it was written s a bash script.
The current main.go follows the same design.

We can make the code simpler and more go-native by getting rid of unnecessary env variables.

Env variables that are necessary before running scorecard are https://github.com/ossf/scorecard-action/blob/main/entrypoint.sh#L26-L28 - basically the ones required via command line.

Other env variables, such as https://github.com/ossf/scorecard-action/blob/main/entrypoint.sh#L29-L34 and https://github.com/ossf/scorecard-action/blob/main/entrypoint.sh#L51-L53 can be safely replaced by native go variables.

@naveensrinivasan is this something you have bandwidth for?

cc @azeemshaikh38 @rohankh532

I set up this action and the scheduled run failed with,

refs/heads/master not supported with 'schedule' event.
Only the default branch 'refs/heads/null' is supported

Since I followed your guide and did not modify this action, I think there is a missing step somewhere?

I just encountered this error in during a run of our scorecard action:

...
Format: sarif
Policy file: /policy.yml
Default branch: refs/heads/main
2022/01/27 14:59:48 GitHub token env var is not set. Please read https://github.com/ossf/scorecard#authentication

The link there seems to be indicating that I should create another PAT and set GITHUB_AUTH_TOKEN to it. I don't understand this. I already pass repo_token to this step (here), and it has the scope (public_repo) recommended in the authentication steps. Why isn't the repo_token being used to avoid the rate limit?

If this absolutely is necessary, then why didn't the default setup instructions include this requirement up front?

The icon looks great but does not contains the usual score 10 like the scorecard logo. Let's update it

The part of the code implicated might be
https://github.com/ossf/scorecard-action/blob/main/entrypoint.sh#L52

Action https://github.com/prisma/prisma/actions/runs/1849483308/workflow

name: Scorecards supply-chain security
on:
  # Only the default branch is supported.
  branch_protection_rule:
  schedule:
    - cron: '21 21 * * 2'
  push:
    branches: [main]

# Declare default permissions as read only.
permissions: read-all

jobs:
  analysis:
    name: Scorecards analysis
    runs-on: ubuntu-latest
    permissions:
      # Needed to upload the results to code-scanning dashboard.
      security-events: write
      actions: read
      contents: read

    steps:
      - name: 'Checkout code'
        uses: actions/checkout@v2
        with:
          persist-credentials: false

      - name: 'Run analysis'
        uses: ossf/[email protected]
        with:
          results_file: results.sarif
          results_format: sarif
          # Read-only PAT token. To create it,
          # follow the steps in https://github.com/ossf/scorecard-action#pat-token-creation.
          repo_token: ${{ secrets.SCORECARD_READ_TOKEN }}
          # Publish the results to enable scorecard badges. For more details, see
          # https://github.com/ossf/scorecard-action#publishing-results.
          # For private repositories, `publish_results` will automatically be set to `false`,
          # regardless of the value entered here.
          publish_results: true

      # Upload the results as artifacts (optional).
      - name: 'Upload artifact'
        uses: actions/[email protected]
        with:
          name: SARIF file
          path: results.sarif
          retention-days: 5

      # Upload the results to GitHub's code scanning dashboard.
      - name: 'Upload to code-scanning'
        uses: github/codeql-action/upload-sarif@v1
        with:
          sarif_file: results.sarif

...

Logs https://github.com/prisma/prisma/runs/5207383913?check_suite_focus=true#step:5:16

Event file: /github/workflow/event.json
Event name: schedule
Ref: refs/heads/main
Repository: null
Private repository: null
Publication enabled: true
Format: sarif
Policy file: /policy.yml
Default branch: refs/heads/null
refs/heads/main not supported with 'schedule' event.
Only the default branch 'refs/heads/null' is supported

This shows that the default branch is not detected (detected as null). It should have been detected as main

refs/heads/main not supported with 'schedule' event.
Only the default branch 'refs/heads/null' is supported

I've just found out that there's a trigger to run workflows when branch protection settings change https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows#branch_protection_rule

As part of the broader effort to use scorecard results for attestation (e.g., SLSA provenance) I think we should enable this trigger for our workflow. In the beginning we'll just ignore the trigger in our implementation. But later, we can create an attestation for this event (including the payload). Besides its use for attestation, it can also be used to complement the data we collect via push events into our BQ table.

cc @azeemshaikh38 @asraa wdut?

We need to automate the step of generating the container image

I am testing this action on one of my repositories (https://github.com/jonasbb/serde_with). One of the problems I noticed is that the detection of the CII Best Practices status is broken and does mostly not detect the status correctly. My project has a CII passing status.

This is the view I have in the security tab.

The message even says, that the passing status is detected, but still tells me to inspect the remediation section. The remediation contains this, so I would expect no warning in the security tab.

We give full credit to projects that meet the passing criteria, which is a significant achievement for many projects.

#400 Introduced the action. At that point, the CII passing status was already established. #401 also got merged without affecting the status.
With #402 it got fixed, but already the next PR #406 broke it. Since then, multiple merges happened, but the status did not get updated.

The action takes an output_format as input, which may be sarif or json.
The action also takes an output_file. I'm wondering if we should simplify and remove the output_format?
We could use the extension of the output_file (results.sarif or results.json) to infer the format the use wants.

I think keeping the output_format is OK, but wanted to touch base and have everyone's opinion.

wdut?

cc @oliverchang @azeemshaikh38 @naveensrinivasan

Submit the starter workflows to GitHub.

The documentation recommends creating a read-only personal access token to run the action, when the action's own token (available as ${{ github.token }}) seems to work just fine, and requires no setup.

edit: AFAIK, the GitHub Actions token also cannot be used after the workflow is complete, unlike a PAT which lasts forever, and could grant an attacker access indefinitely -- though read-only, if configured as recommended.

Was there a reason the PAT is preferred over the GHA token?

If this change sounds fine, I'm happy to send a PR to update the docs.

Is it possible to have a custom icon for our action on the marketplace?
Currently we use a default "mic" https://github.com/marketplace/actions/ossf-scorecard-action.

The documentation at https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#branding seems to suggest we can only pick from the list here https://feathericons.com/

Is that true? Will the icon used in the starter-workflows be used on the marketplace too?

cc @josepalafox

The latest release of Scorecard is spitting out this error on every run:

Some warnings created by this action can be quite noisy, and there does not seem to be a way to silence certain error classes.
It seems that in #16 this feature was removed.

The Pinned-Dependencies check creates many warnings for GitHub Actions if they are not pinned by hash. Every update to the pinned tag will re-surface the same line in the Security Dashboard, even if it was previously selected as "won't fix". This creates unnecessary many alarms and due to the sheer number of "uses" in actions is by far the largest contributor of warnings.

All of GitHub's documentation also uses the form actions/checkout@v3, so this check conflicts with that. Instead, this is recommended actions/checkout@5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f # v2.3.4. The problem here is that the comment will not get updated by tools such as dependabot and thus does not contribute meaningfully, even is harmful as it suggests a different version than actually used.

I can see why somebody would want this check, but for my projects I do not see this as a benefit and would like to disable the check.

https://github.com/ossf-tests/scorecard-action

/usr/bin/docker run --name a95ec22592ac644c40c2acf9b34b0e2c2c7b_f20[8](https://github.com/ossf-tests/scorecard-action/runs/5486923378?check_suite_focus=true#step:4:8)04 --label 2[9](https://github.com/ossf-tests/scorecard-action/runs/5486923378?check_suite_focus=true#step:4:9)a95e --workdir /github/workspace --rm -e INPUT_RESULTS_FILE -e INPUT_RESULTS_FORMAT -e INPUT_REPO_TOKEN -e INPUT_PUBLISH_RESULTS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_REF_NAME -e GITHUB_REF_PROTECTED -e GITHUB_REF_TYPE -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e GITHUB_STEP_SUMMARY -e RUNNER_OS -e RUNNER_ARCH -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/scorecard-action/scorecard-action":"/github/workspace" 29a95e:c22592ac644c40c2acf9b34b0e2c2c7b
Event file: /github/workflow/event.json
Event name: push
Ref: refs/heads/main
Repository: ossf-tests/scorecard-action
Fork repository: null
Private repository: null
Publication enabled: true
Format: sarif
Policy file: /policy.yml
Default branch: refs/heads/null
The 'repo_token' variable is empty.
Please follow the instructions at https://github.com/ossf/scorecard-action#authentication to create the read-only PAT token.

https://github.com/ossf-tests/scorecard-action/runs/5486923378?check_suite_focus=true

test the end-to-end installation steps, run it and review results.
Current repos we have considered:

• distroless
• fulico
• cosign
• rekor

The icon renders too small. Let's update it

I tried using the image generated by our release workflow.
When using ghcr.io in the action:

runs:
  using: "docker"
  image: "ghcr.io://ossf/scorecard-action:latest"

I got the following error

'ghcr.io://ossf/scorecard-action:latest' should be either '[path]/Dockerfile' or 'docker://image[:tag]'

When trying

runs:
  using: "ghcr.io"
  image: "ghcr.io://ossf/scorecard-action:latest"

I got

Error: System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter ''using: ghcr.io' is not supported, use 'docker', 'node12' or 'node16' instead.')

@naveensrinivasan am I doing something wrong or is docker registry the only registry supported?
cc @azeemshaikh38

@laurentsimon @azeemshaikh38 @naveensrinivasan -- Would you mind updating the issue description into the following sections:

• feature parity w/ entrypoint.sh
• new functionality

We should descope the Golang-based rollout to expressly the things that are required for feature parity with the current entrypoint and then take new functionality as follow-up actions.

Originally posted by @justaugustus in #79 (comment)

See https://docs.github.com/en/actions/monitoring-and-troubleshooting-workflows/adding-a-workflow-status-badge
The build workflow should be https://github.com/ossf/scorecard-action/blob/main/.github/workflows/docker-sign.yml

This issue tracks the work being done as part of enabling Scorecard badges through Scorecard Action.

@rohankh532 fyi.

Hi, I just have a quick question/suggestion: I've seen that you set up your inputs so that the user has to create a PAT to use with the action.
Is there a reason why you don't use the built-in token as default, neither recommend doing so?

This issue tracks the work needed to improve test coverage of scorecard-action including e2e tests.

Context:

• #79 (comment)
• #79 (comment)

• #180
• #179
• #84
• #80
• #154
• #78
• #110
• #196
• #198
• ossf-tests/scorecard-action-results#2
• ossf/scorecard#1320 non blocking so skipping
• ossf/scorecard#1850 (comment) First iteration of the issue completed

How to use: an example in https://github.com/octokit/request-action
Optional: some of the docs in https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/creating-a-default-community-health-file#supported-file-types

cc @asraa @olivekl

Some of the text has new lines automatically inserted and does not look good.

Organizations and users may want to enable an action at scale on multiple repos. This requires

1. creating a PAT
2. saving as secrets in each repo
3. Creating pull requests

cc @inferno-chromium

@josepalafox is there an org setting to enable a workflow at scale?

https://github.com/ossf/scorecard-action/blob/main/action.yaml#L48 we need to pin our docker.
However, there's a problem because we currently generate the docker file upon new release generation thru this workflow https://github.com/ossf/scorecard-action/blob/main/.github/workflows/docker-sign.yml

This is a chicken-and-egg problem: in order to generate the release, we need the right hash to pin the action's docker image. But to generate it, we need a release. We may need to split the problem into 2 stages:

1. Generation of docker image

• Manually edit the hash in the workflow https://github.com/ossf/scorecard-action/blob/main/.github/workflows/docker-sign.yml#L40. Get the changes committed via PR/code review.
• Generate the docker image, maybe via a workflow triggered by dispatch even that we trigger manually. This may later be automated via a workflow that looks at every push event whether the hash was edited in the previous PR.

2. Generate the release

• Pin the hash for the docker image https://github.com/ossf/scorecard-action/blob/main/action.yaml#L48 using the hash of the image from step 1. Then get the changes committed via PR/code review.
• Create a release in GitHub UX

@naveensrinivasan @azeemshaikh38 other ideas?

Hello, I am getting the below error after #68 (fix for #66) landed and was picked up by my workflow:

Event file: /github/workflow/event.json
Event name: schedule
Ref: refs/heads/master
Private repository: null
Publication enabled: true
Format: sarif
Policy file: /policy.yml
Default branch: refs/heads/null
refs/heads/master not supported with 'schedule' event.
Only the default branch 'refs/heads/null' is supported

(ref: link)

This is happening on the scheduled runs of the workflow I have configured.

@laurentsimon - when you have a moment to take a look, could you help investigate? The repo in which this is run has the default branch set to master.

Thank you!

Currently, the results uploaded as artifacts to the Action tab don't include "scores" (i.e. X/10). Would it be possible to include the scores in the results?

(This is not an issue but a suggestion. But because this project doesn't have Discussion enabled, I didn't know where else to bring this up. My apologies for cluttering Issues.)

Would it be possible to follow the required structure here to make this more easily discoverable on the marketplace?

The main requirement is that action.yml lives in the root (and that there is only 1 action per repo).

See https://github.com/ossf/scorecard-action/blob/main/.github/workflows/docker-sign.yml#L37: we need to update to scorecard-action instead of scorecard-actions.

In addition, we need to update this line https://github.com/ossf/scorecard-action/blob/main/.github/workflows/docker-sign.yml#L33 to use ./Dockerfile since we've moved the file to the root folder.

I think it's fine for us to keep the hash hard-coded: during a new release fr the action, we will:

1. submit a PR to update the hash to the most recent version of scorecard image
2. create a release, which will trigger the workflow using the hardcoded hash

Things we should be able to test:

=== Unit tests: P0 =====

1. private repos
2. different trigger events: push, schedule, pull_request, workflow_dispatch, etc. Priority should be for push, schedule and pull request
3. different branches: we only support the default branch for push and schedule
4. forks (not supported), should be added to documentation #80
5. this problem #106 (comment)
6. Capture the different threshold scores, e.g. #129

The entrypoint of the action is entrypoint.sh. It expects some variables to be set, see https://github.com/ossf/scorecard-action/blob/main/entrypoint.sh#L25. These variables are the one we need to change for each unit test.

The shell script is part of a dockerfile: the dockefile defines the GitHub action, so we can build an image and call it with different parameters to create our unit tests.

The file $GITHUB_EVENT_PATH should contain all the data we want, except for a GitHub API bug, see https://github.com/ossf/scorecard-action/blob/main/entrypoint.sh#L38. his means we may need to create severals repos for testing purposes until the issue is resolved.
#75 (review)

Here is an example of tests done by a contributor who filed an issue #75 (review)

=== Unit tests: P1 =====

1. ossf/scorecard#1196 should test that command line does not change; and that the same checks are enabled with the same user's input

=== End-to-end tests: P2 ====
Run the workflow/upload the SARIF results to a repo and retrieve the results via an API. Verify the results shown are as expected. This is low priority, I think, because we already test SARIF generation in scorecard itself.

cc @azeemshaikh38 @naveensrinivasan anything missing?

We currently create/sign on every push, let's do it only on new releases

steps:

1. cut a scorecard release and wait for a container image to be created and tagged with new release. Note the hash of the container as CH1. Note: we do not need a scorecard release, we can use any stable version we want.
2. update the hash pin in our dockerfile to use the container hash CH1 from step 1. Once the PR is merged, note the GitHub's commit hash as GH2.
   3. manually trigger the workflow to generate our container image. Note the hash of the container image generated as CH3. It can be found here using the manifest's "digest".~~ ~~4. update the container image hash we use in [action.yaml:L45](https://github.com/ossf/scorecard-action/blob/main/action.yaml#L45), using the hash CH3from step 3. Once the PR is merged, note the GitHub's commit hash asGH4`.
3. test the new hash in a test repo we own. If successful, continue.
4. cut release for the action - the hash of the tagged release should be GH2.
5. send a PR to starter-workflows/code-scanning/scorecards.yml to update the hash to GH2 from step 4.
6. merge a PR to update our documentation's example workflow to use GH2.
7. verify on the market place that the workflow example contains GH2. (the marketplace uses main branch)