Giter Club home page Giter Club logo

check-payload's Introduction

check-payload

About

This application scans container images in an OpenShift release payload, RHEL based nodes, or an operator image for FIPS enabled binaries. The goal is to ensure binaries are compiled correctly for OpenShift.

Build

git clone https://github.com/openshift/check-payload.git
cd check-payload
make

Run

Prerequisites

  • podman should be installed on the node.
  • podman should be configured with pull secrets for the images to be scanned.

Configuration

The binary has a number of built-in configuration files.

A default built-in configuration (config.toml) is used if no options are specified, and no ./config.toml file is available from the current working directory when running the tool.

A specific configuration from a file can be specified using --config path/to/config.toml option. Use --config /dev/null to use an empty configuration.

An additional built-in coniguration tailored for a specific OpenShift version can be specified using -V, --config-for-version option, for example -V 4.11. When this option is specified, the settings from the additional configuration are added to (rather than override) the main configuration (see above). These additional for-version configurations are embedded into the binary during build time from the directories under dist/releases/.

Scan an OpenShift release payload

 sudo ./check-payload scan payload -V 4.11 \
   --url quay.io/openshift-release-dev/ocp-release:4.11.44-x86_64 \
   --output-file report.txt

Here

  • -V specifies the configuration for a particular OpenShift version;
  • --url specifies a payload URL;
  • --output-file specifies a file to write the scan report to.

Scan a Local Unpacked Image Bundle

The scan local subcommand allows you to scan a local unpacked image bundle for FIPS compliance. This is particularly useful for testing or analyzing local images that have been unpacked using tools like umoci.

Usage

To scan a local unpacked image bundle, use the following command:

./check-payload scan local --path /path/to/local/bundle

Here:

  • --path specifies the path to the local unpacked image bundle.

Example

./check-payload scan local --path ./test/resources/mock_unpacked_dir-1

This command will scan the local image bundle located in ./test/resources/mock_unpacked_dir-1.

Use Case

This feature is useful for verifying FIPS compliance of container images in a local development environment, without requiring access to podman mount which is blocked in some pipeline build systems.

Scan a container or operator image

sudo ./check-payload scan operator \
  --spec registry.ci.openshift.org/ocp-priv/4.11-art-assembly-art6883-3-priv@sha256:138b1b9ae11b0d3b5faafacd1b469ec8c20a234b387ae33cf007441fa5c5d567

Scan a node using container image

IMAGE=some.registry.location/check-payload
podman run --privileged -ti -v /:/myroot $IMAGE scan node --root /myroot

How it works

check-payload gathers container images from OpenShift release payloads or container images. The scanner can also be used on a RHEL or RHCOS node to scan the root image. A node scan runs within a container with the host OS mounted within it.

Container Image and Payload Scans

Container Image and Payload Scans gather images and emit file paths to the validation logic. This is done by podman mounting the image to the local machine, walking the directory tree, and emitting the paths for executables to the validation engine.

Node Scans

RHEL or RHCOS nodes can be scanned with check-payload scan node. To gather the file input paths the scanner queries for all the RPMs on the system and walks the paths within the RPMs finding executables. The list of executable paths are then processed by the validation engine.

Diagram

flowchart LR
  subgraph inputs
  Image --> Scanner
  Payload --> Scanner
  Node --> Scanner
  end
  Scanner --> Validations
  Validations --> Printer
  subgraph results
  Printer
  end

Validations

The validation engine uses different logic to validate golang and non-golang executables. The scanner only scans for ELF executables.

All

All scans validate the inclusion of OpenSSL via libcrypto found in /usr/lib64 or /usr/lib. The OpenSSL library is also validated to include {FIPS_mode, fips_mode, or EVP_default_properties_is_fips_enabled}.

Regular Executables

The rules to scan regular executables are:

  1. Must be dynamically linked

Most of RHEL/RHCOS executables are built dynamically to allow for dynamic linking to OpenSSL. There are exceptions for rule (1) which consists of some binaries (ldconfig, build-locale-archive, etc) which are required to be built statically, and/or do not provide cryptographic functionality.

Golang Executables

Golang validations run through a pipeline:

  1. validateGoVersion - enumerates the golang version and compile details
  2. validateGoCgo - ensure CGO_ENABLED=1 is set
  3. validateGoCGOInit - ensure cgo_init is within the binary
  4. validateGoStatic - ensure binary is dynamically linked
  5. validateGoOpenssl - ensure openssl matches the dynamic library within the system
  6. validateGoTags - ensure golang tags are set

Java Development Kit

JDK validations run through a pipeline:

  1. validateFipsHost - ensures the check-payload tool is being run on a FIPS enabled host
  2. validateSystemProperties - ensures pertinent FIPS property values are not being set at runtime
  3. validateAlgorithms - ensures unacceptable algorithms and protocols are disabled at runtime

Printer

The printer aggregates all the results and formats into a table, csv, markdown, etc. If any errors are found then the process exits non-zero. A successful run returns 0.

check-payload's People

Contributors

ashwindasr avatar bentito avatar dbenoit17 avatar dependabot[bot] avatar dominikholler avatar filipb avatar kolyshkin avatar liangxia avatar mrunalp avatar nickboldt avatar openshift-ci[bot] avatar openshift-merge-bot[bot] avatar openshift-merge-robot avatar rhmdnd avatar rphillips avatar stevekuznetsov avatar swghosh avatar tchughesiv avatar tesshuflower avatar xiaojiey avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

check-payload's Issues

OCPNODE-1767: Implement some statistics

When working on #33, I have implemented some bare-bone stat counters (mostly to make sure my changes did not affect the number of files scanned). It used to be part of #33, see dd05cb8

I think we need something like this. A set of global atomic counters would suffice.

Opening this one as this is the last item which was tracked by #68 (so I can finally close it).

Check filenames in config

We should implement checking that all paths (dirs, files) in configuration files are

  • absolute
  • cleaned

Otherwise, something might stop working in some unexpected ways (e.g. when a directory is provided with an ending /, which looks totally fine but won't work).

Rebase/split #33

Unfortunately, #33 grew too big and was not merged before a major refactoring has taken place in #42. This issue is opened to track the progress of rebasing what was in #33 into multiple small PRs.

Here are the commits from #33 and their respective new PRs:

  • 2eb1908 feat: ensure the config is fully parsed (#53)
  • 90a7457 fix: print the entire configuration (#54, #71)
  • 856a827 Check for isGoExecutable errors (part of #56)
  • 869bc3e Refactor ExpectedSyms to return bool; document it (#57)
  • 62daf28 feat: add known errors (#86)
  • e43aa38 feat: ability to ignore individual errors (#86)
  • 2f335d6 Makefile: add verify-clean and verify-generate targets (#86)
  • 1196d62 fix: simplify getComponent and displayExceptions (#58, #59)
  • 8ff413c feat: print exceptions in new ignore_errors format (#86)
  • dbdb94d feat: show error name, component and tag in log (#69, XXX)
  • 96e69fd refactor: rm config.NodeScan (#102)
  • 8381d70 fix: nodeScan: rm nodeVersion (part of #75)
  • e37abb0 Rename node. ignores to rpm. ignores (part of #75)
  • ca19a03 fix: node_scan: log innerPath (part of #84)
  • 190f911 refactor: remove tag arg from validation functions (#60)
  • 720e1c7 refactor: scanBinary: rm component and tag args (part of #84)
  • dd05cb8 feat: print some stats (tracked by #89)
  • 02603bd feat: unify scan logic (replaced by #87)

NB: commits that are crossed out are not going to be ported.

Config merge improvements

Currently, the ignore lists from the add-on config are appended to the main one, and if there are duplicates, they are appended, too. This works just fine, but can be improved in the following ways:

  • duplicates should not be added, as they makes things slower;
  • for embedded configs, we can check there are no duplicates, and fail CI otherwise;
  • for user-specified configs, we can warn (rather than error) on duplicates.

Same about ignore lists -- currently they are appended, but the lists with the same error name should be merged into the same key.

[OCPNODE-1743] Implement `scan node --all`

Per an internal discussion in Slack, we want to implement a different scan node mode.

Currently, scan node gets the list of all rpms and iterates over this list, checking all the files which the rpm contains.

Such a check has some limitations:

  1. it won't work for non-rpm-based system;
  2. it does not check files which do not belong to any rpm.

The idea is to add --all flag to scan node. Adding this flag would make the code to reuse the scan logic we use for payload scan (i.e. traverse the file tree). This way, we will potentially scan more files.

Add support to print the individual binaries scanned by the tool

While conducting the container scan for FIPS, the output lacks specific details regarding which binaries have been scanned. It would be beneficial to have clear information about the binaries that were actually scanned during the process.

output:

I1019 12:17:39.675986 251435 main.go:271] using config file: config.toml I1019 12:17:39.676027 251435 types_config.go:12] using config &{Components:[] FailOnWarnings:false FilterFile: FromFile: FromURL: InsecurePull:false Limit:-1 ContainerImageComponent: ContainerImage: OutputFile: OutputFormat:table Parallelism:5 Java:false PrintExceptions:false PullSecret: TimeLimit:1h0m0s Verbose:false UseRPMScan:false ConfigFile:{FilterFiles:[] FilterDirs:[/lib/firmware /lib/modules /usr/lib/.build-id /usr/lib/firmware /usr/lib/grub /usr/lib/modules /usr/share/app-info /usr/share/doc /usr/share/fonts /usr/share/icons /usr/share/openshift /usr/src/plugins /rootfs /sysroot] FilterImages:[] JavaDisabledAlgorithms:[DH keySize < 2048 TLSv1.1 TLSv1 SSLv3 SSLv2 TLS_RSA_WITH_AES_256_CBC_SHA256 TLS_RSA_WITH_AES_256_CBC_SHA TLS_RSA_WITH_AES_128_CBC_SHA256 TLS_RSA_WITH_AES_128_CBC_SHA TLS_RSA_WITH_AES_256_GCM_SHA384 TLS_RSA_WITH_AES_128_GCM_SHA256 DHE_DSS RSA_EXPORT DHE_DSS_EXPORT DHE_RSA_EXPORT DH_DSS_EXPORT DH_RSA_EXPORT DH_anon ECDH_anon DH_RSA DH_DSS ECDH 3DES_EDE_CBC DES_CBC RC4_40 RC4_128 DES40_CBC RC2 HmacMD5] PayloadIgnores:map[openshift-enterprise-pod-container:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrNotDynLinked Files:[/usr/bin/pod] Dirs:[]}]} operator-lifecycle-manager-container:{FilterFiles:[/usr/bin/cpb /usr/bin/copy-content] FilterDirs:[] ErrIgnores:[]} ose-olm-rukpak-container:{FilterFiles:[/unpack] FilterDirs:[] ErrIgnores:[]}] TagIgnores:map[] RPMIgnores:map[containernetworking-plugins:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrGoMissingTag Files:[] Dirs:[/usr/libexec/cni]}]} cri-o:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrGoMissingTag Files:[/usr/bin/crio /usr/bin/crio-status] Dirs:[]} {Error:ErrNotDynLinked Files:[/usr/bin/pinns] Dirs:[]}]} cri-tools:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrGoMissingTag Files:[/usr/bin/crictl] Dirs:[]}]} glibc:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrNotDynLinked Files:[/usr/sbin/ldconfig /sbin/ldconfig] Dirs:[]}]} glibc-common:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrNotDynLinked Files:[/usr/sbin/build-locale-archive] Dirs:[]}]} ignition:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrGoMissingTag Files:[/usr/lib/dracut/modules.d/30ignition/ignition] Dirs:[]}]} podman:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrGoMissingTag Files:[/usr/bin/podman /usr/libexec/podman/quadlet /usr/libexec/podman/rootlessport] Dirs:[]} {Error:ErrNotDynLinked Files:[/usr/libexec/podman/catatonit] Dirs:[]}]} podman-catatonit:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrNotDynLinked Files:[/usr/libexec/catatonit/catatonit] Dirs:[]}]} runc:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrGoMissingTag Files:[/usr/bin/runc] Dirs:[]}]} skopeo:{FilterFiles:[] FilterDirs:[] ErrIgnores:[{Error:ErrGoMissingTag Files:[/usr/bin/skopeo] Dirs:[]}]}] ErrIgnores:[]}} I1019 12:17:39.676103 251435 main.go:101] "scan" version="0.3.1-18-g27df6ad8" ---- Successful run

Removing the root config file

The config file at the root isn't necessarily needed since we almost always use a versioned (4.12, 4.11, etc) config file.

We can remove the root config file and document how the config files are used.

Ensure ability to read containers rpmdb

Node scan (and, since #78, payload scan) uses host rpm binary to read containers' (images') rpmdb. Surely, host and container can have very different rpm versions, configurations etc.

I have already added a hack so it works when host and container has different rpmdb path in #83 (see 84988f8).

Apparently, some more work is needed to ensure host rpm is working for all containers. In particular, @rphillips reported a failure to read rpmdb in #98 (comment).

It is not yet clear if we can fix this by adding some more options to host rpm, but I hope we can.

The alternative is to use container's rpm via chroot. This will work without further hacks, but the downsides are:

  • we have to trust container's rpm binary (and since we're a security tool to check containers binaries, this sounds crazy);
  • the container must have the same architecture as the host (not an issue currently, I guess).

check-payload fails with permission denied on sudo

image: registry.redhat.io/container-native-virtualization/virt-launcher-rhel9@sha256:f33f2be95e442b03481bc2762334b685cef4ba4f699ba46efc6d5074d363a44d
build: virt-launcher-rhel9-container-v4.16.0-232
check-payload failures:

  • /usr/bin/sudo: open /tmp/local-image-5bc8un30/unpacked/usr/bin/sudo: permission denied
  • /usr/bin/sudoreplay: open /tmp/local-image-5bc8un30/unpacked/usr/bin/sudoreplay: permission denied

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.