Giter Club home page Giter Club logo

clair's Introduction

Clair

Docker Repository on Quay PkgGoDev IRC Channel

Note: The main branch may be in an unstable or even broken state during development. Please use releases instead of the main branch in order to get stable binaries.

Clair Logo

Clair is an open source project for the static analysis of vulnerabilities in application containers (currently including OCI and docker).

Clients use the Clair API to index their container images and can then match it against known vulnerabilities.

Our goal is to enable a more transparent view of the security of container-based infrastructure. Thus, the project was named Clair after the French term which translates to clear, bright, transparent.

The book contains all the documentation on Clair's architecture and operation.

Community

Contributing

See CONTRIBUTING for details on submitting patches and the contribution workflow.

License

Clair is under the Apache 2.0 license. See the LICENSE file for details.

clair's People

Contributors

alecmerdler avatar allda avatar arajkumar avatar arthurlm44 avatar caipre avatar coolljt0725 avatar crozzy avatar davidxia avatar dependabot[bot] avatar djelibeybi avatar greenberga avatar hdonnay avatar jonboulle avatar jzelinskie avatar katee avatar keyboardnerd avatar ldelossa avatar liangchenye avatar mackjm avatar mcoops avatar mrqwer88 avatar philips avatar quentin-m avatar ruokai-lai avatar supereagle avatar supersandro2000 avatar unageanu avatar vbatts avatar vsamidurai avatar yebinama avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

clair's Issues

Calculate CVEs by layer and its parent may not be correct

It seems that layers will be better shared among different images according to Docker 1.10.
https://github.com/docker/docker/releases/tag/v1.10.0-rc1
moby/moby#17924

So there will be images like these:
ImageA : [layer-baseA, layer-a, layer-b]
ImageB : [layer-baseB, layer-a, layer-c, layer-d]
If it happens, layer-a may have two different parents. In this case, we cannot get the correct CVEs of ImageA and ImageB because current Clair does not know the whole layer chain.

Logically layers with different parents should already exist because DockerV2 use a manifest to manage layers of an image (unlike V1). But since imageID is generated randomly, this does not happen.
In my test here: (https://github.com/liangchenye/image-mining/tree/master/parent-layer), no conflicted parents are found after scanning 1000+images.

Add a "get started" section in readme

I'd like to test my containers on my local machine.
How could I use your project to do so?

I saw your amazing video, and I really would like to test my containers.

Thanks!

All images look safe if allowed host IP doesn't match remote host IP

I was happy to see all of our images come up as safe, but when I tried one that was known to be unsafe, it also showed as safe. Have I possibly misconfigured?

I tried Aerospike 3.0.7.2 and got this:
$ docker images | grep aero
aerospike latest 1f38428d7a6f 12 days ago 153.2 MB

$ ./analyze-local-images -endpoint "http://192.168.99.101:6060" -my-address "192.168.99.1" 1f38
Saving 1f38
Getting image's history
Setting up HTTP server (allowing: 192.168.99.101)
Analyzing 11 layers

  • Analyzing c950d63587be10331540688387593721eb728628b75cdaf1dcf266188dfc6e6b
  • Analyzing 3ba3ba0cdebdf6c3fc49dc6ee09f0c01d653796d925ef9c99b4c9ea988bcee7f
  • Analyzing d9376c3ba53cb09b9301a76d07e9a47c784b5d6d4829d6f5142a31339dd41da5
  • Analyzing 77200864887ad2f6373701d0515f3bda7e8e86f530416d0c8c493309d65d7a16
  • Analyzing fbb261c1559b777bcbb16a39a762ad32a478a050b730fedd61f1315ba404c50a
  • Analyzing 38a15afbc1c259497ea86103ded6274289c47e00967eeeab20b00e99e9b892df
  • Analyzing e997a3007992b4ef65921b0d4284b20a59b789da9a887794106e0328a70ae89d
  • Analyzing 975d8801fa166fd701505a6640f368cd70871d64b41d72a527e26f0811c6428d
  • Analyzing fa0642e3ade87b09cbc02c406ad3cc4902376a61ca322643934f4725a60205be
  • Analyzing 63b21889144306f2b65e874677fac6b8453fd9baf569eac399eeb44cfd1b022b
  • Analyzing 1f38428d7a6f1f67c84c1cc3ff8fd19a6b8bf5233e35a30d388992fa14056243
    Getting image's vulnerabilities
    Bravo, your image looks SAFE !

Within the running Clair container I took a look around for CVEs and only found what looks like ubuntu-relevant CVES:

root@d1e39d76aa90:/go/src/github.com/coreos/clair# ls /tmp
aberr k5_skip tce tcloop ubuntu-cve-tracker386367386
root@d1e39d76aa90:/go/src/github.com/coreos/clair# ls /tmp/ubuntu-cve-tracker386367386/repository/active
-- shows around 3932 CVE's from CVE-2002-2439 to CVE-2015-8660

I also tried the curl commands on individual layers of the image and didn't get anything.

During setup, I had let the updater finsh not once but twice:
docker run -it -v /tmp:/tmp -p 6060:6060 -p 6061:6061 clair:0.1 --db-path bolt
2015-12-29 18:50:56.093322 I | database: database at bolt does not exist yet, creating it
2015-12-29 18:50:56.100647 I | api: starting API on port 6060.
2015-12-29 18:50:56.101250 I | api: starting Health API on port 6061.
2015-12-29 18:50:56.102573 I | updater: updater service started. lock identifier: 6b423c83-4eca-4d72-80c8-4094db0fd45c
2015-12-29 18:50:56.104140 I | updater: updating vulnerabilities
2015-12-29 18:50:56.104371 I | updater/fetchers: fetching Ubuntu vulneratibilities
2015-12-29 18:50:56.130798 I | updater/fetchers: fetching Debian vulneratibilities
2015-12-29 18:50:56.137502 I | updater/fetchers: fetching Red Hat vulneratibilities
2015-12-29 19:08:36.082005 N | database: Ignoring 17060 notifications
2015-12-29 19:08:36.118490 I | updater: update finished
2015-12-29 20:08:36.008813 I | updater: updating vulnerabilities
2015-12-29 20:08:36.009312 I | updater/fetchers: fetching Ubuntu vulneratibilities
2015-12-29 20:08:36.012095 I | updater/fetchers: fetching Debian vulneratibilities
2015-12-29 20:08:36.013228 I | updater/fetchers: fetching Red Hat vulneratibilities
2015-12-29 20:09:38.079677 I | updater: update finished

Is the ignoring of 17060 notifications something that I might not want to ignore in my case?

add a `ListVulnerabilities` API

When I use Clair, sometimes I want to know whether my images are really secure or just because my CVE database is empty, so I try to read my CVE database. I found that Clair does not have such API yet. There is a similar API (list namespace), I think maybe we can add this. I already work on it in my repo (liangchenye/clair:getvulns), now the status is 'in progress'.

@Quentin-M @jzelinskie what do you think about this API?

Generic vulnerability link & more metadata

Because otherwise it could link to the Debian vuln on an Ubuntu container and may be confusing.

We could also get more informations about CVEs (Access Vector, Access Complexity, Authentication ...) with additional databases such as NVD.

kubectl submission of config.yml secret fails:

The instructions in the README.md say to submit the clair-kubernetes.yml file first, but this seems it would fail as it's trying to mount a secret that is submitted second. If it fails, it should restart and try to find the newly submitted secret. However, either way it still fails with config.yaml not found:

knepper@MacBook-Pro-2:~/clair/contrib/k8s> kubectl --server="10.3.3.143:8080" create -f clair-kubernetes.yaml 
You have exposed your service on an external port on all nodes in your
cluster.  If you want to expose this service to the external internet, you may
need to set up firewall rules for the service port(s) (tcp:30060,tcp:30061) to serve traffic.

See http://releases.k8s.io/release-1.1/docs/user-guide/services-firewalls.md for more details.
service "clairsvc" created
replicationcontroller "clair" created
replicationcontroller "clair-postgres" created
error validating "clair-kubernetes.yaml": error validating data: field nodePort: is required; if you choose to ignore these errors, turn validation off with --validate=false
jknepper@MacBook-Pro-2:~/clair/contrib/k8s> kubectl --server="10.3.3.143:8080" describe svc
Name:           clairsvc
Namespace:      default
Labels:         app=clair
Selector:       app=clair
Type:           NodePort
IP:         10.100.90.139
Port:           clair-port0 6060/TCP
NodePort:       clair-port0 30060/TCP
Endpoints:      <none>
Port:           clair-port1 6061/TCP
NodePort:       clair-port1 30061/TCP
Endpoints:      <none>
Session Affinity:   None
No events.

Name:           dashboard-api-service
Namespace:      default
Labels:         romulus/lb=vulcan,romulus/route=public
Selector:       app=dashboard-api
Type:           NodePort
IP:         10.100.34.226
Port:           <unnamed>   9003/TCP
NodePort:       <unnamed>   30001/TCP
Endpoints:      10.244.87.11:9292
Session Affinity:   None
No events.

Name:           kubernetes
Namespace:      default
Labels:         component=apiserver,provider=kubernetes
Selector:       <none>
Type:           ClusterIP
IP:         10.100.0.1
Port:           <unnamed>   443/TCP
Endpoints:      10.3.3.143:6443
Session Affinity:   None
No events.

Nonetheless, my problem is the submission of the secret fails with the following output:

jknepper@MacBook-Pro-2:~/clair/contrib/k8s> kubectl --server="10.3.3.143:8080" create -f config.yaml --v=3
F0321 09:31:44.340694   84663 helpers.go:96] error validating "config.yaml": error validating data: Object 'apiVersion' is missing in '{"clair":{"api":{"cafile":null,"certfile":null,"healthport":6061,"keyfile":null,"paginationKey":null,"port":6060,"servername":null,"timeout":"900s"},"database":{"cacheSize":16384,"source":"postgres://postgres:password@postgres:5432/postgres?sslmode=disable"},"notifier":{"attempts":3,"http":{"cafile":null,"certfile":null,"endpoint":null,"keyfile":null,"servername":null},"renotifyInterval":"2h"},"updater":{"interval":"2h"}}}'; if you choose to ignore these errors, turn validation off with --validate=false

My kubernetes version is:

v1.0.3

Pod creation failing:

jknepper@MacBook-Pro-2:~/k8s> kubectl --server="10.3.3.143:8080" describe pods clair-u1p5r
Name:               clair-u1p5r
Namespace:          default
Image(s):           quay.io/coreos/clair
Node:               10.3.3.54/10.3.3.54
Start Time:         Mon, 21 Mar 2016 09:13:47 -0700
Labels:             app=clair
Status:             Running
Reason:             
Message:            
IP:             10.244.87.15
Replication Controllers:    clair (1/1 replicas created)
Containers:
  clair:
    Container ID:       docker://aa09617c867d0d7b1f9a0ad1b42d8d7ca513fce9824da7bceae0202a59331c10
    Image:          quay.io/coreos/clair
    Image ID:           docker://7158ef42f9d805c0498a2231c4fc5d6740f5bd359bf4e3540eda3ade3d985a49
    State:          Running
      Started:          Mon, 21 Mar 2016 09:47:11 -0700
    Last Termination State: Terminated
      Exit Code:        1
      Started:          Mon, 21 Mar 2016 09:47:01 -0700
      Finished:         Mon, 21 Mar 2016 09:47:01 -0700
    Ready:          False
    Restart Count:      11
    Environment Variables:
Conditions:
  Type      Status
  Ready     False 
Volumes:
  secret-volume:
    Type:   Secret (a secret that should populate this volume)
    SecretName: clairsecret
  default-token-i7ba3:
    Type:   Secret (a secret that should populate this volume)
    SecretName: default-token-i7ba3
Events:
  FirstSeen LastSeen    Count   From            SubobjectPath               Reason      Message
  ─────────   ────────    ───── ────            ─────────────             ──────      ───────
  33m       33m     1   {scheduler }                            scheduled   Successfully assigned clair-u1p5r to 10.3.3.54
  33m       33m     1   {kubelet 10.3.3.54} implicitly required container POD   pulled      Pod container image "gcr.io/google_containers/pause:0.8.0" already present on machine
  33m       33m     1   {kubelet 10.3.3.54} implicitly required container POD   created     Created with docker id 2518104c9595
  33m       33m     1   {kubelet 10.3.3.54} implicitly required container POD   started     Started with docker id 2518104c9595
  33m       33m     1   {kubelet 10.3.3.54} spec.containers{clair}          created     Created with docker id 9118b37ec74f
  33m       33m     1   {kubelet 10.3.3.54} spec.containers{clair}          started     Started with docker id 9118b37ec74f
  33m       33m     1   {kubelet 10.3.3.54} spec.containers{clair}          created     Created with docker id 6abc111e5983
  33m       33m     1   {kubelet 10.3.3.54} spec.containers{clair}          started     Started with docker id 6abc111e5983

The error returned from the submitted pods are related to not being able to find the config.yaml:

jknepper@MacBook-Pro-2:~/k8s> kubectl --server="10.3.3.143:8080" logs clair-u1p5r
2016-03-21 16:47:41.652439 C | main: failed to load configuration: open /config/config.yaml: no such file or directory

I also attempted removing references to the secret, the config arguments to the clair container as well as the mount point to get the ""defaults"" specified in the config.yml file as stated when you don't use the secret, and it still failed with the same error but different path '/etc/config/config.yaml' not found.

Add support not only tar layouts

Hello!

How I see clair can add layouts only in tar archives now.
Are planned add layouts like directory and tar.gz?
As example with tar.gz support it be more comfortably to check OpenVZ templates - https://download.openvz.org/template/precreated/ ( if ungzip it - clair work correctly with this templates ).
And if clair can work with directory - it can check already created openvz containers and etc.

Cannot fetching update on rhel-based os

Hello!

I try run clair on fedora and centos 6(binary builded on fedora 21).
After start it have errors like -

2015-11-16 07:44:08.473519 I | updater/fetchers: fetching Ubuntu vulneratibilities
2015-11-16 07:44:08.473851 I | updater/fetchers: fetching Debian vulneratibilities
2015-11-16 07:44:08.474215 I | updater/fetchers: fetching Red Hat vulneratibilities
2015-11-16 07:46:11.600821 E | updater/fetchers: could not parse bzr branch output to get the revision number
2015-11-16 07:46:11.600855 E | updater: an error occured when fetching update 'Ubuntu': could not download requested ressource.

bzr versions 2.6.0 and 2.1.1.

Can you help fix it?

Document usage

How do I actually run clair against an image pushed to a registry?

When I start it, it basically sits and does nothing:

➜  ~  docker run -p 6060:6060 -p 6061:6061 quay.io/coreos/clair --db-path=/tmp/foo.db --log-level=debug
2015-11-14 16:54:44.631509 I | database: database at /tmp/foo.db does not exist yet, creating it
2015-11-14 16:54:44.641627 I | api: starting Health API on port 6061.
2015-11-14 16:54:44.642071 I | api: starting API on port 6060.
2015-11-14 16:54:44.642431 I | updater: updater service started. lock identifier: 1d652969-2f8f-40e0-9ca2-79f0f9125101
2015-11-14 16:54:44.642712 D | updater: attempting to obtain update lock
2015-11-14 16:54:44.644510 I | updater: updating vulnerabilities
2015-11-14 16:54:44.644664 I | updater/fetchers: fetching Ubuntu vulneratibilities
2015-11-14 16:54:44.644901 I | updater/fetchers: fetching Debian vulneratibilities
2015-11-14 16:54:44.647307 I | updater/fetchers: fetching Red Hat vulneratibilities
2015-11-14 17:58:40.143125 N | database: Ignoring 16668 notifications
2015-11-14 17:58:40.167581 I | updater: update finished
2015-11-14 17:58:40.175431 D | database: Lock updater owned by 1d652969-2f8f-40e0-9ca2-79f0f9125101 has expired.
2015-11-14 17:58:40.191551 D | updater: next update attempt scheduled for 2015-11-14 18:58:40.167676167 +0000 UTC.
2015-11-14 19:35:17.925595 I | api: 200 GET /v1/versions (112.378µs)

How can I get it to check a container's flaws?

Cannot analyze image using analyze-local-images

I'm trying to run the analyze-local-images but I get an error like this:

analyze-local-images 778b25451af2
Saving 778b25451af2
Getting image's history
Analyzing 32 layers
- Analyzing a2c33fe967de5a01f3bfc3861add604115be0d82bd5192d29fc3ba97beedb831
2015/11/24 16:00:16 - Could not analyze layer: Got response 404 with message {"Message":"the resource cannot be found"}

Is this an issue with the tool or clair? I'm running docker 1.9.1 with latest clair tag:

quay.io/coreos/clair        latest              bd5cdf49293a        3 hours ago         786.1 MB

Set connection string for database?

I'm trying to get clair docker to connect to my postgres container with at connection string, but it won't work.

My connection string is setup like this
source: postgresql://postgres:[email protected]/clairdb

I also tried to run without the database clairdb which I created manually (not sure if that is needed), but I get the same error when I'm trying to run the container.

2016-03-02 22:10:10.950887 I | pgsql: running database migrations
2016-03-02 22:10:10.955685 E | pgsql: pq: SSL is not enabled on the server
2016-03-02 22:10:10.955803 C | main: database: could not open database

I haven't found any documentation on how to set this up. What am I doing wrong

analyze-local-images: says “the resource cannot be found”

I was trying to use the new analyze-local-images program, but it doesn’t seem to work for me:

In one terminal, I start clair:

$ docker run -it -p 6060:6060 -p 6061:6061 quay.io/coreos/clair --db-path=/db/bolt
2015-11-23 21:37:05.493710 I | database: database at /db/bolt does not exist yet, creating it
2015-11-23 21:37:05.499966 I | api: starting API on port 6060.
2015-11-23 21:37:05.500029 I | updater: updater service started. lock identifier: 78143d76-b946-45b5-86d5-e80dd72c9dce
2015-11-23 21:37:05.500291 I | api: starting Health API on port 6061.
2015-11-23 21:37:05.502117 I | updater: updating vulnerabilities
2015-11-23 21:37:05.502210 I | updater/fetchers: fetching Debian vulneratibilities
2015-11-23 21:37:05.502467 I | updater/fetchers: fetching Ubuntu vulneratibilities
2015-11-23 21:37:05.502744 I | updater/fetchers: fetching Red Hat vulneratibilities
2015-11-23 21:40:37.745528 E | worker: layer cf2616975b4a3cba083ca99bc3f0bf25f5f528c3c52be1596b30f60b0b1c37ff: failed to extract data from /tmp/analyze-local-image-197689302/cf2616975b4a3cba083ca99bc3f0bf25f5f528c3c52be1596b30f60b0b1c37ff/layer.tar: the resource cannot be found
2015-11-23 21:40:37.745605 I | api: 404 POST /v1/layers (390.349µs)
2015-11-23 21:40:58.800581 E | worker: layer 902b87aaaec929e80541486828959f14fa061f529ad7f37ab300d4ef9f3a0dbf: failed to extract data from /tmp/analyze-local-image-040123037/902b87aaaec929e80541486828959f14fa061f529ad7f37ab300d4ef9f3a0dbf/layer.tar: the resource cannot be found
2015-11-23 21:40:58.800611 I | api: 404 POST /v1/layers (175.692µs)

In another one, I run analyze-local-images:

$ analyze-local-images debian:jessie      
Saving debian:jessie
Getting image's history
Analyzing 2 layers
- Analyzing 902b87aaaec929e80541486828959f14fa061f529ad7f37ab300d4ef9f3a0dbf
2015/11/23 22:40:58 - Could not analyze layer: Got response 404 with message {"Message":"the resource cannot be found"}

My docker version:

$ docker version
Client:
 Version:      1.8.3
 API version:  1.20
 Go version:   go1.5.1
 Git commit:   f4bf5c7
 Built:        
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.3
 API version:  1.20
 Go version:   go1.5.1
 Git commit:   f4bf5c7
 Built:        
 OS/Arch:      linux/amd64

I’m not sure what’s going wrong here. Can you advise please?

Add package.json for custom packages

Allow users to add a package.json file (or some other filename) in the root directory of containers where they can specify packages to add for detection, as well as packages to ignore for detection (say, if they are included in the container but never executed).

Priorities seem to be always shown in the Debian context

TL;DR: API returns vulnerabilities considered low by Ubuntu (when I ask for high). Not sure if existing issues like #19 or #54 cover entirely this problem.

Let's say I query /v1/vulnerabilities/CVE-2015-5277 to get a vulnerability's information. Answer:

{"ID":"CVE-2015-5277","Link":"https://security-tracker.debian.org/tracker/CVE-2015-5277","Priority":"High","Description":"The get_contents function in nss_files/files-XXX.c in the Name Service Switch (NSS) in GNU C Library (aka glibc or libc6) before 2.20 might allow local users to cause a denial of service (heap corruption) or gain privileges via a long line in the NSS files database.","AffectedPackages":[{"OS":"debian:9","Name":"glibc","AllVersions":false,"BeforeVersion":"2.21-1"},{"OS":"debian:unstable","Name":"glibc","AllVersions":false,"BeforeVersion":"2.21-1"},{"OS":"ubuntu:14.04","Name":"eglibc","AllVersions":true,"BeforeVersion":""},{"OS":"debian:8","Name":"glibc","AllVersions":true,"BeforeVersion":""}]}
  1. Both Link and Priority fields are shown for Debian. The same vulnerability has a different priority in the Ubuntu CVE tracker. So the same vulnerability can be assigned different priorities by the different trackers.
  2. When querying /v1/layers/{ID}/vulnerabilities with the default minimumPriority (High) using an image based on Ubuntu 14.04, the API will return vulnerabilities that are considered having Low or Medium priorities by the Ubuntu CVE tracker.

I am wondering if this behaviour have been chosen for simplicity, or if there is something I missed. It would be nice if the priority for a vulnerability reflects the OS on which our image is based.

Add 'update reminding' feature

quay.io + Clair is really cool, but one thing is missing from a user's point of view: yes, my image is vulnerable, but is there an easy way to solve this?

To an OS user, we can always use 'apt-get update or zypper update' to keep our system as secure as possible. But to a container user, the scenario is different, most time we need another more secure image to replace the old one that was built months ago. So it would be user friendly if 'Clair' could neither tell the user:
'rebuild your image with the same Dockfile (with 'apt-get update' inside), CVE A, CVE B would be solved.
or 'update your image with another base image(from 14.04 to 15.04), CVE A, CVE B, CVE C would be solved.
With the integration of quay.io, it is easy to get a more secure image.

I'm not sure if we can have this feature in the Clair roadmap.

Use a configuration file

There are already a lot of flags ... Should use a configuration file instead and keep only two flags: log level and path to configuration file.

API: use gRPC and grpc-gateway

In many of our open source projects we are exposing API services we are starting to use gRPC and grpc-gateway. The advantage of using this framework instead of hand-rolled HTTP APIs are:

  • Codegen for the clients/servers
  • High efficiency gRPC based API available
  • Swagger codegen for REST API

This isn't high priority and should be something to consider for the v2 API or perhaps rebasing the v1 API to use it.

clair fetch image layer from openstack swift or ceph s3

Hi,
In my private registry, the images stored in openstack swift which need http auth header to download or get a file, so when I use "POST /v1/layers" API, I can't insert a layer from swift to clair directly. I modify "worker/worker.go func getLayerData()" to add a auth header before "r, err := http.Get(path)" but is not a graceful solution, is there a better one :).

analyze-local-image could not analyze layer: Got response 404 with message

I've got clair running in a docker container:

-sh-4.2$  docker run --env-file ./proxy-env  -p 6060:6060 -p 6061:6061  -v /tmp:/tmp  -v /home/jeremie/quay-config:/config:ro quay.io/coreos/clair:latest --config=/config/config.yaml
2016-02-11 17:27:33.823292 I | database: database at /db/bolt does not exist yet, creating it
2016-02-11 17:27:33.835770 I | api: starting main API on port 6060.
2016-02-11 17:27:33.836521 I | notifier: notifier service is disabled
2016-02-11 17:27:33.836549 I | api: starting health API on port 6061.
2016-02-11 17:27:33.836819 I | updater: updater service started. lock identifier: f3a44bee-bf15-4242-8b0a-60a0c3f70bbe
2016-02-11 17:27:33.839272 I | updater: updating vulnerabilities
2016-02-11 17:27:33.839360 I | updater/fetchers: fetching Debian vulneratibilities
2016-02-11 17:27:33.839437 I | updater/fetchers: fetching Red Hat vulneratibilities
2016-02-11 17:27:33.839863 I | updater/fetchers: fetching Ubuntu vulneratibilities
2016-02-11 17:56:24.140684 N | database: Ignoring 17374 notifications
2016-02-11 17:56:24.492351 I | updater: update finished
2016-02-11 17:58:37.701983 E | worker: layer <missing>: failed to extract data from /tmp/analyze-local-image-257595031/<missing>/layer.tar: the resource cannot be found
2016-02-11 17:58:37.704748 I | api: 404 POST /v1/layers (96.482403ms)

But when I try to analyze-local-images debian:jessie, it is able to save the image, but errors during analyis:

-sh-4.2$ ./work/bin/analyze-local-images debian:jessie
Saving debian:jessie
Getting image's history
Analyzing 2 layers
- Analyzing <missing>
2016/02/11 11:58:37 - Could not analyze layer: Got response 404 with message {"Message":"the resource cannot be found"}

If I look inside the container that's running clair, I do see the images in /tmp/

-sh-4.2$ docker exec insane_bohr ls -l /tmp/analyze-local-image-257595031/
total 20
-rw-r--r-- 1 29857 5002 1395 Jan 25 22:24 9a02f494bef8d0d088ee7533aa1ba4aaa1dbf38a97192d36fa79a51279bc04de.json
drwxr-xr-x 2 29857 5002 4096 Jan 25 22:24 9c3e80c252d138c43eee4c75cdcaaea98b1d2809b9f29263a293cbcae2b30366
drwxr-xr-x 2 29857 5002 4096 Jan 25 22:24 a240a64787deebe0fbf5ac2682dd24f4f93b6275e4d61ded85f7e2a3e6771c0d
-rw-r--r-- 1 29857 5002  279 Jan  1  1970 manifest.json
-rw-r--r-- 1 29857 5002   89 Jan  1  1970 repositories

Any ideas?

Questions: are CVE sources fetched regularly

it is not clear in documentation if remote sources are fetched at regular interval.
At Clair startup I can see the fetch of remove sources (Debian, etc.), but are they updated at regular interval ? Or should I trigger (how?) Clair to update itself ?

At last, once a layer has been analysed by Clair, let's say at time X.
At time X+1, let's say new vulnerabilities are available from remote source and Clair got them (updated its database).
If I query my layer again, will vulnerabilities returned for this layer contain the new vulnerabilities, or should I reupload my layer at regular interval to get it up-to-date against current database?

Thanks

All images are safe !!

Hi,

I'm testing Clair on a local machine. In particular, I'm trying to find vulnerabilities in my own images using the "analyze-local-images" script.
However, I can't find a vulnerable image that is recognized by this script. I also downloaded some images that are to vulnerable to OpenSSL heart bleed but according to Clair they are safe.

Am I missing something? Am I doing something wrong?

Could you please point out a few images in which Clair will find vulnerabilities?

Thanks

namespace API should return objects

In order to future-proof the API for namespaces, we should return a list of objects for the GET route rather than just a list of strings.

Clair Container run error

I just install fresh CentOS and docker engine, I run ubuntu 14.04 image. Now I want to try Clair. So i save config file and run given commands: (I have not changed anything in config file)

$ docker run -p 6060:6060 -p 6061:6061 -v /home/test/config:/config:ro quay.io/coreos/clair:latest --config=/config/config.yaml
2016-01-19 13:27:56.405799 I | api: starting main API on port 6060.
2016-01-19 13:27:56.408493 I | api: starting health API on port 6061.
2016-01-19 13:27:56.408638 I | notifier: notifier service is disabled
2016-01-19 13:27:56.408651 I | updater: updater service started. lock identifier: b6fa8f67-b516-4880-b560-3945808e1640
2016-01-19 13:27:56.408998 I | updater: updating vulnerabilities
2016-01-19 13:27:56.409037 I | updater/fetchers: fetching Ubuntu vulneratibilities
2016-01-19 13:27:56.409216 I | updater/fetchers: fetching Debian vulneratibilities
2016-01-19 13:27:56.413911 I | updater/fetchers: fetching Red Hat vulneratibilities

STDOUTS stops here everytime, i wait for hours even. I am newbie, I am working on some idea's in Docker world so please also put some info in answer which could help me in getting started with Clair.

Optimize paginated notification query (PostgreSQL)

The PostgreSQL query searchNotificationLayerIntroducingVulnerability uses ORDER and LIMIT clause. It also uses id >= $1 instead of OFFSET for performance.

Now, instead of LIMIT, we should really use something like id < $2 to avoid long running queries when the first results are too far from the starting id. It would guarantee that these pagination queries are consistently running quickly, even though they may return no results.

Make the healthcheckers more precise

Concerns especially the updater's healthcheck in which health data is cached and can stay unchanged for a while. It would be better to actively retrieve these.

Doc question: need to create a Postgres database in order to run Clair container

It seems that a Postgres database needs to be created first, otherwise, docker run gave errors:

docker run --rm -v /path/to/image-storage:/path/to/image-storage -p 6060:6060 -p 6061:6061 quay.io/coreos/clair --db-type=sql --db-path='host=postgresdb.example.com port=5432 user=admin password=adminPass' --update-interval=2h
E1117 18:56:31.908102 00001 quadstore.go:59] Couldn't open database at host=postgresdb.example.com> port=5432 user=admin password=adminPass': &pq.Error{Severity:"FATAL", Code:"3D000", Message:"database \"admin\" does not exist", Detail:"", Hint:"", Position:"", InternalPosition:"", InternalQuery:"", Where:"", Schema:"", Table:"", Column:"", DataTypeName:"", Constraint:"", File:"postinit.c", Line:"775", Routine:"InitPostgres"}

I created the database like so:

CREATE ROLE clair with LOGIN CREATEDB PASSWORD 'somepassword;
CREATE DATABASE clair;
GRANT ALL PRIVILEGES ON DATABASE clair to clair;

And then this works:

docker run --rm -v /path/to//image-storage:/path/to/image-storage -p 6060:6060 -p 6061:6061 quay.io/coreos/clair --db-type=sql --db-path='host=postgresdb.example.com port=5432 user=clair password=somepassword' --update-interval=2h
E1117 19:38:15.411549 00001 quadstore.go:319] Couldn't execute horizon: sql: no rows in result set
2015-11-17 19:38:15.412148 I | api: starting Health API on port 6061.
2015-11-17 19:38:15.412261 I | api: starting API on port 6060.
2015-11-17 19:38:15.412458 I | updater: updater service started. lock identifier: 35f76bb1-cfa4-4682-816c-591be9ca52b8
2015-11-17 19:38:15.432099 I | updater: updating vulnerabilities
2015-11-17 19:38:15.432135 I | updater/fetchers: fetching Debian vulneratibilities
2015-11-17 19:38:15.432196 I | updater/fetchers: fetching Red Hat vulneratibilities
2015-11-17 19:38:15.432396 I | updater/fetchers: fetching Ubuntu vulneratibilities

Despite the error from quadstore.go:319, it does work.
Note that the container expects the database name is the same as the db user name, or you can pass in 'database=' in the connection string, which is undocumented.

Is it correct that, unlike 'bolt' db type, a pre-existing sql database is required to run the clair container?

This is great! Once I get it going I was able to insert layers for local images and get reports.

API: add POST based API

For playing around with clair it would be much nicer if I could POST the image for analysis instead of going through hoops, as analyze-local-images does, to pull from a web server. This is a local development nicety but I bet it would make it easier for people to participate. It certainly tripped me up while I tried to use analyze-local-images.

Namespace YAML Configuration

Our YAML configuration should all be under one top-level key such that the file can contain configuration for other projects in addition to Clair.

Also use OWASP Dependency-Check

Cool project. However, Linux-level package managers won't tell you about all the vulnerable components. Many programming languages use their own package management systems.

I suggest starting by integrating OWASP Dependency-Check:
https://www.owasp.org/index.php/OWASP_Dependency_Check

It supports analyzing Java, .NET, Ruby, Node.js, and Python projects, along with limited support for C/C++ source code using CMake or autoconf.

VOLUME /config cannot mount from local folder

When I run the container following : clair/docs/Run.md

Even with the config file in the folder and the correct -v option, my config file is not in the mounted container.

I think is due to the VOLUME /config in the Dockerfile.

I manage to make it running by docker cp config.yaml myClair:/config instead.

Updater always set the latest successful date

Because Update does not return any status, the latest successful update date is always updated even if it failed.

Also, we want to make sure that we don't update the flag of a failed fetcher. Today, fetchers only returns a flag if they succeeded but eventually contributors may start writing fetchers and don't respect that.

Split notifications

In a real environment with millions of layers, notifications can become pretty big and need to be splitted.

Refactor

The code needs serious global refactoring. Should probably start by creating some helpers in the database package.

Clair support for Oracle Linux

Are there any plans to expand the OS support for Oracle Linux. Scanning an OL image flags as unrecognized OS, as this is derived from a RedHat OS with an rpm based package manager would this be an easy fix?

Optimize the Fetching of Debian Security Bug Tracker

As it is now, each time that Debian vulnerabilities are updated, we upsert all of them - and it takes time. It would be great to diff the vulnerabilities and only upsert the ones that actually need to be upserted.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.