Giter Club home page Giter Club logo

clamav-rest's Introduction

Table of Contents

Introduction

This is two in one docker image so it runs open source virus scanner ClamAV (https://www.clamav.net/), automatic virus definition updates as background process and REST API interface to interact with ClamAV process.

Updates

As of May 2024, the releases are built for multiple architectures thanks to efforts from kcirtapfromspace and support non-root read-only deployments thanks to robaca.

The additional endpoint /version is now available to check the clamd version and signature date. Thanks pastral.

Closed a security hole by upgrading our Dockerfile to the alpine base image version 3.19 thanks to Marsup.

Prerequisites

This container doesn't do much on it's own unless you use an additional service or communicator to talk to it!

Installation

Automated builds of the image are available on Registry and is the recommended method of installation.

docker pull hub.docker.com/ajilaag/clamav-rest:(imagetag)

The following image tags are available:

  • latest - Most recent release of ClamAV with REST API
  • YYYYMMDD - The day of the release
  • sha-... - The git commit sha. This version ensures that the exact image is used and will be unique for each build

Quick Start

See this docker-compose file for non-root read-only usage.

Run clamav-rest docker image:

docker run -p 9000:9000 -p 9443:9443 -itd --name clamav-rest ajilaag/clamav-rest

Test that service detects common test virus signature:

HTTP

$ curl -i -F "[email protected]" http://localhost:9000/scan
HTTP/1.1 100 Continue

HTTP/1.1 406 Not Acceptable
Content-Type: application/json; charset=utf-8
Date: Mon, 28 Aug 2017 20:22:34 GMT
Content-Length: 56

{ "Status": "FOUND", "Description": "Eicar-Test-Signature" }

HTTPS

$ curl -i -k -F "[email protected]" https://localhost:9443/scan
HTTP/1.1 100 Continue

HTTP/1.1 406 Not Acceptable
Content-Type: application/json; charset=utf-8
Date: Mon, 28 Aug 2017 20:22:34 GMT
Content-Length: 56

{ "Status": "FOUND", "Description": "Eicar-Test-Signature" }

Test that service returns 200 for clean file:

HTTP

$ curl -i -F "[email protected]" http://localhost:9000/scan

HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Mon, 28 Aug 2017 20:23:16 GMT
Content-Length: 33

{ "Status": "OK", "Description": "" }

HTTPS

$ curl -i -k -F "[email protected]" https://localhost:9443/scan

HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Mon, 28 Aug 2017 20:23:16 GMT
Content-Length: 33

{ "Status": "OK", "Description": "" }

Status Codes

  • 200 - clean file = no KNOWN infections
  • 400 - ClamAV returned general error for file
  • 406 - INFECTED
  • 412 - unable to parse file
  • 501 - unknown request

Configuration

Environment Variables

Below is the complete list of available options that can be used to customize your installation.

Parameter Description
MAX_SCAN_SIZE Amount of data scanned for each file - Default 100M
MAX_FILE_SIZE Don't scan files larger than this size - Default 25M
MAX_RECURSION How many nested archives to scan - Default 16
MAX_FILES Number of files to scan withn archive - Default 10000
MAX_EMBEDDEDPE Maximum file size for embedded PE - Default 10M
MAX_HTMLNORMALIZE Maximum size of HTML to normalize - Default 10M
MAX_HTMLNOTAGS Maximum size of Normlized HTML File to scan- Default 2M
MAX_SCRIPTNORMALIZE Maximum size of a Script to normalize - Default 5M
MAX_ZIPTYPERCG Maximum size of ZIP to reanalyze type recognition - Default 1M
MAX_PARTITIONS How many partitions per Raw disk to scan - Default 50
MAX_ICONSPE How many Icons in PE to scan - Default 100
PCRE_MATCHLIMIT Maximum PCRE Match Calls - Default 100000
PCRE_RECMATCHLIMIT Maximum Recursive Match Calls to PCRE - Default 2000
SIGNATURE_CHECKS Check times per day for a new database signature. Must be between 1 and 50. - Default 2

Networking

Port Description
3310 ClamD Listening Port

Maintenance / Monitoring

Shell Access

For debugging and maintenance purposes you may want access the containers shell.

docker exec -it (whatever your container name is e.g. clamav-rest) /bin/sh

Prometheus

Prometheus metrics were implemented, which can be retrieved as follows

HTTP: curl http://localhost:9000/metrics

HTTPS: curl https://localhost:9443/metrics

Developing

Source Code can be found here: https://github.com/ajilach/clamav-rest

Build golang (linux) binary and docker image:

# env GOOS=linux GOARCH=amd64 go build
docker build . -t clamav-go-rest
docker run -p 9000:9000 -p 9443:9443 -itd --name clamav-rest clamav-go-rest

References

clamav-rest's People

Contributors

ajila-fmeyer avatar ajilaag avatar christianbumann avatar davosian avatar kcirtapfromspace avatar marsup avatar niilo avatar o20ne avatar osterzel avatar parichards avatar pastral avatar philippe-docourt avatar rahearn avatar renovate[bot] avatar rhermanek avatar robaca avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

clamav-rest's Issues

Scanned filename in clamd logs

Currently the clamd logs the following information with the filename missing:

Thu Oct 27 10:01:43 2022 -> instream(127.0.0.1@42164): Eicar-Signature FOUND

It would be good to add a feature to pass the name of the file that is submitted to the api, to clamav logs.
Not sure if it would make more sense to log the files on the http server level.

Thx!!!

Data directory

We should be able to specify where to put the database so we can cache it somewhere, for example on S3

Non-root support

We want to use the docker container in non-root mode with alternative uid/gid. It would be great if the docker image would support that. Currently it seems that it simply cannot apply env vars when started:

$ podman run --rm -it --user 100:101 ajilaag/clamav-rest:20240511

sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
sed: can't create temp file '/etc/clamav/clamd.confXXXXXX': Permission denied
Starting clamav rest bridge...

Start Antivirus Service than check for Virus updates

Starting the container in an environment without internet access doesn't starts the service.

It would be nice to have the service started, and then try to update the virus database...

Background: We want to run integration tests, and our development environment is completely air gapped (no internet). So the update server is not reachable and the service doesn't start. This can be a dangerous behavior.

HTTP/2 clear (h2c) support

Hi

is it possible that you can implement HTTP/2 clear (h2c) support for the clam-av rest? I deployed your Docker image in Google cloud run and hit the 32 MB file size limitation. That means you can't scan files for threads which are bigger than 32 MB. Regarding to the Cloud Run documentation, there is no file size limitation for HTTP/2.

A possible workaround could be to setup a reverse proxy (e.g nginx) in front of your API which can handle HTTP/2.

Sources:
https://cloud.google.com/run/quotas
https://cloud.google.com/run/docs/configuring/http2

Thanks
Umut.

Authentication

Hello,

It would be nice to allow the configuration / check of an ACCESS_TOKEN or BASIC Authentication to secure the access to the service.

Do you consider to add such option?

Thanks

Suggestion: wrap multipart scan response in json array brackets, include faulty status for missing filename

I have a suggestion on the response json from the multipart scanning endpoint, but it's a breaking change. Since it contains one response for each file in the request, it should be wrapped in brackets to form a json array instead of multiple json documents.

My second proposition is that the if statement containing the continue action for when the multipart header filename is missing, should create an entry in the response with either a parse error or a plain error stating that the filename is missing.

Please close this issue if you find my suggestions irrelevant and supply why they are not feasible for implementation. If you find these suggestions valid, I volunteer to fork the project and implement them and then create a PR.

Security updates needed

Various vulnerable versions of libclamav and Go libraries are in use, and the vulnerabilities are rated "High". I don't want to report the details here, but you can see them easily by logging into https://slim.ai, searching for ajilaag/clamav-rest, and clicking on Vulnerabilities. There you'll see the details in the Vulnerabilities and Packages sub-tabs. There are 3 and 8 entries on each tab respectively.

Bug: service does not return proper status when file size is greater than `MAX_FILE_SIZE` value

Description:
The current behavior of the ClamAV HTTP server is to display a warning message in the console when a file size exceeds the configured MAX_FILE_SIZE value, but it does not terminate the connection. This behavior is problematic as it allows oversized files to consume server resources and potentially disrupt the server's performance.

Снимок экрана 2023-06-07 в 17 07 44

Expected Behavior:
When a client uploads a file larger than the defined MAX_FILE_SIZE, the server should terminate the connection and provide a clear error message to the client, indicating that the file size exceeds the limit.

Additional Information:

ClamAV version: ClamAV 0.105.2/26926

Снимок экрана 2023-06-07 в 17 09 50

Feature: Support proxy in entrypoint for freshclam

Referencing an open item we would like to modify the entrypoint.sh to pass proxy information into freshclam.conf

Files to modify:
entrypoint.sh

Description:
Following the pattern present in entrypoint.sh we would like to add a conditional to support an environment variable being set of $PROXY_SERVER. If the environment variables are set for $PROXY_PORT that can also be set, along with $PROXY_USERNAME and $PROXY_PASSWORD for the following items in freshclam.conf:

#HTTPProxyServer https://proxy.example.com
#HTTPProxyPort 1234
#HTTPProxyUsername myusername
#HTTPProxyPassword mypass
if [ -n "$PROXY_SERVER" ]; then
    sed -i 's~^#HTTPProxyServer .*~HTTPProxyServer '"$PROXY_SERVER"'~g' /etc/clamav/freshclam.conf

    # It's not required, but if they also provided a port, then configure it
    if [ -n "$PROXY_PORT" ]; then
        sed -i 's/^#HTTPProxyPort .*$/HTTPProxyPort '"$PROXY_PORT"'/g' /etc/clamav/freshclam.conf
    fi

    # It's not required, but if they also provided a username, then configure both the username and password
    if [ -n "$PROXY_USERNAME" ]; then
        sed -i 's/^#HTTPProxyUsername .*$/HTTPProxyUsername '"$PROXY_USERNAME"'/g' /etc/clamav/freshclam.conf
        sed -i 's~^#HTTPProxyPassword .*~HTTPProxyPassword '"$PROXY_PASSWORD"'~g' /etc/clamav/freshclam.conf
    fi
fi

Freshclam warning about Clamd not being notified of database update

I noticed that I have this warning in /var/log/clamav/freshclam.log:
WARNING: Clamd was NOT notified: Can't connect to clamd through /run/clamav/clamd.sock: No such file or directory

Full log entry:

freshclam daemon 0.104.3 (OS: Linux, ARCH: x86_64, CPU: x86_64)
ClamAV update process started at Sun Aug 21 15:57:23 2022
daily database available for update (local version: 26615, remote version: 26634)
Testing database: '/var/lib/clamav/tmp.10358d49f6/clamav-444b2a996524c77c8813b0b1f33e689f.tmp-daily.cld' ...
Database test passed.
daily.cld updated (version: 26634, sigs: 1998006, f-level: 90, builder: raynman)
main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
WARNING: Clamd was NOT notified: Can't connect to clamd through /run/clamav/clamd.sock: No such file or directory
--------------------------------------

What does that mean for scans requested via the REST API?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

docker-compose
docker-compose-nonroot.yml
docker-compose.yml
dockerfile
Dockerfile
  • alpine 3.19
centos.Dockerfile
github-actions
.github/workflows/ci.yaml
  • actions/checkout v4
  • docker/metadata-action v5
  • docker/setup-qemu-action v3
  • docker/setup-buildx-action v3
  • docker/login-action v3
  • docker/build-push-action v5
  • actions/upload-artifact v4
  • actions/download-artifact v4
  • docker/setup-buildx-action v3
  • docker/metadata-action v5
  • mathieudutour/github-tag-action v6.2
  • ncipollo/release-action v1
  • docker/login-action v3

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.