Giter Club home page Giter Club logo

cyclonedx-bom-repo-server's Introduction

Test CI Docker Image License Website Slack Invite Group Discussion Twitter

CycloneDX BOM Repository Server

A BOM repository server for distributing CycloneDX BOMs.

You can test it out locally using Docker by running:

docker run --env ALLOWEDMETHODS__GET="true" --env ALLOWEDMETHODS__POST="true" --env ALLOWEDMETHODS__DELETE="true" --tty --interactive -p 8000:8080 cyclonedx/cyclonedx-bom-repo-server

Or, if you would like to persist BOM repository storage between runs by using local folder "bomstorage":

mkdir bomstorage
docker run --volume "$(pwd)/bomstorage":/repo --env ALLOWEDMETHODS__GET="true" --env ALLOWEDMETHODS__POST="true" --env ALLOWEDMETHODS__DELETE="true" --tty --interactive -p 8000:8080 cyclonedx/cyclonedx-bom-repo-server

then hit "http://localhost:8000/swagger" from browser

To build your own docker image

git clone github.com:CycloneDX/cyclonedx-bom-repo-server.git
cd cyclonedx-bom-repo-server
./build-and-run.sh

API Endpoints

The server supports Swagger/Open API Specification.

The JSON endpoint is /swagger/v1/swagger.json. The UI can be accessed at /swagger.

A summary of the available endpoints and methods are below:

Path HTTP Method Required Parameters Optional Parameters Description
/v1/bom GET serialNumber version, original If only the serialNumber parameter is supplied, retrieve the latest version of the BOM from the repository. If providing serialNumber and version, a specific version of the BOM will be retrieved. Supports HTTP content negotiation for all CycloneDX BOM formats and versions. If original is true, returns the original, unmodified BOM.
/v1/bom POST BOM content in request body and appropriate Content-Type header Adds a new BOM to the repository. Supports all CycloneDX BOM formats and versions. If the submitted BOM does not have a serial number, one will be generated. If the BOM does not have a version the next version number will be added. The response will contain an appropriate Location header to reference the BOM in the repository.
/v1/bom DELETE serialNumber version If only the serialNumber parameter is supplied, all versions of the BOM will be deleted from the repository. If serialNumber and version are supplied, only the specific version will be deleted from the repository.
/v1/bomexchange GET bomIdentifier Input valid serial number UUID URN or CDX URN to retrieve BOM.
/v1/bomexchange POST BOM content in request body and appropriate Content-Type header Adds a new BOM to the repository. Supports all CycloneDX BOM formats and versions. If the submitted BOM does not have a serial number, one will be generated. If the BOM does not have a version the next version number will be added. The response will contain an appropriate Location header to reference the BOM in the repository.
/v1/search GET One of group, name, version rest of group, name, version Retrieve a list of BOM serial numbers and versions that match the supplied metadata component search criteria.

NOTE: BOM serial numbers should be unique for a particular device/software version. When updating an existing BOM for the same software version, the BOM serial number should remain the same, and the version number should be incremented. For this reason, updating an existing BOM version is not supported. There is, of course, nothing to prevent deleting an existing BOM version and re-publishing it with the same serial number and version. But this is not recommended.

Example cURL Usage

Retrieving a BOM from the repository

curl -X GET 'http://localhost:8000/v1/bom?serialNumber=urn%3Auuid%3A3e671687-395b-41f5-a30f-a58921a69b79' -H 'accept: application/vnd.cyclonedx+json; version=1.4'

Adding a new BOM to the repository

curl -X POST "http://localhost:8000/v1/bom" -H  "accept: */*" -H  "Content-Type: application/vnd.cyclonedx+json; version=1.4" -d "{\"bomFormat\":\"CycloneDX\",\"specVersion\":\"1.3\",\"serialNumber\":\"urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79\",\"version\":1,\"components\":[{\"type\":\"library\",\"name\":\"acme-library\",\"version\":\"1.0.0\"}]}"

Deleting a BOM from the repository

curl -X DELETE 'http://localhost:8000/v1/bom/serialNumber=urn%3Auuid%3A3e671687-395b-41f5-a30f-a58921a69b79?version=1' -H 'accept: */*'

Configuration

The server can be configured by changing the src/CycloneDX.BomRepoServer/appsettings.json file or setting the following environment variables

Environment Variable Name Supported Values Description Default Value
REPO__DIRECTORY Any valid directory path The directory BOMs are stored Repo (set to \repo within the official Container since tbd)
ALLOWEDMETHODS__GET true or false Allows or forbids BOM retrieval false
ALLOWEDMETHODS__POST true or false Allows or forbids BOM creation false
ALLOWEDMETHODS__DELETE true or false Allows or forbids BOM deletion false
RETENTION__MAXBOMVERSIONS Any integer value >= 0 The maximum number of BOM versions that will be retained. If zero, the number of BOM versions are ignored for retention. 0
RETENTION__MAXBOMAGE Any integer value >= 0 The maximum age of a BOM, in days, before it is removed. If zero, the BOM age is ignored for retention. 0

Storage

The server storage type is configured under the Repo section. Currently we support the storage types FileSystem and S3 with options varying depending on the type.

...
"Repo": {
  "StorageType": "FileSystem",
  "Options": {
    "Directory": "Repo"
  }
}
...
...
"Repo": {
    "StorageType": "S3",
    "Options": {
      "Endpoint": "localhost:9000",
      "AccessKey": "bomserver-minioadmin",
      "SecretKey": "bomserver-minioadmin",
      "ForcePathStyle": true,
      "UseHttp": false,
      "BucketName": "bomserver"
  }
}
...

Authentication and Authorization

Authentication and authorization are expected to be configured at the web server or API gateway level.

For simplicity of deployment the allowed methods can be configured, and default to safe options (everything is forbidden by default).

It is recommended to deploy two instances of the BOM repository server.

One, requiring authentication and additional security controls, with GET and POST methods permitted to support publishing BOMs.

And a second instance, public facing, with only the GET method enabled. And authentication configured if required.

More advanced authentication and authorization use cases should be handled with a solution like an API gateway. And are considered out of scope of this project.

NOTE: It is recommended, subject to your operational environment and risk profile, to not require authentication for public facing instances. This enables downstream consumers, who might not have a direct commercial arrangement with your organization, to retrieve BOMs.

System Requirements

The CycloneDX BOM Repository Server has been designed as a lightweight, high performance, BOM repository server. Any production web server should be capable of running it.

However, there is an in memory cache of BOM metadata. Memory requirements will differ based on the amount of BOM metadata that requires caching.

All BOMs are converted to Protocol Buffer format before storage for efficiency.

As an alternative to the Docker image, the server can be hosted using Nginx, Apache, IIS, or Kestrel. More information can be found in the Web server implementations in ASP.NET Core documentation.

High Availability Deployments

The server supports sharing repository storage between multiple frontend instances. Which can be used for full active/active high availability clustering.

When deploying to multiple data centres it is recommended to have one master instance that supports publishing BOMs. And use data replication to any other target data centres used for distributing BOMs.

BOM Metadata Cache & Searching

To support high performance searching, there is an in memory cache of BOM metadata.

This cache is initially populated during startup and updated by a background thread every 10 minutes. For this reason, a newly published BOM may not immediately be returned in search results.

cyclonedx-bom-repo-server's People

Contributors

coderpatros avatar dependabot[bot] avatar k3rnelpan1c-dev avatar magnusp avatar zdtsw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

cyclonedx-bom-repo-server's Issues

Use Docker tooling instead of custom build script

Currently, the README advises users to run the build-and-run.sh script to get started.

However, this script duplicates functionality (in an indirect manner) that is already native to modern versions of Docker.

By using Docker tooling, this eliminates the onus on maintainers to update the build script when Dockerfile and docker-compose.yml already satisfy that need.

This can be achieved by rolling the environment variable, volume mapping, and port settings into the Docker Compose configuration.

Pending PR #380 illustrates this.

/cc @jkowalleck

Retain original unaltered BOM

Especially for XML BOMs, there can be additional information, like extensions and signatures, that are stripped when serializing/deserializing using the core spec data models. This should likely have a separate endpoint to indicate that you are retrieving the original, unaltered BOM.

Add support for BOM publishing key

The first time a serial number is published a publishing key should be optionally generated.

Intention is to support a public BOM repo server and provide a simple mechanism with which to control updates to existing BOMs.

Unable to save bom on filesystem

Start server using script build-and-run.sh (main branch)

on calling the sample Curl POST example, it does not save BOM into the filesystem

show warning,

warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
Failed to determine the https port for redirect.

logs
Sending build context to Docker daemon 664.1kB
Step 1/17 : FROM mcr.microsoft.com/dotnet/sdk:6.0 AS builder
---> d3863aa157b5
Step 2/17 : ARG APP_VERSION=0.0.0
---> Using cache
---> 5f06313c8724
Step 3/17 : COPY . /app
---> Using cache
---> 1597d1aed1a8
Step 4/17 : RUN cd /app && mkdir /app/bin && dotnet publish src/CycloneDX.BomRepoServer/CycloneDX.BomRepoServer.csproj --nologo --configuration Release --output bin --no-self-contained -p:Version=${APP_VERSION}
---> Using cache
---> c2b90f7e9b81
Step 5/17 : FROM mcr.microsoft.com/dotnet/aspnet:6.0
---> 70f39e2150e1
Step 6/17 : ENV TZ=Etc/UTC LANG=C.UTF-8 REPO__DIRECTORY=/repo ASPNETCORE_URLS=http://+:8080
---> Using cache
---> 94001d3d93ed
Step 7/17 : ARG APP_VERSION=0.0.0
---> Using cache
---> 05f395defeb9
Step 8/17 : ARG COMMIT_SHA=unknowen
---> Using cache
---> dc8d50c8b74a
Step 9/17 : ARG UID=1001
---> Using cache
---> cb66b8c21562
Step 10/17 : ARG GID=1001
---> Using cache
---> fcee95ea80f6
Step 11/17 : COPY --from=builder /app/bin /cyclonedx
---> Using cache
---> 50dfcc17c953
Step 12/17 : RUN mkdir -p -m 770 ${REPO__DIRECTORY} && addgroup --system --gid ${GID} cyclonedx || true && adduser --system --disabled-login --ingroup cyclonedx --no-create-home --home /nonexistent --gecos "cyclonedx user" --shell /bin/false --uid ${UID} cyclonedx || true && chown -R cyclonedx:0 ${REPO__DIRECTORY} /cyclonedx && chmod -R g=u ${REPO__DIRECTORY} /cyclonedx
---> Using cache
---> 639d843b9fca
Step 13/17 : USER ${UID}
---> Using cache
---> 6d745f2c0ac9
Step 14/17 : WORKDIR /cyclonedx
---> Using cache
---> 21540b8da67c
Step 15/17 : ENTRYPOINT [ "/cyclonedx/CycloneDX.BomRepoServer" ]
---> Using cache
---> c374b9c053c2
Step 16/17 : EXPOSE 8080
---> Using cache
---> 3a225d11f558
Step 17/17 : LABEL org.opencontainers.image.vendor="CycloneDX" org.opencontainers.image.title="Official CycloneDX BOM Repository Server Container image" org.opencontainers.image.description="CycloneDX BOM Repository Server is a BOM repository server for distributing CycloneDX BOMs" org.opencontainers.image.version="${APP_VERSION}" org.opencontainers.image.url="https://cyclonedx.org/" org.opencontainers.image.source="https://github.com/CycloneDX/cyclonedx-bom-repo-server" org.opencontainers.image.revision="${COMMIT_SHA}" org.opencontainers.image.licenses="Apache-2.0"
---> Using cache
---> 38e5b00739b0
Successfully built 38e5b00739b0
Successfully tagged localhost/cyclonedx-bom-repo-server:latest
info: CycloneDX.BomRepoServer.Services.CacheUpdateBackgroundService[0]
Updating BOM cache...
info: CycloneDX.BomRepoServer.Services.RetentionBackgroundService[0]
Updating BOM cache...
info: Microsoft.Hosting.Lifetime[14]
Now listening on: http://[::]:8080
info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
Content root path: /cyclonedx
warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
Failed to determine the https port for redirect.
info: CycloneDX.BomRepoServer.Services.CacheUpdateBackgroundService[0]
Updating BOM cache...

Spec 1.5 is not supported

I'm trying to deploy this repo server and I'm facing an issue. I currently create the Bom file using trivy in my pipeline, and then I run this to add it to the repo:

curl -v -X POST -H "Content-Type:application/json" -d @trivy_report.json http://10.63.28.54:80/v1/bomexchange

I've tried both /v1/bom and /v1/bomexchange, as both seem to do the same thing according to the documentation, and I get the same error in both of them:

fail: Microsoft.AspNetCore.Server.Kestrel[13]
Connection id "0HN33G40CVVP8", Request id "0HN33G40CVVP8:00000002": An unhandled exception was thrown by the application.
System.ArgumentException: Unsupported specification version: 1.5
at CycloneDX.Models.Bom.set_SpecVersionString(String value)
at System.Text.Json.Serialization.Metadata.JsonPropertyInfo1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader) at System.Text.Json.Serialization.Converters.ObjectDefaultConverter1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
at System.Text.Json.Serialization.JsonConverter1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) at System.Text.Json.Serialization.JsonConverter1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable1 actualByteCount)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan1 json, JsonTypeInfo jsonTypeInfo) at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options) at CycloneDX.Json.Serializer.Deserialize(String jsonString) at CycloneDX.BomRepoServer.Controllers.BomExchangeController.Post() in /app/src/CycloneDX.BomRepoServer/Controllers/BomExchangeController.cs:line 151 at lambda_method21(Closure , Object ) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask1 actionResultValueTask)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

I've found this issue that shows the same error in cyclonedx-cli. Is this an issue in this case as well?

Add support for webhooks

We should add support for configurable webhooks. This would support a lot of automation use cases.

First version would just be a BOM or BOM version has been uploaded. With the webhook payload perhaps constrained to top level BOM information and metadata.

Add Support for Azure Storage

Currently, the supported storage types are FileSystem and S3. It would be useful to extend support to include Azure Storage.

Does someone maintain this project

Hi there,
I am looking for a bom storage solution. I like this small server but i am afraid because the latest commit was 1 year ago. Can someone tell me if this project is alive ?

Regards

Unable to start server: Function not implemented

I'm unable to start the bom-repo-server. I'm testing on an Apple M1 Pro, which can run x86_64 containers under emulation. It works, but its slow. Not sure if platform is related to this error or not though. Here's the stack trace when starting the Docker container:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Unhandled exception. System.IO.IOException: Function not implemented
   at System.IO.FileSystemWatcher.StartRaisingEvents()
   at System.IO.FileSystemWatcher.StartRaisingEventsIfNotDisposed()
   at System.IO.FileSystemWatcher.set_EnableRaisingEvents(Boolean value)
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.TryEnableFileSystemWatcher()
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.CreateFileChangeToken(String filter)
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider.Watch(String filter)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.<.ctor>b__1_0()
   at Microsoft.Extensions.Primitives.ChangeToken.OnChange(Func`1 changeTokenProducer, Action changeTokenConsumer)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider..ctor(FileConfigurationSource source)
   at Microsoft.Extensions.Configuration.Json.JsonConfigurationSource.Build(IConfigurationBuilder builder)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at Microsoft.Extensions.Hosting.HostBuilder.BuildAppConfiguration()
   at Microsoft.Extensions.Hosting.HostBuilder.Build()
   at CycloneDX.BomRepoServer.Program.Main(String[] args) in /home/runner/work/cyclonedx-bom-repo-server/cyclonedx-bom-repo-server/CycloneDX.BomRepoServer/Program.cs:line 33
qemu: uncaught target signal 6 (Aborted) - core dumped

I have attempted to start the container with and without the environment variables, with the same result.

  • REPO__DIRECTORY
  • ALLOWEDMETHODS__GET
  • ALLOWEDMETHODS__POST
  • ALLOWEDMETHODS__DELETE

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.