Giter Club home page Giter Club logo

blob-router-service's Introduction

Blob Router

codecov Codacy Badge

Table of Contents

Purpose

Primary responsibility of this micro service is to retrieve blobs from source blob storage containers and then dispatch them to destination blob storage containers based on source containers. Currently it only routes blobs (zip files) to CFT and Crime blob storage containers. Blobs are uploaded to source storage by the third party scanning supplier. Before dispatching blobs it verifies if the files were uploaded by third party supplier through non repudiation checks. It also provides infrastructure to deploy API management service for retrieving SAS tokens used to upload blobs to source containers.

Building and deploying the application

Building the application

The project uses Gradle as a build tool. It already contains ./gradlew wrapper script, so there's no need to install gradle.

To build the project execute the following command:

  ./gradlew build

Running the application

Create the image of the application by executing the following command:

  ./gradlew assemble

Create docker image:

  docker-compose build

Run the distribution (created in build/install/blob-router-service directory) by executing the following command:

  docker-compose up

This will start the API container exposing the application's port (set to 8584 in this template app).

In order to test if the application is up, you can call its health endpoint:

  curl http://localhost:8584/health

You should get a response similar to this:

  {"status":"UP","diskSpace":{"status":"UP","total":249644974080,"free":137188298752,"threshold":10485760}}

Quick Start

An alternative faster way getting started is by using the automated setup script. This script will help set up all bulk scan/print repos including blob-router-service and its dependencies. See the common-dev-env-bsbp repository for more information. Once set up script has ran successfully you can move the blob-router-service from the newly created common-dev-env-bsbp/apps directory to your desired location.

API gateway

Blob Router uses an (Azure API Management) API to protect its SAS token dispensing endpoint. The API allows only HTTPS requests with approved client certificates and valid subscription keys to reach the service.

Azure API Management is based on public swagger specs. As part of creating API in there documentation had to be published. The full url to documentation can be found here.

If SAS dispensing endpoint has changed in some incompatible way which causes amended specs - the management needs to be notified. This means tiny alteration in terraform file.

In case any new endpoint needs to be included - same treatment must be applied.

Calling the API

In order to talk to the SAS dispensing endpoint through the API, you need to have the following pieces of information:

  • a certificate whose thumbprint is known to the API (has to be added to the list of allowed thumbprints in var.allowed_client_certificate_thumbprints terraform variable)
  • a valid subscription key
  • name of an existing client service (e.g. test)

Preparing client certificate

First, generate client private key, a certificate for that key and import both into a key store:

# generate private key
openssl genrsa 2048 > private.pem

# generate certificate
openssl req -x509 -new -key private.pem -out cert.pem -days 365

# create the key store
# when asked for password, provide one
openssl pkcs12 -export -in cert.pem -inkey private.pem -out cert.pfx -noiter -nomaciter

Next, calculate the thumbprint of your certificate:

openssl x509 -noout -fingerprint -inform pem -in cert.pem | sed -e s/://g

Add this thumbprint to allowed_client_certificate_thumbprints terraform variable for the target environment (e.g. in aat.tfvars file). Your definition may look similar to this:

allowed_client_certificate_thumbprints = ["2FC66765E63BB2436F0F9E4F59E951A6D1D20D43"]

Once you're run the deployment, the API will recognise your certificate.

Retrieving subscription key

You can get the subscription key for the API from the reform-scan-{env} key vault. The available subscription keys in the key vault are:

  • bulk-scan-team-cft-apim-subscription-key
  • exela-cft-apim-subscription-key (supplier key)

Getting the token through the API

You can call the API using the following curl command (assuming your current directory contains the private key and certificate you've created earlier):

curl -v --key private.pem --cert cert.pem https://cft-mtls-api-mgmt-appgw.{env}.platform.hmcts.net/reform-scan/token/{serviceName} -H "Ocp-Apim-Subscription-Key:{subscription key}"

You should get a response with status 200 and a token in the body.

License

This project is licensed under the MIT License - see the LICENSE file for details

blob-router-service's People

Contributors

alivenichoppa avatar amstevenson avatar apereverzin avatar btensay avatar dependabot-preview[bot] avatar dependabot[bot] avatar doncem avatar gauravchawlamoj avatar jasonpaige avatar joshblackmoor avatar justiceia avatar lgonczar avatar mokainos avatar nitinprabhu avatar renovate[bot] avatar rishikrsharma avatar rkondratowicz avatar ruthkirby avatar timja avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

blob-router-service's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • Update dependency org.postgresql:postgresql to v42.7.4
  • Update Terraform azurerm to v3.116.0
  • Update dependency com.azure:azure-storage-blob to v12.27.1
  • Update dependency hashicorp/terraform to v1.9.5
  • Update dependency org.apache.commons:commons-lang3 to v3.17.0
  • Update dependency org.springdoc:springdoc-openapi-starter-webmvc-ui to v2.6.0
  • Update flyway (org.flywaydb:flyway-database-postgresql, org.flywaydb:flyway-core, org.flywaydb.flyway)
  • Update hmctspublic.azurecr.io/imported/azure-storage/azurite Docker tag to v3.31.0
  • Update plugin org.owasp.dependencycheck to v9.2.0
  • Update plugin org.springframework.boot to v3.3.3
  • Update shedlock monorepo to v5.16.0 (net.javacrumbs.shedlock:shedlock-provider-jdbc-template, net.javacrumbs.shedlock:shedlock-spring)
  • Update testcontainers-java monorepo to v1.20.1 (org.testcontainers:junit-jupiter, org.testcontainers:postgresql)
  • Update Terraform azurerm to v4
  • Update actions/cache action to v4
  • Update dependency javax.servlet:javax.servlet-api to v4
  • Update dependency pmd to v7
  • Update plugin org.owasp.dependencycheck to v10
  • Update plugin org.sonarqube to v5
  • ๐Ÿ” Create all rate-limited PRs at once ๐Ÿ”

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

docker-compose
docker-compose.yml
src/integrationTest/resources/docker-compose.yml
  • hmctspublic.azurecr.io/imported/azure-storage/azurite 3.29.0
dockerfile
Dockerfile
  • hmctspublic.azurecr.io/base/java 17-distroless
docker/database/Dockerfile
  • hmctspublic.azurecr.io/imported/postgres 16-alpine
docker/storage/Dockerfile
github-actions
.github/workflows/ci.yml
  • actions/checkout v4
  • actions/cache v3
  • actions/setup-java v4
.github/workflows/publish-openapi.yaml
  • hmcts/workflow-publish-openapi-spec v1
gradle
build.gradle
  • org.postgresql:postgresql 42.7.3
  • org.flywaydb:flyway-database-postgresql 10.13.0
  • io.spring.dependency-management 1.1.6
  • org.springframework.boot 3.2.7
  • org.owasp.dependencycheck 9.0.10
  • com.github.ben-manes.versions 0.51.0
  • org.sonarqube 4.4.1.3373
  • org.flywaydb.flyway 10.13.0
  • checkstyle 9.3
  • pmd 6.55.0
  • org.junit.jupiter:junit-jupiter-api 5.10.1
  • org.junit.jupiter:junit-jupiter-engine 5.10.1
  • org.junit.jupiter:junit-jupiter-params 5.10.1
  • org.junit.platform:junit-platform-commons 1.10.1
  • org.junit.platform:junit-platform-engine 1.10.1
  • org.apiguardian:apiguardian-api 1.1.2
  • org.postgresql:postgresql 42.7.3
  • org.flywaydb:flyway-core 10.13.0
  • net.javacrumbs.shedlock:shedlock-spring 5.14.0
  • net.javacrumbs.shedlock:shedlock-provider-jdbc-template 5.14.0
  • com.azure:azure-storage-blob 12.25.0
  • com.azure:azure-messaging-servicebus 7.17.2
  • org.apache.qpid:qpid-jms-client 1.11.0
  • org.springframework.cloud:spring-cloud-starter-bootstrap 4.1.3
  • org.springframework.cloud:spring-cloud-starter-openfeign 4.1.2
  • io.github.openfeign:feign-httpclient 13.3
  • io.github.openfeign:feign-jackson 13.3
  • org.apache.httpcomponents.client5:httpclient5 5.3.1
  • javax.jms:javax.jms-api 2.0.1
  • javax.servlet:javax.servlet-api 3.1.0
  • org.springdoc:springdoc-openapi-starter-webmvc-ui 2.5.0
  • com.github.ben-manes.caffeine:caffeine 3.1.8
  • com.github.hmcts.java-logging:logging 6.1.4
  • com.github.hmcts.java-logging:logging-appinsights 6.1.4
  • net.logstash.logback:logstash-logback-encoder 8.0
  • com.github.hmcts:service-auth-provider-java-client 4.1.2
  • org.apache.commons:commons-csv 1.11.0
  • org.apache.commons:commons-lang3 3.14.0
  • com.google.guava:guava 33.2.1-jre
  • com.github.hmcts:fortify-client 1.4.3
  • org.springframework.cloud:spring-cloud-contract-wiremock 4.1.3
  • org.testcontainers:postgresql 1.19.3
  • org.testcontainers:junit-jupiter 1.19.3
  • com.icegreen:greenmail 2.1.0-rc-1
  • org.assertj:assertj-core 3.24.2
  • com.jayway.awaitility:awaitility 1.7.0
  • com.typesafe:config 1.4.3
  • org.assertj:assertj-core 3.24.2
  • com.typesafe:config 1.4.3
  • org.flywaydb:flyway-database-postgresql 10.13.0
gradle-wrapper
gradle/wrapper/gradle-wrapper.properties
  • gradle 7.6.4
helm-values
charts/reform-scan-blob-router/values.yaml
helmv3
charts/reform-scan-blob-router/Chart.yaml
  • java 5.2.0
  • servicebus 1.0.6
  • blobstorage 2.0.1
  • blobstorage 2.0.1
  • blobstorage 2.0.1
terraform
infrastructure/cft-api-mgmt.tf
infrastructure/main.tf
infrastructure/state.tf
  • azuread 2.48.0
  • azurerm 3.106.0
infrastructure/tf-db-flexible.tf
infrastructure/versions.tf
  • hashicorp/terraform >= 0.13
terraform-version
infrastructure/.terraform-version
  • hashicorp/terraform 1.7.3

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.