Giter Club home page Giter Club logo

bulk-scan-processor's Introduction

Bulk scan processor

Codacy Badge codecov Known Vulnerabilities

Purpose

Retrieve scanned documents along with information extracted with OCR engine. Store the images and let recipient services fetch the new data.

Getting Started

Prerequisites

  • JDK 17
  • Project requires Spring Boot v3.x to be present

Installation

  • Clone the repository
  • Ensure all required environment variables have been set.

Quick Start (Alternative)

An alternative faster way getting started is by using the automated setup script. This script will help set up all bulk scan/print repos including bulk-scan-processor and its dependencies. See common-dev-env-bsbp repository for more information. Once set up script has ran successfully you can move the bulk-scan-processor from the newly created common-dev-env-bsbp/apps directory to your desired location.

Building and deploying the application

The project uses Gradle as a build tool. It already contains ./gradlew wrapper script, so there's no need to install gradle.

Building the application

To build the project execute the following command:

  ./gradlew build

Running the application

You will either need to add a .env file if one does not already exist, or add environment variables to your Application tasks configuration.

If you choose to add a .env file, you will need to add all the environment variables listed in the application.yaml file.

For example, ${BULK_SCANNING_DB_PASSWORD:} in the application.yaml file will need to be added as BULK_SCANNING_DB_PASSWORD="value from keyvault" in your .env file.

Create the image of the application by executing the following command:

  ./gradlew assemble

Application listens on port 8581 which can be overridden by setting SERVER_PORT environment variable or from .env file.

The application depends upon certain components that are already up and running. Configuration details for each component can be changed by passing values in environment variables:

PostgreSQL

  • BULK_SCANNING_DB_HOST
  • BULK_SCANNING_DB_PORT
  • BULK_SCANNING_DB_NAME
  • BULK_SCANNING_DB_USER_NAME
  • BULK_SCANNING_DB_PASSWORD

Azure Blob Storage

  • STORAGE_ACCOUNT_NAME
  • STORAGE_KEY
  • SAS_TOKEN_VALIDITY

Document Management Storage

  • DOCUMENT_MANAGEMENT_URL working endpoint URL

Service to Service Authentication

  • S2S_URL working endpoint URL
  • S2S_NAME service name
  • S2S_SECRET service secret

Please find more details in infrastructure/main.tf file.

Running smoke tests

Smoke tests expect an address of deployed application to be passed in TEST_URL environment variable. For example:

  TEST_URL=http://localhost:8561 ./gradlew smoke

By default, it will use http://localhost:8581 which is defined in src/smokeTest/resources/application.yaml.

Running integration tests

  ./gradlew integration

Migration

To run migration gradle task expects FLYWAY_URL to be present. In case db requires username/password: FLYWAY_USER and FLYWAY_PASSWORD. Once those variables are exported all flyway tasks are available.

./gradlew flywayMigrate

License

This project is licensed under the MIT License - see the LICENSE file for details

bulk-scan-processor's People

Contributors

adusumillipraveen avatar aliveni avatar alivenichoppa avatar amstevenson avatar apereverzin avatar banderous avatar dependabot-preview[bot] avatar dependabot-support avatar dependabot[bot] avatar doncem avatar ferdi-cocelli avatar gauravchawlamoj avatar hannah38 avatar jasonpaige avatar jordanhoey96 avatar joshblackmoor avatar justiceia avatar karoljastrzebski avatar lgonczar avatar luigibk avatar majid-justice avatar mokainos avatar nitinprabhu avatar renovate[bot] avatar rishikrsharma avatar rkondratowicz avatar ruthkirby avatar someshra avatar timja avatar vakepati avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bulk-scan-processor's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • Update dependency org.postgresql:postgresql to v42.7.4
  • Update dependency hashicorp/terraform to v1.9.5
  • Update dependency org.springdoc:springdoc-openapi-starter-webmvc-ui to v2.6.0
  • Update dependency org.wiremock:wiremock-standalone to v3.9.1
  • Update flyway to v10.18.0 (org.flywaydb:flyway-core, org.flywaydb:flyway-database-postgresql, org.flywaydb.flyway)
  • Update junit5 monorepo (org.junit.platform:junit-platform-engine, org.junit.platform:junit-platform-commons, org.junit.jupiter:junit-jupiter-params, org.junit.jupiter:junit-jupiter-engine, org.junit.jupiter:junit-jupiter-api)
  • Update mcr.microsoft.com/azure-storage/azurite Docker tag to v3.32.0
  • Update plugin au.com.dius.pact to v4.6.14
  • Update plugin org.owasp.dependencycheck to v9.2.0
  • Update plugin org.springframework.boot to v3.3.3
  • Update shedlock monorepo to v5.16.0 (net.javacrumbs.shedlock:shedlock-provider-jdbc, net.javacrumbs.shedlock:shedlock-spring)
  • Update testcontainers-java monorepo to v1.20.1 (org.testcontainers:junit-jupiter, org.testcontainers:postgresql)
  • Update versions.reformLogging to v6.1.6 (com.github.hmcts.java-logging:logging-appinsights, com.github.hmcts.java-logging:logging)
  • Update Terraform azurerm to v4
  • Update actions/cache action to v4
  • Update dependency com.icegreen:greenmail to v2
  • Update dependency javax.servlet:javax.servlet-api to v4
  • Update dependency org.springframework.cloud:spring-cloud-starter-bootstrap to v4
  • Update dependency pmd to v7
  • Update plugin org.owasp.dependencycheck to v10
  • Update plugin org.sonarqube to v5
  • ๐Ÿ” Create all rate-limited PRs at once ๐Ÿ”

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

docker-compose
docker-compose.yml
src/integrationTest/resources/docker-compose.yml
  • mcr.microsoft.com/azure-storage/azurite 3.29.0
dockerfile
Dockerfile
  • hmctspublic.azurecr.io/base/java 17-distroless
docker/database/Dockerfile
  • hmctspublic.azurecr.io/imported/postgres 16-alpine
github-actions
.github/workflows/ci.yml
  • actions/checkout v4
  • actions/cache v3
  • actions/setup-java v4
.github/workflows/publish-openapi.yaml
  • hmcts/workflow-publish-openapi-spec v1
gradle
settings.gradle
build.gradle
  • org.postgresql:postgresql 42.7.3
  • org.flywaydb:flyway-database-postgresql 10.13.0
  • io.spring.dependency-management 1.1.6
  • org.flywaydb.flyway 10.13.0
  • org.springframework.boot 3.2.7
  • org.owasp.dependencycheck 9.1.0
  • com.github.ben-manes.versions 0.51.0
  • org.sonarqube 4.4.1.3373
  • info.solidsoft.pitest 1.15.0
  • au.com.dius.pact 4.3.14
  • checkstyle 9.3
  • pmd 6.55.0
  • org.junit.jupiter:junit-jupiter-api 5.9.2
  • org.junit.jupiter:junit-jupiter-engine 5.9.2
  • org.junit.jupiter:junit-jupiter-params 5.9.2
  • org.junit.platform:junit-platform-commons 1.9.2
  • org.junit.platform:junit-platform-engine 1.9.2
  • org.apiguardian:apiguardian-api 1.1.2
  • org.flywaydb:flyway-database-postgresql 10.13.0
  • net.javacrumbs.shedlock:shedlock-spring 5.14.0
  • net.javacrumbs.shedlock:shedlock-provider-jdbc 5.14.0
  • com.azure:azure-storage-blob 12.25.4
  • com.azure:azure-messaging-servicebus 7.14.7
  • org.springframework.cloud:spring-cloud-starter-openfeign 4.1.0
  • io.github.openfeign:feign-httpclient 13.3
  • io.github.openfeign:feign-jackson 13.3
  • com.github.java-json-tools:json-schema-validator 2.2.14
  • org.flywaydb:flyway-core 10.13.0
  • org.postgresql:postgresql 42.7.3
  • io.hypersistence:hypersistence-utils-hibernate-63 3.8.2
  • org.apache.httpcomponents.client5:httpclient5 5.3.1
  • javax.jms:javax.jms-api 2.0.1
  • javax.servlet:javax.servlet-api 3.1.0
  • org.springdoc:springdoc-openapi-starter-webmvc-ui 2.3.0
  • com.github.hmcts.java-logging:logging 6.0.1
  • com.github.hmcts.java-logging:logging-appinsights 6.0.1
  • net.logstash.logback:logstash-logback-encoder 8.0
  • com.github.hmcts:service-auth-provider-java-client 4.1.2
  • com.github.hmcts:idam-java-client 3.0.3
  • com.github.hmcts:ccd-case-document-am-client 1.59
  • org.springframework.cloud:spring-cloud-starter-bootstrap 3.1.8
  • org.apache.commons:commons-lang3 3.14.0
  • commons-io:commons-io 2.16.1
  • io.vavr:vavr 0.10.4
  • org.apache.commons:commons-csv 1.11.0
  • com.github.ben-manes.caffeine:caffeine 3.1.8
  • io.github.resilience4j:resilience4j-spring-boot2 2.2.0
  • com.launchdarkly:launchdarkly-java-server-sdk 7.5.0
  • com.google.guava:guava 33.2.1-jre
  • com.jayway.awaitility:awaitility 1.7.0
  • com.icegreen:greenmail 1.6.15
  • org.apache.commons:commons-email 1.5
  • io.github.netmikey.logunit:logunit-core 2.0.0
  • com.github.hmcts:fortify-client 1.4.3
  • com.icegreen:greenmail 2.1.0-rc-1
  • io.github.netmikey.logunit:logunit-logback 2.0.0
  • org.springframework.cloud:spring-cloud-contract-wiremock 4.1.1
  • org.wiremock:wiremock-standalone 3.4.2
  • org.testcontainers:postgresql 1.19.7
  • org.testcontainers:junit-jupiter 1.19.7
  • com.revinate:assertj-json 1.2.0
  • com.typesafe:config 1.4.3
  • au.com.dius:pact-jvm-consumer-junit5 4.0.10
gradle-wrapper
gradle/wrapper/gradle-wrapper.properties
  • gradle 7.6.4
helm-values
charts/bulk-scan-processor/values.yaml
helmv3
charts/bulk-scan-processor/Chart.yaml
  • java 5.2.0
  • servicebus 1.0.6
  • blobstorage 2.0.1
terraform
infrastructure/provider.tf
infrastructure/state.tf
  • azuread >= 2.38.0
  • azurerm 3.106.0
infrastructure/tf-db-flexible.tf
infrastructure/versions.tf
  • hashicorp/terraform >= 0.13
terraform-version
infrastructure/.terraform-version
  • hashicorp/terraform 1.6.6

  • Check this box to trigger a request for Renovate to run again on this repository

Use of mutation testing in bulk-scan-processor - Help needed

Hello there!

My name is Ana. I noted that you use the mutation testing tool Pit in the project.
I am a postdoctoral researcher at the University of Seville (Spain), and my colleagues and I are studying how mutation testing tools are used in practice. With this aim in mind, we have analysed over 3,500 public GitHub repositories using mutation testing tools, including yours! This work has recently been published in a journal paper available at https://link.springer.com/content/pdf/10.1007/s10664-022-10177-8.pdf.

To complete this study, we are asking for your help to understand better how mutation testing is used in practice, please! We would be extremely grateful if you could contribute to this study by answering a brief survey of 21 simple questions (no more than 6 minutes). This is the link to the questionnaire https://forms.gle/FvXNrimWAsJYC1zB9.

We apologize if you have already received message multiple times or if you have already had the opportunity to complete the survey. If you have already shared your feedback, we want to convey our appreciation, kindly disregard this message, and please accept our apologies for any inconvenience.

Drop me an e-mail if you have any questions or comments ([email protected]). Thank you very much in advance!!

Dependabot couldn't find a pom.xml for this project

Dependabot couldn't find a pom.xml for this project.

Dependabot requires a pom.xml to evaluate your project's current Java dependencies. It had expected to find one at the path: /pom.xml.

If this isn't a Java project, or if it is a library, you may wish to disable updates for it from within Dependabot.

You can mention @dependabot in the comments below to contact the Dependabot team.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.