Giter Club home page Giter Club logo

projettelecomjava's Introduction

BatchProcessor

GitHub Actions Workflow Status

Introduction

This project is a result of a project for my school. The goal was to create a batch processor that monitors a directory for new CSV files, parses the files to extract user data, and inserts the data into a database. Requirements can be found here

Table of Contents

  1. Architecture
  2. Setup
  3. Usage
  4. Technical Choices
  5. Class Details
  6. Error Handling
  7. Testing
  8. License

Architecture

The application is composed of the following components:

  • Main Class (Program): Entry point of the application.
  • FileWatcher: Monitors a directory for new files.
  • CSVParser: Parses CSV files to extract user data.
  • DatabaseManager: Handles database operations (like create connection).
  • Repository (UserRepository): Handles database operations (like insert in batch).
  • FileMover: Moves processed files to another directory.
  • Models (Reimbursement): Represents the data structure for the user.

Setup

Prerequisites

  • Java Development Kit (JDK) 21 or higher
  • Maven (for dependency management)
  • MariaDB

Configuration

A configuration file (application.properties) is present in resources directory with the following content:

# Database connection properties
db.uri=jdbc:mariadb://localhost:3306/project_telecom_java
db.username=root
db.password=example

Building the Project

Clone the repository and build the project using Maven:

git clone https://github.com/Swansky/ProjetTelecomJava.git
cd ProjetTelecomJava
mvn clean install

Technical Choices

  1. Maven: Used for dependency management and build automation.
  2. WatchService API: Utilized to monitor the file system for new files.
  3. MariaDB: Used because it's open-source and easy to set up and because H2 database for test have a mariadb mode.
  4. OpenCSV: Used for parsing CSV files.

Class Details

Main Class (Program)

Orchestrates the overall workflow by initializing components and handling the sequence of operations.

FileWatcher

Monitors the specified directory for new CSV files using Java's WatchService.

CSVParser

Reads and parses CSV files to extract user data into Reimbursement objects.

DatabaseManager

Handles database operations such as create connection.

FileMover

Moves processed files to a specified directory.

Models (User)

Represents a user record with fields corresponding to the CSV data and database schema.

Error Handling

The application includes basic error handling mechanisms such as try-catch blocks to catch and handle exceptions. It's essential to handle errors gracefully to avoid crashing the application and to log errors for debugging and maintenance purposes.

Logging

I use build-in logging system of Java (java.util.logging) to log errors and information messages.

Testing

To ensure the robustness and reliability of the application, it's crucial to write unit tests and integration tests. Using testing frameworks like JUnit can help automate and streamline the testing process.

Unit Tests

Some unit tests are written to test individual components of the application in isolation. For the database, I use H2 database in memory to test the repository.

License

This project is licensed under the MIT License. See the LICENSE file for details.

projettelecomjava's People

Contributors

swansky avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.