Giter Club home page Giter Club logo

transactions-anomaly-detection-deep-autoencoder-flask's Introduction

Deep Autoencoders based Anomaly Detection for Transactional data

Project Overview

Objective

This project focuses on the development of deep learning models based on autoencoders for the purpose of anomaly detection. Autoencoders are neural networks used to learn compressed representations of raw data, making them effective tools for detecting anomalies in datasets. The project also involves deploying the trained model as an API using Flask.


Aim

The primary objectives of this project include:

  • For normal transactions developing a deep learning model based on autoencoders for anomaly detection.
  • Deploying the model as an API using Flask for real-time anomaly detection.

Data Overview

The dataset used in this project is a transaction dataset containing information on more than 100,000 transactions, each characterized by several features. This data serves as the foundation for training and testing the deep autoencoder model.


Tech Stack

  • Language: Python
  • Packages: Pandas, Numpy, Matplotlib, Keras, Tensorflow
  • API Service: Flask, Gunicorn

Approach

The project follows a structured approach:

  1. Understand the business objective and the importance of anomaly detection.
  2. Perform exploratory data analysis (EDA) to gain insights into the dataset.
  3. Normalize and clean the data, addressing any missing values through imputation.
  4. Delve into the theory behind autoencoders and their architecture.
  5. Build a base autoencoder model using the Keras library.
  6. Fine-tune the model to extract the best performance for anomaly detection.
  7. Make predictions using the trained model to identify anomalies.
  8. Serve the model as an API endpoint using Flask, enabling real-time anomaly detection.

Modular Code

  1. input: Contains the dataset files used for analysis (e.g., final_cred_data.csv, Test-data.csv).

  2. src: The heart of the project, this folder contains modularized code for various steps, including data preprocessing, model building, and deployment. It consists of the ML_pipeline and engine.py files, each containing functions for different functionalities.

  3. output: Contains pre-trained models saved as .pkl files. These models can be conveniently loaded and used without the need for retraining.

  4. lib: A reference folder with the original IPython notebook.

  5. requirements.txt: Lists all required libraries and their versions for easy installation using pip.


transactions-anomaly-detection-deep-autoencoder-flask's People

Contributors

ajnavneet avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.