Giter Club home page Giter Club logo

watcher-public's Introduction

Loggicat solves data leaks by shifting data security left

Follow on Twitter



Getting Started With Loggicat Watcher


Overview

More information is available at Loggicat Cloud Wiki


Versions

There are two versions available. This repo is the open-source version and should only be used to understand how Loggicat Watcher can be integrated in your dev environment, please use the closed-source version for production use and for better performance.

How to scan Scan speed Keep data to local only Limit
Open-Source Ver Use Loggicat Cloud Engine API Slow, this depends on network speed No, the open-source version must send text to Loggicat Cloud 100 MB per user per day for free users
Closed-Source Ver Scan engine is built-in fast Yes, scan engine can be used locally without sending anything to Loggicat Cloud There is no limit

Features

  • Scan local code for secrets and PII (Github repo scan and commit scan will be released in the future releases)
  • Scan local logs for secrets and PII
  • Monitoring local logs, vulnerable lines will be temporarily on-hold until released by an user (Log streaming is currentely not supported)

Prerequisite

GoLang 1.15 or above

Configuration

A configuration json file should contain following information

{
  "operationMode": "",              //scan or monitor
  "scope": "",                      //log or code

  "token": "",                      //Loggicat Cloud API token
  "uuid": "",                        //Loggicat Cloud API token UUID
  
  "engineType": "",                 //cloud or local
  "engineURL": "",                  //only used when engineType is local
  
  "refreshTime": ,                  //time gap to pull releases from Loggicat Cloud
  
  "path": [                         //folders to scan
      "" 
  ],
  
  "outputMode": "",                 //cloud or local
  "outputLocation": ""              //output file location, only used when outputMode is local
}

Operation Modes

  • Monitor : monitor files, changes will be scanned as well, this should be used for logs
  • Scan : one time scan, this should be used for logs and code In the Monitor mode, clean logs and released security findings will be appended to new logs files with .loggicat extension.
    For example, if you are forward myAppLog.txt to a log ingestion platform, you should now use myAppLog.txt.loggicat instead.

Scope

  • Code : Server will try to parse the supported languages such as .go or .xml before scanning
  • Log : Server will try to parse logs in the popular formats before scanning

API Token, UUID

API token and UUID can be generated on Loggicat Cloud.

Engine Type, Engine URL

Put "local" for the type if you use Loggicat Scan Engine

Refresh time

Loggicat Clouds never push contents to Loggicat Watcher so in order to append triaged result to logs, Watcher periodically pulls the result from Loggicat Cloud, this configuration value is in minutes.

Path

Folders to be scanned

Output Modes, Output Location

  • Cloud : Scan results will be sent to Loggicat Cloud
  • local : Generate a local json file to store scan results, many features are not available in this mode. Output location is used i nthis mode.

CLI Usages

Open-source version

//A template will be generated if configs/watcherConfig.json does not exist
go run main.go

Important Notes

  1. Non-nessccary builtin rules should be disabled to speed up the scan speed, however, generic rules such as "Generic Secrets" should always be enabled.
  2. Ignore list has higher priority than redact list, so your finding will be ignored if you have the same keyword in both ignore and redact lists, a feature to improve this behavior is under development.
  3. Monitor mode should only be used for logs, local code monitoring mode and code push monitoring are under development.

watcher-public's People

Contributors

loggicatkt avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.