Giter Club home page Giter Club logo

yildiz's Introduction

Yildiz Graph Database

yildiz

Thin graph database layer on top of Google Bigtable.

Build Status Version MIT License node Swagger Mentioned in Awesome Bigtable

Intro

YildizDB acts as a highly scaleable HTTP layer in front of Google Bigtable. It helps you store billions of relations between nodes in edges and resolve them in milliseconds. A small access layer helps you manage multiple namespaces with ease. YildizDB scales to hundrets of Terabytes. YildizDB excells in $N:N$ non nested relationships.

Features

  • simple namespacing (table separation) with prefixes
  • simple access management through tokens
  • multi-tenancy through table prefixes (with access management)
  • high read and write performance
  • fast read access under heavy write load
  • able to handle billions of edges and nodes
  • scales beyond Terabytes
  • lightweight deployments (small Node.js footprint)
  • ttl feature for all resources
  • HTTP Open API
  • Kubernetes HELM charts

Build for high throughput $N:N$

  • Highly async API based on fastify
  • Thin layer on top of Google's GRPC Bigtable API
  • Hashing and translating all string identifiers into integer representations via murmurhash3 automatically

Fast random access to a node's edge data $1:N$

  • Multiple complex caching layers
  • Custom Bigtable cache table speed up by Google Memorystore (Redis)
  • All Memorystore hits also speed up by In-Memory store
  • Fetch job that keeps active nodes refreshed in cache

Available clients

  • Node.js Client
  • Any Http Client can be used to access the HTTP-Interface

Usage

  • You will need a Google Cloud Project with a running Bigtable cluster
  • Additionally YildizDB requires a Memorystore (or Redis) instance
  • Configure ./config/bigtable.json accordingly
  • Install and start: npm install -g yildiz and yildizdb -p 3058 -l ./config/bigtable.json
  • A word on configuration can be found here
  • Use right alongside your code
  • Spawn server via http interface

Deployment

  • YildizDB is designed to be deployed as simple platform service
  • it requires Node.js > 9.x.x and Redis > 3.x
  • It simply requires its npm module as well as a config file that describes the connections to Bigtable and Memorystore (Redis)
  • It scales best with an HTTP load balancer in front of it e.g. NGINX
  • We run and scale it very successfully in Google's Kubernetes Engine
  • We also offer HELM charts
  • Read some more about the config file here

Metrics & Monitoring

  • Yildiz exposes Prometheus Metrics @ /admin/metrics
  • Read more about it here

Developing YildizDB

Developing YildizDB with Google Bigtable backend

  • Configure ./config/bigtable.json accordingly
  • Run tests via yarn test
  • Start via yarn http

Documentation

Disclaimer

  • This project is not affiliated with Google
  • License is MIT see

yildiz's People

Contributors

dependabot[bot] avatar holgeradam avatar krystianity avatar rjmasikome avatar zrosenbauer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

yacut holgeradam

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.