Giter Club home page Giter Club logo

go-hasql's Introduction

hasql

PkgGoDev GoDoc tests lint Go Report Card codecov

hasql provides simple and reliable way to access high-availability database setups with multiple hosts.

Status

hasql is production-ready and is actively used inside Yandex' production environment.

Prerequisites

Installation

With Go module support, simply add the following import

import "golang.yandex/hasql"

to your code, and then go [build|run|test] will automatically fetch the necessary dependencies.

Otherwise, to install the hasql package, run the following command:

$ go get -u golang.yandex/hasql

How does it work

hasql operates using standard database/sql connection pool objects. User creates *sql.DB objects for each node of database cluster and passes them to constructor. Library keeps up to date information on state of each node by 'pinging' them periodically. User is provided with a set of interfaces to retrieve *sql.DB object suitable for required operation.

dbFoo, _ := sql.Open("pgx", "host=foo")
dbBar, _ := sql.Open("pgx", "host=bar")
cl, err := hasql.NewCluster(
    []hasql.Node{hasql.NewNode("foo", dbFoo), hasql.NewNode("bar", dbBar) },
    checkers.PostgreSQL,
)
if err != nil { ... }

node := cl.Primary()
if node == nil {
    err := cl.Err() // most recent errors for all nodes in the cluster
}

// Do anything you like
fmt.Println("Node address", node.Addr)

ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err = node.DB().PingContext(ctx); err != nil { ... }

hasql does not configure provided connection pools in any way. It is user's job to set them up properly. Library does handle their lifetime though - pools are closed when Cluster object is closed.

Supported criteria

Alive primary|Alive standby|Any alive node, or none otherwise

node := c.Primary()
if node == nil { ... }

Alive primary|Alive standby, or any alive node, or none otherwise

node := c.PreferPrimary()
if node == nil { ... }

Ways of accessing nodes

Any of currently alive nodes satisfying criteria, or none otherwise

node := c.Primary()
if node == nil { ... }

Any of currently alive nodes satisfying criteria, or wait for one to become alive

ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
node, err := c.WaitForPrimary(ctx)
if err == nil { ... }

Node pickers

When user asks Cluster object for a node a random one from a list of suitable nodes is returned. User can override this behavior by providing a custom node picker.

Library provides a couple of predefined pickers. For example if user wants 'closest' node (with lowest latency) PickNodeClosest picker should be used.

cl, err := hasql.NewCluster(
    []hasql.Node{hasql.NewNode("foo", dbFoo), hasql.NewNode("bar", dbBar) },
    checkers.PostgreSQL,
    hasql.WithNodePicker(hasql.PickNodeClosest()),
)
if err != nil { ... }

Supported databases

Since library works over standard database/sql it supports any database that has a database/sql driver. All it requires is a database-specific checker function that can tell if node is primary or standby.

Check out golang.yandex/hasql/checkers package for more information.

Caveats

Node's state is transient at any given time. If Primary() returns a node it does not mean that node is still primary when you execute statement on it. All it means is that it was primary when it was last checked. Nodes can change their state at a whim or even go offline and hasql can't control it in any manner.

This is one of the reasons why nodes do not expose their perceived state to user.

Extensions

Instrumentation

You can add instrumentation via Tracer object similar to httptrace in standard library.

sqlx

hasql can operate over database/sql pools wrapped with sqlx. It works the same as with standard library but requires user to import golang.yandex/hasql/sqlx instead.

Refer to golang.yandex/hasql/sqlx package for more information.

go-hasql's People

Contributors

bbrodriges avatar bruapahe avatar ravener avatar vano468 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-hasql's Issues

Allow creating a new Cluster with alive and down nodes

Scenario:

  • I call hasql.NewCluster with three nodes. One node is unreachable ("context deadline exceeded").
  • All further calls to Cluster.WaitFor* return an error.
  • Calls to Cluster.Alive, Cluster.Primary* and Cluster.Standby* return nil.

Expected: Being able to create a cluster which includes unreachable nodes and still be able to query the alive nodes.

Return error with description when node of requested type is unavailable

Functions like Primary return node and nothing else. If nil is returned it is very hard to understand why exactly it happened - you have to parse logs (if you have those), etc.

We need another set of functions returning both node and error describing why exactly requested node is unavailable.

When we have enough API changes to warrant v2 we should think about removing old versions of these functions leaving only one variant.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.