Type of action

• add

Content

• https://github.com/weaveworks/cortex

Category

Database

Reason (Optional)

• a rewrote of prometheus with long term storage using Dynamo and S3

Not all TSDB listed here are open source, and some of them offer advanced functionalities in enterprise/hosted version. Also even for open source ones, license may vary, due to the recent issues from React, it's important to keep the licenses on track

• license
  • https://github.com/github/choosealicense.com/tree/gh-pages/_licenses is a good to place find a full list of licenses (that is provided by GitHub)
• commercial support
  • company that provides commercial support
  • what is offered in commercial support

Type of action

• add

Content

• https://github.com/m3db/m3db
• https://www.slideshare.net/RobSkillington/go-and-ubers-time-series-database-m3

Category

Database

Reason (Optional)

• used by Uber? (if this count as a reason)

Type of action

• add

Content

• http://siridb.net/
• https://github.com/transceptor-technology/siridb-server
• https://github.com/transceptor-technology/go-siridb-connector
• http insert https://github.com/transceptor-technology/siridb-http#insert-json
• http://siridb.net/blog/how-we-store-time-series-in-siridb/

Category

Database

Reason (Optional)

• saw it long time ago ... didn't put it into the issue list
• ground up in C
• HA
• DSL for query
• use mgpack, qpack(?) support http json

Type of action

• add

Content

One of the many alternative backends for Graphite (I forgot to add Graphite ... 🙀 , I think I should just add it silently)

• https://github.com/criteo/biggraphite
• https://github.com/criteo/biggraphite/blob/master/CASSANDRA_DESIGN.md
• https://github.com/criteo/biggraphite/blob/master/benchmarks/accessor_bench.py

Category

Database

Reason (Optional)

Another TSDB on top of Cassandra

• it sets memtable_flush_period_in_ms, which is not seen in many other Cassandra based TSDB's schema
• It also mentioned SASI (SSTable-Attached Secondary Index) which I didn't know

Type of action

• add

Content

Survey and Comparison of Open Source Time Series Databases

Category

Paper

Reason (Optional)

add: this paper presents a survey on existing TSDBs and describes their features.

Type of action

• add

Content

• https://www.circonus.com/irondb/
• https://twitter.com/IRONdb
• Fred Moyer: Solving the Technical Challenges of Time Series Databases at Scale video
  • introduced the revolution of Circonus's backend (pg -> smooth? -> irondb)
  • CAQL Circonus Analytics Query Language
  • also has a ingestion performance graph w/ InfluxDB and Cassandra
• https://github.com/circonus-labs/reconnoiter (seems to have code for using old PgSQL backend)
• https://www.circonus.com/2017/12/high-volume-ingest (didn't look at it yet, the main point seems to be data has a striped fashion)

Category

Database

Reason (Optional)

• circonus has being doing the monitor a long time, IRONdb is their latest storage

Type of action

• add

Content

• https://github.com/tgres/tgres
• https://grisha.org/blog/2017/01/21/storing-time-seris-in-postgresql-optimize-for-write/

Based on postgres

Category

Database

Reason (Optional)

Also from the blog http://www.iamsysadmin.ninja/2017/03/tsdb-on-postgresql-but-why.html as #17

Type of action

• add

Content

Use Cassandra as Backend, support roll up, but does not support tag like KairosDB

• https://github.com/rackerlabs/blueflood
• http://www.slideshare.net/gdusbabek/blueflood-open-source-metrics-processing-at-cassandraeu-2013

Category

Database

Reason (Optional)

Type of action

• add

Content

• https://github.com/digitalocean/vulcan

Category

Database

Reason (Optional)

• as a backend for prometheus
• use Cassandra for long term storage https://github.com/digitalocean/vulcan/blob/master/cassandra/writer.go

Type of action

• add

Content

• https://github.com/manahl/arctic

Category

Database

Reason (Optional)

• 'supports Pandas, numpy arrays and pickled objects out-of-the-box, with pluggable support for other data types and optional versioning'
• use MongoDB, 'query millions of rows per second per client, achieves ~10x compression on network bandwidth, ~10x compression on disk, and scales to hundreds of millions of rows per second per MongoDB instance'
• provide an example of storing ticker

Type of action

• add

Content

• https://github.com/linkedin/pinot

Category

Database

Reason (Optional)

• columnar store
• found it on twitter, some guy from uber benchmarked it against Druid and says pinot is better etc. https://twitter.com/shirshanka/status/857410394419208192

Currently, the README is generated based on yaml files, it would be better to have a web page for filtering databases based on user selections (i.e. desired backend)

I'm planning to try vue since I'm tired of Angular and never a big fan for react, this could be a start (.w.)

• choose the place to put the generated files, docs folder might be ok, though gh-pages branch is also ok
• auto generated html based on the yaml (i.e. create a large json data based on the yaml file)
• add tag support for databases
• improve the logic between yaml (i.e. you only need to declare the reference for one time) now using js instead of yaml
• might managed to work with the static webpage generator for tongqu https://github.com/tongquhq/about now using nuxt.js
• submit to awesome list
  • not likely, since auto generated README is not accepted

Ref

• https://nuxtjs.org/guide#static-generated might give this a shot, NOTE: I think it would be also ok to rewrite tongqu's about page in this way, use js for config instead of using yaml

When looking at Andy's video about SQL, it showed a web page for comparing SQL standard, and in that website found the term Temporal database http://troels.arvin.dk/db/rdbms/links/#temporal, there are draft called TSQL, seems to be an old concept ....

• http://people.cs.aau.dk/~csj/Thesis/
• a database called TimeDB http://www.timeconsult.com/Software/Software.html
• SQL:2011 has Temporal DBs and Pipelined DML https://youtu.be/6dg6fvJbqu8?t=12m37s

There has been projects putting information of database into database, i.e. https://github.com/cmu-db/dbdb.io which requires a python server and SQL database (and auth for public access).

We serve the static website using gh-pages, so we can't use MySQL etc. However, there are many database (even SQLish) using javascript and run in browser (w/o using WebSQL which is deprecated), we just need to figure out the schema, and turn our existing YAML files into other formats. We want features like join so user can run query in browser (and passing the query in the url should show the filtered result, which is already supported )

Databases that can run in browsers

• https://github.com/kripken/sql.js/ 3.9k star SQLite compiled to javascript using emscripten
  • http://kripken.github.io/sql.js/GUI/
  • Execute millions of sql statements in milliseconds
• https://github.com/agershun/alasql 2.9 k star 'JavaScript SQL database for browser and Node.js. Handles both traditional relational tables and nested JSON data (NoSQL). Export, store, and import data from localStorage, IndexedDB, or Excel'
• https://github.com/pouchdb/pouchdb 9.8k star 'store data locally while offline, then synchronize it with CouchDB and compatible servers when the application is back online'
• https://github.com/typicaljoe/taffydb 1.7k star , no SQL support, js
• https://github.com/louischatriot/nedb 7.5k star, 'Embedded persistent or in memory database for Node.js, nw.js, Electron and browsers, 100% JavaScript, no binary dependency. API is a subset of MongoDB's and it's plenty fast'
• https://github.com/techfort/LokiJS/ 3.8 k in memory document

TODO

• pick one that can be integrated w/ nuxt.js (vue) (l think there might be some react binding, but no for vue ...)
• update the script, and extend YAML format to address problems like license #34
  • I forgot, all the YAML file are changed to javascript ... in #16 f6693a0
• db specific detail, might include content from previous survey and blogs (also other people's blog)
• close tons of issues .... wow, so many time series, such database
• use nelify for PR so we can have preview before merge
• update .github for issue template, we now have benchmark, blog etc. instead of just database
• make use of new dropdown feature github markdown (actually it's HTML <details> and <summary>... not gfm ...)

Type of action

• add

Content

• https://github.com/hawkular/hawkular-metrics
• schema https://github.com/hawkular/hawkular-metrics/blob/master/core/schema/src/main/resources/org/hawkular/schema/bootstrap.groovy

Category

Database

Reason (Optional)

Type of action

• add

Content

• https://kx.com/
• https://github.com/KxSystems/kdb-tick/blob/master/tick.q

Category

Database

Reason (Optional)

• based on kdb+ (w/ APL like q language ...)
• CMU got a series of talk http://db.cs.cmu.edu/seminar2017/

http://db-engines.com/en/ranking/time+series+dbms saw the db engine ranking from https://github.com/gryadka/GryadkaJS but don't know how they get the information, may add this to README

their method http://db-engines.com/en/ranking_definition

Though I don't think this rank really mean something, some databases are easy to setup and play with for small scale projects, but if you want to scale, sorry man.

Type of action

• add

Content

• https://github.com/ChronixDB/chronix.storage
• https://www.slideshare.net/FlorianLautenschlage/chronix-a-fast-and-efficient-time-series-storage-based-on-apache-solr

Category

Database

Reason (Optional)

• Lucene and Solr (quite similar to elasticsearch)

Type of action

• add

Content

• https://github.com/raintank/metrictank
• https://www.percona.com/live/18/sessions/metrictank-building-a-new-time-series-engine-for-grafanacloud

Category

Database

Reason (Optional)

Cassandra-backed, metrics2.0 based, multi-tenant timeseries database for Graphite and friends

• https://github.com/cmu-db/dbdb.io The On-line Database of Databases
• http://db-engines.com/en/ranking/time+series+dbms from #5
• https://misfra.me/2016/04/09/tsdb-list/ from #8
• https://github.com/caesar0301/awesome-public-datasets#time-series

Other awesome lists

• https://github.com/mark-rushakoff/awesome-influxdb I think I can add this list into their readme, they have a section https://github.com/mark-rushakoff/awesome-influxdb#awesome-lists-that-include-links-to-influxdb

Type of action

• add

Content

• https://github.com/Cistern/catena
• http://www.slideshare.net/vividcortex/catena-a-highperformance-time-series-data

Category

Database

Reason (Optional)

Type of action

• add

Content

• https://github.com/timescale/timescaledb
• http://www.timescale.com/
• http://www.timescale.com/how-it-works.html
• http://www.timescale.com/papers/timescaledb.pdf
• TimescaleDB allows JOINs between time-series and business data

They are using custom storage engine for postgres

Category

Database

Reason (Optional)

Found it from the blog http://www.iamsysadmin.ninja/2017/03/tsdb-on-postgresql-but-why.html

Type of action

• add

Content

• https://cloud.tencent.com/product/ctsdb
• 腾讯唯一时序数据库：CTSDB 解密 https://cloud.tencent.com/developer/article/1010045

Category

Database

Reason (Optional)

• yet another tsdb from cloud service providers

Type of action

• add

Content

• https://github.com/mattbostock/timbala
• was called AthensDB mattbostock/timbala@dc8df55

Category

Database

Reason (Optional)

• saw it from go-nuts, talk of FOSDEM https://groups.google.com/forum/#!topic/golang-nuts/vqJpE8BdjOo when searching for interesting authors
• a distributed version using Prometheus's tsdb package https://github.com/prometheus/tsdb

Type of action

• add

Content

• https://github.com/dalmatinerdb/dalmatinerdb

Category

Database

Reason (Optional)

Dataloop has a very detailed comparison of TSDBs which I encountered long time ago and they are still updating it https://docs.google.com/spreadsheets/d/1sMQe9oOKhMhIVw9WmuCEWdPtAoccJ4a-IuZv4fXDHxM/edit#gid=0

Found it when looking at InfluxDB's blog which mentioned percona conf, went through the speakers and found this https://www.percona.com/live/17/users/david-gildeh-0

Type of action

• add

Content

• https://github.com/crate/crate
• https://crate.io/time-series/

Category

Database

Reason (Optional)

• found it when google on 4th page ....

Type of action

• add

Content

• https://github.com/NationalSecurityAgency/timely
• use Apache Accumulo http://accumulo.apache.org/ (similar to HBase? ...) and Grafana

a sorted, distributed key/value store that provides robust, scalable data storage and retrieval.
based on the design of Google's BigTable

Category

Database

Reason (Optional)

• from NSA ....
• https://nationalsecurityagency.github.io/timely/docs/?http#lookup-operation the docs looks pretty good, I think I have seen this kind of API doc (framework/style) before
  • should be slate https://github.com/lord/slate http://lord.github.io/slate/
• it seems they are trying to add a SQL interface NationalSecurityAgency/timely#98

Currently the website is built using https://github.com/nuxt/nuxt.js, however the author of vue released a similar tool https://github.com/vuejs/vuepress

Reason for making the switch

• I forgot how to use nuxt.js since it's been so long for PR #41 , since I have to learn nuxt.js again if I am going to use it, I'd rather learn vuepress
• angular universal is not usable, and it seems few attention is paid into it
• vuepress has
  • builtin markdown support
  • theme for building documentation site

Type of action

• add

Content

• https://cloud.baidu.com/product/tsdb.html
• 百度大规模时序数据存储（一）| 监控场景的时序数据
• 百度大规模时序数据存储（二）| 存储选型及数据模型设计
  • Use HBase
• 百度大规模时序数据存储（三）| 核心功能设计
  • Shard

Category

Database

Reason (Optional)

• like Tencent cloud #45

Type of action

• add

Content

• https://cn.aliyun.com/product/hitsdb
  • used to based on OpenTSDB, but changed a lot now
• Aliware打造史上最强时序数据库，HiTSDB每秒写入时序数据达1000万
• https://github.com/aliyun/HiTSDB-Client

Category

Database

Reason (Optional)

• like #46 #45

Type of action

• add

Content

• https://www.circonus.com/irondb/

Category

Databases

Type of action

• add

Content

Ultimate TSDB Comparison

Category

reading

Reason (Optional)

add: This web site aims for listing all available time series databases. It also offers a feature matrix with filter functionality. It can be seen as superset of the awesome-time-series-database, since it aims for listing all available databases whereas in the awesome list only awesome databases are listed.

Type of action

• add

Content

• https://github.com/Searchlight/khronus A reactive time series database
• Found it from http://kamon.io/backends/khronus/

Category

Database

Reason (Optional)

Type of action

• add

Content

• Why We Built Our Own Distributed Column Store
• inspired by Facebook's Scuba paper
• not open sourced

Category

Database

Reason (Optional)

• used by https://honeycomb.io/ in production
  • I was thinking they were using influxdb since I saw their people in some influxdb related talk, maybe I have bad memory

Type of action

• add

Content

• https://github.com/akumuli/Akumuli

Category

Database

Reason (Optional)

Type of action

• add

Content

• https://github.com/improbable-eng/thanos

Category

Database

Reason (Optional)

• use object store as long term storage for prometheus ... sounds really like
  • https://github.com/weaveworks/cortex
  • https://github.com/digitalocean/vulcan no longer maintained

Type of action

• add

Content

• http://www.taosdata.com/index.html the website and whitepaper are in Chinese
• http://www.taosdata.com/downloads/TDengine_White_Paper_cn.pdf

Category

Database

Reason (Optional)

• written in go, support HA, provide SQL interface
  • I don't think they will open source it though ...

Type of action

• add

Content

• https://github.com/cityzendata/warp10-platform
• https://archive.fosdem.org/2017/schedule/event/iot_warp10/

Category

Database

Reason (Optional)

• has GEO support
• found it in Andy's CMU TSDB talk twitter 's replies ...

Type of action

• add

Content

• https://github.com/openstack/gnocchi
• https://julien.danjou.info/blog/tags/Gnocchi.html
• http://gnocchi.xyz/index.html
• https://docs.openstack.org/developer/gnocchi/

Category

Database

Reason (Optional)

• used by OpenStack, support Ceph, S3 as backend

Type of action

• add

Content

• https://prometheus.io/
• https://github.com/prometheus/prometheus

Category

Database

Reason (Optional)

should have added it long time ago

Type of action

• add

Content

• https://github.com/citusdata/citus
• https://blog.cloudflare.com/scaling-out-postgresql-for-cloudflare-analytics-using-citusdb/ they say cloudflare is their user, however, clickhouse also said cloudflare is using clickhouse in 2017 percona live

Category

Database

Reason (Optional)

• using postgresql
• saw it in VividCortex's slide https://www.slideshare.net/vividcortex/how-vividcortexs-mysqlbased-time-series-database-works

Type of action

• add

Content

• https://github.com/srotya/sidewinder
• http://sidewinder.srotya.com/

Category

Database

Reason (Optional)

• seems to be doing everything from scratch, format, query language, replication

This issue is used as a memo for some monitoring tools, feel free to comment

A lot (almost all) of TSDBs are designed for monitoring, thus monitoring tools like graphing (sort of), analysis, alert always comes with TSDBs, i.e. Grafana, Promtheus.

TODO

• yaml and md generation
• issue template for monitoring tools

Ref

• https://github.com/xephon-contrib some are monitoring related
• https://github.com/cit-lab most forks are monitoring related

Type of action

• add

Content

• https://github.com/dustin/seriesly

Category

Database

Reason (Optional)

Type of action

• add

Content

• in memory, has a query language called stack
• https://github.com/Netflix/atlas

Category

Database

Reason (Optional)

Type of action

• add

Content

• https://github.com/yandex/ClickHouse

Category

Database

Reason (Optional)

• support as a storage engine as graphite https://github.com/yandex/graphouse
• columnar
• use vectorized execution
• found on percona live open source db ecosystem parrt1 https://youtu.be/7-vlmsWtGHc?t=30m4s

Type of action

• add

Content

• http://www.osisoft.com/pi-system/pi-capabilities/pi-server/

Category

Database

Reason (Optional)

• found in influxdata/influxdb-comparisons#4

OSIsoft PI is the de facto time-series historian for much of industry including upstream/downstream oil & gas. PI has been selected as the standard solution by Shell, ExxonMobil, Chevron, BP, Total, ... for example.

• A private company founded in 1980 (older than many existing TSDB developers 🐶 ) based on wiki

Type of action

• add

Content

• Flint: A Time Series Library for Apache Spark
• TimeSeriesRDD
• https://github.com/twosigma/flint

Category

Database

Reason (Optional)

On Spark, though I am not sure if I can call it a database, stream processing maybe, not sure where the border is

Type of action

• add

Content

• https://github.com/salesforce/Argus
• https://medium.com/salesforce-open-source/argus-time-series-monitoring-and-alerting-d2941f67864

Category

Database

Reason (Optional)