Giter Club home page Giter Club logo

relstorage's Introduction

RelStorage is a storage implementation for ZODB that stores pickles in a relational database (RDBMS). PostgreSQL 9.6 and above, MySQL 5.7.19 / 8.0, Oracle 10g and above, and SQLite 3.8.3 and above are currently supported. RelStorage replaced the PGStorage project.

Features

  • It is a drop-in replacement for FileStorage and ZEO, with several enhancements:
    • Supports undo, packing, and object history preservation just like FileStorage.
    • RelStorage can be configured not to keep object histories for reduced disk space usage and improved performance.
    • Multiple processes on a single machine can read and write a local ZODB database using SQLite without needing to start and manage another process (i.e., ZEO).
    • Blobs can be stored on a shared filesystem, or (recommended) in the relational database and only cached locally.
    • Multiple threads in the same process share a high-performance in-memory pickle cache to reduce the number of queries to the RDBMS. This is similar to ZEO, and the ZEO cache trace tools are supported.
    • The in-memory pickle cache can be saved to disk and read when a process starts up. This can dramatically speed up site warmup time by eliminating a flood of RDBMS queries. Unlike ZEO, this cache is automatically shared by all processes on the machine (no need to configure separate client identifiers.)
  • Ideal for large, high volume sites.
    • Multiple Python processes on multiple machines can read and write the same ZODB database concurrently. This is similar to ZEO, but RelStorage does not require ZEO.
    • Supports ZODB 5's parallel commit feature: Database writers only block each other when they would conflict (except for a small window at the end of the twophase commit protocol when the transaction ID is allocated; that still requires a global database lock).
    • According to some tests, RelStorage handles concurrency better than the standard combination of ZEO and FileStorage.
    • Whereas FileStorage takes longer to start as the database grows due to an in-memory index of all objects, RelStorage starts quickly regardless of database size.
    • Capable of failover to replicated SQL databases.
  • Tested integration with gevent for PostgreSQL, MySQL, and SQLite.
  • There is a simple way (zodbconvert) to (incrementally) convert FileStorage to RelStorage and back again. You can also convert a RelStorage instance to a different relational database. This is a general tool that can be used to convert between any two ZODB storage implementations.
  • There is a simple way (zodbpack) to pack databases.
  • Supports zodburi .
  • Free, open source (ZPL 2.1)

Features Supported by Databases

Some of RelStorage's features are only supported on certain versions of certain databases. If the database doesn't support the feature, RelStorage will still work, but possibly with a performance penalty.

Supported Features
Parallel Commit Shared readCurrent locks Non-blocking readCurrent locks Streaming blobs Central transaction ID allocation Atomic lock and commit without Python involvement
PostgreSQL Yes Yes Yes With psycopg2 driver Yes Yes, except with PG8000 driver
MySQL Yes Yes Native on MySQL 8.0, emulated on MySQL 5.7 No, emulated via chunking Yes Yes
Oracle Yes No Yes Yes No (could probably be implemented) No (could probably be implemented)
SQLite No No N/A (there is no distinction in lock types) No, consider using a shared-blob-dir N/A (essentially yes because it happens on one machine) No

Documentation

Documentation including installation instructions is hosted on readthedocs.

The complete changelog is also there.

image

Development

RelStorage is hosted at GitHub:

https://github.com/zodb/relstorage

Continuous integration

A test suite is run for every push and pull request submitted. GitHub Actions is used to test on Linux and macOS, and AppVeyor runs the builds on Windows.

image

image

Builds on CI automatically submit updates to coveralls.io to monitor test coverage.

image

relstorage's People

Contributors

500062397 avatar cutz avatar frapell avatar hathawsh avatar jamadden avatar jimfulton avatar madjar avatar mamico avatar mauritsvanrees avatar mjpieters avatar pfw avatar seanupton avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.