Giter Club home page Giter Club logo

node-database-cleaner's Introduction

Build Status

Node.js Database Cleaner

The simplest way to clean your database.

Say you're doing test cases. How can you clean up your data after each test run? With Database Cleaner you can simply do it with one line of code.

Supported Databases

  • MongoDB
  • Redis
  • CouchDB
  • MySQL
  • Postgres
  • Elasticsearch

Dependencies

Runtime

  • Node 0.8.x+
  • node-mongodb-native (a.k.a mongodb)
  • redis
  • cradle for couchdb (http://cloudhead.io/cradle)
  • mysql server
  • postgres server
  • elasticsearch

Development/Tests

  • mocha
  • should
  • hredis
  • redis
  • cradle
  • mysql
  • pg
  • elasticsearch

Installation

$ npm install database-cleaner

Usage

var DatabaseCleaner = require('database-cleaner');
var databaseCleaner = new DatabaseCleaner(type); //type = 'mongodb|redis|couchdb'

databaseCleaner.clean(database, callback);

Config

By default it attempts to load the config/cleaner-config.js file and will fall back to the default config.

Both MySQL and Postgres support truncation and deletion strategies. To specify this strategy specify the configuration as follows:

{
  postgresql: {
    strategy: 'truncation',
    skipTables: []
  }
}

Examples

Take look at test folder to see how it works.

Look at examples dir.

Running tests

There is some ways to run tests:

$ mocha test # you need mocha globaly installed. `npm install -g mocha`
$ npm test

For tests you need each database running (mongodb, redis, couchdb, mysql) Or run once. Ex: mocha test/redis.test.js

You can also run tests using local mocha. ./node_modules/mocha/bin/mocha

MySQL

To run mysql tests you need to have a database_cleaner database.

mysql -u root -e 'create database database_cleaner;'

Postgres

To run postgres tests you need to have a database_cleaner database.

createdb database_cleaner

To-Do

Author

License:

(The MIT License)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

node-database-cleaner's People

Contributors

amoki avatar chamini2 avatar dakatsuka avatar djhopper01 avatar elliotf avatar emerleite avatar fampinheiro avatar fragglebob avatar gabceb avatar iwz avatar kbariotis avatar lackac avatar mathop avatar phoenixchu10 avatar srghma avatar weexpectedthis avatar xcoderzach avatar yads avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

node-database-cleaner's Issues

[Feat] Include selectTables along with skipTables

I have a very particular case, i'm getting into a legacy database, and i don't touch most of those tables, there's more than 100 tables but only use 2 or 3, and create 3 more.

Basically it makes no sense to me to write down the 100 table names in the skipTables property, maybe it would be better if you provide a selectTables that does the opposite, picks those tables to clean up.

Fails for me on node 0.6.7 and mongodb 2.0.2

Previously (0.4.0) working test fail because db cleaner never invokes its callback. The tests fail as well, here is the output when testing only mongodb

mongodb
0) should delete all collections items
โœ“ should not delete system.indexes collection (1041ms)

โœ– 1 of 2 tests failed:

  1. mongodb should delete all collections items:
    Error: timeout of 2000ms exceeded
    at Object. (/Users/mwawrusch/Documents/npm-packages/node-database-cleaner/node_modules/mocha/lib/runnable.js:111:14)
    at Timer.ontimeout (timers.js:84:39)

Test script to avoid global nodeunit

Today we need a global nodeunit installation to run tests. The target is to have a script to run all tests. This script needs to use nodeunit local dependency

mysql never cleans

It sure looks like no one has ever used the mysql cleaner.

        if (table['Tables_in_database_cleaner'] != 'schema_migrations') {
          db.query("DELETE FROM " + table['Tables_in_database_cleaner'], function() {

From

if (table['Tables_in_database_cleaner'] != 'schema_migrations') {

I'm not really sure what this is supposed to be doing. It appears like we should be comparing table['Table_in_<db_name>'] here, not table['Tables_in_database_cleaner'].

Postgres tables not cleaning

A few problems came up for us:

  1. When database-cleaner lists out the tables to be deleted, the tables are not ordered by foreign key constraints. So we have child with a foreign key to parent, but DELETE FROM "parent" is attempted to be run before DELETE FROM "child", causing the table to not be deleted because there are still rows in child referencing parent

  2. When the above command didn't work, database-cleaner silently failed. I believe database-cleaner should check that the table was indeed cleared, and error otherwise.

Truncation seems to work for us. Is there a reason why truncation is not the default?

database-cleaner doesn't install on systems lacking Postgres

My project is a typical MEAN stack app: MongoDB as my database, no MySQL or Postgres in sight.

I've found I can no longer install the latest database cleaner: it fails at install time!

I create a fresh npm package to demo this problem. Here is my package.json from it

{
  "name": "test_proj",
  "version": "1.0.0",
  "description": "Testing database_cleaner",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "dependencies": {
    "database-cleaner": "*"
  },
  "author": "",
  "license": "ISC"
}

As you see, this is pretty much the simplest Node project you can have, except for the dependency on database-cleaner

When I npm install this package I get:

% npm install                                                                                                             
npm WARN package.json [email protected] No repository field.
npm WARN package.json [email protected] No README data

> [email protected] install /home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/kerberos
> (node-gyp rebuild 2> builderror.log) || (exit 0)

make: Entering directory `/home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/kerberos/build'
  SOLINK_MODULE(target) Release/obj.target/kerberos.node
  SOLINK_MODULE(target) Release/obj.target/kerberos.node: Finished
  COPY Release/kerberos.node
make: Leaving directory `/home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/kerberos/build'

> [email protected] install /home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/bson
> (node-gyp rebuild 2> builderror.log) || (exit 0)

make: Entering directory `/home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/bson/build'
  CXX(target) Release/obj.target/bson/ext/bson.o
  SOLINK_MODULE(target) Release/obj.target/bson.node
  SOLINK_MODULE(target) Release/obj.target/bson.node: Finished
  COPY Release/bson.node
make: Leaving directory `/home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/bson/build'

> [email protected] install /home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/pg
> node-gyp rebuild || (exit 0)

/bin/sh: 1: pg_config: not found
gyp: Call to 'pg_config --libdir' returned exit status 127. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:343:16)
gyp ERR! stack     at ChildProcess.emit (events.js:98:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:810:12)
gyp ERR! System Linux 3.11.0-15-generic
gyp ERR! command "node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/pg
gyp ERR! node -v v0.10.33
gyp ERR! node-gyp -v v1.0.1
gyp ERR! not ok

This makes sense to me: I don't have pg_config installed on my system, because I'm not using Postgres! But I should be able to use database-cleaner without it!

I'm sorry I only have time to file this issue, not track it down further. For now I've reverted to database-cleaner 0.8.0, but it would be nice to use the latest cleaner version ;)

Disable trigger before delete and restart sequences

Hi,

Thanks for that plugin. I'm using it in a postgres db project.

The problem I've is that when the plugin delete all tables, I've constraints errors because some table are linked to others so they can't be deleted. It's a good think to disable the triggers in the the table before doing the delete:

db.query("ALTER TABLE " + "\"" + table['table_name'] + "\" DISABLE TRIGGER ALL", function() {
    ...
}

And after delete we enable triggers

db.query("ALTER TABLE " + "\"" + table['table_name'] + "\" ENABLE TRIGGER ALL", function() {
    ...
}

The second think is also to restart the id sequences after delete. The plugin just delete datas but not restart the sequence.

db.query("ALTER SEQUENCE IF EXISTS " + table['table_name'] + "_id_seq RESTART", function() {
    ...
}

It won't be a good thing to have this features.

Thanks again

Promisifying the cleaner

I'm trying to run the database cleaner in a promise

const Promise = require('bluebird');

const pg = require('pg');
const DbCleaner = require('database-cleaner');
const dbCleaner = new DbCleaner('postgresql');

// Returns a function that, given a Postgres URL, returns a promise
// fulfilled when the database is cleaned out.
// It uses database-cleaner package and omits the 'knex_migrations' table
// (this is specified in the config/cleaner-config.js file)
module.exports = function() {

  // The connection string will be based on dotenv
  var conString = process.env.DATABASE_URL || process.env.TEST_DATABASE_URL;

  return new Promise((resolve, reject) => {

    // Connect to database
    pg.connect(conString, (err, client, done) => {
      if (err) {
        return reject(err);
      }

      // Clean database using the client
      dbCleaner.clean(client, () => {
        done();
        resolve();
      });

    });
  });
};

But I have to call it twice for it to work

afterAll((done) => {
  // Clean the database at the end
  clean()
    .then(function () { return clean() })
    .then(done);
}

Otherwise, some records stay in the database at the end of the test.

Which makes me think I'm not using the Promise correctly, do you have any recommendation on how to do this?

Bug for postgres cleaner

Line 63 should be
db.query('DELETE FROM "' + table['table_name'] + '"', function(err) {

to get the right table name

Doesn't remove indexes

It seems that the cleaner doesn't remove (unique) indexes, which is problematic when you change indexes and the tests still hit a left over unique index.

Hangs on an empty mongodb database

If there are no collections in the database, database-cleaner will hang, because it never calls the callback. collections.forEach will simply do nothing when collections has no items in it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.