emerleite / node-database-cleaner Goto Github PK

It seems that the cleaner doesn't remove (unique) indexes, which is problematic when you change indexes and the tests still hit a left over unique index.

I'm trying to run the database cleaner in a promise

const Promise = require('bluebird');

const pg = require('pg');
const DbCleaner = require('database-cleaner');
const dbCleaner = new DbCleaner('postgresql');

// Returns a function that, given a Postgres URL, returns a promise
// fulfilled when the database is cleaned out.
// It uses database-cleaner package and omits the 'knex_migrations' table
// (this is specified in the config/cleaner-config.js file)
module.exports = function() {

  // The connection string will be based on dotenv
  var conString = process.env.DATABASE_URL || process.env.TEST_DATABASE_URL;

  return new Promise((resolve, reject) => {

    // Connect to database
    pg.connect(conString, (err, client, done) => {
      if (err) {
        return reject(err);
      }

      // Clean database using the client
      dbCleaner.clean(client, () => {
        done();
        resolve();
      });

    });
  });
};

But I have to call it twice for it to work

afterAll((done) => {
  // Clean the database at the end
  clean()
    .then(function () { return clean() })
    .then(done);
}

Otherwise, some records stay in the database at the end of the test.

Which makes me think I'm not using the Promise correctly, do you have any recommendation on how to do this?

Try to figure out a way to automatic integrate with Mocha.

We should do this instead:

  try {
    config = require("../../../config/cleaner-config.js");
  } catch (e) {
    config = require("../config/cleaner-config.js");
  }

Today we need a global nodeunit installation to run tests. The target is to have a script to run all tests. This script needs to use nodeunit local dependency

Could you move dependencies to devDependencies, so I can choose what databases to install?

Currently installing database-cleaner installs all adapters, which is a very heavy operation.

See for example knex: https://github.com/tgriesser/knex/blob/master/package.json

My project is a typical MEAN stack app: MongoDB as my database, no MySQL or Postgres in sight.

I've found I can no longer install the latest database cleaner: it fails at install time!

I create a fresh npm package to demo this problem. Here is my package.json from it

{
  "name": "test_proj",
  "version": "1.0.0",
  "description": "Testing database_cleaner",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "dependencies": {
    "database-cleaner": "*"
  },
  "author": "",
  "license": "ISC"
}

As you see, this is pretty much the simplest Node project you can have, except for the dependency on database-cleaner

When I npm install this package I get:

% npm install                                                                                                             
npm WARN package.json [email protected] No repository field.
npm WARN package.json [email protected] No README data

> [email protected] install /home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/kerberos
> (node-gyp rebuild 2> builderror.log) || (exit 0)

make: Entering directory `/home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/kerberos/build'
  SOLINK_MODULE(target) Release/obj.target/kerberos.node
  SOLINK_MODULE(target) Release/obj.target/kerberos.node: Finished
  COPY Release/kerberos.node
make: Leaving directory `/home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/kerberos/build'

> [email protected] install /home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/bson
> (node-gyp rebuild 2> builderror.log) || (exit 0)

make: Entering directory `/home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/bson/build'
  CXX(target) Release/obj.target/bson/ext/bson.o
  SOLINK_MODULE(target) Release/obj.target/bson.node
  SOLINK_MODULE(target) Release/obj.target/bson.node: Finished
  COPY Release/bson.node
make: Leaving directory `/home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/mongodb/node_modules/bson/build'

> [email protected] install /home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/pg
> node-gyp rebuild || (exit 0)

/bin/sh: 1: pg_config: not found
gyp: Call to 'pg_config --libdir' returned exit status 127. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:343:16)
gyp ERR! stack     at ChildProcess.emit (events.js:98:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:810:12)
gyp ERR! System Linux 3.11.0-15-generic
gyp ERR! command "node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/vagrant/Temp/test_proj/node_modules/database-cleaner/node_modules/pg
gyp ERR! node -v v0.10.33
gyp ERR! node-gyp -v v1.0.1
gyp ERR! not ok

This makes sense to me: I don't have pg_config installed on my system, because I'm not using Postgres! But I should be able to use database-cleaner without it!

I'm sorry I only have time to file this issue, not track it down further. For now I've reverted to database-cleaner 0.8.0, but it would be nice to use the latest cleaner version ;)

Add support to clean redis

Async code and tests with lots of callbacks sucks. Need to refactor or use some library like Do or similar

in https://github.com/emerleite/node-database-cleaner/blob/master/lib/database-cleaner.js#L94

there should be a filter to only get tables AND table_type = 'BASE TABLE' instead of views too.

I have a very particular case, i'm getting into a legacy database, and i don't touch most of those tables, there's more than 100 tables but only use 2 or 3, and create 3 more.

Basically it makes no sense to me to write down the 100 table names in the skipTables property, maybe it would be better if you provide a selectTables that does the opposite, picks those tables to clean up.

If there are no collections in the database, database-cleaner will hang, because it never calls the callback. collections.forEach will simply do nothing when collections has no items in it.

A few problems came up for us:

1. When database-cleaner lists out the tables to be deleted, the tables are not ordered by foreign key constraints. So we have child with a foreign key to parent, but DELETE FROM "parent" is attempted to be run before DELETE FROM "child", causing the table to not be deleted because there are still rows in child referencing parent

2. When the above command didn't work, database-cleaner silently failed. I believe database-cleaner should check that the table was indeed cleared, and error otherwise.

Truncation seems to work for us. Is there a reason why truncation is not the default?

Line 63 should be
db.query('DELETE FROM "' + table['table_name'] + '"', function(err) {

to get the right table name

Use another database (e.g database 1) to not screw up user database during redis tests

Why are you doing such a test when clearly you are dropping the collections?

Support CouchDB cleanup

Hello emerleite,
is there any way how to reset auto increment on mysql by this package?

Do not work with mongodb driver v0.9.6 because the driver api changed. Need to fix it to upgrade

Need to make compatible with Node.JS 0.6.0

Previously (0.4.0) working test fail because db cleaner never invokes its callback. The tests fail as well, here is the output when testing only mongodb

mongodb
0) should delete all collections items
✓ should not delete system.indexes collection (1041ms)

✖ 1 of 2 tests failed:

0. mongodb should delete all collections items:
   Error: timeout of 2000ms exceeded
   at Object. (/Users/mwawrusch/Documents/npm-packages/node-database-cleaner/node_modules/mocha/lib/runnable.js:111:14)
   at Timer.ontimeout (timers.js:84:39)

Hello, thank you for this awesome library!

I'm trying to use this with sequelize (which relies on promises), could you please account for possibly a sequelize connection object and/or promises?
Line 93 doesn't account for promises - just a callback:

https://github.com/emerleite/node-database-cleaner/blob/master/lib/database-cleaner.js#L93

Thank you!

Hi,

Thanks for that plugin. I'm using it in a postgres db project.

The problem I've is that when the plugin delete all tables, I've constraints errors because some table are linked to others so they can't be deleted. It's a good think to disable the triggers in the the table before doing the delete:

db.query("ALTER TABLE " + "\"" + table['table_name'] + "\" DISABLE TRIGGER ALL", function() {
    ...
}

And after delete we enable triggers

db.query("ALTER TABLE " + "\"" + table['table_name'] + "\" ENABLE TRIGGER ALL", function() {
    ...
}

The second think is also to restart the id sequences after delete. The plugin just delete datas but not restart the sequence.

db.query("ALTER SEQUENCE IF EXISTS " + table['table_name'] + "_id_seq RESTART", function() {
    ...
}

It won't be a good thing to have this features.

Thanks again

Support MySQL

It sure looks like no one has ever used the mysql cleaner.

        if (table['Tables_in_database_cleaner'] != 'schema_migrations') {
          db.query("DELETE FROM " + table['Tables_in_database_cleaner'], function() {

From

I'm not really sure what this is supposed to be doing. It appears like we should be comparing table['Table_in_<db_name>'] here, not table['Tables_in_database_cleaner'].