Version 5.4.2 of npm-check just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As npm-check is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

The problem

In Knex, it is possible to define the connection as a single database URL, like:

{
  client: 'pg',
  connection: 'postgres://user:password@localhost:5432/database',
};

This is not supported by the databaseManagerFactory, it requires that the connection property is on the object form, like:

{
    client: 'pg',
    connection: {
      host: 'localhost',
      port: 5432,
      user: 'user,
      password: 'password',
      database: 'database',
    }
}

This makes it inconsistent with Knex. If it is too hard to pull in Knex' way of reading the config, an alternative would be to just document this shortcoming.

Workaround

For what it's worth, I get the values by running RegExp on the database URL, and it works:

const [
  ,
  user,
  password,
  host,
  port,
  database
] = /:\/\/(.*?):(.*?)@(.*?):(.*?)\/(.*)/u.exec(knexConnection);

const databaseManager = databaseManagerFactory({
  knex: {
    client: 'pg',
    connection: {
      host,
      port,
      user,
      password,
      database,
    },
  },
  dbManager: {
    ...
  },
});

Hi,

Have been using this package for awhile but recently started using it for seed files.

I've noticed that populateDb looks for a glob pattern of files, requires those files, and then checks to see if the module is a function. However, when generating knex seed files, the templates look like:

exports.seed = async function() {
  // ..seed data here
}

Where the module being loaded in knex-db-manager is { seed: [AsyncFunction] }. Therefore seed files aren't loading properly.

I have a commit here that fixes this, along with a fix for the wrong populatePathPattern. Can you tell me if this is an appropriate fix or if I'm missing something here? I can open a PR if the former, as well as write a test case for it.

Thanks!

Hi,

I've been struggling to deploy the initial version of a nodejs app with Postgres on Digitalocean.
I get the

Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?

ERROR on the migrateDb function call. How can I find out what exactly is happening on the knex.migrate.latest() used within and traceback to the actual issue?

The dependency knex was updated from 0.16.3 to 0.16.4.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

knex is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

Version 4.1.0 of mssql just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As mssql is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

Hello

When I run await dbManager.createDb('my_db'), The database is created but I get the error (node:4687) Warning: a promise was rejected with a non-error: [object Undefined]. The instruction is in a try catch bloc.

Is there anything I should do to avoid this error?

Thank you

The examples in the knex docs uses client: 'pg' but knex-db-manager requires client: 'postgres'. I'm curious about the reason for this? Could both be supported?

The dependency knex was updated from 0.16.6 to 0.16.7.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

knex is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

Version 2.8.1 of nsp just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As nsp is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

If I use an invalid password knex-db-manager gives this error, which I think can be OK. But it does not exit back to the prompt which I think is a problem.

Unhandled rejection error: password authentication failed for user "userwithrightstocreateusersanddatabases"
    at Connection.parseE (C:\code\XXX\node_modules\pg\lib\connection.js:567:11)
    at Connection.parseMessage (C:\code\XXXnode_modules\pg\lib\connection.js:391:17)
    at Socket.<anonymous> (C:\code\XXX\node_modules\pg\lib\connection.js:129:22)
    at emitOne (events.js:115:13)
    at Socket.emit (events.js:210:7)
    at addChunk (_stream_readable.js:252:12)
    at readableAddChunk (_stream_readable.js:239:11)
    at Socket.Readable.push (_stream_readable.js:197:10)
    at TCP.onread (net.js:588:20)

Version 3.5.2 of bluebird was just published.

This version is covered by your current version range and after updating it in your project the build failed.

bluebird is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

Version 2.16.0 of mysql was just published.

This version is covered by your current version range and after updating it in your project the build failed.

mysql is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Reference issue (not mine) - elixir-ecto/ecto#1925

I'm using the db manager to create multiple databases at once - doing so however gets me this error

{ error: source database "template1" is being accessed by other users
        at Connection.Object.<anonymous>.Connection.parseE (/root/app/node_modules/pg/lib/connection.js:567:11)
        at Connection.Object.<anonymous>.Connection.parseMessage (/root/app/node_modules/pg/lib/connection.js:391:17)
        at Socket.<anonymous> (/root/app/node_modules/pg/lib/connection.js:129:22)
        at emitOne (events.js:115:13)
        at Socket.emit (events.js:210:7)
        at addChunk (_stream_readable.js:252:12)
        at readableAddChunk (_stream_readable.js:239:11)
        at Socket.Readable.push (_stream_readable.js:197:10)
        at TCP.onread (net.js:589:20)

According to the referenced issue, db creation should be done logged into the postgres database - I'm guessing that by default it's logging in using the template1 for superuser.

The encoding is hard coded to 'utf8', which can cause a ER_COLLATION_CHARSET_MISMATCH error.

As 'utf8' is not compatible with a number of collations, e.g. the utf8mb4 variants, some of the functions will not work.

What it should do, is to use the one in config.knex.connection.charset if specified, alternatively we can have a new entry config.dbManager.charset.

I am speaking in the context of MySQL because I'm not too familiar with Postgres, but I guess it's probably the same.

Example -

Version 10 of Node.js (code name Dubnium) has been released! 🎊

To see what happens to your code in Node.js 10, Greenkeeper has created a branch with the following changes:

• Added the new Node.js version to your .travis.yml

If you’re interested in upgrading this repo to Node.js 10, you can open a PR with these changes. Please note that this issue is just intended as a friendly reminder and the PR as a possible starting point for getting your code running on Node.js 10.

Your Greenkeeper Bot 🌴

First of all, thanks for this great project! :)

I saw that the npm package of knex-db-manager points to a fork of this repo, and this fork doesn't support MySQL yet.

As of now, I'm using the version of knex-db-manager in this repo by running this:

npm i Vincit/knex-db-manager

However, it would be great if the npm was linked to this repo as opposed to the fork.

Thanks again! 😄

Version 4.17.5 of lodash was just published.

This version is covered by your current version range and after updating it in your project the build failed.

lodash is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

Version 0.14.0 of knex was just published.

This version is covered by your current version range and after updating it in your project the build failed.

knex is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

The devDependency mocha was updated from 6.1.0 to 6.1.1.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

mocha is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

This package looks awesome and I though it would save me from the dreaded duplicate key value violates unique constraint error.

It seems to be properly configured as I can await dbManager.createDb('some-db'); just fine.
But after inserting some seed data, I try to update the id sequences with await dbManager.updateIdSequences();, but nothing seems to happen.

All the tables keep having have their id sequences set to 1, which makes it impossible to insert data after seeding.

Version 7.4.1 of pg was just published.

This version is covered by your current version range and after updating it in your project the build failed.

pg is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

When I try to connect forcing SSL...there is no way to do this.
For backwards compatibility, this is what I propose:

• Add 'connectionPassthrough' to config.knex
• If true, the pg.Client will be created ONLY with the config.knex.connection data
• If false, the pg.Client will continue being created with _masterConnectionUrl

Here is the change to PostgresDatabaseManager.js (around line 214):

PostgresDatabaseManager.prototype.create_masterClient = function() {
  var self = this;
  return new Promise(function(resolve, reject) {
    let client;
    if (self.config.knex.connectionPassthrough) {
      client = new pg.Client(self.config.knex.connection);
    } else {
      client = new pg.Client(self._masterConnectionUrl());
    }
    client.connect(function(err) {
      if (err) {
        reject(err);
      } else {
        resolve(client);
      }
    });
  });
};

And here is my knexfile.js:

const dotenv = require('dotenv');
const fs = require('fs');

dotenv.load();

module.exports = {
  client: 'pg',
  connection: {
    database: process.env.DB_NAME,
    user: process.env.DB_USER,
    password: process.env.DB_PASSWORD,
    host: process.env.DB_HOST,
    ssl: {
      ca: fs.readFileSync('../deployment/rds-ca-2019-root.pem'),
      rejectUnauthorized: true
    }
  },
  connectionPassthrough: true,
  pool: {
    min: 2,
    max: 10
  },
  migrations: {
    tableName: 'migrations'
  }
};

I have an issue where my the createDb() function (on my instance of the PostgresDatabaseManager) results in the error:

error: must be member of role "<my_db_name>"

(where "<my_db_name>" is the name of the DB I'm creating).

The error doesn't occur in my dev environment (MacOS 10.13.3 running PosgreSQL 9.6.5), only in staging, which is a AWS RDS instance running PostgreSQL 9.6.6.

The problem sounds exactly like this stackoverflow question. The accepted answer, of adding the superuser to the role created for the new DB, also works for me. Eg..

await knexDbManager.knexInstance().raw(`GRANT ${database} TO ${superUser}`);
await knexDbManager.createDb();

Might be worth adding that GRANT statement to the createDb() function.

Version 0.12.7 of knex just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As knex is a direct dependency of this project this is very likely breaking your project right now. If other packages depend on you it’s very likely also breaking them.
I recommend you give this issue a very high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

The devDependency coveralls was updated from 3.0.5 to 3.0.6.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

coveralls is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Version 3.4.0 of mocha just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As mocha is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

Version 0.12.47 of gatsby just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As gatsby is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

I couldn't find this on an npm search and had to write my own. These keywords should help prevent this for someone else:

• copy
• createdb
• dropdb
• truncate

Using the following dbManager config:

  dbManager: {
    // db manager related configuration
    collate: ['latin1_swedish_ci'],
    superUser: user',
    superPassword: 'password'
  }

Gives

ER_COLLATION_CHARSET_MISMATCH: COLLATION 'latin1_swedish_ci' is not valid for CHARACTER SET 'utf8'

But there's no information on how to provide the character set

Hey there, Knex has support for UNIX sockets when socketPath is supplied to the connection: {} object.

The logic behind it is pretty straightforward, when socketPath is detected it ignores the host and port.

As this uses Knex I think it would be useful to add it. There are a lot of performance benefits to using sockets instead of TCP.

Happy to do this myself if you are accepting PRs.

Thoughts appreciated

☝️ Greenkeeper’s updated Terms of Service will come into effect on April 6th, 2018.

Version 5.6.0 of npm-check was just published.

This version is covered by your current version range and after updating it in your project the build failed.

npm-check is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Version 7.1.3 of glob was just published.

This version is covered by your current version range and after updating it in your project the build failed.

glob is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

Apologies for the barrage of truncateDb bugs!

After applying my workaround, I was able to bypass #83. I ran the command in Ubuntu and I was able to get the truncation to work!

I ran the same command on macOS, and it does not do the truncation 😱

I started doing some debugging, and I logged out the result of tableNames from https://github.com/Vincit/knex-db-manager/blob/master/lib/MySqlDatabaseManager.js#L133

[
  RowDataPacket { TABLE_NAME: 'article' },
  RowDataPacket { TABLE_NAME: 'article_history' },
  RowDataPacket { TABLE_NAME: 'knex_migrations' },
  RowDataPacket { TABLE_NAME: 'knex_migrations_lock' },
  RowDataPacket { TABLE_NAME: 'topic' },
  RowDataPacket { TABLE_NAME: 'user' }
]

The output returned seems to an object array with upper case TABLE_NAME as the key name, while the map is looking for the lowercase table_name, which causes tableNames to be an empty array.

Currently, the only workaround I can see for this is to do add an alias as to the select query:

 this._cachedTableNames = knex('information_schema.tables')
	// 👋👋 Force knex to return lowercase `table_name`
    .select('table_name AS table_name')
    .where('table_schema', config.knex.connection.database)
	// .. other stuff

Can you think of any other workarounds? Do you think something could be wrong with the MySQL macOS installation on my Mac?

Currently at least populatePathPattern is missing. Some other might be missing too.

The devDependency mysql was updated from 2.17.0 to 2.17.1.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

mysql is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Package 'pg-escape' is missing in package.json of this module:

Error: Cannot find module 'pg-escape'

What's the deal with mssql? How come its not supported?

Each of my database tests have the following:

beforeEach(done => {
  dbManager.truncateDb(['knex_migrations', 'knex_migrations_lock']).then(() => done());
})

afterEach(done => {
  dbManager.close().then(() => done());
})

However, I'm seeing a whole lost of processes still running:

❯ los feature/teardown-everything* ps aux | ack postgres
et               29148   0.0  0.0  2610452   3488   ??  Ss   11:33AM   0:00.00 postgres: et los_test 127.0.0.1(59912) idle
et               29147   0.0  0.0  2618644   4348   ??  Ss   11:33AM   0:00.00 postgres: et los_test 127.0.0.1(59911) idle
et               29024   0.0  0.0  2635028   6152   ??  Ss   11:33AM   0:00.01 postgres: et los_test 127.0.0.1(59898) idle
et               29023   0.0  0.0  2643220   6052   ??  Ss   11:33AM   0:00.01 postgres: et los_test 127.0.0.1(59897) idle
et               89814   0.0  0.1  2653972   9488   ??  Ss   Tue06PM   0:00.63 postgres: et los_test ::1(59210) idle
et               59583   0.0  0.0  2626836    548   ??  Ss   Fri11AM   0:00.01 postgres: et los_test 127.0.0.1(58241) idle
et               59582   0.0  0.0  2626836    532   ??  Ss   Fri11AM   0:00.01 postgres: et los_test 127.0.0.1(58240) idle
et               59581   0.0  0.0  2626836    544   ??  Ss   Fri11AM   0:00.01 postgres: et los_test 127.0.0.1(58239) idle
et               59580   0.0  0.0  2618644    548   ??  Ss   Fri11AM   0:00.01 postgres: et los_test 127.0.0.1(58238) idle
et               54969   0.0  0.0  2635028    544   ??  Ss   26Jan18   0:00.01 postgres: et los_test 127.0.0.1(57001) idle
et               54968   0.0  0.0  2626836    552   ??  Ss   26Jan18   0:00.01 postgres: et los_test 127.0.0.1(57000) idle
et               54964   0.0  0.0  2635028    556   ??  Ss   26Jan18   0:00.01 postgres: et los_test 127.0.0.1(56995) idle
et               54963   0.0  0.0  2618644    568   ??  Ss   26Jan18   0:00.01 postgres: et los_test 127.0.0.1(56994) idle
et               54960   0.0  0.0  2635028    556   ??  Ss   26Jan18   0:00.01 postgres: et los_test 127.0.0.1(56991) idle
et               54959   0.0  0.0  2643220    588   ??  Ss   26Jan18   0:00.02 postgres: et los_test 127.0.0.1(56990) idle
et               54491   0.0  0.0  2610452    544   ??  Ss   26Jan18   0:00.01 postgres: et los_test 127.0.0.1(56606) idle
et               54490   0.0  0.0  2618644    548   ??  Ss   26Jan18   0:00.01 postgres: et los_test 127.0.0.1(56605) idle

Am I doing the setup and teardown properly?

DB passwords that contain non-URL-safe characters (e.g. hash marks) must be encoded so that pg can properly parse the connection string. Wrapping them in encodeURIComponent() should do the trick.

Nice lib, just checking it out for my needs to initialize databases that use knex migrations.

I noticed the reposiotry field in package.json is pointing to a fork rather than this repo. So for example the number of issues here is not reflected at npmjs.org. Just wanted to check if that is correct?

The devDependency pg was updated from 7.10.0 to 7.11.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

pg is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

I am using knex-db-manager's truncateDb method to remove data after every test. I noticed that when used in conjunction with Objection's knexSnakeCaseMappers, truncateDb does not actually do anything!

I believe this is happening is because of hard-coding of 'table_name' here, while knex is returning 'tableName' - https://github.com/Vincit/knex-db-manager/blob/master/lib/MySqlDatabaseManager.js#L134

Currently, I am removing the fields returning by knexSnakeCaseMappers when I pass knex config into the dbManager's config.

const { omit } = require('lodash')
const { knexSnakeCaseMappers } = require('objection')

// This file has knexSnakeCaseMappers spread into it
// { 
//    // other config props
//    ....knexSnakeCaseMappers()
// }
const knexConfig = require('./knexfile') 

const dbManager = require('knex-db-manager').databaseManagerFactory({
	knex: omit(knexConfig, Object.keys(knexSnakeCaseMappers())),
	dbManager: {...}
})

Is there any other way of dealing with this? Thank you!

I'd be happy to brainstorm and potentially add a PR, if necessary!

The devDependency coveralls was updated from 3.0.2 to 3.0.3.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

coveralls is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴