campbellsoftwaresolutions / mongoose-id-validator Goto Github PK

Hey i'm building a trailsjs based application with a mongo db managed via the trailpack-mongoose addon.

The current release of this addon uses mongoose ^4.7.7.

When i install this plugin in one of my trails models like so:

...
class MyModel extends Model {
  static config() {
    let config  = {}
    config.onSchema = (app, schema)=>{
      schema.plugin(idvalidator);
    }
    return config
  }
  ...

any calls to save a document hang.

let item = new MyModel(object);
item.save().then(neverGetsCalled)

I've been through the internals of the plugin and mongoose. and it seems to be an issue with the exec method in mongoose/lib/query.js on line 3087 that is called by the executeQuery method of this library . If the callback is passed as a second parameter to exec, as this library does, it is called by an internal promise resolve call, but the promise never resolves. I've tried changing this library to use a promise instead of passing a callback, but again it never resolves.

Any help would be greatly appreciated. I find it hard to imagine that this is an unknown mongoose issue with such an important method as query.exec

{
.....
country: { type: mongoose.Schema.Types.ObjectId, ref: 'Countries', required: true },
city: { type: mongoose.Schema.Types.ObjectId, ref: 'Countries.cities' }
.....
}

Validation failed: city: Schema hasn't been registered for model "Countries.cities".\nUse mongoose.model(name, schema)

My schema definition looks like this:

const groupSchema = new mongoose.Schema({
  name: {
    type: String,
    trim: true,
  },
  members: {
    type: [String],
    ref: 'Profile',
    refConditions: inSameOrgRefCondition,
    set: idListSetter,
  },
});

This results in a schemaType that looks like this in [email protected].

SchemaArray {
  casterConstructor:
   { [Function: SchemaString]
     schemaName: 'String',
     _cast: [Function: castString],
     cast: [Function: cast],
     get: [Function],
     _checkRequired: [Function],
     checkRequired: [Function] },
  caster:
   SchemaString {
     enumValues: [],
     regExp: null,
     path: 'members',
     instance: 'String',
     validators: [],
     getters: [],
     setters: [],
     options: {},
     _index: null },
  '$isMongooseArray': true,
  path: 'members',
  instance: 'Array',
  validators: [],
  getters: [],
  setters: [ [Function: idListSetter], [Function: membersSetter] ],
  options:
   { type: [ [Function: String] ],
     ref: 'Profile',
     refConditions: { org: [Function: orgFromDocument] },
     set: [Function: idListSetter] },
  _index: null,
  defaultValue: { [Function: defaultFn] '$runBeforeSetters': true },
  isRequired: true,
  requiredValidator: [Function],
  originalRequiredValue: true }

The following code doesn't recognize the members field as an array:

        if (schemaType.options && schemaType.options.ref) {
            validateFunction = validateId
            refModelName = schemaType.options.ref
            if (schemaType.options.refConditions) {
                conditions = schemaType.options.refConditions
            }
        } else if (schemaType.caster && schemaType.caster.instance &&
            schemaType.caster.options && schemaType.caster.options.ref) {
            validateFunction = validateIdArray
            refModelName = schemaType.caster.options.ref
            if (schemaType.caster.options.refConditions) {
                conditions = schemaType.caster.options.refConditions
            }
        }

After updating to the latest mongoose v5.6.13, I noticed that a bunch of my tests were failing. Turns out they were all due to this plugin no longer functioning properly. It appears that the plugin is no longer working when inserted at the global or schema level.

After cloning the repo and running the tests, I discovered that the following are failing for mongoose ^5.6.0:

1. Should fail validation with custom message on bad ID
2. Should fail validation on bad ID with IdValidator instance
3. Should fail validation if bad ID set after previously good ID value
4. Should fail ID validation in an array of ID references

(node:419) DeprecationWarning: Mongoose: the isAsync option for custom validators is deprecated. Make your async validators return a promise instead: https://mongoosejs.com/docs/validation.html#async-custom-validators

"mongoose": "5.5.1"
"mongoose-id-validator": "0.5.2"

I have a datastructure which is defined thus:

var productSchema = new mongoose.Schema({
name: String,
users: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
required: true
}],
});

var userSchema = new.mongoose.Schema({
username: {
type: String,
required: true,
},
});

/* next piece of code */

badProductBadUser = {
name: 'this should not work',
users: [ { user_id: 'deadbeef667', }, ],
}

/* above product values succeed, when I believe they should not */

Does this make sense?

I am posting issue that first was opened on mongoose github, but realized it has nothing with mongoose, because we use this plugin for bug that is happening

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
I am using version 5.0.1 and tried to update it to latest version, but then some of tests in my code started failing because monogoose stop doing some checks.
What I am doing is that I have schema A with a field arrayOfUsers that is an array of ObjectIds from schema Users.
It is used as a reference and it was doing population when I fetch objects from collection A.

How it was working until version 5.2.14 is if I try to add some new ObjectId in arrayOfUsers, before save(), it was checking does object with new id exist in collection users, and if it does not exist, it would throw me an error.

If the current behavior is a bug, please provide the steps to reproduce.

I am using typescript to show you an example

Schema A

const ASchema: Schema = new Schema(
    {
 
        type: {
            type: String
        },
        arrayOfUsers: {
            type: [UsersSchema]
        },
        private: {
            type: Boolean
        }
    }
);

// hooks to catch error if wrong id is provided
BaseChatSchema.post('save', (err: any, doc: ADocument, next: Function) => {
    /**
     * if there is an error with arrayOfUsers property (usually user with incorrect id)
     * throw an error
     */
    if (err && err.errors && err.errors.hasOwnProperty('arrayOfUsers')) {
        const mongoError: MongoError = new MongoError('user_not_found');
        return next(mongoError);
    }

    // Additional check if any user ID is wrong
    // this is our method to extract error from thrown mongo's error
    if (KError.hasFirstMongoErrorInProperty(err, 'user')) {
        const mongoError: MongoError = new MongoError('user_not_found');
        return next(mongoError);
    }

    return next(err);
});

Users Schema

export const ChatParticipantSchema: Schema = new Schema(
    {
        user: {
            type: ObjectId,
            ref: ModelName.Users,
            required: [true, ErrorType.UserIdRequired]
        },
        lastVisited: { 
            type: Date,
            default: Date.now
        }
    },
    {
        // Don't create id for this sub-schema (array in base chat model)
        _id: false
    }
);

Easy way to reproduce it is to get one document from collection A, and then to do

fetchedDocument.arrayOfUsers.push(new ObjectId());
try {
    await fetchedDocument.save();
} catch (e) {
    console.log(e)
}

Until pointed version, it was throwing an error, but now, it saves document without throwing error

What is the expected behavior?

Expected behavior is that mongoose will check does object in User collection exist with provided ObjectID and to throw error if it does not exist.

Please mention your node.js, mongoose and MongoDB version.
node 8.13
mongoose 5.2.14 when bug occured (5.0.1 is with expected behavior)
mongo 3.6.2

I am trying to run some Mocha tests that all passed a few months ago, but they are all failing now. I have narrowed the problem down to mongoose-id-validator, which is throwing a MissingSchemaError for a schema I have confirmed is loaded.

Here is a simplified version of my test. The user schema references the _id field of event schema and when I attempt to save the user schema, the MissingSchemaError is thrown:

var should = require('should'),
    mongoose = require('mongoose');

console.log(mongoose.modelNames());
console.log(mongoose.model('Event').schema);

var Evnt = mongoose.model('Event'),
    User = mongoose.model('User'),
    Attendees = mongoose.model('Attendees');

var user, evnt;

describe("Model Unit Tests:", function() {
     before(function(done) {
          evnt = new Evnt({
               //Event definition
          });

          evnt.save(function() {
               user = new User({
                    //User definition
               });

               user.save(function() {
                    //MissingSchemaError thrown for model Event.
               });
          });
     });

The result of the console.log(mongoose.modelNames()) is:

[ 'Attendees', 'Candidate', 'Comment', 'Event', 'User' ]

The result of the console.log(mongoose.model('Event').schema) is:

{ paths: 
   { name: 
      { enumValues: [],
        regExp: null,
        path: 'name',
        instance: 'String',
        validators: [Object],
        setters: [Object],
        getters: [],
        options: [Object],
        _index: null },
     start_date: 
      { path: 'start_date',
        instance: 'Number',
        validators: [Object],
        setters: [],
        getters: [],
        options: [Object],
        _index: null },
     end_date: 
      { path: 'end_date',
        instance: 'Number',
        validators: [Object],
        setters: [],
        getters: [],
        options: [Object],
        _index: null },
     location: 
      { enumValues: [],
        regExp: null,
        path: 'location',
        instance: 'String',
        validators: [Object],
        setters: [Object],
        getters: [],
        options: [Object],
        _index: null },
     schedule: 
      { enumValues: [],
        regExp: null,
        path: 'schedule',
        instance: 'String',
        validators: [],
        setters: [Object],
        getters: [],
        options: [Object],
        _index: null,
        defaultValue: 'No schedule specified' },
     _id: 
      { path: '_id',
        instance: 'ObjectID',
        validators: [],
        setters: [Object],
        getters: [],
        options: [Object],
        _index: null,
        defaultValue: [Function: defaultId] },
     __v: 
      { path: '__v',
        instance: 'Number',
        validators: [],
        setters: [],
        getters: [],
        options: [Object],
        _index: null } },
  subpaths: {},
  virtuals: { id: { path: 'id', getters: [Object], setters: [], options: {} } },
  nested: {},
  inherits: {},
  callQueue: [ [ 'pre', [Object] ] ],
  _indexes: [ [ [Object], [Object] ] ],
  methods: {},
  statics: {},
  tree: 
   { name: { validate: [Object], trim: true, type: [Function: String] },
     start_date: { validate: [Object], type: [Function: Number] },
     end_date: { validate: [Object], type: [Function: Number] },
     location: { validate: [Object], trim: true, type: [Function: String] },
     schedule: 
      { default: 'No schedule specified',
        trim: true,
        type: [Function: String] },
     _id: { auto: true, type: [Function: ObjectId] },
     id: { path: 'id', getters: [Object], setters: [], options: {} },
     __v: [Function: Number] },
  _requiredpaths: undefined,
  discriminatorMapping: undefined,
  _indexedpaths: undefined,
  options: 
   { id: true,
     noVirtualId: false,
     _id: true,
     noId: false,
     read: null,
     shardKey: null,
     autoIndex: true,
     minimize: true,
     discriminatorKey: '__t',
     versionKey: '__v',
     capped: false,
     bufferCommands: true,
     strict: true,
     pluralization: true },
  _events: {} }

Obviously the schema is defined before the user model is saved. Here is the error message I receive:

  1) Model Unit Tests: "before all" hook:
     Uncaught MissingSchemaError: Schema hasn't been registered for model "Event".
Use mongoose.model(name, schema)
      at Mongoose.model (/home/c1moore/frankrs/node_modules/mongoose-id-validator/node_modules/mongoose/lib/index.js:323:13)
      at validateId (/home/c1moore/frankrs/node_modules/mongoose-id-validator/lib/id-validator.js:55:29)
      at EmbeddedDocument.<anonymous> (/home/c1moore/frankrs/node_modules/mongoose-id-validator/lib/id-validator.js:33:17)
      at /home/c1moore/frankrs/node_modules/mongoose/lib/schematype.js:623:19
      at Array.forEach (native)
      at ObjectId.SchemaType.doValidate (/home/c1moore/frankrs/node_modules/mongoose/lib/schematype.js:614:19)
      at /home/c1moore/frankrs/node_modules/mongoose/lib/document.js:974:9
      at process._tickDomainCallback (node.js:486:13)

After an hour of head-scratching I found that when using an update method on a model such as .findByIdAndUpdate, validators do not trigger automatically (.save method works fine).

Therefor it is necessary to set an option parameter of { runValidators: true } when using .findByIdAndUpdate

var opt = { runValidators: true };
Model_Name.findByIdAndUpdate(req.params.docID, { $set: req.body }, opt, (err, model)) => {
  /* do something */
}

This isn't a bug for the initiated but since Stackoverflow and Google didn't turn anything up I thought I'd add this here in case anybody else runs into the same issue.

I found the solution in the MongooseJS docs here http://mongoosejs.com/docs/validation.html (under Update Validators)

Update Validators

In the above examples, you learned about document validation. Mongoose also supports validation for update() and findOneAndUpdate() operations. In Mongoose 4.x, update validators are off by default - you need to specify the runValidators option.

To turn on update validators, set the runValidators option for update() or findOneAndUpdate(). Be careful: update validators are off by default because they have several caveats.

I'm using v0.1.10 in my project,

Here is my code:
var playerSchema = new Schema({ weapon: { type: ObjectId, ref: 'Weapons' }, items: [{ type: ObjectId, ref: 'Items' }] }); playerSchema.plugin(idValidator);

I put two items in the items list:

player.items = [ '56c3236161cc8225002df141', '56c3236161cc8225002df141' ]

when I try to save this player. The backend told me that one ID in player.items is non-exist.
I am 100% sure 56c3236161cc8225002df141 is in my items collection.

I trace down the problem:
function validateIdArray(doc, connection, refModelName, values, conditions, respond) { if (values == null || values.length == 0) { return respond(true); } var refModel = connection.model(refModelName); var query = refModel.count().where('_id')['in'](values); executeQuery(query, conditions, values.length, respond); }

The problem should be here.
executeQuery(query, conditions, values.length, respond);
values.length in my case is 2.

Inside the function:
function executeQuery(query, conditions, validateValue, respond) { for (var fieldName in conditions) { query.where(fieldName, conditions[fieldName]); } query.exec(function (err, count) { if (err) { return respond(err); } respond(count === validateValue); }); }

respond(count === validateValue);

However, count is only 1 if the array contain 2 same ObjectId.

I try to modified the code:
`function validateIdArray(doc, connection, refModelName, values, conditions, respond) {
if (values == null || values.length == 0) {
return respond(true);
}
var refModel = connection.model(refModelName);

var unique = function(arr) {
  var n = {},r=[];
  for(var i = 0; i < arr.length; i++)  {
     if (!n[arr[i]]) {
         n[arr[i]] = true; 
        r.push(arr[i]); 
    }
}
return r;
}

var uniqueValues = unique(values);
var query = refModel.count().where('_id')['in'](uniqueValues);
executeQuery(query, conditions, uniqueValues.length, respond);

}`

Then it works as normal.
I guess the biggest problem is duplicate ObjectId in the same array.

I'm using v0.1.8 in my project, and somehow the validation is not happening with arrays referencing other collections.

I have this userschema which references items in a watchlist.

var userSchema = new Schema({
    firstName: {type: String, required: 'First name required!'},
    lastName: {type: String, required: 'Last name required!'},
    _university: {type: ObjectId, ref: 'University'},
    watchlist: [{type: ObjectId, ref: 'Item', unique: true}],
});
userSchema.plugin(idvalidator);

The problem is, that inserting an id that is valid mongo object id but doesn't exist, will fail to validate and actually insert.

I'm using this function to insert the id's.

function pushWatchlist(userId, itemId) {
    var update = {$addToSet: {watchlist: itemId}};
    var options = {new: true, runValidators: true};
    return User.findByIdAndUpdate({_id: userId}, update, options).exec();
}

mongoose 7 no longer accepts a callback if you use it with lean().exec()

I'm using mongoose-id-validator in a FeathersJS project, and I ran this issue when trying to PATCH to a service. Apparently, a Mongoose Query object can have an undefined isModified field (although this is not exactly specified in the v4.75 API).

I was able to fix this by checking isModified in line 61 to see if it's falsey. Will create a pull request.

The specific error I received:

info: TypeError: this.isModified is not a function
at Query. (/home/bekher/redacted/node_modules/mongoose-id-validator/lib/id-validator.js:62:26)
at /home/bekher/redacted/node_modules/mongoose/lib/schematype.js:729:35
at Array.forEach (native)
at ObjectId.SchemaType.doValidate (/home/bekher/redacted/node_modules/mongoose/lib/schematype.js:710:19)
at /home/bekher/redacted/node_modules/mongoose/lib/services/updateValidators.js:70:20
at /home/bekher/redacted/node_modules/async/internal/parallel.js:27:9
at eachOfArrayLike (/home/bekher/redacted/node_modules/async/eachOf.js:57:9)
at exports.default (/home/bekher/redacted/node_modules/async/eachOf.js:9:5)
at _parallel (/home/bekher/redacted/node_modules/async/internal/parallel.js:26:5)
at parallelLimit (/home/bekher/redacted/node_modules/async/parallel.js:85:26)
at /home/bekher/redacted/node_modules/mongoose/lib/services/updateValidators.js:89:5
at Query._execUpdate (/home/bekher/redacted/node_modules/mongoose/lib/query.js:2054:7)
at /home/bekher/redacted/node_modules/kareem/index.js:239:8
at /home/bekher/redacted/node_modules/kareem/index.js:18:7
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickCallback (internal/process/next_tick.js:98:9)

https://mongoosejs.com/docs/populate.html#dynamic-ref

Mongoose supports the ref being specified in another path of the document. It'd be great for this library to also support that.

As of MongoDB Node.JS driver version 3.1, collection.count is deprecated (http://mongodb.github.io/node-mongodb-native/3.1/api/Collection.html#count)

This causes a deprecation warning when the validator runs:
DeprecationWarning: collection.count is deprecated, and will be removed in a future version. Use collection.countDocuments or collection.estimatedDocumentCount instead

Used here:

Should change implementation to use collection.countDocuments or collection.estimatedDocumentCount.

I'm trying to implement mongoose-id-validator in a NestJS project. When I do, the creation of the object gives me a timeout of more than 5 seconds, both when the IDs are valid and when they are not valid.

import * as idValidator from 'mongoose-id-validator';

EventSchema.plugin(idValidator);

{type: [mongoose.Schema.Types.ObjectId], required: true, ref: 'User'}

any plans on TypeScript support?

Like the title says, it crashes when using recursive schemas.
The following code produces the error below.

Code:

var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/test', { useMongoClient: true });
mongoose.Promise = require('bluebird');

var idValidator = require('mongoose-id-validator');

var Tasks = new mongoose.Schema();
Tasks.add({
   title     : String
 , subtasks  : [Tasks]
});

Tasks.plugin(idValidator);

Error:

c:\(...)\test\node_modules\mongoose-id-validator\lib\id-validator.js:33
    schema.eachPath(function (path, schemaType) {
                                              ^
RangeError: Maximum call stack size exceeded
    at c:\(...)\test\node_modules\mongoose-id-validator\lib\id-validator.js:33:30
    at Schema.eachPath (c:\(...)\test\node_modules\mongoose\lib\schema.js:673:5)
    at IdValidator.validateSchema (c:\(...)\test\node_modules\mongoose-id-validator\lib\id-validator.js:33:12)
    at c:\(...)\node_modules\mongoose-id-validator\lib\id-validator.js:37:27
(last 3 lines repeated 4 more times)

Great plugin anyway.

Hi -- I was using the id validator for an application that I am working on. One of my entities had the following attribute:

    RanksAllowed: [{
        type: mongoose.Schema.ObjectId, 
        ref:'Rank'
    }]

In my unit tests I was testing the case when this attribute is null and the id-validator was causing a crash. I implemented a small fix for this #2
Let me know what you think. Thank you.

Explanation

Mongoose allows the field ref to be any of:

• string
• model
• a function that returns string
• a function that returns model

Expected Behavior

When ref is a function, the plugin executes the function
When ref is a model, the plugin takes the name of the model

Actual Behavior

The plugin uses the toString function the value on ref to get the collection name

We have the following schema:

const DocumentSchema = new mongoose.Schema({
  ...
  account: { type: mongoose.Schema.Types.ObjectId, ref: 'Accounts', required: true, index: true }
  items: [SubDocumentSchema]
}, { timestamps: true })

DocumentSchema.plugin(mongooseIdValidator)

const SubDocumentSchema = new mongoose.Schema({
  ...
  user: { type: mongoose.Schema.Types.ObjectId, ref: 'Users' }
  product: { type: mongoose.Schema.Types.ObjectId, ref: 'Products' }
}, { timestamps: true })

As you can see, we try to verify the references in the DocumentSchema but NOT in the SubDocumentSchema, thus we set the plugin only on DocumentSchema. Based on the business requirements, we need to store a subdocument having a product id no matter that product id is valid or not (might not exist).

However, as we apply the plugin on DocumentSchema (to make sure the account field is valid), it also verifies all references in SubDocumentSchema, which we really do NOT want and for that reason we have NOT set the plugin on SubDocumentSchema.

Any way to stop propagation ?

Hi,
it seems I was unable to use the plugin when I changed my code to use multiple db connections (e.g. the ones created with .createConnection).

I think the problem is solvable passing the connection to the plugin, and change the lines where the default mongoose connection is used as var refModel = mongoose.model(refModelName); to use the specific model connection object.

Is there something I missed to make it work as it is, or do you think that this change would be reasonable?

Hi,

I've had to change my application to use "let conn = mongoose.createConnection" it's change created a bug in my application, the initial request stay in loading and doesn't return the response.

When i've used 'mongoose.connect' this plugin works very well!

(node:29759) DeprecationWarning: Implicit async custom validators (custom validators that take 2 arguments) are deprecated in mongoose >= 4.9.0. See http://mongoosejs.com/docs/validation.html#async-custom-validators for more info.

"mongoose": "^4.9.2",
"mongoose-id-validator": "^0.4.1",

Plugin doesn't run validation for arrays of Object Id

Can this library prevents deletion if the reference is used on another document?

I have another issue, that is causing my project problems.

I have this universitySchema

var universitySchema = new Schema({
    name: {type: String, required: 'Name required'}
});

Again I have my userSchema

var userSchema = new Schema({
    firstName: {type: String, required: 'First name required!'},
    lastName: {type: String, required: 'Last name required!'},
    _university: {type: ObjectId, ref: 'University'},
    watchlist: [{type: ObjectId, ref: 'Item', unique: true}],
});
userSchema.plugin(idvalidator);

As you can see, the _university is not required. So the user can sign up with a null value in the _university field. The problem is, that if he was signed up with a university or chooses one later when he edits his profile, and then later on updates his profile listing he is now not in a university anymore, the mongoose-id-validator will validate the field and throw an error, because obviously "null" does not exist in the Universities collection. The problem is, I want to allow that to the user.

How do I solve this problem?

It would be awesome if one could not only specify to check whether the ObjectId reference exists but if it does it would also immediately populate the document!

As this library already makes a query to the DB I thought it might be the right place to introduce an option to populate the query result to the response.

At the moment I need to use this validator + another populate from mongoose. So in total 2 queries are executed, which could have been handled with 1.

I wanted to ask whether you would be interested in such a feature and if yes I could provide a PR

When using $pull on an array. if it's the last item in the list it breaks.

{...., "name":"ValidatorError","properties":{"isAsync":true,"message":"{PATH} references a non existing ID","type":"user defined", .... }