kofrasa / mingo Goto Github PK

The minified file of the library is currently 30+ kB. Is there a way to use only operators and functions which I need? can I compile my own version of the library with help of tools like rollup or webpack?

It'd be good to provide ES6 version of the lib, so people will be able to use tree shaking and remove code which they don't use

Similar to https://www.npmjs.com/package/mongo-query#querygetobj-key

From mongo documentation:

"Adds numbers together or adds numbers and a date. If one of the arguments is a date, $add treats the other arguments as milliseconds to add to the date." documentation link.

The current implementation just sums values, this change would make it work as intended

/**
   * Computes the sum of an array of numbers.
   *
   * @param obj
   * @param expr
   * @returns {Object}
   */
  $add (obj, expr) {
    var args = computeValue(obj, expr)
    var date = args.find(function(argument) {
      return argument instanceof Date;
    })
    if (date) {
      args = args.map(function(argument) {
        if (argument instanceof Date) {
          return argument.getTime()
        }
        return argument
      });
    }
    var reduced = reduce(args, function (acc, num) {
      return acc + num
    }, 0);
    if (date) {
      return new Date(reduced);
    }
    return reduced;
  },

Repro step:

// this works
mingo._internal().computeValue({l1: {l2: [{l3: [{ a: 666}]}]}}, '$l1.l2.l3.a');

// this throws an exception
mingo._internal().computeValue({l1: {l2: [{l3: [{ a: 666}]}, {l4: 'some prop'}]}}, '$l1.l2.l3.a');

Thrown exception:

TypeError: Cannot read property 'a' of undefined
    at getValue (path/to/project/node_modules/mingo/dist/mingo.js:3545:13)
    at _loop (path/to/project/node_modules/mingo/dist/mingo.js:3617:15)
    at resolve (path/to/project/node_modules/mingo/dist/mingo.js:3625:16)
    at path/to/project/node_modules/mingo/dist/mingo.js:3605:16
    at Array.map (native)
    at _loop (path/to/project/node_modules/mingo/dist/mingo.js:3604:21)
    at resolve (path/to/project/node_modules/mingo/dist/mingo.js:3625:16)
    at Object.computeValue (path/to/project/node_modules/mingo/dist/mingo.js:3840:12)
    at repl:1:19
    at sigintHandlersWrap (vm.js:22:35)

It seems like computeValue is trying to find a property a in 'some prop'.

It will be great to manager this library with bower.

Thanks for sharing the library :)

I started using mingo. It's been really great so far.
What I am missing (or don't understand to do) is, finding documents with string functions like: contains, startsWith, endsWith.

How do I do this more elegant (contains):

var result = Query.find(npm.rows, {$where: 'this.id.indexOf("express") !== -1'}).all();
// or
var result = Query.find(npm.rows, {id: /mocha/g}).all();

Given this document:
{ "key0" : [ { "key1" : [ "value" ] }, { "key1" : [ "value1" ] } ] }

This query will match using MongoDB version 3.2.12:
{ "key0.key1": { "$eq": "value" } }
However, mingo fails to match.

There are several places Lodash has functions which Underscore doesn't which could be used internally, and none the other way around.

E.g. the _.result causing #25 would be _.get in Lodash but Underscore has no equivalent.

Is there a possibility Underscore could be replaced?

After mingo package.json added "module": "index.js", webpack seem will pick index.js as the entry point and which is es6.

However, the UglifyJs will complain because it is not es6 ready???

ERROR in app-b0821b8c5ea25b9a55ff.js from UglifyJs
Unexpected token operator «=», expected punc «,» [app-b0821b8c5ea25b9a55ff.js:9081,28]

After I deep into line 9081, I find that UglifyJs complain about default value for arg = null

function each (obj, fn, ctx = null) {

I tried using babili-webpack-plugin instead of uglifyjs-webpack-plugin, which take me forever to compile my code.

My webpack setting

    rules: [
      {
        test: /\.jsx?$/,
        exclude: /(node_modules|bower_components)/,
        loader: 'babel-loader',
        query: {
          presets: [
            [
              'env',
              {
                targets: {
                  browsers: ['>1%', 'last 2 ChromeAndroid versions'],
                  uglify: true,
                },
              },
            ],
          ],
        },
      },

I also have tried include all node_modules into my babel transpile but some other packages will fail to transpile because some devDependencies babel plugins have not be installed.

I am not expert on importing or exporting es6 modules. And seems webpack is not prefect to choose es5 or es6 versions. Not sure how to enforce es5 version for browser and es6 version for node.

Any suggestion on how to use mingo with webpack?

Thanks

• Not yet try rollup, but that require me to convert all webpack configs...

There doesn't seem to be support for dot notation field names: http://docs.mongodb.org/manual/core/document/#dot-notation

You can currently get around by using the $elemMatch operator.

There should be an option to modify the input objects and add a reference to the lookup data, rather than return new (cloned) objects.

This way, if there has been a $project stage, we can safely opt-out of cloning again.

See #59.

Do you plan on adding support for create/update operations?

I was using a selector that filters based on dates:

{
  "postedAt":{
    "$gte":"2017-01-04T15:00:00.000Z",
    "$lt":"2017-01-14T14:59:59.999Z"
  }
}

As far as I can tell, this doesn't work properly when the postedAt field contains date strings (the resulting cursor just came up empty), but it did work when I converted postedAt to date objects.

Is this the expected behavior? Or am I missing something?

I use setup to change the default key from _id. Since the ES6 refactor this is no longer possible.

assert(typeof require('mingo').setup === 'function') Fails

The readme shows setup setup() If it was intended to be remove this should be updated.

I found out that this line is causing problem: some runTest tests that are suppose to fail are not failing because you are comparing 0 with 0 because isNaN('a string') or isNaN([1, 2]) returns true: all the tests that tries to compare strings or arrays are not valid.

repro step:
Try to modify any runTest test in test/string_operators.js or test/array_operators.js to make it fail (hint: it won't!).

I think the proper way to fix it is to do the following:

-          if (isNaN(actual) && isNaN(expected)) actual = expected = 0
+          if (actual !== actual && expected !== expected) actual = expected = 0

because NaN === NaN returns false.

There are 3 tests that fails with the suggested changes, 2 related to unicode characters and string operators, that I am not sure how to solve.

Expected: calling $slice with a single argument in mongodb, gets that many items from the array beginning from zero position.

Actual: Calling mingo with $slice and a single argument takes all items counting from that position in the array.

Your hasing function looks like the hashCode algorhythm. That correct?

Hypothetically

Would you be willing to have any dependencies?
Would you be willing to have different dependencies for the npm/node.js version?

• murmurhash3 (pure javascript) is about 140% as fast as hashCode.
• cityhash (c++ node.js module) is about 280% as fast as hashCode

Seeing this warning with webpack:

webpackHotDevClient.js:196./~/mingo/mingo.js
Critical dependencies:
24:59-66 require function is used in a way in which dependencies cannot be statically extracted
 @ ./~/mingo/mingo.js 24:59-66

webpackHotDevClient.js:196./~/mingo/dist/mingo.min.js
Critical dependencies:
1:5762-5769 require function is used in a way in which dependencies cannot be statically extracted
 @ ./~/mingo/dist/mingo.min.js 1:5762-5769

possibly related to webpack/webpack#2675 (comment) ?

Loving this project by the way! Thanks!

Hi there

I'm trying to use your beautiful library with the Jurassic c# library for executing javascript. It keeps telling me:

TypeError: undefined cannot be converted to an object - This is at line 40 in the library.

Does this absolutely need to work in the browser context?

Im running it in a pure javascript environment (kinda), with no browser or browser objects present...

Help!

I noticed a difference between mongodb and mingo behaviour:

// mongoDB
db.test.insert({l1: [{l2: [{ l3: 'level3', l3bis: 'level3bis'}]}, {l2bis: 'level2bis'}]});
db.test.aggregate([{$project: { l3: '$l1.l2.l3'}}]);
// ==> { "_id" : ObjectId("..."), "l3" : [ [ "level3" ] ] }

// mingo
mingo.aggregate(
    [{_id: 'some-id', l1: [{l2: [{ l3: 'level3', l3bis: 'level3bis'}]}, {l2bis: 'level2bis'}]}],
    [{$project: { l3: '$l1.l2.l3'}}]
);
// ==> [ { _id: 'some-id', l3: 'level3' } ]

Notice that l3 is not an array of array.

It is probably related to this line.

I don't consider this a problematic issue, I just wanted to let you know...

This looks like a great project, just need some more intro info, like what the purpose of your API is and how it compares. Does it work on all collection types or any deserialized JSON? Can it be used to query Firebase documents? Can it augment the Firebase query API?

otherwise it does not work with npm install --production

From experiments, MongoDB only allows one level of nesting when matching values in an array without explicit indexes in the selector.

> db.test.insert({
...     key0: [{
...       key1: [[[{key2: [{a:"value2"}, {a: "dummy"}, {b:20}]}]], {"key2": "value"}],
...       key1a: {key2a: "value2a"}
...     }]
...   })
WriteResult({ "nInserted" : 1 })
> // Test: query deeply nested value
> // 1. specify no array index
> db.test.find({"key0.key1.key2.a": "value2"}) // no match
>
> // 2. specify partial array index selector for nested value
> db.test.find({"key0.key1.0.key2.a": "value2"}) // no match
>
> // 3. specify full array index selector to deeply nested value
> db.test.find({"key0.key1.0.0.key2.a": "value2"}) // match
{ "_id" : ObjectId("583fad3a1828419a48d35fe1"), "key0" : [ { "key1" : [ [ [ { "key2" : [ { "a" : "value2" }, { "a" : "dummy" }, { "b" : 20 } ] } ] ], { "key2" : "value" } ], "key1a" : { "key2a" : "value2a" } } ] }

> // Test: query shallow nested value
> // 1. specify no array index
> db.test.find({"key0.key1.key2": "value"}) // match
{ "_id" : ObjectId("583fad3a1828419a48d35fe1"), "key0" : [ { "key1" : [ [ [ { "key2" : [ { "a" : "value2" }, { "a" : "dummy" }, { "b" : 20 } ] } ] ], { "key2" : "value" } ], "key1a" : { "key2a" : "value2a" } } ] }
>
> // 2. specify array index
> db.test.find({"key0.key1.1.key2": "value"}) // match
{ "_id" : ObjectId("583fad3a1828419a48d35fe1"), "key0" : [ { "key1" : [ [ [ { "key2" : [ { "a" : "value2" }, { "a" : "dummy" }, { "b" : 20 } ] } ] ], { "key2" : "value" } ], "key1a" : { "key2a" : "value2a" } } ] }

Matching whole object is also broken.

> // Test: match whole objects
> db.test.find({"key0.key1": [[{key2: [{a:"value2"}, {a: "dummy"}, {b:20}]}]]}) // match
{ "_id" : ObjectId("583fad3a1828419a48d35fe1"), "key0" : [ { "key1" : [ [ [ { "key2" : [ { "a" : "value2" }, { "a" : "dummy" }, { "b" : 20 } ] } ] ], { "key2" : "value" } ], "key1a" : { "key2a" : "value2a" } } ] }

> db.test.find({"key0.key1.0": [[{key2: [{a:"value2"}, {a: "dummy"}, {b:20}]}]]}) // match
{ "_id" : ObjectId("583fad3a1828419a48d35fe1"), "key0" : [ { "key1" : [ [ [ { "key2" : [ { "a" : "value2" }, { "a" : "dummy" }, { "b" : 20 } ] } ] ], { "key2" : "value" } ], "key1a" : { "key2a" : "value2a" } } ] }

> db.test.find({"key0.key1.0.0": [{key2: [{a:"value2"}, {a: "dummy"}, {b:20}]}]}) // match
{ "_id" : ObjectId("583fad3a1828419a48d35fe1"), "key0" : [ { "key1" : [ [ [ { "key2" : [ { "a" : "value2" }, { "a" : "dummy" }, { "b" : 20 } ] } ] ], { "key2" : "value" } ], "key1a" : { "key2a" : "value2a" } } ] }
>
> db.test.find({"key0.key1.0.0.key2": [{a:"value2"}, {a: "dummy"}, {b:20}]}) // match
{ "_id" : ObjectId("583fad3a1828419a48d35fe1"), "key0" : [ { "key1" : [ [ [ { "key2" : [ { "a" : "value2" }, { "a" : "dummy" }, { "b" : 20 } ] } ] ], { "key2" : "value" } ], "key1a" : { "key2a" : "value2a" } } ] }
>
> db.test.find({"key0.key1.0.0.key2": {b:20}}) // match
{ "_id" : ObjectId("583fad3a1828419a48d35fe1"), "key0" : [ { "key1" : [ [ [ { "key2" : [ { "a" : "value2" }, { "a" : "dummy" }, { "b" : 20 } ] } ] ], { "key2" : "value" } ], "key1a" : { "key2a" : "value2a" } } ] }

Tested with [email protected].

We currently do not handle these cases correctly. Using the sample data.

var Mingo = require("mingo");
var _ = Mingo._internal();

var fixtures = [
    [{"key0.key1.key2.a": "value2"}, [], "should not match without array index selector to nested value "],
    [{"key0.key1.0.key2.a": "value2"}, [], "should not match without enough depth for array index selector to nested value"],
    [{"key0.key1.0.0.key2.a": "value2"}, data, "should match with full array index selector to deeply nested value"],
    [{"key0.key1.0.0.key2": {b:20}}, data, "should match with array index selector to nested value at depth 1"],
    [{"key0.key1.1.key2": "value"}, data, "should match with full array index selector to nested value"],
    [{"key0.key1.key2": "value"}, data, "should match without array index selector to nested value at depth 1"],
    [{"key0.key1.1.key2": "value"}, data, "should match shallow nested value with array index selector"]
  ];

console.log("#Test 1: matching deep nested objects");
fixtures.forEach(function (f) {
   var query = f[0],
       expect = f[1],
       msg = f[2];
   var result = Mingo.find(data, query).all();
   console.log(_.isEqual(result, expect) + "> " + msg);
});
console.log(">>>>>>>>>>>>> finish test 1 <<<<<<<<<<<<<");

console.log("#Test 2: matching whole objects");
fixtures = [
    [{"key0.key1": [[{key2: [{a:"value2"}, {a: "dummy"}, {b:20}]}]]}, "should match full key selector"],
    [{"key0.key1.0": [[{key2: [{a:"value2"}, {a: "dummy"}, {b:20}]}]]}, "should match with key<-->index selector"],
    [{"key0.key1.0.0": [{key2: [{a:"value2"}, {a: "dummy"}, {b:20}]}]}, "should match with key<-->multi-index selector"],
    [{"key0.key1.0.0.key2": [{a:"value2"}, {a: "dummy"}, {b:20}]}, "should match with key<-->multi-index<-->key selector"]
  ];

fixtures.forEach(function (f) {
   var query = f[0],
       msg = f[1];
   var result = Mingo.find(data, query).all();
   console.log(_.isEqual(result, data) + "> " + msg);
});
console.log(">>>>>>>>>>>>> finish test 2 <<<<<<<<<<<<<");
// output below
/* #Test 1: matching deep nested objects 
false> should not match without array index selector to nested value
false> should not match without enough depth for array index selector to nested value
true> should match with full array index selector to deeply nested value
true> should match with array index selector to nested value at depth 1
false> should match with full array index selector to nested value
true> should match without array index selector to nested value at depth 1
false> should match shallow nested value with array index selector
>>>>>>>>>>>>> finish test 1 <<<<<<<<<<<<<
#Test 2: matching whole objects

/Users/francis/workspace/mingo/mingo.js:505
        throw new Error("Invalid query operator '" + operator + "' detected");
              ^
Error: Invalid query operator '0' detected
    at Object.Mingo.Query._processOperator (/Users/francis/workspace/mingo/mingo.js:505:15)
    at Object.Mingo.Query._compile (/Users/francis/workspace/mingo/mingo.js:493:22)
    at Object.Mingo.Query (/Users/francis/workspace/mingo/mingo.js:472:10)
    at Object.Mingo.find (/Users/francis/workspace/mingo/mingo.js:820:13)
    at /Users/francis/workspace/mingo/test/test.js:44:23
    at Array.forEach (native)
    at Object.<anonymous> (/Users/francis/workspace/mingo/test/test.js:41:10)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
*/

Test 1 does not pass all equality checks
Test 2 throws an exception

Tested with [email protected].

Lookup modifies the input object rather than return a new object with reference to the input data.

https://github.com/kofrasa/mingo/blob/master/lib/operators/pipeline/lookup.js#L43

Thanks for your package first. That is really amazing.

I found that the behavior of find({ someField: null }) is not same as mongodb

https://docs.mongodb.com/manual/tutorial/query-for-null-fields/

in which null should also match with undefined field.
Shall mingo align that with mongoDb spec? Or that is expected behavior?

BTW. I have done a little benchmark and found that is much faster than sift and within same level as nedb and lokijs. Thanks a lot.

If an array field in a document is null and we try to project it with a $slice, a type error is thrown.

Expected behaviour is to return null when the array value is null and undefined when the array value is undefined.

Is new mongo 3.2 $lookup support planned?

Hey there. first off: awesome stuff you did with mingo. However I encountered a bug while querying a nested array using $all and $elemMatch. I setup a testcase to illustrate the issue:

test("elemMatch on nested elements", function () {

  var testobject = [{
      user: { username: 'User1', projects: [{ name: "Project 1", rating: { complexity: 6 }}, { name: "Project 2", rating: { complexity: 2 }}] }
  },{
      user: { username: 'User2', projects: [{ name: "Project 1", rating: { complexity: 6 }}, { name: "Project 2", rating: { complexity: 8 }}] }
  }],
  query = {
    'user.projects': {
      "$all": [{
        "$elemMatch": {
          'rating.complexity': { '$gt' : 6 }
        }
      }]
    }
  };
  // It should return one user object
  ok(Mingo.compile(query).test(testobject));

});

Expected behavior: http://docs.mongodb.org/manual/reference/operator/query/all/#use-all-with-elemmatch

I'm not sure what's happening in lookup.js:

each(joinColl, (obj, i) => {
    let k = hashCode(obj[foreignField])
    hash[k] = hash[k] || []
    hash[k].push(i)
  })

  each(collection, (obj) => {
    let k = hashCode(obj[localField])
    let indexes = hash[k] || []
    obj[asField] = map(indexes, (i) => clone(joinColl[i]))
    result.push(obj)
  })

I'm confused by this specifically:

obj[asField] = map(indexes, (i) => clone(joinColl[i]))

Is the $lookup data being cloned, or added by reference?
Cloning, of course, is trivial for small datasets but huge (unnecessary) overhead on big datasets.

From the $addField mongo documentation

Adds new fields to documents. $addFields outputs documents that contain all existing fields from the input documents and newly added fields.

const { Aggregator } = require('mingo');
const oper = [{
  $addFields:{
    accountInfo:{
      $arrayElemAt:["$accounts", 0]
    }
  }
}];
const col = [{
  _id: "59c52580809dd0032d75238a",
  email: "[email protected]",
  accounts: [{
    createdAt: "2017-09-22T15:00:17.418Z",
    updatedAt: "2017-09-22T15:00:17.418Z",
  }],
}];
const aggregatedCollection = new Aggregator([oper]).run(col);
const expected = [{
  _id: "59c52580809dd0032d75238a",
  email: "[email protected]",
  accountInfo: {
    createdAt: "2017-09-22T15:00:17.418Z",
    updatedAt: "2017-09-22T15:00:17.418Z",
  },
  accounts: [{
    createdAt: "2017-09-22T15:00:17.418Z",
    updatedAt: "2017-09-22T15:00:17.418Z",
  }],
}];

// no accounts array
const got = [{
  _id: "59c52580809dd0032d75238a",
  email: "[email protected]",
  accountInfo: {
    createdAt: "2017-09-22T15:00:17.418Z",
    updatedAt: "2017-09-22T15:00:17.418Z",
  },
}];

Is support for geospatial queries planned and what are the expected limitations?

Hi there

I would love a feature to do the following: Test an arbitary object to see if it would pass or fail a "find" kind of expression. I'm sure you have it internally, would it be hard to expose to the API ?

var Mingo = require('mingo');
// or just access Mingo global in browser

// setup the key field for your collection
Mingo.setup({
key: '_id' // default
});

// create a query with criteria
// find all grades for homework with score >= 50
var result = Mingo.Test({ type: "homework", score: 55}, { $gte: 50 } });

if (result)
alert("Passed!");
else
alert("Failed!");

Is this possible?

If you have a nested projection, it fails to match:

var Mingo = require('mingo');

var db = [{ "name": "Steve", "age": 15, "features": { "hair": "brown", "eyes": "brown" } } ];

Mingo.find(db, {}, { "features.hair": 1 }).all(); // [ { } ]

In MongoDB, this would return [ { "_id" : ObjectId("57b86248bc6b3372ac511267"), "features" : { "hair" : "brown" } } ].

This might seem obvious, but the documentation uses the term collection everywhere without ever giving an example of what a collection should look like. I think having a quick illustration of what you mean (array? JS object? something else?) would be useful.

Hello,

According to the mongodb documentation, $concatArrys operator allows multiple elements.

But mingo only allows 2 elements:

repro step:

const data = [{professionalTags: ['tag1', 'tag2'], personalTags: ['tag2'], publicationTags: ['tag2']}];
mingo.aggregate(data, [
  { $project: {
    tags: {$concatArrays: ['$professionalTags', '$personalTags', '$publicationTags'] }
  }}
]);

this throws an exception:

$concatArrays expression must resolve to an array of 2 elements

      at err (node_modules/mingo/dist/mingo.js:140:9)
      at assert (node_modules/mingo/dist/mingo.js:40:26)
      at Object.$concatArrays (node_modules/mingo/dist/mingo.js:2438:5)
      at computeValue (node_modules/mingo/dist/mingo.js:3805:38)
      at node_modules/mingo/dist/mingo.js:3852:23
      at each (node_modules/mingo/dist/mingo.js:182:16)
      at computeValue (node_modules/mingo/dist/mingo.js:3851:7)
      at node_modules/mingo/dist/mingo.js:950:19
      at each (node_modules/mingo/dist/mingo.js:177:14)
      at node_modules/mingo/dist/mingo.js:914:5
      at each (node_modules/mingo/dist/mingo.js:177:14)
      at Object.$project (node_modules/mingo/dist/mingo.js:904:3)
      at node_modules/mingo/dist/mingo.js:1248:48
      at each (node_modules/mingo/dist/mingo.js:177:14)
      at Aggregator.run (node_modules/mingo/dist/mingo.js:1241:9)
      at Object.aggregate (node_modules/mingo/dist/mingo.js:1267:35)
      at Object.<anonymous>.exports.getAllSkills.elementals (src/selectors/index.js:10:26)
      at Object.it (src/selectors/__tests__/index.test.js:7:38)
      at Promise.resolve.then.el (node_modules/p-map/index.js:42:16)
      at process._tickCallback (internal/process/next_tick.js:109:7)

Note that removing one element ($publicationTags for example) doesn't throw.

It seems that there's no reason why aggregations can't be streamed, is there a chance support can be added for this?

If you try to do a $min aggregation on a string field in a collection you receive Infinity instead of the actual result. I still haven't tried, but I think that $max would work the same way. The issue is that _.min returns Ifinity when using strings.

Example data model:

var user = {
  Name: "Mingo",
  Age: 20,
  _id: 1
}

Using something along the line of var query = new Mingo.Cursor(collection, {}, {Name: 0}) I expect the result to be:

{
  Age: 20,
  _id: 1
}

instead it is:

{
  _id: 1
}

Hi,

It looks like mingo doesn't correctly return values in nested arrays. For example, with this data set:

var data = [{
    key0: [{
        key1: [ {key2: "value2"} ],
        key1a: {key2a: "value2a"}
    }]
}];

The expression {"key0.key1a.key2a": "value2a"} returns the object, while the expression {"key0.key1.key2": "value2"} returns an empty array. Am I constructing the query wrong?

Thanks,
Dan

this returns wrong result:

const ObjectId = require('mongoose').Types.ObjectId
const { Query } = require('mingo')
const id = ObjectId()

const q = new Query({ id })

console.log(q.test({ id: id.toString() })) //false
console.log(q.test({ id: ObjectId(id.toString()) })) //false

I expect both checks to return true.

Is there anyway to check equality of ObjectIds?

Meteor also has a pure-js implementation of (a large part of) the mongo API.

I know it's not as feature-complete (e.g. no aggregation), but I was wondering if there has been a performance comparison between the two?

I would be nice to know a little more about this project without having to dig into the code. The project description is only six words and the language makes the project seem like another Mongo Node driver.

It's not a Mongo Node driver per se, but more specifically a Mongo emulator. There's no MongoDB backend required because it's built right in (as I understand it). To me this wasn't clear from the description or the examples. It took a conversation on Reddit to clear up this fact.

When trying to find() a property that is nested in an array of arrays, the record is not returned if the child array has more than one element and one or more of those elements match.

Repro steps:

// no regex - returns expected list: 1 element - ok
mingo.find([{l1: [{ tags: ['tag1', 'tag2'] }, {'notags': 'yep'}]}], {'l1.tags': 'tag1'}).all()

// with regex - but searched property is not an array: ok
mingo.find([{l1: [{ tags: 'tag1'}, {'notags': 'yep'}]}], {'l1.tags': {$regex: '.*tag.*', $options: 'i'}}).all()

// with regex - but searched property is an array, with all elements matching: not ok - expected 1, returned 0
mingo.find([{l1: [{ tags: ['tag1', 'tag2'] }, {'tags': ['tag66']}]}], {'l1.tags': {$regex: 'tag', $options: 'i'}}).all();

// with regex - but searched property is an array, only one element matching: not ok - returns 0 elements - expected 1
mingo.find([{l1: [{ tags: ['tag1', 'tag2'] }, {'notags': 'yep'}]}], {'l1.tags': {$regex: 'tag', $options: 'i'}}).all();

// 

How one could implement $$ROOT variable as in here https://docs.mongodb.com/manual/reference/operator/aggregation/group/#group-documents-by-author?

It's in mongo since 2.6.

Do 'custom operators' fit for this?

I tried to implement it this way, but it doesn't work.

Mingo.addOperators(Mingo.OP_PIPELINE, function (m) {
  return {
    $$ROOT: function (obj) {
      return obj;
    }
  }
})

It throws the error: "Invalid operator name '$$ROOT'"

Hi,

I am playing with your framework, so far I love the way it works.. Anyway I want to "report" the following issue.

On the mongo shell I execute the following query:
db.XXX.aggregate([{"$match":{}},{"$group":{"_id":{"hour":"$date_buckets.hour","keyword":"$Keyword"},"total":{"$sum":1}}},{"$sort":{"total":-1}},{"$limit":5},{"$project":{"_id":0,"hour":"$_id.hour","keyword":"$_id.keyword","total":1}}])

• Notice the multiple field grouping.
  And this is the result:
  { "total" : 2, "keyword" : "Bathroom Cleaning Tips" }
  { "total" : 1, "keyword" : "best way to clean a bathroom" }
  { "total" : 1, "keyword" : "Cleaning Bathroom Tips" }
  { "total" : 1, "keyword" : "unclog bathtub drain" }
  { "total" : 1, "keyword" : "Drain Clogs" }

Same query applied using mingo
var agg = new Mingo.Aggregator(
[{
$match: {}
},{
$group: {
_id: {
hour: "$date_buckets.hour"
, keyword: "$Keyword"
},
total: {
$sum: 1
}
}
},{
$sort: {
total: -1
}
},{
$limit: 5
},{
$project: {
_id:0
, hour: "$_id.hour"
, keyword: "$_id.keyword"
, total:1
}
}]
);

And this is the result:

{ "total" : 6, "keyword" : "Bathroom Cleaning Tips" }
{ "total" : 6, "keyword" : "best way to clean a bathroom" }
{ "total" : 6, "keyword" : "Cleaning Bathroom Tips" }
{ "total" : 6, "keyword" : "unclog bathtub drain" }
{ "total" : 6, "keyword" : "Drain Clogs" }

This is the data I am using it contains 6 rows:
[{"date_buckets":{"date":"2015-04-29T00:17:03.107Z","day":28,"hour":18,"minute":17,"sec":3,"hour_minute":"18:17"},"Keyword ID":"sr3_4VzRD3sp","Creative ID":"5184986203","Keyword":"Bathroom Cleaning Tips","Match Type":"be","Device":"m","Conversions":[1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0],"Revenues":[1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0],"account_id":"baron"},{"date_buckets":{"date":"2015-04-29T00:17:03.107Z","day":28,"hour":18,"minute":17,"sec":3,"hour_minute":"18:17"},"Keyword ID":"sr3_K1iQOeXy","Creative ID":"5184986241","Keyword":"Cleaning Bathroom Tips","Match Type":"bb","Device":"c","Conversions":[5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,0,0,0],"Revenues":[5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,0,0,0],"account_id":"baron"},{"date_buckets":{"date":"2015-04-29T00:17:03.108Z","day":28,"hour":18,"minute":17,"sec":3,"hour_minute":"18:17"},"Keyword ID":"sr3_sl0C3VAYk","Creative ID":"44210589597","Keyword":"best way to clean a bathroom","Match Type":"b","Device":"c","Conversions":[4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0],"Revenues":[4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0],"account_id":"baron"},{"date_buckets":{"date":"2015-04-29T00:17:03.108Z","day":28,"hour":18,"minute":17,"sec":3,"hour_minute":"18:17"},"Keyword ID":"sr3_4VzRD3sp","Creative ID":"5184986204","Keyword":"Bathroom Cleaning Tips","Match Type":"be","Device":"c","Conversions":[1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0],"Revenues":[1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0],"account_id":"baron"},{"date_buckets":{"date":"2015-04-29T00:17:03.107Z","day":28,"hour":18,"minute":17,"sec":3,"hour_minute":"18:17"},"Keyword ID":"sr3_HZAarvKy","Creative ID":"6074827333","Keyword":"Drain Clogs","Match Type":"bp","Device":"c","Conversions":[1,0,0,1,0,0,0,0,0],"Revenues":[5,0,0,5,0,0,0,0,0],"account_id":"baron"},{"date_buckets":{"date":"2015-04-29T00:17:03.107Z","day":28,"hour":18,"minute":17,"sec":3,"hour_minute":"18:17"},"Keyword ID":"sr3_irU8fFk0","Creative ID":"6074827289","Keyword":"unclog bathtub drain","Match Type":"bp","Device":"c","Conversions":[1,0,0,1,0,0,0,0,0],"Revenues":[5,0,0,5,0,0,0,0,0],"account_id":"baron"}]

I will try to figure it out whats going on, but probably you are fastest than me.
Thanks, Robert Baron.

$group operations are insanely slow. Grouping 3000 documents to 25 properties from memory takes about 15 seconds on my laptop. This operation takes 0.1 second from disk in mongodb.

One of the problems seems to be that every OP_GROUP operators ($sum, $push, custom operators, etc) each iterate the collection for themselves. This means that 3000 documents are iterated 25 times.

The collection should be iterated only once, and every operator should work within that iteration.

Please consider the following situation:

// users
[
  {
    _id : ObjectId("1"),
    someKey : "someValue"
  }
]

    var query = new Mingo.Query({_id : 1});
    var cursor = query.find(users);
    return cursor.all(); // [] empty array, but it should contain {_id : ObjectId("1"), someKey : "someValue"}

Looking forward to this!

The MongoDB documentation for the $where operator says that it is evaluated after all other operators:

"MongoDB will evaluate non-$where components of query before $where statements. If the non-$where statements match no documents, MongoDB will not perform any query evaluation using $where."

https://docs.mongodb.com/manual/reference/operator/query/where/

However, mingo does not evaluate $where last.

One thing I find myself wishing for when using Mingo is the ability to define custom operators.

Without the ability to define custom operators, I find myself wrapping aggregators in my own functions with aggregator-like APIs so that I can perform unsupported mutations or actions on data in such a way that the logic is centralized in my application and I don't repeat myself needlessly. If given the ability to define custom operators, the need for wrapping of Mingo's core functions in other functions may remove itself, and Mingo can be used as an implementation of a domain specific language that's very MongoDB-like.

I've given a look at the source code, and it appears that exposing the Ops and *Operators objects via a nice API should do the trick. Of course, users of the extension API need to realize that they might break things severely, but ensuring that they don't would be their own responsibility.

I was wondering what your thoughts were on this. Do you think this is within scope of the project?

Hi,

This works (on NodeJS 7.7):

  let lowScore = {type: "homework", score: 30};
  let highScore = {type: "homework", score: 80};
  mingo.setup({});
  let query = new mingo.Query({
    type: "homework",
    score: { $gte: 50 }
  });
  let isLowScore = query.test(lowScore);
  console.log("Score Above 50 ", isLowScore);
  let isHighScore = query.test(highScore);
  console.log("Score Above 50 ", isHighScore);

and produces:

Score Above 50  false
Score Above 50  true

The below throws an error Criteria must be of type Object. The issue is obviously that criteria is initialized as a string and not an object. In my use case I need to read the criteria from a database and from the database it comes back as a string. Is it possible to to this? I know it might be trivial question.

  let lowScore = {type: "homework", score: 30};
  let highScore = {type: "homework", score: 80};
  let criteria = '{type: "homework", score: { $gte: 50 } }';
  mingo.setup({});
  let query = new mingo.Query(criteria);
  let isLowScore = query.test(lowScore);
  console.log("Score Above 50 ", isLowScore);
  let isHighScore = query.test(highScore);
  console.log("Score Above 50 ", isHighScore);

Thanks
Leon