Giter Club home page Giter Club logo

meteor-reactive-aggregate's Introduction

meteor-reactive-aggregate

Reactively publish aggregations.

meteor add jcbernack:reactive-aggregate

This helper can be used to reactively publish the results of an aggregation.

Usage

ReactiveAggregate(subscription, collection, pipeline[, options])
  • subscription should always be this in a publication.
  • collection is the Mongo.Collection instance to query.
  • pipeline is the aggregation pipeline to execute.
  • options provides further options:
    • observeSelector can be given to improve efficiency. This selector is used for observing the reactive collections. If you wish to have different selectors for multiple reactive collections, use lookupCollections options.
    • observeOptions can be given to limit fields, further improving efficiency. Ideally used to limit fields on your query.
    • delay (default: 250) the time (in milliseconds) between re-runs caused by changes in any reactive collections in the aggregation.
    • lookupCollections is keyed by your $lookup.from collection name(s). it takes observeSelector and observeOptions parameters. see example below. If none is given any change to the collection will cause the aggregation to be reevaluated.

Examples

options applied to the default collection

const options = {
    observeSelector: {
        bookId: { $exists: true },
    },
    observeOptions: {
        limit: 10,
        sort: { createdAt: -1 },
    }
};

options applied to all $lookup reactive collections

const options = {
    lookupCollections: {
        'books': {
           observeSelector: {
              'releaseDate': { $gte: new Date('2010-01-01') }
           },
           observeOptions: {
               limit: 10,
               sort: { createdAt: -1 },
           }
        }
    }
};
  • clientCollection defaults to collection._name but can be overriden to sent the results to a different client-side collection.

Multiple collections observe

By default, any collection instances passed into the aggregation pipeline as a Mongo.Collection instance will be reactive. If you wish to opt out of reactivity for a collection in your pipeline, simply pass the Collection._name as a string.

Example

All collections reactive:

const pipeline = [{
    $lookup: {
        from: Books,
        localField: 'bookId',
        foreignField: '_id',
        as: 'books',
    },
    ...
    $lookup: {
        from: Authors,
        localField: 'authorId',
        foreignField: '_id',
        as: 'authors',
    },
    ...
}];

Only Books collection is reactive:

const pipeline = [{
    $lookup: {
        from: Books,
        localField: 'bookId',
        foreignField: '_id',
        as: 'books',
    },
    ...
    $lookup: {
        from: Authors._name,
        localField: 'authorId',
        foreignField: '_id',
        as: 'authors',
    },
    ...
}];

Quick Example

A publication for one of the examples in the MongoDB docs would look like this:

import { ReactiveAggregate } from 'meteor/jcbernack:reactive-aggregate';

Meteor.publish("booksByAuthor", function () {
    ReactiveAggregate(this, Books, [{
    $group: {
        _id: "$author",
        books: { $push: "$$ROOT" }
    }
    }]);
});

Extended Example

Define the parent collection you want to run an aggregation on. Let's say:

Reports = new Meteor.Collection('Reports');

.. in a location where all your other collections are defined, say lib/collections.js

Next, prepare to publish the aggregation on the Reports collection into another client-side-only collection we'll call, clientReport.

Create the clientReport in the client side (its needed only for client use). This collection will be the destination in which the aggregation will be put into upon completion.

Now you publish the aggregation on the server:

import { ReactiveAggregate } from 'meteor/jcbernack:reactive-aggregate';

Meteor.publish("reportTotals", function() {
    // Remember, ReactiveAggregate doesn't return anything
    ReactiveAggregate(this, Reports, [{
        // assuming our Reports collection have the fields: hours, books
        $group: {
            '_id': this.userId,
            'hours': {
            // In this case, we're running summation.
                $sum: '$hours'
            },
            'books': {
                $sum: 'books'
            }
        }
    }, {
        $project: {
            // an id can be added here, but when omitted,
            // it is created automatically on the fly for you
            hours: '$hours',
            books: '$books'
        } // Send the aggregation to the 'clientReport' collection available for client use
    }], { clientCollection: "clientReport" });
});

We therefore need to subscribe to the above Publish.

Meteor.subscribe("reportTotals");

Then in our Template helper:

Template.statsBrief.helpers({
    reportTotals: function() {
        console.log("I'm working");
        return clientReport.find();
    },
});

Finally, your template:

{{#each reportTotals}}Total Hours: {{hours}} <br/>Total Books: {{books}}{{/each}}

Your aggregated values will therefore be available in client-side and behave reactively just as you'd expect.

Enjoy aggregating reactively!

Multiple collections observe example

import { ReactiveAggregate } from 'meteor/jcbernack:reactive-aggregate';

Meteor.publish("booksByAuthor", function () {
    ReactiveAggregate(this, Books, [{
    $group: {
        _id: "$author",
        books: { $push: "$$ROOT" }
    }
    }], {
    observeSelector: {
        `${Books._name}`: {
            authorId: { $exists: true },
        }
    }, // for Books
        // for Authors get default: {}
    // observeOptions: {} <- default: all reactive collections get no query options
    );
});

meteor-reactive-aggregate's People

Contributors

andrew1431 avatar jcbernack avatar mrauhu avatar pdshah3690 avatar qualitymanifest avatar rkstar avatar seanmavley avatar tiagopaes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

meteor-reactive-aggregate's Issues

Subscribe.ready() doesn't turn true with empty results

Hi there,

I'm opening this issue as follow-up to Issue #11 regarding the subscription .ready() method. My Meteor.subscribe('some_aggregate_pub').ready() always returns false if the aggregate has no $match results. Is there a work-around to check if there are no results and the subscription is ready?

Thank you!

how to get a correct subscription.ready() ?

First of all, this package is great.Thanks. But I have trouble when getting correct subscription ready() value. The ready() is always kept to false status. How can I know the subscription is ready?

Problem with "$lookup"?

I use $lookup aggregate function in Mongo 3.2.

ReactiveAggregate(this, Disbursement, [
        {$match: {_id: _id}},
        {
            $lookup: {
                from: "loan_client",
                localField: "clientId",
                foreignField: "_id",
                as: "clientDoc"
            }
        },
        {$unwind: "$clientDoc"},
.............

It don't work when I tried to change client info.

Problem with Methods deletetion

When I delete a record by Meteor Methods, the reactive don't push the update.
When a do something directly in mongodb, it does normally. I forgot some configuration?

Meteor does not currently support objects other than ObjectID as ids

Hi,

Trying to use this plugin for the first time.

Upon publishing I get the following error:

Error: Meteor does not currently support objects other than ObjectID as ids

This happens when sub.added and sub.changed are called.
Executing the aggregate and then manually running the same functions on the subscription object, it seems to accept strings but not ObjectID objects.

So, to me, the issue seems to be the fact that obj._id is being passed as the second parameter, rather than obj._id.toHexString().

Has anyone run into this situation?
Am I missing something?

I am running Meteor 1.4.2 and v0.7.0 of the package.

THANKS!

sub.ready is not a function

I'm recieving an error of:

I20161017-14:49:52.502(-5)? Exception from sub survey id Y2arbPuvporwGJoso TypeError: sub.ready is not a function                                                                                                        
I20161017-14:49:52.504(-5)?     at ReactiveAggregate (packages/jcbernack_reactive-aggregate/aggregate.js:50:1)                                                                                                           
I20161017-14:49:52.505(-5)?     at Subscription._handler (server/main.js:20:5)                                                                                                                                           
I20161017-14:49:52.505(-5)?     at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1711:12)                                                                                                             
I20161017-14:49:52.506(-5)?     at Subscription._runHandler (packages/ddp-server/livedata_server.js:1026:17)                                                                                                             
I20161017-14:49:52.506(-5)?     at Session._startSubscription (packages/ddp-server/livedata_server.js:845:9)                                                                                                             
I20161017-14:49:52.507(-5)?     at Session.sub (packages/ddp-server/livedata_server.js:617:12)                                                                                                                           
I20161017-14:49:52.507(-5)?     at packages/ddp-server/livedata_server.js:551:43 

I've tried in 2 separate projects, one which is brand new.

The publish looks like this:

import { Meteor } from 'meteor/meteor';
import { ReactiveAggregate } from 'meteor/jcbernack:reactive-aggregate';
import { Books } from  '../imports/Books';

Meteor.publish('aggTest', () => {
    const pipeline = [
        {
            $project: {
                _id: 1,
                amount: true,
            }
        }        
    ]
    ReactiveAggregate(this, Books, pipeline)
})

Meteor Stats:

meteor npm --version
3.10.8
meteor node --version
v4.6.0

Need for a new version number ?

Hi,

Thanks for this great lib.

I've adopted it in my project, and found a bug with the "subscription.ready()" method being called too early. It seems like it's already been fixed in this repo, but no new version has been published, so people installing the package don't get the bugfix, is that correct ?

In the meantime I've copied and pasted the code in my own repo.

If you are not working on this anymore I'd totally understand

Return is undefined

This is my publish:

Meteor.publish("reportTotals", function() {
var theHours = ReactiveAggregate(this, Reports, [{
    $group: {
        '_id': this.userId,
        'hours': {
            $sum: '$hours'
        },
        'magazines': {
            $sum: '$magazines'
        },
        'brochures': {
            $sum: '$brochures'
        },
        'books': {
            $sum: 'books'
        }
    }
}, {
    $project: {
        userId: '$_id',
        hours: '$hours',
        magazines: '$magazines',
        brochures: '$brochures',
        books: '$books'
    }
}]);

console.log(theHours);

});

In console, I get the response: undefined

I might be missing something

TypeError: Object [object global] has no method 'added'

I'm getting this issue:

I20160130-20:56:41.779(11)? Exception from sub competitionLadders id MkTteTfRnqRasF35S TypeError: Object [object global] has no method 'added' I20160130-20:56:41.779(11)? at packages/jcbernack_reactive-aggregate/aggregate.js:17:1 I20160130-20:56:41.779(11)? at Array.forEach (native) I20160130-20:56:41.779(11)? at update (packages/jcbernack_reactive-aggregate/aggregate.js:15:1) I20160130-20:56:41.779(11)? at ReactiveAggregate (packages/jcbernack_reactive-aggregate/aggregate.js:47:1) I20160130-20:56:41.779(11)? at [object Object]._handler (server/publications/competitions.js:182:5)

I have set up competitionLadders as a client side collection.

This is in my publication:

ReactiveAggregate(this, CompetitionFixtures, pipeline, {clientCollection : 'competitionLadders'});

And ideas about the error message?

Non-existent document for aggregation

A document was available for aggregation. That document was deleted. Reactive Aggregate throws this error:

I20160104-10:50:28.341(0)? Exception in queued task: Error: Removed nonexistent document 1
I20160104-10:50:28.342(0)? at [object Object]..extend.removed (livedata_server.js:199:17)
I20160104-10:50:28.342(0)? at [object Object].
.extend.removed (livedata_server.js:388:10)
I20160104-10:50:28.342(0)? at [object Object]..extend.removed (livedata_server.js:1269:19)
I20160104-10:50:28.342(0)? at packages/jcbernack_reactive-aggregate/aggregate.js:27:1
I20160104-10:50:28.342(0)? at Function.
.each..forEach (packages/underscore/underscore.js:113:1)
I20160104-10:50:28.343(0)? at update (packages/jcbernack_reactive-aggregate/aggregate.js:24:1)
I20160104-10:50:28.343(0)? at packages/mongo/observe_multiplex.js:183:1
I20160104-10:50:28.343(0)? at Array.forEach (packages/es5-shim/.npm/package/node_modules/es5-shim/es5-shim.js:417:1)
I20160104-10:50:28.343(0)? at Function.
.each._.forEach (packages/underscore/underscore.js:105:1)
I20160104-10:50:28.343(0)? at Object.task (packages/mongo/observe_multiplex.js:177:1)

Is this issue coming from reactive aggregation?

The error comes up in server console, and doesn't break anything or prevent startup

May be this is a bug

This code has problem

sub._ids[doc._id] = sub._iteration;

If doc._id is mongo ObjectID type, then sub.ids[] will stored "ObjectId('57980bccc33da59766a6e806')" as key, but

sub.added(options.clientCollection,doc._id, doc);

sub will pick '57980bccc33da59766a6e806' as key of publish document, so it will throw exception during sub.removed() as k = ObjectId('57980bccc33da59766a6e806') does not match as 57980bccc33da59766a6e806

if (v != sub._iteration) {
        delete sub._ids[k];
        sub.removed(options.clientCollection, k);
      }

function update() {
    if (initializing) return;
    // add and update documents on the client
    collection.aggregate(pipeline).forEach(function (doc) {
      if (!sub._ids[doc._id]) {
        sub.added(options.clientCollection, doc._id, doc);
      } else {
        sub.changed(options.clientCollection, doc._id, doc);
      }
      sub._ids[doc._id] = sub._iteration;
    });
    // remove documents not in the result anymore
    _.forEach(sub._ids, function (v, k) {
      if (v != sub._iteration) {
        delete sub._ids[k];
        sub.removed(options.clientCollection, k);
      }
    });
    sub._iteration++;
  }

How to sort and limit collection?

Hi, I want to sort the collection by count in descending orden and limit to 5 documents. I tried to use the sort and limit but couldn't make it work.

Meteor.publish("topics", function () {
  ReactiveAggregate(this, Portadas, [
    {
      $project: {
              topic: { $split: ["$title", " "] },
              date_seconds: 1,
              documento: "$$ROOT"
      }
    },
    {
      $unwind: {
              path: "$topic"
      }
    },
    {
      $match: {
              topic: { $nin: ["sports", reg1, reg2,reg3] },
              date_seconds: { $gte: timestamp }
      }
    },
    {
      $group: {
              _id: "$topic",
              count: { $sum: 1 },
              articles: { $push: "$documento" }
      }
    }
  ], { clientCollection: "clientTopic" });
});

reactive aggregation with React

does the package support react implementation for meteor?

I'm trying to using it with the mixin ReactMeteorData but i can't retrieve the aggragation data

ReactiveAggregate returns Mongo types instead of Meteor

Hello,

Because meteor-reactive-aggregate is using rawCollection to call aggregate on collection, result contains plain Mongo types instead of Meteor ones. It is a problem for custom EJSON types returned in aggregate or when ObjectID or Decimal128 is used (see Scale Factor aggregation in https://docs.mongodb.com/manual/tutorial/model-monetary-data/).

This can be fixed by adding same code which is used in meteor/mongo.js for converting types:

diff --git a/aggregate.js b/aggregate.js
index b7e7cc9..2cb5683 100644
--- a/aggregate.js
+++ b/aggregate.js
@@ -1,6 +1,81 @@
 import { Meteor } from 'meteor/meteor';
 import { Mongo } from 'meteor/mongo';
 
+// FIXME: Copied this from mongo.js
+import { NpmModuleMongodb as MongoDB } from 'meteor/npm-mongo';
+import { EJSON } from 'meteor/ejson';
+
+var replaceNames = function (filter, thing) {
+  if (typeof thing === "object" && thing !== null) {
+    if (_.isArray(thing)) {
+      return _.map(thing, _.bind(replaceNames, null, filter));
+    }
+
+    var ret = {};
+
+    _.each(thing, function (value, key) {
+      ret[filter(key)] = replaceNames(filter, value);
+    });
+
+    return ret;
+  }
+
+  return thing;
+};
+
+var unmakeMongoLegal = function (name) {
+  return name.substr(5);
+};
+
+var replaceMongoAtomWithMeteor = function (document) {
+  if (document instanceof MongoDB.Binary) {
+    var buffer = document.value(true);
+    return new Uint8Array(buffer);
+  }
+
+  if (document instanceof MongoDB.ObjectID) {
+    return new Mongo.ObjectID(document.toHexString());
+  }
+
+  if (document instanceof MongoDB.Decimal128) {
+    return Decimal(document.toString());
+  }
+
+  if (document["EJSON$type"] && document["EJSON$value"] && _.size(document) === 2) {
+    return EJSON.fromJSONValue(replaceNames(unmakeMongoLegal, document));
+  }
+
+  if (document instanceof MongoDB.Timestamp) {
+    // For now, the Meteor representation of a Mongo timestamp type (not a date!
+    // this is a weird internal thing used in the oplog!) is the same as the
+    // Mongo representation. We need to do this explicitly or else we would do a
+    // structural clone and lose the prototype.
+    return document;
+  }
+
+  return undefined;
+};
+
+var replaceTypes = function (document, atomTransformer) {
+  if (typeof document !== 'object' || document === null) return document;
+  var replacedTopLevelAtom = atomTransformer(document);
+  if (replacedTopLevelAtom !== undefined) return replacedTopLevelAtom;
+  var ret = document;
+
+  _.each(document, function (val, key) {
+    var valReplaced = replaceTypes(val, atomTransformer);
+
+    if (val !== valReplaced) {
+      // Lazy clone. Shallow copy.
+      if (ret === document) ret = _.clone(document);
+      ret[key] = valReplaced;
+    }
+  });
+
+  return ret;
+};
+//FIXME: END
+
 const defaultOptions = ({
   collection, options
 }) => ({
@@ -48,6 +123,7 @@ export const ReactiveAggregate = function (subscription, collection, pipeline =
       }
       // cursor is not done iterating, add and update documents on the client
       else {
+        doc = replaceTypes(doc, replaceMongoAtomWithMeteor);
         if (!subscription._ids[doc._id]) {
           subscription.added(clientCollection, doc._id, doc);
         } else {

I'm not sure however how to solve this without copy-paste.

Antoni.

observeOptions does nothing / can't use limit and skip

Hello,
The documentation says that I can use 'limit' like this:

observeOptions: {
        limit: 10,
        skip: 0,
        sort: { createdAt: -1 },
}

It does not work, i always get all the documents that corresponds to the query.

Also, i tried adding limit directly in the pipeline:

{
      $lookup: {
        from: 'users',
        localField: 'userId',
        foreignField: '_id',
        as: 'user'
     }
}, 
{
      $project: _adsPublicFields
}, 
{
      $limit: 10
},
{
      $skip: 0
}, 
{
      $sort:  { createdAt: -1 },
}

This works only if skip is 0, if skip is 10, it will retun empty, if skip is 6, it will return 4 results (the limit 10 - skip 6).

What can i do to make it work?

Aggregation takes time with large collections

I'm using the package in many places in the project, it is working perfectly with all collections with med size, once i use it with little bigger collection more than 36k doc, it takes some time till finished more than '8 sec' and the pipeline is not complicated 'only simple match', can you please help to figure out this issue if anyone faced that before.

MongoDB server version: 4.0.6
Meteor version: 1.8.1

Map inner object field

Hi,

Let's say I got this object structure in a collection:

{
  "_id" : "gxDT7d49mgPh65h9P",
  "serviceTypeId" : "ayNc88uLkwZT9kn3m",
  "status" : "Done",
  "projectId" : "g8yCFambJxwFxY6Ms",
  "employee" : {
    "name" : "Adán",
    "lastname" : "Enríquez"
  },
  "serviceType" : {
    "name" : "Tipo2"
  },
  "dateUpdated" : ISODate("2017-04-30T10:48:06.065Z"),
  "plannedStartDate" : ISODate("2017-04-30T10:47:00Z"),
  "plannedEndDate" : ISODate("2017-04-30T10:48:00Z"),
  "executedStartDate" : ISODate("2017-04-30T10:48:00Z"),
  "executedEndDate" : ISODate("2017-04-30T10:48:00Z")
}

I'm using aggregation to get items in a month, grouped by serviceTypeId and plannedStartDate or executedEndDate. I'm using something like this (no date is mandatory):

Meteor.publish("serviceTypesCounts", function(startDate, endDate) {
  // init dates to current month if not provided:
  startDate = startDate || moment().startOf('month').toDate();
  endDate = endDate || moment().endOf('month').toDate();
  // Remember, ReactiveAggregate doesn't return anything
  ReactiveAggregate(
    this,
    Services,
    [
      {
        $match: {
          // match current user's projectId and either planned or executed date exists, and any of those dates are inside startDate, endDate range:
          $and: [
            {projectId: getCurrentProjectId(this.userId)},
            // plannedStartDate or executedStart date exists:
            {$or: [
              {plannedStartDate: {$exists: true}},
              {executedStartDate: {$exists: true}}
            ]},
            // plannedStartDate is in startDate, endDate range or it doesn't exist and executedStartDate is:
            {$or: [
              {
                plannedStartDate: {
                  $gte: startDate,
                  $lte: endDate
                }
              },
              {
                $and: [
                  {
                    executedStartDate: {
                      $gte: startDate,
                      $lte: endDate
                    }
                  },
                  {
                    plannedStartDate: {
                      $exists: false
                    }
                  }
                ]
              }
            ]}
          ]
        }
      },
      {
        $group: {
          '_id': {
            serviceTypeId: "$serviceTypeId",

            year: {
              $cond: [
                { $ifNull: ['$plannedStartDate', 0] },
                { $year: '$plannedStartDate' },
                { $year: '$executedStartDate'}
              ]
            },

            month: {
              $cond: [
                { $ifNull: ['$plannedStartDate', 0] },
                { $month: '$plannedStartDate' },
                { $month: '$executedStartDate'}
              ]
            },

            day: {
              $cond: [
                { $ifNull: ['$plannedStartDate', 0] },
                { $dayOfMonth: '$plannedStartDate' },
                { $dayOfMonth: '$executedStartDate'}
              ]
            },



          },
          'count': { $sum: 1 }
        }
      },
      {
        $project: {
          _id: {
            $concat: [
              '$_id.serviceTypeId',
              '-',
              {$substr: ["$_id.year", 0, 4]},
              {$substr: ["$_id.month", 0, 2]},
              {$substr: ["$_id.day", 0, 2]}
            ]
          },
          year: '$_id.year',
          month: '$_id.month',
          day: '$_id.day',
          count: '$count'
        } // Send the aggregation to the 'clientReport' collection available for client use
      }
    ],
    {
      clientCollection: "ServiceTypesCounts"
    }
  );
});

I works, I'm getting results like this:

{ _id: 'ayNc88uLkwZT9kn3m-2017521', count: 1, year: 2017, month: 5, day: 21 }
{ _id: 'YG873A6AZZ3eEAkqX-2017520', count: 1, year: 2017, month: 5, day: 20 }
{ _id: 'ayNc88uLkwZT9kn3m-2017520', count: 1, year: 2017, month: 5, day: 20 }
{ _id: 'YG873A6AZZ3eEAkqX-201757', count: 26, year: 2017, month: 5, day: 7 }
{ _id: 'YG873A6AZZ3eEAkqX-201758', count: 340, year: 2017, month: 5 ,day: 8 }

Now, thing is: I want to get field serviceType.name inside original objects (or event the full object), how can a I do this?

It would be great to get that field even if I'm not grouping by it, but in this case I don't care as it is the readable representation for serviceTypeId.

Thanks!

returned with empty

//server side code
`UserLogging = new Mongo.Collection('user_logging');

const options = {
observeOptions: {
limit: 10,
sort: { createdAt: -1 },
}
};

Meteor.publish("reportTotals", function() {
// Remember, ReactiveAggregate doesn't return anything
ReactiveAggregate(this, UserLogging, [{
// assuming our Reports collection have the fields: hours, books
$group: {
'_id': "$userId",
count: {
// In this case, we're running summation.
$sum: 1
}
}
}, {
$project: {
// an id can be added here, but when omitted,
// it is created automatically on the fly for you
userId:'$userId',
count: '$count'
} // Send the aggregation to the 'clientReport' collection available for client use
},options], { clientCollection: "ClientUser" });
});`
//client side code

`ClientUser = new Mongo.Collection('clientuser');

Meteor.subscribe("reportTotals");
console.log("****hllojj");

Template.userList.helpers({
user: function() {
console.log("****hllojj");

return UserLogging.find({},{limit:20});

}
});
`
you may have understood what iam doing right...
it is returning with empty
can you please suggest anything..
any new mwthods to achieve reactive publish is also works...
thanks in advance

Efficiency is very low? Why?

I test this package, just using simple pipeline:

ReactiveAggregate(this, loanLogs, [{
    $match: {
        own:user.username,
        time: {
            $gt: new Date("2016-09-01")
        }
    }
}], {
    clientCollection: "todayLoanTotal"
});

It works, but not well, costing 10s~20s to get data at first time .
Then debug your code ,

var handle = query.observeChanges({
    added: update,
    changed: update,
    removed: update,
    error: function (err) {
      throw err;
    }
});

observeChanges is blocked for 10~20 seconds. Why? Do I miss something esle? Thanks!

Fetching all documents in collection

I'm trying to figure out why my application is fetching all the documents in my collections.

I have a publication that looks like this:

Meteor.publish('location_updates', function(){

  let pipeline = [
    {
      $match: {
        userId: this.userId
      }
    },
    {
      $sort: {
        'location.timestamp': -1
      }
    },
    {
      $limit: 1
    },
    {
      $project: {
        'location.coords': 1,
        'location.uuid': 1
      }
    }
  ];

  ReactiveAggregate(this, Location_updates, pipeline);

});

On the client-side I'm selecting a single document:

this.subscribe('location_updates');
....
Location_updates.find({'location.uuid': pos.uuid}, { limit: 1 });

Yet in the Galaxy APM I'm seeing this when I load the app:

image

That's every single document in the location_updates collection.

The same goes for all my other publications that use ReactiveAggregate.

Cannot set property '_ids' of undefined

I get this error message after adding the $match to the pipeline:

Exception from sub grouped-custom-pages id fYKiYT3M2CZ8KTSk6 TypeError: Cannot set property '_ids' of undefined
I20171214-12:31:50.169(1)? at ReactiveAggregate (packages/jcbernack_reactive-aggregate.js:31:12)

this is my publication:

Meteor.publish("grouped-custom-pages", (type) => {
        ReactiveAggregate(this, CustomPages, 
            [
                {
                    $match: {
                        type: type
                    }
                },
                {
                    $group: {
                        _id: "$languageId",
                        createdAt: {$push: "$createdAt"},
                        pages: { $push: {title: "$title", id: "$_id", language: "$language", type: "$type", published: "$published", mainImage: "$mainImage" }}
                    }
                }
            ], 
            { clientCollection: "GroupedCustomPages" }
        );
    });

the query works fine in i.e. Robomongo, so whats the problem here?

Aggregate and filter with full text indexes

Maybe I am missing something, but I can't get the $lookup to work with full text searches. I have a profile object which is a one to one to the users object (Decided to extend it). To pull in the in the users it seems aggregate was the most obvious choice. I want to use the following find in the aggregate:

profile.find(
        { $text: {
            $search: query
          }
        },
        {
          fields: {
            score: {
              $meta: 'textScore'
            }
          },
          sort: {
            score: {
              $meta: 'textScore'
            }
          }
        }
      );

An aggregate without the text query work. But using the above in an aggregate using the property observeSelector get "unknown operator: $text". I think my search translation into the aggregate might be faulty.

clientCollection empty

Hello,
my clientCollection is empty.
Inventories looks like this: "{ "_id" : "5279859913", "userId" : "Jh3DRj3sCwqZiAhAB", "name" : "Test" }"

ReactiveAggregate(this, Inventories, [{ $match: { 'userId': 'Jh3DRj3sCwqZiAhAB' } }, { $project: { 'userId': '$userId', 'name': '$name' } } ], { clientCollection: "clientBotInventory" });

update() should be throttled

The way this is written, if a collection has 100 documents, collection.aggregate will get called 100 times in quick succession, once for every document that's added initially 😄

It should really be throttled with a customizable wait time, and it would be ideal if it could wait until the all the initial documents have been added, but unfortunately there's no way to know when observeChanges is ready...

$lookup returning empty array

a = new Mongo.Collection('a');
b= new Mongo.Collection('b');
c= new Mongo.Collection('c');
d= new Mongo.Collection('d');

a.insert({ "aaa":"hey" "aa":'1' });
b.insert({ "bbb":"hi" "bb": '1' });

Meteor.publish("c", function () {
const pipeline = [{
$lookup: {
from: 'b', localField: 'aa', foreignField: 'bb', as: 'ab', } }];
ReactiveAggregate(this, a, pipeline, { clientCollection: "d" });
});

when i return 'd' it returns
{"aaa":"hey" "aa":'1' "ab": [] }

although "ab" should have "aaa": "hey" I believe?

Error with Meteor 1.3.2.4

import {ReactiveAggregate} from 'meteor/jcbernack:reactive-aggregate';

This is my publish function.

Meteor.publish('skills', () => {
return ReactiveAggregate(this, Skills, [
{$match: {status: 'active'}},
{$limit: 10}
]);
});

And in the meteor console I got this:
I20160422-16:12:21.076(7)? Exception from sub skills id RE76noLCSJ2LeaYeh TypeError: Object [object global] has no method 'added'
I20160422-16:12:21.078(7)? at packages/jcbernack_reactive-aggregate/aggregate.js:17:1
I20160422-16:12:21.079(7)? at Array.forEach (/home/manhpt/.meteor/packages/es5-shim/.4.5.10.ajfzqv++os+web.browser+web.cordova/npm/node_modules/es5-shim/es5-shim.js:435:21)
I20160422-16:12:21.080(7)? at update (packages/jcbernack_reactive-aggregate/aggregate.js:15:1)
I20160422-16:12:21.081(7)? at ReactiveAggregate (packages/jcbernack_reactive-aggregate/aggregate.js:47:1)
I20160422-16:12:21.083(7)? at Subscription._handler (imports/api/skills.js:20:12)
I20160422-16:12:21.084(7)? at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1704:12)
I20160422-16:12:21.085(7)? at Subscription._runHandler (packages/ddp-server/livedata_server.js:1026:17)
I20160422-16:12:21.086(7)? at Session._startSubscription (packages/ddp-server/livedata_server.js:845:9)
I20160422-16:12:21.088(7)? at Session.sub (packages/ddp-server/livedata_server.js:617:12)
I20160422-16:12:21.089(7)? at packages/ddp-server/livedata_server.js:551:43

My app was fine but the result of subscribe method became empty. The problem happend the same to collection:users Meteor.users.

ReactiveAggregate() + collection.update() -> Error: Expected to find a document to change

I'm not sure if I'm doing something wrong or this is a bug. I've taken quite an effort to describe the problem on StackOverflow, so please allow me to simply link to my question there.

In short and for search engines:

Uncaught Error: Expected to find a document to change
    at Object.update (collection.js:207)
    at Object.store.<computed> [as update] (livedata_connection.js:310)
    ...

This (probably) gets thrown when ReactiveAggregate() tries to update the clientCollection. But I don't know what I'm doing wrong here.

Single document published

Hi there

i am new to Meteor, so it might be that i am missing out some basics here.
I am using the package to publish the result of a $lookup + $project + $match aggregation to a reactive native app.
The magic is that, on the react side (within createContainer()), i am getting an array of only one document.
If instead put a $group alone in the pipeline, i get all the expected documents (13).
I am using publish/subscribe on other collections (without aggregation), and they work just fine.
Thanks in advance.

clientCollection name not defined on client side

I have what I thought was a relatively simple test, but I'm not sure where I'm going wrong. The lookup itself works with the regular aggregate function.

Meteor.publish('allPropertyApps', function() {
ReactiveAggregate(this, applicationsTable, [
    {
        $lookup: {
            from: "partyGroups",
            localField: "owner",
            foreignField: "_id",
            as: "application_party"
        }
    }
], { clientCollection: "propApps" });

but when I attempt to access "propApps" in my helper, it's undefined.

class PropertyApplications {
    constructor($scope, $stateParams) {
        'ngInject';

        $scope.viewModel(this);

        this.$scope = $scope;
        this.$stateParams = $stateParams;

        this.$scope.propertyId = this.$scope.propertyApplications.property._id;
        if(this.$scope.propertyId === undefined) {
            this.$scope.propertyId = this.$stateParams.pid;
        }
        Meteor.subscribe('allPropertyApps', this.$scope.propertyId);

        this.helpers({
            allPropertyApps: function () {
                console.log('working!');
                console.log(propApps);
                return propApps.find();
                return false;
            }
        });
    }
}

If I define "propApps" within a client-side collection myself, then it just stays as an empty collection.

Crash on app startup

Hi,

Just added this package to make one aggregation and it was working fine in a local environment, but just after deploy (using mup) to our server, got a crash (not even on subscribing, just on app startup) for memory allocation.

Rolling back to previous version without aggregation does work fine so... it has something to do with this :(

Guess is due to observer or something related to reactivity because when using meteorhacks aggregate don't get this error (it's not a subscription although).

In case you need more info, the aggregation I'm using is the one in issue #26

Thanks!

Can't combine reactive publish with plain cursor

Normally a Publish method can return an array of cursors to publish results from multiple collections. However, since ReactiveAggregate uses the added/changed/removed interface directly, its result can't be combined with a cursor, or another aggregation, as part of a single publish. Either a version of the interface that returns a Cursor should be provided, or ReactiveAggregate should be able to take some cursors to push to the client along with the aggregated data.

Add the ability to stop and restart aggregation

It would be very useful to be able to restart aggregation in cases when you want to observe some unrelated changes which serve as an input to the aggregation for example.

As for implementation, I think returning a cleanup function from ReactiveAggregate would work.

If you like this idea I can help implement it.

Thoughts?

Two collections in one aggregate method

Hi @JcBernack ,

First of all thanks for this package! It make all my aggregate reactive! 👍

is it possible to aggregate two collections in one publication?

Use case:
I have productName and premium fields in enrollments collection.
I have status, products and amountPaid on claim collection.

I want to calculate the loss ratio = total money retained, here's the formula.

(sum of all premium / sum of all amount paid) * 100 = money retained in %

the productName field from collection 1 and products field from collection 2 is basically the same, they are foreign keys from products collection as well as premium.

I've successfully aggregated/calculated money retained per product that matches a certain status if premium is on claim collection but the problem is it will only calculate the total premium of all matching status, the requirement is for all the products(productName) availed in enrollments collection.

Here's my working code:

Meteor.publish( "lossRatio", function () {
  ReactiveAggregate( this, Claim, [ {
    $match: {
      $or: [ {
        'status': "ApprovedRB"
      }, {
        'status': "Approved"
      } ]
    }
  }, {
    $group: {
      '_id': "$products",
      'amountPaid': {
        $sum: '$amountPaid'
      },
      'premium': {
        $sum: '$premium'
      }
    }
  }, {
    $project: {
      status: '$status',
      amountPaid: '$amountPaid',
      premium: '$premium',
      retained: {
        $multiply: [ {
          $divide: [ '$premium', '$amountPaid' ],
        }, 100 ]
      }

    },
  } ], {
    clientCollection: "LossRatio"
  } );
} );

Notice the premium is in claim collection but I mentioned earlier that it is on enrollments collection, this is for testing purposes to see if will work and it works.

The issue is that the calculated total premium should be based on the total productName on enrollments collection not the total products with matching status on claim collection.

Is this possible? I am thinking of creating two separate publications and combine them in one collection. Is this theory possible?

how to use with match date aggregate?

i have this line of code..
$match: { createdAt: {$gte: new Date(dateFrom), $lte: new Date(dateTo)} }, $group: { '_id': "$userName", 'points': { // In this case, we're running summation. $sum: '$points' }, 'reportCount': { $sum: 1 } }

how to approach this ?

Delay between subscription.ready() and data availability

First, I'd just like to say thank you for creating this package! This is really cool. Now, for the problem I'm encountering:

For trying out this package, I have two identical versions of my app running - The only difference being that one subscribes to an aggregate collection that I create outside of meteor and update via a cronjob, and one uses meteor-reactive-aggregate to aggregate the data in real time. Either way, the aggregated data is the exact same size, format, everything.

  • With the app version that aggregates outside of meteor, when subscription.ready() is true, the data is always instantly available on the client, even on slow client connections.

  • With the app version that uses meteor-reactive-aggregate, when subscription.ready() is true, I have an empty array where the data should be. It is easy to check if the array is empty (and if so, continue displaying a loading spinner). The problem is that on slow client connections, after the array is empty but before the array is completely filled, the array will often be partially filled which causes my component to render the partial data before the array is completely filled and the final render happens.

Any idea as to what may be causing this, and if there is anything I can do about it? Is ready() being called early, or is the server just kind of bogged down from performing the aggregation, so it takes longer to publish the data?

Reactive aggregate is not defined

I installed by:

meteor add jcbernack:reactive-aggregate

I created publication

Meteor.publish('notes', function (param) {

        return ReactiveAggregate(this, Note, [{
            $match: {
                userId: getUser(param)._id
            },
        },{
            $lookup: {
                from: 'illnesses',
                localField: 'illnessId',
                foreignField: '_id',
                as: 'illness'
            }
        },{
            $unwind: {
                path: "$illness",
                preserveNullAndEmptyArrays: true
            }
        }]); 

    });

I see error

Exception from sub notes id GGhA6bvomWkPLGbef ReferenceError: ReactiveAggregate is not defined
I20180809-14:03:42.435(2)?     at Subscription._handler (imports/api/publications.js:105:9)
I20180809-14:03:42.435(2)?     at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1767:12)
I20180809-14:03:42.435(2)?     at DDP._CurrentPublicationInvocation.withValue (packages/ddp-server/livedata_server.js:1043:15)
I20180809-14:03:42.436(2)?     at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
I20180809-14:03:42.436(2)?     at Subscription._runHandler (packages/ddp-server/livedata_server.js:1041:51)
I20180809-14:03:42.436(2)?     at Session._startSubscription (packages/ddp-server/livedata_server.js:859:9)
I20180809-14:03:42.436(2)?     at Session.sub (packages/ddp-server/livedata_server.js:625:12)
I20180809-14:03:42.437(2)?     at packages/ddp-server/livedata_server.js:559:43

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.