jcbernack / meteor-reactive-aggregate Goto Github PK
View Code? Open in Web Editor NEWReactively publish aggregations with Meteor.
Reactively publish aggregations with Meteor.
I have what I thought was a relatively simple test, but I'm not sure where I'm going wrong. The lookup itself works with the regular aggregate function.
Meteor.publish('allPropertyApps', function() {
ReactiveAggregate(this, applicationsTable, [
{
$lookup: {
from: "partyGroups",
localField: "owner",
foreignField: "_id",
as: "application_party"
}
}
], { clientCollection: "propApps" });
but when I attempt to access "propApps" in my helper, it's undefined.
class PropertyApplications {
constructor($scope, $stateParams) {
'ngInject';
$scope.viewModel(this);
this.$scope = $scope;
this.$stateParams = $stateParams;
this.$scope.propertyId = this.$scope.propertyApplications.property._id;
if(this.$scope.propertyId === undefined) {
this.$scope.propertyId = this.$stateParams.pid;
}
Meteor.subscribe('allPropertyApps', this.$scope.propertyId);
this.helpers({
allPropertyApps: function () {
console.log('working!');
console.log(propApps);
return propApps.find();
return false;
}
});
}
}
If I define "propApps" within a client-side collection myself, then it just stays as an empty collection.
This code has problem
sub._ids[doc._id] = sub._iteration;
If doc._id is mongo ObjectID type, then sub.ids[] will stored "ObjectId('57980bccc33da59766a6e806')" as key, but
sub.added(options.clientCollection,doc._id, doc);
sub will pick '57980bccc33da59766a6e806' as key of publish document, so it will throw exception during sub.removed() as k = ObjectId('57980bccc33da59766a6e806') does not match as 57980bccc33da59766a6e806
if (v != sub._iteration) { delete sub._ids[k]; sub.removed(options.clientCollection, k); }
function update() {
if (initializing) return;
// add and update documents on the client
collection.aggregate(pipeline).forEach(function (doc) {
if (!sub._ids[doc._id]) {
sub.added(options.clientCollection, doc._id, doc);
} else {
sub.changed(options.clientCollection, doc._id, doc);
}
sub._ids[doc._id] = sub._iteration;
});
// remove documents not in the result anymore
_.forEach(sub._ids, function (v, k) {
if (v != sub._iteration) {
delete sub._ids[k];
sub.removed(options.clientCollection, k);
}
});
sub._iteration++;
}
I get this error message after adding the $match to the pipeline:
Exception from sub grouped-custom-pages id fYKiYT3M2CZ8KTSk6 TypeError: Cannot set property '_ids' of undefined
I20171214-12:31:50.169(1)? at ReactiveAggregate (packages/jcbernack_reactive-aggregate.js:31:12)
this is my publication:
Meteor.publish("grouped-custom-pages", (type) => {
ReactiveAggregate(this, CustomPages,
[
{
$match: {
type: type
}
},
{
$group: {
_id: "$languageId",
createdAt: {$push: "$createdAt"},
pages: { $push: {title: "$title", id: "$_id", language: "$language", type: "$type", published: "$published", mainImage: "$mainImage" }}
}
}
],
{ clientCollection: "GroupedCustomPages" }
);
});
the query works fine in i.e. Robomongo, so whats the problem here?
First, I'd just like to say thank you for creating this package! This is really cool. Now, for the problem I'm encountering:
For trying out this package, I have two identical versions of my app running - The only difference being that one subscribes to an aggregate collection that I create outside of meteor and update via a cronjob, and one uses meteor-reactive-aggregate to aggregate the data in real time. Either way, the aggregated data is the exact same size, format, everything.
With the app version that aggregates outside of meteor, when subscription.ready()
is true, the data is always instantly available on the client, even on slow client connections.
With the app version that uses meteor-reactive-aggregate, when subscription.ready()
is true, I have an empty array where the data should be. It is easy to check if the array is empty (and if so, continue displaying a loading spinner). The problem is that on slow client connections, after the array is empty but before the array is completely filled, the array will often be partially filled which causes my component to render the partial data before the array is completely filled and the final render happens.
Any idea as to what may be causing this, and if there is anything I can do about it? Is ready()
being called early, or is the server just kind of bogged down from performing the aggregation, so it takes longer to publish the data?
Hi,
Just added this package to make one aggregation and it was working fine in a local environment, but just after deploy (using mup) to our server, got a crash (not even on subscribing, just on app startup) for memory allocation.
Rolling back to previous version without aggregation does work fine so... it has something to do with this :(
Guess is due to observer or something related to reactivity because when using meteorhacks aggregate don't get this error (it's not a subscription although).
In case you need more info, the aggregation I'm using is the one in issue #26
Thanks!
A document was available for aggregation. That document was deleted. Reactive Aggregate throws this error:
I20160104-10:50:28.341(0)? Exception in queued task: Error: Removed nonexistent document 1
I20160104-10:50:28.342(0)? at [object Object]..extend.removed (livedata_server.js:199:17)
I20160104-10:50:28.342(0)? at [object Object]..extend.removed (livedata_server.js:388:10)
I20160104-10:50:28.342(0)? at [object Object]..extend.removed (livedata_server.js:1269:19)
I20160104-10:50:28.342(0)? at packages/jcbernack_reactive-aggregate/aggregate.js:27:1
I20160104-10:50:28.342(0)? at Function..each..forEach (packages/underscore/underscore.js:113:1)
I20160104-10:50:28.343(0)? at update (packages/jcbernack_reactive-aggregate/aggregate.js:24:1)
I20160104-10:50:28.343(0)? at packages/mongo/observe_multiplex.js:183:1
I20160104-10:50:28.343(0)? at Array.forEach (packages/es5-shim/.npm/package/node_modules/es5-shim/es5-shim.js:417:1)
I20160104-10:50:28.343(0)? at Function..each._.forEach (packages/underscore/underscore.js:105:1)
I20160104-10:50:28.343(0)? at Object.task (packages/mongo/observe_multiplex.js:177:1)
Is this issue coming from reactive aggregation?
The error comes up in server console, and doesn't break anything or prevent startup
Maybe I am missing something, but I can't get the $lookup to work with full text searches. I have a profile object which is a one to one to the users object (Decided to extend it). To pull in the in the users it seems aggregate was the most obvious choice. I want to use the following find in the aggregate:
profile.find(
{ $text: {
$search: query
}
},
{
fields: {
score: {
$meta: 'textScore'
}
},
sort: {
score: {
$meta: 'textScore'
}
}
}
);
An aggregate without the text query work. But using the above in an aggregate using the property observeSelector get "unknown operator: $text". I think my search translation into the aggregate might be faulty.
Hello,
The documentation says that I can use 'limit' like this:
observeOptions: {
limit: 10,
skip: 0,
sort: { createdAt: -1 },
}
It does not work, i always get all the documents that corresponds to the query.
Also, i tried adding limit directly in the pipeline:
{
$lookup: {
from: 'users',
localField: 'userId',
foreignField: '_id',
as: 'user'
}
},
{
$project: _adsPublicFields
},
{
$limit: 10
},
{
$skip: 0
},
{
$sort: { createdAt: -1 },
}
This works only if skip is 0, if skip is 10, it will retun empty, if skip is 6, it will return 4 results (the limit 10 - skip 6).
What can i do to make it work?
does the package support react implementation for meteor?
I'm trying to using it with the mixin ReactMeteorData but i can't retrieve the aggragation data
I'm recieving an error of:
I20161017-14:49:52.502(-5)? Exception from sub survey id Y2arbPuvporwGJoso TypeError: sub.ready is not a function
I20161017-14:49:52.504(-5)? at ReactiveAggregate (packages/jcbernack_reactive-aggregate/aggregate.js:50:1)
I20161017-14:49:52.505(-5)? at Subscription._handler (server/main.js:20:5)
I20161017-14:49:52.505(-5)? at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1711:12)
I20161017-14:49:52.506(-5)? at Subscription._runHandler (packages/ddp-server/livedata_server.js:1026:17)
I20161017-14:49:52.506(-5)? at Session._startSubscription (packages/ddp-server/livedata_server.js:845:9)
I20161017-14:49:52.507(-5)? at Session.sub (packages/ddp-server/livedata_server.js:617:12)
I20161017-14:49:52.507(-5)? at packages/ddp-server/livedata_server.js:551:43
I've tried in 2 separate projects, one which is brand new.
The publish looks like this:
import { Meteor } from 'meteor/meteor';
import { ReactiveAggregate } from 'meteor/jcbernack:reactive-aggregate';
import { Books } from '../imports/Books';
Meteor.publish('aggTest', () => {
const pipeline = [
{
$project: {
_id: 1,
amount: true,
}
}
]
ReactiveAggregate(this, Books, pipeline)
})
Meteor Stats:
meteor npm --version
3.10.8
meteor node --version
v4.6.0
I installed by:
meteor add jcbernack:reactive-aggregate
I created publication
Meteor.publish('notes', function (param) {
return ReactiveAggregate(this, Note, [{
$match: {
userId: getUser(param)._id
},
},{
$lookup: {
from: 'illnesses',
localField: 'illnessId',
foreignField: '_id',
as: 'illness'
}
},{
$unwind: {
path: "$illness",
preserveNullAndEmptyArrays: true
}
}]);
});
I see error
Exception from sub notes id GGhA6bvomWkPLGbef ReferenceError: ReactiveAggregate is not defined
I20180809-14:03:42.435(2)? at Subscription._handler (imports/api/publications.js:105:9)
I20180809-14:03:42.435(2)? at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1767:12)
I20180809-14:03:42.435(2)? at DDP._CurrentPublicationInvocation.withValue (packages/ddp-server/livedata_server.js:1043:15)
I20180809-14:03:42.436(2)? at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
I20180809-14:03:42.436(2)? at Subscription._runHandler (packages/ddp-server/livedata_server.js:1041:51)
I20180809-14:03:42.436(2)? at Session._startSubscription (packages/ddp-server/livedata_server.js:859:9)
I20180809-14:03:42.436(2)? at Session.sub (packages/ddp-server/livedata_server.js:625:12)
I20180809-14:03:42.437(2)? at packages/ddp-server/livedata_server.js:559:43
I'm not sure if I'm doing something wrong or this is a bug. I've taken quite an effort to describe the problem on StackOverflow, so please allow me to simply link to my question there.
In short and for search engines:
Uncaught Error: Expected to find a document to change
at Object.update (collection.js:207)
at Object.store.<computed> [as update] (livedata_connection.js:310)
...
This (probably) gets thrown when ReactiveAggregate()
tries to update
the clientCollection
. But I don't know what I'm doing wrong here.
I'm getting this issue:
I20160130-20:56:41.779(11)? Exception from sub competitionLadders id MkTteTfRnqRasF35S TypeError: Object [object global] has no method 'added' I20160130-20:56:41.779(11)? at packages/jcbernack_reactive-aggregate/aggregate.js:17:1 I20160130-20:56:41.779(11)? at Array.forEach (native) I20160130-20:56:41.779(11)? at update (packages/jcbernack_reactive-aggregate/aggregate.js:15:1) I20160130-20:56:41.779(11)? at ReactiveAggregate (packages/jcbernack_reactive-aggregate/aggregate.js:47:1) I20160130-20:56:41.779(11)? at [object Object]._handler (server/publications/competitions.js:182:5)
I have set up competitionLadders
as a client side collection.
This is in my publication:
ReactiveAggregate(this, CompetitionFixtures, pipeline, {clientCollection : 'competitionLadders'});
And ideas about the error message?
import {ReactiveAggregate} from 'meteor/jcbernack:reactive-aggregate';
This is my publish function.
Meteor.publish('skills', () => {
return ReactiveAggregate(this, Skills, [
{$match: {status: 'active'}},
{$limit: 10}
]);
});
And in the meteor console I got this:
I20160422-16:12:21.076(7)? Exception from sub skills id RE76noLCSJ2LeaYeh TypeError: Object [object global] has no method 'added'
I20160422-16:12:21.078(7)? at packages/jcbernack_reactive-aggregate/aggregate.js:17:1
I20160422-16:12:21.079(7)? at Array.forEach (/home/manhpt/.meteor/packages/es5-shim/.4.5.10.ajfzqv++os+web.browser+web.cordova/npm/node_modules/es5-shim/es5-shim.js:435:21)
I20160422-16:12:21.080(7)? at update (packages/jcbernack_reactive-aggregate/aggregate.js:15:1)
I20160422-16:12:21.081(7)? at ReactiveAggregate (packages/jcbernack_reactive-aggregate/aggregate.js:47:1)
I20160422-16:12:21.083(7)? at Subscription._handler (imports/api/skills.js:20:12)
I20160422-16:12:21.084(7)? at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1704:12)
I20160422-16:12:21.085(7)? at Subscription._runHandler (packages/ddp-server/livedata_server.js:1026:17)
I20160422-16:12:21.086(7)? at Session._startSubscription (packages/ddp-server/livedata_server.js:845:9)
I20160422-16:12:21.088(7)? at Session.sub (packages/ddp-server/livedata_server.js:617:12)
I20160422-16:12:21.089(7)? at packages/ddp-server/livedata_server.js:551:43
My app was fine but the result of subscribe method became empty. The problem happend the same to collection:users Meteor.users
.
When using $lookup the other collections are not reactive
Hi,
Trying to use this plugin for the first time.
Upon publishing I get the following error:
Error: Meteor does not currently support objects other than ObjectID as ids
This happens when sub.added
and sub.changed
are called.
Executing the aggregate and then manually running the same functions on the subscription object, it seems to accept strings but not ObjectID
objects.
So, to me, the issue seems to be the fact that obj._id
is being passed as the second parameter, rather than obj._id.toHexString()
.
Has anyone run into this situation?
Am I missing something?
I am running Meteor 1.4.2 and v0.7.0 of the package.
THANKS!
Hello,
I want to ask about this package.
If I query collection by using this reactively method, will be it reactively too on the server or not?
Normally a Publish method can return an array of cursors to publish results from multiple collections. However, since ReactiveAggregate uses the added/changed/removed interface directly, its result can't be combined with a cursor, or another aggregation, as part of a single publish. Either a version of the interface that returns a Cursor should be provided, or ReactiveAggregate should be able to take some cursors to push to the client along with the aggregated data.
This is my publish:
Meteor.publish("reportTotals", function() {
var theHours = ReactiveAggregate(this, Reports, [{
$group: {
'_id': this.userId,
'hours': {
$sum: '$hours'
},
'magazines': {
$sum: '$magazines'
},
'brochures': {
$sum: '$brochures'
},
'books': {
$sum: 'books'
}
}
}, {
$project: {
userId: '$_id',
hours: '$hours',
magazines: '$magazines',
brochures: '$brochures',
books: '$books'
}
}]);
console.log(theHours);
});
In console, I get the response: undefined
I might be missing something
Hi,
Let's say I got this object structure in a collection:
{
"_id" : "gxDT7d49mgPh65h9P",
"serviceTypeId" : "ayNc88uLkwZT9kn3m",
"status" : "Done",
"projectId" : "g8yCFambJxwFxY6Ms",
"employee" : {
"name" : "Adán",
"lastname" : "Enríquez"
},
"serviceType" : {
"name" : "Tipo2"
},
"dateUpdated" : ISODate("2017-04-30T10:48:06.065Z"),
"plannedStartDate" : ISODate("2017-04-30T10:47:00Z"),
"plannedEndDate" : ISODate("2017-04-30T10:48:00Z"),
"executedStartDate" : ISODate("2017-04-30T10:48:00Z"),
"executedEndDate" : ISODate("2017-04-30T10:48:00Z")
}
I'm using aggregation to get items in a month, grouped by serviceTypeId
and plannedStartDate
or executedEndDate
. I'm using something like this (no date is mandatory):
Meteor.publish("serviceTypesCounts", function(startDate, endDate) {
// init dates to current month if not provided:
startDate = startDate || moment().startOf('month').toDate();
endDate = endDate || moment().endOf('month').toDate();
// Remember, ReactiveAggregate doesn't return anything
ReactiveAggregate(
this,
Services,
[
{
$match: {
// match current user's projectId and either planned or executed date exists, and any of those dates are inside startDate, endDate range:
$and: [
{projectId: getCurrentProjectId(this.userId)},
// plannedStartDate or executedStart date exists:
{$or: [
{plannedStartDate: {$exists: true}},
{executedStartDate: {$exists: true}}
]},
// plannedStartDate is in startDate, endDate range or it doesn't exist and executedStartDate is:
{$or: [
{
plannedStartDate: {
$gte: startDate,
$lte: endDate
}
},
{
$and: [
{
executedStartDate: {
$gte: startDate,
$lte: endDate
}
},
{
plannedStartDate: {
$exists: false
}
}
]
}
]}
]
}
},
{
$group: {
'_id': {
serviceTypeId: "$serviceTypeId",
year: {
$cond: [
{ $ifNull: ['$plannedStartDate', 0] },
{ $year: '$plannedStartDate' },
{ $year: '$executedStartDate'}
]
},
month: {
$cond: [
{ $ifNull: ['$plannedStartDate', 0] },
{ $month: '$plannedStartDate' },
{ $month: '$executedStartDate'}
]
},
day: {
$cond: [
{ $ifNull: ['$plannedStartDate', 0] },
{ $dayOfMonth: '$plannedStartDate' },
{ $dayOfMonth: '$executedStartDate'}
]
},
},
'count': { $sum: 1 }
}
},
{
$project: {
_id: {
$concat: [
'$_id.serviceTypeId',
'-',
{$substr: ["$_id.year", 0, 4]},
{$substr: ["$_id.month", 0, 2]},
{$substr: ["$_id.day", 0, 2]}
]
},
year: '$_id.year',
month: '$_id.month',
day: '$_id.day',
count: '$count'
} // Send the aggregation to the 'clientReport' collection available for client use
}
],
{
clientCollection: "ServiceTypesCounts"
}
);
});
I works, I'm getting results like this:
{ _id: 'ayNc88uLkwZT9kn3m-2017521', count: 1, year: 2017, month: 5, day: 21 }
{ _id: 'YG873A6AZZ3eEAkqX-2017520', count: 1, year: 2017, month: 5, day: 20 }
{ _id: 'ayNc88uLkwZT9kn3m-2017520', count: 1, year: 2017, month: 5, day: 20 }
{ _id: 'YG873A6AZZ3eEAkqX-201757', count: 26, year: 2017, month: 5, day: 7 }
{ _id: 'YG873A6AZZ3eEAkqX-201758', count: 340, year: 2017, month: 5 ,day: 8 }
Now, thing is: I want to get field serviceType.name
inside original objects (or event the full object), how can a I do this?
It would be great to get that field even if I'm not grouping by it, but in this case I don't care as it is the readable representation for serviceTypeId.
Thanks!
I use $lookup
aggregate function in Mongo 3.2
.
ReactiveAggregate(this, Disbursement, [
{$match: {_id: _id}},
{
$lookup: {
from: "loan_client",
localField: "clientId",
foreignField: "_id",
as: "clientDoc"
}
},
{$unwind: "$clientDoc"},
.............
It don't work when I tried to change client info
.
a = new Mongo.Collection('a');
b= new Mongo.Collection('b');
c= new Mongo.Collection('c');
d= new Mongo.Collection('d');
a.insert({ "aaa":"hey" "aa":'1' });
b.insert({ "bbb":"hi" "bb": '1' });
Meteor.publish("c", function () {
const pipeline = [{
$lookup: {
from: 'b', localField: 'aa', foreignField: 'bb', as: 'ab', } }];
ReactiveAggregate(this, a, pipeline, { clientCollection: "d" });
});
when i return 'd' it returns
{"aaa":"hey" "aa":'1' "ab": [] }
although "ab" should have "aaa": "hey"
I believe?
Hi there,
I'm opening this issue as follow-up to Issue #11 regarding the subscription .ready()
method. My Meteor.subscribe('some_aggregate_pub').ready()
always returns false if the aggregate has no $match
results. Is there a work-around to check if there are no results and the subscription is ready?
Thank you!
I'm using the package in many places in the project, it is working perfectly with all collections with med size, once i use it with little bigger collection more than 36k doc, it takes some time till finished more than '8 sec' and the pipeline is not complicated 'only simple match', can you please help to figure out this issue if anyone faced that before.
MongoDB server version: 4.0.6
Meteor version: 1.8.1
I test this package, just using simple pipeline:
ReactiveAggregate(this, loanLogs, [{
$match: {
own:user.username,
time: {
$gt: new Date("2016-09-01")
}
}
}], {
clientCollection: "todayLoanTotal"
});
It works, but not well, costing 10s~20s to get data at first time .
Then debug your code ,
var handle = query.observeChanges({
added: update,
changed: update,
removed: update,
error: function (err) {
throw err;
}
});
observeChanges
is blocked for 10~20 seconds. Why? Do I miss something esle? Thanks!
Hi, love your project, but could you please publish the 1.0 version to atmosphere? It's stuck on 0.7 for 2 years now.
//server side code
`UserLogging = new Mongo.Collection('user_logging');
const options = {
observeOptions: {
limit: 10,
sort: { createdAt: -1 },
}
};
Meteor.publish("reportTotals", function() {
// Remember, ReactiveAggregate doesn't return anything
ReactiveAggregate(this, UserLogging, [{
// assuming our Reports collection have the fields: hours, books
$group: {
'_id': "$userId",
count: {
// In this case, we're running summation.
$sum: 1
}
}
}, {
$project: {
// an id can be added here, but when omitted,
// it is created automatically on the fly for you
userId:'$userId',
count: '$count'
} // Send the aggregation to the 'clientReport' collection available for client use
},options], { clientCollection: "ClientUser" });
});`
//client side code
`ClientUser = new Mongo.Collection('clientuser');
Meteor.subscribe("reportTotals");
console.log("****hllojj");
Template.userList.helpers({
user: function() {
console.log("****hllojj");
return UserLogging.find({},{limit:20});
}
});
`
you may have understood what iam doing right...
it is returning with empty
can you please suggest anything..
any new mwthods to achieve reactive publish is also works...
thanks in advance
Hi @JcBernack ,
First of all thanks for this package! It make all my aggregate reactive! 👍
is it possible to aggregate two collections in one publication?
Use case:
I have productName
and premium
fields in enrollments
collection.
I have status
, products
and amountPaid
on claim
collection.
I want to calculate the loss ratio
= total money retained
, here's the formula.
(sum of all premium / sum of all amount paid) * 100 = money retained in %
the productName
field from collection 1 and products
field from collection 2 is basically the same, they are foreign keys from products
collection as well as premium
.
I've successfully aggregated/calculated money retained
per product that matches a certain status
if premium
is on claim
collection but the problem is it will only calculate the total premium of all matching status, the requirement is for all the products(productName) availed in enrollments
collection.
Here's my working code:
Meteor.publish( "lossRatio", function () {
ReactiveAggregate( this, Claim, [ {
$match: {
$or: [ {
'status': "ApprovedRB"
}, {
'status': "Approved"
} ]
}
}, {
$group: {
'_id': "$products",
'amountPaid': {
$sum: '$amountPaid'
},
'premium': {
$sum: '$premium'
}
}
}, {
$project: {
status: '$status',
amountPaid: '$amountPaid',
premium: '$premium',
retained: {
$multiply: [ {
$divide: [ '$premium', '$amountPaid' ],
}, 100 ]
}
},
} ], {
clientCollection: "LossRatio"
} );
} );
Notice the premium is in claim
collection but I mentioned earlier that it is on enrollments
collection, this is for testing purposes to see if will work and it works.
The issue is that the calculated total premium
should be based on the total productName
on enrollments
collection not the total products
with matching status
on claim
collection.
Is this possible? I am thinking of creating two separate publications and combine them in one collection. Is this theory possible?
When I delete a record by Meteor Methods, the reactive don't push the update.
When a do something directly in mongodb, it does normally. I forgot some configuration?
Could reactive distinct be a possibility?
https://docs.mongodb.com/manual/reference/method/db.collection.distinct/
It would be very useful to be able to restart aggregation in cases when you want to observe some unrelated changes which serve as an input to the aggregation for example.
As for implementation, I think returning a cleanup function from ReactiveAggregate would work.
If you like this idea I can help implement it.
Thoughts?
The way this is written, if a collection has 100 documents, collection.aggregate
will get called 100 times in quick succession, once for every document that's added initially 😄
It should really be throttled with a customizable wait time, and it would be ideal if it could wait until the all the initial documents have been added, but unfortunately there's no way to know when observeChanges
is ready...
Hi there
i am new to Meteor, so it might be that i am missing out some basics here.
I am using the package to publish the result of a $lookup + $project + $match aggregation to a reactive native app.
The magic is that, on the react side (within createContainer()), i am getting an array of only one document.
If instead put a $group alone in the pipeline, i get all the expected documents (13).
I am using publish/subscribe on other collections (without aggregation), and they work just fine.
Thanks in advance.
i have this line of code..
$match: { createdAt: {$gte: new Date(dateFrom), $lte: new Date(dateTo)} }, $group: { '_id': "$userName", 'points': { // In this case, we're running summation. $sum: '$points' }, 'reportCount': { $sum: 1 } }
how to approach this ?
I think would be nice add an example on readme to show how to import the ReactiveAggregate module on the script because I had some problems to import the module because I did not have one example and other guys had the same problem #45
I like the idea of this package and i remember reading one day that Aggregations can return a cursor.
Im wondering instead of refetching the documents from the DB https://github.com/JcBernack/meteor-reactive-aggregate/blob/master/aggregate.js#L34 and tracking that cursor if its possible to follow the aggregation cursor discussed here https://docs.mongodb.org/v3.0/reference/method/db.collection.aggregate/#cursor-behavior
Hi,
Thanks for this great lib.
I've adopted it in my project, and found a bug with the "subscription.ready()" method being called too early. It seems like it's already been fixed in this repo, but no new version has been published, so people installing the package don't get the bugfix, is that correct ?
In the meantime I've copied and pasted the code in my own repo.
If you are not working on this anymore I'd totally understand
Hello,
Because meteor-reactive-aggregate is using rawCollection to call aggregate on collection, result contains plain Mongo types instead of Meteor ones. It is a problem for custom EJSON types returned in aggregate or when ObjectID or Decimal128 is used (see Scale Factor aggregation in https://docs.mongodb.com/manual/tutorial/model-monetary-data/).
This can be fixed by adding same code which is used in meteor/mongo.js for converting types:
diff --git a/aggregate.js b/aggregate.js
index b7e7cc9..2cb5683 100644
--- a/aggregate.js
+++ b/aggregate.js
@@ -1,6 +1,81 @@
import { Meteor } from 'meteor/meteor';
import { Mongo } from 'meteor/mongo';
+// FIXME: Copied this from mongo.js
+import { NpmModuleMongodb as MongoDB } from 'meteor/npm-mongo';
+import { EJSON } from 'meteor/ejson';
+
+var replaceNames = function (filter, thing) {
+ if (typeof thing === "object" && thing !== null) {
+ if (_.isArray(thing)) {
+ return _.map(thing, _.bind(replaceNames, null, filter));
+ }
+
+ var ret = {};
+
+ _.each(thing, function (value, key) {
+ ret[filter(key)] = replaceNames(filter, value);
+ });
+
+ return ret;
+ }
+
+ return thing;
+};
+
+var unmakeMongoLegal = function (name) {
+ return name.substr(5);
+};
+
+var replaceMongoAtomWithMeteor = function (document) {
+ if (document instanceof MongoDB.Binary) {
+ var buffer = document.value(true);
+ return new Uint8Array(buffer);
+ }
+
+ if (document instanceof MongoDB.ObjectID) {
+ return new Mongo.ObjectID(document.toHexString());
+ }
+
+ if (document instanceof MongoDB.Decimal128) {
+ return Decimal(document.toString());
+ }
+
+ if (document["EJSON$type"] && document["EJSON$value"] && _.size(document) === 2) {
+ return EJSON.fromJSONValue(replaceNames(unmakeMongoLegal, document));
+ }
+
+ if (document instanceof MongoDB.Timestamp) {
+ // For now, the Meteor representation of a Mongo timestamp type (not a date!
+ // this is a weird internal thing used in the oplog!) is the same as the
+ // Mongo representation. We need to do this explicitly or else we would do a
+ // structural clone and lose the prototype.
+ return document;
+ }
+
+ return undefined;
+};
+
+var replaceTypes = function (document, atomTransformer) {
+ if (typeof document !== 'object' || document === null) return document;
+ var replacedTopLevelAtom = atomTransformer(document);
+ if (replacedTopLevelAtom !== undefined) return replacedTopLevelAtom;
+ var ret = document;
+
+ _.each(document, function (val, key) {
+ var valReplaced = replaceTypes(val, atomTransformer);
+
+ if (val !== valReplaced) {
+ // Lazy clone. Shallow copy.
+ if (ret === document) ret = _.clone(document);
+ ret[key] = valReplaced;
+ }
+ });
+
+ return ret;
+};
+//FIXME: END
+
const defaultOptions = ({
collection, options
}) => ({
@@ -48,6 +123,7 @@ export const ReactiveAggregate = function (subscription, collection, pipeline =
}
// cursor is not done iterating, add and update documents on the client
else {
+ doc = replaceTypes(doc, replaceMongoAtomWithMeteor);
if (!subscription._ids[doc._id]) {
subscription.added(clientCollection, doc._id, doc);
} else {
I'm not sure however how to solve this without copy-paste.
Antoni.
Hi, I want to sort the collection by count in descending orden and limit to 5 documents. I tried to use the sort and limit but couldn't make it work.
Meteor.publish("topics", function () {
ReactiveAggregate(this, Portadas, [
{
$project: {
topic: { $split: ["$title", " "] },
date_seconds: 1,
documento: "$$ROOT"
}
},
{
$unwind: {
path: "$topic"
}
},
{
$match: {
topic: { $nin: ["sports", reg1, reg2,reg3] },
date_seconds: { $gte: timestamp }
}
},
{
$group: {
_id: "$topic",
count: { $sum: 1 },
articles: { $push: "$documento" }
}
}
], { clientCollection: "clientTopic" });
});
I'm trying to figure out why my application is fetching all the documents in my collections.
I have a publication that looks like this:
Meteor.publish('location_updates', function(){
let pipeline = [
{
$match: {
userId: this.userId
}
},
{
$sort: {
'location.timestamp': -1
}
},
{
$limit: 1
},
{
$project: {
'location.coords': 1,
'location.uuid': 1
}
}
];
ReactiveAggregate(this, Location_updates, pipeline);
});
On the client-side I'm selecting a single document:
this.subscribe('location_updates');
....
Location_updates.find({'location.uuid': pos.uuid}, { limit: 1 });
Yet in the Galaxy APM I'm seeing this when I load the app:
That's every single document in the location_updates collection.
The same goes for all my other publications that use ReactiveAggregate.
Hello,
my clientCollection is empty.
Inventories looks like this: "{ "_id" : "5279859913", "userId" : "Jh3DRj3sCwqZiAhAB", "name" : "Test" }"
ReactiveAggregate(this, Inventories, [{ $match: { 'userId': 'Jh3DRj3sCwqZiAhAB' } }, { $project: { 'userId': '$userId', 'name': '$name' } } ], { clientCollection: "clientBotInventory" });
I use ReactiveAggregate
in publlication. It work fine when I run it but in test mode it does not work for some reason.
Any Idea why ?
First of all, this package is great.Thanks. But I have trouble when getting correct subscription ready() value. The ready() is always kept to false status. How can I know the subscription is ready?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.