Simple filter-than-group test appears to misbehave about crossfilter HOT 11 CLOSED

This is the expected behavior: the records are filtered, not the groups. In your example, the groups are returned in the expected order by descending value, including those groups whose value happens to be zero.

from crossfilter.

Thanks for the note. I believe this is simply a misunderstanding of the documentation. In your example, the links group definitely is filtered by tokens matching "Jimmy", but it contains empty groups (value == 0). That's why you're seeing three results. From the documentation:

If there are fewer than k groups according to all of the tesseract's filters, then an array smaller than k will be returned. If there are fewer than k non-empty groups, this method may also return empty groups (for example, those with a count of zero).

In your example, k is Infinity, so all the empty groups will be returned. Using this jsfiddle, I got these results:

links.top(Infinity) // no filters
[
  {"key":"http://www.engadget.com/2012/03/17/us-navy-lasr-research-facility-builds-robots-not-ray-guns/",
   "value":20},
  {"key":"http://feedproxy.google.com/~r/oreilly/radar/atom/~3/VUnm8ikk8a4/encyclopaedia-britannica-copyright-paypal-censorship.html",
   "value":10},
  {"key":"http://feeds.arstechnica.com/~r/arstechnica/index/~3/_dcrXs9GTTo/ion-beam-manufacturing-halves-production-cost-of-pv-panels.ars",
   "value":9}
]

links.top(Infinity) // filtering token by "Jimmy"
[
  {"key":"http://www.engadget.com/2012/03/17/us-navy-lasr-research-facility-builds-robots-not-ray-guns/",
   "value":0},
  {"key":"http://feedproxy.google.com/~r/oreilly/radar/atom/~3/VUnm8ikk8a4/encyclopaedia-britannica-copyright-paypal-censorship.html",
   "value":1},
  {"key":"http://feeds.arstechnica.com/~r/arstechnica/index/~3/_dcrXs9GTTo/ion-beam-manufacturing-halves-production-cost-of-pv-panels.ars",
   "value":0}
]

Note that there are the same number of group results in each but the values are different. Also note that I used console.log(JSON.stringify(...)) because tesseract reuses the groups (I think) so inspecting their values after modifying them by filtering is misleading. If I completely misunderstood, please reopen this issue with a clarification. Thanks!

from crossfilter.

Thanks to you both for the clarification--I can see now that the behavior is expected. The jsfiddle results were particularly helpful.

Any advice on how I might get obtain only the set of links that contain the token "Jimmy"? For my purposes, the empty groups are just noise.

from crossfilter.

Just ignore the entries with value == 0.

from crossfilter.

Are you suggesting to post-processing of the result set (e.g. a simple iteration through the results of the group function)? I don't see any "ignore entries" function for the group results listed in the API reference.

from crossfilter.

Correct. This would work:

links.top(Infinity).filter(function(d) { return d.value; })

If you have lots and lots of groups, you could also use bisection (tesseract.bisect) to find the first zero-group and cut the array there. Or, if you're planning on iterating over the groups, just stop iterating when the value is zero.

from crossfilter.

Excellent, thanks.

In investigating why group.top(Infinity) was preferable to group.all(), I encountered this in the group.all() documentation:

Unlike top, however, the returned array may contain empty groups, whose value is the return value from the group's reduce initial function.

Doesn't this conflict with the statement (two or three lines above) that eventualbuddha referenced?

If there are fewer than k non-empty groups, this method may also return empty groups (for example, those with a count of zero).

from crossfilter.

I'll fix the documentation re. group.all(). I don't recall exactly, but I think the behavior may have changed from an earlier (unreleased) version of Tesseract that did not use incremental update for groups.

It might be possible to filter zero-value groups from group.top, but that would either require tracking the count separately (which is a non-significant cost), or by comparing the top groups to the result of the reduceInitial function using the group's order accessor (using bisection, so it'd be fast). It's a bit tricky since the top groups are in descending order, and the bisection assumes ascending order, but in theory it's possible. And, I think it would improve the API.

from crossfilter.

I can't speak for others, but filtering zero-value groups would certainly be useful for my usage of the API. :) I can see the zero-value groups being useful in other cases, though, so I think you'd want to make the filter optional.

On the subject of API adjustments, have you considered aliasing top(Infinity) with an overload of the all function? Using group.all(Boolean clone) would be slightly less astonishing to me, compared to top(Infinity).

from crossfilter.

No; group.top returns groups by descending value, whereas group.all returns groups by ascending key. The top method thus requires selecting + sorting, whereas the all method just returns the groups as-is. Another difference is that group.top returns a copy (of maximum size K), whereas group.all returns the internal array. I don't think it's a good idea to merge these into a single method, given that they do different things.

from crossfilter.

Arrr, I must be getting senile--I thought I read both result sets were in descending order. Sorry.

from crossfilter.