Comments (12)
@neilbowers++ your excellent question helped me find some missing support in the Client.
(real user problems FTW)
So, first I just released it with the following support:
- type 'favorites' in 'all' (== match_all) queries
- 'facets' keyword allowed in the params
Now, theoretically you should be able to do this query using these two features.
I managed to get close with this code: http://pastebin.com/3NB8xHud
and read the results from $favs->facets->{distribution}{terms}.
But I ran into the following problems with it (need some advice here on how to do it properly, @clintongormley? @oalders?):
- I had to perform 2 queries to determine the size, as I can't find a way to iterate over the facets results through the scroller object (don't think it's supported and I read 'facets' is a deprecated feature anyway in ES)
- The number of results are limited because of a server limit (only got ~5K out of ~23K)
- I couldn't find (yet) a way to filter on the count per dist in the facets results from the ES (rather than my plan to do it in Perl)... that might have pseudo-solved (2)
Will appreciate some help in solving this guys.
Cheers,
Mickey
from metacpan-client.
@mickeyn Are dists labelled with whether they have favourites or not? Or are favourites completely separate docs? If, as I suspect, it is the latter, then what you are doing is probably about as good as it gets, but it is a really inefficient way of doing it.
instead, we should look at changing the data model to support these types of queries. Possibly parent-child would be a good solution here.
from metacpan-client.
@clintongormley thanks for the quick reply.
I can't find a reference to the favorite in either
http://api.metacpan.org/v0/distribution/_mapping
or http://api.metacpan.org/v0/release/_mapping
So I guess you suspected correctly.
I guess we'll have to get @oalders's opinion on your suggestion.
from metacpan-client.
If you're open to changes to the data model, then how about putting the number of favourites on the distribution, since that would make it much easier to get at.
In for a penny ...
from metacpan-client.
I agree with @neilbowers that having the favorites count on the distributions will be helpful.
At the same time we remain with the same issue for other aggregations we may need.
(or not, if I'm just missing the right way to do it)
from metacpan-client.
Favourites have their own endpoint: http://api.metacpan.org/v0/favorite/_mapping There's an argument for moving these over to distribution, I think. The issues would be API versioning and also backups, since this is user metadata which cannot be recovered from a re-index. So, it's a non-trivial task, especially since it requires moving the API away from v0. We have to do this at some point anyway, but we don't have a proper plan for it yet.
from metacpan-client.
@oalders I don't think we wanted to move the favorites, just to add the total count ... if that makes sense and/or simplifies the solution.
from metacpan-client.
@neilbowers OK, so I got one important thing wrong - the facets do allow me to get all results, it's just that without the 'all_term => "true"' filter I didn't get the ones with count zero (so no server limit thing) which is what you asked for.
so you can reuse the code snippet above for your answer (as inefficient as it may be :)), or use the full example I just commited:
https://github.com/CPAN-API/metacpan-client/blob/master/examples/all_favorite_counts_per_dist.pl
Cheers,
Mickey
from metacpan-client.
@neilbowers a simpler (one less query) example to fetch the top 20 ones:
https://github.com/CPAN-API/metacpan-client/blob/master/examples/top20_favorite_distributions.pl
from metacpan-client.
@mickeyn thank you -- I'll have a go with this either this evening or tomorrow.
Neil
from metacpan-client.
closing (please reopen if needed)
from metacpan-client.
For reference, this can now be done via https://fastapi.metacpan.org/v1/favorite/agg_by_distributions?distribution=Moose
from metacpan-client.
Related Issues (20)
- testsuite fails on alpine linux HOT 9
- fav.pl shipped? HOT 3
- Include the coverage data in the data returned by the recent call HOT 3
- How to fetch the issues of a release/distribution? HOT 5
- How to fetch kwalitee metrics? HOT 12
- tests fail in fetch() HOT 1
- t/api/reverse-dependencies.t started to fail HOT 2
- ->reverse_dependencies returns only 50 results even if there are more HOT 1
- t/ua_trap.t fails (if WWW::Mechanize::Cached is too old?) HOT 6
- Undeclared dependency LWP::Protocol::https HOT 6
- failed to delete scroller at .../MetaCPAN/Client/Scroll.pm line 162. HOT 4
- Handling of HTTP/transport errors
- t/api/author.t fails HOT 1
- specifying fields in requests can break array type checking HOT 1
- `fields` is documented that it will accept a csv string, but fails if it's not an arrayref HOT 1
- "Wrong type of query arguments" after upgrading from v0-era MetaCPAN::Client HOT 5
- reverse_dependencies() points to an invalid URL HOT 2
- Reverse dependencies method misses some modules HOT 1
- reverse_dependencies broken? HOT 3
- Infinite loop while( my $rs = scroller->next ) HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from metacpan-client.