metacpan / metacpan-client Goto Github PK
View Code? Open in Web Editor NEWHome of the official MetaCPAN Perl API client.
metacpan-client's People
Contributors
Stargazers
Watchers
Forkers
kevindawson sineswiper reneeb kentfredric alexmv closeststorm ansoni-san bk karenetheridge stevan pghmcfc grinnz pkg-perl-tools autarch wolfsage clayne atoomic stigtsp onenos-com cmburn haargmetacpan-client's Issues
CPAN Pull Request Challenge
I was assigned this repo for the April CPAN Pull Request Challenge and am wondering which kinds of contributions you would like to receive.
List of the files in a given distribution?
Is there an easy way to fetch that list?
Can I fetch the list of modules in a distro?
Can I fetch the list of files with POD in a distro?
Question: Convert dependency data obtained by MetaCPAN::Client to cpanfile?
I'm just asking this as I would like to get module dependency data in cpanfile (or CPAN::Meta::Prereqs) format from data fetched via MetaCPAN::Client. I guess it's not unusual case, so someone else might have made some shortcut there. If there's no existing stuffs it's fine for me to do it myself. Thanks!
t/ua_trap.t fails (if WWW::Mechanize::Cached is too old?)
Some of my smokers show the following test failure:
Use of uninitialized value in concatenation (.) or string at /usr/perl5.18.4p/lib/site_perl/5.18.4/HTTP/Tiny.pm line 593.
# Failed test 'No illegal methods called'
# at t/ua_trap.t line 45.
# got: 'Can't use string ("default_expires_in") as a HASH ref while "strict refs" in use at /usr/perl5.18.4p/lib/site_perl/5.18.4/Cache/BaseCache.pm line 430.
# '
# expected: undef
# Can't use string ("default_expires_in") as a HASH ref while "strict refs" in use at /usr/perl5.18.4p/lib/site_perl/5.18.4/Cache/BaseCache.pm line 430.
# Looks like you failed 1 test of 1.
t/ua_trap.t .............
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests
This seems to happen if all prereqs for running ua_trap.t are met, but WWW::Mechanize::Cached is too old (<1.50?). Statistical analysis:
(1)
****************************************************************
Regression 'fail:t/ua_trap.t'
****************************************************************
Name Theta StdErr T-stat
[0='const'] 1.0000 0.0000 38772244432185280.00
[1='eq_1'] -1.0000 0.0000 -7078810960203983.00
R^2= 1.000, N= 60, K= 2
****************************************************************
(2)
****************************************************************
Regression 'qr:(Failed test\s+\S+.*)'
****************************************************************
Name Theta StdErr T-stat
[0='const'] 1.0000 0.0000 38772244432185280.00
[1='eq_Failed test 'No illegal methods called''] -1.0000 0.0000 -7078810960203983.00
R^2= 1.000, N= 60, K= 2
****************************************************************
(3)
****************************************************************
Regression 'qr:WWW::Mechanize::Cached\s+\S+\s+(.*)'
****************************************************************
Name Theta StdErr T-stat
[0='const'] 0.0000 0.0000 5.07
[1='eq_1.50'] 1.0000 0.0000 6859754891225717.00
[2='eq_missing'] 1.0000 0.0000 6668443905700334.00
R^2= 1.000, N= 60, K= 3
****************************************************************
failed to delete scroller at .../MetaCPAN/Client/Scroll.pm line 162.
I have a script which is, in outline:
created MetaCPAN::Client
iterate on releases, collecting information
do a bunch of processing on the information
exit
I came back to see that the script had finished, with the final line of output being:
failed to delete scroller at .../MetaCPAN/Client/Scroll.pm line 162.
I assume resources on the server side time out, which is what's causing this, but given it's cleanup in a DESTROY (another assumption), maybe this could be suppressed?
->modules query prematurely flattens to scalar
The following code worked prior to the ES upgrade.
#!perl
use strict;
use warnings;
use MetaCPAN::Client;
my $name = 'AcePerl-1.92';
my $author = 'LDS';
my $client = MetaCPAN::Client->new();
my $iterator = $client->module(
{
all => [
{ release => $name }, #
{ author => $author },
{ authorized => 1 },
{ indexed => 1 }, #
{ mime => 'text/x-script.perl-module' }, #
]
}
);
while ( my $result = $iterator->next ) {
next unless $result->module;
for my $module (@{ $result->module }) {
printf "%s\n", $module->{name};
}
}On 1.020000 this specific logic works as expected.
On 1.021000, it gets confused at some point and bails quite early as $result->module no longer returns an arrayref.
CPANTESTERS results please
This M::C unrelated snippet currently returns invalid values and often timeouts
# CPANTESTER results
$opt_v > 1 and warn " Fetch cpantesters\n";
!defined $data->{cptst} and
$data->{cptst} =
($r = CPAN::Testers::WWW::Reports::Query::AJAX->new (dist => $dist))
? [ $r->pass, $r->na, $r->fail, $r->unknown ]
: [ "", "", "", "" ];I would like to see valid actual (recent) values for CPANTESTERS results for a given distribution, somewhat like
$ perl -MMetaCPAN::Client -MData::Peek -wE'DDumper (MetaCPAN::Client->new (version => "v1")->distribution ("Text-CSV_XS")->cpantesters)'
{ pass => 214,
fail => 0,
na => 0,
}
dead attributes in objects when 'fields' used as param
Using an example provided by @neilbowers I see the following problem in objects creation:
If we take the extra parameter 'fields' and pass it to the query, thus limiting the result we get back, the object we create (e.g. ::Release) is going to miss information in attributes which were not in 'fields' (undef is returned for those).
We can prevent that by restricting/ignoring the use of 'fields' as a parameter when we return objects (but not in future raw results mode) - any reasons why we shouldn't do it? any suggestions for the solution?
RT and GitHub issues side-by-side
$ perl -MMetaCPAN::Client -MDP -wE'DDumper (MetaCPAN::Client->new (version => "v1")->distribution ("Text-CSV_XS")->bugs)'
{ active => 2,
closed => 100,
new => 0,
open => 1,
patched => 1,
rejected => 19,
resolved => 81,
source => 'https://rt.cpan.org/Public/Dist/Display.html?Name=Text-CSV_XS',
stalled => 0,
type => 'rt'
}
Discussion on IRC learned that github issues might have returned a similar struct, but if RT was also found, it got overwritten. Maybe better would be to add a level
$ perl -MMetaCPAN::Client -MDP -wE'DDumper (MetaCPAN::Client->new (version => "v1")->distribution ("Text-CSV_XS")->bugs)'
{ rt => {
active => 2,
closed => 100,
new => 0,
open => 1,
patched => 1,
rejected => 19,
resolved => 81,
source => 'https://rt.cpan.org/Public/Dist/Display.html?Name=Text-CSV_XS',
stalled => 0,
},
gh => {
active => 0,
closed => 1,
new => 0,
open => 0,
source => 'https://github.com/Tux/Text-CSV_XS/issues',
},
};
Raw Query construction for Request class
This really aught to be a discussion of some kind, because I don't really know what I'm doing here.
request->ssearch() is presently poorly designed for doing raw queries.
You can hack around it with
request->ssearch('author', { bogus => 1 }, { %raw_query_here });And it works because you can stuff {body => } there and blow away the annyoingly constrictive generated one.
But its ugly, and the function is not very obvious. You need the bogus stuff because without it, it simply breaks due to the generated query having $args != 1
I've considered various ways of hacking around it, to make the interface slightly more usable, for instance:
But that solution strikes me as pretty dodgy because of the magical behaviour that is dependent on the value of $args
I even considered
diff --git a/lib/MetaCPAN/Client/Request.pm b/lib/MetaCPAN/Client/Request.pm
index 3c58fcf..2c20bb5 100644
--- a/lib/MetaCPAN/Client/Request.pm
+++ b/lib/MetaCPAN/Client/Request.pm
@@ -158,6 +158,9 @@ sub _build_query_rec {
ref $args eq 'HASH' or croak 'query args must be a hash';
my %query = ();
+
+ return \%query if not keys %{$args};
+
my $basic_element = 1;
KEY: for my $k ( qw/ all either not / ) {But really, that's the same problem all over again, magical behaviour with a magic token.
I think this, slightly more complex approach would be better:
https://gist.github.com/kentfredric/65295334188a7fba4024
Simply because it keeps the existing API exactly the same, doesn't introduce any magical token behaviour, and gives a useful raw search at the same time.
Incidentally, that approach was the first one I tried, and the other ones are later ideas I had after I realised you could already pass custom params.
Handling of HTTP/transport errors
If HTTP::Tiny receives an HTTP error (like server error 500) or encounters a connection error, it will set the error code in the response (599 for connection errors). The documentation should mention if and how MetaCPAN::Client deals with these error conditions.
Can't locate object method "dist" via package "MetaCPAN::Client"
is this a pod issue -> https://metacpan.org/pod/MetaCPAN::Client#distribution
#!perl
use strict;
use warnings;
use Carp::Always::Color;
use Data::Printer;
use MetaCPAN::Client;
my $mcpan = MetaCPAN::Client->new();
my $dist = $mcpan->dist('MetaCPAN-Client');
##Can't locate object method "dist" via package "MetaCPAN::Client" at mcc_01.pl line 11.
__END__
p $mcpan;
MetaCPAN::Client {
Parents Moo::Object
public methods (15) : author, BUILDARGS, distribution, favorite, fetch, file, module, new, post, rating, release, request, reverse_dependencies, ssearch, ua
private methods (5) : _assert_request, _get, _get_or_search, _reverse_deps, _search
internals: {
request MetaCPAN::Client::Request
}
}
So I think you mean the following
my $dist = $mcpan->distribution('MetaCPAN-Client');specifying fields in requests can break array type checking
Elasticsearch isn't always smart about returning an array type field as an array, when field params are specified.
For instance, if you try to search for an array type field such as "provides", and a result only has a single value, Elasticsearch can return this field as a scalar. This breaks the type checking in MetaCPAN::Client::Role::Entity.
$ perl -MMetaCPAN::Client -E 'say MetaCPAN::Client->new->release({name => "App-cpanminus-1.7043"}, {fields => ["provides"]})->next->provides;'
Maybe the type checking should gracefully promote scalars to arrays? Not sure what the best solution would be here.
Rename everything to MetaCPAN::Client:
looks like you missed some
https://github.com/CPAN-API/metacpan-client/blob/master/t/api/module.t#L13
documentation order & indenting is wrong
On page https://metacpan.org/pod/MetaCPAN::Client you get the phrase (under method all):
"When called with a second parameter containing a hash ref, will support the following keys:"
But then the keys don't appear, until AFTER download_url.
download_url should be somewhere else, out of the way, so that the keys appear right underneath all.
version_numified should be string, not number
When I do p $release I get version_numified as a number, not a string.
But version_numified can have many many decimal digits (more than Perl can handle in numbers, I believe), so to be precise and not lose precision, we should be able to process it and store it in a database as a string, not a number.
data {
author "LOCALSHOP",
date "2016-11-29T16:57:37",
distribution "WebService-Xero",
maturity "released",
name "WebService-Xero-0.10",
version 0.10,
version_numified 0.1
},
date "2016-11-29T16:57:37"
Add ability to specify fields to be returned
(This was originally RT #99499.)
When getting large data sets, it's very helpful to be able to pare down the number of fields returned. It would be great to be able to do this via MetaCPAN::Client.
ie it's the difference between http://api.metacpan.org/module/Moose?fields=version and http://api.metacpan.org/module/Moose
v1 API does not return the 'authorized' field, even when explictly requested
Sample query using API v0:
$ perl -MData::Dump -MMetaCPAN::Client -e 'my $mcpan = MetaCPAN::Client->new( version => "v0" ); my $results = $mcpan->release( { all => [ { distribution => "MetaCPAN-Client" }, { version => "1.026001" }, ] }, { fields => "authorized,date,name,version" } ); dd($results->next->{'data'});'
{
authorized => "true",
date => "2016-10-19T21:45:01.000Z",
name => "MetaCPAN-Client-1.026001",
version => 1.026001,
}
Using v1 API:
$ perl -MData::Dump -MMetaCPAN::Client -e 'my $mcpan = MetaCPAN::Client->new( version => "v1" ); my $results = $mcpan->release( { all => [ { distribution => "MetaCPAN-Client" }, { version => "1.026001" }, ] }, { fields => "authorized,date,name,version" } ); dd($results->next->{'data'});'
{
date => "2016-10-19T21:45:01",
name => "MetaCPAN-Client-1.026001",
version => 1.026001,
}
I expect this issue is somewhere upstream of MetaCPAN-Client but I don't know where.
Fails tests without '.' in @INC
On 5.25.10 with -Ddefault_inc_excludes_dot
PERL_DL_NONLAZY=1 "/home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/bin/perl5.25.10" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/api/*.t
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/_search.t line 8.
BEGIN failed--compilation aborted at t/api/_search.t line 8.
# Looks like your test exited with 2 before it could output anything.
t/api/_search.t .........
Dubious, test returned 2 (wstat 512, 0x200)
Failed 19/19 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/_get_or_search.t line 8.
BEGIN failed--compilation aborted at t/api/_get_or_search.t line 8.
# Looks like your test exited with 2 before it could output anything.
t/api/_get_or_search.t ..
Dubious, test returned 2 (wstat 512, 0x200)
Failed 10/10 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/_get.t line 8.
BEGIN failed--compilation aborted at t/api/_get.t line 8.
# Looks like your test exited with 2 before it could output anything.
t/api/_get.t ............
Dubious, test returned 2 (wstat 512, 0x200)
Failed 13/13 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/author.t line 9.
BEGIN failed--compilation aborted at t/api/author.t line 9.
t/api/author.t ..........
Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/download_url.t line 8.
BEGIN failed--compilation aborted at t/api/download_url.t line 8.
# Looks like your test exited with 2 before it could output anything.
t/api/download_url.t ....
Dubious, test returned 2 (wstat 512, 0x200)
Failed 7/7 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/file.t line 8.
BEGIN failed--compilation aborted at t/api/file.t line 8.
# Looks like your test exited with 2 before it could output anything.
t/api/file.t ............
Dubious, test returned 2 (wstat 512, 0x200)
Failed 11/11 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/distribution.t line 9.
BEGIN failed--compilation aborted at t/api/distribution.t line 9.
# Looks like your test exited with 2 before it could output anything.
t/api/distribution.t ....
Dubious, test returned 2 (wstat 512, 0x200)
Failed 9/9 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/favorite.t line 8.
BEGIN failed--compilation aborted at t/api/favorite.t line 8.
# Looks like your test exited with 2 before it could output anything.
t/api/favorite.t ........
Dubious, test returned 2 (wstat 512, 0x200)
Failed 10/10 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/pod.t line 7.
BEGIN failed--compilation aborted at t/api/pod.t line 7.
# Looks like your test exited with 2 before it could output anything.
t/api/pod.t .............
Dubious, test returned 2 (wstat 512, 0x200)
Failed 5/5 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/module.t line 8.
BEGIN failed--compilation aborted at t/api/module.t line 8.
# Looks like your test exited with 2 before it could output anything.
t/api/module.t ..........
Dubious, test returned 2 (wstat 512, 0x200)
Failed 10/10 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/rating.t line 8.
BEGIN failed--compilation aborted at t/api/rating.t line 8.
# Looks like your test exited with 2 before it could output anything.
t/api/rating.t ..........
Dubious, test returned 2 (wstat 512, 0x200)
Failed 7/7 subtests
Can't locate t/lib/Functions.pm in @INC (you may need to install the t::lib::Functions module) (@INC contains: /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/lib /home/kent/.cpanm/work/1488804455.843/MetaCPAN-Client-2.006000/blib/arch /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/site_perl/5.25.10 /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/x86_64-linux /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10) at t/api/release.t line 8.
BEGIN failed--compilation aborted at t/api/release.t line 8.
# Looks like your test exited with 2 before it could output anything.
t/api/release.t .........
Dubious, test returned 2 (wstat 512, 0x200)
Failed 5/5 subtests
#
# Versions for all modules listed in MYMETA.json (including optional ones):
#
# === Configure Requires ===
#
# Module Want Have
# ------------------- ------ ----
# ExtUtils::MakeMaker 7.1101 7.24
#
# === Build Requires ===
#
# Module Want Have
# ------------------- ---- ----
# ExtUtils::MakeMaker any 7.24
#
# === Test Requires ===
#
# Module Want Have
# ------------------- ---- --------
# ExtUtils::MakeMaker any 7.24
# File::Spec any 3.66
# Test::Fatal any 0.014
# Test::More any 1.302073
# Test::Requires any 0.10
# base any 2.24
# blib 1.01 1.06
#
# === Test Recommends ===
#
# Module Want Have
# ---------------------- -------- --------
# CPAN::Meta 2.120900 2.150010
# HTTP::Tiny::Mech any 1.001001
# LWP::Protocol::https any 6.07
# WWW::Mechanize::Cached 1.48 1.50
#
# === Runtime Requires ===
#
# Module Want Have
# --------------- ----- ----------
# Carp any 1.42
# HTTP::Tiny 0.056 0.070
# IO::Socket::SSL 1.42 2.047
# JSON::MaybeXS any 1.003008
# JSON::PP any 2.27400_02
# Moo any 2.003000
# Moo::Role any 2.003000
# Net::SSLeay 1.49 1.80
# Ref::Util any 0.113
# Safe::Isa any 1.000006
# Type::Tiny any 1.000006
# URI::Escape any 3.31
# strict any 1.11
# warnings any 1.37
#
t/entity.t .............. ok
t/00-report-prereqs.t ... ok
t/resultset.t ........... ok
t/request.t ............. ok
Use of uninitialized value in concatenation (.) or string at /home/kent/perl5/perlbrew/perls/5.25.10-nossp-sdbm-nopmc-nodot/lib/5.25.10/HTTP/Tiny.pm line 593.
t/ua_trap.t ............. ok
t/scroll.t .............. ok
Test Summary Report
-------------------
t/api/_search.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 19 tests but ran 0.
t/api/_get_or_search.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 10 tests but ran 0.
t/api/_get.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 13 tests but ran 0.
t/api/author.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
t/api/download_url.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 7 tests but ran 0.
t/api/file.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 11 tests but ran 0.
t/api/distribution.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 9 tests but ran 0.
t/api/favorite.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 10 tests but ran 0.
t/api/pod.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 5 tests but ran 0.
t/api/module.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 10 tests but ran 0.
t/api/rating.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 7 tests but ran 0.
t/api/release.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 5 tests but ran 0.
Files=18, Tests=24, 11 wallclock secs ( 0.07 usr 0.03 sys + 3.06 cusr 0.61 csys = 3.77 CPU)
Result: FAIL
Failed 12/18 test programs. 0/24 subtests failed.
make: *** [Makefile:902: test_dynamic] Error 255
add "changes" endpoint
I'd like to be able to call this endpoint from the metacpan-client:
https://fastapi.metacpan.org/v1/changes/KARJALA/XML-MyXML-0.9001
`fields` is documented that it will accept a csv string, but fails if it's not an arrayref
The docs for all(), says see SEARCH PARAMS, and there it says fields can be passed as a csv string. But if you try using that, it will fail with "fields must be an arrayref".
Example code triggering it:
use MetaCPAN::Client;
my $r = MetaCPAN::Client->new->all('releases', { fields => 'author,date' });
Add support for experimental download_url endpoint
Would be great to support this one (which is v1 only): https://api-v1.metacpan.org/download_url/DBIx::Class::Helpers
Question: best approach to get links to all POD documentation
I'm working on writing a docset for Dash.app and I need to get a list of all of the modules that have pod documentation on metacpan. Don't have enough code done yet to share, but my basic approach is this:
my $mcpan = MetaCPAN::Client->new();
my $all_modules = $mcpan->all( 'modules', { fields => [ "name", "module" ] } );
while ( $module = $all_modules->next )
{
my $mod_list = $module->module;
next unless defined( $mod_list->[0]{name} );
foreach my $mod ( @{$mod_list} )
{
HEAD request for metacpan.org/pod/$mod->{name} to check for existance
}
pause 1/4 second for each 100 of these processed
}
write to database where appropriate etc. Is there a better way to do this?
Edit: Leaving the above in place for context. I think the root of my problem/confusion is that it seems that I'm taking a fairly round-about path to do what I think would be a pretty common task: get a list of packages(?).
I'm not 100% sure I grok the relationship between module/file/package/release etc, which is likely why I'm asking this in the first place. Anyway, is there a direct way to get a list of all things in cpan in the form "MetaCPAN::Client" or at least in a format where I can link to the pod documentation url.
"autocomplete" search not possible with this module
Please make it possible to do "autocomplete" calls to the API
autocompletion query term is not URI-encoded
https://metacpan.org/source/MICKEY/MetaCPAN-Client-2.002000/lib/MetaCPAN/Client.pm#L182
That's why a space in the middle makes the server ignore all terms except the first one. I think it's a bug.
ability to sort the results of ->all by any field I want
I'd like to call ->all('releases', {...}) and have ES sort the search results by any field I want.
Search::Elasticsearch 5.x compatibility
As a follow-on from #48, I have got my own usage of MetaCPAN::Client working again in conjunction with Search::Elasticsearch 5.x by installing Search-Elasticsearch-Client-2_0 and making the following changes in MetaCPAN::Client:
diff -up /usr/share/perl5/vendor_perl/MetaCPAN/Client/Request.pm.orig /usr/share/perl5/vendor_perl/MetaCPAN/Client/Request.pm
--- /usr/share/perl5/vendor_perl/MetaCPAN/Client/Request.pm.orig
+++ /usr/share/perl5/vendor_perl/MetaCPAN/Client/Request.pm
@@ -129,6 +129,7 @@ sub ssearch {
my $params = shift;
my $es = Search::Elasticsearch->new(
+ client => '2_0::Direct',
nodes => [ 'https://' . $self->domain ],
cxn_pool => 'Static::NoPing',
send_get_body_as => 'POST',
diff -up /usr/share/perl5/vendor_perl/MetaCPAN/Client/ResultSet.pm.orig /usr/share/perl5/vendor_perl/MetaCPAN/Client/ResultSet.pm
--- /usr/share/perl5/vendor_perl/MetaCPAN/Client/ResultSet.pm.orig
+++ /usr/share/perl5/vendor_perl/MetaCPAN/Client/ResultSet.pm
@@ -22,7 +22,7 @@ has scroller => (
is => 'ro',
isa => sub {
use Safe::Isa;
- $_[0]->$_isa('Search::Elasticsearch::Scroll')
+ $_[0]->$_isa('Search::Elasticsearch::Client::2_0::Scroll')
or croak 'scroller must be an Search::Elasticsearch::Scroll object';
},
predicate => 'has_scroller',
I've tried it successfully with both the v0 and v1 APIs. I haven't tried the 5_0 or older clients to see if they would also work. Obviously this is just a proof of concept and a proper fix would probably entail supporting different client versions, but that's currently beyond my coding ability.
LWP::Protocol::https not specified as a prereq
Tests fail if LWP::Protocol::https is not installed:
Use of uninitialized value in concatenation (.) or string at /opt/perl-5.24.0t/lib/5.24.0/HTTP/Tiny.pm line 526.
# Failed test 'No illegal methods called'
# at t/ua_trap.t line 45.
# got: 'Failed to fetch 'https://api.metacpan.org/v0/author/KENTNL': Protocol scheme 'https' is not supported (LWP::Protocol::https not installed) at /tmpfs/.cpan-build-cpansand/2016080518/MetaCPAN-Client-1.022001-AMgyxF/blib/lib/MetaCPAN/Client.pm line 201.
# '
# expected: undef
# Failed to fetch 'https://api.metacpan.org/v0/author/KENTNL': Protocol scheme 'https' is not supported (LWP::Protocol::https not installed) at /tmpfs/.cpan-build-cpansand/2016080518/MetaCPAN-Client-1.022001-AMgyxF/blib/lib/MetaCPAN/Client.pm line 201.
# Looks like you failed 1 test of 1.
t/ua_trap.t ..............
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests
Pod Coverage test failing with Moo 1.005
Not sure if this is an issue with MetaCPAN-Client, Moo or Pod-Coverage so I'll start here and see where it goes. Since update Moo from 1.003x to 1.005, MetaCPAN-Client's Pod Coverage test fails like this:
$ make test AUTHOR_TESTING=1 RELEASE_TESTING=1
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/api/*.t
t/00-compile.t ............ ok
t/api/_get.t .............. ok
t/api/_get_or_search.t .... ok
t/api/_search.t ........... ok
t/api/author.t ............ ok
t/api/distribution.t ...... ok
t/api/favorite.t .......... ok
t/api/file.t .............. ok
t/api/module.t ............ ok
t/api/pod.t ............... ok
t/api/rating.t ............ ok
t/api/release.t ........... ok
t/author-critic.t ......... ok
t/entity.t ................ ok
# Failed test 'Pod coverage on MetaCPAN::Client::ResultSet'
# at /usr/share/perl5/vendor_perl/Test/Pod/Coverage.pm line 133.
# Coverage for MetaCPAN::Client::ResultSet is 87.5%, with 1 naked subroutine:
# has_scroller
# Looks like you failed 1 test of 12.
t/release-pod-coverage.t ..
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/12 subtests
t/release-pod-syntax.t .... ok
t/request.t ............... ok
t/resultset.t ............. ok
Test Summary Report
-------------------
t/release-pod-coverage.t (Wstat: 256 Tests: 12 Failed: 1)
Failed test: 4
Non-zero exit status: 1
Files=18, Tests=203, 10 wallclock secs ( 0.07 usr 0.01 sys + 6.20 cusr 0.35 csys = 6.63 CPU)
Result: FAIL
Failed 1/18 test programs. 1/203 subtests failed.
Makefile:838: recipe for target 'test_dynamic' failed
make: *** [test_dynamic] Error 255
has_scroller seems to be being pulled in from Search::Elasticsearch so I don't really think there should be pod for it in MetaCPAN::Client. Perhaps Pod-Coverage should be ignoring this somehow? Any suggestions?
(yes, I know I don't need to run the author/release tests but there's a bug here somewhere and I'd like to help get it fixed)
Add mirrors and/or rating?
There is a little known type in the index for CPAN mirrors:
http://api.metacpan.org/mirror/_mapping
It's there, so it may be worth adding. There is also rating: http://api.metacpan.org/rating/_mapping I'm not sure how useful it is, but I guess you can use it to find changes in favorites for some arbitrary period of time.
is this me or ....
hi mickeyn
#!perl
use strict;
use warnings;
use Carp::Always::Color;
use Data::Printer {caller_info => 1,};
use Try::Tiny;
use MetaCPAN::Client;
my $mcpan = MetaCPAN::Client->new();
my $mc = $mcpan->module('Digest::MD5');
p $mc;
my $mc_v = $mc->{data}{version};
p $mc_v;
my $mc_vn = $mc->{data}{version_numified};
p $mc_vn;
my $cpan_version = $mc->{data}{version_numified};
$cpan_version = 'core' if $mc->{data}{distribution} eq 'perl';
p $cpan_version;
exit(0);and you get
module: invalid args (takes scalar value or search parameters hashref) at /home/kevin/perl5/perlbrew/perls/perl-5.18.2/lib/site_perl/5.18.2/MetaCPAN/Client.pm line 181.
MetaCPAN::Client::_get_or_search(MetaCPAN::Client=HASH(0x90112a0), "module", "Digest::MD5", undef) called at /home/kevin/perl5/perlbrew/perls/perl-5.18.2/lib/site_perl/5.18.2/MetaCPAN/Client.pm line 43
MetaCPAN::Client::module(MetaCPAN::Client=HASH(0x90112a0), "Digest::MD5") called at /home/kevin/perl5/perlbrew/perls/perl-5.18.2/lib/site_perl/5.18.2/MetaCPAN/Client.pm line 181.
MetaCPAN::Client::_get_or_search(MetaCPAN::Client=HASH(0x90112a0), "module", "Digest::MD5", undef) called at /home/kevin/perl5/perlbrew/perls/perl-5.18.2/lib/site_perl/5.18.2/MetaCPAN/Client.pm line 43
MetaCPAN::Client::module(MetaCPAN::Client=HASH(0x90112a0), "Digest::MD5") called at mc2.pl line 12
Undeclared dependency LWP::Protocol::https
It seems that LWP::Protocol::https has to be declared as a prerequisite:
Use of uninitialized value in concatenation (.) or string at /home/cpansand/.cpan/build/2017032321/HTTP-Tiny-0.070-zFPGrC/blib/lib/HTTP/Tiny.pm line 593.
# Failed test 'No illegal methods called'
# at t/ua_trap.t line 45.
# got: 'Failed to fetch 'https://fastapi.metacpan.org/v1//author/KENTNL': Protocol scheme 'https' is not supported (LWP::Protocol::https not installed) at /tmpfs/.cpan-build-cpansand/2017032321/MetaCPAN-Client-2.008001-Syu199/blib/lib/MetaCPAN/Client.pm line 212.
# '
# expected: undef
# Failed to fetch 'https://fastapi.metacpan.org/v1//author/KENTNL': Protocol scheme 'https' is not supported (LWP::Protocol::https not installed) at /tmpfs/.cpan-build-cpansand/2017032321/MetaCPAN-Client-2.008001-Syu199/blib/lib/MetaCPAN/Client.pm line 212.
# Looks like you failed 1 test of 1.
t/ua_trap.t .............
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests
Q, pod suggestion
I think you need a POD example to show, attribute usage for module data
do this
use MetaCPAN::Client;
my $mcpan = MetaCPAN::Client->new();
my $mc = $mcpan->module('Digest::MD5');
my $mc_version = $mc->version();instead of this
use MetaCPAN::Client;
my $mcpan = MetaCPAN::Client->new();
my $mc = $mcpan->module('Digest::MD5');
my $mc_version = $mc->{data}{version};ps it tock me a while to figure the above out from the code, when using Data-Printer you can see the exported methods for $mcpan->distribution, but not for $mcpan->module
hope the above helps, from a newbe perspective to MC-C
Searches with fields param broken with Search::Elasticsearch 5.01
I'm doing lookups like this:
my $mcpan = MetaCPAN::Client->new();
# Look up latest stable release
my $results = $mcpan->release(
{ all => [ { distribution => $distro },
{ maturity => 'released' }, ]
},
{ fields => "authorized,date,name,version" }
);
and that works fine at the moment, but after trying Search::Elasticsearch 5.01 I got:
[Param] ** Unknown param (fields) in (search) request. See docs at: http://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html, called from sub Search::Elasticsearch::Client::5_0::Direct::scroll_helper at /usr/share/perl5/vendor_perl/MetaCPAN/Client/Request.pm line 147.
I've reverted to S::E 2.03 for now. I'm not even sure if this was supposed to work really; I don't know which version of Elasticsearch is being used with MetaCPAN.
t/api/author.t fails
t/api/author.t started to fail recently:
Use of uninitialized value $_ in pattern match (m//) at t/api/author.t line 78.
# Failed test 'This Dave has a Gmail account'
# at t/api/author.t line 78.
(error repeats a couple of times).
t/scroll.t failing in 2.008000
$ cd MetaCPAN-Client-2.008000
$ make test
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/api/*.t
#
# Versions for all modules listed in MYMETA.json (including optional ones):
#
# === Configure Requires ===
#
# Module Want Have
# ------------------- ------ ----
# ExtUtils::MakeMaker 7.1101 7.04
#
# === Build Requires ===
#
# Module Want Have
# ------------------- ---- ----
# ExtUtils::MakeMaker any 7.04
#
# === Test Requires ===
#
# Module Want Have
# ------------------- ---- --------
# ExtUtils::MakeMaker any 7.04
# File::Spec any 3.47
# Test::Fatal any 0.014
# Test::More any 1.302078
# Test::Requires any 0.10
# base any 2.22
# blib 1.01 1.06
#
# === Test Recommends ===
#
# Module Want Have
# ---------------------- -------- --------
# CPAN::Meta 2.120900 2.150010
# HTTP::Tiny::Mech any missing
# LWP::Protocol::https any 6.06
# WWW::Mechanize::Cached 1.48 missing
#
# === Runtime Requires ===
#
# Module Want Have
# --------------- ----- --------
# Carp any 1.38
# HTTP::Tiny 0.056 0.070
# IO::Socket::SSL 1.42 2.047
# JSON::MaybeXS any 1.003009
# JSON::PP any 2.27400
# Moo any 2.003001
# Moo::Role any 2.003001
# Net::SSLeay 1.49 1.80
# Ref::Util any 0.113
# Safe::Isa any 1.000006
# Type::Tiny any 1.000006
# URI::Escape any 3.31
# strict any 1.08
# warnings any 1.23
#
#
# *** WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING ***
#
# The following REQUIRED prerequisites were not satisfied:
#
# ExtUtils::MakeMaker version '7.04' is not in required range '7.1101'
#
t/00-report-prereqs.t ... ok
t/api/_get.t ............ ok
t/api/_get_or_search.t .. ok
t/api/_search.t ......... ok
t/api/author.t .......... ok
t/api/distribution.t .... ok
t/api/download_url.t .... ok
t/api/favorite.t ........ ok
t/api/file.t ............ ok
t/api/module.t .......... ok
t/api/pod.t ............. ok
t/api/rating.t .......... ok
t/api/release.t ......... ok
t/entity.t .............. ok
t/request.t ............. ok
t/resultset.t ........... ok
# Failed test 'can read all matching docs'
# at t/scroll.t line 36.
# got: '52'
# expected: '51'
# Looks like you failed 1 test of 6.
t/scroll.t ..............
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/6 subtests
t/ua_trap.t ............. skipped: Test requires module 'WWW::Mechanize::Cached' but it's not found
Test Summary Report
-------------------
t/scroll.t (Wstat: 256 Tests: 6 Failed: 1)
Failed test: 6
Non-zero exit status: 1
Files=18, Tests=174, 18 wallclock secs ( 0.04 usr 0.02 sys + 1.93 cusr 0.17 csys = 2.16 CPU)
Result: FAIL
Failed 1/18 test programs. 1/174 subtests failed.
Makefile:939: recipe for target 'test_dynamic' failed
make: *** [test_dynamic] Error 255
This is happening consistently for me.
methods implemented in terms of _get_or_search constraint author, module, distribution, and release names too much
_get_or_search, which is used in the implementation of several other methods, will only delegate to ->_get for arguments that are both defined and match the regular expression /^[A-Za-z-:]+$/.
There are valid module, distribution, and release names that aren't matched by the above regex. There's also invalid module names that are matched by it.
It would seem appropriate to me to always dispatch to ->_get if the argument is both not a reference and defined at the same time.
Caching doesn't actually work
With this new client, everything is based on running scrolls, as opposed to grabbing a certain sized query. Caching is based on URL+POST information. However, with ElasticSearch scrolls, the exact same URL is used to acquire different results. Thus, trying to cache the page results in an infinite loop of the same results. (I know; I tried. This is how I figured out how scrolling worked...)
Caching is critical to Dist::Zilla::Role::MetaCPANInterfacer, since the queries would be ran on just about every 'dzil' command.
I have a potential solution, but I wanted your opinion of it first. It uses the fact that MetaCPAN::Client::ResultSet->scroller needs a Search::Elasticsearch::Scroll object. I created something called Search::Elasticsearch::FakeScroll, which acts like a scroll, but doesn't actually use the scroll function. Instead, it would be called with a reduced size parameter, and puts the all of the results in the same kind of result set that Scroll would use.
In fact, there isn't much to the code:
package Search::Elasticsearch::FakeScroll;
use Moo;
use namespace::clean;
extends 'Search::Elasticsearch::Scroll';
#===================================
sub BUILDARGS {
#===================================
my ( $class, $params ) = parse_params(@_);
delete $params->{scroll};
delete $params->{scroll_in_body};
my $es = delete $params->{es};
my $results = $es->search($params);
return {
es => $es,
aggregations => $results->{aggregations},
facets => $results->{facets},
suggest => $results->{suggest},
took => $results->{took},
total_took => $results->{took},
total => $results->{hits}{total},
max_score => $results->{hits}{max_score},
_buffer => $results->{hits}{hits},
is_finished => 1,
};
}
1;
Then MetaCPAN::Client would have some sort of switch to enable this mode (passed to MetaCPAN::Client::Request), so that caching doesn't loop.
RT or Issues
the link in MetaCPAN -> https://rt.cpan.org/Public/Dist/Display.html?Name=MetaCPAN-Client
not here :(
Example on getting # favourites for all dists
I can't work out how to query favourites. Can you add an example showing how you can iterate over all dists that have one or more favourites please, getting back just the dist name?
Cheers,
Neil
Test::Requires required modules cause a BAIL_OUT
Apparently when the modules aren't available, it just causes a BAIL_OUT. Example:
t/ua_trap.t ............... Bailout called. Further testing stopped: Test requires module 'WWW::Mechanize::Cached' but it's not found
FAILED--Further testing stopped: Test requires module 'WWW::Mechanize::Cached' but it's not found
It doesn't skip. :(
autocomplete is not as good as the real thing
When I do: my $ac = $mcpan->autocomplete('mojoli access'); I get 10 results, 8 of which are from the Mojolicious distribution, and none of them is "Mojolicious::Plugin::AccessLog".
On the other hand, when I do: GET https://fastapi.metacpan.org/v1/search/autocomplete?q=mojoli+access, I get very varied and interesting results, the first many of which contain both Mojolicious and Access in their names.
# the following is the same as ->author('MICKEY')
not true
my $mcpan = MetaCPAN::Client->new();
try{
my $author = $mcpan->author('BOWTIE');
p $author; # MetaCPAN::Client::Author
};
try{
my $author = $mcpan->author( { pauseid => 'BOWTIE'} );
p $author; # MetaCPAN::Client::ResultSet
};produces the following
kevin@billy:~/scratch/svn2git/script$ perl authors-02.pl
Printing in line 23 of authors-02.pl:
MetaCPAN::Client::Author {
Parents Moo::Object
public methods (14) : blog, city, country, dir, email, gravatar_url, name, new, pauseid, profile, releases, updated, user, website
private methods (1) : _known_fields
internals: {
data {
blog [
[0] {
feed "",
url "http://kevindawson.github.io/bowtie/"
}
],
country "GB",
dir "id/B/BO/BOWTIE",
email [
[0] "bowtie@cpan.org"
],
gravatar_url "https://secure.gravatar.com/avatar/9f27e15605c9a2d35a4f307a0bbf5dbf?s=130&d=identicon",
name "Kevin Dawson",
pauseid "BOWTIE",
profile [
[0] {
id "kevindawson",
name "coderwall"
},
[1] {
id "kevindawson",
name "github"
},
[2] {
id "BOWTIE",
name "github-meets-cpan"
},
[3] {
id "bowtie",
name "ohloh"
}
],
updated "2013-07-31T10:24:57",
user "m-AM5x8ZSAmcmUZCGkQVTA",
website []
}
}
}
Printing in line 28 of authors-02.pl:
MetaCPAN::Client::ResultSet {
Parents Moo::Object
public methods (9) : BUILDARGS, facets, has_scroller, items, new, next, scroller, total, type
private methods (0)
internals: {
scroller Search::Elasticsearch::Scroll,
total 1,
type "author"
}
}they are not the same to me as I can not extract email from both or from the following
my $author = $mcpan->author({name => 'Kevin Dawson',});All objects should provide a metacpan_url method
It'd be incredibly useful if we could get URLs to a release, distribution, module, etc. on the metacpan.org website with a simple method. As it is, you have to know the exact URL scheme the website uses and construct it from object attributes.
examples/all_favorite_dists.pl~ in distribution
I can't even find examples/all_favorite_dists.pl in the repo.
OTOH I see all_favorite_counts_per_dist.pl in the repo but not on CPAN.
replace current query building with query match_all and filters?
from what I read about ES (not an expert) using:
query => { match_all => {} }
+filters
is more efficient than building our search inside 'query' as we currently do.
also the way we did it doesn't allow queries like 'get all authors'.
any thoughts about such change?
examples/es_filter.pl is broken
olaf$ perl examples/es_filter.pl
[Request] ** [https://fastapi.metacpan.org:443/v1]-[500] {"message":"[Request] ** [http://127.0.0.1:9200]-[400] Bad Request, called from sub Search::Elasticsearch::Role::Client::Direct::ANON at /home/metacpan/metacpan-api/lib/MetaCPAN/Server/Controller/Scroll.pm line 43. With vars: {'request' => {'method' => 'GET','qs' => {'scroll' => '5m'},'path' => '/_search/scroll/c2NhbjszOzEzNjA1NTA2Ol9UTkpwOElSVGF5X0lYeTJUTzhDT2c7MjIxNDE5MTU6bnh2MXI3MUNTZU9xQTlTcWxHVWtEUTsxMzYwNTUwNzpfVE5KcDhJUlRheV9JWHkyVE84Q09nOzE7dG90YWxfaGl0czo5Mzs%3D','serialize' => 'std','body' => undef,'ignore' => []},'status_code' => 400,'body' => {'took' => 18,'timed_out' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),'_shards' => {'successful' => 0,'failed' => 3,'total' => 3,'failures' => [{'index' => undef,'reason' => {'reason' => 'field [tests] isn\'t a leaf field','type' => 'illegal_argument_exception'},'shard' => -1}]},'_scroll_id' => 'c2NhbjswOzE7dG90YWxfaGl0czo5Mzs=','hits' => {'total' => 93,'max_score' => '0','hits' => []}}}\n"}, called from sub Search::Elasticsearch::Scroll::next at /Users/olaf/.plenv/versions/5.24.0/lib/perl5/site_perl/5.24.0/MetaCPAN/Client/ResultSet.pm line 65. With vars: {'status_code' => 500,'request' => {'body' => 'c2NhbjszOzEzNjA1NTA2Ol9UTkpwOElSVGF5X0lYeTJUTzhDT2c7MjIxNDE5MTU6bnh2MXI3MUNTZU9xQTlTcWxHVWtEUTsxMzYwNTUwNzpfVE5KcDhJUlRheV9JWHkyVE84Q09nOzE7dG90YWxfaGl0czo5Mzs=','ignore' => [],'path' => '/_search/scroll','method' => 'POST','mime_type' => 'application/json','qs' => {'scroll' => '5m'},'serialize' => 'std'}}
MetaCPAN::Client does not cache results with WWW::Mechanize::Cached
Hi! Not sure if this should address to MetaCPAN::Client directly , but it seems caching does not happen from time to time. Sometimes it works fine and sometimes client try to download stuff from internet over and over again.
My set up:
my $meta_client = MetaCPAN::Client->new(
ua => HTTP::Tiny::Mech->new(
mechua => WWW::Mechanize::Cached->new(
cache => CHI->new(
driver => 'File',
root_dir => "$ENV{HOME}/.swatman/metacpan/cache/",
),
),
),
);
Switch to JSON::MaybeXS
Any objections to a pull request to switch from JSON to JSON::MaybeXS ?
How to specify time for scroller?
I create a MetaCPAN::Client object and then run lots of queries through it (current release info for all modules I use locally). Of late, I've been getting errors when doing this:
failed to delete scroller at /usr/share/perl5/vendor_perl/MetaCPAN/Client/Scroll.pm line 160.
failed to create a scrolled search at /usr/share/perl5/vendor_perl/MetaCPAN/Client/Request.pm line 129.
I suspect that this is because the scroller has lived longer than the default 5 minutes in MetaCPAN::Client::Scroll. I see that MetaCPAN::Client::Scroll has a 'time' attribute but I don't see any way to set that via MetaCPAN::Client->new / MetaCPAN::Client::Request. Am I missing something?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.