Merge branch 'master' into mi/es2

metacpan · Apr 16, 2016 · d2c3010 · d2c3010
2 parents acb20b5 + 480d931
commit d2c3010
Show file tree

Hide file tree

Showing 82 changed files with 1,884 additions and 454 deletions.
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
@@ -0,0 +1,143 @@
+# How to contribute
+
+We are always after more contributors and suggestions.
+
+### How can I help?
+
+The following issues are tagged as [Volunteer needed](https://github.com/CPAN-API/cpan-api/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+label%3A%22Volunteer+needed%22+no%3Aassignee+)
+
+## Suggestions or issues with MetaCPAN...
+
+#### Does it relate to our API (backend)... ?
+
+ 1. Please check the [previously reported API issues](https://github.com/CPAN-API/cpan-api/issues)
+ 2. Please check the [Wishlist](https://github.com/CPAN-API/cpan-api/wiki/Wishlist).  If you can't find it already there:
+    * If it's a wishlist idea, please edit the [wiki](https://github.com/CPAN-API/cpan-api/wiki/Wishlist) (add a 'wishlist_MYIDEA' page if you need more space!)
+    * If it's an actual bug [create a new issue](https://github.com/CPAN-API/cpan-api/issues/new)
+
+#### If you are not sure, or it is related to https://metacpan.org/ front end:
+
+ 1. Please check the [previously reported Web issues](https://github.com/CPAN-API/metacpan-web/issues)
+ 2. Please check the [Wishlist](https://github.com/CPAN-API/cpan-api/wiki/Wishlist). If you can't find it already there:
+    * If it's a wishlist idea, please edit the [wiki](https://github.com/CPAN-API/cpan-api/wiki/Wishlist) (add a 'wishlist_MYIDEA' page if you need more space!)
+    * If it's an actual bug [create a new issue](https://github.com/CPAN-API/metacpan-web/issues/new)
+
+## Contributing code
+
+Come talk to us on IRC (see below), or send a pull request and we'll respond
+there.  If you implement a new feature, please add a note about it to the
+News.md file at the top level of metacpan-web so that it will appear in our
+news feed.
+
+If you aren't using the VM, remember to enable the pre-commit hook before you start working.
+
+    sh git/setup.sh
+
+These links will get you going quickly:
+
+  * [Using our developer VM](https://github.com/CPAN-API/metacpan-developer) to get you going in minutes (depending on bandwidth)
+  * [Front end bug list](https://github.com/CPAN-API/metacpan-web/issues)
+  * [API (back end) bug list](https://github.com/CPAN-API/cpan-api/issues)
+  * [Wishlist](https://github.com/CPAN-API/cpan-api/wiki/Wishlist) - things that probably need doing
+
+# Git workflow
+
+We try to keep a clean git history, so if it all possible, please rebase to get
+the latest changes from master _before_ submitting a pull request.  You'll only
+need to do the first command (git remote add) once in your local checkout.
+
+    git remote add upstream https://github.com/CPAN-API/metacpan-web.git
+    git pull --rebase upstream master
+
+If you are comfortable rebasing, it is also helpful to squash or delete commits
+which are no longer relevant to your branch before submitting your work.
+
+    git rebase -i master
+
+If you are not comfortable with rebasing, but want to use it, check out the steps
+from [here](https://help.github.com/articles/using-git-rebase/).
+
+# Coding conventions
+
+Please try to follow the conventions already been used in the code base.  This
+will generally be the right thing to do.  Our standards are improving, so even
+if you do follow what you see, we may ask you to make some changes, but that is
+a good thing.  We are trying to keep things tidy.
+
+If you are using the [developer VM](https://github.com/CPAN-API/metacpan-developer) you can run:
+
+```sh
+/home/vagrant/carton/metacpan-web/bin/tidyall
+```
+
+## Perl Best Practices
+
+In general, the concepts discussed in "Perl Best Practices" are a good starting
+point.  Use autodie where possible and MetaCPAN::Web::Types when creating new
+Moose attributes.  Many of the other standards will be enforced by Perl::Critic.
+
+## Clear > Concise
+
+Take pains to use variable names which are easy to understand and to write
+readable code.  We value readable code over concise code.  Use singular nouns
+for class names.  Use verbs for method names.
+
+## Try::Tiny > eval { ... }
+
+You will see many eval statements in the code.  We would like to standardize on
+Try::Tiny, so feel free to swap out any eval with a Try::Tiny and use Try::Tiny
+in all new code.
+
+## Prefer single quotes
+
+Always use single quotes in cases where there is no variable interpolation.  If
+there is a single quote in the quoted item, use curly quotes.
+
+q{Isn't this a lovely day};
+
+## Include a test (or more!)
+
+Any time when a pull request includes a test, it makes it easier for us to
+review and accept, so please do test your changes whenever possible.  If your
+pull request includes visual changes, please include a before and after screen
+shot, so that we can better understand the problem you're trying to solve.
+
+## Dependencies
+
+Introducing new dependencies is fine, if they solve a specific problem which
+current dependencies cannot address.  If we prefer a different module to be used,
+we'll let you know.
+
+## It's OK to be controversial
+
+If a pull request contains any controversial changes, we'll likely wait for some
+feedback from several developers before a merge.  If you think your changes may
+be controversial, feel free to discuss them in a Github issue before starting to
+write any code.
+
+## Travis is your friend
+
+We use Travis to test all code changes.  After submitting your pull request,
+remember to check back to see whether Travis has come back with any test
+failures.  We do get some false negatives.  If your pull request failed for
+reasons unrelated to your changes, we may still be able to merge your work.
+
+# Additional Resources
+
+  * [\#metacpan](http://widget01.mibbit.com/?autoConnect=true&server=irc.perl.org&channel=%23metacpan&nick=) IRC channel on irc.perl.org
+
+# Current Policies
+
+### What is indexed?
+
+ * Perl distributions which contain Perl packages.
+
+### When are issues closed?
+
+We want to keep the issue list manageable, so we can focus on what actually
+needs fixing.  If you feel an issue needs opening again, please add a comment
+explaining why it needs re-opening and we'll look at it again.
+
+ * Issues will be closed and moved to [Wishlist](https://github.com/CPAN-API/cpan-api/wiki/Wishlist) if they are not actual bugs
+ * Issues we think we have addressed will be closed
+ * Issues we are not going to take any further action on without more information will be closed
diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md
@@ -0,0 +1,18 @@
+# Important, please read:
+
+MetaCPAN's core developers need to focus on fixing bugs and improving the
+existing core system.
+
+For this reason, if you have a feature which you would like to see added (there
+are loads we would love to have), please only open an issue _IF_ you are
+prepared to do the work to implement it.  To be clear, we'd love to have a
+bunch of really cool, new, features, but it's more important for us to focus on
+keeping MetaCPAN humming along.
+
+If you're not motivated or otherwise able to send a pull request for your cool,
+new feature, please add it to our wishlist:
+https://github.com/CPAN-API/cpan-api/wiki/Wishlist and someone may get to it
+one day.  Maybe that person will be you!
+
+For more details on issues and contributing please see CONTRIBUTING.md (linked
+above).
diff --git a/.perlcriticrc b/.perlcriticrc
@@ -15,18 +15,21 @@ verbose = 11
 [-Variables::ProhibitPunctuationVars]
 
 [CodeLayout::RequireTrailingCommas]
-severity = 5
+severity = 4
 
 [TestingAndDebugging::RequireUseStrict]
-equivalent_modules = Test::Routine
+equivalent_modules = Test::Routine Mojo::Base
+
+[TestingAndDebugging::RequireUseWarnings]
+equivalent_modules = Test::Routine Mojo::Base
 
 [ValuesAndExpressions::ProhibitEmptyQuotes]
-severity = 5
+severity = 4
 
 [ValuesAndExpressions::ProhibitInterpolationOfLiterals]
 allow_if_string_contains_single_quote = 1
 allow = qq{} qq[]
-severity = 5
+severity = 4
 
 [ValuesAndExpressions::ProhibitNoisyQuotes]
-severity = 5
+severity = 4
diff --git a/.travis.yml b/.travis.yml
@@ -1,12 +1,13 @@
 language: perl
 perl:
+  - "5.22"
   - "5.20"
   - "5.18"
-  - "5.16"
 
 matrix:
   allow_failures:
     - perl: "5.20"
+    - perl: "5.22"
 
 notifications:
   email:
@@ -31,14 +32,17 @@ before_install:
   # https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html
   - wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
   - echo "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list
-  - sudo apt-get update && sudo apt-get install elasticsearch
+  - sudo apt-get update
+
+  # Run update to make libgmp-dev findable (Required by Net::OpenID::Consumer)
+  # postgresql-server-dev-all is required by DBD::Pg
+  - sudo apt-get install elasticsearch libgmp-dev postgresql-server-dev-all
 
   - sudo service elasticsearch restart
   - pwd
 
   - cpanm -n Devel::Cover::Report::Coveralls
   - cpanm -n Carton
-  - sudo apt-get install libgmp-dev
 
   # Carton refuses to update Safe.pm to the version specified in the cpanfile and the
   # version that's core in 5.16 is too old (it fails to work with Devel::Cover).

diff --git a/bin/cpantesters_mini_db_for_testing b/bin/cpantesters_mini_db_for_testing
@@ -0,0 +1,63 @@
+#!/bin/bash
+
+cd `dirname "$0"`
+cd ..
+
+url=http://devel.cpantesters.org/release/release.db.bz2
+in=t/var/tmp/cpantesters-release.db
+out=t/var/cpantesters-release-fake.db
+table=release
+
+download_original () {
+    test -s "$in" || \
+        wget -O "$in.bz2" "$url"
+    test -f "$in.bz2" && \
+        bunzip2 "$in.bz2"
+
+    rm -f "$out" "$out.bz2"
+}
+
+finish () {
+    # Compress the db like cpantesters does.
+    bzip2 "$out"
+}
+
+sqlout () { sqlite3 "$out"; }
+sql () {
+    sqlite3 "$in" | sqlout
+}
+
+dist_version () {
+    local dist="$1" version="$2"
+cat <<SQL | sql
+.mode insert $table
+select * from release where dist = '$dist' and version = '$version';
+SQL
+}
+
+fake_dist () {
+    echo "INSERT INTO $table VALUES('$1', '$2', $3, $4, $5, $6);" | sqlout
+}
+
+populate_db () {
+    # Create tables.
+    echo .schema | sql
+
+    # Get test cases from real data.
+    dist_version 'Devel-GoFaster' '0.000'
+    dist_version 'P' '1.0.20'
+    dist_version 'IPsonar' '0.29'
+    dist_version 'weblint' '++-1.15'
+    dist_version 'WWW-Tumblr' ''
+
+    # Add records for our fake dists.
+    fake_dist 'Some' '1.00-TRIAL' 4 3 2 1
+}
+
+# main
+
+download_original
+
+populate_db
+
+finish
diff --git a/bin/queue.pl b/bin/queue.pl
@@ -0,0 +1,19 @@
+#!/usr/bin/env perl
+
+use strict;
+use warnings;
+
+=head2 DESCRIPTION
+
+Simple script to start Mojo app.
+
+    carton exec -- morbo bin/queue.pl
+
+=cut
+
+# for morbo
+use lib 'lib';
+
+# Start command line interface for application
+require Mojolicious::Commands;
+Mojolicious::Commands->start_app('MetaCPAN::Queue');
diff --git a/cpanfile b/cpanfile
@@ -82,10 +82,14 @@ requires 'List::Util', '1.43';
 requires 'Log::Contextual';
 requires 'Log::Log4perl';
 requires 'Log::Log4perl::Appender::ScreenColoredLevels';
+requires 'Minion', '>= 5.01';
+requires 'Minion::Backend::SQLite';
+requires 'Module::Load';
 requires 'Module::Metadata', '1.000022';
 requires 'Module::Pluggable';
 requires 'Module::Runtime';
 requires 'Moose', ' >= 2.1403';
+requires 'Mojo::Pg';
 requires 'Moose::Role';
 requires 'Moose::Util';
 requires 'MooseX::Aliases';
@@ -112,7 +116,8 @@ requires 'PAUSE::Permissions';
 requires 'Parse::CPAN::Packages::Fast', '0.09';
 requires 'Parse::CSV', '2.04';
 requires 'Parse::PMFile', '0.29';
-requires 'Path::Class', '0.36';
+requires 'Path::Class', '>= 0.36';
+requires 'Path::Iterator::Rule', '>=1.011';
 requires 'Path::Class::File';
 requires 'PerlIO::gzip';
 requires 'Pithub', '0.01033';

diff --git a/docs/API-docs.md b/docs/API-docs.md
@@ -4,7 +4,7 @@ For an introduction to the MetaCPAN API which requires no previous knowledge of
 
 There is also [a repository of examples](https://github.com/CPAN-API/metacpan-examples) you can play with to get up and running in a hurry.  Rather than editing this wiki page, please send pull requests for the metacpan-examples repository.  If you'd rather edit the wiki, please do, but sending the code pull requests is probably the most helpful way to approach this.
 
-_All of these URLs can be tested using the [MetaCPAN Explorer](http://explorer.metacpan.org)_
+_All of these URLs can be tested using the [MetaCPAN Explorer](https://explorer.metacpan.org)_
 
 To learn more about the ElasticSearch query DSL check out Clinton Gormley's [Terms of Endearment - ES Query DSL Explained] (http://www.slideshare.net/clintongormley/terms-of-endearment-the-elasticsearch-query-dsl-explained) slides.
 
@@ -27,13 +27,13 @@ Part of being polite is letting us know who you are and how to reach you.  This
 Available fields can be found by accessing the corresponding `_mapping` endpoint.
 
 
-* [/author/_mapping](http://api.metacpan.org/v0/author/_mapping) - [explore](http://explorer.metacpan.org/?url=/author/_mapping)
-* [/distribution/_mapping](http://api.metacpan.org/v0/distribution/_mapping) - [explore](http://explorer.metacpan.org/?url=/distribution/_mapping)
-* [/favorite/_mapping](http://api.metacpan.org/v0/favorite/_mapping) - [explore](http://explorer.metacpan.org/?url=/favorite/_mapping)
-* [/file/_mapping](http://api.metacpan.org/v0/file/_mapping) - [explore](http://explorer.metacpan.org/?url=/file/_mapping)
-* [/module/_mapping](http://api.metacpan.org/v0/module/_mapping) - [explore](http://explorer.metacpan.org/?url=/module/_mapping)
-* [/rating/_mapping](http://api.metacpan.org/v0/rating/_mapping) - [explore](http://explorer.metacpan.org/?url=/rating/_mapping)
-* [/release/_mapping](http://api.metacpan.org/v0/release/_mapping) - [explore](http://explorer.metacpan.org/?url=/release/_mapping)
+* [/author/_mapping](http://api.metacpan.org/v0/author/_mapping) - [explore](https://explorer.metacpan.org/?url=/author/_mapping)
+* [/distribution/_mapping](http://api.metacpan.org/v0/distribution/_mapping) - [explore](https://explorer.metacpan.org/?url=/distribution/_mapping)
+* [/favorite/_mapping](http://api.metacpan.org/v0/favorite/_mapping) - [explore](https://explorer.metacpan.org/?url=/favorite/_mapping)
+* [/file/_mapping](http://api.metacpan.org/v0/file/_mapping) - [explore](https://explorer.metacpan.org/?url=/file/_mapping)
+* [/module/_mapping](http://api.metacpan.org/v0/module/_mapping) - [explore](https://explorer.metacpan.org/?url=/module/_mapping)
+* [/rating/_mapping](http://api.metacpan.org/v0/rating/_mapping) - [explore](https://explorer.metacpan.org/?url=/rating/_mapping)
+* [/release/_mapping](http://api.metacpan.org/v0/release/_mapping) - [explore](https://explorer.metacpan.org/?url=/release/_mapping)
 
 
 ## Field documentation
@@ -120,6 +120,11 @@ Returns the POD of the given module. You can change the output format by either
 * text/x-pod
 * text/x-markdown
 
+### `/source/{module}`
+
+Returns the full source of the latest, authorized version of the given
+`module`.
+
 ## GET Searches
 
 Names of latest releases by OALDERS:
@@ -158,6 +163,10 @@ Last 50 dists to get a ++:
 
 http://api.metacpan.org/v0/favorite/_search?size=50&fields=author,user,release,date&sort=date:desc
 
+The Changes file of the Test-Simple distribution:
+
+http://api.metacpan.org/v0/changes/Test-Simple
+
 ## Querying the API with MetaCPAN::Client
 
 Perhaps the easiest way to get started using MetaCPAN is with [MetaCPAN::Client](https://metacpan.org/pod/MetaCPAN::Client).  
@@ -313,7 +322,7 @@ curl -XPOST api.metacpan.org/v0/file/_search -d '{
   "fields":["release"]
 }'
 ```
-[example](http://explorer.metacpan.org/?url=%2Ffile&content=%7B%22query%22%3A%7B%22filtered%22%3A%7B%22query%22%3A%7B%22match_all%22%3A%7B%7D%7D%2C%22filter%22%3A%7B%22and%22%3A%5B%7B%22term%22%3A%7B%22file.module.name%22%3A%22DBI%3A%3AProfile%22%7D%7D%2C%7B%22term%22%3A%7B%22file.module.version%22%3A%222.014123%22%7D%7D%5D%7D%7D%7D%2C%22fields%22%3A%5B%22release%22%5D%7D)
+[example](https://explorer.metacpan.org/?url=%2Ffile&content=%7B%22query%22%3A%7B%22filtered%22%3A%7B%22query%22%3A%7B%22match_all%22%3A%7B%7D%7D%2C%22filter%22%3A%7B%22and%22%3A%5B%7B%22term%22%3A%7B%22file.module.name%22%3A%22DBI%3A%3AProfile%22%7D%7D%2C%7B%22term%22%3A%7B%22file.module.version%22%3A%222.014123%22%7D%7D%5D%7D%7D%7D%2C%22fields%22%3A%5B%22release%22%5D%7D)
 
 ### Find all authors with github-meets-cpan in their profiles
 Because of the dashes in this profile name, we need to use a term.