Giter Club home page Giter Club logo

package-quality's Introduction

Package quality Build Status

Package quality

package-quality

Measurements of quality for packages, initially for npm.

Add Your Badge

To show the quality of your npm package, just add one of these images to your GitHub README:

[![Package quality](https://packagequality.com/shield/yourpackage.svg)](https://packagequality.com/#?package=yourpackage)
[![Package quality](https://packagequality.com/badge/yourpackage.png)](https://packagequality.com/#?package=yourpackage)

Or, in HTML markup:

<a href="https://packagequality.com/#?package=yourpackage"><img src="https://packagequality.com/badge/yourpackage.png"/></a>
<a href="https://packagequality.com/#?package=yourpackage"><img src="https://packagequality.com/shield/yourpackage.svg"/></a>

replacing yourpackage with (surprise!) the name of your package. You can choose between badge or mini-badge; the latter is courtesy of shields.io.

Measuring Quality

Any objective measurements of quality are going to be flawed one way or another. package-quality only attempts to give some indications about quality, not be an absolute rating on which to bet your farm. If you don't agree with our ratings, please help us improve them!

Algorithm

The following factors are considered when calculating the quality of a package:

  • Versions quality (v): the more versions a package has, the more quality it is. We calculate v as v=1-1/total_number_of_versions
  • Downloads quality (d): the more downloads a package has, the more quality is it. We calculate d as d=1-1/number_of_downloads_last_year
  • Repo quality (r): if the package has no repo, the value of r is zero. Right now (there is an issue open to fix this), if the package does not have repo on Github, it's given r=0. For those packages with a repo on Github, r is calculated considering these three factors:
    • Total factor (rt), calculated as rt=1-1/total_number_of_issues
    • Open factor (ro): we consider "healthy" to have a 20% of open issues in your repo (20% of the total number of issues). Those packages with 20% or less open issues will have ro=1. For those with more than 20%, ro=1.2-open_issues/total_number_of_issues
    • Long open factor (rlo): we consider long open issues those who have been open more that 1 year and are still open today. This factor is calculated as rlo=1-long_open_issues/total_number_of_issues. If a package has no open issues, then rlo=1.

The repo quality r is calculated as the average of rt, ro and rlo: r=(rt+ro+rlo)/3.

The overall quality of a package is q=v*d*r.

How to Understand Star Ratings

By xkcd

Just joking. New packages will always appear with low stars, until they get enough momentum. Also, packages that "just work" and get no issues will be underrated by our system.

Other Package Managers

Right now the quality is computed only for npm packages. But it is trivial to extend the ratings to other package managers, since there are only external measurements from GitHub and npm: we don't even look at the code.

If you want to extend package-quality to a new package manager, you just need:

  • a complete list of packages,
  • and some way to get the downloads per year.

Let us know and we will set up a new subdomain.

Help Wanted

The following areas are a work in progress:

  • Improve the existing ratings.
  • Take other parameters into account, such as a working or broken Travis-CI build.
  • Extend to other package managers (see above).
  • Just let us know what you don't like about our project.

Just send mail or, even better, a pull request.

License (The MIT License)

Copyright (c) 2014-2015 Alex Fernández [email protected], Diego Lafuente, Sergio García Mondaray [email protected] and contributors.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package-quality's People

Contributors

alexfernandez avatar diegotui avatar grant avatar matiassingers avatar palcarazm avatar sgmonda avatar thecotne avatar tufosa avatar wandmalfarbe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

package-quality's Issues

Use github API response headers to optimize the update of the db

That information is now available in the new X-RateLimit-Reset response header.
$ curl -I https://api.github.com/orgs/octokit

HTTP/1.1 200 OK
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1372700873
...
The X-RateLimit-Reset header provides a Unix UTC timestamp, letting you know the exact time that your fresh new rate limit kicks in.

The reset timestamp is also available as part of the /rate_limit resource.
$ curl https://api.github.com/rate_limit

{
"rate": {
"limit": 60,
"remaining": 42,
"reset": 1372700873
}
}

be more brutal

Everything I looked at seems to be rated pretty well. Suggest perhaps making it harder to get > 4 stars.

Also:
Not clear exactly how metrics are measured. Suggest some brief description as hover tooltip.
Not clear what an author can do to improve quality. "Is this your package? Here's what you can do to improve your score."

Refactoring of estimate.js to make update.js more efficient

Leave pending tasks when several pages of issues are found. Write a getIssuesForRepo(name, from, to, callback) for this purpose. Proccess each chuck, and then go for the peding issues. This way we get better control of the remaining API calls to github.

cache requests

In the new "unify" version, the registry (http://registry.npmjs.org/) is called twice, once for checking that the package exists, and another one to estimate the versions factor. Caching this request for 20 seconds will save us a call.

The rating paradigm

Hi all.

I love this idea, how clean and minimalist it is.

I do worry about how new packages are treated by the calculations. In the very beginning of a package's life it can't (or shouldn't) be treated as a low quality package.

For the sake of clarity, let's assume a package has just been published. No matter how good or well coded it is, it will be a zero-star package assuming most of its considered stats are in zero:

DOWNLOADS = 0;
VERSIONS = 1;
TOTAL_ISSUES = 0; //maybe the package is SO good it doesn't have any issues (just maybe)
OPEN_ISSUES = 0; //idem above
LONG_ISSUES = 0; //because above

v = 1 - 1 / VERSIONS;
// 1 / 1 = 1; 1 - 1 = 0; zero rating so far
d = 1 - 1 / DOWNLOADS;
// 1 / 0 = error; 1 - error = ?; zero rating remains

rt = 1 - 1 / TOTAL_ISSUES;
// 1 / 0 = error; 1 - error = ?; zero rating remains
ro = 1 - 1 / OPEN_ISSUES;
// 1 / 0 = error; 1 - error = ?; zero rating remains
rlo = 1 - 1 / LONG_ISSUES;
// 1 / 0 = error; 1 - error = ?; zero rating remains
r = (rt + ro + rlo) / 3; //zero again

So, how does a package relates to its quality by this calculations? Well, it doesn't.

Quality shouldn't be confused with popularity or maintained package. This stats are only checking for "how much" (a quantitative measure) and not for "how good" (qualitative).

I think a harder work has to be applied to the ratings and how are they calculated. Things like downloads shouldn't be taken on quality measures unless we can find a relation with other packages depending on the one being measured. Documentation is, by far, one of the most important things in a package quality (besides code itself) and is not even mentioned here.

Let's face it, even if someone wrote the perfect package, it would receive zero star rating because it doesn't have any downloads, or because it doesn't have any issues. A package could be perfect (not often) and not need any issues, yet it would still receive a zero star rating.

I'd love to see this tool go further into a deep analysis of the package and post a, relatively, improved version of the ratings.

Code should be hidden on error

image

My package is registered on NPM but it does not find it (doesn't matter) but if there's an error (e.g. not found), the code should not be displayed with "undefined" & co. 😃 thanks!

Use arithmetic average from * to *****.

The results for geometric and harmonic averages are not acceptable: they violate the universal values for star ratings since most packages have zero (and not 1~3). We should use an arithmetic average, with the same weights as now, and start with one star.

Show badges.

It would be great to have badges that show package quality for a repo, using a simple URL scheme. They can then be added to package READMEs.

Store everything in /npm/

Since we have the vocation of being a universal package quality indicator, we should prefix all URLs with /npm/ so that other package managers can reside in /maven/, /pypi/ and so on.

inadequate rating

Just out of curiosity I tried with one of my packages, pg-promise

It dropped the rating a lot based on the number of issues that were open over the package's lifespan.

A popular, high-quality package will naturally receive much attention, and much of that attention is in questions and ideas people bring to the table. In my project all questions have been answered and all ideas have been met, no issues open at all, and yet the rating is dropped because there were questions and ideas published on GitHub? This is nonsense. This absolutely does not reflect the package's quality.

Add preinstalation instructions

Users may need to install "libgif-dev" at operating system level before installing npm dependencies.

This can achieved in debian/ubuntu doing:
sudo apt-get install libgif-dev

And under OSX:
brew install libgif-dev

I will try to update the docs too to include that after the SVG patch.

Package not found

I would like to look at the "quality" of my package. Unfortunately packagequality.com says it could not find it.

Simple badges

Awesome idea!

I would suggest a simpler badge design in addition to the current one. I know a lot of people use the shields.io style badges in their packages, and putting the current package-quality badge next to them doesn't look very well.

Would also be awesome with the same styles as shields.io:


linting code?

I know that this project is code agnostic at the moment, but a metric like this actually indicates something about the quality of the code.

The presence of .jslint.conf, .jshintrc or .eslintrc indicates which linter to use, and the results of the linting could inform the rating.

Add version info

User Statement

As a Package Quality (PQ) user, I would like to see the version of the package I searched for, so that I know what version on npm PQ is looking at.

Details

I know the service currently just looking at the current version on npm but the reassurance is a nice-to-have. I think it could fit well in two places:

  • Underneath the star rating, to the right and vertically aligned with "Package from "
  • Or replacing the text "Package from" entirley.

Implementation Suggestions

Some formatting options:

  • "Version X.X.X from "
  • "vX.X.X from"
  • "X.X.X from"

Great app! Keep up the good work!

Show all quality factors

When a package has a GitHub repo but no issues, it shows with quality factors null. It should show which quality factor has actually lowered its score to zero, for improved transparency.

Sample: microprofiler.

{
"_id":"54cad1fd465dcde6761d2b96",
"name":"microprofiler",
"source":"npm",
"repoQuality":0,
"repoFactors":null,
"quality":0,
"created":"2015-02-06T04:27:50+00:00",
"lastUpdated":"2015-02-06T04:27:50+00:00",
"nextUpdate":"2015-03-06T04:27:50+00:00",
"timesUpdated":0
}

Packages Not Found

I am getting package not found errors for npm packages that are published and are on Github. Try two of mine: stringformatter and propertyinterceptor.

I'm new to publishing using npm, so perhaps the quality of my publishing is so bad, you can't read them!

Open API available?

Hello Alex,
I created an application http://bestof.js.org/ to help developers find the best components of the web platform (front and back end).
Basically it is a curated list of 500 projects on Github about JavaScript in the browser and in node.js, html, css,..
Every day the application checks variations of stars on Github to detect trends.
I would like to display more relevant information about the quality of the projects.
So I wonder if there is a kind of open API to access data generated by http://packagequality.com/ so that I could display the metrics you generate ?
Of course, I would add links to your application and add credits to make obvious the source of the information.
Thank you in advance for your answer and feel free to give me any feedback about bestof.js.org, especially if it is bad feedback ;)

badge on GH doesn't match to the SVG

This issue seems to be similar to #58

The quality on website is 4 stars, check it here
But the badege appears 2.5 stars on GITHUB
Package Quality
However, the svg image seems correct on an independent browser label

I have cleared my browser cache and cookies, but the problem still exists. I wonder why does it happen? 🤣

Here is my repo vis-ui.

Some repositories have quality higher than 1

For some obscure reason their repoTotalQuality is "-Infinity" nad for other ever more obscure its harmonic mean is higher than one:

db.packages.findOne({name:"gulp-filter"})
{
"_id" : ObjectId("54e25979f22796cf67c79252"),
"name" : "gulp-filter",
"source" : "npm",
"created" : "2015-02-16T20:56:10+00:00",
"lastUpdated" : "2015-02-17T15:59:01+00:00",
"nextUpdate" : "2015-02-17T15:59:01+00:00",
"timesUpdated" : 1,
"repoTotalIssues" : [
-Infinity,
1
],
"repoOpenIssues" : [
1,
1
],
"repoLongOpenIssues" : [
1,
1
],
"versions" : [
0.9375,
1
],
"downloads" : [
0.9999973556026845,
1
],
"quality" : 1.2295073972177069
}
db.packages.count({quality:{$gt:1.0}})
73
db.packages.findOne({quality:{$gt:1.0}})
{
"_id" : ObjectId("54e23064f22796cf67c75330"),
"name" : "fedtools",
"source" : "npm",
"created" : "2015-02-16T18:00:52+00:00",
"lastUpdated" : "2015-02-17T13:05:47+00:00",
"nextUpdate" : "2015-02-17T13:05:47+00:00",
"timesUpdated" : 1,
"repoTotalIssues" : [
-Infinity,
1
],
"repoOpenIssues" : [
1,
1
],
"repoLongOpenIssues" : [
1,
1
],
"versions" : [
0.995049504950495,
1
],
"downloads" : [
0.9999830203416308,
1
],
"quality" : 1.2484419119330468
}

Badge shows weird characters

Interesting project 👍 . Having something like this would be hanging when deciding if a dependency is worth being added or removed.

Well, let's not start sucking each other's dicks quite yet. Badges shows a weird text.

badge

Tested with Firefox 41 and Chrome 45 under Ubuntu 14.04.

Looking the code, it seems like you try to place the ASCII (filled or unfilled) stars in SVG but it's not well encoded at server side or decoded at client one. The "one half" character have no issues.

I'll try to help with it if I've time and nobody solves it before.

Show top 100.

We should have a page that displays top 100 packages.

Unite no_mongo and mongo

Currently there are two versions of the site:

  • mongo uses MongoDB to read package information,
  • no_mongo queries GitHub and other datasources concurrently and gets an answer, so the package is checked online.

It would be great to have a single, unified version that uses cached information, and if it's outdated checks online.

In addition to the date, package information might contain a version field checked by the server: if earlier than a given version it is considered outdated and checked online.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.