如果同时同步相同的模块, 会导致qn文件上传冲突.

see here

当开启保护模式, 将启动 admin 权限

所有人都能创建用户

• 在页面上可以配置谁是管理员

只有管理员才能发布二方包

• 非管理员需要通过web页面来申请发布二方包
• 管理员能 unpublish 任意包

所有人都能同步三方包

# 其实这样是从源npm同步包以及包的依赖
cnpm sync [module] 

cdn 可配置

config.nfs

cnpm install connect，如果私有源没有connect模块，则返回官方源，同时开始同步官方源到私有源。
cnpm install [email protected]，如果没有指定版本，同样返回官方源，之后开始同步。

GET http://cnpmjs,org/sync/connect 会发起一次同步。
同步开始之后，url变成http://cnpmjs,org/sync/connect#logid=100

GET http://cnpmjs,org/sync/connect#logid=100将不会重新发起一次同步，将会直接开始获取logid=100的同步日志

这样sync由于请求npmjs.org/connect超时导致未能同步的时候，直接刷新页面就可以重试，如果开始同步，刷新也不会重新触发多一次同步

API

nfs.upload(req, {filename: 'name/name-version.tgz', size: $size}, function (err, result) {
  // result: {url: 'http://xxxx/name/name-version.tgz'}
});

是因为npm包其实都是缓存在七牛CDN上的吗？

由 module 和 tag 表能组合这个结构

{
  "_id": "cnpmjs.org",
  "_rev": "3-c3267f3804042445ff9b708fb0f276cb",
  "name": "cnpmjs.org",
  "description": "cnpmjs.org",
  "dist-tags": {
    "latest": "0.0.0"
  },
  "versions": {
    "0.0.0": {
      "name": "cnpmjs.org",
      "version": "0.0.0",
      "description": "cnpmjs.org",
      "main": "index.js",
      "scripts": {
        "test": "make test-all"
      },
      "dependencies": {},
      "devDependencies": {
        "should": "*",
        "blanket": "*",
        "travis-cov": "*",
        "coveralls": "*",
        "mocha-lcov-reporter": "*",
        "mocha": "*"
      },
      "homepage": "https://github.com/fengmk2/cnpmjs.org",
      "repository": {
      "type": "git",
        "url": "git://github.com/fengmk2/cnpmjs.org.git",
        "web": "https://github.com/fengmk2/cnpmjs.org"
      },
      "bugs": {
        "url": "https://github.com/fengmk2/cnpmjs.org/issues",
        "email": "[email protected]"
      },
      "keywords": [
        "cnpmjs.org"
      ],
      "engines": {
        "node": ">= 0.8.0"
      },
      "author": {
        "name": "fengmk2",
        "email": "[email protected]",
        "url": "http://fengmk2.github.com"
      },
      "license": "MIT",
      "readme": "cnpmjs.org\n=======\n\n[![Build Status](https://secure.travis-ci.org/fengmk2/cnpmjs.org.png)](http://travis-ci.org/fengmk2/cnpmjs.org) [![Coverage Status](https://coveralls.io/repos/fengmk2/cnpmjs.org/badge.png)](https://coveralls.io/r/fengmk2/cnpmjs.org) [![Build Status](https://drone.io/github.com/fengmk2/cnpmjs.org/status.png)](https://drone.io/github.com/fengmk2/cnpmjs.org/latest)\n\n[![NPM](https://nodei.co/npm/cnpmjs.org.png?downloads=true&stars=true)](https://nodei.co/npm/cnpmjs.org/)\n\n![logo](https://raw.github.com/fengmk2/cnpmjs.org/master/logo.png)\n\ncnpmjs.org desc\n\n## Install\n\n```bash\n$ npm install cnpmjs.org\n```\n\n## Usage\n\n```js\nvar cnpmjs.org = require('cnpmjs.org');\n\ncnpmjs.org.foo(function (err) {\n\n});\n```\n\n## License\n\n(The MIT License)\n\nCopyright (c) 2013 fengmk2 <[email protected]>\n\nPermission is hereby granted, free of charge, to any person obtaining\na copy of this software and associated documentation files (the\n'Software'), to deal in the Software without restriction, including\nwithout limitation the rights to use, copy, modify, merge, publish,\ndistribute, sublicense, and/or sell copies of the Software, and to\npermit persons to whom the Software is furnished to do so, subject to\nthe following conditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.\nIN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY\nCLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,\nTORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE\nSOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n",
      "readmeFilename": "README.md",
      "_id": "[email protected]",
      "dist": {
        "shasum": "6da265fb14b151f6b9f4b77967dd90fddfdea1f2",
        "tarball": "http://registry.npmjs.org/cnpmjs.org/-/cnpmjs.org-0.0.0.tgz"
      },
      "_from": ".",
      "_npmVersion": "1.3.14",
      "_npmUser": {
        "name": "fengmk2",
        "email": "[email protected]"
      },
      "maintainers": [
        {
          "name": "fengmk2",
          "email": "[email protected]"
        }
      ],
      "directories": {}
    }
  },
  "readme": "cnpmjs.org\n=======\n\n[![Build Status](https://secure.travis-ci.org/fengmk2/cnpmjs.org.png)](http://travis-ci.org/fengmk2/cnpmjs.org) [![Coverage Status](https://coveralls.io/repos/fengmk2/cnpmjs.org/badge.png)](https://coveralls.io/r/fengmk2/cnpmjs.org) [![Build Status](https://drone.io/github.com/fengmk2/cnpmjs.org/status.png)](https://drone.io/github.com/fengmk2/cnpmjs.org/latest)\n\n[![NPM](https://nodei.co/npm/cnpmjs.org.png?downloads=true&stars=true)](https://nodei.co/npm/cnpmjs.org/)\n\n![logo](https://raw.github.com/fengmk2/cnpmjs.org/master/logo.png)\n\ncnpmjs.org desc\n\n## Install\n\n```bash\n$ npm install cnpmjs.org\n```\n\n## Usage\n\n```js\nvar cnpmjs.org = require('cnpmjs.org');\n\ncnpmjs.org.foo(function (err) {\n\n});\n```\n\n## License\n\n(The MIT License)\n\nCopyright (c) 2013 fengmk2 <[email protected]>\n\nPermission is hereby granted, free of charge, to any person obtaining\na copy of this software and associated documentation files (the\n'Software'), to deal in the Software without restriction, including\nwithout limitation the rights to use, copy, modify, merge, publish,\ndistribute, sublicense, and/or sell copies of the Software, and to\npermit persons to whom the Software is furnished to do so, subject to\nthe following conditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.\nIN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY\nCLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,\nTORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE\nSOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n",
  "maintainers": [
    {
    "name": "fengmk2",
    "email": "[email protected]"
    }
  ],
  "time": {
    "0.0.0": "2013-12-03T09:58:24.368Z"
  },
  "author": {
    "name": "fengmk2",
    "email": "[email protected]",
    "url": "http://fengmk2.github.com"
  },
  "repository": {
    "type": "git",
    "url": "git://github.com/fengmk2/cnpmjs.org.git",
    "web": "https://github.com/fengmk2/cnpmjs.org"
  },
  "_attachments": {
    "cnpmjs.org-0.0.0.tgz": {
      "content_type": "application/octet-stream",
      "revpos": 2,
      "digest": "md5-UCSYBJ+CbiLETV2tAXTryw==",
      "length": 1822,
      "stub": true
    }
  }
}

If admin publish to cnpm, even source npm has the same name of package, should not be sync.

var LIST_BY_AUTHOR_SQL = 'SELECT name, package FROM module WHERE id IN \
                          (SELECT max(id) FROM module WHERE author=?\
                          GROUP BY name )\
                          ORDER BY name';

var LIST_SINCE_SQL = 'SELECT name, package FROM module WHERE id IN \
                     (SELECT max(id) FROM module WHERE gmt_modified > ?\
                      GROUP BY name )\
                     ORDER BY name';

有问题

rt

{"successes":["autofile-pack-macosx-app","accordion","argos","arg-err","angular-kendo","AMD","0","antycs","base64","backbone.geppetto","BigInt","box-view","browserify-on-the-fly","binarysearch","byvoidnodule","c-libnotify","bufferlist","ByteBuffer","cbz-web","catbot-websockets-client","aq","cf-brunch","checkup","components-bootstrap","cls","bag","coops-server-node","concur","cos","cosy-tail","csster","cron","cashew","dancho-npm-test","babel","a","Dateselect","dictation-fix-server","dice-roller","devmailer","difflet-papandreou","docparse-scraper-hes-imacros","DirScanner","DOM","DOMBuilder","ebay-app-meta","emvented","classy","entitiy-system","ep_pad_title","error: forbidden","est","example-manager","EVE","couch-cleaner","exist","cap","fc.lib.timer","faker","comments","file","fis-repos-client","florin-cu-exemplu","frog","get-unique-filename","gi-assert","git-url","google-analytics","glue","first-plugin","gr-event","deferred","grpn-cookies","grunt-contrib-xss","gruntextension-subgrunt","headrest-okay","hellojockchou","helloxxxxxxxx","hp-config-manager","html_json","F","icfp2006-um","idgaf","groundwork","execsync","graph","huk","II","iintranet","ifyouwanttogetthesumoftwonumberswherethosetwonumbersarechosenbyfindingthelargestoftwooutofthreenumbersandsquaringthemwhichismultiplyingthembyitselfthenyoushouldinputthreenumbersintothisfunctionanditwilldothatforyou","EventHub","jade-express","io","javascript.array.extensions","jimmod","joss","jParser","JPath","jq","imageMagick","jQuery","jsconcat","jsdev","jso","json-local","JSON","kevoree-utils","jsonstream","jsonutil","kfc-client","kido-solution","jdataview","jason","leadsec","leto","local-salesforce-dev-example","LiveScript","lolapi","JSONSelect","class","lactate","mdcached","mercury","loggy","mique_math_example2","mmalecki-test-package","dynamodb","mod_config","monoid-llc.github.com","moduleTest","mockery","my_module","my-first-module","mypack_yuki","json2","muffin","md5","ncurl","Mu","nko-website","NetOS","no-skeleton.docpad","node-authfilter","node-dummy","N","node-contentjs","node-gr","node-gerrit","node-invoice","node-picam","node-simon-says","noise-data","NORRIS","now-formatter","npm-fail-hard","npmd-ui-modal","nodevent","Neuro","ø","object-search","packagessssss","Mers","orb","parse-color","pct_enterprise_deployd","percolator","Lingo","pmp-access","printlog","postponejs","pocketbook","purelog-router-redicrect","rc-bin","qixiyumodule","peek","rccli","logstream","react-jquery","replaceapp","o","picomachine","requirejs-google-analytics","riverjs","rfinner","Nonsense","rlo.lib.util","rocksockets","roost-mongo","rx-stats2","Robusta","rollbar-ideame","run.js","s-site","sealion","selections","Scrap","signforms","shenke119","SenseJs","set","simpletest","Simple-Cache","Simplog","smi","socrata-metrics","smtp-protocol-ath","socrata-metrics-api","sourcemap-finder","soda-pop","squirrel","stupid=test","suckle","sugarskull","temp-file-server","SimpleQueue","sardines","test_pkg","test-npm","tempus","thinkpress","timy","tunigo-front-build","tztestjs","undpkg","uber","staticserver","uniform","util-middleware","sLog","Template","upnp-controlpoint","vchange-admin","viprog-watcher","visual-all","wavefront-obj","wod-roll","wintest","workaholic","x-simplemath14","WolfyEventEmitter","xtraicons","zeekay.github.com","thimble","twigjs","front-setting","express-thunder","Vector","index","doctest","iai-oop","ilsjs","dpm2","assert","xerror","iai-is","karma-e2e-dsl","rapgeniusapi","mongoose-access-gen","nodesql","ku4node-kernel","rcrypt","compressible","leaflet.markercluster","jsmv","generator-fedp","nTPL"],"fails":[]}, start sync time: Thu Dec 19 2013 11:58:11 GMT+0800 (CST). +2.6m
cnpmjs.org:sync:index send email with type: log, subject: Sync Finished, html: Sync packages from official registry finished.

• diff npm and cnpm module versions
• sync new versions

安装流程

links

• https://github.com/isaacs/npm/blob/master/lib/publish.js
• npm-registry-client: https://github.com/isaacs/npm-registry-client

当我使用这句话装包的时候(npm --registry=http://registry.cnpmjs.org i express)，返回 401。
相关代码在

由于我是 npm login 过的，所以我的请求自带 http basic auth 的头，于是在这一步被强迫验证从而被返回 401。

当使用 cnpm 的时候就可以成功安装包。因为 cnpm 默认使用空白的 .cnpmrc 文件，覆盖了 npm 原生的 auth 信息。

目前会有旧的版本找不到author的

几个sql就将mysql 塞住了, 我临时屏蔽了 /all 同步接口了

| 12207400 | xxx | localhost:xxx | xxx | Query   | 38626 | Sending data      | 
 |

SELECT name, package FROM module WHERE id IN (SELECT module_id FROM tag WHERE tag="latest" AND name IN (SELECT distinct(name) FROM module WHERE gmt_modified > '2013-12-12 19:24:32.248')) ORDER BY name

cnpmjs.org:sync:status sync [deferred] success +3s
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [c#0] [devmailer] start... +455ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [devmailer] no times and no versions +2ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [devmailer] all versions are exists +0ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [devmailer] synced success, 0 versions: +0ms
cnpmjs.org:sync:status sync [devmailer] success +457ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [c#0] [dice-roller] start... +419ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [dice-roller] no times and no versions +2ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [dice-roller] all versions are exists +0ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [dice-roller] synced success, 0 versions: +0ms
cnpmjs.org:sync:status sync [dice-roller] success +421ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [c#0] [dictation-fix-server] start... +301ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [dictation-fix-server] no times and no versions +3ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [dictation-fix-server] all versions are exists +0ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:33] [dictation-fix-server] synced success, 0 versions: +0ms
cnpmjs.org:sync:status sync [dictation-fix-server] success +304ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:34] [c#0] [difflet-papandreou] start... +495ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:34] [difflet-papandreou] no times and no versions +2ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:34] [difflet-papandreou] all versions are exists +0ms
cnpmjs.org:proxy:sync_module_worker [2013-12-19 11:10:34] [difflet-papandreou] synced success, 0 versions: +0ms
cnpmjs.org:sync:status sync [difflet-papandreou] success +497ms

https://github.com/isaacs/couch-login

http://registry.npmjs.org/-/user/org.couchdb.user:fengmk2

http://registry.npmjs.org/chiron

1. search页找不到指定模块的时候，增加sync按钮
2. package详情页，增加sync按钮

看看npm web是如何获取最近更新的包的。

• 按小时定时同步, 本地合并后, 将备份文件上传到cdn.