Giter Club home page Giter Club logo

wombat-dressing-room's Introduction

Wombat Dressing Room

Google's npm registry proxy. Designed to reduce the attack surface of npm packages.

Build Status Code Style: Google

What it does

  • You publish to Wombat Dressing Room, and it enforces additional security rules, before redirecting to registry.npmjs.org.

  • Publishes are made from a single npm account with 2FA enabled (a bot account).

  • Publishes can be made using the npm CLI, by making Wombat Dressing Room the default registry (npm config set registry https://external-project.appspot.com).

Deployment

This service is deployed in 2 distinct services: an external service for registry access; and a protected service for authentication/authorization (you can use a proxy, such as IAP, to limit access to the authentication server).

Prerequisites

Wombat Dressing Room requires:

Create an npm account

You will need to create an npm account, which will be used for publication. This account should be configured such that 2FA is enabled for authentication and publication. When you are given a QR code to scan for your authenticator app, use a QR code reader to fetch and store the secret associated with the 2FA configuration. You will also need to scan the QR code with an authenticator app, so that you can provide an OTP token to npm.

Create a GitHub OAuth Application

As well as an npm account, you must create a GitHub OAuth application. These credentials are used when performing authenication: both when logging into Wombat Dressing Room, for creating tokens, and when verifying certain types of tokens.

Note: the Authorization callback configured with the OAuth application should be the URL of the internal service, with the suffix /oauth/github.

Setup your environment

Once you've addressed the prerequisites, you should create environment files in the config/ directory populating the appropriate variables.

In order to start this service in development you need to create a config/local.env, in order to deploy you'll need an config/external.env and config/internal.env.

Internal environment variables

NPM_OTP_SECRET={the text value of the otp secret}
NPM_TOKEN={the npm token}
GITHUB_CLIENT_ID={github app id}
GITHUB_CLIENT_SECRET={github app secret}
DATASTORE_PROJECT_ID={project datastore is configured for}
LOGIN_ENABLED=yes-this-is-a-login-server
LOGIN_URL=https://project.appspot.com]
REGISTRY_URL=https://external-project.appspot.com

External environment variables

NPM_OTP_SECRET={the text value of the otp secret}
NPM_TOKEN={the npm token}
GITHUB_CLIENT_ID={github app id}
GITHUB_CLIENT_SECRET={github app secret}
DATASTORE_PROJECT_ID={project datastore is configured for}
LOGIN_ENABLED=this-is-not-enabled
LOGIN_URL=https://project.appspot.com]
REGISTRY_URL=https://external-project.appspot.com

Development environment variables

NPM_OTP_SECRET={the text value of the otp secret}
NPM_TOKEN={the npm token}
GITHUB_CLIENT_ID={github app id}
GITHUB_CLIENT_SECRET={github app secret}
DATASTORE_PROJECT_ID={project datastore is configured for}
LOGIN_ENABLED=yes-this-is-a-login-server
LOGIN_URL=http://127.0.0.1:8080
REGISTRY_URL=hhttp://127.0.0.1:8080

Deploy the application

To configure the Google App Engine services used by Wombat Dressing Room, perform an initial deployment:

  1. install the gcloud command line tool, and run gcloud auth login.
  2. run GCLOUD_PROJECT=my-project npm run deploy, where my-project is the project configured in Prerequisites.

Create a datastore table

The tokens used by Wombat Dressing Room are stored in a datastore table, before accessing the application for the first time you should run:

GCLOUD_PROJECT=my-project npm run create-indexes

To populate this datastore schema.

Note: it takes datastore a while to initialize the first time you run the application. You can view the status of index creation in the Cloud Console.

Protect your application with IAP

Wombat Dressing Room consists of an internal application, used for authorization, and an external app, used for proxing to npm. You should limit access to the internal application, a great way to do so is with IAP: configuring the default application, such that only select accounts have access; and configuring the external application with the allUsers group, such that anyone can access the proxy.

Developing the service locally

Populate config/local.env, and run:

npm run develop

Deploying updates

Populate config/external.env, and config/internal.env, and run:

npm run deploy

Contributing

Contributions welcome! See the Contributing Guide.

License

Apache Version 2.0

wombat-dressing-room's People

Contributors

azu avatar bcoe avatar dependabot[bot] avatar erezrokah avatar justinbeckwith avatar loosebazooka avatar maribethb avatar release-please[bot] avatar renovate-bot avatar soldair avatar zcei avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wombat-dressing-room's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • chore(deps): update dependency @types/express-handlebars to v6
  • chore(deps): update dependency chai to v5
  • chore(deps): update dependency eslint to v9
  • fix(deps): update testing-library monorepo (major) (@testing-library/jest-dom, @testing-library/react)
  • ๐Ÿ” Create all rate-limited PRs at once ๐Ÿ”

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

github-actions
.github/workflows/ci.yaml
  • actions/checkout v3
  • actions/setup-node v3
npm
package.json
  • rimraf ^4.0.0
packages/app/package.json
  • @emotion/react ^11.7.1
  • @emotion/styled ^11.6.0
  • @mui/icons-material ^5.2.4
  • @mui/material ^5.2.4
  • @mui/styled-engine ^5.2.4
  • @testing-library/jest-dom ^5.16.1
  • @testing-library/react ^12.1.2
  • @testing-library/user-event ^14.0.0
  • @types/jest ^29.0.0
  • @types/node ^18.0.0
  • @types/react ^17.0.37
  • @types/react-dom ^18.0.0
  • react ^17.0.2
  • react-dom ^17.0.2
  • react-router-dom ^6.2.1
  • react-scripts 5.0.1
  • react-syntax-highlighter ^15.4.5
  • typescript ^4.5.4
  • eslint ^8.5.0
packages/server/package.json
  • @google-cloud/datastore ^7.0.0
  • @npm/types ^1.0.1
  • @otplib/preset-default ^12.0.0
  • cookie-session ^2.0.0-beta.3
  • dotenv ^16.0.0
  • express ^4.16.4
  • github-url-from-git ^1.5.0
  • is-uuid ^1.0.2
  • morgan ^1.9.1
  • octonode ^0.10.0
  • otplib ^12.0.0
  • request ^2.88.0
  • source-map-support ^0.5.9
  • uuid ^9.0.0
  • validate-npm-package-name ^5.0.0
  • @types/chai ^4.2.7
  • @types/cookie-session ^2.0.36
  • @types/express ^4.16.0
  • @types/express-handlebars ^5.0.0
  • @types/github-url-from-git ^1.5.0
  • @types/is-uuid ^1.0.0
  • @types/mocha ^10.0.0
  • @types/morgan ^1.7.35
  • @types/request ^2.48.1
  • @types/sinon ^10.0.6
  • @types/uuid ^9.0.0
  • @types/validate-npm-package-name ^4.0.0
  • c8 ^7.3.1
  • chai ^4.2.0
  • cross-env ^7.0.0
  • gts ^3.1.0
  • mocha ^10.0.0
  • nock ^13.0.0
  • sinon ^15.0.0
  • typescript ^4.0.0

  • Check this box to trigger a request for Renovate to run again on this repository

Doest not work on Node 13

Currently the version of @google-cloud/datastore we're running with, does not work on Node 13, we should upgrade to the latest version of datastore to address this problem.

add CI/CD

we should add our standard kokoro CI/CD setup to this project.

This is blocked by #16 , which is preventing the build from passing in various environments.

Support npm logout

We would like to be able to use npm logout to log out of wombat proxy. Currently, this does not work as the proxy returns a 404 on the attempt.

Currently, if a logout attempt is made the logout does not occur on either the local client or server sides.

Here is an outline of what happens now if a logout is attempted:

$ npm login --registry https://wombat-dressing-room.appspot.com --no-browser
to complete your login please visit:
https://wombat-dressing-room-internal.googleplex.com/_/token-settings?ott=i-was-unsure-if-this-one-was-identifying-as-well
Logged in on https://wombat-dressing-room.appspot.com/.

$ npm whoami --registry https://wombat-dressing-room.appspot.com
github_user:josephperrott

$ npm logout --registry=https://wombat-dressing-room.appspot.com
npm ERR! code E404
npm ERR! 404 Not Found - DELETE https://wombat-dressing-room.appspot.com/-/user/token/this-is-not-the-token-youre-looking-for

npm ERR! A complete log of this run can be found in:
npm ERR!     /usr/local/google/home/jperrott/.npm/_logs/2021-01-19T17_46_03_319Z-debug.log

$ npm whoami --registry https://wombat-dressing-room.appspot.com
github_user:josephperrott

$ cat ~/.npmrc
//wombat-dressing-room.appspot.com/:_authToken=this-is-not-the-token-youre-looking-for

bug(readme): `used or publication` should be `used for publication`

In the README, under Create an npm account used or publication should be used for publication

Environment details

  System:
    OS: macOS 11.0.1
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 1.56 GB / 32.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 14.9.0 - /usr/local/bin/node
    Yarn: 1.22.5 - /usr/local/bin/yarn
    npm: 6.14.9 - /usr/local/bin/npm
    Watchman: 4.9.0 - /usr/local/bin/watchman

Steps to reproduce

  1. Visit https://github.com/GoogleCloudPlatform/wombat-dressing-room#create-an-npm-account

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Remove usage of `url.parse`

In the gts upgrade I'm getting some warnings about url.parse, which we should drop for the URL constructor.

release-backed/monorepo tokens support release-please generated tags/releases

It would be nice wombat would check per-package (or at least for one package?) the existence of a tag/release created for that package by release-please. It seems like right now it checks for a general-purpose named tag/release?

For monorepos, release-please generates a very specific tag name and release name for each package in the repo:

Given a package named foo you get
tag name: foo-v1.2.3
release name: foo foo-v123

For a scoped package name of @foo/bar you get
tag name: bar-v1.2.3
release name: @foo/bar bar-v1.2.3

Ideally wombat would look these up dynamically at publish time but I suppose additional manual user input (similar to per-package token creation) would be ok if it could be edited.

cc @bcoe

If being used for permissions permsRepo souldn't be able to be deleted.

Users shouldn't be able to delete permsRepo from the version currently being published if repository isn't set or doesn't match latest.permsRepo

This could cause users to lock themselves out of packages and the only solution is a support request.

We already required that the repository is set but this changed and we only require that permRepo or repository is set.
pr #79

consider moving to frontend framework

Currently we're using express for serving, but don't have much structure around our frontend JavaScript/serving static assets in general.

Is there a lightweight framework we could use for this? The benefits I see:

  • better separation between model/view.
  • less likely to have a security hiccup if we use a framework that encourages good practices.
  • potentially easier to make style consistent.

Swallowed error if there are fewer than 11 pages of tags

In the getRelease method, an error is thrown if there is any error from the github API. However, under certain circumstances, this error ends up being swallowed in enforceMatchingRelease and reported as "unknown error" which makes debugging this failure case difficult.

I found this while writing test cases for a case where no matching tag exists. The code does not know/check how many pages of tags exist on GitHub. It will always try to check the first 11 pages of tags. In the tests, the API is mocked, so it makes sense that if you do not mock out all 11 pages of responses, you will get an error in the test as you are trying to make an API call that has no mocked response. Even in this case though, the error should be reported as is rather than being swallowed and showing "unknown error"

It should also be verified that this does not cause a problem in production. If a project only has 4 pages of tags, then this method should not throw an error when it tries to search pages 5-11, it should simply report no matching tags found if it exhausts the list of existing tags.

Environment details

  • OS:
  • Node.js version:
  • npm version:
  • wombat-dressing-room version: 1.1

Steps to reproduce

  1. Create a test case for a release-backed token where the matching tag you are looking for does not exist in the list of tags on GitHub. You would expect this test case to report a 400 error and report that the matching tag was not found.
  2. Do not provide all 11 pages of responses from the mocked GitHub API. Instead of seeing the 400 error you would expect, you'll see a 500 error and "unknown error" message instead of something useful.

Improve local development story

I'm trying to follow the directions, and get the application running locally for development. One of the things I'm getting tripped up on is the redirect Url for the GitHub OAuth application. There's this blurb in the README:

Note: the Authorization callback configured with the OAuth application should be the URL of the internal service, with the suffix /oauth/github.

For local setup - its unclear what that url is supposed to be. I'm not sure if I need to configure this to be a localhost:8080 address, or if I need to actually deploy the service just for the url redirect bits after login.

I'm really just trying to slog my way through local.env.

Full unpublish of package fails

npm unpublish my-package-name --force fails to unpublish a newly published library, even if it's within the 72 hour timeframe.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.