Giter Club home page Giter Club logo

serverless-lerna-yarn-starter's Introduction

Serverless Lerna + Yarn Workspaces Starter Seed Status

A Serverless monorepo starter that uses Lerna and Yarn Workspaces.

  • Designed to scale for larger projects
  • Maintains internal dependencies as packages
  • Uses Lerna to figure out which services have been updated
  • Supports publishing dependencies as private NPM packages
  • Uses serverless-bundle to generate optimized Lambda packages
  • Uses Yarn Workspaces to hoist packages to the root node_modules/ directory

Installation

To create a new Serverless project

$ git clone https://github.com/AnomalyInnovations/serverless-lerna-yarn-starter my-project

Enter the new directory

$ cd my-project

Install NPM packages for the entire project

$ yarn

How It Works

The directory structure roughly looks like:

package.json
/libs
/packages
  /sample-package
    index.js
    package.json
/services
  /service1
    handler.js
    package.json
    serverless.yml
  /service2
    handler.js
    package.json
    serverless.yml

This repo is split into 3 directories. Each with a different purpose:

  • packages

    These are internal packages that are used in our services. Each contains a package.json and can be optionally published to NPM. Any changes to a package should only deploy the service that depends on it.

  • services

    These are Serverless services that are deployed. Has a package.json and serverless.yml. There are two sample services.

    1. service1: Depends on the sample-package. This means that if it changes, we want to deploy service1.
    2. service2: Does not depend on any internal packages.

    More on deployments below.

  • libs

    Any common code that you might not want to maintain as a package. Does NOT have a package.json. Any changes here should redeploy all our services.

The packages/ and services/ directories are Yarn Workspaces.

Services

The Serverless services are meant to be managed on their own. Each service is based on our Serverless Node.js Starter. It uses the serverless-bundle plugin (based on Webpack) to create optimized Lambda packages.

This is good for keeping your Lambda packages small. But it also ensures that you can have Yarn hoist all your NPM packages to the project root. Without Webpack, you'll need to disable hoisting since Serverless Framework does not package the dependencies of a service correctly on its own.

Install an NPM package inside a service.

$ yarn add some-npm-package

Run a function locally.

$ serverless invoke local -f get

Run tests in a service.

$ yarn test

Deploy the service.

$ serverless deploy

Deploy a single function.

$ serverless deploy function -f get

To add a new service.

$ cd services/
$ serverless install --url https://github.com/AnomalyInnovations/serverless-nodejs-starter --name new-service
$ cd new-service
$ yarn

Packages

Since each package has its own package.json, you can manage it just like you would any other NPM package.

To add a new package:

$ mkdir packages/new-package
$ yarn init

Packages can also be optionally published to NPM.

To use a package:

Note that packages should be added by specifying the version number declared in their package.json. Otherwise, yarn tries to find the dependency in the registry.

Libs

If you need to add any other common code in your repo that won't be maintained as a package, add it to the libs/ directory. It does not contain a package.json. This means that you'll need to install any NPM packages as dependencies in the root.

To install an NPM package at the root.

$ yarn add -W some-npm-package

Deployment

We want to ensure that only the services that have been updated get deployed. This means that, if a change is made to:

  • services

    Only the service that has been changed should be deployed. For ex, if you change any code in service1, then service2 should not be deployed.

  • packages

    If a package is changed, then only the service that depends on this package should be deployed. For ex, if sample-package is changed, then service1 should be deployed.

  • libs

    If any of the libs are changed, then all services will get deployed.

Deployment Algorithm

To implement the above, use the following algorithm in your CI:

  1. Run lerna ls --since ${prevCommitSHA} -all to list all packages that have changed since the last successful deployment. If this list includes one of the services, then deploy it.
  2. Run git diff --name-only ${prevCommitSHA} ${currentCommitSHA} to get a list of all the updated files. If they don't belong to any of your Lerna packages (lerna ls -all), deploy all the services.
  3. Otherwise skip the deployment.

Deploying Through Seed

Seed supports deploying Serverless monorepo projects that use Lerna and Yarn Workspaces. To enable it, add the following to the seed.yml in your repo root:

check_code_change: lerna

To test this:

Add the App

  1. Fork this repo and add it to your Seed account.
  2. Add both of the services.
  3. Deploy your app once.

Update a Service

  • Make a change in services/service2/handler.js and git push.
  • Notice that service2 has been deployed while service1 was skipped.

Update a Package

  • Make a change in packages/sample-package/index.js and git push.
  • Notice that service1 should be deployed while service2 will have been skipped.

Update a Lib

  • Finally, make a change in libs/index.js and git push.
  • Both service1 and service2 should've been deployed.

This repo is maintained by Serverless Stack.

serverless-lerna-yarn-starter's People

Contributors

brightobluefin avatar fwang avatar jayair avatar pujitm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

serverless-lerna-yarn-starter's Issues

when to add as a lib vs package

I understand lib could be just common functions used across the repository.

When do we define a package? In an AWS Serverless application, what could be defined as packages?

Does serverless framework support hoisting now?

I have been reading in a bunch of places that serverless framework does not support hoisting and that it needs to be disabled for it. You seem to have enabled it though. Does it work for serverless?

Initial install throws serverless DeprecationWarning DEP0066

Deprecation warning only, still appears to work.

Ran this at command line (Windows 10, Powershell):
serverless install --url https://github.com/AnomalyInnovations/serverless-lerna-yarn-starter --name microservices

Error:
Serverless: Downloading and installing "serverless-lerna-yarn-starter"...
(node:23128) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated

Serverless version:
Framework Core: 1.78.1
Plugin: 3.7.0
SDK: 2.3.1
Components: 2.34.3

Why is serverless-bundle added as a dev-dependency in each service and not at package.json at root level?

Here: https://serverless-stack.com/chapters/share-code-between-services.html

It's suggested that: We use the package.json at the project root to install the dependencies that will be shared across all the services. For example, the serverless-bundle plugin that we are using to optimally package our Lambda functions is installed at the root level. It doesn’t make sense to install it in each and every service.

But in the project, you have added serverless-bundle as dev-dependency in each service. Why is that?

Initial install throws serverless.yml or serverlss.ts not found in

$ serverless install --url https://github.com/AnomalyInnovations/serverless-lerna-yarn-starter --name my-project

Serverless: Downloading and installing "serverless-lerna-yarn-starter"...
(node:24106) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated

  Serverless Error ---------------------------------------

  serverless.yml or serverlss.ts not found in /MY_PATH/my-project

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              12.18.3
     Framework Version:         1.80.0
     Plugin Version:            3.8.2
     SDK Version:               2.3.1
     Components Version:        2.34.9

obvs MY_PATH is my path! 😉

How to make this work with serverless-offline?

Hello,

I have a project with Lerna, Serverless, all glued together with serverless-bundle. I use TypeScript for my packages (but not my services). I would like to use serverless-offline in my project but the issue is that compiling ts files in packages does not reload serverless-offline. Is that possible?

Thank you

using the deployment strategy without having lerna publish the packages

Deployment Algorithm

  • Run lerna ls --since ${prevCommitSHA} -all to list all packages that have changed since the last successful deployment. If this list includes one of the services, then deploy it.
    This command lists all the packages for me since I am not publishing the package anywhere but will only be running lerna run deploy command which will run sls deploy in services/packages.

How can this be addressed? Why does it list all the packages?

Using Yarn 2 or 1?

Hi,
I'm trying to find examples using Yarn 2 workspaces. Are you using 1 or 2?

I don't see

$ yarn policies set-version berry
$ yarn set version berry

anywhere in the instructions.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.