studds / nx-aws Goto Github PK

Currently @nx-aws/build runs one build per entry. This is needlessly inefficient and I don't think it has any actual benefits. Switch to a single build with multiple entries.

Hi @studds,

first of all, thank you for creation of these useful builders.
I have a question specific to @nx-aws/s3:deploy:
What are the possible options? can you please improve the documentation a bit so it's clear how to use it to deploy static files to S3? And can you think of how to improve the setup to zip it before uploading to s3?

Thanks in advance,
Filip

The update to NX 15 in this repo means that it now uses webpack. I see that the builders in this package make use of overwriting the webpack config file. However, I'm now running into issues with some code where I dont see any chunks output from webpack when I expect to the see them.

Specifically, my desire is to have dynamically loaded modules within the lambda functions via await import('@mypackage/api'). However even if I set /* webpackMode: "lazy-once" */ or other options, I still dont see any chunks being created in the dist folder by nx-aws.

Any thoughts?

If the regional parameter isn't defined, should allow to fall back on the default region.

when trying to deploy, I got an error message about the missing dependency @nx-aws/core. Should this be bundled as a dependency or peer dependency? my guess would be dependency but would prefer @studds's input before raising a PR

Can either:

• Be invoked directly
• Be invoked via --frontendProject when creating a new @nx-aws/sam:application

This configuration fails with the error: Required property 'configFileName' is missing

Not sure why, as "config" isn't there...

"deploy": {
      "builder": "@nx-aws/s3:deploy",
      "options": {
          "bucket": {
              "targetName": "ngh-signatures-api:deploy",
              "outputName": "WebBucket"
          },
          "distribution": {
              "targetName": "ngh-signatures-api:deploy",
              "outputName": "DistributionId"
          }
      }
  },

Quick note I added a PR for windows support. Not sure if you got a notification about it so I thought I would leave a message here.

Thanks

The stackNameFormat option of the deploy command seems to have no effect.

The following section of deploy,ts shows that undefined is being passed to funciton formatStackName where I believe options.stackNameFormat should have been passed instead.

When running sam via execute it's loads faster if we skip pulling the image.

I hardcoded it in my repo... thinking it would be nice to have as an option, as I guess you sometimes want to pull the image.

Maybe something like:

nx serve api --skip-pull-image

const args = [
            'local',
            'start-api',
            '--template',
            templatePath,
            '--skip-pull-image'
        ]

Not sure the best place... way to do this. What do you think?

Hello how are you?

Do you have any plans to work on the migration to nx 12? I confess that I tried to make it and send it as a contributing, but I was unsuccessful. This library is awesome and it would be great to keep it. I hope to help more in another opportunity.

this is the error when using with nx 12 version

Running build for entry app/hello/hello
Cannot read property 'length' of undefined

———————————————————————————————————————————————

>  NX   ERROR  Running target "build" failed

Has - DomainName: !Sub '${WebBucket}.s3-${AWS::Region}.amazonaws.com'. Should be - DomainName: !GetAtt WebBucket.RegionalDomainName

In my main API template file, I have added some parameters for the domain name and SSL certificate which I need to manually create.

My app is going to have multiple domains, dev, prod + 1 domain per developer.

Where do you think the best place to set these values is.

1. somewhere in AWS.
2. In workspace.json as a configuration (what I am leaning towards)
3. In some other file SAM TOML file ?

Parameters:
  Domain:
    Description: Domain name
    Type: String
    Default: dev.foo.com
  DomainSSLCertificateARN:
    Description: Domain name SSL certificate
    Type: String
    Default: arn:aws:acm:us-east-1:938175342066:certificate/sgsfdfgfsdgdfg

The main reason I am asking is the configurations maybe will get a bit large when every developer needs an entry. Still, this might be the best place.

At the moment, the outputPath is retrieved from the "build" target and any files at that path are uploaded, and this is hardcoded. If the "build" target doesn't exist or doesn't have the right "outputPath", then it will fail (if the outputPath can't be retrieved) or upload the wrong files (if the outputPath is wrong).

This is the npm page of nx-sam:

These fields are missing:

Adding these 2 row to package.json will resolve it

    "repository": "https://github.com/studds/nx-aws/",
    "website": "https://github.com/studds/nx-aws/edit/main/packages/sam/",

As you can see in this image if you add a second function to the template.yaml when it builds it seems to still build the first function. Pull down this copy of your sample app and build the API to replicate. https://github.com/vespertilian/nx-aws-example

First of all love the project <3

So have project with two sam apps and I want to move them into NX.

Your plugin seems like an awesome way to do that. So as i'm trying it out I feel like the docs are missing some getting started sections.

If I simple try installing it and generating an app, then deploying it I get this error.

Required property 's3Bucket' is missing

As so don't really know where to look, what s3Bucket config was I suppose to have and where should I put it?

ChristopherPHolder/app-speed#30

Is there something more I need to do for the set up?

Im happy to contribute with the docs so make easer for the next person :)

Looking for ideas of the most important schematics to add. @nhammond101 - ideas?

In at least some environments, there are missing dependencies:

• circular-dependency-plugin

I now get that error:

NX Cannot find module '@nrwl/workspace/src/utilities/run-webpack'

This is not working on nx 13, i get the following error:

> nx run mailvideo-sam:build

Running build for entry src\lambdas\hello
Cannot convert undefined or null to object

 ———————————————————————————————————————————————————————————

 >  NX   Running target "mailvideo-sam:build" failed

   Failed tasks:

   - mailvideo-sam:build

   Hint: run the command with --verbose for more details.

   See Nx Cloud run details at https://nx.app/runs/Y0aUShC9QjB

 ELIFECYCLE  Command failed with exit code 1.

build config

{
	"root": "apps/mailvideo/sam",
	"sourceRoot": "apps/mailvideo/sam/src",
	"projectType": "application",
	"targets": {
		"build": {
			"builder": "@nx-aws/sam:build",
			"options": {
				"outputPath": "dist/apps/mailvideo/sam",
				"template": "apps/mailvideo/sam/template.yaml",
				"tsConfig": "apps/mailvideo/sam/tsconfig.app.json",
				"sourceMap": true,
				"maxWorkers": 1
			},
			"configurations": {
				"production": {
					"optimization": true,
					"extractLicenses": true,
					"inspect": false,
					"fileReplacements": [
						{
							"replace": "apps/mailvideo/sam/src/env/env.ts",
							"with": "apps/mailvideo/sam/src/env/env.prod.ts"
						}
					]
				}
			}
		},
		...

src/lambdas/hello.ts

import type { APIGatewayProxyHandler } from 'aws-lambda';

/**
 * A simple example to return payload: 'hello'.
 * @param event {APIGatewayProxyEvent} event - The event passed to the handler.
 */
export const handler: APIGatewayProxyHandler = async () => {
	return {
		body: JSON.stringify({ payload: 'hello' }),
		statusCode: 200,
	};
};

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Backend for nx-aws-example

Globals:
    Function:
        Runtime: nodejs14.x
        Timeout: 5
        MemorySize: 128

Resources:
    Hello:
        Type: AWS::Serverless::Function
        Properties:
            CodeUri: src/lambdas
            Handler: hello.handler

NX Report

   Node : 16.13.2
   OS   : win32 x64
   pnpm : 6.31.0

   nx : 13.7.2
   @nrwl/angular : undefined
   @nrwl/cli : 13.7.2
   @nrwl/cypress : 13.7.2
   @nrwl/detox : undefined
   @nrwl/devkit : 12.10.1
   @nrwl/eslint-plugin-nx : 13.7.2
   @nrwl/express : undefined
   @nrwl/jest : 13.7.2
   @nrwl/js : 13.7.3
   @nrwl/linter : 13.7.2
   @nrwl/nest : undefined
   @nrwl/next : undefined
   @nrwl/node : 12.10.1
   @nrwl/nx-cloud : 13.1.4
   @nrwl/react : undefined
   @nrwl/react-native : undefined
   @nrwl/schematics : undefined
   @nrwl/storybook : 13.7.2
   @nrwl/tao : 13.7.2
   @nrwl/web : undefined
   @nrwl/workspace : 13.7.3
   typescript : 4.5.5
   rxjs : 7.5.4
   ---------------------------------------
   Community plugins:
         @nx-aws/sam: 0.12.2

• Should prompt for a stack name prefix (which necessitates adding a stack name prefix option)

I have been working on setting up multiple environments for my app and locally I was trying to use AWS SSO which is a bit more secure as you do not keep credentials on your machine.

However, I am getting an error:
Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1

I don't think the V2 SDK supports SSO.
aws/aws-sdk-js-v3#1158

Are you good for a PR to update this?

Add documentation for how the s3:deploy gets the bucket name and distro:

Take the following deploy target:

{
  "deploy-s3": {
    "builder": "@nx-aws/s3:deploy",
    "options": {
      "destPrefix": "public",
      "bucket": {
        "targetName": "docs:deploy-cf",
        "outputName": "WebBucket"
      },
      "distribution": {
        "targetName": "docs:deploy-cf",
        "outputName": "DistributionId"
      }
    },
    "configurations": {
      "production": {
        "stackSuffix": "prod"
      }
    }
  }
}

To deploy to S3 we need an s3 bucket and then CloudFront distribution (so that we can trigger an invalidation of the deployed files.) We assume that there is a CloudFormation template which creates the S3 bucket and CF distribution, and outputs the web bucket name and distribution id as stack outputs. The configuration above then retrieves those outputs. It works like this:

1. Retrieve the nx configuration for "docs:deploy-cf" (from targetName)
2. Calculate the stack name - taking into account the current stack suffix
3. Retrieve the stack outputs

So, for example if running nx deploy-s3 docs:

1. Retrieve the nx configuration for "docs-deploy-cf"
2. Re-create the stack name, in this case docs-dev because we're running with the default (dev) stackSuffix.
3. Retrieve outputs from the CloudFormation stack docs-dev
4. Get the values for WebBucket and DistributionId

Alternatively, if running nx deploy-s3 docs --configuration production:

1. Retrieve the nx configuration for "docs-deploy-cf"
2. Re-create the stack name, in this case docs-prod because we've over-ridden the default stackSuffix to prod
3. Retrieve outputs from the CloudFormation stack docs-prod
4. Get the values for WebBucket and DistributionId

Hey,

Thanks for creating this project.

Exploring lots of different serverless options currently.
Do you have any sample apps using nx-aws?
Also do you have an opinion on SAM vs Serverless?

Thanks again!

So I know that under the hood nx-aws uses a nx-webpack build.

I was looking at the bundles and they include the typescript es6 features. I think that as most people would be using a modern version of lambda node > 10 ... maybe even 12 we can make some gains here.

Also, I don't think the current build does any tree shaking. Improving this could also make a big difference.

I plan to look at this sometime after Christmas or early next year. Just logging it here for you to keep in mind for future releases.

I think we should support local invoke for simple functions. I currently have a use case where I want to test some plain functions it's a bit of a pain. Unless you can already do this and I just don't know how.

So I am struggling to get an example of lambda layers working.

My attempt is on the lambda layer branch of this repo https://github.com/vespertilian/nx-aws-example/tree/lambda-layers which is a clone of your example repo.

My real goal is to get bcrypt and similar libraries that require C++ binaries to work with nx-aws. So even if I get this working I think there is still some work I would need to do.

I have done a similar thing once before with Apex lambda .. now deprecated.

It involves installing the node module for a specific target for deployment. Something like this npm install bcrypt --save --target=8.1.0 --target_arch=x64 --target_platform=linux --target_libc=glibc

Then excluding that from the Webpack build.

It would be great to have a layer that could handle this for me.

Any chance we could chat again? I could buy you a coffee or send some $$ you way. Bit stuck and I feel like you could unstick me quick smart. Again just email [email protected]

I get the error in the title after an nx upgrade, here are the deps I've got:

"@nrwl/workspace": "15.3.0",
"@nx-aws/sam": "0.15.1",

When providing values for --parameter-overrides, need to quote values with spaces, eg: --parameter-overrides ParamName="I have spaces"

aws/aws-sam-cli#2383

@studds not sure if you have seen this but warm container support got added to SAM. Make development so much better.

        "serve": {
          "builder": "@nx-aws/sam:execute",
          "options": {
            "buildTarget": "api:build",
            "packageTarget": "api:package",
            "args": ["--skip-pull-image", "--warm-containers=EAGER"]
          }
        },

Hi,

This library looks like a perfect fit for a project I am working on at the moment. However, I am finding it hard to get it to work in my project. I am currently getting this error when running a test from the API Gateway:

Endpoint response body before transformations: {"errorMessage":"Cannot find module 'source-map-support/register'","errorType":"Error","stackTrace":["Function.Module._load (module.js:474:25)","Module.require (module.js:596:17)","require (internal/module.js:11:18)","Object.source-map-support/register (/var/task/lambda.js:2559:18)","webpack_require (/var/task/lambda.js:20:30)","Object../node_modules/@nx-aws/sam/src/build/source-map-install.js (/var/task/lambda.js:2424:1)","webpack_require (/var/task/lambda.js:20:30)","Object.0 (/var/task/lambda.js:2436:1)","webpack_require (/var/task/lambda.js:20:30)"]}
Tue Sep 03 08:02:11 UTC 2019 : Lambda execution failed with status 200 due to customer function error: Cannot find module 'source-map-support/register'. Lambda request id: 1317e79e-6606-4c96-a7ac-c96058d42101
Tue Sep 03 08:02:11 UTC 2019 : Method completed with status: 502

Template.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Lambda with nestjs.

Resources:
LambdaNestJSFunction:
Type: AWS::Serverless::Function
Properties:
# This is the name of the handler file and then the name of the exported handler function
# (standard SAM approach)
Handler: lambda.handler
Runtime: nodejs8.10
# CodeUri should be the directory, relative to template.yaml, where the handler file is found
CodeUri: src/app
MemorySize: 1512
Policies: AWSLambdaBasicExecutionRole
Timeout: 60
Events:
GetResource:
Type: Api
Properties:
Path: /{proxy+}
Method: any

lambda.ts:

require('source-map-support').install();

import { Context, Handler } from 'aws-lambda';
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { Server } from 'http';
import { createServer, proxy } from 'aws-serverless-express';
import { eventContext } from 'aws-serverless-express/middleware';
// @ts-ignore
import express from 'express';

let cachedServer: Server;
const expressApp = require('express')();

// NOTE: If you get ERR_CONTENT_DECODING_FAILED in your browser, this is likely
// due to a compressed response (e.g. gzip) which has not been handled correctly
// by aws-serverless-express and/or API Gateway. Add the necessary MIME types to
// binaryMimeTypes below
const binaryMimeTypes: string[] = [];

async function bootstrapServer(): Promise {
const nestApp = await NestFactory.create(AppModule, expressApp);
nestApp.use(eventContext());
await nestApp.init();
return createServer(expressApp, undefined, binaryMimeTypes);
}

export const handler: Handler = async (event: any, context: Context) => {
if (!cachedServer) {
console.log('Bootstraping server');
cachedServer = await bootstrapServer();
} else {
console.log('Using cached server');
}
return proxy(cachedServer, event, context, 'PROMISE').promise;
};

btw., It would be fantastic if there was a small working nx app with a nestjs back end app and a dummy angular front end app showing this library in action.

Many thanks for any help you could provide.

We generate:

  # Policy to allow CF to access the private webbucket
  WebBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket:
        Ref: WebBucket
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              CanonicalUser: !GetAtt CloudFrontOriginIdentity.S3CanonicalUserId
            Action: 's3:GetObject'
            Resource: !Sub '${WebBucket.Arn}/*'

Should be:

  # Policy to allow CF to access the private webbucket
  WebBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket:
        Ref: WebBucket
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              CanonicalUser: !GetAtt CloudFrontOriginIdentity.S3CanonicalUserId
            Action:
              - 's3:GetObject'
              # ListBucket is required so that a 404 is returned on misses
              - 's3:ListBucket'
            Resource:
              - !Sub '${WebBucket.Arn}/*'
              - !Sub '${WebBucket.Arn}'

Perhaps a way to customize nx-aws to use LocalStack for development and make it easy to configure?

LocalStack could be used for S3 and Lambda locally to prevent need to upload artifacts for development. Would also allow additional services to be hooked into local development such as SQS and others.

Dependabot couldn't find a package.json for this project.

Dependabot requires a package.json to evaluate your project's current JavaScript dependencies. It had expected to find one at the path: /packages/sam/package.json.

If this isn't a JavaScript project, or if it is a library, you may wish to disable updates for it in the .dependabot/config.yml file in this repo.

View the update logs.

Hey,

So I have an account-based AWS setup (separate accounts for each env). I changed the stack suffix from dev to prod as they are all effectively prod.

However, I can not serve locally when using layers as we seem to use a mimic env var which does not set the layer suffix, layers are also deployed as my-layer-name-[dev/prod].

I was looking at trying to add this however I cannot seem to even build the master branch. There was no package.lock file checked in so maybe an update has broken the build?

Can you build the project after an npm install?

Also, I am thinking maybe mimicEnv useDeployedSuffix ? Or does it mimic other aspects of the build?