Giter Club home page Giter Club logo

architect's Introduction

Architect Logo

GitHub CI status npm version Apache-2.0 License

Build ultra scalable database backed web apps on AWS serverless infrastructure with full local, offline workflows, and more. Full documentation found at: https://arc.codes

Requirements

Installation

Make sure you have at least Node.js version 14 installed.

Open your terminal to install arc:

npm i @architect/architect --save-dev

Check the version:

npx arc version

Protip: run arc with no arguments to get help

Work locally

Create a new app:

mkdir testapp
cd testapp
npx arc init

Kick up the local dev server:

npx arc sandbox

Cmd / Ctrl + c exits the sandbox

Deploy to AWS

Deploy the staging stack:

npx arc deploy

Protip: create additional staging stacks with --name

Ship to a production stack:

npx arc deploy --production

Add Architect syntax to your text editor

VS Code

Sublime Text

Vim

Learn more

Head to https://arc.codes to learn more!


Founding team

Amber Costley, Angelina Fabbro, Brian LeRoux, Jen Fong-Adwent, Kristofer Joseph, Kris Borchers, Ryan Block, Spencer Kelley

Special thanks

Pinyao Guo for the Architect GitHub name

architect's People

Contributors

alexdilley avatar aphex avatar austinkelleher avatar brianleroux avatar busticated avatar clintjhill avatar dependabot-preview[bot] avatar filmaj avatar fuentesjr avatar grncdr avatar herschel666 avatar jamietanna avatar jesusgollonet avatar jkarsrud avatar jondavidjohn avatar julianduque avatar kborchers avatar kevinsimper avatar kmelve avatar kristoferjoseph avatar mattbajorek avatar mikeal avatar mikemaccana avatar ryanblock avatar samrocksc avatar shawnhosea avatar supaspoida avatar taphill avatar tbeseda avatar timmak avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

architect's Issues

from `npm run dns` - if certs not issued, throws request error

This line: https://github.com/arc-repos/arc-workflows/blob/master/src/dns/04-create-domain.js#L41 will throw an exception if the domain hasn't finished issuing the SSL certificates. In my case, the prod domain had been issued, but the *.domain.com had not, so this was throwing inside of arc:

❯ npm run dns

> [email protected] dns
> AWS_PROFILE=arc AWS_REGION=us-east-1 arc-dns


Found Route53 hosted zone domain.io

Please ensure your domain registration is using these nameeservers:
ns-1119.awsdns-11.org.
ns-23.awsdns-02.com.
ns-1940.awsdns-50.co.uk.
ns-875.awsdns-45.net.


/Users/remy/Sites/domain/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

TypeError: Cannot read property 'CertificateArn' of undefined
    at Response.<anonymous> (/Users/remy/Sites/domain/node_modules/@architect/workflows/src/dns/04-create-domain.js:41:96)
    at Request.<anonymous> (/Users/remy/Sites/domain/node_modules/aws-sdk/lib/request.js:364:18)
    at Request.callListeners (/Users/remy/Sites/domain/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/Users/remy/Sites/domain/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/Users/remy/Sites/domain/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/Users/remy/Sites/domain/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/Users/remy/Sites/domain/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /Users/remy/Sites/domain/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/Users/remy/Sites/domain/node_modules/aws-sdk/lib/request.js:38:9)

Automatically create /src/shared

The /src/shared folder is super helpful and powerful, but right now it isn't discoverable yet – you need to know about its existence from docs.

Proposing we automatically create that folder when running workflows if it doesn't already exist. (May want to include a file that has some best practices or a link to the docs, since misuse of the /src/shared folder could very easily bloat up folks' Lambdas!

Frontend Workflow RFC

  • Feature Name: Front End Development Support
  • Start Date: 2017-08-25

Summary

Architect supports text/html HTTP endpoints; a well supported frontend development workflow will make working with architect easier.

Motivation

To enable frontend developers the ability to build dynamic AWS Lambda based apps as quickly as single page apps can be developed and deployed.

Use cases:

  • simple single page apps with a simple script embed
  • simple transpiles
  • full client side apps with routes and code splitting
  • fully 'server' side rendered apps that hydrate client side complete with code splitting

What is the expected outcome?

  • setup should be trivial and seamless with existing tooling workflows
  • webpack, babel, rollup and browserify should be fully supported with example docs
  • deployments should be isolated and immutable
  • s3 buckets -> cloudfront distributions -> route53 domains for staging and production
  • provisioning and deployment MUST be as fast as the aws rate limits allow

Guide-level explanation

Setup

Add @static to your .arc file:

@app
testapp

@static
# …has no config! please use this space to instead explain how `npm run build` generates .static
# whats .static? read below!

@domain
wut.click

npm run create generates S3 buckets

  • if @static is defined
  • create two s3 buckets appname-staging-static and appname-production-static with CORS * (?)
  • adds .static to .gitignore
  • buckets are made web accessible

npm run dns sets up static domains

  • fails if buckets do not exist
  • creates CloudFront Distribution of the previously created S3 buckets
  • static.domain.com and static.staging.domain.com are created and CNAME'd to aforementioned cloudfront dist

Deployment

Devs are encouraged to setup npm run build to generate a folder .static with their favorite tool. Webpack. Browserify. Rollup or Babel. It doesn't matter as long as you generate ./.static. Also! .static is not checked in.

npm run deploy static

  • looks for .static and skips if it does not exist with a warning
  • next checks for an S3 bucket appname-staging-static/MAJOR.MINOR.PATCH and halts if it does exist warning that it already exists (where MAJOR.MINOR.PATCH is derived from the root package.json) to ensure immutability of builds
  • otherwise uploads contents of .static to appname-staging-static/MAJOR.MINOR.PATCH/
  • finally print urls of deployed artifacts

Local Development

npm start

  • starts a static file server for .static on localhost:3333 if it exists

Reference-level explanation

  • npm run create generates s3 bukkits
  • npm run dns generates CloudFront dist and maps to static.staging.domain.com and static.domain.com
  • npm run deploy static deploys the contents of .static to static.staging.domain.com/MAJOR.MINOR.PATCH
  • ARC_DEPLOY=production npm run deploy static deploys to static.domain.com/MAJOR.MINOR.PATCH
  • npm start serves assets in .static on http://localhost:3333

Drawbacks

Why should we not do this?
???

Rationale and Alternatives

  • Why is this design the best in the space of possible designs?

Alternative designs could invite drawing front end complexity into the arc codebase.

  • What other designs have been considered and what is the rationale for not choosing them?

We did consider something higher level that baked in serverside render with clientside hydration complete with code splitting. This proposal is a stepping stone to acheiving that with maximal performance and the flexiblity for the community to roll their own approaches.

  • What is the impact of not doing this?

Frontend developers continue deploying single pages applications without the awesome power of a dynamic Node process Lambda behind 'em.

Unresolved questions

  • What parts of the design do you expect to resolve through the RFC process before this gets merged?
  • What parts of the design do you expect to resolve through the implementation of this feature before stabilization?
  • What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC?

deploy doesn't check for empty reply when calling gateway.getRestApis

_get-url checks for an error being returned but doesn't check if result.items is empty.

https://github.com/arc-repos/arc-workflows/blob/master/src/deploy/src/_get-url.js#L24
As a result line 28 will throw a TypeError.

Unsure as of yet how I got here but would think an extra check on items.length would be appropriate.

stacktrace:

TypeError: Cannot read property 'id' of undefined
    at Response._getRestApis (/tmp/arcs/node_modules/@architect/workflows/src/deploy/src/_get-url.js:30:95)
    at Request.<anonymous> (/tmp/arcs/node_modules/aws-sdk/lib/request.js:364:18)
    at Request.callListeners (/tmp/arcs/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/tmp/arcs/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/tmp/arcs/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/tmp/arcs/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/tmp/arcs/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /tmp/arcs/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/tmp/arcs/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/tmp/arcs/node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (/tmp/arcs/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at Request.emit (/tmp/arcs/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/tmp/arcs/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/tmp/arcs/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/tmp/arcs/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /tmp/arcs/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/tmp/arcs/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/tmp/arcs/node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (/tmp/arcs/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at Request.emit (/tmp/arcs/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/tmp/arcs/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/tmp/arcs/node_modules/aws-sdk/lib/request.js:22:10)

[Proposal] Arc Plugins

Why?

As per our discussion early this week, it would be nice if arc provided hooks that can be handled via plugins. This is particularly useful for common tasks that would otherwise have to be implemented using npm scripts.

How?

I propose that we create a new Arc class. This Arc class could contain useful information that can be passed to plugins. For example, we may want a plugin to only be activated in certain configured scenarios. The Arc class could expose the parsed config and other properties.

class Arc {
  constructor (config) {
    this._config = config
    // ... And other stuff
  },

  getConfig () {
    return this._config
  }

  // ... And other stuff
}

Arc plugins could expose the following properties (and probably more):

  • name {String}: Used to log out the plugin name to the console when being installed
  • async install(arc: Arc) {Function): Called once the plugin is being registered. This can return a promise or nothing.
  • async beforeCreate(arc: Arc) {Function): Called once before the create process begins. This can return a promise or nothing.
  • async afterCreate(arc: Arc) {Function): Called once after the create process ends. This can return a promise or nothing.
  • async beforeDeploy(arc: Arc) {Function): Called once before the deploy process begins. This can return a promise or nothing.
  • async afterDeploy(arc: Arc) {Function): Called once after the deploy process ends. This can return a promise or nothing.

example-arc-plugin.js

exports.name = 'example-arc'

exports.beforeCreate = (arc) => Promise.resolve()
exports.afterCreate = (arc) => Promise.resolve()

exports.beforeDeploy = (arc) => Promise.resolve()
exports.afterDeploy = (arc) => Promise.resolve()

Plugins do not necessarily have to return a promise. They can also return nothing:

example-arc-plugin.js

exports.name = 'example-arc'

exports.beforeCreate = (arc) => {
  console.log('Just wanted to log some stuff before creation...')
}

Usage

Arc plugins could be npm installed and registered in a .arc file.

npm install arc-marko --save-dev
@html
get /

@plugins
arc-marko

Additional Thoughts

This part is not necessary, but I wanted to throw it out there regardless. We may find a reason to emit certain events with data that we do not want to be asynchronously handled. The Arc class could extend EventEmitter and emit events:

example-arc-plugin.js

exports.name = 'example-arc'

exports.install = (arc) => {
  arc.on('some-event', (data) => {
    // idk
  })
}

Bug if aws-sdk is install into functions folders

If you install aws-sdk into the functions folders, it will fail before you own code even fails.

It is not because I forgot to set the region.

var arc = require('@architect/functions')

var AWS = require('aws-sdk')

function route(req, res) {
  console.log('kevin')
  AWS.config.update({region: 'us-east-1'});
  console.log(AWS.config)
  var endpoint  = new AWS.Endpoint('http://localhost:5000')
  var dynamo = new AWS.DynamoDB({endpoint})
  dynamo.listTables({}, function _list(err, result) {
    console.log(err, result)
  })
  console.log(JSON.stringify(req, null, 2))
  res({
    json: {msg:`hello world sam`}
  })
}

exports.handler = arc.json.get(route)
Started db @ http://localhost:5000
@json get /api.............................. get-api
@html get /................................. get-index
Started http @ http://localhost:3333
/Users/kevinsimper/Projects/test/architect/src/json/get-api/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

ConfigError: Missing region in config
    at Request.VALIDATE_REGION (/test/architect/src/json/get-api/node_modules/aws-sdk/lib/event_listeners.js:91:45)

"kevin" is never outputted

Dynamo read/write provisioning should be defaulted to 1

The current default is 5 which is probably where people will want to be with their apps but after throwing together a few apps to learn arc, you can start racking up AWS bills for Dynamo tables you aren't even using.

If we default to 1, document clearly that 1 is probably only good for testing and then make that configurable via arc-config, I think that would be the best path forward for Dynamo provisioning.

HTTP methods now need to be uppercase in AWS API Gateway

  • ./src/create/aws/create-slack-endpoints/_create-resource/01-add-method.js line 25 reads: httpMethod, and should read httpMethod: httpMethod.toUpperCase()
  • ./src/create/aws/create-slack-endpoints/_create-resource/02-setup-request line 42 reads: httpMethod, and should read httpMethod: httpMethod.toUpperCase()
  • ./src/create/aws/create-slack-endpoints/_create-resource/03-setup-response line 28 reads: httpMethod, and should read httpMethod: httpMethod.toUpperCase()
  • ./src/create/aws/create-slack-endpoints/_create-resource/03-setup-response line 50 reads: httpMethod, and should read httpMethod: httpMethod.toUpperCase()

bake custom authorizors in

per conversation w @lwcooper

proposed syntax:

@json
auth my-auth-lambda
get /
get /signin
get /reset
get /forgot
get /baz
get /foo
get /bar

everything below the auth tuple is protected by the custom authorizer (which in theory would allow for some routes to be protected and some not / would have to figure out how we'd want to handle subsequent npm run create invocations

DNS issue - 'Certificate must be in 'us-east-1' when the AWS_REGION=us-west-2

First thanks for this repo. Really cool idea, and I'd like to use it - thanks for being a good steward of your community by open sourcing this.

On to the question"
When running the npm run dns step after getting certificates properly accepted, I get an odd error.

$ npm run dns                                                                                                                v6.11.3 | 04:37 pm

> [email protected] dns /Users/j/myproj
> AWS_PROFILE=personal-arc AWS_REGION=us-west-2 arc-dns


Found Route53 hosted zone mydomain.com

Please ensure your domain registration is using these nameeservers:
ns-xx.awsdns-x.net.
ns-xx.awsdns-x.co.uk.
ns-xx.awsdns-x.org.
ns-xx.awsdns-x.com.


/Users/j/myproj/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

BadRequestException: Invalid certificate ARN: arn:aws:acm:us-west-2:xxxxx:certificate/xxx-xxx-xxx. Certificate must be in 'us-east-1'.
    at Object.extractError (/Users/j/myproj/node_modules/aws-sdk/lib/protocol/json.js:48:27)
    at Request.extractError (/Users/j/myproj/node_modules/aws-sdk/lib/protocol/rest_json.js:45:8)
    at Request.callListeners (/Users/j/myproj/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/Users/j/myproj/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/Users/j/myproj/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/Users/j/myproj/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/Users/j/myproj/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /Users/j/myproj/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/Users/j/myproj/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/Users/----/myproj/node_modules/aws-sdk/lib/request.js:685:12)

npm ERR! Darwin 15.6.0
npm ERR! argv "/Users/jono/.nvm/versions/node/v6.11.3/bin/node" "/Users/jono/.nvm/versions/node/v6.11.3/bin/npm" "run" "dns"
npm ERR! node v6.11.3
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE
npm ERR! [email protected] dns: `AWS_PROFILE=personal-arc AWS_REGION=us-west-2 arc-dns`
npm ERR! Exit status 1

Above are the node version and npm version, I'm a bit confused I read here:

Q: Can I use the same certificate in more than one AWS Region?

It depends on whether you’re using Elastic Load Balancing or Amazon CloudFront. To use a certificate with Elastic Load Balancing for the same site (the same fully qualified domain name, or FQDN, or set of FQDNs) in a different Region, you will must request a new certificate for each Region in which you plan to use it. Note that this requires re-validating each name in the certificate for the new Region. To use an ACM certificate with Amazon CloudFront, you must request the certificate in the US East (N. Virginia) region. ACM certificates in this region that are associated with a CloudFront distribution are distributed to all the geographic locations configured for that distribution.

Maybe there is something going on in the background of arc that makes an assumption about zones that don't hold? Or more likely I'm missing something. Any suggestions?

offline docs for tests are failing

Hey future me, its past me, you flailed a demo today. WAY TO GO. The docs for this page: https://arc.codes/guides/offline are, politely, fucked. You need to remove the https from localhost unless you plan on teaching ppl to self sign certs which, even past me agrees would not be fun. Also you need to figure out why the root route slash hangs. IDK!!

add fully qualified url for 302 redirects

need to write some tests buuut I think we need to change this api gateway response pattern to something like below

(.*statusCode\":302.*)|^(http:\/\/|https:\/\/|\/.*)

this will allow

var arc = require('@architect/functions')

function route(req, res) {
  res({
    location: 'https://wtfjs.com'
  })
}

exports.handler = arc.html.post(route)

The regexp above also allows for statusCode to be invoked with a JSON payload to allow headers other than Location be set during 302. (Small enhancement I've been intending for a while.) Also: this will all be completely backwards compatible but old API Gateway routes will need to be deleted in the AWS console and recreated by running npm run create to get the new functionality.

complete `.arc-env` impl

this has had a few fits and starts but I'm ready to finish it. decided we'll use AWS Systems Manager Parameter store for the backing store (because its free) but will make the impl pluggable for those that want to pay for Secrets Manager. both cases we'll have workflows for read/write/verify/sync the variable to a .arc file.

node_modules folder not being created per function

I started another project using workflows 2.1.0 - there appears to be an issue during deployment - if I view the lamda function within AWS Console it only contains the index.js & package.json files.

There's no node_modules folder deployed, and therefore no @architect/functions to import.

To test this theory I've manually run npm install @architect/functions within my lamda folder and re-deployed. This works, without error, and I get the expected response when hitting the endpoint.

Guessing it's something at the point of creating the deployable package??

Node: 8.2.1
Npm: 5.4.1

This is the first project I've created using 2.1.0, have several 1.x.x projects created / deployed successfully from this machine. 👍

[Proposal] Add beforeCreate afterCreate hooks to arc-create

Problem

Currently, arc-deploy supports beforeDeploy and afterDeploy hooks, which allow developers to create plugins. There may be plugins that need to be run before creation and before deploy. In my case, when using arc-plugin-marko, an app needs to run the plugin code before creation otherwise the first deploy (from the create step) will deploy broken code (non-compiled templates, no prebuilt pages, no assets uploaded to S3 etc).

Proposal

Expose beforeCreate and afterCreate hooks.

I would be happy to work on this feature if we decide it's appropriate.

/cc @brianleroux

Duplicate code in create deployments, not sure which runs

In trying to add XML support (progress so far), I've come across some code that can probably be deduplicated, but it's not exactly the same. However, I need more information on this.

The two files that are mostly the same are src/create/aws/create-html-lambda-deployments/index.js and src/create/aws/create-json-lambda-deployments/index.js.

There are two differences in these files (aside from the strings "html" and "json").

  • In the JSON version, _create runs a setTimeout of 2017 milliseconds, but the HTML version runs straight away. Is this timeout necessary?

  • The JSON version includes an _addApiGatewayInvokePermission function that runs in the waterfall, but the HTML version doesn't. There is also an identical _addApiGatewayInvokePermission in 02-setup-request.js. My guess is that the function in 02-setup-request actually does the job and it's not necessary to run it in the create-#{type}-lambda-deployments file too. Can I confirm that? (I'm still working out how to run this in the middle of developing on it, I'll get there, but wanted to ask in case there is anything else to understand about this).

Arc should enforce provider validation rules

Arc should respect cloud provider validation rules.

Repro steps

  • Configure architect for AWS
  • Configure an architect application with a period, example:
@app
mydomain.com

@html
get /
  • Run arc create; it will fail on the period, as period is not a valid character for AWS Lambda

Duplicate fields defined in multiple tables conflict when indexed

Given an .arc file looking something like this

@tables
users
  id *String
organizations
  id *String

@indexes

users
  provider_id *String
organizations
  provider_id *String

Causes the produced dynamo indexes to conflict causing the following error.

@tables create app-staging-users
@tables create app-production-users
@tables create app-staging-organizations
@tables create app-production-organizations
../node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

ValidationException: One or more parameter values were invalid: Duplicate index name: provider_id-index
    at Request.extractError (../node_modules/aws-sdk/lib/protocol/json.js:48:27)
    at Request.callListeners (../node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (../node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (../node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (../node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (../node_modules/aws-sdk/lib/state_machine.js:14:12)
    at ../node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (../node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (../node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (../node_modules/aws-sdk/lib/sequential_executor.js:115:18)

`arc-logs` feature

adds _logs url to API Gateway staging deploy, wires a lambda that prints the latest CloudWatch Logs events, refreshes itself every second

[Bug] npm run dns error

I'm trying to use arc npm run dns with a domain name I've purchased on AWS.

First it was:

/Users/.../node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^
TypeError: Cannot read property 'CertificateArn' of undefined
    at Response.<anonymous> (/.../node_modules/@architect/workflows/src/dns/04-create-domain.js:42:92)
    at Request.<anonymous> (.../node_modules/aws-sdk/lib/request.js:364:18)
    at Request.callListeners (.../node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (.../node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (.../node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/.../node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (.../node_modules/aws-sdk/lib/state_machine.js:14:12)
    at .../node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (.../node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (.../node_modules/aws-sdk/lib/request.js:685:12)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! arc_pizza@1.0.0 dns: `AWS_PROFILE=default AWS_REGION=us-east-1 arc-dns`
npm ERR! Exit status 1

There are two emails from AWS for domain.com and *domain.com. I had missed the second one, so it timed out and caused that error. It might be worth rescuing that error with a more helpful error like "did you validate both certificates?". I'm happy to put in a PR for that.

After validating them both, I now get this error though:

...node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

BadRequestException: The domain name you provided already exists.
    at Object.extractError (/.../node_modules/aws-sdk/lib/protocol/json.js:48:27)
    at Request.extractError (/.../node_modules/aws-sdk/lib/protocol/rest_json.js:45:8)
    at Request.callListeners (/.../node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/.../node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/.../node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/.../node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/.../node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /.../node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/.../node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/.../node_modules/aws-sdk/lib/request.js:685:12)

[Proposal] Re-install modules conditionally using environment variable

Problem

Currently, when arc-prototype-deploy runs, we install modules using cipm as one of the tasks. This is perfect when running from a CI, but if you're running from your local machine, you may have modules linked in that you do not want to get overwritten.

Proposal

We could either inspect NODE_ENV or introduce a new environment variable that arc inspects to determine whether the install task should be run. During local development, I would expect that my linked in modules do not get overwritten.

Table updating - manual or automatic

I'm noticing that if you start with a table with say:

@tables
user
  userID *String

and change it to:

@tables
user
  userID *String
  somethingNew **String

npm run create doesn't actually notice that change and skips. And npm run deploy only deploys functions, not table updates - is there a way that table updates are supposed to happen or do we have to do it manually? Is this a feature we should add or is there a reason it's not there?

new command: npm run sam

  • translates .arc into a SAM YAML
  • delegate create/deploy/etc CloudFormation commands via aws-sdk

tag resources

use @app to tag lambdas, tables, restapis… basically anything generated by arc

Provision API GW methods using uppercase names?

It seems that API GW allows provisioning the method (i.e. HTTP verb) using lowercase and it seems to work fine except when using the in-console test facility. When trying to test a method created as "get" I get...

{
  "cause": "No enum constant com.amazonaws.backplane.executioncore.deployconfig.enums.HttpMethod.get",
  "logref": "e2ed6b96-7516-11e7-afd9-b73280d9a167",
  "message": ""
}

...this is, as far as I can tell, obviously a bug in the API GW console but it might not hurt if Arc created methods using all upper case for HTTP verbs?

(FYI my AWS support case id for this is: 4213986131)

api-gw-sad-panda

Better dynamoDB errors / fixing a `unknown key type`

When attempting to get this table created:

@tables
users
  insert Lambda
  update Lambda
  userID *String
  mac String
  email String
  checkinDay **Number

I'm getting this error:

create @tables worknicer_meraki-production-users
/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

Error: Unknown key type.
    at getKeyType (/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/@architect/workflows/src/create/aws/create-tables/_get-key-type.js:25:11)
    at keys.map.k (/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/@architect/workflows/src/create/aws/create-tables/_get-key-schema.js:7:16)
    at Array.map (<anonymous>)
    at getKeySchema (/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/@architect/workflows/src/create/aws/create-tables/_get-key-schema.js:4:15)
    at Array._createTable (/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/@architect/workflows/src/create/aws/create-tables/_create-table.js:36:26)
    at module.exports (/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/run-waterfall/index.js:24:13)
    at Response._tables (/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/@architect/workflows/src/create/aws/create-tables/_create-table.js:31:9)
    at Request.<anonymous> (/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/aws-sdk/lib/request.js:364:18)
    at Request.callListeners (/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/Users/jono/Dropbox/philia_contracts/worknicer/meraki_cobot_auth/node_modules/aws-sdk/lib/sequential_executor.js:77:10)

I'm running:

$ node -v                                                                                                                                                                  
v8.7.0
$ npm -v                                                                                                                                                                    
5.5.1
# Workflows v 2.1.0
# parser 1.0.6
# functions 1.6.2

Maybe it has something to do with my apps name itself, I'm assuming tables get prefixed with it? worknicer_meraki but that wouldn't explain why the sessions table gets created properly.

I've checked that neither of these tables (user were created using the actual console)

Issue providing status codes for JSON responses

When trying to supply a 403 status to a json response call

res({
  status: 403,
  json: { error: 'Not Authorized' }
})

My local dev environment halts with the following output

undefined:1
[object Object]
 ^

SyntaxError: Unexpected token o in JSON at position 1
    at JSON.parse (<anonymous>)
    at _local (.../node_modules/@architect/workflows/src/sandbox/http/index.js:83:48)
    at Object._err (.../node_modules/@architect/functions/src/http/_err.js:35:3)
    at emitOne (events.js:115:13)
    at process.emit (events.js:210:7)
    at process._fatalException (bootstrap_node.js:329:26)

MissingRequiredParameter error on `npm run create`

When creating a simple hello world app (named pizza because, hey, that's a bit more interesting), I am running into an issue when deploying.

I am not trying to create any tables, but the npm run create script seems to be trying to make Dynamo tables and failing on validation.

Here's the error I see, for both staging and production tables.

create @tables pizza-production-arc-sessions
{ MissingRequiredParameter: Missing required key 'AttributeType' in params.AttributeDefinitions[1]
    at ParamValidator.fail (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/param_validator.js:61:14)
    at ParamValidator.validateMember (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validateList (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/param_validator.js:103:14)
    at ParamValidator.validateMember (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/param_validator.js:90:21)
    at ParamValidator.validateStructure (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/param_validator.js:75:14)
    at ParamValidator.validateMember (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/event_listeners.js:125:42)
    at Request.callListeners (/Users/Lucas/Development/repos/aws_pizza/arc_pizza/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
  message: 'Missing required key \'AttributeType\' in params.AttributeDefinitions[1]',
  code: 'MissingRequiredParameter',
  time: 2017-08-13T04:32:54.301Z }

As of [email protected] install instructions fail

I tried to update my arc-workflows and got some errors about missing packages.

I attempted to reproduce and here is the result:

$ nvm use 6
$ npm init --yes                                                                                                             v6.11.3 | 07:27 am
  npm install @architect/workflows --save
  touch .arc
Wrote to .../test_arc/package.json:

{
  "name": "test_arc",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


[email protected] /.../test_arc
├── @architect/[email protected]  extraneous
├── @smallwins/[email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
├── [email protected]  extraneous
└── [email protected]  extraneous

npm WARN [email protected] No description
npm WARN [email protected] No repository field.
npm ERR! Darwin 15.6.0
npm ERR! argv "/Users/jono/.nvm/versions/node/v6.11.3/bin/node" "/Users/jono/.nvm/versions/node/v6.11.3/bin/npm" "install" "@architect/workflows" "--save"
npm ERR! node v6.11.3
npm ERR! npm  v3.10.10
npm ERR! path /.../test_arc/node_modules/.staging/@iarna/cli-2e39527f
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall rename

npm ERR! enoent ENOENT: no such file or directory, rename '/.../test_arc/node_modules/.staging/@iarna/cli-2e39527f' -> '/.../test_arc/node_modules/cipm/node_modules/@iarna/cli'
npm ERR! enoent ENOENT: no such file or directory, rename '/.../test_arc/node_modules/.staging/@iarna/cli-2e39527f' -> '/.../test_arc/node_modules/cipm/node_modules/@iarna/cli'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! Please include the following file with any support request:
npm ERR!     /.../test_arc/npm-debug.log

When I use node version 8 it works. Can we use node 8 for architect even though lambda only supports 6.10.x?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.