architect / functions Goto Github PK
View Code? Open in Web Editor NEWAWS Lambda Node runtime helpers for Architect apps
Home Page: https://arc.codes
AWS Lambda Node runtime helpers for Architect apps
Home Page: https://arc.codes
Describe the issue
Errors in websocket functions are not shown in sandbox
Steps to reproduce
Make this /src/ws/connect
exports.handler = async function webSocketConnect(event) {
console.log(`started`)
// Now break
undefined()
console.log(`end`)
return {
statusCode: OK
}
}
Expected behavior
Expected to see an error in the console.
Instead the function fails silently.
Arc 6.0.20
This @json
route handler returns a 404 error with a JSON response body:
let arc = require("@architect/functions");
function route(req, res) {
res({
json: { error: "Record not found" },
status: 404,
});
}
exports.handler = arc.json.get(route);
This works fine with npx sandbox
, but once deployed to AWS, the response is malformed:
HTTP/1.1 404 Not Found
Content-Type: application/json
Content-Length: 29
Connection: close
Date: Tue, 09 Oct 2018 01:51:46 GMT
x-amzn-RequestId: e0eb42f5-cb65-11e8-8151-61272b94e470
Set-Cookie:
x-amz-apigw-id: OeZz0GEwPHcF93g=
Location: {"json":{"error":"Record not found"},"status":404,"cookie":"_idx=Bc_8OuTkTXn2rPpp5-YUpJeT.oZB50nUXMa6f1pobTyvDF4lTciizMUZgRyaY8%2BdmrxI; Max-Age=2327449906161; Path=/; Expires=Sat, 03 Oct 2043 01:51:46 GMT; HttpOnly; Secure; SameSite=Lax","statusCode":404}
X-Amzn-Trace-Id: Root=1-5bbc09b2-077d8ab2d93f18d9ff444633;Sampled=0
X-Cache: Error from cloudfront
Via: 1.1 68807936c056006818525c5da31d108e.cloudfront.net (CloudFront)
X-Amz-Cf-Id: YQWk0k2i71QlZZ0OwGGEIA6IspUf4ajJj6WjwNnzUAvq8kIJ8vqGNw==
ord not found"},"status":404
Note the Location
header value!
Describe the issue
Based on the code in https://github.com/architect/functions/blob/master/src/static/index.js, I think that if you deploy to AWS and don't have a domain set up for your app (and thus will have /staging
and /production
URL path prefixes for all routes), any usage of arc.static
without setting the second options
argument to {stagePath: true}
will yield an incorrect path (missing the first staging or production URL path segment). As far as I can tell, the only way to get arc.static
to correctly prefix the stage name is by using this second options argument.
Steps to reproduce
Create a basic arc app with:
@static
pragmapublic/
, e.g. public/foo.css
@http
route@architect/functions
and e.g. print out the path to the static asset via a call to arc.static('foo.css')
Run it in sandbox and load up the http function: displays the right path (/_static/foo.css
).
Deploy it to AWS and load it up: womp womp. You'll get HTTP 403 responses from S3 because the stage URL prefix is missing: blahblahblah.amazonaws.com/_static/foo.css
should be blahblahblah.amazonaws.com/staging/_static/foo.css
.
Expected behavior
Works in both sandbox and deployed to AWS.
Additional context
The workaround is to call arc.static
with an options argument that looks like {stagePath: true}
. In sandbox, stagePath
will be ignored (as there is logic in arc.static
protecting against the local case). When deployed to AWS, it will get used in the path. Feels like this should be default behaviour? Though if I understand correctly, that will mess up apps that have and use a domain. Maybe it's simply a docs issue? Not sure.
Starter project linters fail on the reserved word static
A typical request in lambda looks like so:
{ resource: '/api/signup',
path: '/api/signup',
httpMethod: 'POST',
headers:
{ accept:
'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-US,en;q=0.9,es-MX;q=0.8,es;q=0.7,pl-PL;q=0.6,pl;q=0.5',
'cache-control': 'no-cache',
'content-type': 'application/x-www-form-urlencoded',
cookie: '',
Host: 'a5m898djxa.execute-api.us-west-2.amazonaws.com',
origin: 'https://a5m898djxa.execute-api.us-west-2.amazonaws.com',
pragma: 'no-cache',
referer:
'https://a5m898djxa.execute-api.us-west-2.amazonaws.com/production/signup',
'upgrade-insecure-requests': '1',
'User-Agent':
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',
'X-Amzn-Trace-Id': 'Root=1-5d31c85f-211ec0fbab516c9d1d65b34f',
'X-Forwarded-For': '187.143.154.203',
'X-Forwarded-Port': '443',
'X-Forwarded-Proto': 'https' },
multiValueHeaders:
{ accept:
[ 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3' ],
'accept-encoding': [ 'gzip, deflate, br' ],
'accept-language':
[ 'en-US,en;q=0.9,es-MX;q=0.8,es;q=0.7,pl-PL;q=0.6,pl;q=0.5' ],
'cache-control': [ 'no-cache' ],
'content-type': [ 'application/x-www-form-urlencoded' ],
cookie: [ '' ],
Host: [ 'a5m898djxa.execute-api.us-west-2.amazonaws.com' ],
origin: [ 'https://a5m898djxa.execute-api.us-west-2.amazonaws.com' ],
pragma: [ 'no-cache' ],
referer:
[ 'https://a5m898djxa.execute-api.us-west-2.amazonaws.com/production/signup' ],
'upgrade-insecure-requests': [ '1' ],
'User-Agent':
[ 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36' ],
'X-Amzn-Trace-Id': [ 'Root=1-5d31c85f-211ec0fbab516c9d1d65b34f' ],
'X-Forwarded-For': [ '187.143.154.203' ],
'X-Forwarded-Port': [ '443' ],
'X-Forwarded-Proto': [ 'https' ] },
queryStringParameters: null,
multiValueQueryStringParameters: null,
pathParameters: null,
stageVariables: null,
requestContext:
{ resourceId: 'dhjpnf',
resourcePath: '/api/signup',
httpMethod: 'POST',
extendedRequestId: 'dEw-_EA7vHcFZcQ=',
requestTime: '19/Jul/2019:13:40:47 +0000',
path: '/production/api/signup',
accountId: '417817716689',
protocol: 'HTTP/1.1',
stage: 'production',
domainPrefix: 'a5m898djxa',
requestTimeEpoch: 1563543647997,
requestId: 'd0cd3b88-aa2a-11e9-ab89-3d11659ad920',
identity:
{ cognitoIdentityPoolId: null,
accountId: null,
cognitoIdentityId: null,
caller: null,
sourceIp: '187.143.154.203',
principalOrgId: null,
accessKey: null,
cognitoAuthenticationType: null,
cognitoAuthenticationProvider: null,
userArn: null,
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',
user: null },
domainName: 'a5m898djxa.execute-api.us-west-2.amazonaws.com',
apiId: 'a5m898djxa' },
body:
'mahbase64body',
isBase64Encoded: true }
the middleware code (https://github.com/architect/functions/blob/master/src/middleware/index.js#L10-L22), though, inspects the returned value from middleware/routes and if it contains a method
property, assumes it to be a request.
Note that lambda requests don't contain a method
property.
So I think two things need to be done:
To address (2) above, I think we should consider https://github.com/architect/sandbox/issues/14 to be a related / blocking issue.
Describe the issue
If I put this in my unit-test:
const arc = require('@architect/functions')
I get this error:
Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.
Steps to reproduce
Make a test like this:
/* global describe, it, expect, beforeAll, afterAll */
// this causes async-hang
const arc = require('@architect/functions')
const sandbox = require('@architect/sandbox')
beforeAll(async () => {
console.log('Sandbox: start')
await sandbox.start({ quiet: true })
})
afterAll(async () => {
console.log('Sandbox: end')
await sandbox.end()
})
describe('jest', () => {
it('should have unit-tests', async () => {
expect(1 + 1).toEqual(2)
})
})
run jest
.
If you comment out the require of @architect/functions
, it runs fine and start/stops the sandbox (on 1.5.9-RC.2
.)
Expected behavior
I expect no breaking async side-effects on import, so there is no error.
Solutions
I narrowed the problem down to 3 lines in 2 files, in @architect/functions/src/tables/
, db.js & doc.js:
both have code like this:
if (!testing) {
const agent = new https.Agent({
keepAlive: true,
maxSockets: 50,
rejectUnauthorized: true
})
aws.config.update({
httpOptions: { agent }
})
}
The offending bit is this:
aws.config.update({
httpOptions: { agent }
})
If I remove the whole testing logic-branch (no custom agent) in both files, the issue goes away, but I haven't tested if the client still functions.
{
"devDependencies": {
"@architect/sandbox": "1.5.9-RC.2",
"jest": "^25.1.0"
},
"dependencies": {
"@architect/functions": "^3.5.0"
}
}
node 13.7.0
This is a follow up to this thread: https://architecture-as-text.slack.com/archives/C6BGT0D08/p1566800132296400
I wonder it would be possible to redirect
<https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging>
automagically to<https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging/>
? With the former loading the JS/CSS assets breaks as the paths need to be relative, it tries to load<https://mzix1huilh.execute-api.us-west-2.amazonaws.com/index.js>
instead of<https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging/index.js>
Is your feature request related to a problem? Please describe.
When referencing e.g. <script src="index.js"></script
in public/index.html
, then /index.js
is loaded instead of /staging/index.js
if the opened URL is https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging.
Describe the solution you'd like
https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging would ideally redirect to https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging/
Describe alternatives you've considered
Maybe try to redirect using history.pushState()
before loading any assets, but not sure if that will work, didn't have a chance to try
Describe the issue
Given logic like this
const querystring = require('querystring')
const qs = querystring.stringify({
ids = [1,2,3,4]
})
const res = await fetch(`${rootUrl}records/TEST?${qs}`, { method, headers })
// qs evalues to "ids=1&ids=2&ids=3&ids=4"
you would expect for the underlying lambda to process it the same way
Expected behavior
while using arc.http.asyc
I would expect the value of req.query
to be consistent
However.
In sandbox req.query.ids
is indeed the array of ids as supplied
but when rootUrl
is an actual API Gateway endpoint, req.query.ids
is not an array. It holds the value of the last element in the ids
array. I ended up needing to use req.multiValueQueryStringParameters
.
I think what makes sense is to make it so that req.multiValueQueryStringParameters.ids
is the only way to get these types of query strings and make sandbox behave this way too.
arc.http.helpers.url
is currently hard coded to add /production
but we changed the behavior to append /staging
so it fails
Howdy 👋
I ran into a couple of issues getting the arc-example-ws
repo working. I was able to get the example app working eventually, but my best guess is that the problems I ran into stem from this project.
src/ws/index.js
requires aws-sdk
, but it's declared as a dev dependency in @architect/functions, so it's not available if you just do a normal npm install
aws-sdk
across src/ws/*
AccessDeniedException
is thrown because the role assigned to the src/ws/*
lambda was missing the execute-api:ManageConnections
permission (full error message below)
~\code\clones\arc-example-ws $ npx logs src/ws/ws-default
app ~ test-ws
region ~ us-west-2
profile ~ xxx
version ~ 5.0.7
Feb 03, 05:36:41 PM
ws-default called with { requestContext:
{ routeKey: '$default',
authorizer: '',
messageId: 'UjSOXedLvHcCH1Q=',
integrationLatency: '',
eventType: 'MESSAGE',
error: '',
extendedRequestId: 'UjSOXFftvHcFiow=',
requestTime: '04/Feb/2019:01:36:40 +0000',
messageDirection: 'IN',
stage: 'staging',
connectedAt: 1549244197491,
requestTimeEpoch: 1549244200779,
identity:
{ cognitoIdentityPoolId: null,
accountId: null,
cognitoIdentityId: null,
caller: null,
sourceIp: '73.92.10.73',
accessKey: null,
cognitoAuthenticationType: null,
cognitoAuthenticationProvider: null,
userArn: null,
userAgent: null,
user: null },
requestId: 'UjSOXFftvHcFiow=',
domainName: 'kasodogei2.execute-api.us-west-2.amazonaws.com',
connectionId: 'UjSN2edEvHcCH1Q=',
apiId: 'kasodogei2',
status: '' },
body: '{"text":"hullo"}',
isBase64Encoded: false }
Feb 03, 05:36:41 PM
TypeError: aws.ApiGatewayManagementApi is not a constructor
Object.send (/var/task/node_modules/@architect/functions/src/ws/index.js:54:23)
ws (/var/task/index.js:15:23)
Feb 03, 05:36:41 PM
Billed Duration: 100 ms
Memory Size: 1152 MB
Max Memory Used: 38 MB
Feb 03, 05:55:44 PM
ws-default called with { requestContext:
{ routeKey: '$default',
authorizer: '',
messageId: 'UjVA_ed1PHcCEww=',
integrationLatency: '',
eventType: 'MESSAGE',
error: '',
extendedRequestId: 'UjVA_EL5PHcFeyg=',
requestTime: '04/Feb/2019:01:55:43 +0000',
messageDirection: 'IN',
stage: 'staging',
connectedAt: 1549245340677,
requestTimeEpoch: 1549245343908,
identity:
{ cognitoIdentityPoolId: null,
accountId: null,
cognitoIdentityId: null,
caller: null,
sourceIp: '73.92.10.73',
accessKey: null,
cognitoAuthenticationType: null,
cognitoAuthenticationProvider: null,
userArn: null,
userAgent: null,
user: null },
requestId: 'UjVA_EL5PHcFeyg=',
domainName: 'kasodogei2.execute-api.us-west-2.amazonaws.com',
connectionId: 'UjVAeedwPHcCEww=',
apiId: 'kasodogei2',
status: '' },
body: '{"text":"hullo"}',
isBase64Encoded: false }
Feb 03, 05:55:44 PM
AccessDeniedException: User: arn:aws:sts::226364732856:assumed-role/arc-role/test-ws-staging-ws-default is not authorized to perform: execute-api:ManageConnections on resource: arn:aws:execute-api:us-west-2:********2856:kasod
ogei2/staging/POST/@connections/{connectionId}
Object.extractError (/var/task/node_modules/aws-sdk/lib/protocol/json.js:51:27)
Request.extractError (/var/task/node_modules/aws-sdk/lib/protocol/rest_json.js:52:8)
Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:683:14)
Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)
AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)
/var/task/node_modules/aws-sdk/lib/state_machine.js:26:10
Request. (/var/task/node_modules/aws-sdk/lib/request.js:38:9)
Request. (/var/task/node_modules/aws-sdk/lib/request.js:685:12)
I did end up getting the example app working, but it took a little bit of debugging to get to hello world
.
Thanks! 🤗
When my default WebSocket Lambda is invoked, it attempts to send the received message to all other connected client:
const arc = require('@architect/functions');
exports.handler = async function handler(payload) {
const data = await arc.tables();
const { Items: clients } = await data.connections.scan({});
const { body: json, requestContext: { connectionId: self } } = payload;
const body = JSON.parse(json);
await Promise.all(clients.map(({ connectionId: id }) => {
if (id === self) {
return;
}
const message = { id, payload: body };
arc.ws.send(message);
}));
return { statusCode: 200 };
}
This works locally in Sandbox, but within AWS it throws an error:
AccessDeniedException: User: arn:aws:sts::<id>:assumed-role/<project>Staging-Role-<id>/<project>Staging-WebsocketDefault-<id> is not authorized to perform: execute-api:Invoke on resource: arn:aws:execute-api:eu-west-1:********9001:<id>/staging/POST/@connections/@connections/Bw91Ye3FjoECFKw%3D
(I've stripped some of the IDs out of that for obvious reasons).
Unsure if this is a functions
issue invoking the wrong thing, or deploy
/IAM related?
hit this while trying to set up a proxy to a spa. my get-index route:
let arc = require('@architect/functions');
exports.handler = arc.http.proxy({spa:true})
relevant section of the get-index route's package.json
:
grep -B 1 -A 1 functions package.json
"dependencies": {
"@architect/functions": "^3.2.2"
}
when i run the above in sandbox, i get the following stack:
internal/modules/cjs/loader.js:584
throw err;
^
Error: Cannot find module 'aws-sdk'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:582:15)
at Function.Module._load (internal/modules/cjs/loader.js:508:25)
at Module.require (internal/modules/cjs/loader.js:637:17)
at require (internal/modules/cjs/helpers.js:22:18)
at Object.<anonymous> (/Users/maj/src/arctonic/src/http/get-index/node_modules/@architect/functions/src/events/publish-old.js:3:11)
at Module._compile (internal/modules/cjs/loader.js:701:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
at Module.load (internal/modules/cjs/loader.js:600:32)
at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
at Function.Module._load (internal/modules/cjs/loader.js:531:3)
looks like src/events/publish-old.js
uses aws-sdk
. currently it is set as a dev dependency in this module's package.json
. probably should move it to a full dependency?
I was writing a style-loader here that converts css to js so that the can be imported from js modules. ie. import "styles.css"
and am using it like the following.
exports.handler = arc.proxy.public({
spa: true,
plugins: {
scss: ["@architect/proxy-plugin-sass", "@bvkimball/proxy-plugin-style-loader"],
},
but this means all css and scss files would be converted to js, which is not my intention. I was thinking if the plugin had access to the request.headers.Referrer
i would know what file request it and if it was js,jsx,mjs,etc... i would convert or not.
Alternatively i thought maybe loading the file with a query param, but that is not available in the Key value that is passed to the plugin.
eg.
import "styles.css?module"
You would need to pass the request here.
https://github.com/architect/functions/blob/master/src/http/proxy/public.js#L54
The guide at https://arc.codes/guides/background-tasks shows a callback-style @events handler. But as async @http handlers are supported, I assumed the callback example was just legacy. So I use an async @events handler:
export const handler = arc.events.subscribe(async record => {
console.log(record)
await doSth(record)
})
This works fine when deployed on AWS. However, today I noticed a problem with the sandbox when automating local system tests. The mocha tests would not terminate properly because a subprocess for src/sandbox/events/_subprocess.js
remained. Turns out, this piece of code never gets called: https://github.com/arc-repos/architect/blob/master/src/sandbox/events/_subprocess.js#L16-L20
I am guessing the invocation with (event, context, cb)
is off.
Are async handlers not fully supported or is this a bug in the sandbox?
Describe the bug
Using @architect/functions 2.0.7, arc.http.helpers.static
doesn't return the correct dir for Architect 5.5
`<script src="${static('/js/record-events.js')}"></script>`
when using Architect 5.0.x, correctly return:
http://localhost:3333/js/record-events.js
However when using Architect 5.5.x:
static
should return http://localhost:3333/_static/js/record-events.js (which is where the file now lives)
But instead it returns
http://localhost:3333/js/record-events.js
Which 404s.
In use, arc-functions
does not look for app.arc
, arc.yaml
, or arc.json
.
Lines: https://github.com/arc-repos/arc-functions/blob/master/src/http/helpers/_static.js#L4
sessions are currently hardcoded to 1 week / we should allow to config
Seems to be a logic issue with determining when to use local assets or not.
Wondering specifically about this line:
https://github.com/arc-repos/arc-functions/blob/master/src/http/helpers/_static.js#L13
Not sure why it is checking for arc.static
Seems like you should not have to define @static
if you are working locally.
My project specifies the following buckets:
@static
staging cultureamp-facewall-staging
production cultureamp-facewall-production
In staging, req._static('main.js')
produces this URL:
https://s3.amazonaws.com/cultureamp-facewall-staging/main.js
If you click this, you'll see S3 responds with a PermanentRedirect
XML response, which indicates the correct URL is:
https://cultureamp-facewall-staging.s3.amazonaws.com/main.js
When nosuchkey error on proxy.
Describe the issue
Using static
in Arc 6.0.0 sandbox fails on one of my machines:
Error
ENOENT: no such file or directory, open 'C:\Users\mikem\Code\temp\appless\src\http\get-index\node_modules\@architect\shared\.arc'0.
Error: ENOENT: no such file or directory, open 'C:\Users\mikem\Code\temp\appless\src\http\get-index\node_modules\@architect\shared\.arc'
at Object.openSync (fs.js:443:3)
at Object.readFileSync (fs.js:343:35)
at _static (C:\Users\mikem\Code\temp\appless\node_modules\@architect\functions\src\static\index.js:31:20)
at C:\Users\mikem\Code\temp\appless\src\http\get-index\node_modules\@architect\shared\layout.js:34:25
at Array.forEach ()
at layout (C:\Users\mikem\Code\temp\appless\src\http\get-index\node_modules\@architect\shared\layout.js:33:17)
at http (C:\Users\mikem\Code\temp\appless\src\http\get-index\index.js:23:11)
at process._tickCallback (internal/process/next_tick.js:68:7)
at evalScript (internal/bootstrap/node.js:592:13)
at startup (internal/bootstrap/node.js:265:9)
Versions:
"@architect/architect": "^6.0.0",
"@architect/functions": "^3.3.2",
"@architect/sandbox": "^1.3.6",
Also Windows 10, using arc.json
.
The error occurs in:
https://github.com/architect/functions/blob/master/src/static/index.js#L25
From Slack discussion, arc should no longer expect:
some-lambda/node_modules/@architect/shared/.arc
to exist, but this isn't true according to
https://github.com/architect/functions/blob/master/src/static/index.js#L12
static
, the issue goes away.some-lambda/node_modules/@architect/shared/.arc
, the issue goes away.Repro steps:
let arc = require('@architect/functions');
exports.handler = arc.proxy.public({
spa: true
})
Will be broken without bucket.staging
+ bucket.production
specified in the proxy config.
Describe the issue
Whereas
exports.handler = async req => ({ statusCode: 202 })
returns an empty response,
exports.handler = arc.http.async(async req => ({ statusCode: 202 }))
returns '\n' in the body (both in the sandbox and deployed). I think it might be coming from https://github.com/architect/functions/blob/master/src/http/_res-fmt.js#L17
This looks like a bug to me.
Expected behavior
I would still expect an empty response.
Additional context
@architect/[email protected]
Describe the issue
In this commit we removed the proxy config in favor of setting an environment variable (ARC_STATIC_BUCKET
), which Architect 6 does automatically.
This breaks Architect 5 users, and provides not an automatic means of setting the bucket without manually defining that env var.
Steps to reproduce
Steps to reproduce the behavior:
3.2.2
or later + Architect 5proxy
without ARC_STATIC_BUCKET
env var setExpected behavior
Previously documented and released config params should be respected.
We can also read the Architect project manifest and automatically populate ARC_STATIC_BUCKET
, but the user may want to specify a different bucket. (Unlikely, but possible.)
similar to what we provide in sandbox today: https://github.com/architect/sandbox/blob/master/src/http/index.js#L17-L29
In previous versions of architect we could get the table name using data._name('users')
. This appears to be removed in version 6.
The helper was great when we had to drop down and use AWS SDK's or needed finer control of the AWS operation, such as when wanting to add condition expressions etc.
How do I achieve the following in version 6?
const tableName = 'users';
const data = await arc.tables();
const tableName = data._name(tableName);
await data._doc.put({
TableName: tableName,
Item: {
emailAddress: '[email protected]'
},
ConditionExpression: 'attribute_not_exists(emailAddress)',
});
Per my in-person conversation with @brianleroux earlier today, we need to set up some deep integration tests on live AWS infra that exercise everything in Functions, with environments for:
sandbox
Describe the issue
Calling arc.events.publish returns 'topic not found' on topics after the first ten returned from SSM.
Steps to reproduce
I seems that ssm.getParametersByPath is limited to 10 results, which breaks in
function lookupTopics(callback) {
let ssm = new aws.SSM
let Path = `/${process.env.ARC_CLOUDFORMATION}`
ssm.getParametersByPath({Path, Recursive:true}, function done(err, result) {
if (err) callback(err)
else {
let topic = param=> param.Name.split('/')[2] === 'events'
let topics = result.Parameters.filter(topic).reduce((a, b)=> {
a[b.Name.split('/')[3]] = b.Value
return a
}, {})
callback(null, result.Parameters)
}
})
}
More of question than an issue.
I noticed on arc.codes we always prefix the session methods with await
. Looking at the code in this repo, though, neither the old session code nor the new provider-based code leverage async
functions.
So my question is: is it necessary? If no, why use it in the docs? Hedging your bet against future changes? What is the impact of using await
with a synchronous function?
https://github.com/arc-repos/arc-functions/blob/master/src/http/helpers/_url.js#L7
I don't think req._url
is available anymore.
Describe the bug
Per Twitter discussion between @brianleroux and I: currently to respond to a websocket, we do:
arc.ws().send()
arc.ws()
takes a connectionId
and an apiId
, and send()
takes a connectionId
It would be better to:
connectionId
onceapiId
if possibleSomething like arc.ws.send(connectionId, payload)
would be great. This means we could store connectionId in DynamoDB / DocumentDB and use it to respond to websockets from outside the /src/ws
- for example we could have an HTTP POST (called from an admin panel via /src/https/post-xxx-xxx-xxx
) that sends a particular message to a subset of connected users via websockets.
B seemed to like this, so filing it here!
I might be able to get rid of that param (
apiId
)
By looking it up once and caching
We do that for events and queues
Hmm, ya we should
xnoɹǝʃ uɐıɹq 🐻
(thx for talking thru this w me)
When using the put
function to add an item to dynamodb, it would be useful to be able to add parameters to the request (e.g. ConditionExpression etc.). This would allow us to use put in a manner that does not overwrite a previous item if one already exists in the table.
var params = {
Item: {
partitionKey: partitionKey,
sortKey: sortKey,
data: data
},
ConditionExpression: 'attribute_not_exists(partitionKey) AND attribute_not_exists(sortKey)'
};
This would stop us having to do the following (which consumes RCUs) when inserting new items:
let data = await arc.tables();
const exists = await data.users.get({ _id: id })
if (exists) {
throw new Error(`User ${model._id} already exists.`);
}
await data.users.put(model);
remove all calls to aws-sdk
and add these to the deps
arc.http
https://www.npmjs.com/package/@aws-sdk/client-s3-nodearc.events
https://www.npmjs.com/package/@aws-sdk/client-sns-nodearc.queues
https://www.npmjs.com/package/@aws-sdk/client-sqs-nodearc.ws
https://www.npmjs.com/package/@aws-sdk/client-apigatewaymanagementapi-node and maybe https://www.npmjs.com/package/@aws-sdk/client-apigatewayv2-nodearc.tables
https://www.npmjs.com/package/@aws-sdk/client-dynamodb-v2-nodeAdditionally we need to replace calls to SSM with
Say you have the following file saved to S3: /foo/index.html
If you'd like to read that file out via the Functions proxy, you have to access it as either:
/foo/index.html
, or/foo/
(note trailing slash)I'd like to implement path peeking, so that a request for /foo
would:
/foo
/foo
is not found, then it should look for /foo/index.html
/foo
nor /foo/index.html
are round, only then 404Describe the issue
We recently started updating our arc projects to use the macro-http-api
macro which pulls in the proxy function from this repo and uses it to serve static assets. Our first upgrades we're successful, but later upgrades would not load static assets due to a change in this repo. It would fail with a "Key not Found" error.
After a deep dive through working / non working lambdas we found that a line similar to if (Key.startsWith('_static/')) Key = Key.replace('_static/', '')
had been removed from the code causing the keys from the /public
folder to be malformed.
Steps to reproduce
Steps to reproduce the behavior:
.arc
project./public/anything.css
macro-http-api
.arc.static
helper./_static/anything.css
- but will 404 with a "Key not found" error.Expected behavior
I expect the asset to load properly.
Screenshots
NA
Additional context
None.
Is your feature request related to a problem? Please describe.
Currently when we proxy / read blobs out of S3, we set anti-cache headers for HTML and JSON; everything else, unless specified, gets a 1 day cache.
This bites many users whose content gets cached in CloudFront and can't be easily purged. This isn't only new users – this has happened to me!
Describe the solution you'd like
We have support for 304/etag but aren't doing enough with it right now. We should more aggressively utilize etag as the default, or prioritize other means of ensuring content updates are good.
With static env vars, arc.http.helpers.static()
craches with:
{
"errorMessage": "Cannot read property 'forEach' of undefined",
"errorType": "TypeError",
"stackTrace": [
"getBucket (/var/task/node_modules/@architect/functions/src/http/helpers/static.js:32:10)",
"Object._static [as static] (/var/task/node_modules/@architect/functions/src/http/helpers/static.js:20:16)",
"http (/var/task/index.js:90:103)"
]
}
https://github.com/architect/arc-functions/blob/master/src/http/helpers/static.js#L20
Just curious about code style, any particular reason why you've used var
and no let
or const
?
Edit: found some let
usage in /index.js
;)
Describe the issue
Static helper (arc.static
) does not necessarily return the correct path if being used with a bare, non-DNS/CF'd API gateway. See: https://github.com/architect/functions/blob/master/src/static/index.js#L16-L18
Steps to reproduce
Steps to reproduce the behavior:
/staging
path suffixExpected behavior
The static helper should present the correct path prefix for a bare APIG, or accept options to allow that to be specified/altered.
Thanks, @jessehattabaugh!
need to look into this! avoids having to upgrade the aws-sdk in the lambdas
Following on from #33 I've been testing Arc Functions v4.
Using 6c43f8bc70190f8ede03791ebcc3eca40864bfbf
from git, await arc.ws.send(connectionID, payload)
will break a lambda in the sandbox
No logging, including console.log()
before the arc.ws.send()
, will be shown if the arc.ws.send()
is enabled
Inspecting the arc.ws.send()
code it looks like the promise is missing from arc.ws.send()
, so wrapping it in a promisify:
let arc = require('@architect/functions'),
makePayload = require('@architect/shared/make-payload'),
util = require("util");
const webSocketSend = util.promisify(arc.ws.send);
require('@architect/shared/globals')
exports.handler = async function webSocketConnect(event) {
let connectionID = event.requestContext.connectionId
log(`Sending chat default greeting to new client at connection ID: ${connectionID}`)
var payload = makePayload(`Hey there whats up!`, 'Mike')
// BUG - if uncommented this will never return
// await webSocketSend(connectionID, payload)
log(`ws connect finished`)
return {
status: OK
}
}
(Obviously I've tried it without the promisify too)
How can I make arc.ws.send()
work in the sandbox?
Great project - I've hit an issue that I'm struggling to resolve - I'm having difficulty setting the status
param of the response function.
res({
status: 403,
json: { msg: 'unauthorised'}
})
This results in a JSON parse error SyntaxError: Unexpected token o in JSON at position 1
in my terminal
If I remove the status param, the response correctly returns the JSON payload {msg: "unauthorised"}
when the request object doesn't include a valid authorization header.
I'm not sure if I've misunderstood the documentation, or if there is fact an issue, my full route below.
Thanks!
var arc = require('@architect/functions')
function route(req, res) {
console.log(JSON.stringify(req, null, 2))
res({
json: {hello:'world'}
});
}
function check(req, res, next) {
if (req.headers.authorization == 'xxxx') {
next()
}
else {
res({
status: 403,
json: { msg: 'unauthorised'}
})
}
}
exports.handler = arc.json.post(check, route)
Describe the issue
Architect v5 projects using arc.events.publish
may unexpectedly fail in the following conditions:
In this block events.publish
loops through all topics until it finds what it's looking for, then fires the event; however, by default SNS listTopics
can only run 30tps, so enough loops through at the same time and your function will unexpectedly error.
Is your feature request related to a problem? Please describe.
arc.http
(ie callback
style) supports forward compatibility to Architect 6, but not the arc.http.middleware
(ie async/await
style) path. We should add this forwards-compatibility.
Describe the solution you'd like
Likely it'll just be a pure function to remap params right here: https://github.com/architect/functions/blob/master/src/http/middleware/index.js#L30
I'm guessing we'll just key on process.env.ARC_CLOUDFORMATION
.
According to the docs, calling res
with an Error
object lets you return a 404 error if you set the code
property of the error object. I'm attempting to do this as follows in a @json
route handler:
let arc = require("@architect/functions");
function route(req, res) {
const error = Error("Record not found");
error.code = 404;
res(error);
}
exports.handler = arc.json.get(route);
With npx sandbox
, requesting this route crashes the server with an error:
/Users/kyank/code/project/node_modules/aws-sdk/lib/request.js:31
throw err;
^
Error: Record not found
at Object.route (/Users/kyank/code/project/src/json/get-record-000recordId/index.js:22:19)
at _iter (/Users/kyank/code/project/src/json/get-surveys-000surveyId/node_modules/@architect/functions/src/http/_request.js:73:12)
at _read (/Users/kyank/code/project/src/json/get-surveys-000surveyId/node_modules/@architect/functions/src/http/_request.js:74:9)
at _find (/Users/kyank/code/project/src/json/get-surveys-000surveyId/node_modules/@architect/functions/src/http/session/read.js:37:5)
at Response._get (/Users/kyank/code/project/src/json/get-surveys-000surveyId/node_modules/@architect/functions/src/http/session/_find.js:17:9)
at Request.<anonymous> (/Users/kyank/code/project/node_modules/aws-sdk/lib/request.js:364:18)
at Request.callListeners (/Users/kyank/code/project/node_modules/aws-sdk/lib/sequential_executor.js:109:20)
at Request.emit (/Users/kyank/code/project/node_modules/aws-sdk/lib/sequential_executor.js:81:10)
at Request.emit (/Users/kyank/code/project/node_modules/aws-sdk/lib/request.js:683:14)
at Request.transition (/Users/kyank/code/project/node_modules/aws-sdk/lib/request.js:22:10)
With the project deployed to AWS, requesting this route returns a 500 error, and logs this in CloudWatch:
{
"errorMessage": "{\"statusCode\":404,\"json\":{\"message\":\"Record not found\",\"code\":404,\"time\":{},\"name\":\"Error\",\"stack\":\"Error: Record not found\\n at Object.route (/var/task/index.js:22:19)\\n at _iter (/var/task/node_modules/@architect/functions/src/http/_request.js:73:12)\\n at _read (/var/task/node_modules/@architect/functions/src/http/_request.js:74:9)\\n at _find (/var/task/node_modules/@architect/functions/src/http/session/read.js:37:5)\\n at Response._get (/var/task/node_modules/@architect/functions/src/http/session/_find.js:17:9)\\n at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:364:18)\\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)\\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)\\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)\\n at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)\"}}"
}
Presently we support body parsing (among other things) keying on application/json
, but not application/vnd.api+json
. We should add that.
/ht @jkarsrud!
Describe the issue / Steps to reproduce
If the sandbox is launched where NODE_ENV
is set to something unexpected (in my case, I used tunnelled
, arc.static()` will return non-usable URLs.
This happened when I hapepned to reuse a terminal from another node app.
The issue comes from:
https://github.com/architect/functions/blob/master/src/static/index.js#L22
This defines
let runningLocally = !env || env === 'testing' || process.env.ARC_LOCAL
Which fails in the case where NODE_ENV
exists but isn't a known value. Other parts of arc, like arc.http.helpers.url
use different parsing logic, looking for explicit strings then falling back.
Ideally, have one function only that is called from anywhere that wants to look up the env, fixing issues like this (also allowing changes to be made more rapidly because one place needs to be modified, rather than many.) This should be like arc.http.helpers.url
use different parsing logic, looking for explicit strings then falling back.
few use or like this interface so low priority
Describe the issue
A clear and concise description of the bug
Steps to reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen
Screenshots
If applicable, add screenshots to help explain your problem
Desktop
Please complete the following information (if appropriate):
Mobile
Please complete the following information (if appropriate):
Additional context
Add any other context or notes about the problem here
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.