We should (optionally) be smarter for URL types and parse them into url crate structs rather than keep them as strings.

The authentication field in ApiGatewayV2httpRequestContext should be optional since the field is not present for public APIs.

In aws/aws-lambda-go@ee817dd codepipeline is only an alias. We should either blacklist it or correctly parse / handle the alias.

We should generate a variant that borrows rather than owns (so &str rather than String for example). This will reduce allocations.

If a batch of SQS messages cannot be processed a lambda may return an SQSBatchResponse with batchItemFailures. Only the message_ids in batchItemFailures get put back on the originating queue. This avoids reprocessing. This lib should probably include this. There was an issue for this in the .NET equivelant library, and the change looks like this.

I can do a PR for this if you like? Or I was wondering if you'd rather wait for the Go code to be updated and pull it in from there?

Hello,

I have configured a private rest API and tried to send a body from AWS Console without setting headers.

{
    "a":1
}

The error is:
Error: Error("invalid type: null, expected lots of things can go wrong with HeaderMap", line: 1, column: 85)

The code:

use lambda_runtime::{handler_fn, Context, Error};
use aws_lambda_events::event::apigw::ApiGatewayProxyRequest;

#[tokio::main]
async fn main() -> Result<(), Error> {
    lambda_runtime::run(handler_fn(|event: ApiGatewayProxyRequest, ctx: Context| {
        execute(event, ctx)
    }))
    .await?;
    Ok(())
}

pub async fn execute(event: ApiGatewayProxyRequest, _ctx: Context) -> Result<Value, Error> {
    println!("EVENT {:?}", event);
    Ok(json!({
        "statusCode": 201
    }))
}

The input that is coming in is:

{
   "body": "{\r\n\t\"a\": 1\r\n}",
   "headers":null,
   "httpMethod":"POST",
   "isBase64Encoded":false,
   "multiValueHeaders":null,
   "multiValueQueryStringParameters":null,
   "path":"/myPath",
   "pathParameters":null,
   "queryStringParameters":null,
   "requestContext":{
      "accountId":"xxxxx",
      "apiId":"xxxxx",
      "domainName":"testPrefix.testDomainName",
      "domainPrefix":"testPrefix",
      "extendedRequestId":"NvWWKEZbliAFliA=",
      "httpMethod":"POST",
      "identity":{
         "accessKey":"xxxxx",
         "accountId":"xxxxx",
         "apiKey":"test-invoke-api-key",
         "apiKeyId":"test-invoke-api-key-id",
         "caller":"xxxxx:xxxxx",
         "cognitoAuthenticationProvider":null,
         "cognitoAuthenticationType":null,
         "cognitoIdentityId":null,
         "cognitoIdentityPoolId":null,
         "principalOrgId":null,
         "sourceIp":"test-invoke-source-ip",
         "user":"xxxxx:xxxxx",
         "userAgent":"aws-internal/3 aws-sdk-java/1.12.154 Linux/5.4.156-94.273.amzn2int.x86_64 OpenJDK_64-Bit_Server_VM/25.322-b06 java/1.8.0_322 vendor/Oracle_Corporation cfg/retry-mode/standard",
         "userArn":"arn:aws:sts::xxxxx:assumed-role/xxxxx/xxxxx"
      },
      "path":"/myPath",
      "protocol":"HTTP/1.1",
      "requestId":"e5488776-afe4-4e5e-92b1-37bd23f234d6",
      "requestTime":"18/Feb/2022:13:23:12 +0000",
      "requestTimeEpoch":1645190592806,
      "resourceId":"ddw8yd",
      "resourcePath":"/myPath",
      "stage":"test-invoke-stage"
   },
   "resource":"/myPath",
   "stageVariables":null
}

Not sure if this can be considered an issue, but from APIGW POST - Method Test, it is not mandatory to set Headers, so not sure if null can handle it.

Dan

Currently getting the following error when trying to use ApiGatewayWebsocketProxyRequest on a lambda set as an integration target for an AWS API Gateway Websocket API without an authoirzer configured.

Error("missing field `authorizer`", line: 1, column: 1246)'

PR #26 solved the issue on my end

There is a bunch of DynamoDB-specific code here:

https://github.com/aws/aws-lambda-go/blob/e8b9e003487a29d8e5b8cc4ee6a051dc4e36124c/events/attributevalue.go

We should probably just manually port it to Rust as it looks like a one-off.

Kafka headers are a byte array, not a base64 encoded string.

Why not converting only null to None and leaving empty strings empty? The problem is loosing the information about whether the value was actually null or an empty string, neither of string-null-empty nor string-null-none preserve this knowledge

Take this simple Lambda:
https://serverlessland.com/patterns/sns-sqs-lambda-sam-rust (the code is not up to date but 8 months ago it was working)

If you update the code and the libs

use aws_lambda_events::event::{sns::SnsMessage, sqs::SqsEvent};
use futures::future::join_all;
use lambda_runtime::{service_fn, Error, LambdaEvent};
use serde::Deserialize;

#[tokio::main]
async fn main() -> Result<(), Error> {
    tracing_subscriber::fmt()
        .with_ansi(false)
        .without_time()
        .init();

    lambda_runtime::run(service_fn(|event: LambdaEvent<SqsEvent>| execute(event))).await?;

    Ok(())
}

pub async fn execute(event: LambdaEvent<SqsEvent>) -> Result<(), Error> {
    println!("Input {:?}", event);

    let mut tasks = Vec::with_capacity(event.payload.records.len());
    for record in event.payload.records.into_iter() {
        tasks.push(tokio::spawn(async move {
            if let Some(body) = &record.body {
                let sns_message = serde_json::from_str::<SnsMessage>(&body).unwrap();
                let sns_message = sns_message.message;
                let request = serde_json::from_str::<MyStruct>(&sns_message);
                if let Ok(request) = request {
                    // Do something
                    println!("Request {:?}", request);
                }
            }
        }));
    }

    join_all(tasks).await;

    Ok(())
}

#[derive(Deserialize, Debug, Clone, PartialEq, Eq)]
pub struct MyStruct {
    pub name: String,
    pub surname: String,
}

cargo.toml

[package]
name = "sns-sqs-lambda-rust"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[[bin]]
name = "handler"
path = "src/bin/handler.rs"

[dependencies]
aws_lambda_events = { version = "0.7.2", default-features = false, features = ["sqs", "sns"] }
futures = "0.3.17"
lambda_runtime = "0.7.2"
serde = {version = "1.0", features = ["derive"] }
serde_json = "1.0.68"
simple-error = "0.2"
tokio = { version = "1", features = ["full"] }
tracing-subscriber = "0.3"

You get an exception:
`Input LambdaEvent { payload: SqsEvent { records: [SqsMessage { message_id: Some("44dec5ec-0128-4661-9c54-a06485ead572"), receipt_handle: Some(".....="), body: Some("{\n "Type" : "Notification",\n "MessageId" : "5a1c0085-ee71-5e38-9d84-5c9bdcb02ad5",\n "TopicArn" : "arn:aws:sns:eu-central-1:xxxxx:sam-app-MySnsTopic-BWiabVkXTYg2",\n "Message" : "{\n \"name\": \"Daniele\",\n \"surname\": \"Frasca\"\n}",\n "Timestamp" : "2022-12-22T10:25:05.377Z",\n "SignatureVersion" : "1",\n "Signature" : ".....==",\n "SigningCertURL" : "https://sns.eu-central-1.amazonaws.com/SimpleNotificationService-56e67fcb41f6fec09b0196692625d385.pem",\n "UnsubscribeURL" : "https://sns.eu-central-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:eu-central-1:xxxxx:sam-app-MySnsTopic-BWiabVkXTYg2:44fb3b0d-6b6a-48cc-9183-f52a3d238175"\n}"), md5_of_body: Some("ab0cac5be52987991d1d1b2b57a08480"), md5_of_message_attributes: None, attributes: {"SentTimestamp": "1671704705405", "SenderId": "AIDAISDDSWNBEXIA6J64K", "ApproximateFirstReceiveTimestamp": "1671704705410", "ApproximateReceiveCount": "1"}, message_attributes: {}, event_source_arn: Some("arn:aws:sqs:eu-central-1:xxxxx:sam-app-MySqsQueue-nDLzmzS8vEaH"), event_source: Some("aws:sqs"), aws_region: Some("eu-central-1") }] }, context: Context { request_id: "d187ab64-f92b-52a2-a81b-f77acb9cde03", deadline: 1671704735726, invoked_function_arn: "arn:aws:lambda:eu-central-1:xxxxxx:function:sam-app-MyFunction-QqyqP0mMMxOS", xray_trace_id: Some("Root=1-63a43081-22df4f205924b3de2b0c3b46;Parent=54651cb434f944d8;Sampled=0"), client_context: None, identity: None, env_config: Config { function_name: "sam-app-MyFunction-QqyqP0mMMxOS", memory: 128, version: "$LATEST", log_stream: "2022/12/22/[$LATEST]7da35658a61a4b0cb323242f7f20526a", log_group: "/aws/lambda/sam-app-MyFunction-QqyqP0mMMxOS" } } }

thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("missing field SigningCertUrl", line: 11, column: 1)', src/bin/handler.rs:29:77

`

Please let me know if it is not clear.

I'm trying to upgrade some of our projects from 0.6 to 0.7.

It seems like aws_lambda_events::s3::S3Event is missing.

https://docs.rs/aws_lambda_events/0.7.0/aws_lambda_events/index.html?search=S3Event

vs

https://docs.rs/aws_lambda_events/0.6.3/aws_lambda_events/index.html?search=S3Event

When autocompleting in an IDE, aws_lambda_events::event::s3:: gives me exactly the same thing as aws_lambda_events::event::, so maybe there's some recursive definition going on or something.

I'm looking for a way to deserialize the HashMap<String, AttributeValue> in the StreamRecord.newImage and StreamRecord.oldImage to a struct. Any suggestions on a good approach? Is there e.g. some way I can utilize serde?

I'm getting this error in my AWS Lambda, being used in a AWS Connect Contact Flow:

invalid type: null, expected struct ConnectQueue

This happens during the deserialization of the LambdaEvent, whose function signature is:

async fn function_handler(event: LambdaEvent<ConnectEvent>) -> Result<Value, Error> {}

I don't know how can I log the event in this kind of error, to log the request, otherwise I would attach the request json. However it seems quite clear that the reason is that the LambdaEvent is getting a null queue attribute here https://github.com/LegNeato/aws-lambda-events/blob/master/aws_lambda_events/src/generated/connect.rs#L63 . I am not using any connect que in my AWS Connect Contact Flow, because it's entirely automated with no agents involved. I'm not sure if I'm doing that wrong, or it is just a bug.

I suggest:

1. Please update the generated ConnectEvent to allow a null value in the queue attribute if it's a bug. Probably needs to be updated in the upstream go source.
2. If it's not a bug and I'm doing something wrong.. what is it?

Many structs can be used part of the regular function code. It's very cumbersome to create them by hand if you only need to add a few fields, and want to leave the rest as defaults. We should include Default as part of the derivations, or implement it when possible, so it's easier to work with events manually.

A couple of examples that could be simplified:

Ok(ApiGatewayV2httpResponse {
        status_code: 200,
        headers: HeaderMap::new(),
        multi_value_headers: HeaderMap::new(),
        body: Some(Body::Text(response)),
        is_base64_encoded: Some(false),
        cookies: vec![],
    })

https://github.com/cargo-lambda/cargo-lambda/blob/main/crates/cargo-lambda-watch/src/trigger_router.rs#L98-L132

I'm trying to create a custom ApiGatewayV2 authorizer lambda for a WebSocket API using ApiGatewayV2CustomAuthorizerV1Request or ApiGatewayV2CustomAuthorizerV2Request, but the lambda fails to run in both cases.
It runs the main() function with no issues, but code inside the handler_fn is being completely ignored, and no error whatsoever is being thrown. I've tried using other types, such as ApiGatewayCustomAuthorizerRequest, and they work just fine, but they're incompatible with the API Gateway version and return incorrect/missing data.

Sometimes it outputs an "Unknown.Runtime" error message directly in the lambda logs.

I'm using lambda_runtime and aws_lambda_events, both v0.7.0.
I've tried v0.7.1 and v0.7.2 and had the exact same result.

Here's the code:

use aws_lambda_events::apigw::ApiGatewayV2CustomAuthorizerV2Request;
use lambda_runtime::{service_fn, Error, LambdaEvent};

#[tokio::main]
async fn main() -> Result<(), Error> {
    println!("Running main function");

    let handler = service_fn(|event| handler_fn(event));

    lambda_runtime::run(handler).await?;

    println!("Finished running");

    Ok(())
}

async fn handler_fn(
    event: LambdaEvent<ApiGatewayV2CustomAuthorizerV2Request>,
) -> Result<ApiGatewayV2CustomAuthorizerV2Request, Error> {
    println!("Query params:");
    println!("{:#?}", event.payload.query_string_parameters);

    Ok(event.payload)
}

Here are the lambda logs:

2023-01-05T20:53:34.722-03:00 | Running main function
2023-01-05T20:53:34.729-03:00 | START RequestId: e320b0aa-6596-4396-abb5-2989ebfad615 Version: $LATEST
2023-01-05T20:53:34.731-03:00 | END RequestId: e320b0aa-6596-4396-abb5-2989ebfad615
2023-01-05T20:53:34.731-03:00 | REPORT RequestId: e320b0aa-6596-4396-abb5-2989ebfad615 Duration: 1.57 ms Billed

And here's my lambda configuration:

Also, what's the difference between ApiGatewayV2CustomAuthorizerV1Request and ApiGatewayV2CustomAuthorizerV2Request?

Thanks!

Hello! Thank you for this wonderful crate!

I'm using this crate in a Lambda that processes DynamoDB stream events coming from Kinesis.

I can't deserialize the events to aws_lambda_events::dynamodb::EventRecord because the eventVersion field is missing from them. The aws_lambda_events::dynamodb::EventRecord struct has the event_version field as being of type String rather than of type Option<String>. I'm not sure if this is a bug or if I'm just using the wrong type.

I've been able to get around this by duplicating the type, changing the type of the field to Option<String>, and deriving serde::Deserialize on it.

Here is the base64 decoding of the problematic EventRecord. Note the lack of eventVersion field.

{
  "awsRegion":"eu-west-1",
  "eventID":"00000000-0000-0000-0000-000000000000",
  "eventName":"INSERT",
  "userIdentity":null,
  "recordFormat":"application/json",
  "tableName":"examples",
  "dynamodb":{
    "ApproximateCreationDateTime":1649809356015,
    "Keys":{
      "id":{
        "S":"00000000-0000-0000-0000-000000000000"
      }
    },
    "NewImage":{
      "id":{
        "S":"00000000-0000-0000-0000-000000000000"
      },
      "created":{
        "S":"2022-02-16T15:12:00.14Z"
      }
    },
    "SizeBytes":292
  },
  "eventSource":"aws:dynamodb"
}

Here is what the above looks like as the data field of a lambda_runtime::LambdaEvent<aws_lambda_events::event::kinesis::KinesisEvent>.

{
  "Records":[
    {
      "kinesis":{
        "kinesisSchemaVersion":"1.0",
        "partitionKey":"00000000000000000000000000000000",
        "sequenceNumber":"49627941615570055579864951581810255285859251490098511874",
        "data":"eyJhd3NSZWdpb24iOiJldS13ZXN0LTEiLCJldmVudElEIjoiMDAwMDAwMDAtMDAwMC0wMDAwLTAwMDAtMDAwMDAwMDAwMDAwIiwiZXZlbnROYW1lIjoiSU5TRVJUIiwidXNlcklkZW50aXR5IjpudWxsLCJyZWNvcmRGb3JtYXQiOiJhcHBsaWNhdGlvbi9qc29uIiwidGFibGVOYW1lIjoiZXhhbXBsZXMiLCJkeW5hbW9kYiI6eyJBcHByb3hpbWF0ZUNyZWF0aW9uRGF0ZVRpbWUiOjE2NDk4MDkzNTYwMTUsIktleXMiOnsiaWQiOnsiUyI6IjAwMDAwMDAwLTAwMDAtMDAwMC0wMDAwLTAwMDAwMDAwMDAwMCJ9fSwiTmV3SW1hZ2UiOnsiaWQiOnsiUyI6IjAwMDAwMDAwLTAwMDAtMDAwMC0wMDAwLTAwMDAwMDAwMDAwMCJ9LCJjcmVhdGVkIjp7IlMiOiIyMDIyLTAyLTE2VDE1OjEyOjAwLjE0WiJ9fSwiU2l6ZUJ5dGVzIjoyOTJ9LCJldmVudFNvdXJjZSI6ImF3czpkeW5hbW9kYiJ9",
        "approximateArrivalTimestamp":1648225669.343
      },
      "eventSource":"aws:kinesis",
      "eventVersion":"1.0",
      "eventID":"shardId-000000000000:49627941615570055579864951581810255285859251490098511874",
      "eventName":"aws:kinesis:record",
      "invokeIdentityArn":"arn:aws:iam::111111111111:role/service-role/body-role-00000000",
      "awsRegion":"eu-west-1",
      "eventSourceARN":"arn:aws:kinesis:eu-west-1:000000000000:stream/examples_changes"
    }
  ]
}

I am using localstack. If no authorization (authorizationType = NONE) is configured for the lambda proxy integration with API gateway, the authorizer field is omitted in the lambda event body.

Currently this is required, causing serde to fail.
https://github.com/LegNeato/aws-lambda-events/blob/master/aws_lambda_events/src/generated/apigw.rs#L121

Error: data did not match any variant of untagged enum LambdaRequest

This is an example lambda event body:

{
  "path": "/",
  "headers": {
    "remote-addr": "172.17.0.1",
    "host": "localhost:4566",
    "connection": "keep-alive",
    "upgrade-insecure-requests": "1",
    "user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36",
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "sec-gpc": "1",
    "sec-fetch-site": "none",
    "sec-fetch-mode": "navigate",
    "sec-fetch-user": "?1",
    "sec-fetch-dest": "document",
    "accept-encoding": "gzip, deflate, br",
    "accept-language": "en-US,en;q=0.9",
    "x-forwarded-for": "172.17.0.1, localhost:4566, 127.0.0.1, localhost:4566",
    "x-localstack-edge": "http://localhost:4566",
    "authorization": "AWS4-HMAC-SHA256 Credential=__internal_call__/20160623/us-east-1/apigateway/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=1234",
    "x-localstack-tgt-api": "apigateway",
    "__apigw_request_region__": "us-east-1"
  },
  "multiValueHeaders": {
    "remote-addr": ["172.17.0.1"],
    "host": ["localhost:4566"],
    "connection": ["keep-alive"],
    "upgrade-insecure-requests": ["1"],
    "user-agent": [
      "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36"
    ],
    "accept": [
      "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
    ],
    "sec-gpc": ["1"],
    "sec-fetch-site": ["none"],
    "sec-fetch-mode": ["navigate"],
    "sec-fetch-user": ["?1"],
    "sec-fetch-dest": ["document"],
    "accept-encoding": ["gzip, deflate, br"],
    "accept-language": ["en-US,en;q=0.9"],
    "x-forwarded-for": [
      "172.17.0.1, localhost:4566, 127.0.0.1, localhost:4566"
    ],
    "x-localstack-edge": ["http://localhost:4566"],
    "authorization": [
      "AWS4-HMAC-SHA256 Credential=__internal_call__/20160623/us-east-1/apigateway/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=1234"
    ],
    "x-localstack-tgt-api": ["apigateway"],
    "__apigw_request_region__": ["us-east-1"]
  },
  "body": "",
  "isBase64Encoded": false,
  "httpMethod": "GET",
  "queryStringParameters": {},
  "multiValueQueryStringParameters": {},
  "pathParameters": {},
  "resource": "/",
  "requestContext": {
    "path": "/prod/",
    "resourcePath": "/",
    "apiId": "qw9nx3npwz",
    "domainPrefix": "qw9nx3npwz",
    "domainName": "qw9nx3npwz.execute-api.localhost.localstack.cloud",
    "accountId": "000000000000",
    "resourceId": "9htszz4dh3",
    "stage": "prod",
    "identity": {
      "accountId": "000000000000",
      "sourceIp": "127.0.0.1",
      "userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36"
    },
    "httpMethod": "GET",
    "protocol": "HTTP/1.1",
    "requestTime": "2021-11-01T15:45:14.341Z",
    "requestTimeEpoch": 1635781514341
  },
  "stageVariables": {}
}

This issue is similar to other issues about optional fields:
#27
#25

Hi,

Can you point me to an example of how to use this crate with LogEntry event? I'm trying to write a lambda that uses this event when a new log is sent to a log stream however it isn't clear to me on how to use this crate for this purpose. I'm new to Rust so this also doesn't help.

The error I'm getting from the compiler is the following:

error[E0277]: the trait bound `fn(LogEntry) -> Result<(), Box<(dyn StdError + Send + Sync + 'static)>> {function_handler}: Service<LambdaEvent<_>>` is not satisfied
   --> src/main.rs:18:9
    |
18  |     run(function_handler).await?;
    |     --- ^^^^^^^^^^^^^^^^ the trait `Service<LambdaEvent<_>>` is not implemented for fn item `fn(LogEntry) -> Result<(), Box<(dyn StdError + Send + Sync + 'static)>> {function_handler}`
    |     |
    |     required by a bound introduced by this call
    |
    = help: the following other types implement trait `Service<Request>`:
              <&'a mut S as Service<Request>>
              <&hyper::client::client::Client<C, B> as Service<Request<B>>>
              <AndThen<S, F> as Service<Request>>
              <Box<S> as Service<Request>>
              <BoxCloneService<T, U, E> as Service<T>>
              <BoxService<T, U, E> as Service<T>>
              <Either<A, B> as Service<Request>>
              <FutureService<F, S> as Service<R>>
            and 16 others
note: required by a bound in `run`
   --> /Users/stefano/.cargo/registry/src/github.com-1ecc6299db9ec823/lambda_runtime-0.7.0/src/lib.rs:231:8
    |
231 |     F: Service<LambdaEvent<A>>,
    |        ^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `run`

This is where I'm currently at.

use aws_lambda_events::{cloudwatch_logs, event::cloudwatch_logs::LogEntry};
use lambda_runtime::{run, service_fn, Error, LambdaEvent};

#[tokio::main]
async fn function_handler(event: LogEntry) -> Result<(), Error> {
    Ok(())
}

#[tokio::main]
async fn main() -> Result<(), Error> {
    tracing_subscriber::fmt()
        .with_max_level(tracing::Level::INFO)
        .with_target(false)
        .without_time()
        .init();

    let func = service_fn(function_handler);
    lambda_runtime::run(function_handler).await?;
    Ok(())
}

Thanks in advance!

Hi there!

I recently updated my ALB configuration, enabling the multi-header feature. This move introduced a breaking change, preventing me to handle ALB Requests. The root cause, if my diagnose is correct, lies in the fact that, with multi-header enabled in ALB, the header entry is empty and might've been removed during the serialization process. As a result, serde tries to deserialize the received bytes into an AlbTargetGroupRequest instance but fails as header is mandatory and has no default implementation available.

Here you can find the raw JSON that was received by the Lambda Runtime.

I know that the AlbTargetGroupRequest is generated based on the Go SDK, but for the sake of the test, I've manually patched it and it worked as expected.

Problem

I am unable to use the ApiGatewayWebsocketProxyRequest, and therefore, ApiGatewayWebsocketProxyRequestContext, to serialize WebSocket events from API Gateway because the field http_method does not exist on the emitted event.

Actual Received Event

{
    "headers": {
        "Accept-Encoding": "gzip, deflate, br",
        "Accept-Language": "en-US,en;q=0.9",
        "Cache-Control": "no-cache",
        "Host": "/* host */",
        "Origin": "http://localhost:3000",
        "Pragma": "no-cache",
        "Sec-WebSocket-Extensions": "permessage-deflate; client_max_window_bits",
        "Sec-WebSocket-Key": "/* key */",
        "Sec-WebSocket-Version": "13",
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67",
        "X-Amzn-Trace-Id": "/* id */",
        "X-Forwarded-For": "/* ip */",
        "X-Forwarded-Port": "443",
        "X-Forwarded-Proto": "https"
    },
    "isBase64Encoded": false,
    "multiValueHeaders": {
        "Accept-Encoding": [
            "gzip, deflate, br"
        ],
        "Accept-Language": [
            "en-US,en;q=0.9"
        ],
        "Cache-Control": [
            "no-cache"
        ],
        "Host": [
            "/* host */"
        ],
        "Origin": [
            "http://localhost:3000"
        ],
        "Pragma": [
            "no-cache"
        ],
        "Sec-WebSocket-Extensions": [
            "permessage-deflate; client_max_window_bits"
        ],
        "Sec-WebSocket-Key": [
            "/* key */"
        ],
        "Sec-WebSocket-Version": [
            "13"
        ],
        "User-Agent": [
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67"
        ],
        "X-Amzn-Trace-Id": [
            "/* id */"
        ],
        "X-Forwarded-For": [
            "/* ip */"
        ],
        "X-Forwarded-Port": [
            "443"
        ],
        "X-Forwarded-Proto": [
            "https"
        ]
    },
    "requestContext": {
        "apiId": "/* appId */",
        "connectedAt": 1628718944028,
        "connectionId": "/* connectionId */",
        "domainName": "/* domain */",
        "eventType": "CONNECT",
        "extendedRequestId": "/* id */",
        "identity": {
            "sourceIp": "/* ip */",
            "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67"
        },
        "messageDirection": "IN",
        "requestId": "/* id */",
        "requestTime": "11/Aug/2021:21:55:44 +0000",
        "requestTimeEpoch": 1628718944030,
        "routeKey": "$connect",
        "stage": "dev"
    }
}

We could use https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows#schedule to look for changes in the go sdk and automatically put up a PR.

Pull in a couple of API Gateway v2 fixes:

• aws/aws-lambda-go#280
• aws/aws-lambda-go#285

I'm building a password-less authentication poc using Cognito and found out CognitoEventUserPoolsDefineAuthChallengeResponse.issue_tokens, CognitoEventUserPoolsDefineAuthChallengeResponse.fail_authentication and CognitoEventUserPoolsVerifyAuthChallengeResponse.answer_correct can actually be null.

I took a look at this crate's code and from what i understand, a field will be generated as Option<T> in Rust if the field has the omitempty tag in the Go code.

I'm happy to submit a PR to generate these fields as Option<T> if you can tell me what's the preferred way of doing this type of thing.

https://github.com/LegNeato/aws-lambda-events/blob/632a850ed90503dbe07fceff1f354fc1df289c4c/aws_lambda_events/src/sns/mod.rs#L89

New SDK type: https://docs.rs/aws-sdk-sns/latest/aws_sdk_sns/model/struct.MessageAttributeValue.html

This way, they could support binary values.

(I discovered this because I'm making a lambda, that copies SNS messages cross account. Currently I have to manually convert.

Thank you for this great work.

I'm using this to return some image data from a lambda:

    let resp = AlbTargetGroupResponse {
        status_code: 200,
        body: Some(Body::Binary(buf)),
        headers: HeaderMap::new(),
        is_base64_encoded: true,
        multi_value_headers: HeaderMap::new(),
        status_description: Some(String::from("200 OK")),
    };

It appears that when using body: Some(Body::Binary(buf)), this library automatically encodes the payload in base64.

In this case, is_base64_encoded should be set to true. However, it might make sense to drop the is_base64_encoded field entirely, and auto-generate that based on the type of body:

• If body is Body::Text -> is_base64_encoded = false.
• If body is Body::Binary -> is_base64_encoded = true.
• If body is Body::Empty -> irrelevant...?

Given that most of the code around this is auto-generated, I'm not sure if this is even possible.

Are there any fields that are auto-generated based on other's value? Do you think my suggestion makes sense, or is it simpler to just add a note in the docs explaining that binary content is always base64 encoded?

Please consider re-exporting exposed types (query_map::QueryMap, for example). This would help avoid complications in dependencies downstream where the consumer of this package might need a different version of QueryMap for something else. When they need to QueryMap (or any other type), that consumer could simply use the QueryMap exported by this crate.

I specifically ran into this when dealing with the lambda_http crate. I can't easily update the version of QueryMap it depends on because this crate (as of right now) depends on an out of date version of it.

When a lambda function is attached to the $CONNECT route in a Websocket API, I get parsing errors e.g.

Error: Error("data did not match any variant of untagged enum LambdaRequest", line: 0, column: 0)

The raw request looks like this :

{
    "headers": {
        "Host": "blah.execute-api.us-west-2.amazonaws.com",
        "Sec-WebSocket-Key": "Q2062s5dM2uhnYdgQyQ8kg==",
        "Sec-WebSocket-Version": "13",
        "X-Amzn-Trace-Id": "Root=1-62153479-78192ed94f7a987b6cdbfc41",
        "x-api-key": "theapikey",
        "X-Forwarded-For": "0.0.0.0",
        "X-Forwarded-Port": "443",
        "X-Forwarded-Proto": "https"
    },
    "multiValueHeaders": {
        "Host": [
            "blah.execute-api.us-west-2.amazonaws.com"
        ],
        "Sec-WebSocket-Key": [
            "Q2062s5dM2uhnYdgQyQ8kg=="
        ],
        "Sec-WebSocket-Version": [
            "13"
        ],
        "X-Amzn-Trace-Id": [
            "Root=1-62153479-78192ed94f7a987b6cdbfc41"
        ],
        "x-api-key": [
            "theapikey"
        ],
        "X-Forwarded-For": [
            "0.0.0.0"
        ],
        "X-Forwarded-Port": [
            "443"
        ],
        "X-Forwarded-Proto": [
            "https"
        ]
    },
    "requestContext": {
        "routeKey": "$connect",
        "eventType": "CONNECT",
        "extendedRequestId": "N9UjFHPGvHcF99g=",
        "requestTime": "22/Feb/2022:19:07:37 +0000",
        "messageDirection": "IN",
        "stage": "dev",
        "connectedAt": 1645556857902,
        "requestTimeEpoch": 1645556857902,
        "identity": {
            "apiKey": "theapikey",
            "apiKeyId": "blah",
            "sourceIp": "0.0.0.0"
        },
        "requestId": "N9UjFHPGvHcF99g=",
        "domainName": "blah.execute-api.us-west-2.amazonaws.com",
        "connectionId": "N9UjFc6FPHcCIOw=",
        "apiId": "blah"
    },
    "isBase64Encoded": false
}

The lambda handler code is very simple :

use lambda_http::{service_fn, Error, IntoResponse, Request, RequestExt};
use tracing::{debug, info};

#[tokio::main]
async fn main() -> Result<(), Error> {
    tracing_subscriber::fmt()
        .with_max_level(tracing::Level::TRACE)
        .with_ansi(false)
        .without_time()
        .init();

    lambda_http::run(service_fn(func)).await?;
    Ok(())
}

async fn func(event: Request) -> Result<impl IntoResponse, Error> {
    debug!("Raw event : {:?}", event);
    info!("Received request ID : {}", event.lambda_context().request_id);
    Ok(format!("Hello world !").into_response())
}

The code above parses the fixture file correctly.

#97 changed the way that the SNS message event deserializes SigningCertURL and UnsubscribeURL. Unfortunately it seems like the AWS .NET SDK unit tests are wrong, and that the fields should actually be SigningCertUrl and UnsubscribeUrl. I verified this by running a Lambda built with the latest commit of aws-lambda-events and had it fail to deserialize the SNS event due to the casing mismatch.

For reference, https://github.com/aws/aws-lambda-go/blob/0dae56425b1a25ef20494ce922b36baeeaf92863/events/sns.go#L28-L30 is what the AWS Go SDK uses.

I am working on switching the payload type from 1.0 to 2.0 for an HTTP API I'm working on. When I did that with a stock Cognito JWT authorizer, I started receiving "scopes": null from my API Gateway. The definition in this crate is scopes: Vec<String>, so that doesn't work.

This appears to be a common input from API Gateway -> Lambda, as seen in these examples:

• https://www.jeremydaly.com/verifying-self-signed-jwt-tokens-with-aws-http-apis/
• https://dev.to/martzcodes/token-authorizers-with-apigatewayv2-tricks-apigwv1-doesn-t-want-you-to-know-41jn

The aws-lambda-go repo defines this as Scopes []string json:"scopes"``.

I don't know enough about Go and the codegen you're doing to know if the problem is in the initial Go data structure or in the codegen.

Rust assumes that the version field in the CodePipelineEventDetailType should be a number, when in reality (the JSON CloudWatch sends) contains a string.

I'm looking through the crate right now and I'm not seeing where support for S3 Event Test Events are. These are getting sent by S3 to SQS when I set up a new Event Notification.

These are documented here:
https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html

For now I can just write the struct myself, but it would make sense to support it directly here until the AWS team can publish their own official crate to go along with their runtime.

Odd that test events have a different structure than the S3 Events themselves.

I could be missing something obvious here, but how should users access the actual data in a Base64Data? Normal tuple access fails because the field is private.

If it's going to stay private, it seems like there should be fn inner(&self) -> &[u8] or impl Deref or something along those lines. I'd be happy to open a PR if you know which version you'd prefer.

Eventually we'll probably want to have Rust as the source of truth rather than generating from Go.

• Rust's type system is more expressive and can be more correct and ergonomic than what we can infer from Go.
• There are not many maintainers of the Go code so we will always lag behind / wait on them.

We would only want to break with the codegen when we have a critical mass of people using this lib and willing to provide event defs directly in Rust. I don't think we are there but wanted to open this so people can tell me when we are 😄 .

We use example files inside the generated module, and also outside of that module. It might make sense to move the fixtures directory outside the generated module.

This happens on a basic example when testing through api gateway. When I test lambda it works, but when testing through GW this happens...

First off, thank you for this fantastic crate!

Overview
This crate exports over 200 structs, and compiling all of them takes a long time! It would be great if we could introduce feature flags so users can pull in the subset of types they need for their project.

Context
When running cargo bloat --time on my project, I noticed aws_lambda_events takes just under 30 seconds to compile, more than any other dependency in my project, and more than double the next worst crate (tokio).

I also found using aws_lambda_events added ~4 seconds to my iterative debug build.

My Specific Use Case
My project is only using these four structs:

use aws_lambda_events::event::apigw::{
    ApiGatewayProxyRequest, ApiGatewayProxyRequestContext, ApiGatewayProxyResponse, ApiGatewayRequestIdentity,
};

I'd love it if I could add default-features = false, features = ["apigw_proxy"] to my Cargo.toml file to reduce code bloat and build times

Difference in 0.3 to 0.4, httpMethod/Method should be optional. This change breaks usability for websocket connection based events.

https://github.com/LegNeato/aws-lambda-events/blob/b86ba2835ff062d4771f500f82022fbe7525a33f/aws_lambda_events/src/generated/apigw.rs#L22

Quite frequently, I've got a use case where I have a Lambda subscribed to an SQS Queue that is in turn subscribed to an SNS topic, to which I'm posting a custom struct serialized to JSON as the message.

It would be great to have some way of having SQS and SNS events that, instead of having message / body attributes that are String, are able to JSON decode their payload so that I can for example define functions like this:

#[derive(Deserialize)]
struct CustomMsgStruct {
  // ... blah
}

pub async fn handler(e: LambdaEvent<SqsEvent<SnsEvent<CustomMsgStruct>>>) -> Result<(), Error> {
    
    let sqs_messages: Vec<SnsEvent<CustomMsgStruct>> = e.payload;

    for message in sqs_messages.into_iter() {
        let sns_records: Vec<SnsRecord<CustomMsgStruct>> = message.body.records;

        for record in sns_records.into_iter() {
            let msg: CustomMsgStruct = record.sns.message;
        
            // ... Do whatever I need to do with my deserialized custom struct
        }
    }

    Ok(())
}

Are there any good patterns out there that people have come across for doing this ergonomically? I'm currently using the SqsEvent from the lambda_sqs crate to handle deserializing SnsEvents out of the SQS message, followed by just calling serde_json manually on the SnsMessage body, but it feels very clunky to do this and it feels like something that it would be nice for aws_lambda_events to support itself.

With #50, I was actually aiming at the CloudTrail event in particular, more specifically the request_parameters field. It's generic in the sense that it may hold any type of AWS API calls. Structs for most of these are available in other crates, but there are two problems at the moment:

1. The official AWS SDK does not annotate any API calls with serde (de)-serialization attributes (e.g. https://github.com/awslabs/aws-sdk-rust/blob/2ad8b0abab2f54ac23d3c83f429873ae28d905fb/sdk/route53/src/input.rs#L16243-L16250)
2. Rusoto does indeed annotate their structs, but does not transform from camelCase to snake_case...

Guessing that the official SDK will be the go-to crate, I don't think contributing to Rusoto (at this point) makes much sense, but then again - I suppose there's a good reason why the official SDK doesn't depend on serde at all.

Does it make sense to consider generating said structs from the official SDK and serve them with the appropriate attributes in this crate?

Hi,

I was working with DynamoDB Stream, and I noticed something.
Please hear me out possible I am wrong.

Considering this
pub async fn function_handler(event: LambdaEvent<Event>) -> Result<(), Error> {

To access the DynamoDB records, you have something like this:

event.payload.records.iter().for_each(|record| {
        let record_set = MyTable::from_dynamodb(&record.change.new_image);
....
    });

new_image is type HashMap<String, AttributeValue> where AttributeValue is from aws_lambda_events::dynamodb::attributes::AttributeValue and it is quite different from aws_sdk_dynamodb::AttributeValue

Because there is no way to do it like node DynamoDB.Converter.unmarshall(record.dynamodb.NewImage) you have to implement your logic.

The aws_sdk_dynamodb::AttributeValue has methods like:

    pub fn as_s(&self) -> std::result::Result<&std::string::String, &Self> {
        if let AttributeValue::S(val) = &self {
            Ok(val)
        } else {
            Err(self)
        }
    }

While the aws_lambda_events::dynamodb::attributes::AttributeValue has none and so you have to implement your methods:

fn get_key(key: &str, t: ValueType, item: &HashMap<String, AttributeValue>) -> Option<String> {
        let attr = item.get(key).expect("Valid param");
        match t {
            ValueType::N => {
                if let AttributeValue::Number(val) = attr {
                    Some(val.to_string())
                } else {
                    None
                }
            }
            ValueType::S => {
                if let AttributeValue::String(val) = attr {
                    Some(val.to_string())
                } else {
                    None
                }
            }
        }
    }

The question is:
Is it possible to have the same type of AttributeValue? In this case, pointing to aws_lambda_events::dynamodb::attributes::AttributeValue

Decoding an s3 ObjectRemoved:Delete event, I'm getting this error. I think it's because size is not optional here: https://github.com/LegNeato/aws-lambda-events/blob/baffdab31e42eea24046dd7194074f07fe968569/aws_lambda_events/src/generated/s3.rs#L89.

Hey @LegNeato,

It's been a while since my last contribution to this project. I recently joined AWS and I'd like to help maintaining this project if you're open to it.

Can you email me to dcalaver @ amazon.com so we can talk about it?

We should make Rust enums out of Go code like:

https://github.com/aws/aws-lambda-go/blob/master/events/dynamodb.go#L127

In version 0.7.0 released recently, specifying only the dynamodb feature fails to build. This previously worked fine in 0.6.0.

This is reproducible with the following sequence of commands.

$ git clone https://github.com/LegNeato/aws-lambda-events
$ cd aws-lambda-events
$ git switch -d v0.7.0
$ cargo build --no-default-features --features dynamodb
error[E0432]: unresolved import `crate::event::streams`
 --> aws_lambda_events/src/dynamodb/mod.rs:2:19
  |
2 | use crate::event::streams::DynamoDbBatchItemFailure;
  |                   ^^^^^^^ could not find `streams` in `event`

Building with dynamodb and events seems to work fine.

$ cargo build --no-default-features --features dynamodb --features events

I ran into this because serde_dynamo only needs enough of aws-lambda-events to bring AttributeValue into scope.

For 0.6.0, the Cargo.toml invocation I used was

__aws_lambda_events_0_6 = { package = "aws_lambda_events", version = "0.6", default-features = false, features = ["dynamodb"], optional = true }

which fails for 0.7.0.

At this point I think I'll just use features = ["dynamodb", "events"] to support the already published 0.7.0, but I thought I'd bring it up as an issue in case you felt it needed to be addressed?

When you run cargo audit on the current master branch you get these vulnerabilities:

Crate:     rustc-serialize
Version:   0.3.24
Title:     Stack overflow in rustc_serialize when parsing deeply nested JSON
Date:      2022-01-01
ID:        RUSTSEC-2022-0004
URL:       https://rustsec.org/advisories/RUSTSEC-2022-0004
Solution:  No fixed upgrade is available!
Dependency tree:
rustc-serialize 0.3.24
└── rustc-test 0.3.1
    └── go_to_rust 0.1.1
        └── aws_lambda_events_codegen 0.1.2

Crate:     time
Version:   0.1.44
Title:     Potential segfault in the time crate
Date:      2020-11-18
ID:        RUSTSEC-2020-0071
URL:       https://rustsec.org/advisories/RUSTSEC-2020-0071
Solution:  Upgrade to >=0.2.23
Dependency tree:
time 0.1.44
├── rustc-test 0.3.1
│   └── go_to_rust 0.1.1
│       └── aws_lambda_events_codegen 0.1.2
└── chrono 0.4.22
    └── aws_lambda_events 0.7.0

One can be solved with: #110 but the others they cannot.

The project rustc-serialize is archived, so I don't know solve this. But since it is only used when generation it is not so bad?

There is no way of specifying the type parameters for the request context (ApiGatewayWebsocketProxyRequestContext) of ApiGatewayWebsocketProxyRequest so even though the former supports type parameters for the authorizer and message ID there's no way of specifying them.

The type parameter for ApiGatewayCustomAuthorizerResponse is also used incorrectly, it's used as pub context: HashMap<String, T1> whereas pub context: T1 would make much more sense.

The type parameters could also be better named - they all seem to be named T1, T2, etc.

This probably applies to more events, but these are the ones I'm working with right now so I haven't investigated the rest of the events.

I haven't tried to run the codegen at all, just scanned through the list of event types. It appears that Dynamo events are missing - is this a known issue?

Thanks!

Some modules have multiple example test event json files (such as aws/aws-lambda-go@c941d49). We should support synthesizing tests for those cases.