Giter Club home page Giter Club logo

aws-lambda-ses-forwarder's Introduction

AWS Lambda SES Email Forwarder

npm version Travis CI test status Test coverage status

A Node.js script for AWS Lambda that uses the inbound/outbound capabilities of AWS Simple Email Service (SES) to run a "serverless" email forwarding service.

Instead of setting up an email server on an EC2 instance to handle email redirects, use SES to receive email, and the included Lambda script to process it and send it on to the chosen destination.

Limitations

Set Up

  1. Modify the values in the config object at the top of index.js to specify the S3 bucket and object prefix for locating emails stored by SES. Also provide the email forwarding mapping from original destinations to new destination.

  2. In AWS Lambda, add a new function and skip selecting a blueprint.

  • Name the function "SesForwarder" and optionally give it a description. Ensure Runtime is set to Node.js 16.x. (Node.js 18.x can be used if the AWS SDK v2 module is also installed.)

  • For the Lambda function code, either copy and paste the contents of index.js into the inline code editor or zip the contents of the repository and upload them directly or via S3.

  • Ensure Handler is set to index.handler.

  • For Role, choose "Basic Execution Role" under Create New Role. In the popup, give the role a name (e.g., LambdaSesForwarder). Configure the role policy to the following:

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
            "logs:CreateLogGroup",
            "logs:CreateLogStream",
            "logs:PutLogEvents"
         ],
         "Resource": "arn:aws:logs:*:*:*"
      },
      {
         "Effect": "Allow",
         "Action": "ses:SendRawEmail",
         "Resource": "*"
      },
      {
         "Effect": "Allow",
         "Action": [
            "s3:GetObject",
            "s3:PutObject"
         ],
         "Resource": "arn:aws:s3:::S3-BUCKET-NAME/*"
      }
   ]
}
  • Configure the Memory and Timeout settings according to your use case. For simple text emails, a memory limit of 128 MB and timeout of 10 seconds should be sufficient. For emails with large attachments, a memory limit of 512 MB or more and timeout of 30 seconds may be required.
  1. In AWS SES, verify the domains for which you want to receive and forward email. Also configure the DNS MX record for these domains to point to the email receiving (or inbound) SES endpoint. See SES documentation for the email receiving endpoints in each region.

  2. If you have the sandbox level of access to SES, then also verify any email addresses to which you want to forward email that are not on verified domains.

  3. If you have not configured inbound email handling, create a new Rule Set. Otherwise, you can use an existing one.

  4. Create a rule for handling email forwarding functionality.

  • On the Recipients configuration page, add any email addresses from which you want to forward email.

  • On the Actions configuration page, add an S3 action first and then an Lambda action.

  • For the S3 action: Create or choose an existing S3 bucket. Optionally, add an object key prefix. Leave Encrypt Message unchecked and SNS Topic set to [none].

  • For the Lambda action: Choose the SesForwarder Lambda function. Leave Invocation Type set to Event and SNS Topic set to [none].

  • Finish by naming the rule, ensuring it's enabled and that spam and virus checking are used.

  • If you get an error like "Could not write to bucket", follow step 7 before completing this one

  • If you are asked by SES to add permissions to access lambda:InvokeFunction, agree to it.

  1. The S3 bucket policy needs to be configured so that your IAM user has read and write access to the S3 bucket. When you set up the S3 action in SES, it may add a bucket policy statement that denies all users other than root access to get objects. This causes access issues from the Lambda script, so you will likely need to adjust the bucket policy statement with one like this:
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "GiveSESPermissionToWriteEmail",
         "Effect": "Allow",
         "Principal": {
            "Service": "ses.amazonaws.com"
         },
         "Action": "s3:PutObject",
         "Resource": "arn:aws:s3:::S3-BUCKET-NAME/*",
         "Condition": {
            "StringEquals": {
               "aws:Referer": "AWS-ACCOUNT-ID"
            }
         }
      }
   ]
}
  1. Optionally set the S3 lifecycle for this bucket to delete/expire objects after a few days to clean up the saved emails.

Extending

By loading aws-lambda-ses-forwarder as a module in a Lambda script, you can override the default config settings, change the order in which to process tasks, and add functions as custom tasks.

The overrides object may have the following keys:

  • config: An object that defines the S3 storage location and mapping for email forwarding.
  • log: A function that accepts log messages for reporting. By default, this is set to console.log.
  • steps: An array of functions that should be executed to process and forward the email. See index.js for the default set of steps.

See example for how to provide configuration as overrides.

Troubleshooting

Test the configuration by sending emails to recipient addresses.

  • If you receive a bounce from AWS with the message "This message could not be delivered due to a recipient error.", then the rules could not be executed. Check the configuration of the rules.

  • Check if you find an object associated with the message in the S3 bucket.

  • If your Lambda function encounters an error it will be logged in CloudWatch. Click on "Logs" in the CloudWatch menu, and you should find a log group for the Lambda function.

Credits

Based on the work of @eleven41 and @mwhouser from: https://github.com/eleven41/aws-lambda-send-ses-email

aws-lambda-ses-forwarder's People

Contributors

arithmetric avatar bensower avatar ccjmne avatar cgonser avatar ctr49 avatar dansmith65 avatar jakubboucek avatar mariusrumpf avatar michaellasmanis avatar morenoh149 avatar paolobarbolini avatar pr1ntr avatar rbiro avatar saeger avatar yanokwa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-lambda-ses-forwarder's Issues

Cost?

How much do you typically get charged by using this method?

I guess a more static way of doing things would be to use G Suit, then just forward emails through there.

Any thoughts?

'copyObject() returned error:'

I have followed exactly what you've spelt out on https://github.com/arithmetric/aws-lambda-ses-forwarder but am stuck. No amount of Google search or No playing with options either has helped. A client is about to terminate the contract - we've stuck at this error for too long.

I have verified my domain. I receive email from anywhere through my verified address to my designated S3 bucket. I also changed the '"functionArn":xxx..."' to '"functionArn": "arn:aws:lambda:us-west-2:MY-ACCOUNT-ID:function:fnForwardEmailToGmail"'.

However the received emails are never forwarded and when I test the function using "SES Email Receiving" I get the error below.

START RequestId: 7f2cd5ed-83ea-11e7-913f-55748388c69f Version: $LATEST
2017-08-18T07:54:44.467Z 7f2cd5ed-83ea-11e7-913f-55748388c69f { level: 'info',
message: 'Fetching email at s3://MY-S3-BUCKET/MY-PREFIX/o3vrnil0e2ic28fgfdstrm7dfhrc2v0clambda4nbp0g1' }
2017-08-18T07:54:46.068Z 7f2cd5ed-83ea-11e7-913f-55748388c69f { level: 'error',
message: 'copyObject() returned error:',
error:
{ [AccessDenied: Access Denied]
message: 'Access Denied',
code: 'AccessDenied',
region: null,
time: Fri Aug 18 2017 07:54:46 GMT+0000 (UTC),
requestId: 'A6285517D1AF2B9D',
extendedRequestId: 'dfH3csS5kHLsYN4ZgIWVliYmuVb1OgCVl6KdUSdZdqwX2T+JdkfZwIyPa5KEgYFiJfZmrwXjXDI=',
cfId: undefined,
statusCode: 403,
retryable: false,
retryDelay: 32.49475641641766 },
stack: 'AccessDenied: Access Denied
at Request.extractError (/var/task/node_modules/aws-sdk/lib/services/s3.js:473:35)
at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:615:14)
at Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /var/task/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request. (/var/task/node_modules/aws-sdk/lib/request.js:38:9)
at Request. (/var/task/node_modules/aws-sdk/lib/request.js:617:12)
at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:115:18)' }
2017-08-18T07:54:46.127Z 7f2cd5ed-83ea-11e7-913f-55748388c69f {"errorMessage":"Error: Could not make readable copy of email."}
END RequestId: 7f2cd5ed-83ea-11e7-913f-55748388c69f
REPORT RequestId: 7f2cd5ed-83ea-11e7-913f-55748388c69f Duration: 1993.85 ms Billed Duration: 2000 ms Memory Size: 128 MB Max Memory Used: 32 MB

LAMBDA ROLE FUNCTION POLICY
{
"Version": "2016-03-04",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:::"
},
{
"Effect": "Allow",
"Action": "ses:SendRawEmail",
"Resource": "
"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::MY-S3-BUCKET/*"
}
]
}

S3 BUCKET POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "GiveSESPermissionToWriteEmail",
"Effect": "Allow",
"Principal": {
"Service": "ses.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::MY-S3-BUCKET/*",
"Condition": {
"StringEquals": {
"aws:Referer": "MY-ACCOUNT-ID"
}
}
}
]
}

InvalidParameterValue: Extra route-addr

I get InvalidParameterValue: Extra route-addr when forwarding an email. Not sure what causes this, but I noticed that the email that fails has charset=windows-1252, and the one that succeeds has charset=UTF-8.

Fails:

...
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="------------F2C3840841699E1CBA46489D"

This is a multi-part message in MIME format.
--------------F2C3840841699E1CBA46489D
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 8bit

some text

--------------F2C3840841699E1CBA46489D
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit

<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=windows-1252">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <p><font face="Helvetica, Arial, sans-serif">no sign</font><br>
    </p>
    <br>
    <div class="moz-signature">-- <br>
      <br>
      <b>John Doe</b><br>
      <a href="https://example.com">Example</a><br>
    </div>
  </body>
</html>

--------------F2C3840841699E1CBA46489D--

Doesn't fail:

...
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary=94eb2c18fe78175025054b289093

--94eb2c18fe78175025054b289093
Content-Type: text/plain; charset=UTF-8

Hello

--94eb2c18fe78175025054b289093
Content-Type: text/html; charset=UTF-8

<div dir="auto">Hello</div>

--94eb2c18fe78175025054b289093--

mail content

Hi,
I'm having some trouble in forwarding email.
Every time when mail forwarding is done, these kind of codes are included in the mail.
Is there any way to get rid of these codes?

X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
bh=VcUwljfBB3Ih9IZnGfiyRq86HN5fYES5CKp76GqkLsE=;
b=Kg02Ur+JrglkIC/oKiheGqdcAQ5YnKq0PYHLA7yCFgTi5AoNkDLrlR9vx96F85hWgh
5gih0q1GySuWWrW8MljCg5gtqdu0SAWIhfij2GR5DiFXE1L0NQL/9MAEm+U3qHiNsXjP
6y2f2BMFNl9o1JXKRYq4wv4XMhJXgC6lNRsI/q1Wynm7zxXXnYrPW9zkgA5LKs0afQHw
aupdrBZmTTa8VcFBBu8t51s9aTqiAwpZSDC/iT9XJ9UjPGutnI7Mr+TUXXWXEnj4QlZG
hj5RrMTOb1v/UpTPNbt4jfPAOXz30A66TeoaOX/yExLYrouwwy4Ss3IXXbFQ25xB/bjr
gSUw==

--94eb2c0b8d5ef56d5d05528a0878
Content-Type: text/plain; charset="UTF-8"

Support ability to ignore emails originating from verified domains/email addresses

Awesome work, there is only one thing that I would like to see different. If I am sending a mail from, say, a PHP app to a user, let's say a notification, I want this script to ignore this email and not change it and just let it pass through unharmed. If it comes from another domain not in my list of verified domains then it should do its thang.

Forgive me if there is already a way of doing this. If not then this is something I would love to see.

Fails to forward mail with more than 50 recipients

If the Lambda function is invoked with an e-mail that has more than 50 original recipients, it will create an error in SES.

Example: User is part of an e-mail with 100 different addresses. The forwarded e-mail should only be sent to the user's forward address (1 recipient) but instead an error is thrown in SES for attempting to send an e-mail with more than 50 recipients.

I'll update this if I can configure a fix or find the logs

Functionality request - AWS store and forward for backup MX

Hi Joe

I am interested in using your code to create a store and forward backup MX on AWS.

As far as I can tell, this would require changing the code to forward to the original addressee. This would appear trivial.

The major difference I think is that the forward would need to be triggered by a poll on the main MX showing it is up and then looping through all emails in the S3 bucket rather than just forwarding the one email in the current code.

I think the store and forward requirement is typical. There may be other code that does this.

If not I'm prepared to contribute to beer money...

Example policy displays an error

This policy contains the following error: Has prohibited field Principal For more information about the IAM policy grammar, see AWS IAM Policies.

I was interested in implementing this. I got quite far, but when I attempt to create the example policy listed I see the error listed above.

Any advise?

1 Mail getting redirected (sent) ~5 times

I've come across a weird situation where occasionally when an email is sent to the domain where I have the redirect set up, the redirected to domain gets five identical copies of the email.

It looks like lambda is fired off five times one after another but the execution of the lambdas still occur in parallel. I.e. the lambda doesn't happen back to back but all at once.

Has anyone else come across this problem before? I'm still in the process of debugging the cause so any help would be appreciated.

Problems identifying original sender

Got it all set up nicely for multiple domains, just have one issue.

Some clients (Gmail) are showing messages up as from myself.
I think I have it configured correctly but suspect it is DKIM signing.
Because we're processing the email and using SES to send it on, it's being signed by my AWS identity.

Any way to stop this?

I presume the original mail is being signed by the original sender, but it will be stripped out when SES downloads it to the S3 bucket?

e.g.
From line says: "Other person at [email protected] < [email protected] > "
Reply-to : "[email protected] [email protected]"
Signed-by: my.domain.com

level: 'error', message: 'Step returned error: Error: Received invalid SES message.'

Hi Guys,

I´ve spent a few hours double checking 99% of the settings according to setup yet I still cannot get over this error message. Any tips?

{ level: 'error',
message: 'Step returned error: Error: Received invalid SES message.',
error: [Error: Error: Received invalid SES message.],
stack: 'Error: Error: Received invalid SES message.\n at exports.parseEvent (/var/task/index.js:59:27)' }

level: 'error',
message: 'Step returned error: Error: Could not make readable copy of email.',
error: [Error: Error: Could not make readable copy of email.],
stack: 'Error: Error: Could not make readable copy of email.\n at Response. (/var/task/index.js:144:11)\n at Request.

PR for domainMapping?

Would you support a PR to add a domain mapping feature to forward all mail from one domain to another with the same username? Configuration might be like:

{
    config: {
      fromEmail: "[email protected]",
      emailBucket: "s3-bucket-name",
      emailKeyPrefix: "emailsPrefix/",
      forwardMapping: {
        "[email protected]": [
          "[email protected]",
          "[email protected]"
        ],
        "[email protected]": [
          "[email protected]"
        ]
      },
      domainMapping: {
        "@foo.example.com": "@example.com"
      }
    }
  }

So [email protected] would forward to [email protected]. The forwardMapping would still take precedent, but if there aren't any matches for forwardMapping, it would check the domainMapping.

Thoughts?

I've currently done this by overriding the transformRecipients function in my own module, but it's not as succinct as it would be integrated directly. I'm happy to provide tests, implementation, and documentation.

Thanks!

Nick

X-Autoreply: yes

Thank you to the creators of this script! This is a terrific script and I've been using it for over a month now. It's been working great.

One curious issue I've come across recently is due to people with 'Gmail autoreplies' set up. Seems like between Gmail and Amazon, the original user's from address is getting stripped before it is even seen by this script. This results in undesired behavior:

  1. Email is sent to Gmail user (FROM [email protected]), with auto reply configured
  2. Auto-reply sends email back to ([email protected]), however, From address is listed as [email protected].

From header reports amazonses.com (??)

To header reports the same email address as the one sent.

To: (same as sender address)

Possibly-related MIME headers:

Precedence: bulk
X-Autoreply: yes
Auto-Submitted: auto-replied

At any rate, I can think of a few ways to deal with this:

  • Look for and ignore any mail with X-Autoreply: yes header set
  • Ignore any email with FROM: [email protected]

General thoughts?

Email declared as spam

When I receive an email, it is marked as spam (outlook.com).

However, if I specify in the inbound rule that mails are encrypted on S3, it is not marked. But you can't read the mail anymore, because it is encrypted.

Anybody know what I can do about it?

SignatureDoesNotMatch error

I am attempting to use this function, and it appears to be executing when I send an email to my verified address, as it is logging an error. However, I'm not sure how to fix this.

SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.

Where do I enter the signature in my settings or config? Presumably this is something I need to do in Lambda, or SES, or in IAM, or where? I do not see any config in this particular script.

mails from local domain forwarded result in a SES domain DKIM check failure

Let's say yourdomain.com is the domain the forwarder is sending emails from, and anotherdomain.com is another domain. Consider these cases:

  1. [email protected] sends a mail to [email protected]. The mail is forwarded to recipients correctly without any checks failed;
  2. [email protected] sends a mail to [email protected]. The mail ends up with a DKIM(on SES domain) check failure.

After looking at the DKIM-Signature part I found it's weird that the failed one has TWO Feedback-ID's in the h field. I guess it might be the cause and tried to fix the forwarding code to remove the field from the header, and it works.

But I don't know what this field means and what side-effects the removal would introduce.

Here's excerpt from the message fails DKIM validation:

Authentication-Results: mx.google.com;
       dkim=fail [email protected];
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
    s=gdwg2y3kokkkj5a55z2ilkup5wp5hhxx; d=amazonses.com; t=1470162493;
    h=Message-ID:Date:Subject:From:Reply-To:To:MIME-Version:Content-Type:Content-Transfer-Encoding:Feedback-ID:Feedback-ID;
    bh=cIc4+mNxsFp+dUwkGlbCwYelaS0phVkPmIICqAGQvPs=;
    b=oQRKCP7Qz2BZ7rNOt64kJZLMQiAw85afNkIS0ZBezqdObXAfEINMPtrG0K+oZps2
    CmPVoLW8UkSkFU4bTtaiaXYw0lsmAjXvpsipQyYQFPYN1MfEWIFQZBxTEJJEVnozkyb
    M31UJ95rnkf437YCCPyFvSMsYHdmhlceK++Fk6QY=

And here is a success one after I removed the field:

Authentication-Results: mx.google.com;
       dkim=pass [email protected];
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
    s=gdwg2y3kokkkj5a55z2ilkup5wp5hhxx; d=amazonses.com; t=1470166761;
    h=Date:From:To:Subject:Content-Type:MIME-Version:Message-ID:Reply-To:Feedback-ID;
    bh=EEznD2tyqczD8Zpld/7JgAHSIj9dxh/xhszpF4qOYyQ=;
    b=cMic0Xdgjvkrr5mgutcRyVmxDw2apNOWABeTRx41q9PqrTEymFATlZObJ2kq9Bbz
    X27rFRkfkT+XN6z4qwzvrXXG6VSDXnfZ4B9aM4/9ntQ9wyfjcyZXHaZlbr9i98L2qLY
    cQqvz2ET1Dg7FOsOh6tCUXEMTaCbpE0xo598cN/o=

Subject Prefix Mod. Feel free to incorporate into original code

A small bug encountered when emailing back and forth with replies via the SES forwarder script in that successive subjectPrefixes can accumulate on each reply received.

Following code corrects, feel free to add:

  // Add a prefix to the Subject
  if (data.config.subjectPrefix) {
    header = header.replace(/^Subject: (.*)/mg,
      function(match, subject) {
        if (subject.indexOf(data.config.subjectPrefix) < 0) {
            return 'Subject: ' + data.config.subjectPrefix + subject;
        } else {
            return 'Subject: ' + subject;
        }
        
      });
  }

Could not make readable copy of email

I double-checked all the permissions according to readme, but still get the error:

START RequestId: af224ff3-fb47-11e6-9444-1f9e512246cc Version: $LATEST
2017-02-25T10:46:38.924Z	af224ff3-fb47-11e6-9444-1f9e512246cc	{ level: 'info',
  message: 'Fetching email at s3://musweb.mail/o3vrnil0e2ic28trm7dfhrc2v0clambda4nbp0g1' }
2017-02-25T10:46:40.344Z	af224ff3-fb47-11e6-9444-1f9e512246cc	{ level: 'error',
  message: 'copyObject() returned error:',
  error: 
   { [AccessDenied: Access Denied]
     message: 'Access Denied',
     code: 'AccessDenied',
     region: null,
     time: Sat Feb 25 2017 10:46:40 GMT+0000 (UTC),
     requestId: '4CC93788EF0116A6',
     extendedRequestId: 'nGtVxfT3aT/uqPpufrdvjJrLy/ClZ4WeJoqh8AK9eXy5keqqx3Q89SSnkrvIYyxnSS31STpGF00=',
     cfId: undefined,
     statusCode: 403,
     retryable: false,
     retryDelay: 50.05418912041932 },
  stack: 'AccessDenied: Access Denied
    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/s3.js:538:35)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:671:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:673:12)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)' }
2017-02-25T10:46:40.404Z	af224ff3-fb47-11e6-9444-1f9e512246cc	{ level: 'error',
  message: 'Step returned error: Error: Could not make readable copy of email.',
  error: [Error: Error: Could not make readable copy of email.],
  stack: 'Error: Error: Could not make readable copy of email.
    at Response.<anonymous> (/var/task/index.js:145:11)
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:358:18)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:671:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:673:12)' }
2017-02-25T10:46:40.405Z	af224ff3-fb47-11e6-9444-1f9e512246cc	{"errorMessage":"Error: Step returned error.","errorType":"Error","stackTrace":["/var/task/index.js:302:28","process._tickDomainCallback (node.js:407:9)"]}
END RequestId: af224ff3-fb47-11e6-9444-1f9e512246cc
REPORT RequestId: af224ff3-fb47-11e6-9444-1f9e512246cc	Duration: 1942.54 ms	Billed Duration: 2000 ms 

[q] SNS trigger

Hi!

This is not an issue, but an educational question. I wonder why did you not set up this lambda with an SNS trigger (rule with 1 action: write to S3 + SNS topic) instead of rule with 2 actions: write to S3, trigger lambda?

Thanks!

Remove DKIM-Signature header from message

Amazon SES refuses to send email if there are duplicate DKIM-Signature headers.

See the following error message from CloudWatch logs:

InvalidParameterValue: Duplicate header 'DKIM-Signature'.] message: 'Duplicate header \'DKIM-Signature\'.', code: 'InvalidParameterValue', time: Sat Mar 05 2016 20:19:22 GMT+0000 (UTC), requestId: '8c87c2e0-e30f-1234-a6f4-616802c8702d', statusCode: 400, retryable: false, retryDelay: 30 }

Unable to import module 'index'

In Lambda function logs I see:

START RequestId: 12ea793b-9ef2-11e7-b1ef-b360a5b0eab4 Version: $LATEST
Unable to import module 'index': Error at Module.require (module.js:353:17)
END RequestId: 12ea793b-9ef2-11e7-b1ef-b360a5b0eab4
REPORT RequestId: 12ea793b-9ef2-11e7-b1ef-b360a5b0eab4	Duration: 1.88 ms	Billed Duration: 100 ms Memory Size: 128 MB	Max Memory Used: 21 MB

What does it mean?

modifying the "From" header causes DKIM to fail

Sending mail from the primary domain of my Google Apps account to this forwarder works, but causes DKIM to fail when received by a Gmail account the email was forwarded to. The reason it fails is because Google Apps originally signed the email's using the From header as one of it's inputs, but that value is modified by this forwarder, so verifying that signature fails on the final receiving end.

I haven't tested, but if the sender had dmarc setup to reject non-aligned messages, then I suspect this email would be dropped/bounced by the receiving mail server. Even without using dmark, having a failed dkim header could potentially introduce delivery issues.

Since the "From" address must be a verified address, as per SES, this may just be a limitation of using SES/Lambda as a mail forwarder.

TypeError: Cannot read property 'log' of undefined

Getting this error:

017-12-01T13:23:19.809Z cbb36a51-d69a-11e7-966a-e777dc99be75 { level: 'error',
message: 'Step returned error: Cannot read property 'log' of undefined',
error:
TypeError: Cannot read property 'log' of undefined
at exports.fetchMessage (/var/task/index.js:127:7),
stack: 'TypeError: Cannot read property 'log' of undefined\n at exports.fetchMessage (/var/task/index.js:127:7)' }

Failed due to duplicate Message-ID header

I've been using this project for a while and it's awesome, thanks a lot! I stumbled upon an error for the first time today. Error log from CloudWatch is below:

2017-05-03T11:38:51.802Z	f2ab5592-2ff4-11e7-97ca-a1db4277fedb	{ level: 'error',
message: 'sendRawEmail() returned error.',
error: 
{ [InvalidParameterValue: Duplicate header 'Message-ID'.]
message: 'Duplicate header \'Message-ID\'.',
code: 'InvalidParameterValue',
time: Wed May 03 2017 11:38:51 GMT+0000 (UTC),
requestId: '14956f7e-2ff5-11e7-8290-6fdc1cb44eda',
statusCode: 400,
retryable: false,
retryDelay: 16.828257404267788 },
stack: 'InvalidParameterValue: Duplicate header \'Message-ID\'.\n at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/query.js:47:29)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:673:14)\n at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)\n at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)\n at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:675:12)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)' }

Repro step: create a Skype account using [email protected]. The email containing the verification code fails to be forwarded.

Adding a custom header

I've got this working successfully but wanted to track my bounces and errors via CloudWatch.

In order to do this it seems like I need to add a custom header (X-SES-CONFIGURATION-SET) to my forwarded emails. http://docs.aws.amazon.com/ses/latest/DeveloperGuide/event-publishing-send-email.html

I've tried doing this but I'm some how mutilating the header and it's throwing this error:
message: 'sendRawEmail() returned error.', error: { [InvalidParameterValue: Duplicate header 'Content-Type'.] message: 'Duplicate header \'Content-Type\'.', code: 'InvalidParameterValue',

I inserted the following code into the process.message function, at the very end:
header = header + 'X-SES-CONFIGURATION-SET: cloudwatch_SES_destination\r'; data.log({level: "info", message: "Added Configuration Group Tag"});

original mail header sent in body of messages

When I send an e-mail to an assigned address it gets forwarded now (after the Return-Path merge), but all original email header is put into the body of the forwarded mail, so it also loses all properties like subject and message ID.

Any idea what might be causing this? I can forward an example if needed.

Bursts and throttling, working on bigger scale

Have you considered bigger scale events and bursts? Let's say, SES received 5000 messages just now - your script will try to invoke 5000 Lambdas and send out 5000 emails which will break two limits at once - Lambdas limit for concurrent runs and SES send-out rate-limit.

It would be nice to have a pacing mechanism. Or at least support the dead-letter queue (not ideal).

For instance, one possibility could be: when SES email is received, it's saved in S3 and an SNS event is broadcasted (no Lambda is invoked). An SQS queue is subscribed to this SNS topic so that there is always a queue of emails that can be processed synchronously/later on.

Then, Lambda is invoked by CloudWatch Schedule instead, let's say one once every minute. Maximum execution time is set to 1 minute so that it can run always, if necessary. This shouldn't make it more expensive than current model - AWS charges for total, sum time, not invocations.

Then, this smart Lambda function checks how much in trouble the queue is (how many queued messages there are) and decides the number of outgoing SES threads based on that. The number will never go higher than current SES send-rate (could be a part of the config).

So your current send-rate is 30 emails per second, Lambda function sees 1000 messages in the queue - it will run 30 threads to send out emails until AWS kills it after 1 minute. Then, it will be invoked again.

If queue is empty, Lambda will die gracefully, it won't idle for 60 seconds. Or it could poll lazily - every 5-10 seconds.

Receiving double emails

I configured this for my aws account. The problem that I'm facing is that when I sent an email to my domain.com email, I got 2 of the same emails in my inbox.

When I had a subject prefix added, I received 1 email without subject prefix and 1 with the subject prefix.

Any idea why would this happen?

parseEvent() received invalid SES message:

Getting this error in CloudWatch logs. Best I can tell it's because data.event.Records[0].eventVersion is coming through as "2.0" and index.js requires that it's "1.0". Do I need to change my AWS setup somewhere to force 1.0, or do I change index.js to allow this through?

Thanks!

Won't forward for unverified addresses.

Hi there,

Your script only forwards if the email is sent FROM an already verified address.

I'm not running in a sandbox, but SES always seems to think the from address is the original from address when forwarding, and it fails with "Email address is not verified".

If I verify the sender's address in SES, then the script works, but obviously I cannot verify everyone who is going to send me email.

I tried removing the From: original via Recipient in lieu of just From: recipient, but it still fails. Maybe because of the Reply-To, or maybe because of the Return-Path? Not sure.

Forward historical S3 contents to email

There was a period of time when the email forwarding lambda was not working so our email account (in gmail) did not receive the emails via AWS, but the files for these emails are stored in our s3 bucket.

Right now, only new emails are being forwarded. How can I export / forward all the contents of the s3 bucket which contains those historical emails that weren't received by our email client (gmail)?

Thanks

Do not touch From header

Why the script is changing From header to own address?
Are there some reason why it is doing it?
How to turn it off?

InvalidParameterValue: Duplicate header 'DKIM-Signature'

It looks like you addressed a similar issue in #8, but my use-case is a little different.

I use a google apps account with an alias domain so I can send mail from two different addresses, although both are "mailed by" the first:

Both domains have DKIM configured, so when I send email from the alias domain, two DKIM-Signature headers are included; one for each domain. This works fine when sending mail to most places, but it fails when being processed by this lambda forwarder. I get this error:

{
    level : 'error',
    message : 'sendRawEmail() returned error.',
    error :
    {
        [InvalidParameterValue : Duplicate header 'DKIM-Signature'.]message : 'Duplicate header \'DKIM-Signature\'.',
        code : 'InvalidParameterValue',
        time : Fri Mar 25 2016 02 : 50 : 53 GMT + 0000(UTC),
        requestId : '64105019-f234-11e5-a148-3d2a02a5aa36',
        statusCode : 400,
        retryable : false,
        retryDelay : 80.16784060746431
    },
    stack : 'InvalidParameterValue: Duplicate header \'DKIM-Signature\'.\n at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/query.js:40:29)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:596:14)\n at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:21:10)\n at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:37:9)\n at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:598:12)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)'
}

I suspect this issue is actually with SES, but I wanted to document it here, so people at least know that it will affect their use of this project.

Support SES receive action

Currently it depends on storing incoming e-mails to S3. Why it don't fetch e-mails from event, so we don't need S3?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.