Giter Club home page Giter Club logo

griddler's Introduction

Deprecated as of June 6, 2024

Griddler has been deprecated in favor of ActionMailer since this is built into Rails now.

Griddler

Receive emails in your Rails app

Griddler is a Rails engine that provides an endpoint for services that convert incoming emails to HTTP POST requests. It parses these POSTs and hands off a built email object to a class implemented by you.

Tutorials

  • SendGrid wrote a great tutorial on integrating Griddler with your application.
  • We have our own blog post on the subject over at Giant Robots.

Installation

  1. Add griddler and an adapter gem to your application's Gemfile and run bundle install.

  2. A route is needed for the endpoint which receives POST messages. To add the route, in config/routes.rb you may either use the provided routing method mount_griddler or set the route explicitly. Examples:

    # config/routes.rb
    
    # mount using default path: /email_processor
    mount_griddler
    
    # mount using a custom path
    mount_griddler('/email/incoming')
    
    # the DIY approach:
    post '/email_processor' => 'griddler/emails#create'

Configuration Options

An initializer can be created to control some of the options in Griddler. Defaults are shown below with sample overrides following. In config/initializers/griddler.rb:

Griddler.configure do |config|
  config.processor_class = EmailProcessor # CommentViaEmail
  config.email_class = Griddler::Email # MyEmail
  config.processor_method = :process # :create_comment (A method on CommentViaEmail)
  config.reply_delimiter = '-- REPLY ABOVE THIS LINE --'
  config.email_service = :sendgrid # :cloudmailin, :postmark, :mandrill, :mailgun
end
Option Meaning
processor_class The class Griddler will use to handle your incoming emails.
email_class The class Griddler will use to represent an incoming e-mail. It must support an initializer that receives a hash as the only argument. We recommend inheriting it from Griddler::Email or checking this class to see all the methods it responds to.
processor_method The method Griddler will call on the processor class when handling your incoming emails.
reply_delimiter The string searched for that will split your body. You can also supply an array of string - useful if you need translated versions of the delimiter
email_service Tells Griddler which email service you are using. The supported email service options are :sendgrid (the default), :cloudmailin (expects multipart format), :postmark, :mandrill and :mailgun. You will also need to have an appropriate adapter gem included in your Gemfile.

By default Griddler will look for a class named EmailProcessor. The class is initialized with a Griddler::Email instance representing the incoming email, and has a process method to actually process the email. For example, in ./lib/email_processor.rb:

class EmailProcessor
  def initialize(email)
    @email = email
  end

  def process
    # all of your application-specific code here - creating models,
    # processing reports, etc

    # here's an example of model creation
    user = User.find_by_email(@email.from[:email])
    user.posts.create!(
      subject: @email.subject,
      body: @email.body
    )
  end
end

Griddler::Email attributes

Attribute Description
#to An array of hashes containing recipient address information. See Email Addresses for more information.
#from A hash containing the sender address information.
#cc An array of hashes containing cc email address information.
#subject The subject of the email message.
#body The full contents of the email body unless there is a line in the email containing the string -- REPLY ABOVE THIS LINE --. In that case .body will contain everything before that line.
#raw_text The raw text part of the body.
#raw_html The raw html part of the body.
#raw_body The raw body information provided by the email service.
#attachments An array of File objects containing any attachments.
#headers A hash of headers parsed by Mail::Header, unless they are already formatted as a hash when received from the adapter in which case the original hash is returned.
#raw_headers The raw headers included in the message.
#to_h A hash of the above attributes.

Email Addresses

Gridder::Email provides email addresses as hashes. Each hash will have the following information of each recipient:

Key Value
:token All the text before the email's "@". We've found that this is the most often used portion of the email address and consider it to be the token we'll key off of for interaction with our application.
:host All the text after the email's "@". This is important to filter the recipients sent to the application vs emails to other domains. More info below on the Upgrading to 0.5.0 section.
:email The email address of the recipient.
:full The whole recipient field (e.g., Some User <[email protected]>).
:name The name of the recipient (e.g., Some User).

Testing In Your App

You may want to create a factory for when testing the integration of Griddler into your application. If you're using factory_girl this can be accomplished with the following sample factory:

factory :email, class: OpenStruct do
  # Assumes Griddler.configure.to is :hash (default)
  to { [{ full: '[email protected]', email: '[email protected]', token: 'to_user', host: 'email.com', name: nil }] }
  from { { token: 'from_user', host: 'email.com', email: '[email protected]', full: 'From User <[email protected]>', name: 'From User' } }
  subject { 'email subject' }
  body { 'Hello!' }
  attachments { '0' }

  trait :with_attachment do
    attachments {[
      ActionDispatch::Http::UploadedFile.new({
        filename: 'img.png',
        type: 'image/png',
        tempfile: File.new("#{File.expand_path(File.dirname(__FILE__))}/fixtures/img.png")
      })
    ]}
  end
end

Bear in mind, if you plan on using the :with_attachment trait, that this example assumes your factories are in spec/factories.rb and you have an image file in spec/fixtures/.

To use it in your tests, build with email = build(:email) or email = build(:email, :with_attachment).

Adapters

Depending on the service you want to use Griddler with, you'll need to add an adapter gem in addition to griddler.

Service Adapter
sendgrid griddler-sendgrid
cloudmailin griddler-cloudmailin
mandrill griddler-mandrill
mailgun griddler-mailgun
postmark griddler-postmark
sparkpost griddler-sparkpost
ses (amazon) griddler-amazon_ses

Writing an Adapter

Griddler can theoretically work with any email => POST service. In order to work correctly, adapters need to have their POST parameters restructured.

Griddler::Email expects certain parameters to be in place for proper parsing to occur. When writing an adapter, ensure that the normalized_params method of your adapter returns a hash with these keys:

Parameter Contents
:to The recipient field
:from The sender field
:subject Email subject
:text The text body of the email
:html The html body of the email, or an empty string
:attachments Array of attachments to the email. Can be an empty array.
:headers The raw headers of the email. Optional.
:charsets A JSON string containing the character sets of the fields extracted from the message. Optional.

All keys are required unless otherwise stated.

Adapters should be provided as gems. If you write an adapter, let us know and we will add it to this README. See griddler-sendgrid for an example implementation.

Credits

Griddler was written by Caleb Hearth and Joel Oliveira.

Thanks to our contributors!

About thoughtbot

thoughtbot

This repo is maintained and funded by thoughtbot, inc. The names and logos for thoughtbot are trademarks of thoughtbot, inc.

We love open source software! See our other projects. We are available for hire.

griddler's People

Contributors

alexbaldwin avatar bradpauly avatar brentd avatar calebhearth avatar chrishunt avatar diego-aslz avatar dvanderbeek avatar e3matheus avatar emilford avatar gabebw avatar gxespino avatar heidi avatar jayroh avatar joeadcock avatar kle-roy avatar lulalala avatar mgodwin avatar nhocki avatar pbougie avatar petergoldstein avatar pftg avatar r38y avatar rickbutton avatar sauron avatar shime avatar stefannibrasil avatar steveklabnik avatar theycallmeswift avatar wingrunr21 avatar wireframe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

griddler's Issues

Body parse fails when there is no content before -----Original Message-----

NoMethodError: undefined method `gsub' for nil:NilClass

vendor/bundle/ruby/1.9.1/bundler/gems/griddler-a9a4458c87ae/lib/griddler/email_parser.rb:29

This should be a failing test case:

  it 'handles "-----Original Message-----" format' do
    body = <<-EOF
      -----Original Message-----
      From: [email protected]
      Sent: Today
      Subject: Awesome report.

      Check out this report!
    EOF

    body_from_email(:text, body).should eq ''
  end

Failures:

  1) Griddler::Email body formatting handles "-----Original Message-----" format
     Failure/Error: email = Griddler::Email.new(params).process
     NoMethodError:
       undefined method `gsub' for nil:NilClass
     # ./lib/griddler/email_parser.rb:29:in `extract_reply_body'
     # ./lib/griddler/email.rb:55:in `extract_body'
     # ./lib/griddler/email.rb:16:in `initialize'
     # ./spec/griddler/email_spec.rb:233:in `new'
     # ./spec/griddler/email_spec.rb:233:in `body_from_email'
     # ./spec/griddler/email_spec.rb:97:in `block (2 levels) in <top (required)>'

NoMethodError (undefined method `split' for nil:NilClass) in v0.6.3

This is my stack trace

vendor/bundle/ruby/2.0.0/gems/griddler-0.6.3/lib/griddler/email_parser.rb:59:in `extract_email_address'
vendor/bundle/ruby/2.0.0/gems/griddler-0.6.3/lib/griddler/email.rb:49:in `extract_address'
vendor/bundle/ruby/2.0.0/gems/griddler-0.6.3/lib/griddler/email_parser.rb:15:in `parse_address'
vendor/bundle/ruby/2.0.0/gems/griddler-0.6.3/app/controllers/griddler/emails_controller.rb:4:in `new'
vendor/bundle/ruby/2.0.0/gems/griddler-0.6.3/lib/griddler/email.rb:13:in `initialize'

Error when parsing "CC" from "message-headers" with Mailgun adapter

Mailgun adapter: receiving mail without CC field:

It seems params["message-headers"] is a string , not an array.


Started POST "/email_processor" for 127.0.0.1 at 2014-04-01 17:11:46 -1000
Processing by Griddler::EmailsController#create as HTML
  Parameters: {"recipient"=>"[email protected]",
  "sender"=>"[email protected]", "subject"=>"FW: xxx",
  "from"=>"\"R&D Recruit Mail\" ",
  "Received"=>"from aaaaa ([180.16.63.178])\tby 21CN-ent7-3(MEDUSA 10.27.101.7) with ESMTP id 1396407625.17611 for [email protected]\t;\tWed Apr 2 11:00:43 2014",
  "X-Envelope-From"=>"", "Hmm_source_ip"=>"10.27.101.7:36370.1075920735", "Hmm_attache_num"=>"0001",
  "Hmm_source_type"=>"SMTP", "0/x-Total-Score"=>"0:", "3/x-Brightmail-Tracker"=>"AAAAAA==", "X-Filter-Score"=>"to=<9595959594854f828594836195828d868f958d8a9495944f84908e>, score=<1396407643RPt4uhjjjjjTjj2jjLjT2LHEuf5Qbbbbb9bbCbbdb9Cd> ",
  "X-Real-From"=>"[email protected]", "X-Receive-Ip"=>"180.166.63.178 [email protected]",
  "From"=>"\"R&D Recruit Mail\" ",
  "To"=>"",
  "Subject"=>"FW: aaa", "Date"=>"Wed, 2 Apr 2014 11:00:37 +0800",
  "Message-Id"=>"<[email protected]>",
  "Mime-Version"=>"1.0",
  "Content-Type"=>"multipart/mixed; boundary=\"----=_NextPart_000_0013_01CF4E62.D00C1CD0\"",
  "X-Mailer"=>"Microsoft Office Outlook 12.0",
  "Thread-Index"=>"Ac9IL7jpKzuy9FReQkqj6X93yR9k4AF7/k6Q",
  "Content-Language"=>"zh-cn",
  "X-Mailgun-Incoming"=>"Yes",
  "X-Mailgun-Sflag"=>"No",
  "X-Mailgun-Sscore"=>"0.0",
  "X-Mailgun-Spf"=>"Neutral",
  "message-headers"=>"[[\"Received\", \"by luna.mailgun.net with SMTP mgrt 8766208669373; Wed, 02 Apr 2014 03:00:55 +0000\"], [\"X-Envelope-From\", \"\"], [\"Received\", \"from corp.21cn.com (corp.forptr.21cn.com [121.14.129.51]) by mxa.mailgun.org with ESMTP id 533b7d5e.7f508c1bd960-in2; Wed, 02 Apr 2014 03:00:46 -0000 (UTC)\"], ....]"
Completed 500 Internal Server Error in 0.5ms

NoMethodError (private method `select' called for #):
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:42:in `extract_header_cc'
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:36:in `ccs'
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:16:in `normalize_params'
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:10:in `normalize_params'
  griddler (0.6.4) app/controllers/griddler/emails_controller.rb:12:in `normalized_params'
  griddler (0.6.4) app/controllers/griddler/emails_controller.rb:3:in `create'
  actionpack (3.2.16) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  actionpack (3.2.16) lib/abstract_controller/base.rb:167:in `process_action'
  actionpack (3.2.16) lib/action_controller/metal/rendering.rb:10:in `process_action'
  actionpack (3.2.16) lib/abstract_controller/callbacks.rb:18:in `block in process_action'
  activesupport (3.2.16) lib/active_support/callbacks.rb:403:in `_run__3226707876278453871__process_action__698180469685376029__callbacks'
  activesupport (3.2.16) lib/active_support/callbacks.rb:405:in `__run_callback'
  activesupport (3.2.16) lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
  activesupport (3.2.16) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (3.2.16) lib/abstract_controller/callbacks.rb:17:in `process_action'

Extract SendGrid adapter

Due to the fact that thoughtbot don't use email POST services other than SendGrid, we've run into a problem with the other service adapters: they don't always work correctly, and they add a maintenance burden to the main project.

Because of this, we've determined that we want to extract all adapters from the Griddler gem into subgems, and direct users to add the specific gem for the service they use.

While I'm happy to continue maintaining griddler-sendgrid, if someone else wants to take that over as the lead I'm open to that as well. If any of you are interested, please let me know.

Logistics conversations is happening in #124.

Previous contributors to the adapter: @r38y and myself
SendGrid: @sendgrid
Also: @theycallmeswift

Invalid Byte Sequence Error UTF-8

On some incoming emails, I'm receiving the following error when using SendGrid inbound parse - > Griddler.

ArgumentError: invalid byte sequence in UTF-8

griddler/emails#create

vendor/bundle/ruby/1.9.1/gems/mail-2.4.4/lib/mail/core_extensions/string.rb:4

any ideas or suggestions on how to fix this?

here's some additional information from the backtrace:
vendor/bundle/ruby/1.9.1/gems/mail-2.4.4/lib/mail/core_extensions/string.rb:4:in gsub' vendor/bundle/ruby/1.9.1/gems/mail-2.4.4/lib/mail/core_extensions/string.rb:4:into_crlf'
vendor/bundle/ruby/1.9.1/gems/mail-2.4.4/lib/mail/header.rb:39:in initialize' vendor/bundle/ruby/1.9.1/gems/griddler-0.5.0/lib/griddler/email_parser.rb:45:innew'
vendor/bundle/ruby/1.9.1/gems/griddler-0.5.0/lib/griddler/email_parser.rb:45:in extract_headers' vendor/bundle/ruby/1.9.1/gems/griddler-0.5.0/lib/griddler/email.rb:59:inextract_headers'
vendor/bundle/ruby/1.9.1/gems/griddler-0.5.0/lib/griddler/email.rb:21:in `initialize'

NoMethodError (private method `select' called for #<String:0x00000006f2ef08>):

Any ideas why I get this when receiving a mailgun "Send A Sample POST" test email:

I, [2014-05-28T11:07:07.182971 #1296]  INFO -- : Completed 500 Internal Server Error in 3ms
F, [2014-05-28T11:07:07.189249 #1296] FATAL -- : 
NoMethodError (private method `select' called for #<String:0x00000006f2ef08>):
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:42:in `extract_header_cc'
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:36:in `ccs'
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:16:in `normalize_params'
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:10:in `normalize_params'
  griddler (0.6.4) app/controllers/griddler/emails_controller.rb:12:in `normalized_params'
  griddler (0.6.4) app/controllers/griddler/emails_controller.rb:3:in `create'
  actionpack (4.0.3) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  actionpack (4.0.3) lib/abstract_controller/base.rb:189:in `process_action'
  actionpack (4.0.3) lib/action_controller/metal/rendering.rb:10:in `process_action'
  actionpack (4.0.3) lib/abstract_controller/callbacks.rb:18:in `block in process_action'
  activesupport (4.0.3) lib/active_support/callbacks.rb:423:in `_run__16735415116720946__process_action__callbacks'
  activesupport (4.0.3) lib/active_support/callbacks.rb:80:in `run_callbacks'
  actionpack (4.0.3) lib/abstract_controller/callbacks.rb:17:in `process_action'
  actionpack (4.0.3) lib/action_controller/metal/rescue.rb:29:in `process_action'
  actionpack (4.0.3) lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
  activesupport (4.0.3) lib/active_support/notifications.rb:159:in `block in instrument'
  activesupport (4.0.3) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  activesupport (4.0.3) lib/active_support/notifications.rb:159:in `instrument'
  actionpack (4.0.3) lib/action_controller/metal/instrumentation.rb:30:in `process_action'
  actionpack (4.0.3) lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
  activerecord (4.0.3) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
  actionpack (4.0.3) lib/abstract_controller/base.rb:136:in `process'
  actionpack (4.0.3) lib/abstract_controller/rendering.rb:44:in `process'
  actionpack (4.0.3) lib/action_controller/metal.rb:195:in `dispatch'
  actionpack (4.0.3) lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
  actionpack (4.0.3) lib/action_controller/metal.rb:231:in `block in action'
  actionpack (4.0.3) lib/action_dispatch/routing/route_set.rb:80:in `call'
  actionpack (4.0.3) lib/action_dispatch/routing/route_set.rb:80:in `dispatch'
  actionpack (4.0.3) lib/action_dispatch/routing/route_set.rb:48:in `call'
  actionpack (4.0.3) lib/action_dispatch/journey/router.rb:71:in `block in call'
  actionpack (4.0.3) lib/action_dispatch/journey/router.rb:59:in `each'
  actionpack (4.0.3) lib/action_dispatch/journey/router.rb:59:in `call'
  actionpack (4.0.3) lib/action_dispatch/routing/route_set.rb:680:in `call'
  newrelic_rpm (3.8.1.221) lib/new_relic/rack/error_collector.rb:55:in `call'
  newrelic_rpm (3.8.1.221) lib/new_relic/rack/agent_hooks.rb:32:in `call'
  newrelic_rpm (3.8.1.221) lib/new_relic/rack/browser_monitoring.rb:27:in `call'
  warden (1.2.3) lib/warden/manager.rb:35:in `block in call'
  warden (1.2.3) lib/warden/manager.rb:34:in `catch'
  warden (1.2.3) lib/warden/manager.rb:34:in `call'
  rack (1.5.2) lib/rack/etag.rb:23:in `call'
  rack (1.5.2) lib/rack/conditionalget.rb:35:in `call'
  rack (1.5.2) lib/rack/head.rb:11:in `call'
  actionpack (4.0.3) lib/action_dispatch/middleware/params_parser.rb:27:in `call'
  actionpack (4.0.3) lib/action_dispatch/middleware/flash.rb:241:in `call'
  rack (1.5.2) lib/rack/session/abstract/id.rb:225:in `context'
  rack (1.5.2) lib/rack/session/abstract/id.rb:220:in `call'
  actionpack (4.0.3) lib/action_dispatch/middleware/cookies.rb:486:in `call'
  activerecord (4.0.3) lib/active_record/query_cache.rb:36:in `call'
  activerecord (4.0.3) lib/active_record/connection_adapters/abstract/connection_pool.rb:626:in `call'
  actionpack (4.0.3) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
  activesupport (4.0.3) lib/active_support/callbacks.rb:373:in `_run__3385064544836038165__call__callbacks'
  activesupport (4.0.3) lib/active_support/callbacks.rb:80:in `run_callbacks'
  actionpack (4.0.3) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  actionpack (4.0.3) lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
  actionpack (4.0.3) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
  actionpack (4.0.3) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
  railties (4.0.3) lib/rails/rack/logger.rb:38:in `call_app'
  railties (4.0.3) lib/rails/rack/logger.rb:20:in `block in call'
  activesupport (4.0.3) lib/active_support/tagged_logging.rb:67:in `block in tagged'
  activesupport (4.0.3) lib/active_support/tagged_logging.rb:25:in `tagged'
  activesupport (4.0.3) lib/active_support/tagged_logging.rb:67:in `tagged'
  railties (4.0.3) lib/rails/rack/logger.rb:20:in `call'
  actionpack (4.0.3) lib/action_dispatch/middleware/request_id.rb:21:in `call'
  rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
  rack (1.5.2) lib/rack/runtime.rb:17:in `call'
  activesupport (4.0.3) lib/active_support/cache/strategy/local_cache.rb:83:in `call'
  rack (1.5.2) lib/rack/sendfile.rb:112:in `call'
  railties (4.0.3) lib/rails/engine.rb:511:in `call'
  railties (4.0.3) lib/rails/application.rb:97:in `call'
  unicorn (4.8.3) lib/unicorn/http_server.rb:576:in `process_client'
  unicorn (4.8.3) lib/unicorn/http_server.rb:670:in `worker_loop'
  unicorn (4.8.3) lib/unicorn/http_server.rb:525:in `spawn_missing_workers'
  unicorn (4.8.3) lib/unicorn/http_server.rb:140:in `start'
  unicorn (4.8.3) bin/unicorn:126:in `<top (required)>'

Extract CloudMailin adapter

Due to the fact that thoughtbot don't use email POST services other than SendGrid, we've run into a problem with the other service adapters: they don't always work correctly, and they add a maintenance burden to the main project.

Because of this, we've determined that we want to extract all adapters from the Griddler gem into subgems, and direct users to add the specific gem for the service they use.

We'd love to get someone involved in that adapter gem who would be able to provide more knowledgeable maintenance of the adapter. I'm including mentions to everyone who's contributed to the adapter, as well as to the CloudMailin organization and the one person who I could find who was a member.

If any of you are interested, please let me know.

Subtask of #124

History of file (to determine churn and approximate workload)
Issues related to cloudmailin

Previous contributors to the adapter: @r38y @ErwinM
CloudMailin: @cloudmailin @scsmith

Different "On so and so dude wrote" in Gmail

I've been noticing that Gmail can use two different lines for some date that someone wrote something. One of them is already stripped by Griddler but the other is not.

This is what I'm talking about:

this is my awesome discussion - me r38y com - r38y com mail 2013-12-15 14-13-04

this is my awesome discussion - me r38y com - r38y com mail 2013-12-15 14-12-03

Have any of you encountered this?

Extract adapters to standalone gems

Adapters have proven to be:

  1. Embarrassing, as they seem to often be broken, and
  2. Difficult to maintain as neither I nor others at thoughtbot use the other 'supported' services.
  • Extract gems for each service, including SendGrid
    • Extract Sendgrid #130
    • Extract Mandrill #128
    • Extract CloudMailin #126
    • Extract Postmark #129
    • Extract Mailgun #127

I'm willing to continue working on griddler-sendgrid and on griddler proper, but after extraction I don't plan to continue actively maintaining the adapters and will request that those using them or representatives of the companies providing the service take over.

We'll add links to the various adapters in the README and make sure that people understand the division of responsibility as far as issues / pull requests.

config not accepted

Rails 4.1.1, griddler (0.6.4)

here is my file in initializers:

Griddler.configure do |config|
config.processor_class = EmailProcessor # MyEmailProcessor
config.processor_method = :process # :custom_method
config.to = :email # :full, :email, :token
config.cc = :full # :full, :hash, :token
config.from = :email # :full, :token, :hash

:raw => 'AppName [email protected]'

:email => '[email protected]'

:token => 's13.6b2d13dc6a1d33db7644'

:hash => { raw: [...], email: [...], token: [...], host: [...], name: [...] }

config.reply_delimiter = '-- REPLY ABOVE THIS LINE --'
config.email_service = :mandrill # :cloudmailin, :postmark, :mandrill, :mailgun
end

but when in rails console, I got this:

2.1.2 :001 > Griddler.configure.email_service
=> Griddler::Adapters::SendgridAdapter

What am I doing wrong?

Problems with names that contain comma

I know it little strange to have comma in name, but this what we got. To reproduce you need first create contact with name like an example in yours address book (at least I can't enter it right in mail app). Also possible emails without <> so, split by '>,' is not the option too.

Actual result:

> SendgridAdapter.normalize_params({ to: 'John M. Somebuddy,SOMECOMPANY <[email protected]>, [email protected]' })
=> {:to=>
  ["John M. Somebuddy",
   "SOMECOMPANY <[email protected]>",
   " [email protected]"],
 :cc=>[],
 :attachments=>[]}

Expected result:

> SendgridAdapter.normalize_params({ to: 'John M. Somebuddy,SOMECOMPANY <[email protected]>, [email protected]' })
=> {:to=>
  ["John M. Somebuddy,SOMECOMPANY <[email protected]>",
   "[email protected]"],
 :cc=>[],
 :attachments=>[]}

Undefined method error when email body is empty

I have successfully integrated Griddler into my app, and it works perfectly as long as the email body has some text in it.

However when I test with an email with an empty body, I keep on getting the following error from the email_parser (which looks to be because the body is empty):

NoMethodError (undefined method split' for nil:NilClass):
griddler (0.3.1) lib/griddler/email_parser.rb:26:in extract_reply_body' griddler (0.3.1) lib/griddler/email.rb:63:in extract_body'
griddler (0.3.1) lib/griddler/email.rb:12:in initialize' griddler (0.3.1) app/controllers/griddler/emails_controller.rb:3:in new'
griddler (0.3.1) app/controllers/griddler/emails_controller.rb:3:in create'

--Edit--

After some investigating, this appears to be happening because of the way the format is set in the params in the incoming email. Specifically, when the email body has content, in the params the format is set as "text"=>"String\n", however when the body is empty, this becomes "text"=>"".

This would seem to be the root cause, because in my rspec tests,
post :create, to: "email-token", from: "[email protected]", text: "some string" passes, but post :create, to: "email-token", from: "[email protected]", text: "" fails.

Extract Mandrill adapter

Due to the fact that thoughtbot don't use email POST services other than SendGrid, we've run into a problem with the other service adapters: they don't always work correctly, and they add a maintenance burden to the main project.

Because of this, we've determined that we want to extract all adapters from the Griddler gem into subgems, and direct users to add the specific gem for the service they use.

We'd love to get someone involved in that adapter gem who would be able to provide more knowledgeable maintenance of the adapter. I'm including mentions to everyone who's contributed to the adapter, as well as to the MailChimp organization and the few members I saw on the organization page.

If any of you are interested, please let me know.

Subtask of #124

History of file (to determine churn and approximate workload)
Issues related to Mandrill

Previous contributors to the adapter: @wingrunr21 @pbougie @lagartoflojo and @r38y
MailChimp: @mailchimp @amro @stephenmartin @stevenosloan

Not setting config values in Rspec

So I am running tests on my EmailProcessor class, but for some reason despite loading Griddler, the configuration is being initialized to the default values rather than what I set them to.

My spec helper runs this:

require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
require 'rspec/autorun'

My config works in dev/production, but when I try and print out Griddler.configuration.xxx in EmailProcessor when it's run from a spec it just outputs the defaults and ignores the initializer.

Still relatively new to rspec, is there something I'm missing? Thanks in advance for the help

Griddler config undefined method processor_class

I created config/initializers/griddler.rb per the github's instructions:

Griddler.configure do |config|
config.processor_class = EmailProcessor
config.to = :email # :full, :email, :hash
config.reply_delimiter = '-- REPLY ABOVE THIS LINE --'
config.email_service = :mandrill
end
On running I get this error:

/app/config/initializers/griddler.rb:2:in block in <top (required)>': undefined methodprocessor_class=' for #Griddler::Configuration:0x00000004b1eb18 (NoMethodError)
The file is located at ./config/initializers/griddler.rb . I'm not sure why it shows /app there.

http://stackoverflow.com/questions/16952206/griddler-config-undefined-method-processor-class

Empty reply from server with attachment

Hi,

I am trying to receive emails with attachment via mandrill but I have some troubles: Mandrill is always failing with this message:

Error: POST to http://56d3.localtunnel.com/email_processor failed: Empty reply from server

Everything works fine when I haven't any attachment.

Any ideas?

Handle email with no body

I know this was partially addressed by #28 and #29. If an email is sent with an attachment only though, I see an EmailBodyNotFound exception. Our use case is using email to upload files to a registered account, where the existence of the body is optional (as a note for the uploaded file).

I have a patch that just removes this entirely, if there's interest in pulling it. Since it deletes this exception entirely, there is no test for it. The test case that expects the exception is also deleted so that the existing suite passes.

the adapter for mailgun does not parse the several parts of the recipient email addresses correctly

Thanks bradpauly for doing the mailgun adapter (here: #84 ). Could I submit a small bug report? It has to do with sending emails to "full" email addresses. The bug occurs with 1 or multiple recipients but I will do 2 recipients here:

If you send an email to (two recipients, each with a "full" address):
Monkey Q. Baby [email protected]
Foo B. Baz [email protected]

the current mailgun adapter returns an email.to (inside the EmailProcessor self.process(email) method) of:
[{:token=>"mp7", :host=>"stone-house.org", :email=>"[email protected]", :full=>"[email protected]", :name=>nil}, {:token=>"mep", :host=>"stone-house.org", :email=>"[email protected]", :full=>" [email protected]", :name=>nil}]

The problem is that the :name is nil, and :full is not correct (:full should be the whole thing with the human-readable name plus the email address as submitted).

Instead, we want email.to to be like this:
[{:token=>"mep", :host=>"stone-house.org", :email=>"[email protected]", :full=>""Monkey Q. Baby" [email protected]", :name=>""Monkey Q. Baby""}, {:token=>"mp7", :host=>"stone-house.org", :email=>"[email protected]", :full=>""Foo B. Baz" [email protected]", :name=>""Foo B. Baz""}]

You can get that result with a single line change in mailgun_adapter.rb. Change line 28 from:
params[:recipient].split(',')
to:
params["To"].split(',').map(&:strip)

Note, I also added a :strip because, if you notice in the output of the current version of the mailgun adapter, there is a leading space before the email address in :full=>" [email protected]" .

Thanks!

P.S. Note, this bug is only evident when you set config.to = :hash in Griddler.configure .

Removing attachments associated with a signature

So, I'm a little worried about littering my app with images that are just part of the signature. I haven't looked too far into how this works but I'm willing to take this on. Do you think it would be worthwhile?

How do you think we should do it? Add another parameter to the attachment hash indicating it might be part of the signature. This way people could decide for themselves?

Extract Postmark

Due to the fact that thoughtbot don't use email POST services other than SendGrid, we've run into a problem with the other service adapters: they don't always work correctly, and they add a maintenance burden to the main project.

Because of this, we've determined that we want to extract all adapters from the Griddler gem into subgems, and direct users to add the specific gem for the service they use.

We'd love to get someone involved in that adapter gem who would be able to provide more knowledgeable maintenance of the adapter. I'm including mentions to everyone who's contributed to the adapter, as well as to the Wildbit organization and a few members who seemed to be involved in the ruby portions.

If any of you are interested, please let me know.

Subtask of #124

History of file (to determine churn and approximate workload)
Issues related to Postmark

Previous contributors to the adapter: @r38y @nhocki
Wildbit: @wildbit @temochka @isabanin

Remove "[image: Inline image 1]" from bodies

Hey there, it looks like gmail inserts "[image: Inline image 1]" into an email body if one of the attachments is just dragged and dropped into the middle of the body. They come through as attachments so I was wondering if we could strip these out of the body.

Generator for example files?

I just started using Griddler and wondering if this is a functionality you guys would like to add?

Most gems that require initializers and libraries have a generator to construct these example files.

I'd gladly submit a pull request.

Inconsistent reply_delimiter default with README display

The reply_delimiter default value is "Reply ABOVE THIS LINE" in the library but in the README its shown first as "-- Reply ABOVE THIS LINE --" and finally (in the configuration block example) as "-- REPLY ABOVE THIS LINE --". I can submit a pull request to address this but I wasn't sure whether to correct the default value in the library or to fix the inconsistencies in the README?

raw_html is empty

Hi, I using cloudmailin and I am finding that the raw_html parameter is empty. I can see the html data in the POST message data but when I try to process email.raw_html it is not present.

My initialiser looks like this;

require 'email_processor'

Griddler.configure do |config|
  config.processor_class = EmailProcessor # MyEmailProcessor
  config.processor_method = :process # :custom_method
  config.to = :hash # :full, :email, :token
  config.cc = :email # :full, :hash, :token
  config.from = :email # :full, :token, :hash
  # :raw    => 'AppName <[email protected]>'
  # :email  => '[email protected]'
  # :token  => 's13.6b2d13dc6a1d33db7644'
  # :hash   => { raw: [...], email: [...], token: [...], host: [...], name: [...] }
  config.reply_delimiter = '-- REPLY ABOVE THIS LINE --'
  config.email_service = :cloudmailin # :sendgrid, :postmark, :mandrill, :mailgun
end

Any help wold be much appreciated.
Neil

Extract Mailgun adapter

Due to the fact that thoughtbot don't use email POST services other than SendGrid, we've run into a problem with the other service adapters: they don't always work correctly, and they add a maintenance burden to the main project.

Because of this, we've determined that we want to extract all adapters from the Griddler gem into subgems, and direct users to add the specific gem for the service they use.

We'd love to get someone involved in that adapter gem who would be able to provide more knowledgeable maintenance of the adapter. I'm including mentions to everyone who's contributed to the adapter, as well as to the Mailgun organization.

If any of you are interested, please let me know.

Subtask of #124

History of file (to determine churn and approximate workload)
Issues related to Mailgun

Previous contributors to the adapter: @r38y @bradpauly
Mailgun: @mailgun

ArgumentError (invalid byte sequence in UTF-8)

In accepting emails from SendGrid I was receiving some invalid byte sequences in the raw_html incoming field. The technique at http://robots.thoughtbot.com/fight-back-utf-8-invalid-byte-sequences didn't work for me without forcing the encoding to ISO-8859-1 first. I ended up solving this error by defining this method in email_processor.rb:

def self.to_utf8(str)
  str = str.force_encoding("UTF-8")
  return str if str.valid_encoding?
  str = str.force_encoding("ISO-8859-1")
  str.encode("UTF-8", invalid: :replace, undef: :replace)
end

and then using it like so inside the self.process(email) method:

# Save the email as a letter
letter = Letter.new do |l|
  l.to = email.to
  l.from = email.from
  l.cc = email.cc
  l.subject = email.subject
  l.body = to_utf8(email.body) if email.body.present?
  l.raw_text = to_utf8(email.raw_text) if email.raw_text.present?
  l.raw_html = to_utf8(email.raw_html) if email.raw_html.present?
  l.raw_body = to_utf8(email.raw_body if email.raw_body.present?
  l.headers = email.headers
end
letter.save

Multiple delimiter strings

Hey, if I added the option to pass an array of delimiter strings, would you accept it? I want to switch split strings but need to support the old one.

I think I would turn them into a regex that would get passed into split instead of just the single specified string.

Extracting inline images from MMS

Hey

Thanks for all your hard work on Griddler.

I've got it working nicely with emails and "proper" attachments, but I'm trying to get it to work with MMS messages sent to an email address.

Using pry, I can see that MMS messages have inline images that are base64 encoded in the raw_body of the params.

Has anybody got any suggestions as to how I can try to extract this encoded image and convert it to a "real" image file, that I can associate with an object in my app?

Not sure if this question belongs here or it's more for StackOverflow. Just looking for some pointers really.

Thank you
Chris

parsing questions

Just recently starting using griddler and I'm noticing that in a lot of cases it isn't excluding things from the original message.

Example:

John Smith responded with this note.


Sent via the Samsung Galaxy S™ III, an AT&T 4G LTE smartphone

-------- Original message --------
From: [email protected] 
Date: 
To: [email protected] 
Subject: My New Subject 

Shouldn't it be checking for things like Original Message... or From: [email] ?

Message Forwarding

How would I use ActionMailer to just forward this straight along - attachments and all ? Thanks!

Mailgun support?

Just found out about griddler via ruby5, looks awesome.

I spent last week building inbound message support into our application. Till now i'd planned to extract the code and release it as a gem. Implementation is crazy close to griddler, even the naming conventions. For example we've got Mailgun::Message and MessageProcessors.

Thought i'd take a stab at using our code to add mailgun support to griddler, wanted to see 1) if that makes sense and 2) if anyone has feedback before I get started...

UTF-8 encoding works as expected, not desired

Hi,
I will be processing a lot of email written in Spanish, so I need to keep accented characters, etc.
The clean_invalid_utf8_bytes uses 'binary' to encode the messages by default, and otherwise falls back to params[:charsets]. The problem is that, as noted in issue #27, I am losing all accented characters (Hernán comes out as Hernn), and not all adapters provide charset information (Mandrill, which we use, doesn't).
I'm not sure how to solve this issue... The ideal solution would be to parse the raw email and extract the charsets from there, but that's a pain.
Maybe adding a config to set the default encoding, in order to override 'binary'?
Thanks!

Routing problems with and without route helper

When I use post '/email_processor' => 'griddler/emails#create', I get the following error:

Invalid route name, already in use: 'email_processor'  (ArgumentError)
You may have defined two routes with the same name using the `:as` option, or you may be overriding a route already defined by a resource with the same naming. For the latter, you can restrict the routes created with `resources` as explained here:
http://guides.rubyonrails.org/routing.html#restricting-the-routes-created

When I use the route helper mount_griddler without any arguments, I get this error:

undefined local variable or method `mount_griddler' for #<ActionDispatch::Routing::Mapper:0x007f95a6d78f50> (NameError)

Clearly not desired behavior, but not really sure why this might be happening. Any thoughts?

undefined method error when adding config file

I'm super exited about griddler but having some issues getting it to run properly. I'v installed the gem on both rails 3 and 4 in a simple test app with a minimal gemset. I added the EmailProcessor class in my models (same as in the demo app).

The issue I'm having occurs when I add the custom config file griddler.rb in my config/initializers folder. when I start my server, rails throws back the following error in my console:

/Users/username/code/appname/config/initializers/griddler.rb:3:in block in <top (required)>': undefined methodprocessor_method=' for #Griddler::Configuration:0x007f9b7d9e5438 (NoMethodError)

from /Users/username/.rvm/gems/ruby-2.0.0-p353@rails4/gems/griddler-0.6.1/lib/griddler/configuration.rb:8:in `configure'

from /Users/username/code/appname/config/initializers/griddler.rb:1:in `<top (required)>'
etc... (long stack of error messages

Parsed headers support

Would this be desired? It could be baked in by default or as an option to parse the headers string from Sendgrid using Mail::Header.new(email.headers). It's not hard to do it yourself if you want it, but does take some knowledge of the underlying tools, the Mail gem and it's api, to accomplish it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.