How to handle incoming emails with multiple large attachments from a Rails application?

I need to setup a Ruby / Rails application whi开发者_高级运维ch can accept incoming emails, the body of the emails will be ignored, each email will have one or more large attachments (graphics) up to 20MB in size, the app will receive approx 10-20 of these emails an hour.

I need to use a custom email address / domain.

Quick processing of the emails is not a big priority, however each email should get handled in at most 5 minutes after it arrives.

I was thinking of having all emails automatically forwarded from my info@mydomain.com mail account to a GMail inbox, then have a Rails lib script use MMS2R & TMail access GMail, download any new mails & add them to a message queue (delayed job), this script would be invoked by a cron job once a minute.

This solution just does not "smell right" & I would be concerned about its ability to scale; I don't want to load up the whole Rails stack each time the cron / script is run.

Does anyone have experience with incoming mail & attachment processing in Rails?

---

If your main concern is about loading rails environment every minute to run the script, I would suggest mailgun along with some cron alternatives.

I use a combination of bluepill + clockwork + delayed_job to handle this. I have custom jobs, enqueing which is as simple as Delayed::Job.enqueue MyJob.new. The logic is captured in job's perform method, which helps minimize dependencies if there are any.

clockwork is used to enqueue delayed jobs only loading minimal dependencies, keeping the memory footprint minimal.

bluepill is used to monitor the delayed workers and clockwork process and start them back up if they fail or restart if they are getting out of control.

General memory usage: 20MB(bluepill) + 25MB(clockwork) + 80-90MB(delayed worker)

Example of clockwork.rb:

ENV['RAILS_ENV'] ||= "development"
ENV['RACK_ENV'] = ENV['RAILS_ENV']

require 'clockwork'
require 'mongoid'
require 'delayed_job'
require 'delayed_job_mongoid'

include Clockwork

base_directory = File.absolute_path(File.dirname(__FILE__))
require File.join(base_directory, "../jobs/fetch_comments_job.rb")
require File.join(base_directory, "../jobs/news_letter_job.rb")

Mongoid.load!(File.join(base_directory, "../config/mongoid.yml"))

every(1.minute, "pop.comments") { Delayed::Job.enqueue FetchCommentsJob.new}
every(1.day, "send.newsletter", :at => '22:30') { Delayed::Job.enqueue NewsLetterJob.new}

---

I wrote a blog post about a number of possible options for receiving email in Rails. The cron approach works but but there can be a number of issues if the processing takes longer than your suggested 5 minutes and your then spinning up another instance whilst the old one is still going.

The alternatives look at how you can send the email directly to your web app so that you can use the existing app rather than having to spin up another instance each time the cron task runs. There are obviously potential downsides to doing this too but systems like CloudMailin allow you to send the attachments directly to S3 to get around this. You can then just use a background processing system to download the attachments and resize them.

I also had to modify paperclip a little to get it to resize the attachments.