Design Advice on Image Processing Architecture

I'm working on a stock-standard, ASP.NET MVC 3 web application (hosted on IIS 7). The site allows users to upload photos, among other things.

The upload process is as follows:

1. User makes use of widget (currently plupload) to select files from their PC.
2. AJAX call happens to my server, with image in HTTP POST (Request.Files)
3. Server resizes photo N amount of times
4. Each resized photo is uploaded to Amazon S3

At the moment, the above is implemented with a "fire and forget" technique using .NET 4.0's TPL.

I would like to make the above more flexible and robust. For example, if the image processing fails (it's using GDI, so it's likely), or S3 is down (which happens), i or the user won't know about it.

I'm thinking about hosting a WCF service as a Windows Service, which polls a folder for images.

My main website would simply FTP the image to the "watched" folder, then the service would take care of the image processing and the uploading.

The user doesn't need to be notified "immediately" that the photo is done. In other words, right now we show a "your image is being processed an开发者_如何学Cd will be available shortly" message.

To sum up, the service needs to:

1. Resize images
2. Upload images to S3
3. Read/write to database
4. Ability to "retry" failed images

Any advice? Is FileSystemWatcher a good option?

---

In my current project we implemented a similar middleware service responsible for data processing using FileSystemWatcher with relative success. Some things to remember about:

1. Be sure to implement some sort of queueing for core processing. Starting 100 image conversion processes at the same time is not a good idea. Consider using a ThreadPool.
2. FileSystemWatcher will give notifications as soon as the file gets created, at which point it may still be write-only locked - you will have to perform periodic checks to determine the right moment to start processing. Probably using a main loop and a queue.
3. Keep track of finely grained status changes (like file_created, file_processing, file_processed, file_uploading etc). You might really need them for debugging.

Hope this helps and good luck.