app engine incoming mail handling

I am developing a Google App Engine application.

I wish to receive mails under '%username%@appid.appspotmail.com', where %username% belongs to a user of the application.

I just can't figure out what to define in web.xml file.

Any similar solution such as mails to:

• '%username%.usermailbox@appid.appspotmail.com'
• 'usermailbox.%username%@appid.appspotmail.com'

is acceptable (if it makes it easier with the wildcards).

I've tried (as su开发者_C百科ggested by Gopi)

mapping the relevant servlet to <url-pattern>/_ah/mail/user.*</url-pattern> within the web.xml file. It's not working.

The client gets a bounce message, whereas the server logs, do show a relevant request received by the app, but rejected with a 404. No "No handlers matched this URL." INFO is added to the log entry. In addition, when GETing the generated URL, I don't get a 'This page does not support GET', but rather a plain 404.

If I however send mail to say 'info@appid.appspotmail.com', the logs show a 404 (which they should, as it's not mapped in the web.xml). In addition, for such a request, a "No handlers matched this URL." INFO is added to the relevant log entry.

Needless to say that, Incoming mail IS found under Configured Services.

---

This change happened when App Engine started using a true Java web server (and so Toby's explanation is spot on... sadly I can't seem to recover my login to vote it up!). My recommendation is to use a Filter. I played around with the filter below when writing a toy app for GAE. You once you've defined the base class at the end of this post, you can can create a series of mail handlers (like the following). All you have to do is register each filter in your web.xml to handle /_ah/mail/*.

public class HandleDiscussionEmail extends MailHandlerBase {

  public HandleDiscussionEmail() { super("discuss-(.*)@(.*)"); }

  @Override
  protected boolean processMessage(HttpServletRequest req, HttpServletResponse res)
    throws ServletException 
  { 
    MimeMessage msg = getMessageFromRequest(req); 
    Matcher match = getMatcherFromRequest(req);
    ...
 }

}

---

public abstract class MailHandlerBase implements Filter {

  private Pattern pattern = null;

  protected MailHandlerBase(String pattern) {
    if (pattern == null || pattern.trim().length() == 0)
    {
      throw new IllegalArgumentException("Expected non-empty regular expression");
    }
    this.pattern = Pattern.compile("/_ah/mail/"+pattern);
  }

  @Override public void init(FilterConfig config) throws ServletException { }

  @Override public void destroy() { }

  /**
   * Process the message. A message will only be passed to this method
   * if the servletPath of the message (typically the recipient for
   * appengine) satisfies the pattern passed to the constructor. If
   * the implementation returns <code>false</code>, control is passed
   * o the next filter in the chain. If the implementation returns
   * <code>true</code>, the filter chain is terminated.
   *
   * The Matcher for the pattern can be retrieved via
   * getMatcherFromRequest (e.g. if groups are used in the pattern).
   */
  protected abstract boolean processMessage(HttpServletRequest req, HttpServletResponse res) throws ServletException;

  @Override
  public void doFilter(ServletRequest sreq, ServletResponse sres, FilterChain chain)
      throws IOException, ServletException {

    HttpServletRequest req = (HttpServletRequest) sreq;
    HttpServletResponse res = (HttpServletResponse) sres;

    MimeMessage message = getMessageFromRequest(req);
    Matcher m = applyPattern(req);

    if (m != null && processMessage(req, res)) {
      return;
    }

    chain.doFilter(req, res); // Try the next one

  }

  private Matcher applyPattern(HttpServletRequest req) {
    Matcher m = pattern.matcher(req.getServletPath());
    if (!m.matches()) m = null;

    req.setAttribute("matcher", m);
    return m;
  }

  protected Matcher getMatcherFromRequest(ServletRequest req) {
    return (Matcher) req.getAttribute("matcher");
  }

  protected MimeMessage getMessageFromRequest(ServletRequest req) throws ServletException {
    MimeMessage message = (MimeMessage) req.getAttribute("mimeMessage");
    if (message == null) {
      try {
        Properties props = new Properties();
        Session session = Session.getDefaultInstance(props, null);
        message = new MimeMessage(session, req.getInputStream());
        req.setAttribute("mimeMessage", message);

      } catch (MessagingException e) {
        throw new ServletException("Error processing inbound message", e);
      } catch (IOException e) {
        throw new ServletException("Error processing inbound message", e);
      }
    }
    return message;
  }



}

---

the following provides a plausable explanation, thanks to url-pattern and wildcards which refers to http://jcp.org/aboutJava/communityprocess/mrel/jsr154/index2.html (scroll to section 11.2)

In the url-pattern the * wildcard behaves differently to how one would assume, it is treated as a normal character, except -when the string ends with /* for "path mapping" -or it begins with *. for "extension mapping"

Too bad, would have been nice to wildcard-match email recipient addresses to different servlets, as depicted in Google's API doc samples. I'm using absolute matches now which isn't as clean as the appid needs to be included.

---

I think putting an entry similar to below into your web.xml should work to match your second case 'usermailbox.%username%@appid.appspotmail.com

<servlet>
  <servlet-name>handlemail</servlet-name>
  <servlet-class>HandleMyMail</servlet-class>
</servlet>
<servlet-mapping>
  <servlet-name>handlemail</servlet-name>
  <url-pattern>/_ah/mail/usermailbox.*</url-pattern>
</servlet-mapping>

---

Well... After trying every possible solution/url-mapping, I went with fast and ugly one.
The gist is to have a single "catch all" mail servlet, to work as a dispatcher to other, specific, servlets. It's like a giant switch, where the parameter is the request URL.
This is NOT what I wished for, but it works, and seems to be the only one that does.

I have a single servlet IncomingMail that handles ALL incoming mail. period.
So now, the only mapping of URLs under /_ah/mail/ is the following:

<servlet>
    <servlet-name>IncomingMail</servlet-name>
    <servlet-class>IncomingMail</servlet-class>
</servlet>
<servlet-mapping>
    <servlet-name>IncomingMail</servlet-name>
    <url-pattern>/_ah/mail/*</url-pattern>
</servlet-mapping>

In addition, I have the following servlet, mapped as a "plain-old-servlet":
(notice the <url-pattern>, not a "mail mapped" servlet)

<servlet>
    <servlet-name>GetUserMail</servlet-name>
    <servlet-class>GetUserMail</servlet-class>
</servlet>
<servlet-mapping>
    <servlet-name>GetUserMail</servlet-name>
    <url-pattern>/serv/userMail</url-pattern>
</servlet-mapping>

The catch-all servlet (would eventually) look like a giant switch:

public class IncomingMail extends HttpServlet {
    private final String USER_MAIL_PREFIX="http://appid.appspot.com/_ah/mail/user.";
    private final String USER_MAIL_SERVLET="/serv/userMail";
    ...
    public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
        String url = req.getRequestURL().toString();
        System.out.println("IncomingMail called, with URL: "+url);
        String email;
        String servlet;

        if (url.startsWith(USER_MAIL_PREFIX)) {
            email=url.replace(USER_MAIL_PREFIX, "");
            servlet=USER_MAIL_SERVLET;
        }//userMail 
        if (url.startsWith(OTHER_PREFIX)) {
            //Redirect to OTHER servlet
        }
        ...
        System.out.println("forward to '"+servlet+"', with email '"+email+"'");
        RequestDispatcher dispatcher=req.getRequestDispatcher(servlet);
        try {
            req.setAttribute("email", email);
            dispatcher.forward(req, resp);
        } catch (ServletException e) {              
            System.err.println(e);
        }           

    }
}

The destination servlet (GetUserMail in this case), does a getRequestParameter("email"), to see the specific destined mailbox.
It will receive all mails sent to 'user.%un%@appid.appspotmail.com', where %un% is a username in the application space.
The email parameter received by the servlet would be of the form '%un%@appid.appspotmail.com', without the discerning prefix.
Each such "specific" servlet, would get "its cut" from the mail dispatcher servlet, with the email parameter already without the discerning prefix.

One note I will add under security:
If you're worried of bogus requests to the "specific servlets", just define them all under a common virtual namespace say /servmail/ in your site, and define a new <security-constraint> to allow requests to originate only within the application itself.
Like so (inside web.xml):

    <security-constraint>
        <web-resource-collection>
            <web-resource-name>MailServlets</web-resource-name>
            <description>policy for specific mail servlets</description>
            <url-pattern>/servmail/*</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>admin</role-name>
        </auth-constraint>
    </security-constraint>

Would still love to hear from someone that tried and succeeded in doing a wildcard <url-pattern> mail mapping, other than a catch-all one.

---

I had a similar problem (using Python, so yaml config files rather than XML) and the cause turned out to be because I put the:

- url: /_ah/mail/.+ 
  script: handle_incoming_email.py 
  login: admin

before an existing catch-all entry:

- url: /.*
  script: main.py

This gave 404s on the server and "Message send failure" when sending test messages.

Moving it after the catch-all entry solved the problem.

---

I'm fairly sure the problem is just that you're trying to use .*. URL expressions in web.xml are globs, not regular expressions, so you should use just * instead - .* will only match strings starting with a dot.