Google app engine: NullpointerException while storing image in Blobstore through a java servlet

I'm trying to upload images via a JSP file upload and a validation servlet to the Blobstore. the JSP part is as follows:

<form action="/testuploadmimevalidation?provider-key=testprovider" method="post" enctype="multipart/form-data">
    <input type="text" name="foo">
    <input type="file" name="myfile" >
    <input type="submit" value="Submit">
</form>

The java class TestUploadMimeValidation is as follows:

public class TestUploadMimeValidation extends HttpServlet {

    private BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
    private static final Logger log = Logger.getLogger(TestUploadMimeValidation.class.getName());
    private static final boolean PRODUCTION_MODE = SystemProperty.environment.value() == SystemProperty.Environment.Value.Production;
    private static final String URL_PREFIX = PRODUCTION_MODE ? "" : "http://127.0.0.1:8080";

    public void doPost(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException {
        InputStream in = req.getInputStream();
        int formDataLength = req.getContentLength();
        byte dataBytes[] = new byte[formDataLength];
        int len;
        ByteArrayOutputStream bos = new ByteArrayOutputStream();

        while ((len = in.read(dataBytes, 0, formDataLength)) != -1)
            bos.write(dataBytes, 0, len);

        dataBytes = bos.toByteArray();
        String urlStr = URL_PREFIX + BlobstoreServiceFactory.getBlobstoreService().createUploadUrl("/testupload");
        URLFetchService urlFetch = URLFetchServiceFactory.getURLFetchService();
        HTTPRequest request = new HTTPRequest(new URL(urlStr), HTTPMethod.POST, FetchOptions.Builder.withDeadline(20.0));
        request.setHeader(new HTTPHeader("Content-Type", "multipart/form-data"));
        request.setPayload(dataBytes);

        System.out.println("step1");

        try {
            HTTPResponse response = urlFetch.fetch(request);
            System.out.println("step2");
        } catch (IOException e) {

        } catch (NullPointerException e) {

        }

        System.out.println("step3");
    }
}

This class uploads the image 开发者_StackOverflowto the Blobstore just fine, but I get a NullPointerException. the stacktrace is as follows:

WARNING: /_ah/upload/ag5tOGJ5dXMtZGV2ZWxvcHIbCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGAIM
java.lang.NullPointerException
        at javax.mail.internet.MimeMultipart.writeTo(MimeMultipart.java:143)
        at com.google.appengine.api.blobstore.dev.UploadBlobServlet.handleUpload(UploadBlobServlet.java:180)
        at com.google.appengine.api.blobstore.dev.UploadBlobServlet.access$000(UploadBlobServlet.java:72)
        at com.google.appengine.api.blobstore.dev.UploadBlobServlet$1.run(UploadBlobServlet.java:101)
        at java.security.AccessController.doPrivileged(Native Method)
        at com.google.appengine.api.blobstore.dev.UploadBlobServlet.doPost(UploadBlobServlet.java:98)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
        at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:58)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:122)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
        at com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:70)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:351)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Because of this NPE my code in the TestUpload.java class does not get executed. The printlns come out fine. It prints "step3". The image gets stored into the Datastore. But I just can't get the code in the TestUpload to run. Do you have any idea what might be causing the exception? I've tried to mess with the multipart, but wasn't very succesful.

Any help with this issue would be really appreciated.

---

I use

<form action="<%= blobstoreService.createUploadUrl("/catalog/actions/add") %>" method="post" enctype="multipart/form-data" >

---

First you can see how to do your POST request here: Using java.net.URLConnection to fire and handle HTTP requests

String param = "value";
File textFile = new File("/path/to/file.txt");
File binaryFile = new File("/path/to/file.bin");
String boundary = Long.toHexString(System.currentTimeMillis()); // Just
// generate some unique random value.
String CRLF = "\r\n"; // Line separator required by multipart/form-data.
URLConnection connection = new URL(url).openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type",
    "multipart/form-data; boundary=" + boundary);
PrintWriter writer = null;
try {
    OutputStream output = connection.getOutputStream();
    writer = new PrintWriter(new OutputStreamWriter(output, charset),
            true); // true = autoFlush, important!
    // Send normal param.
    writer.append("--" + boundary).append(CRLF);
    writer.append("Content-Disposition: form-data; name=\"param\"")
            .append(CRLF);
    writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF);
    writer.append(CRLF);
    writer.append(param).append(CRLF).flush();
    // Send text file.
    writer.append("--" + boundary).append(CRLF);
    writer.append(
        "Content-Disposition: form-data; name=\"textFile\"; filename=\""
            + textFile.getName() + "\"").append(CRLF);
    writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF);
    writer.append(CRLF).flush();
    BufferedReader reader = new BufferedReader(new InputStreamReader(
            new FileInputStream(textFile), charset));
    try {
        for (String line; (line = reader.readLine()) != null;) {
            writer.append(line).append(CRLF);
        }
    } finally {
        try { reader.close(); } catch (IOException logOrIgnore) {}
    }
    writer.flush();
    // Send binary file.
    writer.append("--" + boundary).append(CRLF);
    writer.append(
        "Content-Disposition: form-data; name=\"binaryFile\"; filename=\""
            + binaryFile.getName() + "\"").append(CRLF);
    writer.append(
        "Content-Type: "
            + URLConnection.guessContentTypeFromName(binaryFile
                .getName())).append(CRLF);
    writer.append("Content-Transfer-Encoding: binary").append(CRLF);
    writer.append(CRLF).flush();
    InputStream input = new FileInputStream(binaryFile);
    try {
        byte[] buffer = new byte[1024];
        for (int length = 0; (length = input.read(buffer)) > 0;) {
            output.write(buffer, 0, length);
        }
        output.flush(); // Important! Output cannot be closed. Close of
                        // writer will close output as well.
    } finally {
        try { input.close(); } catch (IOException logOrIgnore) {}
    }
    writer.append(CRLF).flush(); // CRLF is important! It indicates end
                                    // of binary boundary.
    // End of multipart/form-data.
    writer.append("--" + boundary + "--").append(CRLF);
} finally {
    if (writer != null) writer.close();
}

I tried with apache http-client library and it worked but only locally. Because of socket limitation we can't use it. https://developers.google.com/appengine/docs/java/sockets/#limitations-and-restrictions We are stucked to use: java.net.URL; java.net.URLConnection;

For images use binary from the link above. It was pain in the ass till I get working code :)

---

TL;DR: For those who got this specfically weird error when trying to use AJAX to upload instead of the traditional form submission route, the solution is to NOT set the "Content-Type" header when creating the AJAX request. Upon removing this, it started working for me.

More info: Apparently, passing in a FormData object as the body for the AJAX request will automatically set the "Content-Type" header with "multipart/form-data", as well the required boundary string (which is what would be missing if we just set the content-type ourselves).

Additional info: This boundary string is, AFAIK, randomly generated, and we cannot define our own boundary via the "Content-Type" header (I've tried, and the boundary used in the body is still the randomly generated string, causing a heap space error). The AJAX request must be assigning the randomly generated boundary string in the header and setting our own content type is overwriting that header.

I got the clue from here: html form enctype on ajax (though jQuery related, the logic of setting the Content-type header is native webkit/browser logic)