Absurd connection timeout in HttpURLConnection in Java

In one of my applications, we hit another server using HttpURLConnection - the application worked for fine for months, and now suddenly all hits are facing a connection timeout. Intermittently, a few calls (1 in 500) succeed while all others fail. The application is deployed on Linux running on Java 5 with Tomcat 5.5. I have tried a curl and wget from command line which work fine.

Another Java process using exactly the same code base is able to make connections from co开发者_开发问答mmand line. We have restarted the tomcat server as well as the machine but to no avail. We have thoroughly checked the machine and there are no blocking processes/firewalls hindering with this (evident that other Java processes are able to make connections).

In intermittent bursts (say once a day) the application is able to establish a few connections and then returns back to the broken state. Not much is evident from the stack trace as well.

Any suggestions where it might be going wrong?

[Update] In case the server DNS changes while the application is running, Java will not pick this up and connections will timeout from the older IP (considering the IP is down). Thus, in case of CDN if a node goes down and it is replaced with another IP, the problem may occur.

---

I had the same problem and it was caused by HttpURLConnection's handling of keepalive. The problem went away when we disabled keepalive by setting this system property,

http.keepAlive=false

We have another issue related to firewall. If the destination URL is blocked by firewall, it takes long time to time out (over 2 minutes). We have to run all our HttpURLConnection in another thread so we can interrupt it after a few seconds.