Java application performs very slow (10-100 times slower than on Windows, Linux, AIX)

I need your help about performance problems running our corporate Java application on HP\UX server. A开发者_JAVA技巧pplication is standalone tool which synchronizes data over several data bases into one, communicates with remote control on XML-RPC protocol and uses local Derby (Java DB) data base instance to hold configuration data etc. We don not have performance problems on other environments on the same load like Windows XP, Linux and AIX which use Sun JVM. After series of test we found out that most time consuming was communication with Derby data base. Most time is spent on reading from socket and this time is greater in 10-100 times than on other platforms. We know for sure that Derby works fine, we’ve got CPU reserve (usage is about 30%-40%), so most probable reason is transport layer between local data base and application.

Is there a way to diagnose socket I\O problems on HP-UX or maybe there is some possible limitations that can be configured? Maybe there is necessary JVM option? Any ideas from your side would be highly appreciated.

We’ve tried to optimize JVM options according to http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websphere.wsfep.multiplatform.doc/info/ae/ae/tprf_tunejvm_v61.html but didn’t get any significant improvement.

JVM info: Java HotSpot(TM) 64-Bit Server VM (19.1-b02-jinteg:2011mar11-16:46 PA2.0W (aCC_AP), mixed mode) Java: version 1.6.0.10, vendor "Hewlett-Packard Company"

We use following instance: OS: HP-UX (B.11.23) Architecture: PA_RISC2.0W 64bit Processors: 2

Total physical memory size: 4 088 MB Swap size: 4 090 MB

Here is example of slow running code. It takes several seconds to execute on HP while on Windows it takes 10-30ms:

/** Template to communicate with local db. */
SimpleJdbcTemplate jdbcTemplate;

@Transactional(readOnly = true)
public List<JobLogEntry> getLastLogs(Integer dbnr, JobDataType dtype) {
    try {
        String uid = jdbcTemplate.queryForObject("SELECT session_uuid FROM "
                                                         + tableName + " WHERE id=(SELECT max(id) FROM "
                                                         + tableName + " WHERE dbnr=? AND dtype=?)",
                                                         String.class, dbnr, dtype.name());
        List<JobLogEntry> list = jdbcTemplate.query("SELECT id, dbnr, dtype, zeit, level, message FROM "
                                                             + tableName
                                                             + " WHERE dbnr=? AND dtype=? AND session_uuid=? ORDER BY ID",
                                                             new ConRowMapper(), dbnr, dtype.name(), uid);
        return list;
    } catch (org.springframework.dao.EmptyResultDataAccessException e) {
        return new ArrayList<JobLogEntry>();
    }
}



class ConRowMapper implements RowMapper<JobLogEntry> {
    private final SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy.MM.dd HH:mm:ss");

    /**
     * Maps rows.
     */
    public JobLogEntry mapRow(ResultSet rs, int rowNum) throws SQLException {
        return new JobLogEntry(rs.getInt("dbnr"), 
                               rs.getString("dtype"), 
                               dateFormat.format(rs.getTimestamp("zeit")), 
                               rs.getString("level"), 
                               rs.getString("message"));
    }
}

Thanks in advance for all your ideas

---

I wonder about the method getLastLogs(). Why query to get the session UUID and then turn around and use it in another query? I would guess that it's possible to do it in one query.

When you say Derby, it makes me think that only Java accesses that database. Is that true? Do you know that it's optimized well (e.g. proper indexes for every WHERE clause)?

Do you use connection pooling? That way you can pay the cost of creating connections up front and amortize it over all the queries you run.

I see jdbcTemplate, so you must be using Spring. I'd get the debug or trace interceptor wired in and see where the time is being spent.

I'd also recommend Visual VM 1.3.2 will all the plugins installed. It will give you a lot more data.

---

Probable reason could be slow and blockong GC work on HP-UX. Try removing redundant System.gc() calls and use some JVM GC options to otimize :)

See nice presentation about HP performance tuning: http://www.scribd.com/doc/47433278/Javamemorymanagemen