I'm currently load testing the default REST API of activemq. All my 100 users write the same message text to the same destination as fast as they can. After a few seconds i get:
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at org.apache.activemq.thread.TaskRunnerFactory.execute(TaskRunnerFactory.java:92)
at org.apache.activemq.thread.TaskRunnerFactory.execute(TaskRunnerFactory.java:85)
at org.apache.activemq.broker.TransportConnector$1.onAccept(TransportConnector.java:211)
at org.apache.activemq.transport.vm.VMTransportServer.connect(VMTransportServer.java:91)
at org.apache.activemq.transport.vm.VMTransportFactory.doCompositeConnect(VMTransportFactory.java:145)
at org.apache.activemq.transport.vm.VMTransportFactory.doConnect(VMTransportFactory.java:53)
at org.apache.activemq.transport.TransportFactory.doConnect(TransportFactory.java:51)
at org.apache.activemq.transport.TransportFactory.connect(TransportFactory.java:80)
at org.apache.activemq.ActiveMQConnectionFactory.createTransport(ActiveMQConnectionFactory.java:243)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:258)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:230)
at org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:178)
at org.apache.activemq.web.WebClient.getConnection(WebClient.java:246)
at org.apache.activemq.web.WebClient.createSession(WebClient.java:348)
at org.apache.activemq.web.WebClient.getSession(WebClient.java:239)
at org.apache.activemq.web.MessageServletSupport.getDestination(MessageServletSupport.java:319)
at org.apache.activemq.web.MessageServletSupport.getDestination(MessageServletSupport.java:267)
at org.apache.activemq.web.MessageServlet.doPost(MessageServlet.java:110)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:713)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:527)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1216)
at org.apache.activemq.web.SessionFilter.doFilter(SessionFilter.java:45)
at开发者_如何学运维 org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1187)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:421)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:493)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:930)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:358)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:866)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:456)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113)
at org.eclipse.jetty.server.Server.handle(Server.java:351)
at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:594)
at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1059)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:764)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:217)
at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:424)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:506)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:436)
at java.lang.Thread.run(Thread.java:662)
Since I'm already running with 10G heapspace and only simulating 10 REST users I don't understand why activemq is generating 8000 threads. I hope someone can help me fix this.
The reason for this memory leak lies within activemq's connection handling. The default servlet creates one connection for each httpsession (which is a really bad idea). Me loadtesting tool doesn't care about sessions and doesn't send a session-id. So every message that was send is a new connection, is a new thread and after 17.000 threads the jvm is done.
Basicly I'm going to write a new servlet that uses one connection per worker thread. That should limmit the threadnumber to a bare minimum.
精彩评论