I’ve seen a lot of MemoryLeaks caused by wrong usage patterns of ThreadLocal variables. As I had this problem in one of our own applications yesterday, I want to blog about this case.
ThreadLocal variables are used in Java to bind a variable to the current Thread – each thread will have its own, independend instance of that variable. These kind of variables are normally used to hold status information like the user, that are needed in the entire application and you don’t want to have in every method signature.
The lifecyle of ThreadLocal variables is directly interconnected to the lifecyle of its corespondig thread. If the thread is terminated and collected by the Garbage Collector, its coresponding ThreadLocal variables will also be good candidates for Garbage Collcetion.
Memory problems will mainly occur if ThreadLocal variables are used inside of Java EE applications running in an application server. Application Servers manage threads by using thread pools to safe resources and boost performance. (See configuration of Tomcat 6 Http Conncector as an example). If for example a HttpServletRequest is send to the ServletEngine of the Application Servers, a free worker thread is taken out of the pool and is associated with the processing of the servlet’s application logic. If the servlet or its called Java classes are using ThreadLocal variables, these variables are associated with the current worker thread. If the servlet is finished and the response is sent to the client, the associated thread is put back to the thread pool, so that it can handle another request. That means that the Thread object and its associated ThreadLocal variables are not garbage collected, as the thread object is still alive.
Depending on the number of threads in the pool (> 100 threads are normal in productive environements) and the size of the object in the ThreadLocal variable, critical memory problems can occur. If for example 200 threads are configured for the thread pool and the ThreadLocal variables are 5MB big, this could result in 1 GB of heaps space occupied by just these ThreadLocal variables. This will lead to an GC overhead and can crash the JVM with an OutOfMemoryError.
In our concrete case the servers had to be restartet every day to avoid OutOfMemoryError problems. To analyse the problem we took a heapdump during runtime (see our recent blog entry how to do this). I did the analysis of the dump with Eclipse MAT which showed very quickly that we had a ThreadLocal memory problem.
The above screenshot shows the Dominator Tree of the dump. I’ve marked 6 Threads each occupying about 14MB of the heap.
The next screenshot shows the details of one of these threads. The high memory consumption is the result of a ThreadLocal variable that references a DOMParser and its parsed document with a total size of 14MB. Because MAT can also show the content of the objects, we could easily see that the parsers were caused by our WebServices. The WebService classes were generated by JBoss WS . Consulting Google showed that this was because of a Bug in jbossws version 1.2.0, which was fixed in 1.2.1: “DOMUtils doesn’t clear thread locals”. So we had to patch our JBoss AppServer with the latest jbossws release and the problem was gone.
This exampe shows that you have to be careful if you use ThreadLocal inside an application server. It is essential to take care that the ThreadLocal variables are dereferenced if they are not used anymore. We normally use ServletFilters to do this job.
Was this post helpful?
More articles
fromMirko Novakovic
21.2.2014 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Mirko Novakovic
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.