What Is On My Threads – Debugging Java Using JStack

Through the nature of my job for last decade I spend most of my time with Microsoft technologies. I have troubleshoot .NET, COM, IIS, SharePoint, BizTalk and of course SQL Server and as part of my job frequently used Microsoft Debugging Tools for Windows (WINDBG) and multitude of extensions for stability and performance troubleshooting and performance analysis. However, my current customer is a mixed shop with fairly large Java presence. Therefore systems I troubleshoot and tune fairly commonly interface with systems built in J2EE. Lots of times it surprised me that fairly common production debugging techniques that I used with Windows\Microsoft -centric applications were virtually unknown to Java developers that attempted to reproduce issues in production on their development workstation with Eclipse\NetBeans IDE to no avail or resorted to guessing and providing fixes for that guess.

Meanwhile, Java does have tools that can be used for production debugging. One of such tools is jstack. JStack is JVM troubleshooting tool that prints stack traces of all running threads of a given JVM process, a Java core file, or remote debug server. First I have seen tool in action with one of our banking clients on performance tuning engagement few years ago and it was useful. Below I will create a simple HangMe application that will allow me to create a “hang issue” and successfully show it via JStack.

Below is simple Java Console Application that uses Thread.Sleep to essentially create that hang:

package hangme;

/** * * @author gennadyk */

public class HangMe {

/** * @param args the command line arguments */

public static void main(String[] args) {

Runtime rt = Runtime.getRuntime();

System.out.println("Lets Go To Sleep");

try {Thread.sleep(1000*60*60);}

catch (InterruptedException e) {} }

}

Now lets compile and run this program. Next step is to run jps utility to get a PID for HangMe that is being executed:

C:\Program Files\Java\jdk1.7.0_25\bin>jps -l –m

26192 C:\Users\gennadyk\Documents\NetBeansProjects\HangMe2\dist\HangMe.jar

Now lets use JStack to get thread stack information on PID 26192

:\Program Files\Java\jdk1.7.0_25\bin>jstack 26192

2014-08-06 14:18:20 Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode):

"Service Thread" daemon prio=6 tid=0x00000000028b8000 nid=0x2d10 runnable [0x000 0000000000000] java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" daemon prio=10 tid=0x00000000028b1000 nid=0x5574 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" daemon prio=10 tid=0x00000000028af800 nid=0x3950 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE

"Attach Listener" daemon prio=10 tid=0x000000001b061000 nid=0x231c waiting on co ndition [0x0000000000000000] java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x000000001b060800 nid=0x31c4 runnable [0 x0000000000000000] java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x000000001906e800 nid=0x1a34 in Object.wait() [0x 000000001b03f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x0000000756405608> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(Unknown Source) - locked <0x0000000756405608> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

"Reference Handler" daemon prio=10 tid=0x0000000019068000 nid=0x2f98 in Object.w ait() [0x000000001af3f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x0000000756405190> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:503) at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source) - locked <0x0000000756405190> (a java.lang.ref.Reference$Lock)

"main" prio=6 tid=0x00000000027be000 nid=0x6758 waiting on condition [0x00000000 027af000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at hangme2.HangMe2.main(HangMe2.java:20) "VM Thread" prio=10 tid=0x0000000019064800 nid=0x24e0 runnable "GC task thread#0 (ParallelGC)" prio=6 tid=0x00000000027d7800 nid=0x40ec runnabl e "GC task thread#1 (ParallelGC)" prio=6 tid=0x00000000027d9800 nid=0x6648 runnabl e "GC task thread#2 (ParallelGC)" prio=6 tid=0x00000000027db000 nid=0x120c runnabl e "GC task thread#3 (ParallelGC)" prio=6 tid=0x00000000027dc800 nid=0x43fc runnabl e "GC task thread#4 (ParallelGC)" prio=6 tid=0x00000000027df000 nid=0x319c runnabl e "GC task thread#5 (ParallelGC)" prio=6 tid=0x00000000027e0800 nid=0x6068 runnabl e "GC task thread#6 (ParallelGC)" prio=6 tid=0x00000000027e4800 nid=0x2114 runnabl e "GC task thread#7 (ParallelGC)" prio=6 tid=0x00000000027e6000 nid=0xbb8 runnable "VM Periodic Task Thread" prio=10 tid=0x00000000028bc000 nid=0x26d0 waiting on c ondition JNI global references: 105

As you can see above you can easily note my Thread.Sleep call on main above as a wait cause.

Some more on JStack parameters

pId = Java process id (use jps to get the info)

exe = Java executable from which the core dump was produced.

core = core file for which the stack trace is to be printed.

remote-hostname-or-IP = Debug server’s (see jsadebugd) hostname or IP address.

server-id= (optional) unique server id

More on JStack – http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstack.html and http://docs.oracle.com/javase/7/docs/technotes/tools/share/jstack.html

Advertisements

3 thoughts on “What Is On My Threads – Debugging Java Using JStack

  1. Pingback: Using nmnon To Monitor Linux System Performance On Azure | A posteriori
  2. Pingback: Javacore Dump Analysis using JCA – IBM Thread and Monitor Dump Analyzer for Java | A posteriori
  3. Pingback: Let Me Count The Ways – Various methods of generating stack dump for JVM in production | A posteriori

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s