Tomcat 6 JMX How-To
Subscribe

From OpenNMS

Jump to: navigation, search


Contents

Summary

I just want to write down my steps how I managed it to collect Garbage Collector data from Tomcat 6

My environment

  • OS on Tomcat6 servers : Ubuntu Jaunty - Server Edition - 64 Bit
  • OpenNMS : 1.7.4

Steps

Enable JMX on Tomcat6

/etc/default/tomcat6 

# Forcing JVM to use: 'Concurrent Mark-Sweep Collector (aka Low-Latency Collector)' as Garbage Collector
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC"

# Enabling JMX 
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.port=9003"
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.password.file=/etc/tomcat6/jmxremote.password"
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.access.file=/etc/tomcat6/jmxremote.access"
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.ssl=false"

In the first line I force the JVM to use a specific Garbage Collector. Reason is, that one of the servers I want to monitor is a Virtual Machine and the JVM then chooses a Garbage Collector which is more suitable. More info: Sun JVM Memory Management


/etc/tomcat6/jmxremote.password 

opennms OpenNMS

Username and password for remote authentification. These will be used by OpenNMS.

/etc/tomcat6/jmxremote.access 

opennms readonly

Here we restrict our previously created user to have only read access

Pitfalls

The config files must have correct permissions. Check logfiles for the following entries:

/var/log/syslog 

Error: Password file read access must be restricted: /etc/tomcat6/jmxremote.password
Error: Password file not readable: /etc/tomcat6/jmxremote.password

Assigning correct permissions: 

chmod 600 /etc/tomcat6/jmxremote.access
chmod 600 /etc/tomcat6/jmxremote.password

chown tomcat6:tomcat6 /etc/tomcat6/jmxremote.access
chown tomcat6:tomcat6 /etc/tomcat6/jmxremote.password

Testing JMX

For testing the JMX connection and simple datacollection, you can use jconsole and cmdline-jmxclient

  • jconsole is included in sun-java6-jdk
  • cmdline-jmxclient can be downloaded from cmdline-jmxclient


Pitfalls

If jconsole does not connect successfully, run it in debug mode 

jconsole -debug IP:Port

java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is:
java.net.ConnectException: Connection refused: connect

If you see output like the previous one, then you need to adapt your hosts file on the server! (NOT on the client you want connecting from). Here is an example:

/etc/hosts

from 

127.0.1.1      tomcat6.server-test.net tomcat6

changed to 

192.168.0.100    tomcat6.server-test.net tomcat6

Collecting simple data: 

java -jar cmdline-jmxclient.jar USER:PASSWORD HOST:PORT MBEAN

e.g. 

java -jar cmdline-jmxclient-0.10.3.jar opennms:OpenNMS 192.168.0.100:9003 java.lang:type=OperatingSystem FreePhysicalMemorySize

Configure Opennms

Capsd

/etc/opennms/capsd-configuration.xml 

...
<capsd-configuration
...
...
<protocol-plugin protocol="JVM" class-name="org.opennms.netmgt.capsd.plugins.Jsr160Plugin" scan="on" user-defined="false">
    <property key="port" value="9003"/>
    <property key="type" value="default"/>
    <property key="factory" value="PASSWORD-CLEAR"/>
    <property key="username" value="opennms"/>
    <property key="password" value="OpenNMS"/>
    <property key="protocol" value="rmi"/>
    <property key="urlPath" value="/jmxrmi"/>
    <property key="retry" value="2"/>
    <property key="timeout" value="2000"/>
</protocol-plugin>
...
...
</capsd-configuration>

Datacollection

/etc/opennms/collectd-configuration.xml 

...
<collectd-configuration
...
...
<package name="example1">
...
<service name="JVM" interval="300000" user-defined="false" status="on">
    <parameter key="port" value="9003"/>
    <parameter key="retry" value="2"/>
    <parameter key="timeout" value="3000"/>
    <parameter key="protocol" value="rmi"/>
    <parameter key="urlPath" value="/jmxrmi"/>
    <parameter key="factory" value="PASSWORD-CLEAR"/>
    <parameter key="username" value="opennms"/>
    <parameter key="password" value="OpenNMS"/>
    <parameter key="ds-name" value="jmx"/>
    <parameter key="friendly-name" value="jsr160"/>
    <!-- This must match the collection name in the jmx-datacollection.xml
         that defines the set of mbeans you want -->
    <parameter key="collection" value="jsr160"/>
</service>
...
</package>
...
...
...
<collector service="JVM"  class-name="org.opennms.netmgt.collectd.Jsr160Collector"/>
...
...
...
</collectd-configuration>

/etc/opennms/jmx-datacollection-config.xml 

...
<jmx-collection name="jsr160">
...
<mbeans>
...
    <mbean name="JVM Garbage Collector" objectname="java.lang:type=GarbageCollector,name=ParNew">
        <attrib name="CollectionCount"        alias="CollectionCount"          type="gauge"/>
    </mbean>
    <mbean name="JVM Garbage Collector2" objectname="java.lang:type=GarbageCollector,name=ConcurrentMarkSweep">
        <attrib name="CollectionCount"        alias="CollectionCount2"          type="gauge"/>
    </mbean>
...
</mbeans>
</jmx-collection>

Graphing

Note: this was broken in 1.6 but works in 1.6.7

At the beginning of snmp-graph.properties I added jvm.garbage.count to the reports variable and somewhere in the middle I defined the graphs.

/etc/opennms/snmp-graph.properties 

...
reports=...,
...
..., \
jvm.garbage.count
...
...
report.jvm.garbage.count.name=JVM Garbage Collection
report.jvm.garbage.count.columns=CollectionCount,CollectionCount2
report.jvm.garbage.count.type=interfaceSnmp
report.jvm.garbage.count.command=--title="JVM Garbage Collection" \
 DEF:collections={rrd1}:CollectionCount:AVERAGE \
 DEF:collections2={rrd2}:CollectionCount2:AVERAGE \
 LINE2:collections#458B00:"Collections (ParNew)" \
 LINE2:collections2#0000ff:"Collections (ConcurrentMarkSweep)" \
 GPRINT:collections:AVERAGE:" Avg ( ParNew ) \\: %8.2lf %s\\t" \
 GPRINT:collections2:AVERAGE:" Avg ( ConcurrentMarkSweep ) \\: %8.2lf %s\\n"
...
...


Here is an example

Jvm-garbage.png

Retrieved from "http://www.opennms.org/wiki/Tomcat_6_JMX_How-To"