JMXTrans Setup (Monitoring a proprietary Java Application)

I recently had the need to improve monitoring of a proprietary java application.  After poking around at the options available to me I found that there was extensive monitoring available in the administration console, but that it was a challenge to scrape, parse, or export.  Luckily the internal monitoring all seemed to be provided by mbeans which I quickly verified by attaching to the app with jconsole.

Its at this point I set about writing my own mbean poller and while working on some challenges I was referred to JMXTrans by these guys (thanks to Lars).  While I am eager to try their javaagent solution as well I initially set up the poller:

This entry was almost pointless as the installation docs are great!!!!

Making sure that mbeans are exposed is easy. For a NON-production server that is BEHIND a firewall add the following parameters to start up

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.local.only="false"
-Dcom.sun.management.jmxremote.port=
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Djava.rmi.server.hostname="the public hostname"

I still need to set up jmx authentication, so more on that soon…

RPM install the latest stable version from: https://github.com/jmxtrans/jmxtrans/downloads

$ sudo rpm -Uhv https://github.com/downloads/jmxtrans/jmxtrans/jmxtrans-20121016.145842.6a28c97fbb-0.noarch.rpm


$ cd /var/lib/jmxtrans/
vim server_to_poll.json

And then populate it, this is an example for writing to graphite. It is really helpful to connect via jconsole and browse the mbeans there so you can get the “obj” and “attr” information cleanly


{
    "servers" : [ {
        "port" : "",
        "host" : "",
        "alias" : "",
        "queries" : [ {
            "obj" : "java.lang:type=Memory",
            "resultAlias": "Memory",
            "attr" : [ "HeapMemoryUsage", "NonHeapMemoryUsage", "ObjectPendingFinalizationCount" ],
            "outputWriters" : [ {
                "@class" : "com.googlecode.jmxtrans.model.output.GraphiteWriter",
                "settings" : {
                    "port" : "${mygraphiteport}",
                    "host" : "${mygraphitehost}"
                }
            } ]
        } , {
            "obj" : "java.lang:type=Threading",
            "resultAlias": "Threads",
            "attr" : [ "DaemonThreadCount", "PeakThreadCount", "CurrentThreadCpuTime", "CurrentTheeadUserTime", "ThreadCount", "TotalStartedThreadCount" ],
            "outputWriters" : [ {
                "@class" : "com.googlecode.jmxtrans.model.output.GraphiteWriter",
                "settings" : {
                    "port" : "${mygraphiteport}",
                    "host" : "${mygraphitehost}"
                }
            } ]
        } , {
            "obj" : "java.lang:type=GarbageCollector,name=Copy",
            "resultAlias": "GCCopy",
            "attr" : [ "CollectionCount", "CollectionTime" ],
            "outputWriters" : [ {
                "@class" : "com.googlecode.jmxtrans.model.output.GraphiteWriter",
                "settings" : {
                    "port" : "${mygraphiteport}",
                    "host" : "${mygraphitehost}"
                }
            } ]
        } , {
            "obj" : "java.lang:type=GarbageCollector,name=MarkSweepCompact",
            "resultAlias": "GCCMS",
            "attr" : [ "CollectionCount", "CollectionTime" ],
            "outputWriters" : [ {
                "@class" : "com.googlecode.jmxtrans.model.output.GraphiteWriter",
                "settings" : {
                    "port" : "${mygraphiteport}",
                    "host" : "${mygraphiteport}"
                }
            } ]
        } ]
    } ]
}


${mygraphiteport} and ${mygraphiteport} are not variables for substitution. Instead these are specified in

$ sudo vim /etc/sysconfig/jmxtrans
EDIT: export JMXTRANS_OPTS="-Dmygraphiteport= -Dmygraphitehost="