MBeans

  domain:attribute1=value1,attribute2=value2, .....

E.g.

  java.lang:type=Memory

points to the MBean which lets you access the memory information of the target server.

Unfortunately, except for so called MXBeans (<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/management/package-summary.html>) there is no standard naming for MBeans. Each platform uses its own. There used to be a naming standard defined in JSR77 (<http://jcp.org/en/jsr/detail?id=77>), unfortunately it was never widely adopted.

There are various ways for identifying MBeans on a server:

• Use "jmx4perl --list" to list all registered MBeans. In addition "jmx4perl --attributes" dumps out all known MBean attributes along with their values. (Be careful, the output can be quite large)
• Use "j4psh" for interactively exploring the JMX namespace.
• Use an alias. An alias is a shortcut for an MBean name, predefined by JMX::Jmx4Perl. All known aliases can be shown with "jmx4perl aliases". Since each platform can have slightly different MBean names for the same information, this extra level of indirection might help in identifying MBeans. See "Aliases" for more about aliases.
• Use a predefined check. "check_jmx4perl" comes with quite some checks predefined in various configuration files. These are ready for use out of the box. "Predefined checks" are described in an extra section.
• Ask your Java application development team for application specific MBean names.

Attributes and Operations

Simple data types can be used directly in threshold checking. I.e. the following data types can be used directly

• Integer
• Long
• Float
• Double
• Boolean
• String

"String" and "Boolean" can be used in string checks only, whereas the others can be used in both, numeric and string checks (see "String checks").

For numeric checks, the threhsholds has to be specified according to the format defined in <http://nagiosplug.sourceforge.net/developer-guidelines.html#THRESHOLDFORMAT>

Paths

For more complex types, "check_jmx4perl" provides the concept of so called paths for specifying an inner attribute of a more complex value. A path contains parts separated by slashes (/). It is similar to an XPath expression for accessing parts of an XML document. Each part points to an inner level of a complex object.

For example, the MBean "java.lang:type=Memory" exposes an attribute called "HeapMemoryUsage". This attribute is a compound data type which contains multiple entries. Looking with "jmx4perl" at this attribute

 $ jmx4perl http://localhost:8080/jolokia read java.lang:type=Memory HeapMemoryUsage
 {
   committed => 85000192,
   init => 0
   max => 129957888,
   used => 15106608,
 }

it can be seen, that there are 4 values coming with the reponse. With a path "used" one can directly pick the used heap memory usage (8135440 bytes in this case) which then can be used for a threshold check.

 $ check_jmx4perl --url http://localhost:8080/jolokia \ 
                  --mbean java.lang:type=Memory \ 
                  --attribute HeapMemoryUsage \
                  --path used \
                  --critical 100000000
 OK - [java.lang:type=Memory,HeapMemoryUsage,used] : Value 10136056 in range | ...

Attributes

Attributes are values obtained from MBean properties. Complex values are translated into a JSON structure on the agent side, which works for most types. To access a single value from a complex value, the path mechanism described above can be used. Thresholds can be applied to simple data types only, so for complex attributes a path is required.

Operations

The return values of operations can be used for threshold checking, too. Since a JMX exposed operation can take arguments, these has to be provided as extra arguments on the command line or in the configuration via the "Args" configuration directive. Due to the agent's nature and the protocol used (JSON), only simple typed arguments like strings, numbers or booleans ("true"/"false") can be used.

Example:

 $ check_jmx4perl --url http://localhost:8888/jolokia \
                  --mbean jolokia:type=Runtime \
                  --operation getNrQueriesFor \
                  --critical 10 \
                  "operation" \
                  "java.lang:type=Memory" \
                  "gc"

This example contacts a MBean "jolokia:type=Runtime" registered by the jolokia agent in order to check for the number of queries for a certain MBean via this agent. For this purpose an JMX operation "getNrQueriesFor" is exposed which takes three arguments: The type ("operation"/"attribute"), the MBean's ObjectName and the operation/attribute name which was called.

If the operation to be called is an overloaded operation (i.e. an operation whose name exists multiple times on the same MBean but with different parameter types), the argument types must be given within parentheses:

     --operation checkUserCount(java.lang.String,java.lang.String)

Aliases

Aliases are not extensible and can not take any parameters. All availables aliases can be viewed with

  jmx4perl aliases

A much more flexible alternative to aliases are parameterized checks, which are defined in a configuration file. See "CONFIGURATION" for more details about parameterized checks.

Relative Checks

The base value has to be given with "--base" (configuration: "Base"). The argument provided here is first tried as an alias name or checked as an absolute, numeric value. Alternatively, you can use a full MBean/attribute/path specification by using a "/" as separator, e.g.

  ... --base java.lang:type=Memory/HeapMemoryUsage/max ...

If one of these parts (the path is optional) contains a slash within its name, the slash must be escaped with a backslash (\/). Backslashes in MBean names are escaped with a double backslash (\\).

Alternatively "--base-mbean", "--base-attribute" and "--base-path" can be used to specify the parts of the base value separately.

Example:

   check_jmx4perl --url http://localhost:8080/jolokia \ 
                  --value java.lang:type=Memory/HeapMemoryUsage/used \ 
                  --base java.lang:type=Memory/HeapMemoryUsage/max \ 
                  --critical 90

   check_jmx4perl --url http://localhost:8080/jolokia \ 
                  --value java.lang:type=Memory/HeapMemoryUsage/used \ 
                  --base-mbean java.lang:type=Memory \
                  --base-attribute HeapMemoryUsage \
                  --base-path max \ 
                  --critical 90

This check will trigger a state change to CRITICAL if the used heap memory will exceed 90% of the available heap memory.

Incremental Checks

Incremental checks are switched on with the "--delta" option (configuration: "Delta"). This option takes an optional argument which is interpreted as seconds for normalization.

Example:

  check_jmx4perl --url http://localhost:8080/jolokia \ 
                 --mbean java.lang:type=Threading \ 
                 --attribute TotalStartedThreadCount \ 
                 --delta 60 \ 
                 --critical 5

This will fail as CRITICAL if more than 5 threads are created per minute (60 seconds). Technically "check_jmx4perl" uses the history feature of the jolokia agent deployed on the target server. This will always store the result and the timestamp of the last check on the server side and returns these historical values on the next check so that the velocity can be calculated. If no value is given for "--delta", no normalization is used. In the example above, without a normalization value of 60, a CRITICAL is returned if the number of threads created increased more than 5 between two checks.

"--delta" doesn't work yet with "--base" (e.g. incremental mode for relative checks is not available).

String checks

For string checks, "--critical" and "--warning" are not treated as numerical values but as string types. They are compared literally against the value retrieved and yield the corresponding Nagios status if matched. If the threshold is given with a leading "!", the condition is negated. E.g. a "--critical '!Running'" returns "CRITICAL" if the value not equals to "Running". Alternatively you can also use a regular expression by using "qr/.../" as threshold value (substitute "..." with the pattern to used for comparison). Boolean values are returned as "true" or "false" strings from the agent, so you can check for them as well with this kind of string comparison.

No performance data will be generated for string checks by default. This can be switched on by providing "--perfdata on" (or ""PerfData on"" in the configuration). However, this probably doesn't make much sense, though.

Output Tuning

  B  - Byte
  KB - Kilo Byte
  MB - Mega Byte
  GB - Giga Byte
  TB - Terra Byte
  
  us - Microseconds
  ms - Milliseconds
  s  - Seconds
  m  - Minutes
  h  - Hours
  d  - Days

The unit will be used for performance data as well as for the plugin's output. Large numbers are converted to larger units automatically (and reverse for small number that are smaller than 1). E.g. "2048 KB" is converted to "2 MB". Beautifying by conversion is only performed for the plugin output, not for the performance data for which no conversions happens at all.

Beside unit handling, you can provide your own label for the Nagios output via "--label". The provided option is interpreted as a pattern with the following placeholders:

 %v   the absolute value 
 %f   the absolute value as floating point number
 %r   the relative value as percentage (--base)
 %q   the relative value as ratio of value to base (--base)
 %u   the value's unit for the output when --unit is used (after shortening)
 %w   the base value's unit for the output when --unit is used (after shortening)
 %b   the absolut base value as it is used with --base 
 %c   the Nagios exit code in the Form "OK", "WARNING", "CRITICAL" 
      or "UNKNOWN"
 %t   Threshold value which failed ("" when the check doesn't fail)
 %n   name, either calulated automatically or given with --name
 %d   the delta value used for normalization when using incremental mode
 %y   WARNING threshold as configured
 %z   CRITICAL threshold as configured

Note that %u and %w are typically not the same as the "--unit" option. They specify the unit after the conversion for the plugin output as described above. You can use the same length modifiers as for "sprintf" to fine tune the output.

Example:

 check_jmx4perl --url http://localhost:8888/jolokia \
                --alias MEMORY_HEAP_USED \
                --base MEMORY_HEAP_MAX \ 
                --critical :80 \
                --label "Heap-Memory: %.2r% used (%.2v %u / %.2b %w)" \
                --unit B

will result in an output like

 OK - Heap-Memory: 3.48% used (17.68 MB / 508.06 MB) | '[MEMORY_HEAP_USED]'=3.48%;;:80

Security

In addition to this user/password authentication, the jolokia-agent uses a policy file for fine granular access control. The policy is defined with an XML file packaged within the agent. In order to adapt this to your needs, you need to extract the war file, edit it, and repackage the agent with a policy file. A future version of jmx4perl might provide a more flexible way for changing the policy.

In detail, the following steps are required:

• Download jolokia.war and a sample policy file jolokia-access.xml into a temporary directory:

     $ jolokia
     $ jolokia --policy
      

• Edit the policy according to your needs.

     $ vi jolokia-access.xml
      

• Repackage the war file

     $ jolokia repack --policy jolokia.war
      

• Deploy the agent jolokia.war as usual

The downloaded sample policy file jolokia-access.xml contains inline documentation and examples, so you can easily adapt it to your environment.

Restrictions can be set to on various parameters :

Client IP address

Access to the jolokia-agent can be restricted based on the client IP accessing the agent. A single host, either with hostname or IP address can be set or even a complete subnet.

Example:

  <remote>
    <host>127.0.0.1</host>
    <host>10.0.0.0/16</host>
  </remote>

Only the localhost or any host in the subnet 10.0 is allowed to access the agent. If the "<remote>" section is missing, access from all hosts is allowed.

Commands

The access can be restricted to certain commands.

Example:

   <commands>
     <command>read</command>
   </commands>

This will only allow reading of attributes, but no other operation like execution of operations. If the "<commands>" section is missing, any command is allowed. The commands known are

read

Read an attribute

write

Write an attribute (used by "check_jmx4perl" only when using incremental checks)

exec

Execution of an operation

list

List all MBeans (not used by "check_jmx4perl")

version

Version command (not used by "check_jmx4perl")

search

Search for MBean (not used by "check_jmx4perl")

Specific MBeans

The most specific policy can be put on the MBeans themselves. For this, two sections can be defined, depending on whether a command is globaly enabled or denied.

<allow>

The "<allow>" section is used to switch on access for operations and attributes in case "read", "write" or "exec" are globally disabled (see above). Wildcards can be used for MBean names and attributes/and operations.

Example:

  <allow>
    <mbean>
      <name>jolokia:*</name>
      <operation>*</operation>
      <attribute>*</attribute>
    </mbean>
    <mbean>
      <name>java.lang:type=Threading</name>
      <operation>findDeadlockedThreads</operation>
    </mbean>
    <mbean>
    <name>java.lang:type=Memory</name>
      <attribute mode="read">Verbose</attribute>
    </mbean>

  </allow>
    

This will allow access to all operation and attributes of all MBeans in the "jolokia:" domain and to the operation "findDeadlockedThreads" on the MBean "java.lang:type=Threading" regardless whether the "read" or "exec" command is enabled globally. The attribute "Verbose" on "java.lang:type=Memory" is allowed to be read, but cannot be written (if the "mode" attribute is not given, both read and write is allowed by default).

<deny>

The "<deny>" section forbids access to certain MBean's operation and/or attributes, even when the command is allowed globally.

Example:

  <deny>
    <mbean>
      <!-- Exposes user/password of data source, so we forbid this one -->
      <name>com.mchange.v2.c3p0:type=PooledDataSource*</name>
      <attribute>properties</attribute>
    </mbean>
  </deny>
    

This will forbid the access to the specified attribute, even if "read" is allowed globally. If there is an overlap between <allow> and <deny>, <allow> takes precedence.

Proxy mode

• The target server needs to export its MBeans via JSR-160. The configuration for JMX export is different for different JEE Server. <http://labs.consol.de> has some cooking recipes for various servers (JBoss, Weblogic).
• A dedicated proxy server needs to be setup on which the jolokia.war gets deployed. This can be a simple Tomcat or Jetty servlet container. Of course, an already existing JEE Server can be used as proxy server as well.
• For using "check_jmx4perl" the target JMX URL for accessing the target server is required. This URL typically looks like

    service:jmx:rmi:///jndi/rmi://host:9999/jmxrmi
      

  but this depends on the server to monitor. Please refer to your JEE server's documentation for how the export JMX URL looks like.

• "check_jmx4perl" uses the proxy mody if the option "--target" (configuration: <Target>) is provided. In this case, this Nagios plugin contacts the proxy server specified as usual with "--url" (config: Url in Server section) and put the URL specified with "--target" in the request. The agent in the proxy then dispatches this request to the real target and uses the JMX procotol specified with in the target URL. The answer received is then translated into a JSON response which is returned to "check_jmx4perl".

  Example:

     check_jmx4perl --url http://proxy:8080/jolokia \
                    --target service:jmx:rmi:///jndi/rmi://jeeserver:9999/jmxrmi
                    --alias MEMORY_HEAP_USED
                    --base MEMORY_HEAP_MAX
                    --critical 90
      

  Here the host proxy is listening on port 8080 for jolokia requests and host jeeserver exports its JMX data via JSR-160 over port 9999. (BTW, proxy can be monitored itself as usual).

  So, what mode is more appropriate ? Both, the agent mode and the proxy mode have advantages and disadvantages.

Advantages

• No agent needs to be installed on the target server. This might be useful for policy reasons.
• Compared to other Nagios JMX plugin's no JVM startup is required since the proxy server is already running.

Disadvantages

• It takes two hops to get to the target server
• Exporting JMX via JSR-160 is often not that easy as it may seem. (See post series on remote JMX on labs.consol.de)
• Some features like merging of MBean Servers are not available in proxy mode. (i.e you need to know in advance which MBean-Server on the target you want to contact for a certain MBean, since this information is part of the JMX URL)
• Bulk request needs to be detangled into multiple JMX request since JSR-160 doesn't know anything about bulk requests.
• jmx4perl's fine granular security policy is not available, since JSR-160 JMX is an all-or-nothing thing. (except you are willing to dive deep into Java Security stuff)
• For JSR-160 objects to be transferable to the proxy, the proxy needs to know about the Java types and those types must be serializable. If this is not the case, the proxy isn't able to collect the information from the target. So only a subset of MBeans can be monitored this way.

  The agent protocol is more flexible since it translates the data into a JSON structure before putting it on the wire.

To summarize, I would always recommend the agent mode over the proxy mode except when an agentless operation is required (e.g. for policy reasons).

<Server>

Url

The URL under which the jolokia agent can be reached.

User, Password

If authentication is switched on, the user and the credentials can be provided with the User and Password directive, respectively. Currently only Basic Authentication is supported.

Product

The type of application server to monitor. This configuration can speed up checks significantly, but only when aliases are used. By default when using aliases, "check_jmx4perl" uses autodetection for determine the target's platform. This results in at least one additional HTTP-Request. This configuration does not has any effect when MBeans are always used with their full name.

Proxy

A HTTP Proxy URL and credentials can be given with the "<Proxy>" sub-section. Example:

  <Server>
  ....
    <Proxy>
      Url = http://proxy.company.com:8001
      User = woody
      Password = buzz
    </Proxy>
  </Server>
    

Target

With this directive, the JMX-Proxy mode can be switched on. As described in section "Proxy mode", "check_jmx4perl" can operate in an agentless mode, where the agent servlet is deployed only on an intermediated, so called JMX-Proxy server, whereas the target platform only needs to export JMX information in the traditional way (e.g. via JSR-160 export). This mode is especially useful if the agent is not allowed to be installed on the target platform. However, this approach has some drawbacks and some functionality is missing there, so the agent-mode is the recommended way. A sample JMX-Proxy configuration looks like:

  <Target>
     Url = service:jmx:rmi:///jndi/rmi://tessin:6666/jmxrmi
     User = max
     Password = frisch 
  </Target>
    

For a discussion about the advantages and disadvantages of the JMX-Proxy mode, please have a look at <http://labs.consol.de/> which contains some evaluations of this mode for various application servers (e.g. JBoss and Weblogic).

Single Check

Example:

  <Check memory_heap>
    Value = java.lang:type=Memory/HeapMemoryUsage/used
    Base = java.lang:type=Memory/HeapMemoryUsage/max
    Label = Heap-Memory:
    Name = Heap
    Critical = 90
  </Check>

The "<Check>" section knows about the following directives:

Mbean

The "ObjectName" of the MBean to monitor.

Attribute

Attribute to monitor.

Operation

Operation, whose return value should be monitored. Either "Attribute" or "Operation" should be given, but not both. If the operation takes arguments, these need to be given as additional arguments to the "check_jmx4perl" command line call. In the rare case, you need to call an overloaded operation (i.e. an operation whose name exists multiple times on the same MBean but with different parameter types), the argument types can be given within parentheses:

  <Check>
     ....
     Operation = checkUserCount(java.lang.String,java.lang.String)
     ...
  </Check>
    

Argument

Used for specifying arguments to operation. This directive can be given multiple times for multiple arguments. The order of the directive determine the order of the arguments.

  <Check>
     ....
     Operation checkUserCount(java.lang.String,java.lang.String)
     Argument  Max
     Argument  Morlock    
  </Check>
    

Alias

Alias, which must be known to "check_jmx4perl". Use "jmx4perl aliases" to get a list of all known aliases. If "Alias" is given as configuration directive, "Operation" and/or "Attribute" is ignored. Please note, that using "Alias" without "Product" in the server section leads to at least one additional HTTP request.

Path

Path to apply to the attribute or operation return value. See "Paths" for more information about paths.

Value

Value is a shortcut for specifying "MBean", "Attribute" and "Path" at once. Simply concatenate all three parts via "/" (the "Path" part is optional). Slashes within MBean names needs to be escaped with a "\" (backslash). Example:

  Value = java.lang:type=Memory/HeapMemoryUsage/used
    

is equivalent to

  MBean = java.lang:type=Memory
  Attribute = HeapMemoryUsage
  Path = used
    

Base

Switches on relative checks. See "Relative Checks" for more information about relative checks. The value specified with this directive defines the base value against which the relative value should be calculated. The format is the same as for "Value":

  Base = java.lang:type=Memory/HeapMemoryUsage/max
    

For relative checks, the "Critical" and "Warning" Threshold are interpreted as a value between 0% and 100%.

BaseMBean, BaseAttribute and BasePath

As an alternative to specifying a base value in a combined fashion the different parts can be given separately. "BaseMBean" and "BaseAttribute" switches on relative checks and specifies the base value. An optional "BasePath" can be used to provide the path within this base value.

The example above can be also written as

  BaseMBean = java.lang:type=Memory
  BaseAttribute = HeapMemoryUsage
  BasePath = max
    

Delta

Switches on incremental mode as described in section "Incremental Checks". The value given is used for normalization the increase rate. E.g.

  Delta = 60
    

measures the growth rate per minute (60 seconds). If no value is given, the absolute increase between two checks is used.

Numeric

This directive switches on numeric mode, i.e. the given threshold values are compared numerically against the returned JMX value. By default, the check mode is determined by a heuristic algorithm.

String

String checks, which are switched on with this directive, are useful for non-numeric thresholds. See "String checks" for more details.

Name

The name to be used in the performance data. By default, a name is calculated based on the MBean and attribute/operation name.

MultiCheckPrefix

If this check is used within a multi check, this prefix is used to identify this particular check in the output of a multicheck. It can be set to an empty string if no prefix is required. By default the name as configured with "Name" is used.

Label

Format for setting the plugin output (not the performance data, use "Name" for this). It takes a printf like format string which is described in detail in "Output Tuning".

PerfData

By default, performance data is appended for numeric checks. This can be tuned by setting this directive to "false" (or "0", "no", "off") in which case performance data is omitted. If using this in a base check, an inherited check can switch performance data generation back on with "true" (or "1", "yes", "on")

For relative checks, the value can be set to '%'. In this case, performance data is added as relative values instead of the absolute value measured.

Unit

This specifies how the return value should be interpreted. This value, if given, must conform to the unit returned by the JMX attribute/operation. E.g. for "java.lang:type=Memory/HeapMemoryUsage/used" unit, if set, must be "B" since this JMX call returns the used memory measured in bytes. The value given here is only used for shortening the plugin's output automatically. For more details and for what units are available refer to section "Output Tuning".

Critical

Specifies the critical threshold. If "String" is set (or the heuristics determines a string check), this should be a string value as described in "String checks". For relative checks, this should be a relative value in ther range [0,100]. Otherwise, it is a simple numeric value which is used as threshold. For numeric checks, the threshhold can be given in the format defined at <http://nagiosplug.sourceforge.net/developer-guidelines.html#THRESHOLDFORMAT>.

Warning

Defines the warning threshold the same way as described for the "Critical" threshold.

Null

Replacement value when an attribute is null or an operation returns a null value. This value then can be used in string checks in order to check against null values. By default, this value is ""null"".

Method

HTTP Method to use for the check. Available values are "GET" or "POST" for GET or POST HTTP-Requests, respectively. By default a method is determined automatically. The value can be given case insensitively.

Use

In order to use parent checks, this directive specifies the parent along with any parameters passed through. For example,

  Use = memory_relative_base(80,90),base_label
    

uses a parent check named "memory_relative_base", which must be a check defined in the same configuration file (or an imported on). Additionally, the parameters 80 and 90 are passed to this check (which can be accessed there via the argument placeholders $0 and $1). See "Parent checks" and "Parameterized checks" for more information about check inheritance.

Multiple parents can be given by providing them in a comma separated list.

Script

For complex checks which can not be realized with the configurations described until yet, it is possible to use a Perl script snippet to perfrom arbitrary logic. The content of this script is typically provided as an HERE-document (see example below). It comes with a predefined variable $j4p which is an instance of JMX::Jmx4Perl so that it can be used for a flexible access to the server. Note that this scriptlet is executed separately and doesn't not benefit from the optimization done for bulk or relative checks. Check parameters can be accessed as ${0}, ${1}, .. but since these are also valid Perl variables (and hence can be overwritten accidentially), it is recommended to assign them to local variable before using them. In summary, script based checks are powerful but might be expensive.

Example:

  Script <<EOT
  
  my $pools = $j4p->search("java.lang:type=MemoryPool,*");
  my @matched_pools;
  my $pattern = "${0}";
  for my $pool (@$pools) {   
     push @matched_pools,$pool if $pool =~ /$pattern/;   
  }
  return $j4p->get_attribute($matched_pools[0],"Usage","used");
  
  EOT
    

Includes

  include tomcat.cfg
  include threads.cfg
  include memory.cfg

If given as relative path, the configuration files are looked up in the same directory as the current configuration file. Absolute paths can be given, too.

Parent checks

For example, consider the following configuration:

  <Check grand_parent>
     Name grand_parent
     Label GrandPa
     Critical 10
  </Check>

  <Check parent_1>
     Use grand_parent
     Name parent_1
     Critical 20
  </Check>

  <Check parent_2>
     Name parent_2
     Warning 20
  </Check> 

  <Check check>
     Use parent_1,parent_2
     Warning 40
  </Check>

In this scenario, when check "check" is used, it has a "Name" ""parent_2"" (last parent check in "Use"), a "Label" "GrandPa" (inherited from "grand_parent" via "parent_1"), a "Critical" 20 (inherited from "parent_1") and a "Warning" 40 (directly give in the check definition).

A parent value of a configuration directive can be refered to with the placeholder $BASE. For example:

  <Check parent>
    Name Parent
  </Check>

  <Check check>
    Use parent
    Name Child: $BASE
  </Check>

This will lead to a "Name" ""Child: Parent"" since $BASE is resolved to the parent checks valus of "Name", "Parent" in this case. The base value is searched upwards in the inheritance hierarchy (parent, grand parent, ...) until a value is found. If nonen is found, an empty string is used for $BASE.

Parameterized checks

Example:

  <Check parent>
    Name $0
    Label $1
  </Check>

  <Check child_check>
    Use parent($0,"Check-Label")   
    ....
  </Check>

  $ check_jmx4perl --check child_check .... "Argument-Name"
  OK - Check-Label | 'Argument-Name'= ....

As it can be seen in this example, arguments can be propagated to a parent check. In this case, $0 from the command line ("Argument-Name") is passed through to the parent check which uses it in the "Name" directive. $1 from the parent check is replaced with the value ""Check-Label"" given in the "Use" directive of the child check.

Parameters can have default values. These default values are taken in case an argument is missing (either when declaring the parent check or missing from the command line). Default values are specified with ${arg-nr":"default"}". For example,

 <Check relative_base>
   Label = %.2r% used (%.2v %u / %.2b %w)
   Critical = ${0:90}
   Warning = ${1:80}
 </Check>

defines a default value of 90% for the critical threshold and 80% for the warning threshold. If a child check uses this parent definition and only wants to ommit the first parameter (but explicitely specifying the second parameter) it can do so by leaving the first parameter empty:

  <Check child>
     Use relative_base(,70)
  </Check>

Multichecks

If a single check within a multi check fails with an exception (e.g. because an MBean is missing), its state becomes "UNKNOWN". "UNKNOWN" is the highest state in so far that it shadows even "CRITICAL" (i.e. if a single check is "UNKNOWN" the whole multi check is "UNKNOWN", too). This can be changed by providing the command line option "--unknown-is-critical" in which case all "UNKNOWN" errors are mapped to "CRITICAL".

A multi-check can be defined with the directive "<MultiCheck>", which contain various references to other "<Check>" definitions or other multi check definitions.

Example:

  <MultiCheck all>
    MultiCheck memory
    MultiCheck threads
  </MultiCheck>

  <MultiCheck memory>
    Check memory_heap($0,80)
    Check memory_pool_base("CMS Perm Gen",90,80)
  </MultiCheck>

  <MultiCheck threads>
    Check thread_inc
    Check thread_deadlock
  </MultiCheck>

Here a multi check group memory has been defined with reference to two checks, which must exist somewhere else in the configuration file. As it can be seen, parameters can be given through to the check in the usual way (literally or with references to command line arguments). The group all combines the two groups memory and thread, containing effectively four checks.

A multi-check is referenced from the command line like any other check:

  $ check_jmx4perl .... --check all 90

(90 is the argument which replaces $0 in the definition above).

The summary label in a multi check can be configured, too.

Example:

  <MultiCheck memory>
    SummaryOk All %n checks are OK
    SummaryFailure %e of %n checks failed [%d]
    ...
  </MultiCheck>

These format specifiers can be used:

  %n        Number of all checks executed 
  %e        Number of failed checks
  %d        Details which checks failed

Predefined checks

common.cfg

Common check definitions, which can be used as parents for own checks. E.g. a check "relative_base" can be used as parent for getting a nicely formatted output message.

memory.cfg

Memory checks for heap and non-heap memoy as well as for various memory pools. Particularly interesting here is the so called Perm Gen pool as it holds the java type information which can overflow e.g after multiple redeployments when the old classloader of the webapp can't be cleared up by the garbage collector (someone might still hold a reference to it).

threads.cfg

Checks for threads, i.e. checking for the tread count increase rate. A check for finding out deadlocks (on a JDK 6 VM) is provided, too.

jetty.cfg

Various checks for jetty like checking for running servlets, thread count within the app server, sessions (number and lifing time) or requests per minute.

tomcat.cfg

Mostly the same checks as for jetty, but for tomcat as application server.

websphere.cfg

WebSphere specific checks, which uses the configuration files below the `websphere/` directory. For this checks to work, a customized Jolokia agent with JSR-77 extensions is required. The GitHub project for this enhanced agents can be found at <https://github.com/rhuss/jolokia-extra> and downloaded at Maven Central (<http://central.maven.org/maven2/org/jolokia/extra/jolokia-extra-war/>)