Hyperic HQ Syslog Integration
Subscribe

From OpenNMS

Jump to: navigation, search
⚠
This page is obsolete. Please see Hyperic HQ Integration instead.

This article describes an abandoned first attempt at creating an integration between OpenNMS and Hyperic HQ.

Contents

HQ Server Configuration

Here's the doc from Hyperic's site on setting up syslog logging of escalation alerts.

Configure the HQ server database (may be unnecessary)

Quote:

Separately, you'll need to enable the alert gui by setting the SYSLOG_ALERTS_ENABLED attribute in the HQ database's EAM_CONFIG_PROPS table to 'true'.

Hyperic's docs indicate that this step is unnecessary if you will be using syslog only from alert escalations on the HQ server.

Execute the following SQL update statement against the HQ database:

UPDATE eam_config_props SET propvalue = 'true' WHERE propkey = 'CAM_SYSLOG_ACTIONS_ENABLED';

You can enter this query through the HQ webui at http://hqhost:hqport/admin/sql.jsp (e.g. http://10.11.12.13:7080/admin/sql.jsp) as outlined in this article from Hyperic's site.

Alternately, it may work (assuming that HQ Server is using the embedded PostgreSQL database) to reach the database directly from the HQ server host:

psql -U hqadmin -h localhost -p 9432 hqdb

Configure log4j.xml

On the HQ server, edit HQ_SERVER_HOME/conf/templates/log4j.xml. Add or uncomment the following section:

<appender name="SYSLOG" class="org.apache.log4j.net.SyslogAppender">
  <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>
  <param name="Facility" value="LOCAL7"/>
  <param name="FacilityPrinting" value="false"/>
  <param name="SyslogHost" value="localhost"/>
  <layout class="org.apache.log4j.PatternLayout">
    <param name="ConversionPattern" value="%c{1}[%r]: %m%n"/>
  </layout>
</appender>

<!-- More appenders here -->

<!-- Put this category after the last appender element or you will lose -->
<category name="org.hyperic.hq.bizapp.server.action.log.SyslogAction">
  <appender-ref ref="SYSLOG"/>
</category>

Change the value of the SyslogHost parameter from localhost to the IP address of the OpenNMS server where Syslogd is running (or to whatever syslog aggregation server is configured to forward messages to OpenNMS). Note that the default log4j.xml has the Facility parameter set to SYSLOG, which will quite possibly not work; you want to change this to a real syslog facility name (e.g. LOCAL7).

In the log4j.xml distributed with HQ server, the categoryelement is inside the same comment immediately following the corresponding appender element. You *MUST* move this category element after the last appender element in this file, or HQ Server will be unable to unmarshal the config file. It will mention this in logs/server.log but will continue to start up anyway.

The default log4j.xml also includes, within the category for syslog, a priority element. I had to remove this child element (leaving only the appender-ref child element, as above) in order to get this to work at all.

Finally, the examples on Hyperic's site use the wrong class name for the category element (they have net.hyperic.hq.bizapp.server.action.log.SyslogAction, which does not exist -- the package name must start with org.hyperic instead). The log4j.xml distributed with HQ Server does not contain this error.

OpenNMS Syslogd Configuration

Notes on break-down of syslog message format

Hyperic's docs say the message format is:

SyslogAction[ALERT_ID]: DB_1 4 META/PRODUCT/VERSION RESOURCE_NAME : ALERT_NAME - ALERT_CONDITION

An example message, grabbed from /var/log/syslog on the HQ appliance VM (with FacilityPrinting set to false):

SyslogAction[413523]: DB_1 5 TestMeta123/TestProduct123/TestVersion123 hq-server.hyperic.net :Cheesy Test Alert - 
    If Free Memory < 8.0 GB (actual value = 323.4 MB)
  • SyslogAction -- Hard-wired, nice way to recognize them
  • [413523]: -- Alert ID in HQ, worth keeping
  • DB_1<tt> -- looks hard-wired per Hyperic docs
  • <tt>5 -- probably the alert priority in the context of HQ, need to check (this alert was set to HIGH)
  • TestMeta123/TestProduct123/TestVersion123 -- These are the three fields required when configuring an escalation. Maybe just discard.
  • hq-server.hyperic.net : -- The name by which HQ knows this resource; might be gethostname(), need to check
  • Cheesy Test Alert - -- Name for alert within HQ, with (space, dash, space, newline, four spaces) appended
  • If Free Memory < 8.0 GB (actual value = 344.8 MB) -- HQ's description of the triggering situation, how hard to parse?

In OpenNMS 1.3.8, we can break apart some of these values using a regex match. For now, though, here's a syslogd-configuration.xml for 1.3.7 systems that works:

Working syslogd-configuration.xml for OpenNMS 1.3.7

<?xml version="1.0"?>
<syslogd-configuration>
    <configuration
            syslog-port="514"
            new-suspect-on-message="true"
            forwarding-regexp="^((.+?) (.*))\n?$"
            matching-group-host="2"
            matching-group-message="3"
            />

    <!-- Use the following to convert UEI ad-hoc -->
    <ueiList>
        <ueiMatch>
            <match>CRISCO</match>
            <uei>CISCO</uei>
        </ueiMatch>

        <ueiMatch>
            <match>SyslogAction[</match>
            <uei>uei.opennms.org/vendors/hyperic/hq/alerts/SyslogAction</uei>
        </ueiMatch>
    </ueiList>

    <!-- Use the following to remove a syslog message from the event-trail -->

    <hideMessage>
        <hideMatch>
            <match>TEST</match>
        </hideMatch>
    </hideMessage>


</syslogd-configuration>

Event Definition for OpenNMS

The corresponding event definition, included from eventconf.xml. In 1.3.8, we will be able to take advantage of the regex matching in Syslogd to create useful alarm data and other awesomeness.

<event>
 <uei>uei.opennms.org/vendors/hyperic/hq/alerts/SyslogAction</uei>
 <event-label>Hyperic-HQ defined event: SyslogAction</event-label>
 <descr><p>Hyperic HQ Server has forwarded an alert via
 syslog. Details are contained in the message body.</p>
                        Node ID: %nodeid%<br>
                        Host: %nodelabel%<br>
                        Interface: %interface% <br>
                        Message: %parm[syslogmessage]% <br>
                        Process: %parm[process]% <br>
                        PID: %parm[processid]%
 </descr>
 <logmsg dest='logndisplay'><p>Hyperic HQ Syslog Alert: HQ Server has forwarded an alert.</p></logmsg>
 <severity>Indeterminate</severity>
</event>