Sibling column storage strategy specification
Subscribe

From OpenNMS

Jump to: navigation, search

Overview

Today, there are three broad kinds of resources that can have performance data in OpenNMS:

  1. Node-level resources (exactly one resource per node)
  2. Interface-level resources (one resource per ifTable row on a node)
  3. Generic indexed resources (one resource per instance of a particular SNMP columnar object or WMI multi-valued object)

Storage strategies are used by OpenNMS to decide on the pathname of the time-series file in which performance data gathered by the collectors will be persisted. The storage strategies for node-level and interface-level resource data are not configurable. The storage strategy for a generic-indexed resource type is set in the resourceType element of the data collection configuration:

   <resourceType name="bgpPeerEntry" label="BGP Peer"
                 resourceLabel="ASN ${bgpPeerRemoteAs}, Peer ${bgpPeerRemoteAddr}">
     <persistenceSelectorStrategy class="org.opennms.netmgt.collectd.PersistAllSelectorStrategy"/>
     <storageStrategy class="org.opennms.netmgt.dao.support.IndexStorageStrategy"/>
   </resourceType>

Until OpenNMS 1.6.8, only the IndexStorageStrategy was available for generic-indexed resource types. This strategy stores data under the repository in the form:

[nodeID]/[typeName]/[instanceName]/[attributeName].[suffix]

Where:

nodeID 
The numeric database ID of the node to which the data belongs
typeName 
The textual name of the resource type. In the example above, bgpPeerEntry.
instanceName 
A version of the instance name returned by the collector, minimally transformed to play nicely with all filesystems.
attributeName 
The name of the attribute as returned by the collector. Corresponds to the alias attribute of an SNMP mibObj.
suffix 
Usually one of rrd, jrb depending on the RRD strategy in use.

The Problem

The IndexStorageStrategy is great for resources whose instances map directly to an unshifting identifier (e.g. the BGP peer's IP address for the bgpPeerEntry resource type). It's less than ideal, though, for types that have a purely numeric instance whose mapping may change across restarts or reconfigurations of the node. Examples would include the filesystem and device tables in most SNMP MIBs; if instance 7 maps to /mnt/cdrom and instance 8 to /mnt/floppy, but the user unmounts both filesystems and later remounts them in reverse order, the data persisted for the two instances will not correspondingly reverse locations.

Alejandro Galue added a couple of new storage strategies in trunk to address this problem for the specific cases of frame-relay PVC and host-resources storage resources. I (User:Jeffg) set out to write corresponding storage strategies for other commonly collected SNMP data, mostly filesystems, and found that I was writing the same class over and over, changing only the object identifier of the columnar object used to construct a better instance name. Instead of going down that road, I decided to create a strategies that can be configured by adding param child elements under the storageStrategy of a resource type definition.

SiblingColumnStorageStrategy

High-level description

This strategy should fetch the value of a columnar object using the same instance identifier as the one returned by the collector, and use that value as the basis of the instance name in the path that it returns. The "sibling" column must belong to either the same table as all objects being collected for the given resource type, or to a table that augments (either explicitly or implicitly) that table. For instance, a column from the IETF ipMRouteTable (.1.3.6.1.2.1.83.1.1.2) could be used with this strategy along with columns for the Cisco-proprietary ciscoIpMRouteTable (.1.3.6.1.4.1.9.10.2.1.1.2) because the latter augments the former and therefore uses the same index. The OID of the sibling column should be specified by the user as a parameter to the storage strategy in the resource type definition. In the interest of giving the user the greatest possible control over the returned instance name, this strategy should substitute only path-separator characters (/ and \) and allow the user to specify any further string-replacement operations through additional parameters to the storage strategy.

Configuration Example

  <resourceType name="dskIndex" label="Disk Table Index (UCD-SNMP MIB)" resourceLabel="${ns-dskPath}">
    <persistenceSelectorStrategy class="org.opennms.netmgt.collectd.PersistAllSelectorStrategy"/>
      <storageStrategy class="org.opennms.netmgt.dao.support.SiblingColumnStorageStrategy">
        <parameter key="sibling-column-oid" value=".1.3.6.1.4.1.2021.9.1.2" />
        <parameter key="replace-first" value="s/^-$/_root_fs/" />
        <parameter key="replace-all" value="s/^-//" />
        <parameter key="replace-all" value="s/\\s//" />
        <parameter key="replace-all" value="s/:\\\\.*//" />
      </storageStrategy>
  </resourceType>

This resource type duplicates the behavior of the existing HostFileSystemStorageStrategy for data in the the UCD-SNMP-MIB dskTable. Now instead of a numeric dskIndex instance identifier that is subject to shifts, data for each conceptual row will be stored according to the value of the dskPath column in the same conceptual row. This strategy can be reused for any table with an unstable numeric index that has a reliably unique column whose value is suitable for use as an auxiliary instance ID.

Important Changes since 1.9.90

In order to provide a solution for Bug NMS-4494, some changes have been made on the configuration for the SiblingColumnStorageStrategy:

The new Sibling Strategy, will use any MibObj attribute defined as String. So, they should need sibling-column-name instead of sibling-column-oid.

Example:

<resourceType name="hrStorageIndex" label="Storage (MIB-2 Host Resources)" resourceLabel="${hrStorageDescr}">
   <persistenceSelectorStrategy class="org.opennms.netmgt.collectd.PersistAllSelectorStrategy"/>
   <storageStrategy class="org.opennms.netmgt.dao.support.SiblingColumnStorageStrategy">
      <parameter key="sibling-column-name" value="hrStorageDescr" />
      <parameter key="replace-first" value="s/^-$/_root_fs/" />
      <parameter key="replace-all" value="s/^-//" />
      <parameter key="replace-all" value="s/\s//" />
      <parameter key="replace-all" value="s/:\\.*//" />
   </storageStrategy>
</resourceType>

If you want to upgrade from previous version of OpenNMS you should take this in consideration, because otherwise, all resources which belongs to a resource type which uses a bad configured SiblingColumnStorageStrategy will be ignored (i.e., no RRD/JRB will get updated).