From OpenNMS

Contents 1 Provisioning Integration 1.1 Assumptions 1.2 Implementation 1.3 Operational Parameters 1.3.1 Logging 1.3.2 Exported XML 1.3.3 File Format and Location 1.3.4 Performance 1.3.5 Importer Operation 1.3.5.1 Reload Import Event 1.3.5.2 Import Started Event 1.3.5.3 Import Successful Event 1.3.5.4 Import Failure Event 1.3.6 JMX MBean

Provisioning Integration

Assumptions

• TEC is the name of your Provisioning DB

Implementation

• How to do partial imports?
• How to force an import?

Brief description of the integration process Adding/Deleting/Changing sites works by..

Operational Parameters

• Integration is configured in the file: model-importer.properties

#
#http://quartz.sourceforge.net/javadoc/org/quartz/CronTrigger.html
#Field Name             Allowed Values          Allowed Special Characters
#Seconds                0-59                            , - * /
#Minutes                0-59                            , - * /
#Hours                  0-23                            , - * /
#Day-of-month   1-31                            , - * ? / L W C
#Month                  1-12 or JAN-DEC         , - * /
#Day-of-Week    1-7 or SUN-SAT          , - * ? / L C #
#Year (Opt)             empty, 1970-2099        , - * /

importer.importSchedule = 0 0 0 1 1 ? 2023

# Import URL
importer.importURL = file:/tmp/tec_dump.xml

#
# For SNMP scans
importer.scanThreads = 10
#
# Database persistence threads
importer.writeThreads = 8

• How Changes are picked up:

OpenNMS recognizes changes to the model-importer.properties file when the OpenNMS service is started. Note that in the next generation of OpenNMS, all services will either dynamically detect their respective config file changes or an event will be used to tell the service to re-read its configuration/add to its configuration.

Logging

OpenNMS uses the Log4J library for logging. (see the /etc/opennms/log4j.properties file).

Exported XML

If the export from TEC contains special characters, there needs to be special entity definitions in the XML header. Example:

<?xml version="1.0" encoding="utf-8"?>
  <!DOCTYPE myData [
    <!ENTITY auml    "ä" >
    <!ENTITY ouml    "ö" >
    <!ENTITY uuml    "ü" >
    <!ENTITY szlig   "ß" >
  ]>

File Format and Location

• Integration XML Format

Performance

The OpenNMS team has worked hard provide this integration with the performance required for very large networks. The current default settings in the model-importer.properties file allowed for an initial import of almost 45,000 devices in less than 1 hour. Subsequent runs are more intensive and take approximately 2 hours to complete. This performance improves tremendously when all devices provisioned with the SNMP service are responding to SNMP. At least half of the update time was spent waiting for SNMP timeouts. There is opportunity for improving performance by increasing the number of SNMP scanning threads in the model-importer.properties file.

#URL
importer.importURL = file:/tmp/tec_dump.xml
#importer.importURL = https://provisioning.mycompany.com/tecadmin/export?Username=XXXXXX&Password=XXXXXX

#Schedule
importer.importSchedule = 0 0 3 * * ?

#Performance Tuning
importer.scanThreads = 50
importer.writeThreads = 8
importer.maxActiveConnections = 10

Importer Operation

Four new OpenNMS events were added to support operation of the Importer service:

 uei.opennms.org/internal/importer/reloadImport
 uei.opennms.org/internal/importer/importStarted
 uei.opennms.org/internal/importer/importSuccessful
 uei.opennms.org/internal/importer/importFailure

Reload Import Event

An event that can be used to cause the OpenNMS Importer service to be run immediately. See: How to force an import

Import Started Event

An event is sent at the beginning of the Import. This can be viewed in the OpenNMS event browser.

Import Successful Event

An event is sent upon successful completion of the OpenNMS Import integration. This event contains statistics about the performance of the import.

 ImporterService: Finished Importing: Deletes: 0 Updates: 1 Inserts: 855
 Importing: 3m 18s 871ms 
   Loading Resource: URL [file:/tmp/tec_dump.xml]: 873ms
   Auditing: 643ms Preprocessing: 2m 45s 882ms
   Total Scan Time: 326.047s  Avg Scan Time: 380.8960280373832ms per node
   Processing: 3m 17s 355ms
     Total Write Time: 212.918s Avg Write Time: 248.73598130841123ms per node
     Total Event Sending Time: 4.057s Avg Event Sending Time: 4.739485981308412ms per node Avg 0.5312295403954432ms per event

Import Failure Event

An event is sent upon the failure of the OpenNMS Importer integration. A parameter contains a message for the cause of the failure. This event should be configured to create an alarm.

JMX MBean

OpenNMS has instrumented the Importer service with JMX. Import statistics can be pulled from the JMX MBean and also viewed here:

http://localhost:8280/mbean?objectname=OpenNMS%3AName%3DImporterd