SNMP — Accessing Raw Data :: SAS(R) IT Resource Management 3.3: Administrator's Guide

Sources of SNMP Data

Simple Network Management Protocol (SNMP) data can be collected from the following sources:

HP Network Node Manager (HP NNM)
Round-Robin Database

Reporting on the data available from the SNMP adapter can be enhanced by mapping devices to their device type. For information about how to enable this functionality, see Working with the ITMS_SNMP_DeviceType Lookup Table.

The SNMP adapter is designed with a data model that contains MIB2 data and can be extended to other MIBs. For information about how to do so, see ForWorking with Management Information Base (MIB) Definitions for SNMP Staged Tables.

In order to read the raw data from either HP NNM or rrdtool, it is necessary to create Perl scripts as part of the staging process. This means that Perl must be installed on the machine that the SAS program runs on, that is, the SAS Workspace Server, the SAS batch server, or interactive SAS.

The Perl scripts that are used by the staging code are generated as part of the code generation of the staging job. This code can be viewed and modified as necessary in the Code screen of the job. To run these scripts in batch mode, it is necessary to set the xcmd option for the Batch server. To do so, change set USERMODS_OPTIONS= to

set
                     USERMODS_OPTIONS=XCMD

.

By default, on the Workspace server, this option is turned off. To turn the option on, perform the following steps:

Launch SAS Management Console. Log on as an administrator.
Expand the Server Manager in the left panel.
Expand SASITRM. Then expand SASITRM-Logical Workspace Server.

Expanded Left Panel of SAS Management Console
Right-click SASITRM-Workspace Server and select Properties. The Workspace Server Properties dialog box appears.
Select the Options tab and click Advanced Options.
Select the Launch Properties tab to open the following dialog box.

Launch Properties Tab of the Advanced Options Dialog Box
Select the Allow XCMD check box.
Click OK to close all the open dialog boxes.
Stop and then restart your Object Spawner service.

Windows Specifics: To stop your Object Spawner service, select Start ProgamsSASSAS Configuration<configuration-name>Object SpawnerStop. To restart your Object Spawner service, select StartProgamsSASSAS Configuration<configuration-name>Object SpawnerStop.

UNIX Specifics: To stop and then restart your Object Spawner service, from the command line, change directories to SAS-config-dir/Lev1/SASMain/ObjectSpawner. Stop the object spawner by issuing this command: $ ./ObjectSpawner.sh stop. When you receive a confirmation that the object spawner has stopped, start it again by issuing this command: $ ./ObjectSpawner.sh start. You should receive a confirmation that the object spawner has started.

z/OS Specifics: To stop and restart your Object Spawner service, stop and restart the z/OS started task associated with the Object Spawner. Follow the stop and restart procedures that are established at your site.

Extracting Data By Using HP NNM Tools

For versions of HP NNM that are earlier than HP NNMi 8.x, there are tools that produce a repository of SNMP data that can be processed the SAS IT Resource Management solution.

The two commands to produce the SNMP data are:

snmpCollect, which gathers SNMP data into binary files
snmpColDump, which reads the binary files and writes the data into ASCII files that can be processed by the SNMP staging transformation

The snmpCollect executable uses a file called snmpCol.conf to determine the metrics to collect and the devices from which to gather data.

You can create the snmpCol.conf file by using either of the following methods.

Use the sample UNIX script (snmpmkcf) that SAS IT Resource Management provides. The sample script is located in the sasmisc folder. It reads the snmphost and the snmpvars files, which are also located in the sasmisc folder, and creates the snmpCol.conf file.
- The snmphost file contains a list of the device names for which SNMP data is being gathered.
  
  Note: The job will run successfully even if, in the Perl script, the alias of the host name contains a hyphen (-) or an underscore (_).
- The snmpvars file contains a list of the SNMP metrics that are to be gathered from each device. By default, the values in this list are the metrics that are in the recommended staging tables.
Note: This method creates the snmpCol.conf file more quickly than using the NNM graphical user interface.
Use the HP NNM graphical user interface to create snmpCol.conf. The user interface prompts you to select the metrics that you want to gather and the devices about which you want to gather data. Using your responses, it then creates the snmpCol.conf file.

When the snmpCol.conf file is created, you can run the snmpCollect command to gather data. You can run the collect as often as you choose. The command collects a set of files in the location that you specify. For each metric and interface, two files are generated:

metricName.interface-number, that contains the raw data
metricName.interface-number!, that contains the information about the metrics that are collected

Consider the following example. For the metric called ifOctets and for interface 3, the ifOctets.3 and ifOctets.3! files are generated. The SNMP staging code runs the snmpColDump command against these files to get the rawdata that is to be read and loaded into staged tables. In order to read the data, the snmpColDump executable requires both the file that contains the binary data and file that contains the information about the metrics that are collected.

Tip

The binary data files continue to accumulate until they are deleted. For best results, these data files (whose names do not end with "!") should be moved to another location before staging. The snmpCollect command then creates a new set of binary files into which the data is to be collected. The files that contain the information about the metrics (whose names end with "!") should be copied (not moved) to the other location as well. In this way, the snmpCollect command can continue to use these metrics as it collects the raw data. The staging code can then read the files from the location to which they were moved or copied.

Extracting Data By Using the Round-Robin Database Tool (RRDTOOL)

The SNMP adapter can read data from a round-robin database that was created with rrdtool.

Note: The round-robin database tool, rrdtool, is a freeware package that is available for download from Tobias Oetiker. This software is widely used, often in conjunction with Tobias Oetiker's MRTG package, which is a real-time monitor of data.

The SNMP adapter reads only those round-robin databases that contain one round-robin database for each combination of the device, the network interface to which the device is connected, and the metric. The data can be gathered at any interval, and any metrics and devices can be gathered.

In order to produce the correct set of round-robin databases, SAS IT Resource Management provides a sample Perl script for data collection and storage in the round-robin database. This script, called snmpuprd.pl, is available in the sasmisc folder. The section at the top of the script enables you to specify modifications that control how the data is collected. The Perl script uses the snmphost and snmpvars files, which are located in the sasmisc folder, to determine which metrics to gather from your devices. It also uses the snmpwalk command to collect the snmp data. When you have the script set up correctly, you can use your operating system's scheduling facility to run the script at the appropriate interval.

Note: If the data is being collected by a tool other than the snmpuprd.pl script provided with SAS IT Resource Management, then it might be necessary to modify the staging code to handle the format differences. This code can be modified by one of the following methods:

Edit the code from the staging job’s Code window. To do so, change the Code Generation Mode option at the top of the screen to All user written.
Deploy the job. The deployed code can be modified from an external editor. The code to be modified begins with this comment line:
```
/*---- Create script to read data from an RRD  ----*/
```

Note: You can backload data from the rrdtool. To so, specify the Number of days to load option in the Staging Parameters property tab of the SNMP staging transformation. (This option is available only if the chosen access command is RRDTool.)

Notes about the SNMPWALK Command

The snmpwalk command is an executable that is available as part of the Net-SNMP package. This package is available as freeware that you can download from the Sourceforge.net website. The snmpwalk command provides an easy way to read SNMP data from specified devices.

The SNMP adapter uses snmpwalk in two places:

Using rrdtool: If you choose to store your rawdata using rrdtool, the script that is provided with SAS IT Resource Management uses snmpwalk to gather the data to put in the round-robin database.
Staging the data: The SNMP staging transformation provides the ability to collect the non-numeric data at staging time instead of at the normal regular intervals that the rest of the SNMP data is collected. This can be useful for character data that does not change frequently. If you want to collect character data and are using rrdtool to gather your rawdata, then you must collect it at staging time, because rrdtool does not store character data. If you are using HP NNM, you can set it up to gather the character data at regular intervals along with the rest of the data. Alternatively, you can choose to collect it at staging time.

If you want to use the snmpwalk method, go to the Staging Parameters tab of the SNMP transformation's property window. On the Use snmpwalk to gather character data option, select Yes. Then fill in values for the following options:
- snmpwalk Executable: This is the path and exe file that contains the snmpwalk command.
- HostFile for snmpwalk: This is the path and file that contains a list of the devices to poll for character data.
- Community value for snmpwalk: This option is usually "public." Check with your SNMP administrator to ascertain the correct value for your site.
  
  If you choose to use snmpwalk, then a script is generated and executes when the staging code runs. This script executes the snmpwalk command in order to gather the SNMP data for specified character metrics and adds that data directly to the staged tables.

Notes about the SSH Host Command

The secure shell host command is an executable object that is available as part of the SNMP adapter’s functionality when data is only in a round-robin database. This command specifies the rsh or ssh version of the command and the name of the host for running the rrdtool command. The rrdtool command facilitates reading data from round-robin database files that are located on other hostnames. Entering the ssh version of the command triggers this ssh functionality. This is the format of the command:

ssh
                     user@hostname

. It is entered in the rsh/ssh host command field on the Staging Parameters tab of the Properties dialog box for the SNMP staging transformation.

If you enter a value in the rsh/ssh host command field that begins with ssh, then SAS IT Resource Management assumes that this job is running on a UNIX environment. The Perl script is changed so that it uses the UNIX find command to get the list of round-robin database files from the other hostnames.

To enable this functionality, set up ssh authentication using the SSH key-gen file and copy the file that stores the key to the host where the rrdtool executable and raw data are located. After the SSH key-gen file is copied, make sure that you can access the target host that is issuing the ssh command from the source host. If you have alias names for your target host, make sure that you execute the ssh command manually from the source host with that alias. By doing this, the command generates the host/RSA key for that alias host name so that the expected warning or error message is avoided during execution of the SNMP staging job. The following message is an example of the warning or error message:

Host key verification failed

To use this ssh method, go to the Staging Parameters tab of the Properties dialog box for the SNMP staging transformation. Then fill in values for the following options:

Raw data input directory: Enter the location of the raw data in the target host.
rsh/ssh host command: Start this command with SSH (for example: ssh user@hostname).
Use snmpwalk to gather character data: Select Yes to use snmpwalk. A script is generated and executes when the staging code runs. This script executes the snmpwalk command to gather the SNMP data for specified character metrics and adds that data directly to the staged tables.

The default value for this option is No.
Choose access command: Enter RRDTool.
rrdtool executable: Enter the rrdtool location of the target host machine.

Working with the ITMS_SNMP_DeviceType Lookup Table

In order to provide better reporting, the SNMP adapter provides a SAS data table that enables you to indicate the type of device for each of the devices for which you are gathering data. This table is located by default in the SASHELP library, and contains no data. To use this table to define your device types, perform the following steps from the SAS IT Resource Management client:

Locate, or create, the IT data mart that you want to use for staging SNMP data.
Locate the Administrative folder. Right-click the library within that folder. (The library is called Adminnnnn, where nnnn is a random number that was generated when the IT data mart was created.)

Select View Libname.
Select the entire LIBNAME statement in the displayed box. Then right-click the LIBNAME statement and select Copy.
Launch an interactive session of SAS. Use the Paste function to paste the LIBNAME statement into an Editor window.
Submit the LIBNAME statement.
In the Explorer window, locate the SAS data table called Itms_snmp_devicetype by drilling down into the SASHELP library.
Use the Copy and Paste functions to copy this table to the Adminnnnn library.
Now you can edit the table that you copied to Adminnnnn library. To do so, use VIEWTABLE to add and update rows. (To access VIEWTABLE, double-click the table in the Explorer window.)
The value in the Device column should be the name of the device, and deviceType is the type of device.
- The name of the device should match the device for which SNMP is gathering data. Make sure that the device names use the same case.
- DeviceType can be any character string that helps identify the device. For example, "Router," "Switch," or "Server" would be reasonable choices for device Type.
You can also update this table programmatically as long as the table has the device and deviceType columns.

Once this table is populated, when you run the staging code, the deviceType column in all the SNMP staging tables will have the correct value from the lookup table. If a device is not in the deviceType table, then deviceType is set to "Unknown."

If your devices are named using a convention (for example, the device type is always the first part of the name, like router_2123), you can create the deviceType value programmatically. To do so, change the expression for the deviceType column in the staged table metadata. By default, the expression for the deviceType column is: put(device, ?? $deviceType.)

You can change this to a different expression. An example is:

upcase(scan(device,
                     1, "_"))

Or, you can use standard SAS DATA step code to create an expression. An example is:

if (device eq: "R") then 
		deviceType = "Router";
	else if (device eq: "W") then 
		deviceType = "Switch";
	else if (device eq: "S") then 
		deviceType = "Server";
	else
		deviceType = "Unknown";

Working with Management Information Base (MIB) Definitions for SNMP Staged Tables

The SAS macro %rmmssnmp enables the SNMP adapter to read Management Information Base (MIB) definitions and create the metadata for SAS IT Resource Management staged tables. This macro enables you to generate staged data from any MIBs that you have. The SAS macro is available in the SAS autocall macro library so that you can use it from any SAS session.

To use the macro, you must first create a job with an SNMP transformation in SAS IT Resource Management. To do so, perform the following steps:

In the SAS IT Resource Management client, navigate to the IT Data Marts tree.
Navigate to an existing IT data mart, or create a new one. (To create a new IT data mart, from the menu bar, select NewIT Resource ManagementIT Data Mart. The IT Data Mart wizard opens and prompts you to enter information to specify your new IT data mart.)
Create a folder in the IT data mart for your new job and tables.
In the new folder, create a new job. (To create a new job, select NewJob. Then enter the information to specify your new job.)
In the process flow diagram, drag the SNMP transformation from the ITRM Transformations folder of the Transformations tree and drop it onto the job.
Save and close the job.
Select the job in the tree and in the Basic Properties window, scroll to the Metadata ID property, and make a note of the value. This is the job ID that you need to run the macro in the next set of steps.

After the job is created, you can run the %rmmssnmp macro. To do so, perform the following steps:

Invoke an interactive SAS session.
Specify the parameters for the %rmmssnmp macro. For information, see Introduction to the Macros in SAS IT Resource Management.
Submit the macro call to read the MIB definitions. The macro can generate a report of the staged tables that would be created from this definition, or it can create the staged table metadata.