How MSU and MIPS Columns Are Included in Staged Tables for the SMF Adapter
Overview of the MIPS and MSU Automation Process
In SAS IT Resource Management
3.2 and later, additional columns are included. These columns enhance
the reporting of resource utilization for various categories of SMF
activity, such as systems, LPARs, workloads, workload periods, jobs,
TSO and OMVS sessions, and more. The additional columns reflect MSU
(millions of service units) and MIPS (millions of instructions per
second) based on CPU time expended during the activity. These columns
are distinct from other MSU-related columns that are already present
in the SMF rawdata. The following topics explain how the population
of these new resource utilization columns in several of the SMF performance-based
adapters and tables were automated.
Calculation Dependencies for MIPS and MSU Columns
The calculation of MIPS
and MSU columns for SMF staged tables depend on the following items:
As noted in the preceding
list of required items, the new MSU and MIPS columns require that
the MXG RMF Interval dataset be created. This table requires SMF record
types 70, 71, 72, 75 and 78 to be present in the rawdata that is being
processed. If these record types are not present, the RMF Interval
dataset will be empty and the MIPS columns will be missing. For more information,
see New MSU and MIPS Fields in the XJOBS Table That Depend on RMF Data.
The presence of these
record types is especially important when staging the XJOBS table
if you have a separate staging job for the XJOBS staged table. If
the RMF data is not stored on the same rawdata file as the SMF records
for jobs, concatenate the two SMF files together so that both job-related
and RMF-related SMF records are available to the XJOBS staging job.
It is not necessary to add the RMF Interval table, XRMFINT, to the
XJOBS staging job.
Working with LSPR Tables
SAS IT Resource Management
supplies an initial copy of a SAS data set that is called ITMS_SMF_LSPRDATA.
The version of the LSPR table that forms the basis for the table that
is shipped with SAS IT Resource Management 3.3 is z/OS V1 R11. It
is dated July 2012.
This data set is stored
in the SASHELP library and contains Large System Performance Reference
(LSPR) information that is based on the IBM table for z/OS V1 R9.
The data set also contains columns that enhance the reporting of resource
utilization for various categories of System Management Facility (SMF)
activity. Among these categories are systems, logical partitions (LPARs),
workloads, workload periods, TSO and OMVS sessions, and jobs. The
additional columns are distinct from other MSU-related columns that
are already present in the SMF raw data. They reflect MSU and MIPS
and are based on the CPU time expended during the activity. You can
use the LSPR tables to prepare and initialize the MIPS and MSU columns.
SAS IT Resource Management
can merge the LSPR information with staged Resource Management Facility
(RMF) interval data. For information about this topic, see Staging Methodology for MIPS and MSU Columns.
Note: IBM might update this information
with new processors or other information at a later date. IBM might
also change the location of the LSPR table. Therefore, the LSPR information
might need to be updated. If an update is needed, then one of the
following two methods can be used to create an updated copy of the
LSPR information.
Initialization and Preparation of the MIPS and MSU Columns
Methods for Including MIPS and MSU Columns in Staging Tables
Tip
The following methods update
the IT Resource Management Admin library. Therefore, make sure that
you have a backup copy of that library before you proceed.
To import MIPS and MSU
columns for inclusion in your staging tables and to save them into
a permanent SAS data set, perform one of the two following methods:
filename lsprurl url 'http://www-304.ibm.com/servers/
resourcelink/lib03060.nsf/pages/
lsprITRzOSv1r13?OpenDocument&pathID=';
libname admin "<location-of-the-ADMIN-library>";
data work.lsprdata;
infile lsprurl;
retain found_toc found_thead found_th td_idx 0;
retain CPCFMODL LSPRCPUS LSPRPCI LSPRMSU;
length CPCFMODL $8;
keep CPCFMODL LSPRCPUS LSPRPCI LSPRMSU;
array td_array{3} LSPRCPUS LSPRPCI LSPRMSU;
input @;
if not found_toc and index(_infile_,'Table of contents') = 0
then return;
found_toc=1;
if index(_infile_,'<thead>') then found_thead=1;
if index(_infile_,'</thead>') then found_thead=0;
if not found_thead and index(_infile_,'<th ')
then do;
found_th = 1;
CPCFMODL = scan(_infile_,2,'<>');
end;
else if not found_thead and found_th and index(_infile_,'<td ')
then do;
td_idx + 1;
td_array{td_idx} = input(scan(_infile_,2,'<>'),comma10.);
if td_idx = 3
then do;
output;
td_idx = 0;
found_th = 0;
end;
end;
run;
proc sort data=work.lsprdata out=admin.itms_smf_lsprdata;
by CPCFMODL;
run;-
Create a Microsoft Excel spreadsheet that contains selected information from published (and currently available) IBM LSPR information. (You must reference the current published location of the IBM LSPR information. This information is currently located at
https://www-304.ibm.com/servers/resourcelink/lib03060.nsf/pages/lsprindex?OpenDocument.)Copy and paste a selected set of columns from the published LSPR data into a Windows Excel spreadsheet. The resulting workbook should look like the following table and reflect the information that is documented in the topic called Calculation Dependencies for MIPS and MSU Columns. -
/* Read the Microsoft Excel workbook that contains extracted information from IBM LSPR documentation and create a SAS data set from it. */ PROC IMPORT OUT=WORK.LSPRDATA DATAFILE=<"location-of-spreadsheet"> DBMS=EXCEL REPLACE; SHEET="Sheet1$"; GETNAMES=NO; MIXED=NO; SCANTEXT=YES; USEDATE=YES; SCANTIME=YES; RUN; /* Extract selected fields of information from the vendor-supplied LSPR resource information. */ DATA WORK.LSPRDATA(keep=CPCFMODL LSPRCPUS LSPRPCI LSPRMSU); SET WORK.LSPRDATA; LENGTH CPCFMODL $ 16; IF _n_=1 THEN DELETE; CPCFMODL=F1; LSPRCPUS=F2; LSPRPCI=F3; LSPRMSU=F4; run; /* Store IBM LSPR resource information in SAS data set in IT Resource Management Admin library. */ LIBNAME ADMIN <'location-of-ITRM-Admin-library'>; PROC SORT DATA=WORK.LSPRDATA out=ADMIN.ITMS_SMF_LSPRDATA; BY CPCFMODL; RUN;
How to Retain the Original LSPR Table
Applying software maintenance
or hot fixes might automatically update the LSPR table in SASHELP.
If you backed up the SASHELP library from your original SAS IT Resource
Management 3.3 installation, you can restore the default LSPR table
information that was included in that library. To do so, perform
the following steps:
Note: You need to have exclusive
access to the SASHELP library before you restore the original LSPR
table in the SASHELP library.
-
/*Allocate the backup copy of the SASHELP library */ /*that uses PROC COPY to copy only the LSPR table */ /*with the SELECT statement. */ /*Make sure you have exclusive udate access to the */ /*SASHELP SAS library before you submit code. */ libname bkuphelp '<physical-location-of-backup-sashelp>'; proc copy in=bkuphelp out=SASHELP; select ITMS_SMF_LSPRDATA; run;//jobname JOB //STEP1 EXEC SASPROC,OPTIONS=’SASHELP=SASHELP’ //SASHELP DD DSN='& prefix.llWn.SASHELP',DISP=OLD //SYSIN DD * libname bkuphelp '<physical-location-of-backup-sashelp> '; proc copy in=bkuphelp out=SASHELP; select ITMS_SMF_LSPRDATA; run; /*Ouput contents of LSPR data information */ PROC PRINT=SASHELP.ITMS_SMF_LSPRDATA; RUN;
Staging Methodology for MIPS and MSU Columns
The following steps
describe the methodology that SAS IT Resource Management uses to stage
RMF interval data.
-
After MXG stages the RMF interval data and before IT Resource Management begins its staging transformation processing, code is executed to extract the necessary columns from the RMF interval data (MXG data set RMFINTRV). This code captures the distinct resource utilization entries from the RMF interval data that has just been staged. Another code step also extracts the current contents of the existing Resource Utilization table.
-
Using the recently staged RMF interval data, code is executed to determine whether the available CPU models in the data are full capacity models or sub-capacity models. This model capacity information and a calculation of total available CPUs for the full capacity models are included with the rest of the RMF interval data extracted previously.
-
The latest available LSPR information from IBM is read so that it can be included in the Resource utilization table. The latest available LSPR information is available from one of the following two sources:
-
an updated copy in an IT Resource Management Admin library that was created by the customer. For information about this process, see Working with LSPR Tables.
-
The combined Resource Utilization table information (from the previous resource utilization table data and the new RMF interval data) and the extracted IBM LSPR resource information are merged together by the common column CPCFMODL. The merged result is the adjusted CPU model. This merge creates a new current copy of the Resource Utilization table, which replaces the existing copy. (The staging code created an empty Resource Utilization Table for this purpose.)If, as a result of this merge, a machine in your RMF data is not listed in your LSPR table, SAS IT Resource Management will halt the staging job and write the following ERROR message to the SAS log.
ERROR: The SMF data for system is associated with a mainframe model ERROR: <number> which does not appear in your LSPR table. ERROR: To preserve the integrity of your data mart, this job will stop.-
If the LSPR information and the subsequent calculations that are based on it are important to you, you should update your LSPR table as soon as possible. Do not attempt to rerun the staging job until the updated table has been implemented and stored in your Admin library. For information about updating your LSPR table, see Working with LSPR Tables.
-
If you are not interested in the LSPR information, you can bypass the error and continue processing the data. To do so, set the LSPR_ERROR_ON macro variable at the beginning of your staging job. The following code bypasses the error and enables processing to continue:
%LET LSPR_ERROR_ON=N;
This code sets all subsequent calculations that are based on the LSPR information to missing values. This results in other computed columns also having missing values. This situation is not statistically incorrect, but it can misrepresent MIPS and MSU usage in aggregations. The SAS log will contain notes about this instead of errors.NOTE: The SMF data for system is associated with a mainframe model NOTE: <number> which does not appear in your LSPR table. NOTE: You coded LSPR_ERROR_ON=N to ignore this problem.
-
-
The newly updated Resource Utilization table data can be used to include various parts of this information in staging extracts so that resource utilization computed columns for MSU and MIPS can be calculated. Lookups can be done using System, CEC, or CPU Model columns present in the Resource Utilization table. The following table shows the type of columns and the corresponding information for those columns that is available in the Resource Utilization table.
Note: The RMF CPU Resource Table
consists of 17 columns and a variable number of rows. (In the following
example, the table contains sample data for two rows.) In order to
present all the columns that are available in the table in a legible
format, the table was rotated 90 degrees. Each column was displayed
as rows, and each row was displayed as columns.
New MSU and MIPS Fields in the XJOBS Table That Depend on RMF Data
In SAS IT Resource Management
3.2 and later, there are new MSU and MIPS columns in the XJOBS staging
table. These columns require Resource Management Facility (RMF) interval
data to be present in the input System Management Facility (SMF) file.
If the SMF data does not contain the RMF 70, 71, 72, 75, and 78 type
data records, the following XJOBS MIPS-related columns are set to
missing:
To ensure that these MIPS columns are calculated in
the XJOBS table, process the RMF and XJOBS SMF data together in the
same staging job. If the SMF records that populate XJOBS were separated
from the ones that populate the RMF tables XRMFSYS, XRMFINT, XASM70L,
XASMCEL, XRMFWKL, XRMFWKP, and XTY72GO, then concatenate the two files
together. This enables the staging job to make the RMF data accessible
for the MSU and MIPS fields in the XJOBS table. You do not have to
add the RMF tables to the staging job for this technique to work correctly.
For more information about these columns, see Additional Notes about MIPS and MSU Calculations.
Additional Notes about MIPS and MSU Calculations
The following notes
pertain to the calculation of the MIPS and MSU columns.
-
When dealing with a full capacity model, any additional specialty CPU engines, such as IFA and ZIP, that have been configured for your system must be included in the calculations. These specialty CPU engines are added to the Total CPU Engines and this new value is stored in the resource table under a column labeled Total CPU Engines (calculated if full capacity model, else standard CPUs from RMF).The CPU Model ID value is adjusted to reflect the addition of any specialty engines. For example, suppose there are three additional specialty engines on a full capacity model (2094-706): two engines for IFA and one for ZIP. As a result, the CPU model value is adjusted from 2094-706 to 2094-709 to account for the three additional specialty engines. The latter model ID specification is used to match up to the IBM LSPR information to retrieve the relevant MSU and MIPS ratings. These ratings are then used to calculate the MSU/MIPS conversion factor. This refined methodology for the full capacity CPU model results in a different MsuMipsConversionFactor. The difference is due to using an adjusted CPU model and the LSPR MSU and MIPS ratings that are associated with this adjusted CPU model.In our sample, without using an adjusted CPU model, the calculation of the MsuMipsConversionFactor is 7.32. Using an adjusted CPU model for the calculation, the MsuMipsConversionFactor is calculated to be 7.48. This results in higher MIPS amounts for CPU activity on the full capacity models that include additional specialty engines.
-
The calculation of MIPS is done in two distinct ways, depending on the IT Resource Management table that is being staged.
-
For systems (IT Resource Management tables XRMFSYS and XRMFINT) and LPARs (IT Resource Management tables XASM70L and XASMCEL), MSU is calculated using the hardware-based active CPU times from the RMF type 70 records and the hardware service adjustment factor (SUSEC). MIPS are then calculated by multiplying the MSU result by the appropriate MsuMipsConversionFactor for the associated processor type.
-
For workloads and workload periods (IT Resource Management tables XRMFWKL, XRMFWKP, XTY72GO) and batch jobs, TSO sessions, and OMVS sessions (XJOBS), SAS IT Resource Management calculates MSU by using the application-based active CPU times from the RMF type 72 records and the software service adjustment factor (CECSUSEC). MIPS are then calculated by multiplying the MSU result by the appropriate MsuMipsConversionFactor for the associated processor type.
-
-
If comparisons are performed between system and LPAR MIPS values that are based on RMF type70 hardware-based active CPU times and other MIPS values, such as workload, workload period, jobs, and so on, that are based on RMF type72 application-based active CPU times, the system- and LPAR-based MIPS will be larger. The reason that the times are different is because the uncaptured CPU time is not available in the type72 record. The difference in values is typically in the 5-15% range.