Improving Memory Performance

Out-of-memory issues can cause ETL jobs to fail.

To resolve out-of-memory issues on UNIX platforms, set the following SAS options as shown:

MEMSIZE=512M
SORTSIZE=256M
BUFSIZE=48K
BUFNO=1500

To resolve out-of-memory issues on z/OS platforms, set the following SAS options as shown:

SORTSIZE=256M
BUFSIZE=48K
BUFNO=1500

Note: In addition, using SAS IT Resource Management software on z/OS requires that batch jobs have a REGION of at least 1024M. A parallel ASSIZEMAX of 1024M is also required.

To resolve out-of-memory issues on Windows (64-bit) platforms, set the following SAS options as shown:

MEMSIZE=2G
SORTSIZE=256M

Tip

SAS options can be set at configuration time, invocation time, or run time.

If you use the Adapter Setup wizard and specify multiple domains, particularly if you simultaneously specify a full configuration level or multiple time periods, you might experience job failure due to an out of memory condition. To correct this problem, split the jobs into smaller pieces by running the Adapter Setup wizard several times with fewer domain categories, for example, only three domains.

Note: If you continue to experience out-of-memory issues, you might need to further increase the value of the MEMSIZE option. For information about metadata memory issues, see Metadata Server Memory Issues.

Limiting the Processing of SMF Records by System

An aggregation filter can be used to limit the SMF records that are processed into an aggregation table to specific systems. However, the staging code must process the data that will not be aggregated. In addition, the filter must be set on each SMF aggregation. It is more efficient to filter data during the staging job processing.

To limit the SMF records to process only specific systems during a staging job process, add an IF statement in the IMACFILE MXG SOURCLIB member. For example, if you want to process only those records from the 'PROD' and 'TEST' systems, place a copy of the IMACFILE MXG SOURCLIB member into your MXGUSER source library. Modify the user IMACFILE MXGUSER member with the following SAS statement:

IF SYSTEM IN ('PROD','TEST');   /*Only process records for these systems */

where the systems for which you want to process SMF records are listed in the IN clause. For additional examples or information about coding the IF statement, please refer to the documentation found in the IMACFILE MXG SOURCLIB member.

Note: Make sure you have a working backup copy of your SAS IT Resource Management installation files and IT data marts before you make any changes.

Staging Jobs Fail Due to Lack of Space

Staging job fails due to lack of space on the WORK library on Windows.

The location of the Work library defaults to the user's C: drive. The space on this drive might not be adequate for running a staging job. To resolve this issue, edit the sas.cfg file to point WORK to a location that has more space.

Aggregation Jobs Require Lengthy Processing Times

Some aggregation jobs that contain rank variables or join columns can require a long time to finish.

For best results, minimize the number of rank variables or join columns that you specify in a summarized aggregation table.

Information Map Jobs Require Lengthy Execution Times

Executing a job that contains multiple information maps can take a long time. Information maps facilitate the process of generating reports on your IT data and are the required input to SAS Web Report Studio. Jobs that generate many information maps can take a long time to execute. Although SAS Enterprise Guide can generate reports from tables or information maps, using information maps is preferred.

The Adapter Setup wizard creates separate jobs for Information Map transformations. You do not need to schedule these Information Map jobs to run nightly. These Information Map jobs need to be run initially only once—after the underlying tables are written, and later if something changes about the structure of the tables for which information maps are to be created. For example, if you change the class columns or statistics in an Aggregation transformation, that will change the resulting aggregation table. In that situation, the corresponding Information Map must be re-created as well.

If you are sure that you will never use specific information maps, remove the unneeded Information Map transformation from the Process Flow Diagram before you deploy the job.