Monitoring performance with WRKSYSSTS

Log in to save

WRKSYSSTS is the first screen anyone opens when somebody says the system is slow, and it is also the one read worst: people look at the CPU percentage at the top, and if it is low they conclude the problem lies elsewhere. Almost always the answer is further down, in the memory pool table.

Before reading, reset

The counters shown are cumulative since the last sample. If nobody ever resets them, you are looking at averages that include the night, the weekend and three restarts.

WRKSYSSTS RESET(*YES)

Then press F5 to refresh after a defined interval. The correct way is: reset, wait a few minutes of real work, refresh, read.

The numbers at the top

  • % CPU used — useful, but not enough. A system at 30% CPU can be doing nothing at all because it is waiting on disk.
  • % DB capability — how much of the CPU is going into database operations. A high value is not a problem in itself: on a business machine it is normal.
  • Jobs in system, Active jobs — if the number of active jobs grows without anyone having launched anything, something is accumulating them.

The pool table: where the answer is

This is the part that counts. For each memory pool, among other columns, you get:

  • DB Faults and Non-DB Faults — how many times per second the system had to fetch from disk something it did not have in memory.

A fault is not an error: it is how virtual memory normally works. The problem is the quantity. If an application pool shows faults in the hundreds per second, that pool does not have enough memory for the work running in it, and the machine is spending its time reading and rewriting the same pages.

There is no universal threshold valid for every installation: it depends on the number of processors, the disk configuration and the type of workload. The useful figure is the one for your own machine on a quiet day, written down once and used as a reference. It is the only comparison that means anything.

Pool 1, the machine pool, is a case apart: there, faults must be extremely low, in the single digits per second. If the machine pool starts faulting, the whole system suffers, because it holds the operating system code.

The right pool for the right work

On many installations all the batch and all the interactive work share the same pool. While the load is light it works; when a heavy batch starts reading a file with millions of rows, it takes the memory away and interactive users start waiting.

Separating the pools is a reversible, low-risk change: create a shared pool, assign the batch subsystem to it, and give each what it needs. The system can also adjust them itself if QPFRADJ is set to do so — which it usually already is, and is worth leaving on.

The disks are not on this screen

WRKSYSSTS says nothing about disk usage. You need the companion command:

WRKDSKSTS RESET(*YES)

The columns that matter are % Used (how full it is) and % Busy (how hard it is working). A disk consistently above 80-90% full degrades the performance of the whole system, because the code that manages space has to work harder to find free space.

For % Busy the same rule as for faults applies: what counts is the comparison with your own normal. One arm far busier than the others, though, almost always signals an imbalance in how the data is spread, and is worth looking at.

Who is consuming it

Once you have found the problem, you still need to know who is causing it:

WRKACTJOB SEQ(*CPUPCT)

That sorts jobs by CPU consumption and puts the culprits at the top. From there, option 5 and then 10 for the job log, or option 11 to see the calls in progress.

Anyone with Performance Tools installed also has WRKSYSACT, which shows consumption in real time rather than cumulatively — far more useful for catching a spike while it is happening.

A reading order that works

  1. WRKSYSSTS RESET(*YES), wait a few minutes, F5.
  2. Look at the faults per pool, not the CPU.
  3. WRKDSKSTS to rule out space or an unbalanced disk.
  4. WRKACTJOB SEQ(*CPUPCT) to put a name to the culprit.
  5. Write down the values from a normal day, so next time there is something to compare against.

Step 5 is the one nobody does and the one that makes all the others useless: without knowing what the machine looks like when it is healthy, any number read during a problem is just a number.


Which versions this works on

WRKSYSSTS, WRKDSKSTS, WRKACTJOB — long-standing commands, present on every release in use. WRKSYSACT, on the other hand, requires the Performance Tools product, licensed separately.

The SQL route: QSYS2.SYSTEM_STATUS_INFO — the same information as WRKSYSSTS can also be read as an SQL view. It is documented at least from IBM i 7.3 and has been extended with new columns in later releases:

SELECT ELAPSED_CPU_USED, SYSTEM_ASP_USED,
       CURRENT_TEMPORARY_STORAGE
  FROM QSYS2.SYSTEM_STATUS_INFO;

The advantage over the screen is that the value becomes available to a program: that is what makes automated checks possible. Before this view existed, the same data could only be obtained from system APIs (QWCRSSTS) called from a program, or by reading the screen by eye.

There is also QSYS2.SYSTEM_STATUS_INFO_BASIC, lighter to query when you only need the essential values.

The number of columns in these views grows with every release. If a column mentioned here does not exist on your machine, it is because it was added later: the documentation page for your version lists exactly what is there.

Sources

← Back to blog

Comments

No comments yet. Be the first to comment!

You need an account to comment. Log in · Sign up