NFSiostat Plotter V3

Introduction

I wrote NFSiostat plotter to plot output from NFSiostat when I ran some tests. I had several requests for improvements and this latest version, v3, incorporates these suggestions. I’ve made a few changes to the basic code since the article:


  • Normally nfsiostat doesn’t capture the CPU utilization but this can be an important part of analyzing NFS client performance. In this version of the code you need to also run iostat while you run nfsiostat so that the CPU utilization is captured. With this version of code, you have to run iostat and run iostat_plotter_v3.py to produce the iostat output (which is read by nfsiostat_plotter).
  • Moved the lengend outside the plot to the top right
  • Shrunk the size of the legend so you can plot more NFS file systems (up to about four before the legend lables start to overlap).
  • The command option “-c” was added so you could combine the NFS file systems onto a single plot (total of 4 plots)
  • The code can read either sysstat v9.x or sysstat v10.x format for nfsiostat.
  • The legend labels are auto-sized based on the string length (it’s a heuristic algorithm).
  • A simple “-h” option as added that produces a small help output.

The changes were pretty simple and I hope they prove useful.

This tool, iostat_plotter_v3.py simply takes the output from iotstat, parsers the data to some degree, and creates plots and an HTML report. The intent is to allow plot the time history of the IO rather than have to stare at numbers flashing by on the command line.

This version is covered under the GPLv2 license.

With this version of the nfsiostat plotting tool, you can either plot the nfsiostat output from a single NFS file system or as many NFS file systems as you want. If you run nfsiostat_plotter_v3.py without any options it will create a series of four plots for each NFS file system. If you have 2 NFS file systems it will create a total of 8 plots (4 * 2). If you run it with the “-c” option, if will “combine” the NFS file systems into the plots resulting in a total of 4 plots regardless of the number of the number of NFS fiel systems in the nfsiostat output.

To begin, you need to get the nfsiostat output using a command such as:

[laytonj@home8 IOSTAT]$ nfsiostat -h -m -t 1 100 > nfsiostat.out


which collects information on all NFS file systems.The example capture the output every second for a total of 100 seconds. Most importantly, the output is in MB/s via the “-m” option. If you change the output to something other than MB/s, you will need to change the plot labels (this isn’t hard to do but just remember that you have to do this).

In conjunction with nfsiostat, you should run iostat so that the CPU utilization information is captured. A simple way to do this is to put both commands in the background using the following command:

/usr/bin/iostat -c -d -x -t -m 1 160 > iostat.out&
/usr/bin/nfsiostat -h -m -t 1 160 > nfsiostat.out&


By using the anpersand symbol at the end of the line, the command are put into the background. Both commands are run on the NFS client node.

Right after you run these two commands, you can run the application you want to profile. For this example, the application runs for less than 160 seconds (be sure you check this since you could miss information after 160 seconds).

Then you need to run iostat_plotter first to process the iostat information. Follow the directions on this page for running iostat_plotter. After it is done running a file named “iostat.picklet” will be produced. You need this file when you run nfsiostat (it contains the CPU utilization information).

After running iostat_plotter_v3.py you then run nfsiostat_plotter_v3.py to create the HTML document with the plots. The command is,

[laytonjb ~]$ nfsiostat_plotter_v3.py nfsiostat.out


where “nfsiostat.out” is the output from nfsiostat (don’t forget to run iostat_plotter_v3.py first!) The code is written in Python (obviously) and uses the shlex, time, os, and matplotlib modules. If you run it this way, a set of 4 plots for each NFS file systems will be created. If you want to combined the NFS file systems into a single set of plots, then you use the following command:

[laytonjb ~]$ nfsiostat_plotter_v3.py -c nfsiostat.out


The “-c” option will “combine” the NFS file systems.

When nfsiostat_plotter_v3 is done it will create a subdirectory “HTML_REPORT” that contains the plots and an html file “report.html”. Open that html file in a browser or word processor and you will see the plots and a small write-up about them. Feel free to modify the code but please send back changes since others might like to see what you have done.

If you want to get the help output just run the application as,

[laytonjb ~]$ nfsiostat_plotter_v3.py -h


I don’t have an exampl at the moment but stay tuned for one. You can download the file using this link (sorry – I have to use Dropbox because WordPress does not allow .py files).

One Response to NFSiostat Plotter V3

  1. Pingback: IOSTAT Plotter and NFSiostat plotter updates (V3!) | clusterbuffer

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: