NFSometer

From Linux NFS

(Difference between revisions)
Jump to: navigation, search
(News)
 
(50 intermediate revisions not shown)
Line 1: Line 1:
-
NFSometer is a framework for running and reporting performance characteristics of workloads across NFS protocol versions, options and Linux kernels.
+
NFSometer is a performance measurement framework for running workloads and reporting results across NFS protocol versions, NFS options and Linux NFS client implementations.
-
== Motivation ==
+
The NFSometer project was started at [http://www.netapp.com/ NetApp] as a way to automate performance testing of the Linux NFS client.  Since then it has grown to include many other features, notably the generation of reports.  It is our hope that by releasing this project under the GPLv2 license, the Linux NFS community will benefit from a better understanding of NFS performance characteristics and contribute improvements to NFSometer.
-
NFSometer was created to:
+
NFSometer is not designed to replace existing filesystem benchmarks.  Instead, it is designed to automate the running of existing filesystem benchmarks, gather NFS specific statistics and generate reports that make benchmark output more understandable.
 +
 
 +
Goals:
* Automate the time consuming process of running a variety of workloads looking for performance regressions between kernel versions, NFS protocol versions and NFS options.
* Automate the time consuming process of running a variety of workloads looking for performance regressions between kernel versions, NFS protocol versions and NFS options.
-
* Generate a report that allows the user to quickly understand the results and comparisons of workload runs, instead of having to manually dig through a lot of data.
+
* Generate reports that allows the user to understand the performance characteristics of workloads and easily compare performance characteristics of different configurations.
 +
* Provide a simple way to define workloads.  This allows users to evaluate NFS deployment scenarios with workloads representative of their unique needs.
-
== Example ==
+
== News ==
-
The following command will run the Connectathon test suite once for each NFS protocol version (v3, v4, v4.1) and generate a report:
+
* '''Wed June 28, 2017''': NFSometer v1.8 released! This version has only one bug fix that allows testing of v4.2 pnfs servers.
-
<pre>
+
* '''Thu March 3, 2016''': NFSometer v1.8 released! This version includes a few bugfixes.
-
$ ./nfsometer.py zero:/export cthon
+
-
Requested: 1 workloads X 3 nfsopts X 1 runs = 3 test runs
+
* '''Wed Jan 16, 2014''': NFSometer v1.7 released! This version includes bugfixes including a bad graphing bug, as well as many cleanups. Also, updated the example report.
-
Need to run 3 of 3 requested test runs
+
-
cthon - needs 1 runs of vers=3
+
-
cthon - needs 1 runs of vers=4
+
-
  cthon - needs 1 runs of vers=4,minorversion=1
+
-
Trace 1/3: 1/1 with workload: cthon, nfsopts: vers=3
+
* '''Tue Oct  8, 2013''': NFSometer v1.6 released! This version includes bugfixes, code cleanups and speedups and style/layout changes to reports
-
< SETUP >
+
-
Mounting: zero:/export...
+
-
remove run directory: /mnt/nfsometer_runroot/cthon
+
-
Unmounting: zero:/export...
+
-
< RUN >
+
-
Mounting: zero:/export...
+
-
Running command: cd /home/dros/nfsometer/tests/cthon/cthon && NFSTESTDIR="/mnt/nfsometer_runroot/cthon" ./runtests -a
+
-
Unmounting: zero:/export...
+
-
Results copied to: nfsometer_trace-cthon-vers=3-0
+
-
Trace 2/3: 1/1 with workload: cthon, nfsopts: vers=4
+
* '''Wed Mar  6, 2013:''' NFSometer v1.5 released! This version includes several bugfixes, including one that caused a traceback when parsing results from older kernels.
-
< SETUP >
+
-
Mounting: zero:/export...
+
-
remove run directory: /mnt/nfsometer_runroot/cthon
+
-
Unmounting: zero:/export...
+
-
< RUN >
+
-
Mounting: zero:/export...
+
-
Running command: cd /home/dros/nfsometer/tests/cthon/cthon && NFSTESTDIR="/mnt/nfsometer_runroot/cthon" ./runtests -a
+
-
Unmounting: zero:/export...
+
-
Results copied to: nfsometer_trace-cthon-vers=4-0
+
-
Trace 3/3: 1/1 with workload: cthon, nfsopts: vers=4,minorversion=1
+
* '''Fri Feb 22, 2013:''' NFSometer v1.4 released! This version has a redesign of reports, many bugfixes and several new features.
-
< SETUP >
+
-
Mounting: zero:/export...
+
-
remove run directory: /mnt/nfsometer_runroot/cthon
+
-
Unmounting: zero:/export...
+
-
< RUN >
+
-
Mounting: zero:/export...
+
-
Running command: cd /home/dros/nfsometer/tests/cthon/cthon && NFSTESTDIR="/mnt/nfsometer_runroot/cthon" ./runtests -a
+
-
Unmounting: zero:/export...
+
-
Results copied to: nfsometer_trace-cthon-vers=4,minorversion=1-0
+
-
Successfully ran 3 traces!
+
* '''Fri Dec  7, 2012:''' NFSometer v1.3 released! This version includes some major cleanup and bugfixes.
-
Generating reports:
+
* '''Fri Nov 30, 2012:''' NFSometer v1.2 released! This version includes, new options (notably -o replaces -m), basic loadgen functionality, probe mountopts and detect tags, replace poorly supported admin-script and client-script functionality with tags (-t option), bugfixes, etc
-
Saved: index.html
+
* '''Thu Jun 21, 2012:''' [http://www.netapp.com NetApp] is pleased to announce the GPLv2 release of NFSometer!
-
Saved: Report_Averages.html
+
 +
== Download ==
-
graph summary: 47 generated, 0 cached, 0 mismatch, 0 pruned,
+
The current release of nfsometer is version 1.9:
-
    0 hash files and 0 other files deleted
+
-
Report index: /home/dros/nfsometer/results/index.html
+
[http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.9.tar.gz nfsometer-1.9.tar.gz] (md5sum: 10121aa2fe48416bb8a38eb73c502468)
-
</pre>
+
-
[http://linux-nfs.org/~dros/nfsometer/nfsometer-report-screenshot-1.png Screenshot of the generated report]
+
Old releases:
-
== Download ==
+
* [http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.8.tar.gz nfsometer-1.8.tar.gz] (md5sum: 9b9e0cd8299f611e4d6fe51f71464fde)
 +
* [http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.7.tar.gz nfsometer-1.7.tar.gz] (md5sum: eabd49e42006bb103be023e086dc27a9)
 +
* [http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.6.tar.gz nfsometer-1.6.tar.gz] (md5sum: f82b981a050cc787d128edb076a1ea0f)
 +
* [http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.5.tar.gz nfsometer-1.5.tar.gz] (md5sum: 4669fd1554ca51307c2f744e51a108a8)
 +
* [http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.4.tar.gz nfsometer-1.4.tar.gz] (md5sum: ef6e3031b2c39c7eb439dd35197e6b76)
 +
* [http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.3.tar.gz nfsometer-1.3.tar.gz] (md5sum: f6ee81e838753fe1d07e49fff653f5cf)
 +
* [http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.2.tar.gz nfsometer-1.2.tar.gz] (md5sum: 86ca117b3265ce40408b2015e96482b5)
 +
* [http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.1.tar.gz nfsometer-1.1.tar.gz] (md5sum: efbca65ec863b63cb852baac5ff409cf)
 +
* [http://www.linux-nfs.org/~dros/nfsometer/releases/nfsometer-1.0.tgz nfsometer-1.0.tgz] (md5sum: 9d8873517c0365dc04d28b585e0ba31b)
Main git repository:
Main git repository:
 +
 +
'master' branch is current release, 'testing' branch is where all new development is happening.
 +
* git clone git://git.linux-nfs.org/projects/dros/nfsometer.git
* git clone git://git.linux-nfs.org/projects/dros/nfsometer.git
-
* [[http://git.linux-nfs.org/?p=dros/nfsometer.git;a=summary NFSometer gitweb]]
+
* [http://git.linux-nfs.org/?p=dros/nfsometer.git;a=summary NFSometer gitweb]
 +
 
 +
== Example Report ==
 +
 
 +
Here is an example nfsometer report. The reports have notes explaining what was run.
 +
 
 +
[http://linux-nfs.org/~dros/nfsometer/example1/report.html example 1]: run cthon workload with and without a loaded server.
 +
 
 +
== More Examples ==
 +
 
 +
Examples from nfsometer-1.5 manpage / "nfsometer examples":
 +
 
 +
<pre>
 +
 
 +
Example 1: See what workloads are available
 +
 
 +
  $ nfsometer workloads
 +
 
 +
  This command lists available workloads and will tell you why
 +
  workloads are unavailable (if any exist).
 +
 
 +
 
 +
Example 2: Compare cthon, averaged over 3 runs,
 +
          across nfs protocol versions
 +
 
 +
  nfsometer -n 3 server:/export cthon
 +
 
 +
  This example uses the default for -o: "-o v3 -o v4 -o v4.1".
 +
  To see the results, open results/index.html in a web browser.
 +
 
 +
 
 +
Example 3: Compare cthon, averaged over 3 runs,
 +
          between v3 and v4.0 only
 +
 
 +
  nfsometer -n 3 -o v3 -o v4 server:/export cthon
 +
 
 +
  This example specifies v3 and v4 only.
 +
  To see the results, open results/index.html in a web browser.
 +
 
 +
 
 +
Example 4: Compare two kernels running iozone workload, averaged
 +
          over 2 runs, across all nfs protocol versions
 +
 
 +
  nfsometer can compare two (or more) kernel versions, but
 +
  has no way of building, installing or booting new kernels.
 +
  It's up to the user to install new kernels.
 +
  In order for these kernels to be differentiated, 'uname -a'
 +
  must be different.
 +
 
 +
  1) boot into kernel #1
 +
 
 +
  2) nfsometer -n 2 server:/export iozone
 +
 
 +
  3) boot into kernel #2
 +
 
 +
  4) nfsometer -n 2 server:/export iozone
 +
 
 +
  5) open results/index.html in a web browser
 +
 
 +
  To see the results, open results/index.html in a web browser.
 +
 
 +
 
 +
Example 5: Using tags
 +
 
 +
  Tags (the -t option) can be used to mark nfsometer runs as
 +
  occurring with some configuration not captured by mount options
 +
  or detectable tags, such as different sysctl settings (client side),
 +
  different server side options, or different network conditions.
 +
 
 +
  1) set server value foo to 2.3
 +
 
 +
  2) nfsometer -o v4 -o v4.1 -t foo=2.3
 +
 
 +
  3) set server value foo to 10
 +
 
 +
  4) nfsometer -o v4 -o v4.1 -t foo=10
 +
 
 +
  What is passed to -t is entirely up to the user - it will not be
 +
  interpreted or checked by nfsometer at all, so be careful!
 +
 
 +
  To see the results, open results/index.html in a web browser.
 +
 
 +
 
 +
Example 6: Always options
 +
 
 +
  The -o flag specifies distinct option sets to run, but sometimes
 +
  there are options that should be present in each.  Instead of
 +
  writing each one out, you can use the -a option:
 +
 
 +
  nfsometer -o v3 -o v4 -a sec=krb5 server:/export iozone
 +
 
 +
  this is equivalent to:
 +
 
 +
  nfsometer -o v3,sec=krb5 -o v4,sec=krb5 server:/export iozone
 +
 
 +
 
 +
Example 7: Using the "custom" workload
 +
 
 +
  A main use case of nfsometer is the "custom" workload - it allows
 +
  the user to specify the command that nfsometer is to run.
 +
 
 +
  NOTE: the command's cwd (current working directory) is the runroot
 +
        created on the server.
 +
 
 +
  export NFSOMETER_CMD="echo foo > bar"
 +
  export NFSOMETER_NAME="echo"
 +
  export NFSOMETER_DESC="Writes 4 bytes to a file"
 +
  nfsometer server:/export custom
 +
 
 +
  This will run 3 traces (v3, v4, v4.1) against server:/export of
 +
  the command: echo foo > bar.
 +
 
 +
 
 +
Example 8: Using the loadgen mode
 +
 
 +
Loadgen runs several instances of a workload without capturing
 +
traces. The idea is that you use several clients to generate
 +
load, then another client to measure performance of a loaded
 +
server. The "real" run of nfsometer (not loadgen) should mark
 +
the traces using the -t option.
 +
 
 +
1) On client A, run the cthon workload to get a baseline of
 +
    a server without any load.
 +
 
 +
  nfsometer trace server:/export cthon
 +
 
 +
2) When that's done, start loadgen on client B:
 +
 
 +
  nfsometer -n 10 loadgen server:/export dd_100m_1k
 +
 
 +
    This runs 10 instances of dd_100m_1k workload on server:/export.
 +
    It can take several minutes to start in an attempt to stagger
 +
    all the workload instances.
 +
 
 +
3) once all instances are started, run the "real" nfsometer
 +
    trace on client A.  Use the -t option to mark the traces
 +
    as having run under load conditions:
 +
 
 +
  nfsometer -t "10_dd" trace server:/export cthon
 +
 
 +
4) Explain how the tests were set up in the result notes.
 +
    This should be run on client A (which has the traces:
 +
 
 +
  nfsometer notes
 +
 
 +
5) Now generate the reports:
 +
 
 +
  nfsometer report
 +
 
 +
Example 8: Long running nfsometer trace
 +
 
 +
  The nfsometer.py script currently runs in the foreground.  As
 +
  such, it will be killed if the tty gets a hangup or the connection
 +
  to the client is closed.
 +
 
 +
  For the time being, nfsometer should be run in a screen
 +
  session, or run with nohup and the output redirected to a file.
 +
 
 +
  1) screen -RD
 +
  2) nfsometer -n 2 server:/export iozone
 +
  3) close terminal window (or ^A^D)
 +
  ...
 +
  4) reattach later with screen -RD
 +
  5) once nfsometer.py is done, results will be in results/index.html
 +
 
 +
</pre>
 +
 
 +
== Feedback ==
 +
 
 +
Please send any bug reports, questions or comments to dros(AT)monkey.org
 +
 
 +
For contributions, see [http://git.linux-nfs.org/?p=dros/nfsometer.git;a=blob;f=howto-contribute.txt;hb=refs/heads/master howto-contribute.txt].

Latest revision as of 03:49, 29 June 2017

NFSometer is a performance measurement framework for running workloads and reporting results across NFS protocol versions, NFS options and Linux NFS client implementations.

The NFSometer project was started at NetApp as a way to automate performance testing of the Linux NFS client. Since then it has grown to include many other features, notably the generation of reports. It is our hope that by releasing this project under the GPLv2 license, the Linux NFS community will benefit from a better understanding of NFS performance characteristics and contribute improvements to NFSometer.

NFSometer is not designed to replace existing filesystem benchmarks. Instead, it is designed to automate the running of existing filesystem benchmarks, gather NFS specific statistics and generate reports that make benchmark output more understandable.

Goals:

  • Automate the time consuming process of running a variety of workloads looking for performance regressions between kernel versions, NFS protocol versions and NFS options.
  • Generate reports that allows the user to understand the performance characteristics of workloads and easily compare performance characteristics of different configurations.
  • Provide a simple way to define workloads. This allows users to evaluate NFS deployment scenarios with workloads representative of their unique needs.

Contents

News

  • Wed June 28, 2017: NFSometer v1.8 released! This version has only one bug fix that allows testing of v4.2 pnfs servers.
  • Thu March 3, 2016: NFSometer v1.8 released! This version includes a few bugfixes.
  • Wed Jan 16, 2014: NFSometer v1.7 released! This version includes bugfixes including a bad graphing bug, as well as many cleanups. Also, updated the example report.
  • Tue Oct 8, 2013: NFSometer v1.6 released! This version includes bugfixes, code cleanups and speedups and style/layout changes to reports
  • Wed Mar 6, 2013: NFSometer v1.5 released! This version includes several bugfixes, including one that caused a traceback when parsing results from older kernels.
  • Fri Feb 22, 2013: NFSometer v1.4 released! This version has a redesign of reports, many bugfixes and several new features.
  • Fri Dec 7, 2012: NFSometer v1.3 released! This version includes some major cleanup and bugfixes.
  • Fri Nov 30, 2012: NFSometer v1.2 released! This version includes, new options (notably -o replaces -m), basic loadgen functionality, probe mountopts and detect tags, replace poorly supported admin-script and client-script functionality with tags (-t option), bugfixes, etc
  • Thu Jun 21, 2012: NetApp is pleased to announce the GPLv2 release of NFSometer!

Download

The current release of nfsometer is version 1.9:

nfsometer-1.9.tar.gz (md5sum: 10121aa2fe48416bb8a38eb73c502468)

Old releases:

Main git repository:

'master' branch is current release, 'testing' branch is where all new development is happening.

  • git clone git://git.linux-nfs.org/projects/dros/nfsometer.git
  • NFSometer gitweb

Example Report

Here is an example nfsometer report. The reports have notes explaining what was run.

example 1: run cthon workload with and without a loaded server.

More Examples

Examples from nfsometer-1.5 manpage / "nfsometer examples":


Example 1: See what workloads are available

  $ nfsometer workloads

  This command lists available workloads and will tell you why
  workloads are unavailable (if any exist).


Example 2: Compare cthon, averaged over 3 runs,
           across nfs protocol versions

   nfsometer -n 3 server:/export cthon

  This example uses the default for -o: "-o v3 -o v4 -o v4.1".
  To see the results, open results/index.html in a web browser.


Example 3: Compare cthon, averaged over 3 runs,
           between v3 and v4.0 only

  nfsometer -n 3 -o v3 -o v4 server:/export cthon

  This example specifies v3 and v4 only.
  To see the results, open results/index.html in a web browser.


Example 4: Compare two kernels running iozone workload, averaged
           over 2 runs, across all nfs protocol versions

  nfsometer can compare two (or more) kernel versions, but
  has no way of building, installing or booting new kernels.
  It's up to the user to install new kernels.
  In order for these kernels to be differentiated, 'uname -a'
  must be different.

   1) boot into kernel #1

   2) nfsometer -n 2 server:/export iozone

   3) boot into kernel #2

   4) nfsometer -n 2 server:/export iozone

   5) open results/index.html in a web browser

  To see the results, open results/index.html in a web browser.


Example 5: Using tags

  Tags (the -t option) can be used to mark nfsometer runs as
  occurring with some configuration not captured by mount options
  or detectable tags, such as different sysctl settings (client side),
  different server side options, or different network conditions.

  1) set server value foo to 2.3

  2) nfsometer -o v4 -o v4.1 -t foo=2.3

  3) set server value foo to 10

  4) nfsometer -o v4 -o v4.1 -t foo=10

  What is passed to -t is entirely up to the user - it will not be
  interpreted or checked by nfsometer at all, so be careful!

  To see the results, open results/index.html in a web browser.


Example 6: Always options

  The -o flag specifies distinct option sets to run, but sometimes
  there are options that should be present in each.  Instead of
  writing each one out, you can use the -a option:

  nfsometer -o v3 -o v4 -a sec=krb5 server:/export iozone

  this is equivalent to:

  nfsometer -o v3,sec=krb5 -o v4,sec=krb5 server:/export iozone


Example 7: Using the "custom" workload

  A main use case of nfsometer is the "custom" workload - it allows
  the user to specify the command that nfsometer is to run.

  NOTE: the command's cwd (current working directory) is the runroot
        created on the server.

  export NFSOMETER_CMD="echo foo > bar"
  export NFSOMETER_NAME="echo"
  export NFSOMETER_DESC="Writes 4 bytes to a file"
  nfsometer server:/export custom

  This will run 3 traces (v3, v4, v4.1) against server:/export of
  the command: echo foo > bar.


Example 8: Using the loadgen mode

 Loadgen runs several instances of a workload without capturing
 traces. The idea is that you use several clients to generate
 load, then another client to measure performance of a loaded
 server. The "real" run of nfsometer (not loadgen) should mark
 the traces using the -t option.

 1) On client A, run the cthon workload to get a baseline of
    a server without any load.

   nfsometer trace server:/export cthon

 2) When that's done, start loadgen on client B:

   nfsometer -n 10 loadgen server:/export dd_100m_1k

    This runs 10 instances of dd_100m_1k workload on server:/export.
    It can take several minutes to start in an attempt to stagger
    all the workload instances.

 3) once all instances are started, run the "real" nfsometer
    trace on client A.  Use the -t option to mark the traces
    as having run under load conditions:

   nfsometer -t "10_dd" trace server:/export cthon

 4) Explain how the tests were set up in the result notes.
    This should be run on client A (which has the traces:

   nfsometer notes

 5) Now generate the reports:

   nfsometer report

Example 8: Long running nfsometer trace

  The nfsometer.py script currently runs in the foreground.  As
  such, it will be killed if the tty gets a hangup or the connection
  to the client is closed.

  For the time being, nfsometer should be run in a screen
  session, or run with nohup and the output redirected to a file.

   1) screen -RD
   2) nfsometer -n 2 server:/export iozone
   3) close terminal window (or ^A^D)
   ...
   4) reattach later with screen -RD
   5) once nfsometer.py is done, results will be in results/index.html

Feedback

Please send any bug reports, questions or comments to dros(AT)monkey.org

For contributions, see howto-contribute.txt.

Personal tools