NFSTEST_RDMA
Section: nfstest_rdma 1.0 (1)
Updated: 19 July 2021
Index
Return to Main Contents
NAME
nfstest_rdma - NFS-over-RDMA functional tests
SYNOPSIS
nfstest_rdma --server <server> [options]
DESCRIPTION
Verify correct functionality of NFS-over-RDMA
Remote Direct Memory Access (RDMA) provides fast data transfers between
servers and storage. NFS over RDMA is best used when a large amount of data
needs to be transferred with higher performance than regular NFS. NFS over
RDMA is usually used over InfiniBand which provides higher performance and
a lower latency.
Although NFS over RDMA is mostly used over InfiniBand, Ethernet could be
used as the link protocol as well. RDMA over Converged Ethernet (RoCE) which
allows RDMA over Ethernet by encapsulating the InfiniBand transport packet
over Ethernet. RoCE provides a couple of variants: RoCEv1 and RoCEv2.
One is RoCEv1 which is an Ethernet link layer protocol and provides RDMA
functionality between two hosts in the same Ethernet broadcast domain.
While the second is RoCEv2 or RRoCE (Remote RoCE) which is an internet
layer protocol so these packets can be routed. RoCEv2 runs over UDP/IPv4
or UDP/IPv6. There is also another variant called iWARP which runs over
the TCP protocol. Testing is currently supported for all of these variants
except for iWARP.
NFS over RDMA has a couple of extra layers in the packet: InfiniBand layer
and RPC-over-RDMA or RPCoRDMA layer. The InfiniBand layer contains the
OpCode which specifies the type of RDMA operation to perform and the PSN
which is the packet sequence number. The RPCoRDMA layer contains the XID
and the RDMA chunk lists. The RDMA read chunk list is used to transfer
DDP (Direct Data Placement) data from the NFS client to the server, e.g.,
NFS write call. On the other hand, the RDMA write chunk list is used to
transfer DDP data from the NFS server back to the client, e.g., NFS read
reply. Only certain NFS operations can be transferred using DDP and only
the opaque part of the operation is transferred using either RDMA reads
or writes while the rest of the NFS packet is transferred via the receive
buffer using the RDMA SEND operation. Finally, the RDMA reply chunk is
used to transfer replies having a variable length reply which could be
larger than the receive buffer and could not be transferred using the
write chunk list because it does not contain a single large opaque item.
Tests are divided into three groups: basic, read and write. The basic tests
deal mostly with verifying NFS packets using the reply chunk and some other
basic RDMA functionality. The read tests deal with verifying NFS read which
in turn verify the RDMA write functionality. Finally, the write tests deal
with verifying NFS write which in turn verify the RDMA read functionality.
Also, if the NFS read or write is small enough the client could not use the
RDMA write or read functionality, but instead could use the receive buffer
and transfer the data using the RDMA SEND operations.
Tests verify the RPCoRDMA layer is sent when necessary and that the RDMA
chunk lists are sent with the correct information which includes the number
of chunks, number of segments in each chunk and the correct information for
each segment. Tests verify each segment information is correct and their
corresponding RDMA read or write information which includes correct handle,
virtual offset, DMA length and the XDR position for the case of RDMA reads.
In addition, the correct number of RDMA I/O fragments is also verified and
their corresponding lengths and packet sequence numbers.
OPTIONS
- --version
-
show program's version number and exit
- -h, --help
-
show this help message and exit
- -f FILE, --file=FILE
-
File where options are specified besides the system wide file
/etc/nfstest, user wide file $HOME/.nfstest or in the current
directory .nfstest file
NFS specific options:
- -s SERVER, --server=SERVER
-
Server name or IP address
- -e EXPORT, --export=EXPORT
-
Exported file system to mount [default: '/']
- --nfsversion=NFSVERSION
-
NFS version, e.g., 3, 4, 4.1, etc. [default: 4.1]
- -m MTPOINT, --mtpoint=MTPOINT
-
Mount point [default: '/mnt/t']
- -p PORT, --port=PORT
-
NFS server port [default: 20049]
- --proto=PROTO
-
NFS protocol name [default: 'rdma']
- --sec=SEC
-
Security flavor [default: 'sys']
- --nconnect=NCONNECT
-
Multiple TCP connections option [default: '1']
- -o MTOPTS, --mtopts=MTOPTS
-
Mount options [default: 'hard']
- --datadir=DATADIR
-
Data directory where files are created, directory is created on the
mount point [default: '']
Logging options:
- -v VERBOSE, --verbose=VERBOSE
-
Verbose level for debug messages [default: 'none']
- --tverbose=TVERBOSE
-
Verbose level for test messages [default: '1']
- --createlog
-
Create log file
- --rexeclog
-
Create rexec log files
- --warnings
-
Display warnings
- --tag=TAG
-
Informational tag, it is displayed as an INFO message [default: '']
- --notty
-
Do not use terminal colors on output
Packet trace options:
- --createtraces
-
Create a packet trace for each test
- --tbsize=TBSIZE
-
Capture buffer size for tcpdump [default: 192k]
- --trcdelay=TRCDELAY
-
Seconds to delay before stopping packet trace [default: 0.0]
- --keeptraces
-
Do not remove any trace files [default: remove trace files if no
errors]
- --rmtraces
-
Remove trace files [default: remove trace files if no errors]
- -i INTERFACE, --interface=INTERFACE
-
Device interface [default: automatically selected]
File options:
- --nfiles=NFILES
-
Number of files to create [default: 0]
- --filesize=FILESIZE
-
File size to use for test files [default: 64k]
- --rsize=RSIZE
-
Read size to use when reading files [default: 4k]
- --wsize=WSIZE
-
Write size to use when writing files [default: 4k]
- --iodelay=IODELAY
-
Seconds to delay I/O operations [default: 0.1]
- --offset-delta=OFFSET_DELTA
-
Read/Write offset delta [default: 4k]
Path options:
- --sudo=SUDO
-
Full path of binary for sudo [default: '/usr/bin/sudo']
- --kill=KILL
-
Full path of binary for kill [default: '/usr/bin/kill']
- --nfsstat=NFSSTAT
-
Full path of binary for nfsstat [default: '/usr/sbin/nfsstat']
- --tcpdump=TCPDUMP
-
Full path of binary for tcpdump [default: '/usr/sbin/tcpdump']
- --iptables=IPTABLES
-
Full path of binary for iptables [default: '/usr/sbin/iptables']
- --messages=MESSAGES
-
Full path of log messages file [default: '/var/log/messages']
- --trcevents=TRCEVENTS
-
Full path of tracing events directory [default:
'/sys/kernel/debug/tracing/events']
- --trcpipe=TRCPIPE
-
Full path of trace pipe file [default:
'/sys/kernel/debug/tracing/trace_pipe']
- --tmpdir=TMPDIR
-
Temporary directory [default: '/tmp']
Debug options:
- --nocleanup
-
Do not cleanup created files
- --notimestamps
-
Do not display timestamps in debug messages
- --bugmsgs=BUGMSGS
-
File containing test messages to mark as bugs if they failed
- --nomount
-
Do not mount server and run the tests on local disk space
- --basename=BASENAME
-
Base name for all files and logs [default: automatically generated]
- --nfsdebug=NFSDEBUG
-
Set NFS kernel debug flags and save log messages [default: '']
- --rpcdebug=RPCDEBUG
-
Set RPC kernel debug flags and save log messages [default: '']
- --tracepoints=TRACEPOINTS
-
List of trace points modules to enable [default: '']
- --nfsstats
-
Get NFS stats [default: 'False']
- --pktdisp
-
Display main packets related to the given test
- --nfserrors
-
Fail every NFS error found in the packet trace
- --client-ipaddr=CLIENT_IPADDR
-
IP address of localhost
Reporting options:
- --xunit-report
-
Generate xUnit compatible test report
- --xunit-report-file=XUNIT_REPORT_FILE
-
Path to xout report file
Test options:
- --runtest=RUNTEST
-
Comma separated list of tests to run, if list starts with a '^' then
all tests are run except the ones listed [default: 'all']
- --small-filesize=SMALL_FILESIZE
-
File size to use for small files [default: 4k]
- --large-filesize=LARGE_FILESIZE
-
File size to use for large files [default: 1m]
TESTS
basic
Run all NFS-over-RDMA basic functionality tests: basic01, basic02,
basic03, basic04, basic05
basic01
Verify basic NFS-over-RDMA functionality
basic02
Verify NFS-over-RDMA reply chunk on EXCHANGE_ID/SETCLIENTID
basic03
Verify NFS-over-RDMA reply chunk on READDIR
basic04
Verify NFS-over-RDMA reply chunk on READLINK
basic05
Verify NFS-over-RDMA reply chunk on GETATTR(FATTR4_ACL)
read
Run all NFS-over-RDMA functionality tests where file is opened for
reading: read01, read02, read03, read04
read01
Verify NFS-over-RDMA functionality on a file opened for reading (very small file)
read02
Verify NFS-over-RDMA functionality on a file opened for reading (small file)
read03
Verify NFS-over-RDMA functionality on a file opened for reading (medium file)
read04
Verify NFS-over-RDMA functionality on a file opened for reading (large file)
write
Run all NFS-over-RDMA functionality tests where file is opened for
writing: write01, write02, write03, write04
write01
Verify NFS-over-RDMA functionality on a file opened for writing (very small file)
write02
Verify NFS-over-RDMA functionality on a file opened for writing (small file)
write03
Verify NFS-over-RDMA functionality on a file opened for writing (medium file)
write04
Verify NFS-over-RDMA functionality on a file opened for writing (large file)
all
Run all tests: basic01, basic02, basic03, basic04, basic05, read01,
read02, read03, read04, write01, write02, write03, write04
EXAMPLES
The only required option is --server
$ nfstest_rdma --server 192.168.0.11
NOTES
The user id in the local host must have access to run commands as root
using the 'sudo' command without the need for a password.
SEE ALSO
formatstr(3),
nfstest.test_util(3),
nfstest_alloc(1),
nfstest_cache(1),
nfstest_delegation(1),
nfstest_dio(1),
nfstest_fcmp(1),
nfstest_file(1),
nfstest_interop(1),
nfstest_io(1),
nfstest_lock(1),
nfstest_pkt(1),
nfstest_pnfs(1),
nfstest_posix(1),
nfstest_sparse(1),
nfstest_ssc(1),
nfstest_xid(1),
packet.application.rpc_const(3),
packet.application.rpcordma_const(3),
packet.nfs.nfs3_const(3),
packet.nfs.nfs4_const(3),
packet.transport.ib(3)
BUGS
No known bugs.
AUTHOR
Jorge Mora (mora@netapp.com)
Index
- NAME
-
- SYNOPSIS
-
- DESCRIPTION
-
- OPTIONS
-
- NFS specific options:
-
- Logging options:
-
- Packet trace options:
-
- File options:
-
- Path options:
-
- Debug options:
-
- Reporting options:
-
- Test options:
-
- TESTS
-
- basic
-
- basic01
-
- basic02
-
- basic03
-
- basic04
-
- basic05
-
- read
-
- read01
-
- read02
-
- read03
-
- read04
-
- write
-
- write01
-
- write02
-
- write03
-
- write04
-
- all
-
- EXAMPLES
-
- NOTES
-
- SEE ALSO
-
- BUGS
-
- AUTHOR
-
This document was created by
man2html,
using the manual pages.
Time: 16:08:34 GMT, July 19, 2021