NFS and IO testing : Bonnie++

November 23rd, 2010
Been spending time lately testing out and trying to tune NFS mounts.
For testing IO, I’ve been using
  • dd
  • iozone
  • bonnie++
  • orion
This first option, dd, is fine for file creation tests and/or sequential reads but less flexible for random reads or random writes
To test random reads I thought I’d use iozone but don’t see a solely random read test.
Then I thought I’d use the Oracle supplied tool, orion, to do random read test on an NFS mounted file system, but this doesn’t work, at least on AIX 6.1, with my mount settings.
First orion test gave this error
$ orion -run simple -testname orion
ORION: ORacle IO Numbers — Version
rwbase_read_luncfg: SlfFopen error on orion.lun
orion_parse_args: rwbase_read_luncfg failed

OK, have to create “orion.lun” with either my lun locations or my file locations. I put in file locations:

$ cat orion.lun
Now get another error
$ orion -run simple -testname orion
ORION: ORacle IO Numbers — Version
Test will take approximately 9 minutes
Larger caches may take longer
orion_spawn: skgpspawn failed: Error category: 27155, Detail: 2
orion_main: orion_spawn failed
Non test error occurred
Orion exiting
Illegal instruction(coredump)
Looks like the “orion” executable wasn’t being found, at least not by execve
$ truss -f orion -run simple -testname orion
700502: execve(“orion”, 0x0FFFFFFFFFFBF2D0, 0x0FFFFFFFFFFFFB30) Err#2 ENOENT
so I ran it from my bin directory where the orion executable could be found. Now get another error
$ orion -run simple -testname orion
ORION: ORacle IO Numbers — Version
Test will take approximately 9 minutes
Larger caches may take longer
storax_skgfr_openfiles: File identification failed on /kyle/system01.dbf
OER 27054: please look up error in Oracle documentation
Additional information: 6
rwbase_lio_init_luns: lun_openvols failed
rwbase_rwluns: rwbase_lio_init_luns failed
orion_thread_main: rw_luns failed
Non test error occurred
Orion exiting
If the datafile was on “/tmp” it word fine but if it was on my NFS mount it failed with the above error. Hmm – doesn’t work over NFS?
Finally I decided to try bonnie++. I download, unziped, tar extracted bonnie++ , ran “./configure” and then make, but got error
$ make
make: 1254-002 Cannot find a rule to create target bon_io.o from dependencies.
For some reason the rule for the “.o” doesn’t seem to work:
%.o: %.cpp %.h bonnie.h port.h
$(CXX) -c $&lt
Not feeling like trying to figure make out I just did it by hand

$ for i in *cpp; do
name=`echo $i | sed -e ‘s/.cpp//’`
echo “g++ -c $name.cpp $name.h bonnie.h port.h”
done > makeo
$ sh makeo
$ make

That worked. Test results coming


  1. Trackbacks

  2. No trackbacks yet.

  2. Alex Gorbachev
    November 23rd, 2010 at 22:17 | #1

    ORION is somewhat picky on NFS. I did manage to run it on NFS but I remember I had to play with strace for a while to get to the bottom of it. I think it was issue with ASYNC IO settings.

    Btw, it still didn't work on NFS with ASYNC IO on RHEL 4. Even though IOs were done via async syscall, based on strace I concluded that there was serialization somewhere and ORION couldn't schedule multiple concurrent IOs.

  3. Kyle Hailey
    November 23rd, 2010 at 23:07 | #2

    Did get ORION working. Looks like Orion, as well as Oracle, require an entry in /etc/filesystems. For example I added

    dev = /kyle
    vfs = nfs
    mount = true
    options = rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600
    account = false

    and now Orion works.
    Will do some testing …

  4. EricGrancher
    November 24th, 2010 at 12:43 | #3

    good morning Kyle and all readers,
    I have been using the fio ( tool with pleasure. It enables to have direct IO, async IO, traces of IO time… quite flexible.
    I have to say that I have used it exclusively on Linux (but the README file mentions AIX so maybe it works as well, I do not have an AIX box to test).

  5. Kyle Hailey
    November 26th, 2010 at 16:26 | #4

    Eric, thanks for the pointer. Look forward to trying fio out.

You must be logged in to post a comment.