NFS and IO testing : Bonnie++
November 23rd, 2010
Been spending time lately testing out and trying to tune NFS mounts.
For testing IO, I’ve been using
- dd
- iozone
- bonnie++
- orion
This first option, dd, is fine for file creation tests and/or sequential reads but less flexible for random reads or random writes
To test random reads I thought I’d use iozone but don’t see a solely random read test.
Then I thought I’d use the Oracle supplied tool, orion, to do random read test on an NFS mounted file system, but this doesn’t work, at least on AIX 6.1, with my mount settings.
First orion test gave this error
$ orion -run simple -testname orionORION: ORacle IO Numbers — Version 11.1.0.7.0orion_20101123_1503rwbase_read_luncfg: SlfFopen error on orion.lunorion_parse_args: rwbase_read_luncfg failed
OK, have to create “orion.lun” with either my lun locations or my file locations. I put in file locations:
$ cat orion.lun/tmp/system01.dbf
Now get another error
$ orion -run simple -testname orionORION: ORacle IO Numbers — Version 11.1.0.7.0orion_20101123_1508Test will take approximately 9 minutesLarger caches may take longerorion_spawn: skgpspawn failed: Error category: 27155, Detail: 2orion_main: orion_spawn failedNon test error occurredOrion exitingIllegal instruction(coredump)
Looks like the “orion” executable wasn’t being found, at least not by execve
$ truss -f orion -run simple -testname orion…700502: execve(“orion”, 0x0FFFFFFFFFFBF2D0, 0x0FFFFFFFFFFFFB30) Err#2 ENOENT
so I ran it from my bin directory where the orion executable could be found. Now get another error
$ orion -run simple -testname orionORION: ORacle IO Numbers — Version 11.1.0.7.0orion_20101123_1510Test will take approximately 9 minutesLarger caches may take longerstorax_skgfr_openfiles: File identification failed on /kyle/system01.dbfOER 27054: please look up error in Oracle documentationAdditional information: 6rwbase_lio_init_luns: lun_openvols failedrwbase_rwluns: rwbase_lio_init_luns failedorion_thread_main: rw_luns failedNon test error occurredOrion exiting
If the datafile was on “/tmp” it word fine but if it was on my NFS mount it failed with the above error. Hmm – doesn’t work over NFS?
Finally I decided to try bonnie++. I download, unziped, tar extracted bonnie++ , ran “./configure” and then make, but got error
$ makemake: 1254-002 Cannot find a rule to create target bon_io.o from dependencies.Stop.
For some reason the rule for the “.o” doesn’t seem to work:
%.o: %.cpp %.h bonnie.h port.h$(CXX) -c $<
Not feeling like trying to figure make out I just did it by hand
$ for i in *cpp; doname=`echo $i | sed -e ‘s/.cpp//’`echo “g++ -c $name.cpp $name.h bonnie.h port.h”done > makeo$ sh makeo$ make
That worked. Test results coming
Oracle L
Trackbacks
Comments
ORION is somewhat picky on NFS. I did manage to run it on NFS but I remember I had to play with strace for a while to get to the bottom of it. I think it was issue with ASYNC IO settings.
Btw, it still didn't work on NFS with ASYNC IO on RHEL 4. Even though IOs were done via async syscall, based on strace I concluded that there was serialization somewhere and ORION couldn't schedule multiple concurrent IOs.
Did get ORION working. Looks like Orion, as well as Oracle, require an entry in /etc/filesystems. For example I added
/kyle:
dev = /kyle
vfs = nfs
mount = true
options = rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600
account = false
and now Orion works.
Will do some testing …
good morning Kyle and all readers,
I have been using the fio (http://freshmeat.net/projects/fio/) tool with pleasure. It enables to have direct IO, async IO, traces of IO time… quite flexible.
I have to say that I have used it exclusively on Linux (but the README file mentions AIX so maybe it works as well, I do not have an AIX box to test).
cheers,
Eric
Eric, thanks for the pointer. Look forward to trying fio out.
Best
Kyle