Mac OS X specific notes
*** Instructions before Starting libMicro ***
# Disable Open directory and LDAP using Directory Utility app
# Turn off airport
# Turn off spotlight. In terminal, execute the following:
launchctl unload /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
# Turn off Time Machine in System Preferences
# Wait at least 2 minutes after boot to desktop for boot cache to settle down
*** Make and run quickstart ***
make
./bench >output.txt
runs the libMicro test suite excluding the lmbench tests and gives you a text file named output.txt with the results of one run.
./multiview output1.txt output2.txt >compare.html
gives you a html file comparing two runs.
*** To run libMicro testsuite with stepper disabled ***
*** For Desktop use coreos_bench script***
To get a more consistent result of libMicro benchmark run, we need to disable the
stepper to prevent it from causing wide variations in results. See rdar://6243819
for details.
So to run libMicro test suite with stepper disabled, use 'coreos_bench' script
instead of 'bench' script.
For example:
./coreos_bench > output.txt
runs the libMicro test suite excluding the lmbench tests and gives you a text file named output.txt with the results of one run, with stepper disabled.
Note:
1) We need '/usr/local/bin/pstates' to disable the stepper. Install AppleInternal package
which provides '/usr/local/bin/pstates'.
2) 'coreos_bench' script is used exactly like the 'bench' script. All the usage examples for
'bench' script in this readme file also holds true for 'coreos_bench' script.
*** Makefile ***
The Makefile invokes Makefile.Darwin which invokes Makefile.com.Darwin.
Just invoke make, with options if necessary, and everything should
build correctly. The binaries are placed in a directory called
bin-ARCH where ARCH is the default or specified when building via
the ARCH flag.
Note:
1) The binaries of apple added tests are placed in a directory called
apple/bin-ARCH
2) All the binaries under bin-ARCH and apple/bin-ARCH are code signed
during build.
options for invoking Makefile are:
ARCH defaults to i386
to build fat/multi architecture, specify
make ARCH=fat
the makefile will automatically build with ARCH_FLAG="-arch i386 -arch x86_64" and put the results in bin-fat
to build for ARM architecture,
first set an environment variable 'SDKROOT' to point to iPhone internal sdk
For example:
$export SDKROOT="/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.Internal.sdk/"
to build use:
make ARCH=ARM_ARCH where ARM_ARCH can be for e.g. armv6, armv7
this will put the results in bin-ARM_ARCH
to build with only two of the architectures see below
ARCH_FLAG defaults to -arch $(ARCH)
to build fat/multi architecture, specify
make ARCH_FLAG="-arch i386" ARCH=fat
this will put the results in bin-fat
OPT_FLAG defaults to -g
to build optimized, specify make OPT_FLAG=-Os
SEMOP_FLAG defaults to -DUSE_SEMOP
to eliminate SEMOP usage, specify make SEMOP_FLAG=
this is needed on some lower-end systems (e.g. M63)
These can be combined, e.g.
make ARCH=i386 SEMOP_FLAG=
*** Before running benchmarks ***
The shell script create_stuff should be run before any benchmarking
this script takes care of raising the process limits which would
otherwise cause several of the tests to fail - if not you will see:
Running: pipe_pst1
fork: Resource temporarily unavailable
in your stderr during the runs. After you run create_stuff, the
system then needs to be rebooted.
*** running the benchmarks ***
The shell script "bench" will run all the benchmarks, or you can
pass it a parameter to run a single benchmark, e.g.
*** To run libMicro on the embedded platform, use "embd_bench" script. 'embd_bench' script is used exactly like the 'bench' script. All the usage examples for
'bench' script in this readme file also holds true for 'embd_bench' script. ***
bench lmbench_bw_unix
By default the script will run only the libMicro testsuite excluding the lmbench tests.
To run the libmicro testsuite with the lmbench tests included, just pass the -l parameter. e.g,
bench -l
To run only the lmbench testsuite
bench lmbench
To display the usage, just do
bench -h
Watch for:
# WARNINGS
# Quantization error likely;increase batch size (-B option) 4X to avoid.
in the output
To see an example run the supplied testbench script
Add or adjust the -B parameter for any benchmark that fails. The
Quantization error will refer to the benchmark preceding the error,
not the one following...
A typical run:
$ make clean
$ make
$ ./create_stuff
$ ./bench > output1
Running: getpid
for 0.13353 seconds
Running: getppid
for 3.65609 seconds
Running: getenv
for 0.20924 seconds
Running: getenvT2
for 0.37437 seconds
Running: gettimeofday
for 0.58077 seconds
etc...
Use the supplied multiview script to compare runs like:
multiview output1 output2 > compare.html
open compare.html (safari launches)
will show output2 results as a percentage change from the output1 results
*** Adding additional benchmark tests ***
Look at the sample file trivial.c. This demonstrates how to do
argument passing, the flow of control of a benchmark, etc. for the
trivial case. The tests starting with "lmbench_" were ported from
the lmbench suite, so they might be good examples as well.
*** A note regarding future changes in bench.sh script ***
coreos_bench.sh script is almost identical to bench.sh script, except that it
has additional code to disable the stepper during libmicro benchmark run.
In future, if bench.sh script is modified, make sure the changes reflect
in coreos_bench.sh script also.
*** Things to do ***
* port the rest of the lmbench benchmarks into this framework
* create website that will allow easy ability to compare many builds
across many machines with historical repository of runs
* document better how to write a benchmark for this framework
(started in trivial.c)
* check this into xnu/test
* create new benchmarks
*** Leopard notes ***
Due to rdar://4654956 and its original, rdar://2588252 you cannot
run these tests on Leopard without removing the cascade_lockf test.
There may be other tests which panic a Leopard system.
*** benchDS notes ***
From rdar://problem/7468995 add the ability to benchmark the key APIs
for server daemons. In particular, a test binary is added for each of:
ODQueryCreateWithNode() (standard User, Groups, and Hosts records)
getaddrinfo() (hosts and ports)
mbr_check_service_membership()
mbr_check_membership()
getpwnam()
getpwuid()
getgrgid()
getpwent()
getgrent()
getgrnam()
The script benchDS is provided to run a standard set of tests presuming
that the tests are run by root on a system configured with an OD binding.
The OD server (local or remote) must have a set of accounts created with
od_acount_create shell script. This script must also be run as root,
and passed a single argument of the number of users to create. It creates
od_test_{1..N}, and all belong to a ds_test_group1(gid 1211). In addition,
ds_test_group2(gid 1212) is created which has no users as members. User ids are
set sequentially from 5000. In order to administer the OD server, it assumes
user 'diradmin' and password 'admin' are the OD admin.
Also, these tests consult the APIs listed, which can be run against the local
account info, or even Active Directory.
Thus, the quick recipe is:
Install X Server
Enable OD, and create directory admin user 'diradmin' with password 'admin'
As root run: od_account_create 1000
Now run the test, as root: ./benchDS 1000 > output-file
In addition, od_account_delete 1000 will delete the 1000 users created with od_account_create.