FERRET benchmarks README file

Running the benchmarks and interpreting results.

After building Ferret, make a few simple tests for command-line reading,
script reading, file reading, for example

   > ferretdods_gui
   yes? list/L=1:5 L
   yes? go ptest

   yes? use coads_climatology
   yes? show data

When Ferret is running, the benchmark tests will verify its performance. To run the benchmarks, 
in the /bench directory we will run the script run_all. Look at the start of run_all, and if you 
like, customize the section for machine type. This information is used to make the log file, and
also to look for a machine-specific stream binary file.

Run run_all and answer the questions. When you first run the shell script run_all, you will be 
coached to create a stream binary file for the machine by compiling and running the program 
make_stream_file.F. Do this, and then run run_all again.  The benchmarks may be run with or without 
the shared-object external functions. 
If the benchmark scripts run correctly, the benchmark job will finish with 
Display no longer set, about to run batch gif test
Display still not set, about to run batch ps test
Ended at Wed Nov 1 09:35:09 PST 2006
Checking PLT, GIF and PS files.......
    PLT files look good.....
    GIF files look good.....
    PS files look good......

The output is contained in several files:

  all_01nov06at0959.sol_log
  all_01nov06at0959.sol_err
  all_01nov06at0959.sol_plt

where the name contains the date and time of the benchmark run, and the extension refers to the 
machine type or operating system. In addition a number of plot output files are created and 
compared to reference output by the benchmark script.

In the benchmark directory are "official" output files from a number of operating systems.  To 
compare your output logs, choose one to compare with your output. There are lines in the benchmark 
output which may differ from one run of the benchmarks to another and which do not indicate problems 
with the benchmark run. We may remove them by running the script "clean_ultra" and piping the output 
to a new file, for the official benchmark log file and the one just created

    > clean_ultra ansley_official.x86_64-linux_log > cleaned_ansley_official.x86_64-linux_log
    > clean_ultra all_01nov06at0959.sol_log > cleaned_all_01nov06at0959.sol_log 

    > xdiff cleaned_ansley_official.x86_64-linux_log cleaned_all_01nov06at0959.sol_log 

Some differences will still always exist: 

1) The date of the Ferret run and the operating system are included in various outputs such 
as the Ferret symbol SESSION_DATE, values of labels which are written to the logs, or file 
attributes which are listed. These differences may be ignored.

2) Values of PPL$XPIXEL and PPL$YPIXEL will differ; these are computed based on the display of the terminal where the benchmark job is run

3) If you are comparing a log from a different operating system, there may be differences in the 
values of data in output. This might show up as missing_value=-9.9999998e+33f vs missing_value=-1.e-34f, or 
listings may differ in the least-significant positions. Most all Ferret results are single-prescision, so 
diffreences of that size are OK. Try to compare with output from a similar machine (32- or 64-bit for
instance).

4) At the end of the log files, there is a collection of outputs from "spawn ncdump file.nc" commands. 
Differences in the form of ncdump output, such as differently-placed commas, may exist especially if you
are comparing logs from different operating systems.

5) Some benchmark scripts involve the output of a spawn command.  The speed with which this output is
written to the log file may vary from run to run or system to system.  Occasional garbled output is the 
result of this effect. 


