CPSC 330 - Fall 2008 Project 1 - Benchmark Analysis Email notification of team and benchmark choice due by September 5 Paper due at class time on September 15 You should work individually or in teams of at most two. You should prepare a five page (plus or minus) analysis report for a non-toy, non-Dhrystone computer benchmark. The paper should report on a single program rather than a complete benchmark suite, but the program you choose can be a member of one or more benchmark suites. Model your paper after the Alan Weiss paper, "Dhrystone Benchmark: History, Analysis, 'Scores', and Recommendations," available on-line as http://www.ebenchmarks.com/download/ECLDhrystoneWhitePaper.pdf Following the example of Weiss, your paper should have the following sections: * history of the benchmark * technical characteristics of the benchmark (identify strengths and/or weaknesses of each characteristic) * benchmark scores - examples of recent use of the benchmark * brief summary of any run rules for the benchmark * how to win at the benchmark the easy way - any known cheats, shortcuts, and/or targeted compiler optimizations) * conclusions - is this benchmark reliable and representative? * references - bibliographic citations, URLs, etc. Please use section titles to delimit the sections (the titles do not have to be match verbatim the section descriptions above). If you cut and paste a table or a figure from the web or other source, be sure to cite the source both in the caption and in the references section. If the information is available, give a workload characterization of the benchmark in an extra appendix. E.g., see the header comment to Dhrystone, or Tables VIII-XIII in Weicker's 1984 CACM paper, which includes workload information such as the distribution of operators: * 2. Operators * ------------ * number approximate * percentage * * Arithmetic 32 50.8 * * + 21 33.3 * - 7 11.1 * * 3 4.8 * / (int div) 1 1.6 * * Comparison 27 42.8 * * == 9 14.3 * /= 4 6.3 * > 1 1.6 * < 3 4.8 * >= 1 1.6 * <= 9 14.3 * * Logic 4 6.3 * * && (AND-THEN) 1 1.6 * | (OR) 1 1.6 * ! (NOT) 2 3.2 * * -- ----- * 63 100.1 For some benchmarks on some platforms, the instruction mix may be available, e.g., SPEC CPU95 using gcc 2.8.1, -O2 flag, on SPARC: load 18% cond. branch 15% compare 13% logic 12% add/sub 11% store 7% shift 7% load imm. 2% save/restore 2% Other benchmarks might be better characterized with workload data such as bytes/message, number of instructions between sends, TCP accepts per session, SSL handshakes per session, file size, vertices (or triangles) per frame, etc. (Broad-brush characterizations about, say, the prevalence of pointer dereferences or floating-point operations would likely be better placed in the technical characteristics section than in an appendix.)