DAP2 - Portable Data Abstraction Penalty Benchmark, version 2

Project's site at Sourceforge.net is http://sourceforge.net/projects/dap2/

Introduction

Every programmer knows that performance usually degrades when higher levels of abstraction are used in programming. One can say that compilers/execution environments impose a data abstraction penalty. Ok, it doesn't stop programmers from using abstractions. However, it would be helpful to know how much we "pay" for a convinience.

Well, it depends on many factors, most obviously on the programming language. Alex Stepanov developed the Stepanov Kernels benchmark for C++. He introduced the concept of data abstraction penalty measurement. The idea was to perform the same calculation in several ways, starting with the lowest possible level of abstraction in the code to higher levels. Next step is to calculate ratios of execution times of those test to the very first test. Then the penalty was defined as a geometrical mean of the ratios. Ideally, it should be exactly one.

Inspired by Stepnaov Kernels benchmark, in 2001 Argyn Kuketayev wrote DAP Benchmark for Java. It should be noted that it was impossible to port the original benchmark to Java. Although the definition of the abstraction penalty remained the same, DAP for Java has a different set of tests.

Why DAP2?

Original DAP was designed for Java only, as well as Stepanov Benchmark is for C++ only. DAP2 is an attempt to define language neutral asbtraction penalty benchmark. So it would be possible to implement it in Java and C#, for example.

Status

The original DAP benchmark has been updated to version 1.1, it's in the "old" directory. We performed the test runs on Red Hat Linux and Windows 2000 for x86 CPU using JDKs from Sun, IBM, and BEA. Raw test results are available, as well as the draft paper written in Russian language.

The DAP2 sources are in Beta state. The DAP2 benchmark's test set is defined. We have sample implementations for Java and C#. Also, the we have sample test results for several Java and C# SDKd and VMs.

References

Definition of DAP2 test set

http://www.adtmag.com/joop/article.asp?id=4597&mon=8&yr=2001 - The Data Abstraction Penalty (DAP) Benchmark for Small Objects in Java by Argyn Kuketayev in JOOP, August 2001.

http://www.acl.lanl.gov/Pooma96/abstracts/robison.html- The Abstraction Penalty for Small Objects in C++ by Arch D. Robison.

Draft version of DAP article in Russian by Argyn Kuketayev