Benchmarking of Java Cryptoalgorithms

Contents

1 Introduction
1.1 Objectives of this paper
1.2 Related work

2 SETUP of Cryptobenchmarking
2.1 Security Levels
2.2 Selecting the candidates for benchmarking
2.3 The test environment
2.3.1 Hardware platform
2.3.2 Operating Systems
2.3.3 Software Environment
2.4 Performance measuring in modern systems
2.5 The bench code for JAVA

3 RESULTS of Cryptobenchmarking
3.1 Results JAVA
3.1.1 Key Setup for symmetric cryptography
3.1.2 Key Generation for asymmetric cryptography
3.1.3 Symmetric Encryption and Decryption
3.1.4 Asymmetric Encryption and Decryption
3.1.5 Hash Generation and MAC Generation with HMac
3.1.6 Signing and verifying with asymmetric cryptography
3.2 Comparison JAVA to C++

4 Conclusion

5 Appendix

List of Figures

1 Key Generation for symmetric keys

2 Asymmetric Key Generation

3 Symmetric Encryption and Decryption using Windows Vista

4 Symmetric Encryption and Decryption using Ubuntu

5 Asymmetric Encryption using Windows Vista

6 Asymmetric Decryption using Windows Vista

7 Benchmarking of hash function using Windows Vista

8 Benchmarking of hash functions using Linux

9 Signing and verifying using Windows Vista: Security Level 4

10 Comparison Symmetric Operations C++ and JAVA

11 Comparison Asymmetric Operations C++ and JAVA

12 Comparison Hashing C++ and JAVA

13 Symmetric Key Setup (1-3) at operating system Windows Vista

14 Symmetric Key Setup(4) at operating system Windows Vista

15 Asymmetric Key Generation at operating system Windows Vista

16 Symmetric Encryption and Decryption(1+2) at operating system Windows Vista

17 Symmetric Encryption and Decryption(3) at operating system Windows Vista

18 Symmetric Encryption and Decryption(4) at operating system Windows Vista

19 Asymmetric Encryption and Decryption at operating system Windows Vista

20 Signing and Verifying at operating system Windows Vista

21 Hashing at operating system Windows Vista

22 HMAC Operation at operating system Windows Vista

23 Symmetric Key Generation using Ubuntu: Security Level 1 and 2

24 Symmetric Key Generation using Ubuntu: Security Level 3

25 Symmetric Key Generation using Ubuntu: Security Level 4

26 Asymmetric Key Generation using Ubuntu

27 Symmetric Encryption / Decryption using Ubuntu: Security Level 1 and 2

28 Symmetric Encryption / Decryption using Ubuntu: Security Level 3

29 Symmetric Encryption / Decryption using Ubuntu: Security Level 4

30 Asymmetric Encryption / Decryption using Ubuntu

31 Signing and Verifying using Ubuntu

32 Hashing using Ubuntu

33 HMACs using Ubuntu

List of Tables

1 security levels with keylength and recommending institute

2 overview of the tested symmetric block ciphers

3 overview of the tested symmetric stream ciphers

4 overview of the tested asymmetric ciphers

5 overview of the tested hashfunctions

6 hardware specifications

7 iterations for cryptographic operations

8 Recommandatons for Keylengths in regard to security

9 Recommandatons for Keylengths in regard to security(2)

1 Introduction

1.1 Objectives of this paper

Cryptoalgorithms are intensivly used in different fields of computing. Examples are: the cre­ation of checksums of documents on the internet using trap door hash functions, the encryption of hard disks in public administration to secure data with symmetric encryption, or the use of digital signatures and asymmetric cryptography in business communication. Everybody who uses cryptoalgorithms in software development may ask themselves which algorithm or library is best for their project. In order to provide some help on this decision we measured a certain selection of cryptoalgorithms of nearly all available java crypto libraries with regards to perfor­mance. Moreover we chose to use the latest Windows operation system Vista and a up-to-date Linux distribution to find out if the choice of the operation system has influence on the perfor­mance of the algorithms. Although the performance measuring is our main approach, we want to present in our spadework how long algorithms with respect to their key lengths should be used in order to guarantee security. Finally, we compare the benchmarking results of the JAVA libraries to the benchmarking results of RSA BSsafe, a C++ library, to provide a relation between java byte code and native machine code of C++. This comparison is of special interest, because RSA provided us with the same library for JAVA (namly JSafe) and C++ (namely BSafe) and therefore one can directly compare the isolated influence of the programming language on cryptographic performance.

1.2 Related work

A measurement of cryptoalgorithms has been performed by various groups in the past. NIST(National Institute of Standards and Technology) as a non-regulatory federal agency within the U.S. Department of Commerce wanted to look for a new standard algorithm for symmetric encryption in 1997 (see [13]). Of course a part of their selection process was benchmarking per­formance that included both software and hardware testing. In comparison to our work the num­ber of tested algorithms was limited due to the fact that only symmetric cipher algorithms were benchmarked. ECRYPT (European Network of Excellence for Cryptology) started in November 2004 with a similar approach. The Phase 2 report shows the current progress [8]. The difference was that the objective of ECRYPT was to identify new stream ciphers that might become suitable for widespread adoption. This project is called eSTREAM and is still going on until May 2008 when phase 3 will be finished. The NESSIE project (New European Schemes for Signatures, Integrity and Encryption) is a project within the Information Society Technolgy (IST) Program of the European Commission[6]. The main goal of it was to find a portfolio of strong crypto­graphic primitives. Performance measuring was also a part of this selection process. All of these described works were examined and are basis of our proceedings.

2 SETUP of Cryptobenchmarking

2.1 Security Levels

At the beginning of our work we allocated the different kinds of crypto algorithms to security levels we defined.

The basis of our security levels are the appraisements of various groups including ECRYPT and NIST which we already mentioned. Additonal BSI (German Federal Office for Information Se­curity) and DCSSI (Central Information Systems Security Division) published recommandations of keylengths to maintain certain periods of secureness. The only scientific work available was done by Lenstra [11]. All of these groups use the key length as an indicator of how long crypto algorithms might be secure in the future including symmetric block ciphers, symmetric stream ciphers, asymmetric cryptography and hash functions. After the evaluation of these papers we defined four security levels similar to the annual report of ECRYPT[4]. The difference of our security levels to those of this European institute is that we only defined 4 levels in comparison to 7 levels of ECRYPT. The reason for that is the weakness of the first two levels, that describe very insecure security levels, which we thought are not applicable nowadays anymore. Additionally, we also ignored the highest security level of ECRYPT because we are of the opinion that secu­rity in cryptography cannot be foreseen for more that 30 years (which is already very optimistic). So we defined security equivalents for 2 to 3 years, 10 years, 20 years and 30 years. Based on that we decided not simply to overtake the key length values of ECRYPT, but to evaluate all the recommendations of BSI[2], NIST[7], DCSSI[3], Lenstra[1][11] and also ECRYPT to create a broad basis for our levels. The detailed recommendations can be seen in the appendix. It became clear quickly that these organizations / authors have more or less similar estimations. Only ECR-PYT differs significantly for the last security level 4. According to Lenstra the recommended key length for asymmetric encryption should be more than 15000 bits, that is approximately four times the estimations of all the other groups. Moreover we realized that Lenstra has very optimistic estimations in comparison to the other groups. This is understandable when we read Lenstra carefully. Namely he only defines a minimum limit for keylength instead of a real secu­rity level. The BSI does not give a prognosis for more than 5 year from now on. Additionally, the draft paper of the BSI only cares about digital signatures, so we couldn’t use this for the defini­tions of our security levels. In order to be prudent we decided to choose the maximum keylength of all these different recommondations for each kind of crypto algorithm except this one case where ECRPYT has very different results. In this case we chose the average of all publications which lead to a key length of 6136 bit for asymmetric cryptography and security level 4.

The defined security levels have two main implications for our work: On the one hand they provide a niveau to compare algorithms which provide different level of security in the same class e.g. MD5 vs. Whirlpool for hashing. On the other hand we wanted to provide an easy way to look up results with regard to the future time frames of security. The complete overview of our defined security levels can be seen in table 1.

illustration not visible in this excerpt

Table 1: security levels with keylength and recommending institute

illustration not visible in this excerpt

Table 2: overview of the tested symmetric block ciphers

2.2 Selecting the candidates for benchmarking

In the following paragraphs we would like to answer the question: ”Which algorithms are to be benchmarked?”. We will present the selected algorithms grouped by kind and explain the criteria of our selection procedure. The basis of the algorithm selection was again the recommandations of the various mentioned groups like NIST or ECRYPT. Additionally to their recommendations, we included algorithms that seemed to be interesting for the comparsion results due to their widespread use for example DES for symmetric cryptography or MD5 for the hashing algo­rithms. Lastly, we also had a look at the candidates of the libaries we wanted to test, to conclude which algorithms are implemented in most of our testing candidates.

The finally choosen algorithms grouped by kind can be seen in tables 2, 3, 4 and 5.

Not shown extra in the tables above but also included in the benchmarking are Message Au­thentication Codes (MAC), which are based on hash functions. Also not shown above is digital

illustration not visible in this excerpt

Table 3: overview of the tested symmetric stream ciphers

illustration not visible in this excerpt

Table 5: overview of the tested hashfunctions

signature cryptography, which is also based on those hash functions together with asymmetric cryptography.

After the selection of the algorithms the next task was to choose the libaries for testing. In this context we included a comprehensive set of the available libraries for the programming language JAVA. We included the following seven so called JAVA crypto provider, or libraries, to our performance testing:

- JCE SUN The security provider of SUN Microsystems is included in JDK. We used ver­sion 1.6 which is available from http://java.sun.com

- Flexiprovider is developed by the Cryptography and Computer Algebra Research Group of Prof. Dr. Johannes Buchmann at the Departement of Computer Sci­ence at Technische Universität Darmstadt; The current version is 1.5p1 available from http://www.flexiprovider.de.

- For Bouncy Castle from the group ”Legion of BouncyCastle” there are two available versions: a full version and a lightweight one. We tested the full version, which is available in version 1.38 from http://www.bouncycastle.org.

- Cryptix from the Cryptix Foundation is tested in snapshot version 20050328. It is avail­able from http://www.cryptix.org.

- IAIK-JCE The Stiftung Secure Information and Communication Technologies(SIC), es­tablished by the Institute for Applied Information Processing and Communication(IAIK) of the Graz University of Technology is responsible for devoloping this library. We used the version 3.16 available from http://jce.iaik.tugraz.at.

· GNU Crypto is part of the GNU project that is maintained by the Free Software Founda­tion (FSF). For our tests we included verion 2.0.1 from http://www.gnu.org.

- RSA JSafe the cryptographic components for JAVA in version 3.6 were provided by RSA the security division of EMC.

illustration not visible in this excerpt

Table 6: hardware specifications

We also included the library BSafe from RSA Security in our performance testing to provide a comparison with a library which is written in C++ and has therefore native machine code performance.

- RSA BSafe the cryptographic components for C in version 6.3.2 were provided by RSA the security division of EMC.

2.3 The test environment

After selecting theoretically which algorithms and libraries are to be tested, we had to care about the setup of the testing environment. These considerations involved the hardware as well as the software environment for testing.

2.3.1 Hardware platform

Thinking about the underlying hardware platform for the benchmarks we had to decide how modern the constituating hardware should be. In general it is better to use a more modern per­sonal computer than an older pc, because the results will then be interesting in both regards, in absolute terms as well as relative figures. We must nonetheless admit that the absolute figures are going to be outdated in two or more years anyway. For the interpretation of the results the speed of the actual testing hardware is not important because the results are to be read and interpreted relativly to each other. One restriction to this assertion can be added: If the situation eventuates that a newer cpu has special optimizations which are supported by some single libraries (not by all), then this would influence the relative library ranking and therfore be an exception to the rule that ”the hardware basis is not important”.

Our test system was composed of a intel core 2 duo processor with 2 Ghz clock speed and 2048 MB of 667 Mhz DDR2 RAM sitting on a Intel Crestline-GM 965 Northbridge with a Front Side Bus Speed of 800 Mhz. More detailed information is presented in table 6.

2.3.2 Operating Systems

As well as the influence of the hardware we made some considerations about the underlying operating systems for benchmarking. Our main concern about the operating system was that a modern OS may influence benchmarking metrics through its inherent time sharing mechanisms, mainly by interrupt requests (IRQ) of the kernel to the central processor. The starting point of our thoughts was to use a real-time operating system for benchmarking, in order to get the most accurate results and to minimize the influence of interrupt requests on benchmarking metrics. However we did not have time and money to acquire commercial real-time plugins for Microsoft Windows an Linux. We finally prefered to set up a ”real life” testing environment. This lead to a standard installation of the two common up-to-date operating systems: Microsoft Windows Vista and Ubuntu Linux 7.10. Moreover we came to the conclusion that results for standard in­stallations of operating systems may be more interesting than a special testing operating system. In addition we switched off every service of the operating system which could trigger a long lasting kernel interrupt. Here is a short list of the deactivated or not installed services of standard operating system installation:

- Anti-Virus services
- Indexing services
- Task planner services

2.3.3 Software Environment

Additionally to the operating system environment, the software development environment also influences benchmarking metrics. For our JAVA testing the software development environment consists of the ”JAVA Development Kit 6 Update 3” with accompanying ”JAVA Virtual Machine” from SUN Microsystems and the current releases of the available crypto libraries (see above). To do the JAVA benchmarking we wrote a single JAVA program which makes use of the JAVA Cryptography Architecture (JCA). This feature of the JAVA programming language provides an interface to access all the different crypto libraries via a standardized java interface in the same manner. By the use of the JCA technology it was then possible to integrate every single library into one benchmarking program that handles the complete benchmarking process. In practice there were some libraries that differed slightly from the JCA standard. The GNU crypto library differs significantly from the SUN JCA standard, but our testing program made it possible to implement some extra methods and branches for those special libraries.

For the single C/C++ library BSafe from RSA we chose to use the Microsoft Visual Studio 2005 IDE as the library itself was tested by RSA with that product.

2.4 Performance measuring in modern systems

In modern hardware, different time sources with different qualities for time measuring and soft­ware development exist. As to Microsoft, there exist five sources of timing/counting in a modern

x86 pc [12]:

- 8254 PIT
- RTC
- APIC Timer
- PM Clock
- High Precision Event Timer
- Time Stamp Counter (TSC): fast, completely unreliable. Frequency changes, CPUs di­verge over time.

All these timers have different qualities like resolution, wether the timer is cpu internal or exter­nal, etc. Our criteria for a good timer for performance benchmarking are that the timer is highly precise to detect the differences between the runtime of various crypto libraries and secondly that the timer is reliable even when the program is for example switching cores in our dual core processor. The advantage of the timer selection procedure was that when coding a benchmarking program you do not have to care in low-level programming about the appropriate timing source. All we had to do is find out which method of the coding language gives us the best timer with regards to our criteria.

For JAVA it is very easy to find a timer which matches our criteria, because of the: 1 System . nanoTime ( ) ;

method, which offers the feasibility to make use of the most precise timer on a very abstract level. This means that the method is independent of the operating system and can thus be used under Windows and Linux for high precision timing because it automatically returns the most precise timer available on the system. It is recommend for benchmarking by the ”Software Design and Quality Institute” of the University of Karlsruhe[15] as a first choice. We also evaluated other possibilties for benchmarking in JAVA for example the hrlib-library which was developed by Vladimir Roubtsov [14]. However none of the evaluated possiblities offers more ease or precision than ”System.nanoTime()”.

For C/C++ you have to differentiate between Windows and Linux operating system because the abstraction layer of the JAVA runtime environment is missing and thus you have to use the most appropriate timing source depending on the API of the operating system. We made test runs with every provided method and chose the best one. For the Microsoft Windows operating system we identified the ”QueryPerformanceCounter” method of the Windows API as the most exact and suitable method for our benchmarking.

2.5 The bench code for JAVA

The provided abstraction layer made it relativly easy to use the different crypto libraries. A so-called ”provider” that is responsible for the matching between the defined standard and the intern functions has to be installed the following way:

Listing 1: register Security Provider

1 Security . addProvider (new packagepath . ProviderConstuctor());

This approach offers a standardized way to access all cryptographic operations of all the different libraries. The relevant part of the benchmarking core of our program written in pseudo JAVA looks like this:

Listing 2: Time Measuring

illustration not visible in this excerpt

For our benchmarking we took the current time in nanoseconds before and after the main crypto operation call including its initialization. In a first version of the benchmarking program we measured, in addition to the average runtime, the minimum and maximum runtime of the cryp­tographic operation. When we analysed the output later we found that the maximum runtime of an algorithm did not make much sense in our results, because the reason for this maximum lies more or less in the interrupt calls of the kernel and is therefore stochastic with regards to the perspective of our benchmarking program. In a later version we took the measurement of the overall time taken and calculated the average of the overall runtime, because in contrast to the minimum runtime the average represents a more real life performance value. To compensate for the mistakes of measuring times with an interrupting kernel, we did the benchmarking runs of the cryptographic primitives a certain numbers of iterations and caluclated the average time at the end. Our minimum runtimes and iterations for the different cryptographic primitives for JAVA can be seen in table 7.

Statistical deviations were not examined in detail. We desisted from verifing the test results with a t-test or similar statistical methods. Several reason were to be said against it. First, tests showed that there are only slight differences between the single iterations of the benchmark runs. Only a few outliers were found during the runs. They were smoothed out by a minimum number of benchmark iterations for each algorithm and the fact that we always compared the average values. On the other hand the additional work for recording and analysing each result of every iteration would have produced a large amount of data an adding only a minimal advantage for verifing results. More detailed verification may be done by future work and thus is not a part of this work. However, we hold that our results reflect the real speed of the algorithms.

illustration not visible in this excerpt

Table 7: iterations for cryptographic operations

The input dimensions to our core for the testing were:

- Provider or library
- Algorithms as far as supported by a provider
- Keylengths to be seen in table 1
- To examine if the plaintext length has significant influence on the algorithm speed (length-/time) we created random texts with the lengths: 128kbyte, 256kbyte, 512kbyte, 1mbyte and 2mbyte and ran the benchmarking for every plaintext length. Of course we also veri­fied the correctness of our implementation with encrypted and decrypted text.

3 RESULTS of Cryptobenchmarking

3.1 Results JAVA

In order to give a clear picture of the results we will now only present the key facts, split into the security levels defined at the beginning. In this section we only consider the JAVA language and compare the results for Linux and Windows together. We tested with 5 different input lengths as described above, but in the discussion we will focus on the results of 1 megabyte plaintext length input. The reason for this is that after a review of the output data we detected a linear coherence. That means that for example the encryption of 128 kilobytes lasts half the time of encryption of a file with 256 kilobytes. That does not proove a statistical coherence in general, as we did not use statistical tests. For our interpretation, however, it does not make a difference as our benchmark data behaves as if this coherence was given. First we will present both key setups, symmetric and asymmetric. Then proceed with encryption and decryption for both. This is followed by the digital signing and verifying. In the end we will describe the hashing and generation of hash based message authentication codes.

[...]