Enduro/X benchmarks


Table of Contents

1. Overview
2. Preparation
2.1. Preparing on Ubuntu/Debian like system
3. Asynchronous tpacall()
4. Local tpcall()
5. Networked tpcall()
6. Multi-process/multi-thread tpcall()
7. Persisted storage benchmark, tpenqueue()
8. Single-threaded tpcall() on cached service
9. Running the benchmarks

1. Overview

This document generally contains Enduro/X benchmark overview, performed on different platforms. Different aspects are being tested and performance results are analyzed. Testing will cover fundamental message exchange between client and server, synchronous and asynchronous aspects are tested. Document also covers persistent storage benchmarks.

2. Preparation

This section gives some notes how to prepare for benchmark. What packages are required to be installed. This assumes that standard Enduro/X build installation is performed.

2.1. Preparing on Ubuntu/Debian like system

$ sudo apt-get install r-base

3. Asynchronous tpacall()

This tests uses one way calls to the server process. At the end of the calls it is ensured that all messages are processed by server. Only then results are plotted.

benchmark/04_tpacall.png

4. Local tpcall()

This test includes locally running ATMI client and ATMI server. The interprocess comunication happens with help of kernel queues (kq) or shared memory (shm). The polling mechanisms are either epoll() (works on linux, most efficient way) and usual poll() for which event chain is bit longer.

benchmark/01_tpcall.png

From the results can be seen that performance mostly stays stable at different data size loads.

5. Networked tpcall()

At this test basically same works is done as for above, but instead two instances of application servers are started and they are interconnected with network.

benchmark/02_tpcall_network.png

Here we see that network performance slightly fluctuate. That could be related with fact now bridge is involved to transfer the message over the network, and process count is bigger than CPU core count. Thus scheduler comes in work.

6. Multi-process/multi-thread tpcall()

This test cases employs five ATMI servers, and one ATMI client which inside have 5x threads, sending massages to the servers

benchmark/03_tpcall_threads.png

7. Persisted storage benchmark, tpenqueue()

This test gets much lower results, because all messages are being saved to disk. Also note that internally Enduro/X uses distributed transaction manager to coordinate the save of the message, thus processing of XA transaction takes some disk resources too.

benchmark/05_persistent_storage.png

8. Single-threaded tpcall() on cached service

This tests performs benchmark over the service for which results are cached. For best results cache database is stored on RAM driver (on Linux system).

benchmark/06_tpcache.png

9. Running the benchmarks

It is possible to run benchmarks on your own system. Note that for chart plotting R language is used. If R will not be installed, then charts will not be generated. But results still can be read from data files. Benchmark script is located in doc/benchmark folder, named build.sh <configuration name>. The configuration name is arbitrary description of your system on which you perform the tests. Shall not contain spaces. Results are plotted into text files located in same directory.

Note

That you must enable the Q message size up till 56000 bytes. On Linux system that would mean that /etc/rc.local needs to be set to:

...
echo 56000 > /proc/sys/fs/mqueue/msgsize_max
...

And in setndrx we shall also enable that message size:

...
# Max message size (in bytes)
export NDRX_MSGSIZEMAX=56000
...

For our sample user (user1), then running could look like:

$ cd /home/user1/endurox/doc/benchmark
$ ./build.sh my_system,linux,ssd
$ ls -1
01_tpcall.png
01_tpcall.txt
02_tpcall_dom.txt
02_tpcall_network.png
03_tpcall_threads.png
03_tpcall_threads.txt
04_tpacall.png
04_tpacall.txt
05_persistent_storage.png
05_persistent_storage.txt
build.sh
genchart.r