News

Enduro/X 7.5 Feature release

Added by Madars Vitolins about 1 month ago

Overview

This is major strategic release. Major changes in this release is that, there is no more requirements of the large process stack are. Now Enduro/X uses dynamic allocated memory for internal buffer management. To keep the performance levels high, Enduro/X employs Fast Pool Allocator (FPA) which caches certain sizes of allocated memory blocks, so that lot of malloc() calls are eliminated. Previously new thread stack size was based on message size factor estimations. Now NDRX_THREADSTACKSIZE environment variable is used for this purposes (and if not sent default 8M is used). Other news is that this version implements more compliant build tools, including buildserver, buildclient and buildtms tools. Now buildserver generates the main() entry points of the server processes. Also when generating, main function, automatic service advertise is provided too. Build tools includes '-r' flag for static XA resource manager symbol application, thus for these cases libndrxxatmsx.so driver shall be used to use built-in xa switch handler.

New version of Enduro/X supports XATMI dispatcher multi-threading, supports callbacks of tpsvrthrinit() and tpsvrthrdone() via _tmstartserver(). Enduro/X event dispatching server tpevsrv is upgraded to multi-thread support via mindispatchthreads/maxdispatchthreads settings.

UBF Buffer now is optimized for BFLD_STRING field handling. The UBF buffer internally caches the string length, previously any iteration over the UBF buffer made strlen() for each string field occurrence, now cached length is used.

Generic System V mode has been reworked. Instead of previously using internal unnamed was utilized on every timed message processing. Now System V time-out scanning thread wakes ups every NDRX_SCANUNIT milliseconds, checking for queue thread time-out. This boost performance for System-V as no other internal message roundtrip is needed, but now periodic wakups are performed by XATMI processes for time-out scanning.

Thanks to IBM AIX extension now polling on System-V queues are performed in the same fashion as it is done on Linux.

Due to poor performance, and fact that no customers are using GPG-ME bridge traffic encryption, this functions is removed from the tpbridge process.

PDF documentation is removed from build processes, and binary distributions which includes documentation release, now includes HTML only documents.

For better analysis, logging is moved to microseconds format.

API changes

  • BF_LENGTH/F_LENGTH value is changed from 32 to 64 (Max UBF field length)
  • _tmstartserver() added arguments to tmsvrargs_t for p_tpsvrthrinit/p_tpsvrthrdone

ABI changes

  • UBF buffer format changes due to string len caching. Thus any binary blob saved from previous Enduro/X version will not work in new version (if there are string fields)
  • If server processes are started by _tmstartserver(), then there might be possible segmentation fault, as new Enduor/X version probes for tmsvrargs_t for p_tpsvrthrinit/p_tpsvrthrdone fields. If _tmstartserver() was not used, then there are no ABI changes to this aspect.

Fixes

Feature #547 - BFLD_STRING shall store cached string length internally
Support #553 - Move tpexport/tpimport to dynamic memory
Feature #545 - Add call descriptor ranges for conversational IPC, avoid fast cd reuse
Feature #540 - Threadpool optimizations for waiting for free slot
Feature #218 - buildserver support for automatic server advertise
Feature #368 - xadmin start dead process detailed reason infos
Feature #398 - MINDISPATCHTHREADS and MAXDISPATCHTHREADS
Feature #440 - AIX Message queue poll feature
Feature #463 - System V queue interface optimizations
Feature #497 - Allow tpacall with TPNOREPLY to be executed from tpsrvinit()
Feature #511 - Move logs to microsecond format
Feature #539 - Thread pool shall use spinlocks for protecting internal counters
Bug #542 - tpgetrply for same thread multiple active call descriptors might drop call descriptor if reading the reply for different active all descriptor
Feature #549 - Moved to dynamic memory instead of stack allocation
Bug #576 - tmqueue at startup might get race conditions and create folder likes: ./QSPACE1/prepared/prepared
Feature #577 - tpbridge -i should accept host names too
Bug #580 - ini config duplicate key values in same section might cause memory leak
Support #582 - Move packaged documentation to HTML format
Support #562 - macos slow xadmin stop
Bug #575 - Binary with atmictx client starting first and only after server may cause to go out of shared memory mode
Bug #584 - server <sysopt>, <appopt> and cpmsrv client cmdline string argument quote issues
Support #585 - Remove gpg-me support for bridge traffic encryption
Support #558 - bridge overload
Bug #560 - tpcall() with TPETIME does not cancel the message
Support #557 - xadmin output updates
Bug #486 - if using tpacall() mixing with tpcall() may cause to drop pending async responses

Release availability

See https://www.mavimax.com/downloads

Enduro/X 7.0 Release

Added by Madars Vitolins 11 months ago

  1. Priority Subject
    #465 Bug test038_tpnotify failure of tpbridge with core
    #454 Feature tplogsetreqfile() feature for attempt to create folder, if file is missing and fails to create
    #460 Feature Plugin for debug interface
    #459 Support In SystemV mode test060_ndxdfork might cause locked child processes
    #457 Support add unit test for TPETIME when doing tpconnect
    #456 Feature xadmin start /stop shall wait if ndrxd is in non normal context
    #433 Bug ulimit -s unlimited causes tmsrv and tpbridge boot failure
    #436 Bug ud process core dumps when cache reallocates the buffer and output buffer is different (for tpcall) than input buffer
    #437 Feature new cache flag nosvcok
    #441 Bug cache key string build error
    #446 Bug move atmi buffer management to mutex locks instead of spinlocks
    #76 Feature Java bindings using JNI
    #158 Feature Tuxedo style MIB API
    #452 Bug ndrx_str_env_subs_len might process length incorrectly
    #415 Support endurox-java JNI Class/MethodID/FieldID cache
    #399 Support Add unit test for tmqueue transaction timeout.
    #421 Bug In Automatic transaction mode tmqueue forwarder fails to finish the failed transaction even tries exceeded
    #425 Bug ndrxd died queue reply to servers not waiting for reply (called with tpacall/TPNOREPLY)
    #409 Feature tmsrv optimizations
    #426 Support auto-buffer deallocation during the service processing
    #428 Bug xadmin rc non_existing_client core dumps
    #418 Feature Implement RM transaction manipulations - recoverlocal, commitlocal, abortlocal and forgetlocal
    #417 Bug tpcall in distributed transaction does not return correct tperrno using TPTRANSUSPEND
    #423 Bug test021 - too many arguments output error
    #414 Bug tpalloc/tprealloc calculates target buffer in fields, not in bytes
    #422 Support Base64 handling security improvements
    #413 Support XID internal processing
    #403 Support tpenqueue/ex shall allow NULL buffers
    #411 Feature xadmin - Terminal input handling improvement - command edit/auto completion/history
    #408 Bug tpdequeue might give wrong order messages

Release binary builds are in progress.

Enduro/X 6.0 released

Added by Madars Vitolins almost 2 years ago

Mavimax Ltd announced release of Enduro/X 6.0 middleware. With this release significant development have been done for Unix operating system support. As with time, it was shown that Posix queues are not reliability implemented on other Unix operating systems than GNU/Linux and FreeBSD. Thus to ensure that Enduro/X can perform well on such systems as IBM AIX and Oracle Solaris, Mavimax have invested significant resources into providing new version of Enduro/X. Now standard releases for Unixes are done on System V message queue IPC resources. Other improvements of Enduro/X includes new APIs for being more compatible with competitor software. This version of Enduro/X also includes works done for upcoming Java package, thus product is able to handle different styles on how server binaries are started, including modes with wrapper scripts.

Version 6.0 also introduces support of Oracle SPARC CPU architecture for enterprise grade processing. Release cycle now delivers Enduro/X for IBM AIX 7.1 (powerpc), Oracle Solaris 10 (sparc), For x86_64 architecture release is made for Oracle Solaris 11, Suse Enterprise Linux Server (SLES) 12 and 15, Ubuntu Server 14, 16, 18, Red Hat Enterprise Linux 6 and 7.

As usual release includes bug fixes, discovered in the development cycle.

List of fixes and updates can be located here: https://www.endurox.org/issues/328

Oracle Linux releases UEK 5

Added by Madars Vitolins about 2 years ago

With this announced release [[https://blogs.oracle.com/linux/announcing-the-general-availability-of-the-unbreakable-enterprise-kernel-release-5]] it is highly recommended to move upgrade to this kernel when using Enduro/X middle-ware.

The benefit for Enduro/X is related with fact that this kernel version offers EPOLLEXCLUSIVE feature. Meaning that highly load balanced server processes won't be waken up for nothing, just to get the notification that some other process have already pulled workload from queue. The EPOLLEXCLUSIVE ensures that only one sleeping process is waken if there are some message arrived on service queue. Even if say 500 load balanced XATMI servers are waiting for message.

The tests was performed here [[https://mvitolin.wordpress.com/2015/12/05/endurox-testing-epollexclusive-flag/]]. This now this feature is available in enterprise grade Operating system. After some 3 years :)

Community feedback

Added by Madars Vitolins over 2 years ago

https://publicwork.wordpress.com/2018/06/27/kafka-vs-rabbitmq-vs-enduro-x-vs-activemq-on-go-benchmark-of-the-2018/

During the testing the results shows that Enduro/X shows goods results with RPC
style calls. For Oneway testing (publish only), Apache Kafka was the best. But
downside for Kafka is that calls were not blocked, thus “dirty” solution was to
restart the sending message to the queue while the Enduro/X no “dirty fixes” are
needed, as when the queue is full, the process blocks and waits for free space.
Then after all the ActiveMQ with Go STOMP client was also showing good results
when message size increased. But the downside with STOMP client was that, during
the testing, the client got disconnected for some reason. RabbitMQ also seems to
be stable platform and no special fixes were needed to execute the test cases
(except that channel send must be used to get “blocked” style calls). RabbitMQ
showed quite good stable results.

Enduro/X 5.3 released

Added by Madars Vitolins over 2 years ago

New version of Enduro/X offers configuration of XATMI service cache by means of administrative configuration. No additional programming is required to enable service cache. The service cache allows to increase the performance of existing applications which are using Enduro/X. Cache using proven LMDB memory mapped file technology, which allows to create a persisted and non persisted caches. The Enduro/X configuration allows to create distributed caches where data of caches is synchronized between cluster nodes. The configuration allows to define rules when data shall be cached and shall be invalidated. Caches can be limited by different strategies. Cached data can be marked with expiry too.

For configuration details see tpcached(3) man page. For configuration examples see https://github.com/endurox-dev/endurox/tree/master/atmitest/test048_cache.

This version also includes other bugfixes and small improvements.

List of fixes:
Feature #272, Bug #291, Feature #287, Bug #290, Support #279, Feature #294, Bug #293

Stable release version is: 5.3.4+.

Enduro/X 5.2 Released

Added by Madars Vitolins almost 3 years ago

Major work on support for PCI/DSS mandatory configuration encryption have been performed. Now Enduro/X is capable of encrypt and decrypt sections of configuration file. Provided command line utilities "exencrypt" and "exdecrypt" for encryption use in scripts. Introduction of plugin architecture (currently used for crypto key providers) So that software vendors may implement it's own algorithms for symmetrical encryption key storage.

Implemented tpconvert() ATMI call. Other feature now is that "@global" section for ini files are read twice, meaning that in ini file user may reference to previously defined env/global variable.

Overall fixes: #261 Bug, #118 Feature, #237 Feature, #236 Bug, #245 Feature, #258 Support, #259 Support, #255 Bug, #254 Bug.

Enduro/X 5.1 released

Added by Madars Vitolins almost 3 years ago

Enduro/X 5.1 have been released. The main changes in this version is support for big message sizes. Up till version 5.0 the max message size in system was limited to 64KB. Now with version 5.1 and upwards, the limit is removed. Now message size is limited by the Operating System - POSIX Message queue subsystem. More details about message queue, please see Enduro/X Admin Manual.

Other notable changes which are included in this release, is Client Process Monitor (cpmsrv) and corresponding command line utilities like "xadmin bc", "xadmin rc" and "xadmin sc" now are capable to handle the client tags and subsections by wild card matching. Symbol "%" can be used for substituting any string. Also CPMSRV command line utilities accept tag/subsection with out any extra arguments. For more information see xadmin's man page

There are few bugfixes made in tpbridge. The tpbridge protocol mode TLV is changed, now data length is encoded in 4 bytes (instead of 2). There are few performance improvements made in system with this release. Changes in UBF library to not to perform free space memset to zero. Also few buffer copies are eliminated from tpbridge. Tpbridge also got extra data validity checks to avoid buffer overruns in case of defective TCP packets.

Changes: Feature #127, Bug #229, Feature #230, Bug #234, Feature #244, Bug #243, Feature #248, Bug #240, Bug #238.

1 2 (1-10/19)

Also available in: Atom