This is major strategic release. New release includes several overall improvements for the system stability, compatibility and performance. Major feature implemented is support for sub-UBF buffers (recursive buffers), support for PTR type fields in UBF and VIEW fields in UBF. New APIs have been added tpsetcallinfo() and tpgetcallinfo(). Platform independent network protocol mode for tpbridge has been refactored and receives performance boost as well as release includes full protocol description document. Another big feature for this release is inclusion of new tool "ubb2ex" which allows to perform automatic Oracle Tuxedo configuration file migration to Enduro/X format in matter of seconds. Tool performs best possible mapping to Enduro/X configuration, including, support for clustering, persistent queues, servers, services, routing. Migration document has been created which describes how manually to migrate from Oracle Tuxedo to Enduro/X. With this version AIX 7.3 builds are provided.
- Bgetr(), Bgetrv(), CBgetr(), CBgetrv(), Bfindr(), Bfindrv(), CBfindr(), CBfindrv(), Bpresr(), Bpresrv(), CBvgetr(), CBvgetrv(), CBgetallocr(), CBgetallocrv(), Bvnullr(), Bvnullrv(), CBvgetallocr(), CBvgetallocrv() - for quick recursive field access
WARNING: Any UBF buffers saved as blobs shall be completed prior upgrade, due to UBF internal structure changes.
WARNING: Application must be re-built if migrating from 7.5, 7.0 or earlier Enduro/X version.
Support #63 - Network protocol document
Support #265 - xatmi.h TPEVCTL and TPSVCINFO service name length fixes
Support #753 - Restore SIGCHLD/SIGHUP/SIGTERM/SIGINT handlers to default after ndrxd/cpmsrv forks
Support #752 - mkfldhdr allow duplicate field IDs
Bug #701 - System-V switching server rqaddr requires full restart
Feature #211 - Write a document for Migration from Tuxedo to Enduro/X
Support #753 - Restore SIGCHLD/SIGHUP/SIGTERM/SIGINT handlers to default after ndrxd/cpmsrv forks
Support #752 - mkfldhdr allow duplicate field IDs
Feature #226 - Recursive UBF field support in UBF
Bug #601 - Bprint() shall escape BFLD_CHAR fields
Feature #518 - Add support for tpgetcallinfo() / tpsetcallinfo()
Support #622 - Baddfast() shall not allow to add bfldid lesser then previously added.
Feature #517 - New data types: PTR , FML32 and VIEW32
Support #599 - Parallel build support - issue with viewc
Support #554 - Bcmp / Fcmp32 shall return -2 on error
Bug #555 - Bextread / ud equal (=) operator might use corrupted data
Support #759 - ndrxd shall unlink local died local service queue, regardless of shared network service available or not
Bug #639 - ndebug.h api consider const char * to use for NDRX_LOG, etc. So that C++ would bark about warning: ISO C++ forbids converting...
Support #267 - Update getting started doc to use common configuration with xadmin provision
Bug #756 - tpcall sequence number may duplicate
Bug #755 - Request file logging tplogsetreqfile() may close stderr FILE desciptor, if one was used
Support #758 - Updated UID generatione algorythm
Support #761 - ndrxd reduce log levels
Support #742 - For consistency, tmqueue default queue workers shall be set to 1 instead of 2
Bug #745 - XATMI server startup hangs, in case if tpsvrthrinit() callback returns -1
Bug #747 - debug.conf threaded=y may produce invalid file names
Support #748 - Fix for CVE-2021-41556
Bug #749 - TPGOTSIG not reported, instead error TPEBADDESC is returned
Bug #750 - Incorrect XA switch is used for IBM DB2
Feature #447 - Implement: tpgblktime(), tpsblktime()
Bug #602 - TMQ invalid prepare files shall be dropped (i.e if size is bellow the must have size)
Bug #624 - SystemV mode Enduor/X core dump
Support #634 - Review global transaction abort rules when particpated server fails
Bug #666 - UBF Carray convert to string, even if carray contains 0x00, the 0x00 is not being added to output string, but converted to maximum carray len, with random string bytes starting from 0x00 byte pos
Support #667 - tmqueue keep in progress (uncommitted) message status after the tmqueue restart
Support #671 - If calling server in transaction mode, the callers transaction shall be automatically suspend
Support #673 - stderr logrotate for xatmi servers
Bug #675 - If autotran setting for service is set to N, but defaults was set to Y, then service still gets the Y setting (enabled autotrans)
Support #677 - tmsrv two phase commit state machine review
Feature #699 - errorq setting for tmqueue
Support #703 - tmsrv corrupted file housekeeping
Bug #704 - tpbridge in protocol mode (-f) cannot handle UBF messages larger than 32K bytes
Feature #709 - Tool for orphaned transaction rollback
Feature #714 - tmqueue internal transaction accouting
Feature #715 - ndrxdebug.conf threaded mode new setting for mutex locking
Feature #717 - corrid compatibility for tmqeueue
Bug #719 - tpalloc/tprealloc/tpfree race condition issue in highly multi-threaded environment
Support #721 - tmqueue QoS support forwarder message distribution
Bug #725 - tmqueue might deadlock when performing tpenqueue() or tpdequeue() in global transaction
Feature #726 - tmqueue optimization - forwarder thread wakeup in case if sleeping and having enqueued forwardable msg
Bug #730 - tmqueue / forward transaction/XATMI call timeout
Support #729 - Reduce error logging for read only transactions
Bug #734 - Transaction Processing: Read only committed transaction can corrupt the commit flow
Support #737 - XA resource reconnect engine shall be improved - support for tmsrv reconnects
This is major strategic release. Major changes in this release is that, there is no more requirements of the large process stack are. Now Enduro/X uses dynamic allocated memory for internal buffer management. To keep the performance levels high, Enduro/X employs Fast Pool Allocator (FPA) which caches certain sizes of allocated memory blocks, so that lot of malloc() calls are eliminated. Previously new thread stack size was based on message size factor estimations. Now NDRX_THREADSTACKSIZE environment variable is used for this purposes (and if not sent default 8M is used). Other news is that this version implements more compliant build tools, including buildserver, buildclient and buildtms tools. Now buildserver generates the main() entry points of the server processes. Also when generating, main function, automatic service advertise is provided too. Build tools includes '-r' flag for static XA resource manager symbol application, thus for these cases libndrxxatmsx.so driver shall be used to use built-in xa switch handler.
New version of Enduro/X supports XATMI dispatcher multi-threading, supports callbacks of tpsvrthrinit() and tpsvrthrdone() via _tmstartserver(). Enduro/X event dispatching server tpevsrv is upgraded to multi-thread support via mindispatchthreads/maxdispatchthreads settings.
With this release Data Dependent Routing (DDR) support has been introduced. With DDR service call priority handling is added (only for Linux and FreeBSD), as well auto-transactions are added for XATMI services. See ndrxconfig.xml(5) manage for configuration details.
This version introduces new framework - Latent Command Frameworks - which allows to publish certain commands in shared memory and Enduro/X libraries processes these commands at debug points (which are in many places in Enduro/X). The published command checking is done very efficiency and does not introduce any overhead. With help of these commands now in the real-time logging configuration for processes can be changed and also loss-less logrotate can be performed. Framework allows for developers to publish custom commands to processes via plugin interface and such new API commands as: ndrx_lcf_func_add(), ndrx_lcf_xadmin_add(). With this LCF framework new shared memory segment is open by Enduro/X at startup. After the shutdown the shared memory segment is kept open (but all command data is reset). To fully remove all shared memory segments, use xadmin down -y command.
UBF Buffer now is optimized for BFLD_STRING field handling. The UBF buffer internally caches the string length, previously any iteration over the UBF buffer made strlen() for each string field occurrence, now cached length is used.
Generic System V mode has been reworked. Instead of previously using internal unnamed was utilized on every timed message processing. Now System V time-out scanning thread wakes ups every NDRX_SCANUNIT milliseconds, checking for queue thread time-out. This boost performance for System-V as no other internal message roundtrip is needed, but now periodic wakups are performed by XATMI processes for time-out scanning.
Thanks to IBM AIX extension now polling on System-V queues are performed in the same fashion as it is done on Linux.
Due to poor performance, and fact that no customers are using GPG-ME bridge traffic encryption, this functions is removed from the tpbridge process.
PDF documentation is removed from build processes, and binary distributions which includes documentation release, now includes HTML only documents.
For better analysis, logging is moved to microseconds format.
With this release, new endurox-connect version 2.5.x is released, which includes Distributed Transaction orchestration API over the web services. endurox-connect v2.5.x is not compatible with earlier Enduro/X versions.
- BF_LENGTH/F_LENGTH value is changed from 32 to 64 (Max UBF field length)
- _tmstartserver() added arguments to tmsvrargs_t for p_tpsvrthrinit/p_tpsvrthrdone
- UBF buffer format changes due to string len caching. Thus any binary blob saved from previous Enduro/X version will not work in new version (if there are string fields)
- If server processes are started by _tmstartserver(), then there might be possible segmentation fault, as new Enduro/X version probes for tmsvrargs_t for p_tpsvrthrinit/p_tpsvrthrdone fields. If _tmstartserver() was not used, then there are no ABI changes to this aspect.
- If programs were using Enduro/X logging framework via ndebug.h provided macros, applications needs to be re-built. This also applies to any modules which were using endurox-go.
Feature #547 - BFLD_STRING shall store cached string length internally
Support #553 - Move tpexport/tpimport to dynamic memory
Feature #545 - Add call descriptor ranges for conversational IPC, avoid fast cd reuse
Feature #540 - Threadpool optimizations for waiting for free slot
Feature #218 - buildserver support for automatic server advertise
Feature #368 - xadmin start dead process detailed reason infos
Feature #398 - MINDISPATCHTHREADS and MAXDISPATCHTHREADS
Feature #440 - AIX Message queue poll feature
Feature #463 - System V queue interface optimizations
Feature #497 - Allow tpacall with TPNOREPLY to be executed from tpsrvinit()
Feature #511 - Move logs to microsecond format
Feature #539 - Thread pool shall use spinlocks for protecting internal counters
Bug #542 - tpgetrply for same thread multiple active call descriptors might drop call descriptor if reading the reply for different active all descriptor
Feature #549 - Moved to dynamic memory instead of stack allocation
Bug #576 - tmqueue at startup might get race conditions and create folder likes: ./QSPACE1/prepared/prepared
Feature #577 - tpbridge -i should accept host names too
Bug #580 - ini config duplicate key values in same section might cause memory leak
Support #582 - Move packaged documentation to HTML format
Support #562 - macos slow xadmin stop
Bug #575 - Binary with atmictx client starting first and only after server may cause to go out of shared memory mode
Bug #584 - server <sysopt>, <appopt> and cpmsrv client cmdline string argument quote issues
Support #585 - Remove gpg-me support for bridge traffic encryption
Support #558 - bridge overload
Bug #560 - tpcall() with TPETIME does not cancel the message
Support #557 - xadmin output updates
Bug #486 - if using tpacall() mixing with tpcall() may cause to drop pending async responses
Support #657 - System-V mode service semaphore locking algorithm review
Feature #416 - exbench Enduro/X benchmark tool experimental release
Feature #286 - async logger and latent command framwork (LCF)
Bug #608 - xadmin cli failures
Support #633 - Bboolev with fields longer than 511 chars might get invalid results
Feature #655 - Static analysis of the source
Feature #401 - DDR for services
Support #611 - Excompat cleanup, leave only used fields
Feature #594 - Add support for TPEXIT flag
Feature #613 - ndrxconfig.xml shall allow no-entities - flatten the XML
Support #623 - Bridge clock adjustment improvements
Feature #598 - svqids and svsemids shall work in similar way as shms. Added headers for shms command
- Priority Subject
#465 Bug test038_tpnotify failure of tpbridge with core
#454 Feature tplogsetreqfile() feature for attempt to create folder, if file is missing and fails to create
#460 Feature Plugin for debug interface
#459 Support In SystemV mode test060_ndxdfork might cause locked child processes
#457 Support add unit test for TPETIME when doing tpconnect
#456 Feature xadmin start /stop shall wait if ndrxd is in non normal context
#433 Bug ulimit -s unlimited causes tmsrv and tpbridge boot failure
#436 Bug ud process core dumps when cache reallocates the buffer and output buffer is different (for tpcall) than input buffer
#437 Feature new cache flag nosvcok
#441 Bug cache key string build error
#446 Bug move atmi buffer management to mutex locks instead of spinlocks
#76 Feature Java bindings using JNI
#158 Feature Tuxedo style MIB API
#452 Bug ndrx_str_env_subs_len might process length incorrectly
#415 Support endurox-java JNI Class/MethodID/FieldID cache
#399 Support Add unit test for tmqueue transaction timeout.
#421 Bug In Automatic transaction mode tmqueue forwarder fails to finish the failed transaction even tries exceeded
#425 Bug ndrxd died queue reply to servers not waiting for reply (called with tpacall/TPNOREPLY)
#409 Feature tmsrv optimizations
#426 Support auto-buffer deallocation during the service processing
#428 Bug xadmin rc non_existing_client core dumps
#418 Feature Implement RM transaction manipulations - recoverlocal, commitlocal, abortlocal and forgetlocal
#417 Bug tpcall in distributed transaction does not return correct tperrno using TPTRANSUSPEND
#423 Bug test021 - too many arguments output error
#414 Bug tpalloc/tprealloc calculates target buffer in fields, not in bytes
#422 Support Base64 handling security improvements
#413 Support XID internal processing
#403 Support tpenqueue/ex shall allow NULL buffers
#411 Feature xadmin - Terminal input handling improvement - command edit/auto completion/history
#408 Bug tpdequeue might give wrong order messages
Release binary builds are in progress.
Mavimax Ltd announced release of Enduro/X 6.0 middleware. With this release significant development have been done for Unix operating system support. As with time, it was shown that Posix queues are not reliability implemented on other Unix operating systems than GNU/Linux and FreeBSD. Thus to ensure that Enduro/X can perform well on such systems as IBM AIX and Oracle Solaris, Mavimax have invested significant resources into providing new version of Enduro/X. Now standard releases for Unixes are done on System V message queue IPC resources. Other improvements of Enduro/X includes new APIs for being more compatible with competitor software. This version of Enduro/X also includes works done for upcoming Java package, thus product is able to handle different styles on how server binaries are started, including modes with wrapper scripts.
Version 6.0 also introduces support of Oracle SPARC CPU architecture for enterprise grade processing. Release cycle now delivers Enduro/X for IBM AIX 7.1 (powerpc), Oracle Solaris 10 (sparc), For x86_64 architecture release is made for Oracle Solaris 11, Suse Enterprise Linux Server (SLES) 12 and 15, Ubuntu Server 14, 16, 18, Red Hat Enterprise Linux 6 and 7.
As usual release includes bug fixes, discovered in the development cycle.
List of fixes and updates can be located here: https://www.endurox.org/issues/328
With this announced release [[https://blogs.oracle.com/linux/announcing-the-general-availability-of-the-unbreakable-enterprise-kernel-release-5]] it is highly recommended to move upgrade to this kernel when using Enduro/X middle-ware.
The benefit for Enduro/X is related with fact that this kernel version offers EPOLLEXCLUSIVE feature. Meaning that highly load balanced server processes won't be waken up for nothing, just to get the notification that some other process have already pulled workload from queue. The EPOLLEXCLUSIVE ensures that only one sleeping process is waken if there are some message arrived on service queue. Even if say 500 load balanced XATMI servers are waiting for message.
The tests was performed here [[https://mvitolin.wordpress.com/2015/12/05/endurox-testing-epollexclusive-flag/]]. This now this feature is available in enterprise grade Operating system. After some 3 years :)
During the testing the results shows that Enduro/X shows goods results with RPC
style calls. For Oneway testing (publish only), Apache Kafka was the best. But
downside for Kafka is that calls were not blocked, thus “dirty” solution was to
restart the sending message to the queue while the Enduro/X no “dirty fixes” are
needed, as when the queue is full, the process blocks and waits for free space.
Then after all the ActiveMQ with Go STOMP client was also showing good results
when message size increased. But the downside with STOMP client was that, during
the testing, the client got disconnected for some reason. RabbitMQ also seems to
be stable platform and no special fixes were needed to execute the test cases
(except that channel send must be used to get “blocked” style calls). RabbitMQ
showed quite good stable results.
Also available in: Atom