DESCRIPTION

This is special ATMI server which is used for distributed transaction coordination. For new transactions started with tpbegin(), tmsrv generates new XID and passed it back to transaction initiator. At the same time transaction is remembered by tmsrv as active transaction and time-out counter is checked by background thread.

In Enduro/X XA Resource Managers are numeric identifiers, which are allowed to be in range of 1..32. Enduro/X’s Resource Manager ID (RMID) is same identifier as Group Number or grpno known in other ATMI systems.

If during distributed transaction processing new resource manager is associated with transaction, then process notifies initial transaction manager that new association must be made. In active phase this too is stored in process memory.

When transaction comes to commit() phase, then it is logged to disk. Every transaction is logged to separate file. File name contains resource manager ID and transaction XID. During the prepare phase status of every Resource Manager is logged. The same is with commit phase. Once all resources are completed, transaction file is removed. If tmsrv crashed for in-progress transaction, then transaction files are read from disk, and appropriate actions according to state machine are performed (aborted or rolled back).

If running transaction did time-out, then background thread will abort it automatically, and for caller process commit() will fail with abort-only message.

In cluster environment, other transaction manager of corresponding resource managers are involved for prepare/abort/commit actions. Also as every service doing as part of single transaction, must have a cluster link to initial transaction manager (for registration purposes). Thus means that all involved nodes, must have direct cluster visibility, otherwise transactions will fail.

Transaction managers can be load balanced with ndrxconfig.xml with min/max attributes. In load balance manner at tpbegin() corresponding free transaction manager will be selected. Later at transaction process this manager is responsible for full cycle of the transaction. Other resource managers for this transaction will help for prepare/commit/aborts of other RMs. These other TMs will be selected in load balanced mode.

Every instance of tmsrv will advertise following list of services

For example for Resource Manager ID 1, Cluster Node ID 6, Enduro/X Server ID 10 services will look like:

Currently 1. format is used for starting the transaction, and accepting prepare/commit/abort calls from master TM of the transaction. Service Nr 3. is used by transaction initiator for subsequent calls of the tpcommit()/tpabort(). Also 3. is used by services involved in transaction to register new Resource Manager ID as part of the transaction.

For XA processing, resource manager drivers are loaded via dynamic loadable shared libs. Resource manager should expose xa_switch in shared lib. For every different resource manager, there is different Enduro/X server running. Enduro/X process gets associated with corresponding RMDI via NDRX_XA_RES_ID environment variable.

To configure different RMID’s for different processes or tmsrvs. Use the Enduro/X build in facility of environment variable override. See the manpage of ex_env(5).

Enduro/X support static and dynamic XA registration.

LOGGING AND RECOVERY

tmsrv register all activities of the transactions and resource managers in the machine readable log file. Log file is used for crash recovery, where last state of the transaction is read and transaction is completed according to the target state set in log file.

Each log file line contains a CRC32 checksum, which is verified during the recovery, any bad line is ignored, which might happen in case if data have not fully flushed to the disk. If during the recovery process some of the lines are invalid, they are ignored, and tmsrv acts with knowledge of last known state.

When transaction is started or when new resource manager joins the transaction or when commit/abort request is made, logging is mandatory, i.e. if disk is full or permissions error, the transaction is either not started or state not changed.

When transaction is finalized (committed or aborted) the transaction and resource states are logged optionally, thus write errors are ignored (but logged to ULOG). Thus if recovery is necessary at this stage, transaction would be finalized according to any last valid data logged.

If after crash recovery some transaction still exist in Resource Manager as not completed, following xadmin(8) commands may be used to finish them at particular Resource Manager level:

WARNING: These commands does not consult with the originatig transaction managers for the transaction statuses, thus these command shall be used only when system is idling (not processing any useful workload and it is known that there some records at resources locked / stuck at prepare stage).

To collect and rollback any orphaned prepared transactions, it is recommended to configure singled tmrecoversv(8) copy at the end of ndrxconfig.xml(5) server startup sequence, so that this server automatically would collect any broken transaction branches. Or after the system startup, manually invoke tmrecovercl(5) command line tool to perform transaction collection and rollback.

By default tmsrv writes log files to disk and uses fflush() unix call to persist the data. This call submits the message only to Operating System kernel, but does not guarantee that data is written to disk. Thus if power outage happens some transaction information may be lost. Thus for critical systems it is recommended to use special flags which instruct the tmsrv to perform disk synchronization when commit decision have been taken. These flags shall be set in NDRX_XA_FLAGS: FSYNC or FDATASYNC and DSYNC. The FSYNC or FDATASYNC corresponds to fsync() and fdatasync() unix calls to flush the transaction log data to disk. DSYNC ensures that log file directory structure is flushed to disk. FDATASYNC is bit faster than FSYNC, as it does not update the file last change and other insignificant attributes). Usage of these flags may significantly reduce the transaction TPS performance. DSYNC usage depends on the operating system. It is necessary for Linux and Solaris operating systems. For other operating systems, please consult with vendors manuals, when directory fsync is needed for new files to be persisted.

OPTIONS

XA RECOVER SETTINGS FOR ORACLE DB

The -R mode might not be enabled in database for user. I.e. user is not allowed to see open transactions. Thus must be enabled by following commands on DB user set in XA open string:

grant select on pending_trans$ to <database_user>;
grant select on dba_2pc_pending to <database_user>;
grant select on dba_pending_transactions to <database_user>;
grant execute on dbms_system to <database_user>;  (If using Oracle 10.2)
grant execute on dbms_xa to <database_user>; (If using Oracle 10.2)

ORACLE RAC SETTINGS

If planing to use Oracle RAC, the to successfully process distributed transaction across binaries which are connected to different RAC nodes, Oracle RAC DTP Service must be configured with Singleton option, so that only one node actively serves the transactions.

If this above is not configured and say two binaries are working with same XA transaction, one binary is connected to first RAC node and other binary with second RAC node, the transaction will not work, as XA API will not see the transaction on other node than where it was started.

Enduro/X does not support RAC XA Affinity and does not process Oracle Fast Application Notifications (FAN).

For more details consult with Oracle instructions, as basically Enduro/X uses plain X/Open XA API for managing the transactions, and it is expected that Oracle DB provides support for XA API.

LIMITATIONS

When using dynamic registration xa switches with the RECON XA flag functionality, to keep the process working in case if communications are lost while executing non XA AP code e.g. SQL statements, the process by it self must perform tpclose(3)/tpopen(3) until it succeeds, or process shall perform exit so that Enduro/X would restart it. This extra logic is needed due to fact, that if outside of XA API communications are lost, the Enduro/X by it self would not see that comms status have changed because ax_start() is executed only when resource is modified by the application. If comms are not working in the application, the resource is not modified and thus ax_start() is not invoked.

When process joins the transaction (either initiator or participating XATMI server), firstly it register with tmsrv and only then performs xa_start() API call. If transaction at tmsrv expires concurrently while joining process have not yet called the xa_start(), there is possibility that orphan transaction may be created (i.e. created active transaction in the resource, but transaction is not managed by Enduro/X as already rolled back). To overcome this limitation, careful transaction timeout planing shall be performed which applies to tpbegin() setting and timeout setting at the resource for inactive transactions.

If transaction expires at tmsrv, this fact does not terminate any tpcall(3) operations, except that if called service’s associate resource manager is not registered with given global transaction.

EXIT STATUS

BUGS

Report bugs to support@mavimax.com

SEE ALSO

ex_env(5) buildtms(8) xadmin(8) tmrecoversv(8) tmrecovercl(8)

COPYING

© Mavimax, Ltd