Feature #409

Feature #357: Enduro/X 7.0 master task

Feature #76: Java bindings using JNI

tmsrv optimizations

Added by Madars Vitolins 3 months ago. Updated 15 days ago.

Status:ClosedStart date:04/11/2019
Priority:NormalDue date:
Assignee:-% Done:

100%

Category:-
Target version:-

Description

For mysql XA there is no possiblity to join transaction, thus when tmsrv creates transaction the atmi_xa_start_entry and atmi_xa_end_entry calls possibly are not needed.

History

#1 Updated by Madars Vitolins 3 months ago

  • Subject changed from tmsrv optimiztaions to tmsrv optimizations

For some databases like mysql and postgresl we might need to support for one resource manager several (may be thousands) of branch qualifiers This may be built by cluster node id, resource manager id, process id, timestamp, and sequence number within the process.

Each time process starts the transaction, it requests the new xid tmsrv. then this "branch" shall be associated with the "resource manager id"/group. These xids must be logged. When performing the prep/commit/abort of the transaction, we shall loop over the xids assoced and perform the action. The result of each xid must be logged. The final result of "RM" shall be given as worst status of we have.

For postgresql:

- with new begin, we generate xid
- when process performs xa_end, we shall call prepare for that transaction
- when tmsrv calls prepare again for all of the transactions, if any of them fails, then we shall go to abort phase, due to fact that some process have not completed it's duties.

As the we use guide in txn id, ten bqual needs to be appended with transaction id assigned by tmsrv, this could be a long 64bit number. Also this will make less pressure on log file format.

Thus as now we have guid+rmid, then we get new XID in format of guid+rmid+tid

#2 Updated by Madars Vitolins 3 months ago

  • Parent task set to #76

#3 Updated by Madars Vitolins 3 months ago

tpsuspend() -> for prosgres we prepare tran. For Mysql we end tran.
tpresume() -> we request new XID from TMSRV.

When flag "TPTRANSUSPEND" is set for tpcall() no suspend is needed, because target invocation will be in different branch, thus no locking will be required.

#4 Updated by Madars Vitolins 2 months ago

For posgresql support, seems that TLOG on disk we need to open at the begining of the transaction, because there is no such thing on PG as active transaction (globally), it is either active locally or prepared (which is done on xa_end).

Thus we need to track all xids to do their abort.

We need some kind of flag to be passed to TMAPI to request new xid which shall be part of the transaction.

#5 Updated by Madars Vitolins about 2 months ago

The XID by it self as XIDSTR we could leave as is, no need to append any bqual data there.

#6 Updated by Madars Vitolins about 2 months ago

postgresql xid string format:

    return xid.getFormatId() + "_" 
        + Base64.encodeBytes(xid.getGlobalTransactionId(), Base64.DONT_BREAK_LINES) + "_" 
        + Base64.encodeBytes(xid.getBranchQualifier(), Base64.DONT_BREAK_LINES);

#7 Updated by Madars Vitolins about 2 months ago

mysql:

    private static void appendXid(StringBuilder builder, Xid xid) {
        byte[] gtrid = xid.getGlobalTransactionId();
        byte[] btrid = xid.getBranchQualifier();

        if (gtrid != null) {
            StringUtils.appendAsHex(builder, gtrid);
        }

        builder.append(',');
        if (btrid != null) {
            StringUtils.appendAsHex(builder, btrid);
        }

        builder.append(',');
        StringUtils.appendAsHex(builder, xid.getFormatId());
    }

#8 Updated by Madars Vitolins about 2 months ago

/**
 * Get connection handle for non XA resources.
 * This shall perform callback initialized by ndrx_get_xa_switch().
 * the ndrx_get_xa_switch() shall provide the function pointer by ndrx_xa_setgetconn()
 */
void * tpgetconn();

#10 Updated by Madars Vitolins about 1 month ago

Posgresql mode C mode driver shall work in ECPG:

/* Open / close */
bool     ECPGconnect (int lineno, int c, const char *name, const char *user, const char *passwd, const char *connection_name, int autocommit) 
bool     ECPGdisconnect (int lineno, const char *connection_name)

/*
 * Returns current connection name for PG
 */
tpgetconn();

with this you may access:

- get dynamic connection:
- PGconn * ECPGget_PGconn (const char *connection_name)

- get esql connection:
- ecpg_get_connection(const char *connection_name)

and thus provide connections for both embeded env & dynamic env. Both mode shall be tested.

#12 Updated by Madars Vitolins about 1 month ago

for postgres to avoid the commits where process have works still in progress, we shall do following protocol:

- at start of TID, the PG RM goes into Unknown state ('u').
- when process reaches end, it updates the TMSRV with XA_RM_STATUS_PREP if prepare transaction was ok
- if prepare transaction was failure (PG performs automatic rollback), TMSRV shall be updated with status XA_RM_STATUS_ABORTED.

- In case if tmsrv reaches timeout, and transaction is in state of 'u', the transaction must be assigned for abort
- The status change shall be allowed only when state is 'u' or states matches (for repeated send). If status not changed or transaction is not found, error shall be reported back to caller. In case of unmatched state/not 'u' or transaction is not found, the local process must roll back it.

#13 Updated by Madars Vitolins 19 days ago

when doing tpabort(), posgresql must be rolled back with out attempt of doing prepare transaction at xa_end().
The XA driver shall export the callback function for local transaction abort.

#14 Updated by Madars Vitolins 15 days ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

#15 Updated by Madars Vitolins 15 days ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF