Enduro/X Core

Some infos: https://www3.ntu.edu.sg/home/ehchua/programming/java/JavaNativeInterface.html

http://stackoverflow.com/questions/6152747/jni-not-support-types-as-void-unsigned-int-what-to-do

http://stackoverflow.com/questions/11803927/accessing-a-java-object-in-a-java-object-in-c-using-jni

Cmake samples for JNI: https://github.com/bureau14/qdb-api-jni/blob/master/CMakeLists.txt

JATMI:

- http://sqltech.cl/doc/oas11gR1/apirefs.1111/e13941/weblogic/wtc/jatmi/package-summary.html

- http://docs.oracle.com/cd/E79994_01/tuxedo/docs1222/javadoc/javasev/com/oracle/tuxedo/tjatmi/TuxAppContext.html

XA java transactions: https://docs.oracle.com/cd/B10501_01/java.920/a96654/xadistra.htm

http://www.tricoder.net/blog/?p=197

Technology sample: http://java-gnome.sourceforge.net/HACKING.html
$ git clone git://github.com/afcowie/java-gnome.git java-gnome

https://github.com/remram44/java-cpp-example

About views: https://docs.oracle.com/cd/E13203_01/tuxedo/tux71/html/dvlibra5.htm

https://john.nachtimwald.com/2017/06/17/calling-java-from-c/

https://www.nag.co.uk/IndustryArticles/CallingCLibraryRoutinesfromJava.pdf

https://github.com/mkowsiak/jnicookbook/tree/master/recipeNo035

https://cmake.org/cmake/help/v3.7/module/UseJava.html

https://stackoverflow.com/questions/819536/how-to-call-java-functions-from-c

https://www.javatpoint.com/java-naming-conventions

https://stackoverflow.com/questions/36849642/cmake-generate-a-jar-file-using-multiple-jar-dependencies-out-of-source-build

http://hildstrom.com/projects/jni/index.html

https://www.cl.cam.ac.uk/teaching/0910/CandC++/lecture8-6up.pdf

error handling: https://www.developer.com/java/data/exception-handling-in-jni.html

- Inside JNI functions, firstly we shall set the context manually, and then use standard tp commands (not O).

- When JNI error occurs, throw runtime exception. For ATMI errors, instantiate ATMI exception. For example if context is NULL.

ndrxconfig.xml could look like:

        <servers>
                <server name="someprocess">
                        <srvid>1</srvid>
                        <min>1</min>
                        <max>1</max>
                        <interpret>java -jar someprocess_package.jar</interpret>
                        <sysopt>-e /tmp/RECOVER -r</sysopt>
                        <appopt>-c10</appopt>
                </server>

So "someprocess" becomes logical name of the given server. But shall give options for formatting the command. There could be option for logical name in format string and the sysopt and appopt too:

        <servers>
                <server name="someprocess">
                        <srvid>1</srvid>
                        <min>1</min>
                        <max>1</max>
                        <sysopt>-e /tmp/RECOVER -r</sysopt>
                        <appopt>-c10</appopt>
                        <interpret>java -jar ${NDRX_PROCESS}_package.jar -- ${NDRX_SYSOPT} ${NDRX_APPOPT}</interpret>
                </server>

Which would in the final make following runnable:

java -jar someprocess_package.jar -e /tmp/RECOVER -r  -c10

This will require some changes in housekeep processes, as real executable will be java.

jni java instance in c: https://www.codeproject.com/Articles/993067/Calling-Java-from-Cplusplus-with-JNI

https://www.math.uni-hamburg.de/doc/java/tutorial/native1.1/implementing/method.html

https://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.html

https://stackoverflow.com/questions/44276882/jni-what-type-of-callstaticmethod-to-use

https://stackoverflow.com/questions/10408972/how-to-obtain-a-description-of-a-java-exception-in-c-when-using-jni

Throw pending: https://stackoverflow.com/questions/230689/best-way-to-throw-exceptions-in-jni-code/9812796

http://ant.apache.org/manual/tutorial-HelloWorldWithAnt.html

We need following interface for service advertise:

interface Server {
   public void tpSvrInit(string [] argv);
   public void tpSvrDone();
}

interface Service {
  public void tpService(TpSvcInfo svcinfo);
}

https://stackoverflow.com/questions/8051863/how-can-i-close-the-socket-in-a-proper-way

xa api: https://docs.oracle.com/cd/A97630_01/java.920/a96654/xadistra.htm

Seems for server's we need something like

    <defaults>
        <envs group="ENV1">
            <env>SOME_OTHER_ENV=HELLO WORLD</env>
        <envs>
        <envs group="ENV2">
            <env>SOME_OTHER_ENV2=HELLO WORLD</env>
        <envs>
    </defaults>
    <server name="cconfsrv">
        <srvid>1700</srvid>
        <min>1</min>
        <max>1</max>
        <sysopt>-e /tmp/CCONF</sysopt>
        <envs>
            <usegroup>ENV1</usegroup>
            <usegroup>ENV2</usegroup>
            <env>JAVA_OPTS=-some -flags ${SOME_OTHER_ENV}</env>
            <env>JAVA_CLASSPATH=-some2 -flags2</env>
        ...
        </envs>
    </server>
..
        <clients>
        <envs group="ENV1">
            <env>SOME_OTHER_ENV=HELLO WORLD</env>
        </envs>
                <client cmdline="/opt/bin/my_binary -t ${NDRX_CLTTAG} -s ${NDRX_CLTSUBSECT}">
            <env>
                <usegroup>ENV1</usegroup>
                <usegroup>ENV2</usegroup>
                <env>JAVA_OPTS=-some -flags ${SOME_OTHER_ENV}</env>
                <env>JAVA_CLASSPATH=-some2 -flags2</env>
            <env/>
                        <exec tag="BINARY1" subsect="1" autostart="Y" />
                        <exec tag="BINARY2" subsect="2" autostart="Y" />
                </client>
                <client cmdline="firefox"> <exec tag="FF"/> </client>
                <client cmdline="/tmp/test.bin" env="/tmp/yopt.env"> <exec tag="TT" log="/tmp/test.out"/> </client>
                <client cmdline="mongod -dbpath /home/mvitolin/tmp/data/db" >
                        <exec tag="mongo" autostart="Y"/>
                </client>
        </clients>

1. So we need to keep in defaults hash list of envs by set id.
2. The server keeps the linked list of envs
3. Server keeps linked list of pointer to linked resolved sets.

The same approach goes with client admin.

we shall provide exjc (Enduro/X Java Compiler) so that we could embed the JVM and compile in the jar files, which at the process boot we could extract to disk (/dev/shm preffered) load the jars, and start the binary.

This way we can get the normal executable, and customer have zero config requirements for the binary configuration.

Alos we might want to export server id too to env variables. So that any startup scripts might want to adjust their tcp port numbers or any other resources according to the server id

Also needs to consider what happens in cases when atmi server runs in background and the atmi server parent went off.

Dynamic class loader: https://github.com/poidasmith/winrun4j/blob/master/WinRun4J/src/java/JNI.cpp

Load jar from ZIP stream:

https://stackoverflow.com/questions/16602668/creating-a-classloader-to-load-a-jar-file-from-a-byte-array

+

https://stackoverflow.com/questions/8367126/how-can-i-convert-byte-array-to-zip-file

For the compiler, probably we need to join all jar files into single one and use it for the custom class loader.

Using custom class loader:

https://dzone.com/articles/java-classloader-handling

https://www.concretepage.com/java/custom-classloader-java-example

seems exembedfile adds some un needed zero at end of the array...

Seems like "setContextClassLoader" is needed for exjld for runner context to set the class loader.

We should have new package for enduroxjava https://github.com/Kitware/CMake/blob/master/Modules/UseJava.cmake

https://www.mkyong.com/java/java-custom-annotations-example/

https://keyholesoftware.com/2014/09/15/java-annotations-using-reflection/

https://stackoverflow.com/questions/32289952/how-can-i-switch-on-the-data-type-of-an-object-field

https://www.mkyong.com/java/java-find-class-fields-with-specified-data-type/

https://www.javatips.net/api/javax.transaction.xa.xaresource

1) For java application XA driver we need a driver to which from java side we can assign database handler for given XATMI conext. I.e. for example AtmiCtx.set(db XAResource),

The C side should have common driver like libexrmjdrv.so (java resource manager) for which there should be a thread local storage assigned AtmiContext with XAResource inside, and callbacks from call must be redirected back this context. EXRMJDRV shall be provided with some global reference to java env.

2) For tmsrv we need shared library which would start the java inside similarly like exjld does. Then it should create a XAResource and assign with ATMI Context and should work like library in step 1. For example loaders:
libndrxxajd.so, libndrxxajs.so (these should load and create JVM too), and backs: libexrmjdrv.so (java resource manager).

Needs to think how multi-threading will be done here. Each thread should have it's own AtmiContext. in case of 1) context can be set by java tpbegin from java caller...
for tmsrv which is multi-threaded, for each thread at init we need to create new AtmiContext and set the XAResource accordingly. There for EXRMJDRV there could be a thread local storage which if found to be NULL, will create new AtmiCtx. and then further will operate with it.

3) For normal apps which will use tpcall interface, we will need some kind of dummy xa driver (e.g. libndrxxadumd.so, libndrxxadums.so) and also dummy backend libexrmdumdrv.so which will hold just dummy xa functions.

https://stackoverflow.com/questions/8095016/unpacking-an-array-using-reflection

To track finalizer issues: https://heaphero.io https://dzone.com/articles/what-happens-behind-the-scene-finalize-method

Java finalizers and try-with-resources Statement https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html

might be used to deallocate C resources.

We could simulate JOLT api too: http://documentation.microfocus.com/help/index.jsp?topic=%2Fcom.microfocus.silkperformer.doc%2FSILKPERF-11C1E1E0-JOLTGETSHORTFUNCTION-REF.html

https://docs.oracle.com/cd/E13203_01/tuxedo/tux80/atmi/dvintro4.htm#6397

#
# Service type maps for jolt
#
[@joltmap]
service DEBIT = {"req":"UBF", "rsp":"VIEW:JVIEW2"}
service LASTAUTH = {"req":"UBF", "rsp":"VIEW:JVIEW2"}

Probably JOLT we shall build on rest-in api.

There shall be separate link URL transaction management, i.e. begin, commit.
The transaction ID shall be transferred in HTTP/S header with the original request buffer.
Currently no AUTH will be supported.
The service repo shall be some ini file accessible from java.

This way we could keep the max compatibility with Tuxedo JOLT...

https://www.mkyong.com/java/how-to-send-http-request-getpost-in-java/

Call Enduro/X services from java: https://dzone.com/articles/how-to-parse-json-data-from-a-rest-api-using-simpl
https://www.tutorialspoint.com/json/json_java_example.htm

java bin [have xa_switch] ---> load the driver which would use
per thread global variable has Java XA Switch.
- load the dll which would ndrxj_javabin_get_switch(), from the libexjava.so
-

if we want tmsrv to work with java code, we need to create a shared library which would:
- start VM
- load the "libexjava.so", call ndrxj_tmsrv_get_switch(), tpopen shall load the jars accordingly, set the Java Env variable (global) and check the AtmiCtx is there one for thread or not saved within libexjava.so. This would also load NDRX_XA_RMLIB jar jdbc library.

here question is about visibility, are the symbols global when loaded by jni.

https://www.postgresql.org/message-id/4674E56C.8060802@enterprisedb.com

Re: a simple example of XA (not working)
From:    Heikki Linnakangas <heikki(at)enterprisedb(dot)com>
To:    Luca Ferrari <fluca1978(at)infinito(dot)it>
Cc:    pgsql-jdbc(at)postgresql(dot)org
Subject:    Re: a simple example of XA (not working)
Date:    2007-06-17 07:40:28
Message-ID:    4674E56C.8060802@enterprisedb.com
Views:    Raw Message | Whole Thread | Download mbox
Thread:    
Lists:    pgsql-jdbc
Luca Ferrari wrote:
> Hi all,
> I'm trying to build a simple "manual" example of the use of the XA extension, 
> the following is the code:
> 
>     PGXADataSource dataSource = new PGXADataSource();
>     dataSource.setDatabaseName("hrpm");        
>     dataSource.setUser("luca");                
>     dataSource.setServerName("localhost");        
>     
>     XAConnection connection = dataSource.getXAConnection();
>     System.out.println("Connesso!");
>     XAResource resource = connection.getXAResource();
>     MyXid identifier = new MyXid();
>     identifier.setBranchQualifier(new byte[] {0x01, 0x02, 0x03, 0x04, 0x05});
>     identifier.setGlobalTransactionId(new byte[] {0x05, 0x04, 0x03, 0x02, 0x01});
>     identifier.setFormatId(100);
>     
>     try{
>         resource.start(identifier, XAResource.TMNOFLAGS);
>         
>         
>         Connection jdbcConnection = connection.getConnection();
>         Statement statement = jdbcConnection.createStatement();
>         String sql = "INSERT INTO ...";
>         int inserted = statement.executeUpdate(sql);
> 
>         resource.end(identifier, XAResource.TMSUCCESS);
>         
>         int commit = resource.prepare(identifier); 
>         if( commit == XAResource.XA_OK )
>         resource.commit(identifier, false);
>         else
>         resource.rollback(identifier);
> 
> 
> now the exception that is raised when the program commits the transaction is 
> the following:
> 
> javax.transaction.xa.XAException: org.postgresql.util.PSQLException: ERROR: 
> prepared transaction with identifier "100_BQQDAgE=_AQIDBAU=" does not exist
>     at org.postgresql.xa.PGXAConnection.commitPrepared(PGXAConnection.java:412)
>     at org.postgresql.xa.PGXAConnection.commit(PGXAConnection.java:339)
> 
> and the log of the database says:
> 
> 2007-06-16 16:05:48 CEST LOG:  execute <unnamed>: INSERT INTO ....
> 2007-06-16 16:05:48 CEST LOG:  execute <unnamed>: PREPARE 
> TRANSACTION '100_BQQDAgE=_AQIDBAU='
> 2007-06-16 16:05:48 CEST WARNING:  non c'è nessuna transazione in corso
> 2007-06-16 16:05:48 CEST LOG:  execute <unnamed>: COMMIT 
> PREPARED '100_BQQDAgE=_AQIDBAU='
> 2007-06-16 16:05:48 CEST ERROR:  prepared transaction with 
> identifier "100_BQQDAgE=_AQIDBAU=" does not exist
> 2007-06-16 16:05:48 CEST STATEMENT:  COMMIT PREPARED '100_BQQDAgE=_AQIDBAU='
> 2007-06-16 16:05:49 CEST LOG:  unexpected EOF on client connection
> 
> that is an error since it seems no one transaction has been started. However, 
> debugging the application, I've seen that the driver disables the autocommit 
> mode, thus where is the problem here?
> I'm sorry, I'm not a XA expert, just learning.

This seems to be an undesired side-effect of this patch that was 
committed in December:

> date: 2006/12/01 10:13:46;  author: jurka;  state: Exp;  lines: +19 -22
> A XAConnections default autocommit state should be true.  The
> driver was previously setting autocommit to false and assuming it
> would stay that way.  This caused two problems.  First, some
> applications expected to be able to issue local autocommit
> transactions prior to begin().  Second, some TMs (Geronimo) set the
> autocommit state to true themselves, which causes problems as the
> driver did not expect it to be changed.  This patch correctly disables
> and enables autocommit around begin, prepare, commit, and rollback
> methods.

getConnection sets autocommit to false, so even though start set it to 
true, it's reset to false in the call to getConnection. Attached patch 
fixes that by explicitly setting autocommit to the right mode in 
PGXAConnection.getConnection.

As a work-around, just call jdbcConnection.setAutoCommit(false) in your 
program before executing any queries.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

Attachment    Content-Type    Size
xa-autocommit-fix.patch

https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html-single/Transactions_JTA_Development_Guide/index.html#sect-Transactions_JTA_Programmers_Guide-Test-Externally_controlled_connections

https://docs.oracle.com/cd/E11882_01/java.112/e16548/xadistra.htm#JJDBC28878

probably we need to close the connection before commit, otherwise hold seem to hold on.
https://stackoverflow.com/questions/32664186/closing-jdbc-connections-in-jta

After commit call, we shall re-establish connection.

Needs to test with suspend.

http://amplifino.com/html/chapter6.html

we need to close connection when we are about to commit.