Feature #76

Feature #357: Enduro/X 7.0 master task

Java bindings using JNI

Added by Madars over 5 years ago. Updated over 2 years ago.

Status:ClosedStart date:04/23/2018
Priority:Normal (Code 4)Due date:
Assignee:-% Done:

100%

Category:-
Target version:-

CallingCLibraryRoutinesfromJava.pdf (523 KB) Madars, 11/13/2017 02:35 PM


Subtasks

Feature #307: error codes in string format - needed for JAVA exception buildingClosed

Feature #409: tmsrv optimizationsClosed

History

#1 Updated by Madars over 5 years ago

  • Subject changed from Java bindings to Java bindings using JNI

#9 Updated by Madars about 5 years ago

Technology sample: http://java-gnome.sourceforge.net/HACKING.html
$ git clone git://github.com/afcowie/java-gnome.git java-gnome

#20 Updated by Madars about 4 years ago

  • Tracker changed from Bug to Feature

#24 Updated by Madars about 4 years ago

- Inside JNI functions, firstly we shall set the context manually, and then use standard tp commands (not O).

- When JNI error occurs, throw runtime exception. For ATMI errors, instantiate ATMI exception. For example if context is NULL.

#25 Updated by Madars about 4 years ago

ndrxconfig.xml could look like:

        <servers>
                <server name="someprocess">
                        <srvid>1</srvid>
                        <min>1</min>
                        <max>1</max>
                        <interpret>java -jar someprocess_package.jar</interpret>
                        <sysopt>-e /tmp/RECOVER -r</sysopt>
                        <appopt>-c10</appopt>
                </server>

So "someprocess" becomes logical name of the given server. But shall give options for formatting the command. There could be option for logical name in format string and the sysopt and appopt too:

        <servers>
                <server name="someprocess">
                        <srvid>1</srvid>
                        <min>1</min>
                        <max>1</max>
                        <sysopt>-e /tmp/RECOVER -r</sysopt>
                        <appopt>-c10</appopt>
                        <interpret>java -jar ${NDRX_PROCESS}_package.jar -- ${NDRX_SYSOPT} ${NDRX_APPOPT}</interpret>
                </server>

Which would in the final make following runnable:

java -jar someprocess_package.jar -e /tmp/RECOVER -r  -c10

#26 Updated by Madars about 4 years ago

This will require some changes in housekeep processes, as real executable will be java.

#31 Updated by Madars about 4 years ago

We need following interface for service advertise:

interface Server {
   public void tpSvrInit(string [] argv);
   public void tpSvrDone();
}

interface Service {
  public void tpService(TpSvcInfo svcinfo);
}

#34 Updated by Madars almost 4 years ago

Seems for server's we need something like

    <defaults>
        <envs group="ENV1">
            <env>SOME_OTHER_ENV=HELLO WORLD</env>
        <envs>
        <envs group="ENV2">
            <env>SOME_OTHER_ENV2=HELLO WORLD</env>
        <envs>
    </defaults>
    <server name="cconfsrv">
        <srvid>1700</srvid>
        <min>1</min>
        <max>1</max>
        <sysopt>-e /tmp/CCONF</sysopt>
        <envs>
            <usegroup>ENV1</usegroup>
            <usegroup>ENV2</usegroup>
            <env>JAVA_OPTS=-some -flags ${SOME_OTHER_ENV}</env>
            <env>JAVA_CLASSPATH=-some2 -flags2</env>
        ...
        </envs>
    </server>
..
        <clients>
        <envs group="ENV1">
            <env>SOME_OTHER_ENV=HELLO WORLD</env>
        </envs>
                <client cmdline="/opt/bin/my_binary -t ${NDRX_CLTTAG} -s ${NDRX_CLTSUBSECT}">
            <env>
                <usegroup>ENV1</usegroup>
                <usegroup>ENV2</usegroup>
                <env>JAVA_OPTS=-some -flags ${SOME_OTHER_ENV}</env>
                <env>JAVA_CLASSPATH=-some2 -flags2</env>
            <env/>
                        <exec tag="BINARY1" subsect="1" autostart="Y" />
                        <exec tag="BINARY2" subsect="2" autostart="Y" />
                </client>
                <client cmdline="firefox"> <exec tag="FF"/> </client>
                <client cmdline="/tmp/test.bin" env="/tmp/yopt.env"> <exec tag="TT" log="/tmp/test.out"/> </client>
                <client cmdline="mongod -dbpath /home/mvitolin/tmp/data/db" >
                        <exec tag="mongo" autostart="Y"/>
                </client>
        </clients>

1. So we need to keep in defaults hash list of envs by set id.
2. The server keeps the linked list of envs
3. Server keeps linked list of pointer to linked resolved sets.

The same approach goes with client admin.

#35 Updated by Madars almost 4 years ago

we shall provide exjc (Enduro/X Java Compiler) so that we could embed the JVM and compile in the jar files, which at the process boot we could extract to disk (/dev/shm preffered) load the jars, and start the binary.

This way we can get the normal executable, and customer have zero config requirements for the binary configuration.

#36 Updated by Madars almost 4 years ago

Alos we might want to export server id too to env variables. So that any startup scripts might want to adjust their tcp port numbers or any other resources according to the server id

#37 Updated by Madars almost 4 years ago

Also needs to consider what happens in cases when atmi server runs in background and the atmi server parent went off.

#40 Updated by Madars almost 4 years ago

seems exembedfile adds some un needed zero at end of the array...

#41 Updated by Madars almost 4 years ago

Seems like "setContextClassLoader" is needed for exjld for runner context to set the class loader.

#42 Updated by Madars almost 4 years ago

#43 Updated by Madars almost 4 years ago

  • Parent task set to #328

#44 Updated by Madars almost 4 years ago

  • Status changed from New to In Progress

#50 Updated by Madars over 3 years ago

1) For java application XA driver we need a driver to which from java side we can assign database handler for given XATMI conext. I.e. for example AtmiCtx.set(db XAResource),

The C side should have common driver like libexrmjdrv.so (java resource manager) for which there should be a thread local storage assigned AtmiContext with XAResource inside, and callbacks from call must be redirected back this context. EXRMJDRV shall be provided with some global reference to java env.

2) For tmsrv we need shared library which would start the java inside similarly like exjld does. Then it should create a XAResource and assign with ATMI Context and should work like library in step 1. For example loaders:
libndrxxajd.so, libndrxxajs.so (these should load and create JVM too), and backs: libexrmjdrv.so (java resource manager).

Needs to think how multi-threading will be done here. Each thread should have it's own AtmiContext. in case of 1) context can be set by java tpbegin from java caller...
for tmsrv which is multi-threaded, for each thread at init we need to create new AtmiContext and set the XAResource accordingly. There for EXRMJDRV there could be a thread local storage which if found to be NULL, will create new AtmiCtx. and then further will operate with it.

3) For normal apps which will use tpcall interface, we will need some kind of dummy xa driver (e.g. libndrxxadumd.so, libndrxxadums.so) and also dummy backend libexrmdumdrv.so which will hold just dummy xa functions.

#51 Updated by Madars over 3 years ago

  • Parent task changed from #328 to #357

#54 Updated by Madars over 3 years ago

Java finalizers and try-with-resources Statement https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html

might be used to deallocate C resources.

#56 Updated by Madars about 3 years ago

#
# Service type maps for jolt
#
[@joltmap]
service DEBIT = {"req":"UBF", "rsp":"VIEW:JVIEW2"}
service LASTAUTH = {"req":"UBF", "rsp":"VIEW:JVIEW2"}

#57 Updated by Madars about 3 years ago

Probably JOLT we shall build on rest-in api.

There shall be separate link URL transaction management, i.e. begin, commit.
The transaction ID shall be transferred in HTTP/S header with the original request buffer.
Currently no AUTH will be supported.
The service repo shall be some ini file accessible from java.

This way we could keep the max compatibility with Tuxedo JOLT...

https://www.mkyong.com/java/how-to-send-http-request-getpost-in-java/

#59 Updated by Madars about 3 years ago

java bin [have xa_switch] ---> load the driver which would use
per thread global variable has Java XA Switch.
- load the dll which would ndrxj_javabin_get_switch(), from the libexjava.so
-

if we want tmsrv to work with java code, we need to create a shared library which would:
- start VM
- load the "libexjava.so", call ndrxj_tmsrv_get_switch(), tpopen shall load the jars accordingly, set the Java Env variable (global) and check the AtmiCtx is there one for thread or not saved within libexjava.so. This would also load NDRX_XA_RMLIB jar jdbc library.

here question is about visibility, are the symbols global when loaded by jni.

#60 Updated by Madars about 3 years ago

https://www.postgresql.org/message-id/4674E56C.8060802@enterprisedb.com

Re: a simple example of XA (not working)
From:    Heikki Linnakangas <heikki(at)enterprisedb(dot)com>
To:    Luca Ferrari <fluca1978(at)infinito(dot)it>
Cc:    pgsql-jdbc(at)postgresql(dot)org
Subject:    Re: a simple example of XA (not working)
Date:    2007-06-17 07:40:28
Message-ID:    4674E56C.8060802@enterprisedb.com
Views:    Raw Message | Whole Thread | Download mbox
Thread:    
Lists:    pgsql-jdbc
Luca Ferrari wrote:
> Hi all,
> I'm trying to build a simple "manual" example of the use of the XA extension, 
> the following is the code:
> 
>     PGXADataSource dataSource = new PGXADataSource();
>     dataSource.setDatabaseName("hrpm");        
>     dataSource.setUser("luca");                
>     dataSource.setServerName("localhost");        
>     
>     XAConnection connection = dataSource.getXAConnection();
>     System.out.println("Connesso!");
>     XAResource resource = connection.getXAResource();
>     MyXid identifier = new MyXid();
>     identifier.setBranchQualifier(new byte[] {0x01, 0x02, 0x03, 0x04, 0x05});
>     identifier.setGlobalTransactionId(new byte[] {0x05, 0x04, 0x03, 0x02, 0x01});
>     identifier.setFormatId(100);
>     
>     try{
>         resource.start(identifier, XAResource.TMNOFLAGS);
>         
>         
>         Connection jdbcConnection = connection.getConnection();
>         Statement statement = jdbcConnection.createStatement();
>         String sql = "INSERT INTO ...";
>         int inserted = statement.executeUpdate(sql);
> 
>         resource.end(identifier, XAResource.TMSUCCESS);
>         
>         int commit = resource.prepare(identifier); 
>         if( commit == XAResource.XA_OK )
>         resource.commit(identifier, false);
>         else
>         resource.rollback(identifier);
> 
> 
> now the exception that is raised when the program commits the transaction is 
> the following:
> 
> javax.transaction.xa.XAException: org.postgresql.util.PSQLException: ERROR: 
> prepared transaction with identifier "100_BQQDAgE=_AQIDBAU=" does not exist
>     at org.postgresql.xa.PGXAConnection.commitPrepared(PGXAConnection.java:412)
>     at org.postgresql.xa.PGXAConnection.commit(PGXAConnection.java:339)
> 
> and the log of the database says:
> 
> 2007-06-16 16:05:48 CEST LOG:  execute <unnamed>: INSERT INTO ....
> 2007-06-16 16:05:48 CEST LOG:  execute <unnamed>: PREPARE 
> TRANSACTION '100_BQQDAgE=_AQIDBAU='
> 2007-06-16 16:05:48 CEST WARNING:  non c'รจ nessuna transazione in corso
> 2007-06-16 16:05:48 CEST LOG:  execute <unnamed>: COMMIT 
> PREPARED '100_BQQDAgE=_AQIDBAU='
> 2007-06-16 16:05:48 CEST ERROR:  prepared transaction with 
> identifier "100_BQQDAgE=_AQIDBAU=" does not exist
> 2007-06-16 16:05:48 CEST STATEMENT:  COMMIT PREPARED '100_BQQDAgE=_AQIDBAU='
> 2007-06-16 16:05:49 CEST LOG:  unexpected EOF on client connection
> 
> that is an error since it seems no one transaction has been started. However, 
> debugging the application, I've seen that the driver disables the autocommit 
> mode, thus where is the problem here?
> I'm sorry, I'm not a XA expert, just learning.

This seems to be an undesired side-effect of this patch that was 
committed in December:

> date: 2006/12/01 10:13:46;  author: jurka;  state: Exp;  lines: +19 -22
> A XAConnections default autocommit state should be true.  The
> driver was previously setting autocommit to false and assuming it
> would stay that way.  This caused two problems.  First, some
> applications expected to be able to issue local autocommit
> transactions prior to begin().  Second, some TMs (Geronimo) set the
> autocommit state to true themselves, which causes problems as the
> driver did not expect it to be changed.  This patch correctly disables
> and enables autocommit around begin, prepare, commit, and rollback
> methods.

getConnection sets autocommit to false, so even though start set it to 
true, it's reset to false in the call to getConnection. Attached patch 
fixes that by explicitly setting autocommit to the right mode in 
PGXAConnection.getConnection.

As a work-around, just call jdbcConnection.setAutoCommit(false) in your 
program before executing any queries.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

Attachment    Content-Type    Size
xa-autocommit-fix.patch

#67 Updated by Madars about 3 years ago

probably we need to close the connection before commit, otherwise hold seem to hold on.
https://stackoverflow.com/questions/32664186/closing-jdbc-connections-in-jta

After commit call, we shall re-establish connection.

Needs to test with suspend.

#68 Updated by Madars about 3 years ago

http://amplifino.com/html/chapter6.html

we need to close connection when we are about to commit.

#69 Updated by Madars over 2 years ago

  • Status changed from In Progress to Resolved

#70 Updated by Madars over 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF