endurox-connect:v8.0.x:manuals:restincl.8

restincl

Name

restincl — Enduro/X HTTP REST incoming call handler (HTTP Server, XATMI Client)

Synopsis

restincl

DESCRIPTION

This is XATMI client which accepts HTTP/HTTPS REST requests, calls the XATMI server specified in configuration. The binary works with go routines (very cheap threads). But the number of concurrent invocations to XATMI sub-system is limited by workers initialization configuration flag. In case if all workers are busy, the incoming thread will wait for the XATMI worker to become free.

The configuration is defined in mixed for with INI and JSON formats. The configuration file contains keys which value can be set as multi-line JSON. Configuration file contains defaults key which defines the default parameters for the route. Then when new route is added, firstly it is initialized from defaults and then additionally it is overridden by route it self.

Incoming HTTP REST server (XATMI client), supports multiple buffer formats and different error handling schemes. Buffer formats can be:

json2ubf - JSON Converted to UBF XATMI buffer type;
json2view - JSON Converted to VIEW XATMI buffer type;
json - JSON text directly set in UBF JSON buffer type;
text - XATMI Typed String buffer format. Incoming text is copied to String buffer;
raw - XATMI Carray buffer format; The incoming data is copied to Carray buffer type;
ext - UBF buffer is loaded with request body, headers and POST/PUT form values;

The error handling can be done in following ways:

http - mapped HTTP error codes are being used for indicating XATMI failure codes;
json - XATMI error codes are projected to JSON text root level two fields: code and message;
text - The error code is printed in output response buffer text by using format specified in config;
json2ubf - The error code is set in Enduro/X standard buffer fields and later Converted to JSON buffer.
json2view - The error is converted to code/message fields specified for the view errors. If views does not hold response field, the separate view name can be configured in order to provide responses in error cases.

' ext - Special services and UBF buffers are used for handling the error cases.

The operation modes of the REST interface can by:

Synchronous - with http call XATMI services are invoked in Synchronous mode, waiting for the sub-system answer or timeout that provides the corresponding response error code.
Asynchronous - invocation does the asynchronous call to XATMI service. The answer to http rest call is return of formatted error code and message. Indicating success or fail of async service invocation.

The typical restincl configuration looks like:

[@restin]
port=8080
ip=0.0.0.0
#
# Defaults: conv=json2ubf
# async - call service in async way, if submitted ok, just reply back with ok
defaults={"errors":"http"
        ,"reqlogsvc": "GETFILE"
        ,"noreqfilersp": true
        }

# Service 1
/svc1={"svc":"DATASV1", "notime":false, "conv":"json2ubf", "errors":"json", "async":false
        ,"errfmt_json_code":"\"error_code1\":%d"
        ,"errfmt_json_msg":"\"error_message1\":\"%s\""}

# Echo service:
/echo={"notime":false, "conv":"json2ubf", "errors":"json", "async":false, "echo":true}

HTTPS mode is also supported. In this case configuration file contains ssl certificate file and key file. The format of these files are in x509 format.

It is possible to configure per request logging files. Then the parameter reqlogsvc must be set which indicates the service name which to invocate to acquire the request logging file name.

In case if multiple REST-IN handlers are needed with different binding IP/PORTS/HTTPS settings. it is possible to use NDRX_CCTAG setting (environment or in client process monitor configuration).

Next sections will describe supported buffer formats and it’s handling in the Enduro/X.

Buffer conversion modes

Following sub-chapters describes the modes, how buffers are converted from network message to Enduro/X format and vice-versa.

Conversion buffer type: ext - External handling

External handling mode, incoming message is loaded into generic UBF fields. Field names are following:

EX_IF_URL - URL called;
EX_IF_REQDATA - is loaded with HTTP Message body, byte array;
EX_IF_REQHN - request header names. Multi-occurrence header values are supported;
EX_IF_REQHV - request header values, same occurrence with names;
EX_IF_REQCN - request cookie names;
EX_IF_REQCV - request cookie value, same occurrence with names;
EX_IF_REQFORMN - form submit field names;
EX_IF_REQFORMV - form submit field values;
EX_IF_REQQUERYN - URL Query parameter names names;
EX_IF_REQQUERYV - URL Query parameter names values;

If fields are prepared OK, list of comma separated services found in finman are executed with UBF buffer. This can be used to build up the target request buffer. In case if any service fails from mandatory list, it is treated as general error and then incoming error service chain finerr is executed (as optional). When generating response EX_NETRCODE field with expected http code is read. If field is missing in buffers, then restincl will respond with error 500, without any body. In case of error and if buffer contains EX_NETRCODE, then system will attempt to return the EX_IF_RSPDATA.

If there was no error, after all incoming filter chains done, target service is executed. Once that is done, the response chain is activated. If there was no error during target service call, the foutman mandatory chain is executed. If it succeed, then optional chain foutopt chain is executed.

In case if destination service was called in synchronous mode, the user return value tpurcode(3) from tpcall(3) is added to UBF buffer field EX_IF_TPURCODE.

If there was error during the target service execution or mandatory out filter chain execution (foutman), the fouterr service chain is executed. If EX_NETRCODE is found in the body, then UBF content is used for response generation and http status is set to EX_NETRCODE. If in case of error EX_NETRCODE is not found, the response is set to 500 with out any content.

The response is generated in following way:

http body is loaded from EX_IF_RSPDATA;
http headers are loaded from EX_IF_RSPHN and EX_IF_RSPHV
http response cookies are loaded from EX_IF_RSPCN, EX_IF_RSPCV, EX_IF_RSPCPATH, EX_IF_RSPCDOMAIN, EX_IF_RSPCEXPIRES, EX_IF_RSPCMAXAGE, EX_IF_RSPCSECURE, EX_IF_RSPCHTTPONLY
http content type is according to ubftab.EX_IF_RSPHN (Content-Type) value found in EX_IF_RSPHV. If Conent-Type is not provided in response, text/plain is used.

NOTE: The conv mode works only with ext error handling mode. And error handling mode ext works only with buffer conversion mode ext. See more details bellow for ext error handler mode.

For response EX_IF_RSPHN and EX_IF_RSPHV header field pairs generally supports multi-occurrence headers, exception is for Content-Type: for which if multi-occurrence are used, only last value will be present in response (or field will be set automatically set depending to the scenario).

File Upload

restincl supports file upload to temporary folder TEMP_DIR set by tempdir flag. File upload works only in ext buffer mode. Upload mode is enabled by setting fileupload parameter to true.

The upload process performs following steps:

Processes finman, finopt filters (with parsed headers)
Downloads the uploaded files to temporary directory to temporary file names. Multiple files may be uploaded. Attributes of each of the files are loaded into UBF buffer occurrences.
Target service is invoked.
When response is received from service, temporary files are deleted.
If for given file ordered file UBF field occurrence EX_IF_RSPFILEACTION value contains K (Keep flag), the file is not deleted. If EX_IF_RSPFILEACTION occurrence is not present or set to other value (including value is D - delete), file is deleted as in 4.

Following list of fields are added of the uploaded files:

EX_IF_REQFILENAME - Logical file name which user selected in HTML form.
EX_IF_REQFILEFORM - Form field name e.g. files.
EX_IF_REQFILEMIME - Detected file MIME code.
EX_IF_REQFILEDISK - Full path to temporary file on disk.

Following HTML form may be used for data upload:

<html>
    <body>
        <form enctype="multipart/form-data" action="http://localhost:8080/ext_fileupload" method="post">
            <input type="file" name="files" multiple />
            <input type="submit" value="upload" />
        </form>
    </body>
</html>

Upload call of 3 files to service would look like:

EX_NREQLOGFILE  /home/user1/endurox-connect/tests/01_restin/runtime/log/TRACE_1606490385644802141
EX_IF_URL       /ext_fileupload
EX_IF_METHOD    POST
EX_IF_REQFILEDISK       /home/user1/endurox-connect/tests/01_restin/runtime/tmp/@restin-162189307
EX_IF_REQFILEDISK       /home/user1/endurox-connect/tests/01_restin/runtime/tmp/@restin-410813726
EX_IF_REQFILEDISK       /home/user1/endurox-connect/tests/01_restin/runtime/tmp/@restin-398043621
EX_IF_REQFILENAME       312329.image0.jpg
EX_IF_REQFILENAME       312332.image0.jpg
EX_IF_REQFILENAME       consoleText (1).txt
EX_IF_REQFILEMIME       image/jpeg
EX_IF_REQFILEMIME       image/jpeg
EX_IF_REQFILEMIME       text/plain
EX_IF_REQFILEFORM       files
EX_IF_REQFILEFORM       files
EX_IF_REQFILEFORM       files

If wanting to keep third file in temp directory, following reply must be given:

EX_IF_RSPFILEACTION     D
EX_IF_RSPFILEACTION     D
EX_IF_RSPFILEACTION     K

Additional caveats:

If form contains non file-upload type fields (e.g. text), for each of the values temporary file will be created, content is written to file.
If form is submitted empty (with out upload files) single temporary file will be created for which EX_IF_REQFILENAME is set to empty "" string, but all other fields are filled.
Temporary file name strategy may change in future. Currently it is "@restincl-<CCTAG><RAND_STRING>". If CCTAG is not set, then it would look like "@restincl-<RAND_STRING>".
Files are downloaded after the incoming filter. Thus during the filter execution files are not available for processing.

Conversion buffer type: json2ubf - JSON converted to UBF message handling

With JSON2UBF mode, it is expected that configured web service will receive JSON document formatted in one level, where basically data is encode in key:value format. Array’s types is supported. The array elements are loaded into UBF buffer field occurrences. The BLOB elements are encoded as Base64 data and are loaded into UBF buffer’s BFLD_CARRAY typed fields in decoded (binary form). If target field is not BFLD_CARRAY, then it is treated as string data and loaded into field via conversion functions.

The JSON2UBF POST REST data of service invocation of would look like:

{
        "T_CHAR_FLD":"A",
        "T_SHORT_FLD":123,
        "T_LONG_FLD":444444444,
        "T_FLOAT_FLD":1.33,
        "T_DOUBLE_FLD":4444.3333,
        "T_STRING_FLD":["HELLO", "WORLD"],
        "T_CARRAY_FLD":"SGVsbG8="
}

That would be converted into following UBF buffer:

T_CHAR_FLD      A
T_SHORT_FLD     123
T_LONG_FLD      444444444
T_FLOAT_FLD     1.33
T_DOUBLE_FLD    4444.3333
T_STRING_FLD    HELLO
T_STRING_FLD    WORLD
T_CARRAY_FLD    Hello

When response is generated for caller, the UBF buffer coming back from Enduro/X IPC would be in the same JSON format as in request - single level JSON document with arrays if necessary i.e. have multiple occurrences for field.

The restincl for incoming data does not check the MIME type, but in response MIME type will be set to: text/plain.

Conversion buffer type: json2view - JSON converted to VIEW message handling

With JSON2VIEW mode, it is expected that configured web service will receive JSON document formatted in two levels, outer level is object with view name (which is configured in Enduro/X environment VIEWDIR and VIEWFILES. The second level of the JSON document basically is fields encoded in key:value format. Array’s types is supported. The array elements are loaded into UBF buffer field occurrences. The BLOB elements are encoded as Base64 data and are loaded into VIEW buffer’s carray fields typed fields in decoded (binary form). The standard UBF data conversation functions (CBchg(3)) are used for data converting.

The JSON2VIEW POST REST data of service invocation of would look like:

{
    "MYVIEW":{
        "char_fld":"a",
        "short_fld":123,
        "long_fld":444444444,
        "float_fld":1.33,
        "double_fld":4444.3333,
        "string_fld":["hello", "world"],
        "carray_fld":"SGVsbG8="
    }
}

That would be converted into following VIEW buffer:

VIEW MYVIEW
#type    cname      fbname              count   flag    size    null

char    char_fld    -                   1       -       -       -
short   short_fld   -                   1       -       -       -
long    long_fld    -                   1       -       -       -
float   float_fld   -                   1       -       -       -
double  double_fld  -                   1       -       -       -
string  string_fld  -                   2       -       20      -
carray  carray_fld  -                   1       -       25      -

# optional response fields, used if configured so:

string  rspmessage  -                   1       -       255      -
short  rspcode     -                    1       -       -        -
END

When response is generated for caller, the VIEW buffer coming back from Enduro/X IPC would be in the same JSON format as in request - two level JSON document with arrays if necessary i.e. have multiple occurrences for field.

The restincl for incoming data does not check the MIME type, but in response MIME type will be set to: text/plain.

For error handling if configured so (using json2view errors), the restincl can install the ATMI error code and message in the VIEW before converting to JSON, Thus rspcode and rspmessage can be produced back in the JSON with corresponding content. In case of wrong configuration (errfmt_view_rsp does not contain response fields) or errfmt_view_rsp is not set at all, but error mechanism is json2view and response view does not have response fields, the restincl will generate {} - empty JSON object error. The caller shall assume this as format error or timeout (because there are no knowledge to caller of what have happened at the service).

Conversion buffer type: json - Direct JSON buffer

In this case JSON text is received in POST message and buffer is loaded into XATMI buffer type JSON. Buffer is sent to target service. It is expected that target service will respond with valid JSON text back which is returned in HTTP response. In this case too, the response type is set to text/plain.

Conversion buffer type: text - Arbitrary text message

In this case arbitrary string is received from POST message. The string is loaded into Enduro/X buffer type STRING. And with this buffer the message is delivered to destination service (svc field from route configuration). The response also is generated as pure string, with MIME set to text/plain.

Conversion buffer type: raw - BLOB message

In this case arbitrary binary (BLOB) data is received from POST message. The BLOB is loaded inti CARRAY typed buffer and destination service is invoked with this buffer. If service invocation is success, then the received BLOB message from XATMI sub-system is returned to caller. In this case response will be generated as application/octet-stream.

Error handling

restincl supports different error handling strategies for different URL setting/targets. Following sub-sections describes each handling mode.

Error handling type: ext - external service mode

As described in buffer ext buffer conversion mode, this error handling mechanism consists of following key features:

If incoming mandatory filters or buffer preparation fails, then finerr services is called. If after service calls buffer contains EX_NETRCODE (http code), then UBF buffer contents are used for generating response. Otherwise http error code 500 is returned with out any body.
If incoming filters where succeed, but target service or outgoing mandatory filters did fail, the fouterr chain is executed. And in the same way if EX_NETRCODE in final stages are present in buffer, then value from this is used for http code and buffer contents are used for sending back responses. If return code is not present, then error code 500 is returned, with any further processing.

Ext mode of error handling includes additional diagnostics information for the service errors. When incoming processing fails (including target service), following fields are add to the buffer and are available for filters:

EX_IF_ECODE XATMI error code.
EX_IF_EMSG XATMI error message.
EX_IF_ERRSRC Indicate the response source. F means incoming filter (finman) chain, R means internal restincl handling (i.e. http handler, buffers), S error raised by service call.

Error handling type: http - return error codes in HTTP protocol

With this error handling method, the error codes are returned within HTTP protocol. The error code can be mapped from XATMI subsystem to HTTP codes by using errors_fmt_http_map parameter in service or default parameter block. The default mapping which is set if errors_fmt_http_map is not present, is following:

atmi.TPMINVAL (0) = http.StatusOK (200)
atmi.TPEABORT (1) = http.StatusInternalServerError (500)
atmi.TPEBADDESC (2) = http.StatusBadRequest (400)
atmi.TPEBLOCK (3) = http.StatusInternalServerError (500)
atmi.TPEINVAL (4) = http.StatusBadRequest (400)
atmi.TPELIMIT(5) = http.StatusRequestEntityTooLarge (413)
atmi.TPENOENT (6) = http.StatusNotFound (404)
atmi.TPEOS (7) = http.StatusInternalServerError (500)
atmi.TPEPERM (8) = http.StatusUnauthorized (401)
atmi.TPEPROTO (9) = http.StatusBadRequest (400)
atmi.TPESVCERR (10) = http.StatusBadGateway (502)
atmi.TPESVCFAIL (11) = http.StatusInternalServerError (500)
atmi.TPESYSTEM (12) = http.StatusInternalServerError (500)
atmi.TPETIME (13) = http.StatusGatewayTimeout (504)
atmi.TPETRAN (14) = http.StatusInternalServerError (500)
atmi.TPERMERR (16) = http.StatusInternalServerError (500)
atmi.TPEITYPE (17) = http.StatusInternalServerError (500)
atmi.TPEOTYPE (18) = http.StatusInternalServerError (500)
atmi.TPERELEASE (19) = http.StatusInternalServerError (500)
atmi.TPEHAZARD (20) = http.StatusInternalServerError (500)
atmi.TPEHEURISTIC (21) = http.StatusInternalServerError (500)
atmi.TPEEVENT (22) = http.StatusInternalServerError (500)
atmi.TPEMATCH (23) = http.StatusInternalServerError (500)
atmi.TPEDIAGNOSTIC (24) = http.StatusInternalServerError (500)
atmi.TPEMIB (25) = http.StatusInternalServerError (500)
Anything else (\*) = http.StatusInternalServerError (500)

Error handling type: json - response code embedded JSON response message

This is suitable for json and json2ubf buffer types. On response the JSON block is appended at then end with two fields. The fields are set with format string %s for error message in errfmt_json_msg parameter, for example "error_message":"%s". The error code format is set with %d in errfmt_json_code parameter, for example: "error_code":%d. The error codes are XATMI standard defined in xatmi.h. For example if calling JSON service and call times out, then response will be look like:

{"error_code":13,"error_message":"13:TPETIME (last error 13: ndrx_mq_receive failed: Connection timed out)"}

Error handling type: json2ubf - UBF format field in JSON response message

With this error handling mechanisms, which is suitable for JSON2UBF buffer conversion mode, the error message is loaded into top level JSON field EX_IF_ECODE and error message is loaded into EX_IF_EMSG field. This is suitable in case if using restout on the other Enduor/X server to bridge the servers using HTTP/Rest method.

Error handling type: text - Free format text error code and message

The error code and message is generated in free form text which is provided by errfmt_text service parameter block field. The first parameter in format string must be %d - for XATMI error, and next parameter in format string must be %s- for error message. For example errfmt_text could be set to %d: %s.

Error codes and it’s meaning

No matter of which error handling mechanism is selected http/json/json2ubf/text, the list of Enduro/X error codes is following:

0 - Succeed (TPMINVAL)
1 - Transaction abort (TPEABORT)
2 - Bad call descriptor (TPEBADDESC)
3 - Blocking condition found (TPEBLOCK)
4 - Invalid parameters passed to function or service (TPEINVAL)
5 - System limit (TPELIMIT)
6 - Service not found (TPENOENT)
7 - Operating system error (TPEOS)
8 - No permissions (TPEPERM)
9 - Protocol error, service invoked in invalid order (TPEPROTO)
10 - Service hard failure, crashed (TPESVCERR)
11 - Service soft failure, returning error (TPESVCFAIL)
12 - System error (TPESYSTEM)
13 - Time out condition (TPETIME)
14 - Transaction error (TPETRAN)
16 - Resource manager error (TPERMERR)
17 - Invalid input data type to service (TPEITYPE)
18 - Invalid data type received from service (TPEOTYPE)
19 - Invalid program release (TPERELEASE)
20 - Transaction partially completed (TPEHAZARD)
21 - Transaction partially completed (TPEHEURISTIC)
22 - Event occurred, only for conversational messages, RFU (TPEEVENT)
23 - Identifier not matches (TPEMATCH)
24 - Diagnostic info provided, RFU (TPEDIAGNOSTIC)
25 - RFU (TPEMIB)

CONFIGURATION

port = PORT_NUMBER: Port number to listen on which http server will listen on. Mandatory option.
ip = IP_ADDRESS: Ip address on which http server is listening for incoming connections. Default 0.0.0.0.
workers = NUMBER_OF_XATMI_SESSIONS: Number XATMI sessions. These sessions are use for serving the incoming calls - i.e. doing the calls to XATMI sub-system. If the number is less than incoming calls, the calls will be suspended while there will be no XATMI session free. Once it is made free, then call will be served (i.e. called corresponding XATMI counterpart). The default value for parameter is 10. Note that every new HTTP request to the restincl is processed in new lightweight thread (go-routine), where each of such thread tries to get (or blocks on getting) a XATMI worker. This means, that handling large amount of parallel requests, does not make any significant pressure on TCP backlog, even if the system is loaded, i.e. requests gets queued within the process instead of the backlog.
gencore = GENERATE_CORE_FILE: If set to 1, then in case of segmentation fault, the core dump will be generated instead of Golang default signal handler which just prints some info in stderr. The default value is 0 which means use default Golang panic handling mechanisms.
tls_enable = ENABLE_HTTPS: With this setting set to 1, HTTPS will be enabled. Default value is 0. To complete the HTTPS activation, configuration flags tls_cert_file and tls_key_file must be set too. Otherwise program will run in HTTP mode.
defaults = 'SERVICE_CONFIGURATION_JSON*: This is JSON string (can be multiline), setting the defaults for the services. It is basically a service descriptor which is used as base configuration for services. Once the service is being setup, firstly it uses this defaults config block and then overrides it by additional flags in service definition block. The details within the JSON are described in bellow section SERVICE CONFIGURATION.
/some/service/url = 'SERVICE_CONFIGURATION_JSON*: This is the same configuration as for default, but describes the service route. The REST-IN process might have as many as needed the service mapping routes.

SERVICE CONFIGURATION

svc = MAPPED_XATMI_SERVICE_NAME: This is the name of the target XATMI service which needs to be invoked when POST message is received. The invocation might be synchronous, in which case caller will wait for service to complete or receive XATMI time-out. If service completes, then response is generated back to caller in configured format (conv parameter). If service fails or times-out. The error is returned. Depending on error handling mode (errors parameter), the data buffer can be returned too, including the erroneous buffer data, because XATMI services at application level errors TPESVCFAIL returns the data buffer from service even the error occurred. The data is returned with error in case of following error handling methods: http, json, json2ubf.
errors = ERROR_HANDLING: The parameter can be set to following values http, json, json2ubf and text. See the working modes of each of the modes in above text. The default value for this parameter is json.
notime = NO_TIMEOUT: Set value to true if service call shall be run with out XATMI sub-system tpcall() timeout value. Default is 0, meaning that standard timeout settings applies on the destination service call.
errfmt_text = TEXT_BUFFER_ERROR_FORMAT_STRING: Format string for buffer to return in case if destination service invocation fails. Format text will be invoked with "%d" representing the error code and then with "%s" representing the error message. This error string is used in case if errors parameter is set to text. The format string will be used only in case of tpcall(3) error. If no error occurs, then service answer message is returned. If case if asynchronous invocation is done to service (async param set to true), the format string will be used, the status code will be returned, including 0 if tpacall(3) did succeed.
errfmt_json_msg = JSON_BUFFER_ERROR_FORMAT_STRING_MESSAGE: JSON error message field format string. Normally this would be set to JSON field like syntax. This field is used in case if errors parameter is set to json. The field is always present in case of unsuccessful invocation of XATMI service. If service parameter errfmt_json_onsucc is set to true, then field is present in case of successful XATMI service invocation too. The field is present also in case of successful async service invocation .e. if async is set to true. The default value for the field is "error_message":"%s".
errfmt_json_code = JSON_BUFFER_ERROR_FORMAT_STRING_CODE: JSON message field format for presenting XATMI error code occurred while doing the service invocation. This field is together with errfmt_json_msg field. The format string for the field is %d, with meaning of XATMI error code. The default value for this parameter is "error_code":%d.
errfmt_json_onsucc = ADD_JSON_ERROR_FIELDS_ON_SUCCEESS: If set to true, in case of successful synchronous service invocation, then error fields defined in errfmt_json_msg and errfmt_json_code will be added to JSON message ending.
errfmt_view_code = ERRFMT_VIEW_CODE: Field name into which store the response XATMI error code in case of json2view errors. Parameter is mandatory for json2view error handling mechanism.
errfmt_view_msg = ERRFMT_VIEW_MSG: Field name into which store the response XATMI error message in case of json2view errors. Parameter is mandatory for json2view error handling mechanism.
errfmt_view_onsucc = ERRFMT_VIEW_ONSUCC: If set to true, the restincl will try to install the errfmt_view_code and errfmt_view_msg errors fields in the response buffer in case if call is success, if fields are not found in the response buffer, the condition is ignored and pure response buffer (converted to JSON) is returned. Default is true.
view_notnull = VIEW_NOTNULL: Do not convert NULL fields (according to view) to the outgoing JSON. For more infos see, the tpjsontoview(3) or CBvget(3) man pages. Basically the functionality will omit the matched NULL fields in the VIEW. In case of arrays, dropped will be only last NULL fields, if there is some valid value in the middle and start is filled with NULLs, then start NULLs will be converted to JSON anyway. Default is false.
errfmt_view_rsp = ERRFMT_VIEW_RSP: View name to be used when sending the errors json2view errors. In case of 1) if there is error and response buffer does not have error fields, view errfmt_view_rsp will be instantiated and fields will set with the error code. If errfmt_view_rsp is invalid, then response will contain {} - empty JSON object. In that case the caller should threat the error as format error or timeout. 2) In case of async is set to true and asyncecho is set to false, in this case errfmt_view_rsp is mandatory. 3) If errfmt_view_rsp_first is set, then errfmt_view_rsp must be set too, as in this case error will be charged into configured object.
errfmt_view_rsp_first = ERRFMT_VIEW_RSP_FIRST: If set to true, the system will instantiated errfmt_view_rsp view and set error there instead of response view.
async = 'DO_ASYNC_SERVICE_INVOCATION*: Set to true if target service should be invoked asynchronously with TPNOREPLY flag set, meaning that message is enqueued to service with out waiting for response. Regardless of error reporting method set in errors parameter, the status of the invocation is provided back to caller, by using the configured method. The default value for parameter is false.
asyncecho = ECHO_BACK_BUFFER_DURING_ASYNC_INVOCATION: Set to true if on successful invocation, the request buffer shall be echoed back to caller. In this case if service invocation was successful, the request buffer is sent back to HTTP caller with no error fields set. In case if failure happened, then corresponding error is reported back to caller with configured errors mechanism. The default value for parameter is false.
conv = BUFFER_CONVERTION_TYPE: Request/response buffer conversion method. Available constants json2ubf, json, text and raw. Buffer methods are described above in manpage. Shortly: json2ubf - converts incoming JSON formatted document (with one level key:value (including arrays)) to Enduro/X UBF buffer format. json makes the JSON XATMI data buffer, text makes STRING XATMI data buffer. The raw method load the data into CARRAY XATMI buffer. The default value for this parameter is json2ubf. If static file serving is required then conv type shall be set to "static". For static serving parameter
reqlogsvc = REQUEST_LOGGING_SERVICE: Request logging service. If the service name is set and buffer conversion type is set to json2ubf, then when request is received and is converted to UBF XATMI buffer type, the service is invoked by tplogsetreqfile(3) XATMI function. If service returns EX_NREQLOGFILE UBF field, then current request logging is switched to given request file. When service invocation is completed and data is returned back to caller, the restincl will close the request logging by invoking tplogclosereqfile(3). This mechanism is useful if per session or some other correlator (for example banking card number (PAN) hash is used for separating the logs) based logging is needed. Thus to employ the functionality fully, the target services should use the request logging too. The default value for this parameter is empty - not set.
errors_fmt_http_map = HTTP_ERROR_CODES_MAPPING: Error mapping between XATMI error code and HTTP. This is optional remap string which will override the default mode described above. The parameter is effective only in case if errors parameter is set to http. The syntax for the string is following:
staticdir = STATIC_DIR_OF_FILES: In case if conv is set to static, then this parameter denotes the folder where static contents is kept. Note that if file name at the route does not match, then file server will attempt to to upload index.html. If index.html is not available, then directory listing will be provided back to caller.
parseheaders = PARSE_HEADERS: If set to true, for conv modes json2ubf and ext HTTP headers will be parsed and will be provided back in response. Fields used header transporting are following: Request headers are loaded into: EX_IF_REQHN (request header name) and EX_IF_REQHV (request header value) UBF field occurrences and on response, headers are read from EX_IF_RSPHN (response header name) and EX_IF_RSPHV (response header value) UBF field occurrences. Default setting is false.

"errors_fmt_http_map":"<ATMI_ERROR_CODE_1>:<HTTP_STATUS_CODE_1>,...,
<ATMI_ERROR_CODE_N>:<HTTP_STATUS_CODE_N>,*:<HTTP_STATUS_CODE_FOR_ANY_OTHER>"

for example:

"errors_fmt_http_map":"13:404,*:200"

means that XATMI error code 13 (time-out) will be mapped to HTTP status code 404. In case of any other XATMI error (*), the HTTP status code will be set to 200. The default value is as described above.

noreqfilersp = DO_NOT_SEND_REQUEST_FILENAME_BACK_TO_CALLER: If set to true, that will indicate the request logging file name shall not be provided back in buffer to caller in response. Default value for field is false, meaning that if conversion type is set to json2ubf and request logging is used, then field EX_NREQLOGFILE will be provided back in response to caller.
echo = ECHO_MODE: In echo mode the incoming request buffer is directly sent back to caller in response message. This is useful for making link testing routines. The default value for parameter is false.
format = ROUT_FORMAT: Format of the provided rout. Possible values: r, regexp. Default or empty means that regexp compiler will not be used. r and regexp means that rout should have regular expression which will be used to map url. Regular expression matching will be used in case exact path is not found.
urlfield = URL_FIELD: Field to store URL path for json and json2ubf conversion methods in case regular expression format is used. Default value is EX_IF_URL.
parseform = true|false: If set to true then URL and Post body is parsed and EX_IF_REQFORMN/EX_IF_REQFORMV pairs are filled. In this case EX_IF_REQDATA field is not setup. This flag functions only in conv/error mode ext. Flag cannot be used together with fileupload.
finman = SERVICE_LIST: Comma separated list of services to call before target service invocation. This is mandatory list. Any failed service will terminated request chain and error will be returned. Default is empty.
finopt = SERVICE_LIST: Comma separated list of services to call before target service invocation. This is optional list. Any failed service will be ignored and chain execution will continue. Default is empty.
finerr = SERVICE_LIST: Comma separated list of services to be executed when in ext mode incoming mandatory filters or buffer setup failed. In case if EX_NETRCODE is present, it is assumed that buffer content is ready for response generation. This is optional service list. Default is empty.
foutman = SERVICE_LIST: Comma separated list of services to be executed when in ext mode input filters and target service was OK. This is mandatory list, any service error will trigger fouterr chain to process.
foutopt = SERVICE_LIST: Comma separated list of services to be executed when in ext mode input filters, target service was OK and foutman list were executed OK. This is optional list, any service errors will be ignored.
fouterr = SERVICE_LIST: Comma separated list of services to be executed when in ext mode target service or outgoing mandatory filters have failed. In case if EX_NETRCODE is present (set by this or previous services), it is assumed that buffer content is ready for response generation. This is optional service list. Default is empty.
fileupload = true|false: Enable file upload for given URL. If enabled, multipart/form-data will be processed, files will be uploaded to TEMP_DIR directory and corresponding UBF buffer with file names are sent to target service. See File Upload section for more details. This flag functions only in conv/error mode ext. Flag cannot be used together with parseform
tempdir = TEMP_DIR: Temporary directory where to upload the files. This is used only for fileupload URL mode. Parameter is optional, and default setting is OS temp directory which usually is "/tmp".
transaction_handler = true|false: If this flag is set to true, then route is configured as destination for transaction management. I.e. this opens a REST API with which transactions may be started, committed or aborted. Default value for this parameter is false. The REST API details are described bellow in section TRANSACTION MANAGEMENT API. This mode may be configured as single parameter for whole route. Internally it uses conv = ext, so it is possible to use incoming and outgoing filters around the transaction API. So svc is required for this route.
txnoabort = true|false: Do not abort transaction, in case if service failed (the XATMI rules of tpcall(3) here). The default is false, meaning that if service fails, transaction ID in response header endurox-tptranid-rsp is updated with marking transaction must be aborted with commit will be requested.
txnooptim = true|false: Do not optimize the list of involved resource managers (e.g. databases) in global transaction. This setting allows to detect expired transactions when performing the calls. Otherwise expired transaction is detected at commit or abort point. The default value is true.

STATIC ROUTES EXAMPLE

For example:

/static.*={"svc":"@STATIC", "format":"regexp", "conv":"static", "staticdir":"${NDRX_APPHOME}/static"}
/={"svc":"@STATIC", "conv":"static", "staticdir":"${NDRX_APPHOME}/static"}
/index.html={"svc":"@STATIC", "conv":"static", "staticdir":"${NDRX_APPHOME}/static"}

will perform following logic:

sub-folder. For example /static/app1/test.ccs will be provided.
/ will strictly on root host provide only index.html from static resources directory.
index.html in the same way if at host root index is access then, will be provided from static folder.

TRANSACTION MANAGEMENT API

This section describes special built-in API which purpose is to allow to invoke XATMI services with-in global transaction. Where the global transaction is managed by web services consumer.

To enable transactions API, special route shall be configured. For example:

/tran_api={"transaction_handler":true}

Such configured route basically works as ext mode destination, thus filter calls such finman, finopt, etc.. are available and can be configured for the route.

API works as performing http POST of JSON message to the route. And in reply API provides result JSON with error code, error message and transaction id (if applicable).

In any case of success or error JSON body is returned (if it was possible to generate response). In case if request was success, HTTP code 200 is returned. In case if request is invalid (invalid args, etc), response code is set to 400, in case if some Enduro/X internal issues has happened, status code 500 is returned.

Legend of data types used in messages

Table 1. List of basic data types

Type name	Description
<Type>..X	Field length (range) form 0 to X.
<Type>X..Y	Field length (range) from X to Y.
<Type>X	Fixed field length (range) X.
LONG	Signed long field. 32bit or 64bit value. Depending on platform. In case if range is specified and upper value is greater than 10, then for 32bit platforms this is defaulted to 10.
STRING	String value. May contains all ASCII characters according to JSON standard.
INT	Signed integer type, 32bit type.
ULONG	Unsigned long number. 32 or 64 bit value. . Depending on platform. In case if range is specified and upper value is greater than 10, then for 32bit platforms this is defaulted to 10.

Request message

Request shall be HTTP POST JSON to the specified URL.

Table 2. API Request message

JSON field	Format	Cond	Description
operation	STRING1..16	Mand	Values: tpbegin - begin transaction. tpcommit - commit transaction. tpabort - abort transaction.
timeout	ULONG1..20	Optional	Transaction timeout in seconds. If not specified, default value 0 is used, which means maximum transaction time. Field is used only when operation is tpbegin, otherwise ignored.
flags	LONG1..20	Opt	Reserved for future use, and if specified, shall be set to 0.
tptranid	STRING1..256	Cond	Transaction token value. Used for tpcommit, tpabort.

Request example message - begin transaction

{
    "operation":"tpbegin"
    ,"timeout":60
    ,"flags":0
}

Request example message - commit transaction

{
    "operation":"tpcommit"
    "flags":0
    ,"tptranid":"AABZWlQzb1VCQWtKUXdQNVF2UyttVTlpMlh5cDdyc0FFQUFnREkAAAAAAAAAAAAAAAAAAAEAAgDIAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA"
}

Response message

Table 3. API response message

JSON field	Format	Cond	Description
operation	STRING1..16	Mand	Echo from original call.
error_code	INT1..2	Mand	XATMI Error code. See section Error codes and it’s meaning for codes. In case of success 0 is returned.
error_message	STRING1..1024	Mand	Error description. In case of success success, value Succeed is returned.
tptranid	STRING1..256	Cond	Transaction token value returned by tpbegin or echo from tpcommit or tabort calls. Field is returned whenever transaction identifier is available.

NOTE: The system which processes responses shall ignore any unknown (for which system is not aware of) which might appear in future releases on restincl.

Response example message - begin transaction

{
    "operation":"tpbegin"
    ,"error_code":0
    ,"error_message":"Succeed"
    ,"tptranid":"AABZWlQzb1VCQTdZMElsem9ZVGpPREl4ZkZVMkxHUHdFQUFnREkAAAAAAAAAAAAAAAAAAAEAAgDIAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA"
}

Response example message - commit transaction

{
    "operation":"tpcommit"
    ,"error_code":0
    ,"error_message":"Succeed"
    ,"tptranid":"AABZWlQzb1VCQTdZMElsem9ZVGpPREl4ZkZVMkxHUHdFQUFnREkAAAAAAAAAAAAAAAAAAAEAAgDIAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA"
}

Response example message - commit transaction (error)

{
    "operation":"tpcommit"
    ,"error_code":1
    ,"error_message":"1:TPEABORT (last error 1: tpcommit: Transaction was marked for abort and aborted now)"
    ,"tptranid":"AQBZWlQzb1VCQXBnVDBiaVQ1VHRtaFplY0JkamhhOXdFQUFnREkAAAAAAAAAAAAAAAAAAAEAAgDIAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA"
}

Transaction token/identifier (tptranid) notes

Value is platform architecture dependent (CPU, OS version, data model). Value may be re-used in the same Enduro/X cluster strictly if other server has the same architecture and the other server has accessible originating tmsrv(8) instance.

Calling services in transaction mode.

XATMI service may be put in the transactions mode, in case if in request special header is set with the transaction id previously returned by transactions API. This header is used for what ever conv mode used. When XATMI server process finishes it returns updated transaction id header. For next service call or API operations last known transaction id must be used (if available).

Transaction mode is supported only synchronous service invocations, i.e. flags async and asyncecho must be false (which is default values). Transaction headers are ignored in these modes.

To call XATMI service in transaction mode, following steps must be accomplished (assuming that service’s transaction group is configured and that restincl is configured with NULL switch, routes for transaction handler and resource are defined):

Call the transaction_handler API for tpbegin operation, e.g. /tran_api
Calling service route, ensure that header endurox-tptranid-req is set to value returned by 1. field tptranid. Service response return endurox-tptranid-rsp header. This shall be saved for next operation.
Calling another service route, set header endurox-tptranid-req to value returned by 2. response header endurox-tptranid-rsp.
To perform commit, call transaction_handler API with operation set to tpcommit. tptranid must be set here to header endurox-tptranid-rsp value returned from step 3. In case if 3. did time-out, then use endurox-tptranid-rsp from step 2.

For sample code of transaction usage see endurox-connect/tests/01_restin/src/trancl/trancl.go source code.

Additional notes for error handling in transaction mode calls

This section lists additional aspects that needs to be considered while calling services in transaction modes.

Transaction timed-out by tmsrv while handling some REST calls

If route flag txnooptim (default true) is set to true and transaction is expired when performing service calls with endurox-tptranid-req header set, the error TPESYSTEM (code 12) is returned, as internally tpresume(3) call would fail with this error code. If flag txnooptim is set to false and call is made to previously already called resource manager (RM), then it is possible that TPETRAN (code 14) would be returned. Recommended way is to to leave txnooptim as true, as this ensures that transaction is terminated as soon as possible, and there is less chance for orphaned transaction, in case if for known RM on TMJOIN (if supported) on rolled back transaction does not return error code.

Performing commit twice (in case if HTTP REST call time-out)

Retry commits currently are not supported. Thus if particular tptranid did succeed with tpcommit (error was 0), if performing commit again, error TPEABORT (code 1) would be returned. This is due to fact that tmsrv(8) does not hold the history of transactions, and by trying commit unknown transaction tmsrv reports it is TPEABORT this would be the typical case when unknown transaction id is being committed but tmsrv rolled it back due to time-out.

Sample configuration

restincl must be configured with standard NULL switch (libndrxxanulls.so), other switches are not supported. NULL switch ensures that process may participate in global transaction by not touching any real resources by it self.

To put restincl in NULL switch mode, requires to certain configuration to be used and also instance of tmsrv(8) is required.

Fragment from ndrxconfig.xml(5):

<?xml version="1.0" ?>
<endurox>
...
    <servers>
    ...
                <server name="tmsrv">
                        <min>1</min>
                        <max>1</max>
                        <cctag>TRAN</cctag>
                        <srvid>200</srvid>
                        <sysopt>-e ${NDRX_APPHOME}/log/tmsrv-rm1.log -r -- -t1 -l${NDRX_APPHOME}/tmlogs/rm2</sysopt>
                </server>
    </servers>
    ...
        <clients>
        ...
                <client cmdline="restincl">
                        <exec tag="TRAN" autostart="Y" cctag="TRAN" subsect="" log="${NDRX_APPHOME}/log/restin-tran.log"/>
                </client>
        ...
        </clients>
...
</endurox>

Fragment from app.ini(5) definition of NULL transaction group:

...
[@global/TRAN]
NDRX_XA_RES_ID=1
NDRX_XA_OPEN_STR=-
NDRX_XA_CLOSE_STR=-
NDRX_XA_DRIVERLIB=libndrxxanulls.so
NDRX_XA_RMLIB=-
NDRX_XA_LAZY_INIT=1
...

EXIT STATUS

0: Success
1: Failure

EXAMPLE

To see the usage different usage settings, see tests/01_restin/runtime/conf/restin.ini.

BUGS

Report bugs to support@mavimax.com

Golang compiler problems:

go1.14 introduced use of SIGURG for internal purposes of the language runtime. This causes Enduro/X Unix system calls to interrupt with EINTR, which can lead to incorrect work of the binary using Enduro/X Go bindings.

The bug is reported here: https://github.com/golang/go/issues/50521

While this bug is not fixed, the restincl shall be started with following environment variable set:

export GODEBUG="asyncpreemptoff=1"

The setting may be applied to app.ini in [@global] section as:

[@global]
...
GODEBUG=asyncpreemptoff=1