This is XATMI server which provides external REST service mappings for XATMI sub-system. The restoutsv can be started with in multiple copies in order to gain better system performance and have higher stability (so that administrator can reload the configuration, e.g. adding some new service mapping). The data to HTTP REST services are send in POST format.

This is XATMI REST gateway server can send requests in HTTP/HTTPS format. Different content types (say XATMI buffer types are supported). The output format which is sent to network is mapped from input XATMI buffer type. All Enduro/X buffers are supported:

UBF/FML buffer is converted to JSON message and send as mime type application/json. The buffer format it self is one level key:value json with possible array elements (UBF occurrences). The buffer is converted by tpubftojson(3) C function. The error handling is made via two fields defined in Exfields UBF field definition file. The field names are EX_IF_ECODE for XATMI error code and EX_IF_EMSG for error message. The other option is to use HTTP standard error codes.
JSON with this XATMI buffer the message with mime type application/json is directly sent to network. The error fields can be provided in configuration. Other option is to use HTTP standard error codes.
STRING with this XATMI buffer the string message is directly printed to HTTP request stream. the mime type is set to text/plain. The errors are matched in response message by regular expressions or HTTP error code.
CARRAY with this buffer type the received message is sent to HTTP server in binary format as mime type application/octet-stream. The errors are match in response message by regular expressions or HTTP error code.
VIEW are converted to two level JSON. The first level includes one object. The object name matches view name. The for conversion from VIEW to JSON functions like 'tpviewtojson(3)* and tpjsontview(3) are used. If error view based error handling mechanism is selected (errors="json2view"), then views shall contain one "short" type field for error code and another C string field (char array) for error message.

The restincl and restoutsv can be combined so that it would make a HTTP bridge between the systems.

restousv supports service monitoring and setting the services to be depend on echo server. For echo server there could be set max failures to remove services from shared memory (service board) and min echo OK messages, to advertise services back to shared memory.

Internally the restoutsv is multi-threaded XATMI server, where main thread consumes the service requests and then it they are dispatched to XATMI client sessions associated go routine (workers). The number is configured by workers configuration parameter (default 5). When all workers are busy the main thread will wait for free worker. During this time if any other restoustsv is started, it they can handle the traffic.

The restoustsv does not keep the http connections open between multiple requests, this is linked with fact that with go version 1.7, the open connections was leaking thus this functionality is disabled.

The error handling supports for following error types:

Service error
No entry (no service)
Timeout

Whenever format error is occurred (the response cannot be parsed), the XATMI error type 13 (TPETIME) is generated.

The typical restousv configuration looks like:

[@restout]
# Enduro/X debug string
debug = tp=5 ndrx=5 iflags=detailed file=
gencore=0
workers=5
scan_time=7
defaults={"urlbase":"https://example.com"
           ,"errors":"http"
           ,"noreqfilersp": true
           ,"sslinsecure": true
        }

# this will post to: https://example.com/svc1
service XATMISERVICE1={
        "url":"/svc1"
        ,"errors":"json2ubf"
        ,"timeout":5
        }

# this will post to: https://example.com/svc2
service XATMISERVICE2={
        "url":"/svc2"
        ,"errors":"json2ubf"
        ,"timeout":5
        }

Services with dependency on echo example

This section list the example where JSON echo service is defined and once there is enough succeed messages, the dependent services are advertised to service board.

[@restout]
# Enduro/X debug string
debug = tp=5 ndrx=5 iflags=detailed file=
gencore=0
workers=5
scan_time=7
defaults={"urlbase":"https://example.com"
           ,"errors":"http"
           ,"noreqfilersp": true
           ,"sslinsecure": true
        }

# This echo host
service ECHO_JSON={
        "url":"/echo_json"
        ,"errors":"json"
        ,"echo":true
        ,"echo_time":1
        ,"echo_max_fail":2
        ,"echo_min_ok":3
        ,"echo_conv":"json"
        ,"echo_data":"{\"StringField\":\"Hello Echo\"}"
        }

service DEP_JSON={
        "url":"/dep_jsonhte"
        ,"errors":"http"
        ,"timeout":5
        ,"depends_on":"ECHO_JSON"
        }

From above example it could be seen that, the https://example.com/echo_json will be requested every 1 second (echo_time) and if the succeed count is 3 (echo_min_ok) the service will advertise any dependent services, in this case DEP_JSON. In case if the service fails 2x times (echo_max_fail), the DEP_JSON is unadvertised.

The data which is send to service are {"StringField":"Hello Echo"} (echo_data), the buffer type is json.

The advertise and unadvertised operations by echo threads are actually scheduled for XATMI servers main thread and executed with interval of the 7 seconds (scan_time).

Error handling type: http - return error codes in HTTP protocol

With this error handling method, the error codes are directly used from HTTP protocol. and mapped to XATMI return codes. The error code can be mapped from XATMI subsystem to HTTP codes manually by using errors_fmt_http_map parameter in service or default parameter block. The default mapping which is set if errors_fmt_http_map is not present, is following:

http.StatusOK (200) = atmi.TPMINVAL(0)
http.StatusGatewayTimeout (504) = atmi.TPETIME(13)
http.StatusNotFound (404) = atmi.TPENOENT(6)
Anything else (\*) = atmi.TPESVCFAIL(11)

This method is suitable for all buffer formats. Also in case if using other error handling method and the HTTP error code is not http.StatusOK(200), then default http error mapping is used or the one set by errors_fmt_http_map.

Error handling type: json2ubf - UBF request buffer error handling method

This error handling mechanism is suitable for XATMI UBF buffer type. The error message by server must be loaded into top level JSON field EX_IF_ECODE and error message is loaded into EX_IF_EMSG field. This is suitable in case if using restincl on the other Enduor/X server to bridge the servers using HTTP/Rest method and sending data over UBF buffers. The response fields

Error handling type: json - response code embedded JSON response message

This is suitable for json buffer type. It is expected that at root level of json message there are two fields: One for XATMI error code and another for error message. The default value for configuration parameter errfmt_json_code is set to error_message and the default value for parameter errfmt_json_msg is set to errfmt_json_msg.

For example:

{"error_code":13,"error_message":"13:TPETIME (last error 13: ndrx_mq_receive failed: Connection timed out)"}

Error handling type: text - Free format text error code and message

This is free format text field provided back by server. The restoutsv can parse the response and if parse is ok (got 2x arguments) then it is treated as response and error code and value is extracted. If error format is not matched, then it is assumed that there is no error and data is loaded back into original request buffer (either STRING or CARRAY). The regular expression for parsing the error is defined by errfmt_text parameter and default is set to "^([0-9]+):(.\*)$", the first group must match the number and second part is the string containing the error.

For example if response will be:

13:TPETIME (last error 13: ndrx_mq_receive failed: Connection timed out)

Then error code is extracted as 13 and error

Error handling type: json2view - VIEW request buffer error handling method

This is suitable for VIEW buffer type. When these errors are used, two specific parameters must be defined: errfmt_view_code - view field name where XATMI error code is stored and errfmt_view_msg - view field where the error message is stored.

CONFIGURATION

workers = NUMBER_OF_XATMI_SESSIONS: Number XATMI sessions. These sessions are use for serving the outgoing calls - i.e. doing the calls to HTTP server and responding back to XATMI caller. If the number is less than outgoing calls, the calls will be suspended while there will be no XATMI session free. Once it is XATMI session is found free, then call will be served (i.e. called HTTP counterpart). So meanwhile this parameter means max number of concurrent outgoing HTTP calls. The default value for parameter is 5.
gencore = GENERATE_CORE_FILE: If set to 1, then in case of segmentation fault, the core dump will be generated instead of Golang default signal handler which just prints some info in stderr. The default value is 0 which means use default Golang panic handling mechanisms.
scan_time = SCAN_TIME_SECONDS: This is restoutsv wide configuration parameter which is used in case if any echo services are define. This is time when main service thread is interrupted for doing any advertise or un-advertise as a result of echo threads have scheduled the changes in service board.

The default is 1 (second) and used only if there is at-least one echo service.

defaults = SERVICE_CONFIGURATION_JSON: This is JSON string (can be multi-line) with defaults setting for the services. It is basically a service descriptor which is used as base configuration for services. Once the service is being setup, firstly it uses this defaults config block and then overrides it by additional flags in service definition block. The details within the JSON are described in bellow section SERVICE CONFIGURATION.
service <SERVICE_NAME> = SERVICE_CONFIGURATION_JSON: This is the same configuration as for default, but describes the service routes. The REST-OUT process can have as many as needed the service mapping routes. Then <SERVICE_NAME>is XATMI service name to be advertised as outgoing for given configuration.

SERVICE CONFIGURATION

urlbase = URL_BASE: This is first part of the URL to be used when compiling the final url to make requests to. urlbase is used in case if url parameter starts with leading slash symbol (/), then urlbase is contacted with url to get the final destination. If url starts with any other symbol (like "http…"), then it is assumed that URL is full and not partial. Schemes supported are: HTTP and HTTPS.
url = URL: Full or partial HTTP/HTTPS url to do the postings to. If the parameter starts with leading / symbol, then urlbase from given definition or from defaults are used as the start of the request address.
sslinsecure = SSL_INSECURE: If set to true the work with self-signed certificates on HTTPS server side are accepted. The default is false, meaning that requests to self signed hosts will be rejected with error.
timeout = REQUEST_TIMEOUT_SECONDS: Number of seconds in which HTTP/HTTPS server must respond. If the request goes out of the scope of the given seconds, then time-out error TPETIME is returned to caller process.
errors = ERROR_HANDLING: The parameter can be set to following values http, json, json2ubf and text. See the working modes of each of the modes in above text. The default value for this parameter is json2ubf.
errfmt_json_code = JSON_ERRORS_CODE: In case if errors handling mechanism is set json, then this field indicates the top level JSON field which is expected in response. The codes used here are XATMI error codes. The default value is error_code.
errfmt_json_msg = JSON_ERRORS_MESSAGE: In case if error handling mechanism is selected in parameter errors to json, then this field is used to indicate top level JSON field to store the response message. The default value is errfmt_json_msg.
errfmt_json_onsucc = EXPECT_JSON_ERROR_FIELDS_ON_SUCCEESS: If set to true in case of errors driving by json errors, then system will expect to have the errfmt_json_code and errfmt_json_msg fields to be present in response even the service invocation did succeed.
errfmt_view_code = VIEW_ERRORS_CODE: In case if errors handling mechanism is set json2view, then this field indicates view buffer field in which error code should be installed according to JSON-To-VIEW buffer formatting rules. See tpjsontoview(3) and tpviewtojson(3) for message format.
errfmt_view_msg = VIEW_ERRORS_MESSAGE: In case if error handling mechanism in parameter errors is set to to json2view, then this field is used to indicates VIEW field into which responses message is stored. The message it self comes in the JSON message. The JSON message must be formatted according to the JSON-To-VIEW rules, see tpjsontoview(3) and tpviewtojson(3).
errors_fmt_http_map = HTTP_ERROR_CODES_MAPPING: Error mapping between HTTP return codes and XATMI error codes used for returning the value to XATMI service client.

The syntax for the string is following:

"errors_fmt_http_map":"<HTTP_STATUS_CODE_1>:<ATMI_ERROR_CODE_1>,...,
<HTTP_STATUS_CODE_N>:<ATMI_ERROR_CODE_N>,*:<ATMI_ERROR_CODE_FOR_ANY>"

for example (actually a default setting of the parameter):

"errors_fmt_http_map":"200:0,504:13,404:6,*:11"

Means that HTTP return code 200 is translated to Succeed, error code 504 is translated to TPETIME (timeout) and 404 is translated on TPENOENT (no service) and any other error (\*) is translated to 11 which is service failure.

parseonerror = PARSE_ON_ERROR: If set to true and the error response is received from target service, then then service response buffer is populated with corresponding data received. Thus the XATMI service will return the error and the return buffer will be changed too. If set to false in case of error, from XATMI service only error will be returned and service request buffer is not changed.

If error handling is different than http for given rest service, but http error is received, then regardless of this parameter, the XATMI service call buffer is not changed.

The default value for this parameter is false.

echo = ENABLE_ECHO: Make particular service a background echo daemon. This means that given service definition will make periodical remote HTTP/HTTPS REST service invocations. The service will count the succeeds or fails, and will schecdule advertise or unadvertise any services from service board, for which depends_on is set to given echo service name.

The default is false.

echo_time = ECHO_INTERVAL_TIME: Echo interval time. This is the time interval which is used for sending out echo messages. Paramter is used only when echo is set to true.

The default is 5 (seconds).

echo_max_fail = ECHO_MAX_FAIL: Max number of failure transmissions to echo URL. After the ECHO_MAX_FAIL is reached, all services which are depending on this echo server will mark for unadvertise. The unadvertise will happen at period of the scan_time.

The default value is 2.

echo_min_ok = ECHO_MIN_OK: Minimum number of succeed echo messages after which dependent services will be scheduled for advertise by main thread. Main thread is executed at interval scan_time. Note that if service is marked

The default is 3.

echo_conv = ECHO_DATA_FORMAT: Data buffer format which is used for sending the data to remote HTTP service. Possible values are json2ubf, json, text and raw. This parameter sets how echo_data parameter is parsed to prepare the outgoing XATMI buffer that is converted to target HTTP service data format.

The default is json2ubf.

echo_data = ECHO_DATA: Echo data is used for data preparation for sending to target service for doing diagnostics of the HTTP service availability. The value of this JSON configuration parameter depends on value configured echo_conv value.

Formats of data are following according to echo_conv:

json2ubf: The format is according to tpjsontoubf(3) function. One level json with UBF fields inside. Arrays are supported, which are loaded into UBF buffer occurrences. For example:

# This echo host
service ECHO_JSON2UBF={
        "url":"/echo_json2ubf"
        ,"errors":"json2ubf"
        ,"echo":true
        ,"echo_time":1
        ,"echo_max_fail":2
        ,"echo_min_ok":3
        ,"echo_conv":"json2ubf"
        ,"echo_data":"{\"T_STRING_FLD\":\"Some echo data...\"}"
        }

json: This is arbitrary JSON string. The double quotes shall be prefixed with back slash symbol (\). For example:

service ECHO_JSON={
        "url":"/echo_json"
        ,"errors":"json"
        ,"echo":true
        ,"echo_time":1
        ,"echo_max_fail":2
        ,"echo_min_ok":3
        ,"echo_conv":"json"
        ,"echo_data":"{\"StringField\":\"Hello Echo\"}"
        }

text: This is arbitrary text buffer. Any double quotes shall be prefixed with back slash symbol (\). Example:

# This echo host
service ECHO_STRING={
        "url":"/echo_string"
        ,"errors":"text"
        ,"echo":true
        ,"echo_time":1
        ,"echo_max_fail":2
        ,"echo_min_ok":3
        ,"echo_conv":"text"
        ,"echo_data":"This is echo string!"
        }

raw: This is raw data encoded in base64. For example:

service ECHO_RAW={
        "url":"/echo_raw"
        ,"errors":"text"
        ,"echo":true
        ,"echo_time":1
        ,"echo_max_fail":2
        ,"echo_min_ok":3
        ,"echo_conv":"raw"
        ,"echo_data":"AQIDBAUGBwgJEBESExQV"
        }

json2view: the data is encoded as described in tpviewtojson(3) C call. For example:

service ECHO_JSON2VIEW={
        "url":"/echo_json2view"
        ,"errors":"json2view"
        ,"echo":true
        ,"echo_time":1
        ,"echo_max_fail":2
        ,"echo_min_ok":3
        ,"echo_conv":"json2view"
        ,"errfmt_view_msg":"rspmessage"
        ,"errfmt_view_code":"rspcode"
        ,"echo_data":"{\"REQUEST1\":{\"tshort1\": 5,\"tlong1\": 77777,\"tstring1\": [\"\",\"INCOMING TEST\"]}}"
        }

depends_on = DEPENDS_ON: This parameter is used by normal services (non echo), to mark that the defined service is depending on echo service. Thus if echo service name is specified in this parameter, then given service will be advertised if echo_min_ok is reached by echo. And unadvertised when echo_max_fail is reached. When restoutsv is booting, any services which depends on echo are not automatically advertised, instead they wait for echo service schedule.

The default value is empty, thus service is advertised automatically and does not depend on echo service.

EXIT STATUS

0: Success
1: Failure

EXAMPLE

To see the usage different usage settings, see tests/03_restout/runtime/conf/restout.ini and the corresponding rest-in services are defined in tests/03_restout/runtime/conf/restin.ini.

BUGS

Report bugs to support@mavimax.com

Golang compiler problems:

go1.14 introduced use of SIGURG for internal purposes of the language runtime. This causes Enduro/X Unix system calls to interrupt with EINTR, which can lead to incorrect work of the binary using Enduro/X Go bindings.

The bug is reported here: https://github.com/golang/go/issues/50521

While this bug is not fixed, the restoutsv shall be started with following environment variable set:

export GODEBUG="asyncpreemptoff=1"

The setting may be applied to app.ini in [@global] section as:

[@global]
...
GODEBUG=asyncpreemptoff=1

COPYING

Sidebar

Table Of Contents

Name

Synopsis

DESCRIPTION

Services with dependency on echo example

Error handling type: http - return error codes in HTTP protocol

Error handling type: json2ubf - UBF request buffer error handling method

Error handling type: json - response code embedded JSON response message

Error handling type: text - Free format text error code and message

Error handling type: json2view - VIEW request buffer error handling method

CONFIGURATION

SERVICE CONFIGURATION

EXIT STATUS

BUGS

SEE ALSO

COPYING

User Tools

Site Tools

Sidebar

Table Of Contents

Name

Synopsis

DESCRIPTION

Services with dependency on echo example

Error handling type: http - return error codes in HTTP protocol

Error handling type: json2ubf - UBF request buffer error handling method

Error handling type: json - response code embedded JSON response message

Error handling type: text - Free format text error code and message

Error handling type: json2view - VIEW request buffer error handling method

CONFIGURATION

SERVICE CONFIGURATION

EXIT STATUS

BUGS

SEE ALSO

COPYING

Page Tools