Enduro/X Core

For timeout handling, we shall also use pthread_kill() by background thread. This means that client process will require background thread too. As it might perform timed calls (like tpcall with timeout). As there could be many user threads, the timeout thread shall perform scheduler for them all.

For socket processing (fd add extension) I guess we need another thread. So that if message comes in, we put big lock, so that other queues hold on their processing. While we get something. And basically the main thread while doing polling basically should wait for message to appear in some kind of buffer. Locks would be faster than pipes.

Probably we need following tables

1. Mapping for string names -> msgqid, with rwlocks segmented... Also needs to think by which param it will be indexed. probably by q string name

The string names could include RQ address path, or unique RQ address path per binary. if RQA not used. Then binaries needs to register RQADDR strings in the service shared memory entries. Length is 1...30

So Enduro/X service shared mem block, pointers to either one or multiple RQADDRs. Either binaries share RQA or each generates own RQA. The RQA strings are generic Q strings mapped to MSGID.

Regarding message dispatching.

So we get 3x threads and one lock W and one Job queue. In job queue messages from admin Q or poll() thread are put. Also we have

Main thread does following:

1. init
1. START:
1. lock S
1. goes to msgrcv waiting for service dispatching.
1. unlock S
1. if any message received or msgrcv was interrupted -> lock W
1. if message was received put that in job Q.
1. process each message in Job Q.
1. release lock W
1.. continue START

thread 2, admin Q:
2. init
2. START: goes into msgrcv
2. if message received, we lock W
2. put message in job Q
2. unlock W
2. while trylock S and it is locked already, do pthread_kill to main thread so that w guarantee that we interrupt the msgrcv of 1., reschedule our thread, and repeat while we get the lock... If we get the lock then continue to work to next step
2. continue START

thread 3, poll & timeouts:
3. init
3. START: goes into poll() with timed option in case if periodic callback is set
3. if message received, we lock W
3. put message in job Q
3. unlock W
3. while trylock S and it is locked already, do pthread_kill to main thread so that w guarantee that we interrupt the msgrcv of 1., reschedule our thread, and repeat while we get the lock... If we get the lock then continue to work to next step
3. continue START

For worker threads if we need to generate TPETIME when doing tpcall, then we have to assign our thread id and timeout parameter to thread 3. so that it can interrupt our msgrcv when timeout is reached. Also this must be deliver to thread 3. in some atomic fashion, also wait for mutex to clear, send kills, till second mutex becomes free.

https://www.softprayog.in/programming/interprocess-communication-using-system-v-message-queues-in-linux

RW sempahore: https://cboard.cprogramming.com/c-programming/95848-reader-writer-locks-shared-mem-using-semaphores.html

Tuxedo probably for timeout processing uses BBL to send timeout messages to clients when they do timeout. Enduro/X uses per process internal mechanisms for timeout processing and does not relay on BBL... thus Enduro/X is able to crash survival.

Registered issue for Ubuntu 18.04, https://bugs.launchpad.net/ubuntu/+source/gcc-defaults/+bug/1796858 - Building process fails with hidden symbol `pthread_atfork'