Too many open files when running xadmin

I am trying to run the xadmin start command and end up receiving the error in the log as below

N:NDRX:2:67aded1f:17863:77b2b22824c0:000:20251113:130508299195:_open_rply_q:config.c:0122:Failed to open queue: [/master,sys,bg,xadmin,17863] - ERROR ! Queue error, check logs err: Too many open files

This seems like something to do with the message queues however I have right configuration in place as mentioned n the https://www.endurox.org/dokuwiki/doku.php?id=endurox:v8.0.x:guides:ex_adminman#_linux_setup file. Can someone guide on whats the issue here?

Replies (2)

RE: Too many open files when running xadmin - Added by Lauris 18 days ago

Hi,

So for Linux you need this

$ sudo -s
# cat << EOF >> /etc/security/limits.conf

# Do not limit message Q Count.
# Some Linux 3.x series kernels have a bug, that limits 1024
# queues for one system user.
# In 2.6.x and 4.x this is fixed, to have
# unlimited count of queues (memory limit).
# ealrier and later Linuxes have fixed this issue.
*               soft    msgqueue        -1
*               hard    msgqueue        -1

# Increase the number of open files
*               soft    nofile  1024
*               hard    nofile  65536

EOF
# exit
$

$ sudo -s

# cat << EOF >> /etc/sysctl.conf

# Max Messages in Queue
fs.mqueue.msg_max=10000

# Max message size, to pass unit tests, use 1M+1K
fs.mqueue.msgsize_max=1049600

# Max number of queues system-wide
fs.mqueue.queues_max=10000

EOF

# Apply kernel parameters now
$ sudo sysctl -f /etc/sysctl.conf

# to check the values, use (print all) and use grep to find:
$ sudo sysctl -a | grep msgsize_max

After that reboot your linux server (or try to logout login) to apply new security settings.

Let me know is this helps.

Thanks

RE: Too many open files when running xadmin - Added by Madars 6 days ago

Reply from the Client:

I see this issue is resolved now. However for a process I get this error when I run the xadmin start -y &

Can you help me with this ?

The process log:
N:NDRX:2:67aded1f:08345:770beae07000:001:20251124:094240054733:ric_q_send_2:iutils.c:0489:Failed to send data to queue [/master,sys,bg,ndrxd] with error: 90:Message too long

$ cat /dev/mqueue/master,sys,bg,ndrxd
QSIZE:0          NOTIFY:0     SIGNO:0     NOTIFY_PID:0

$ cat xadmin.log
N:NDRX:2:67aded1f:08308:78b761577d80:000:20251124:094227866298:_open_rply_q:config.c:0133:Reply queue [/master,sys,bg,xadmin,8308] opened!

$ cat ndrxd.log
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508057283:tart_process:pmodel.c:0934:*********processing for startup ORATMS/158 (state: 2)*********
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508058213:espawn_check:espawn.c:0184:next try after: 5 sty
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508058274:espawn_check:espawn.c:0173:Respawning server: srvid: 159, name: [ORATMS], seq try: 34, already not running: 5 secs, singleton_attempt: 0, do_wait: 0
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508058299:tart_process:pmodel.c:0934:*********processing for startup ORATMS/159 (state: 2)*********
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508059194:espawn_check:espawn.c:0184:next try after: 5 sty
N:NDRX:3:67aded1f:08309:7fb5b4d15700:000:20251124:094508069948:handle_child:pmodel.c:0205:sigchld: PID: 8630 exit status: 0
N:NDRX:2:67aded1f:08309:7fb5b4d15700:000:20251124:094508069998:handle_child:pmodel.c:0215:Process normal shutdown!
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508070092:cmd_notify  :rtstop.c:0148:Got notification for pid:8630 srvid:0
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508070106:cmd_notify  :rtstop.c:0198:Removing resources allocated for process [ORATMS]
N:NDRX:2:67aded1f:08309:7fb5b7d24900:000:20251124:094508070109:rom_pid_hash:pmodel.c:0445:Removing pid 8630 from pidhash
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508070127:cmd_notify  :rtstop.c:0220:cmd_notify returns with status 0
N:NDRX:3:67aded1f:08309:7fb5b4d15700:000:20251124:094508070540:handle_child:pmodel.c:0205:sigchld: PID: 8631 exit status: 0
N:NDRX:2:67aded1f:08309:7fb5b4d15700:000:20251124:094508071613:handle_child:pmodel.c:0215:Process normal shutdown!
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508071704:cmd_notify  :rtstop.c:0148:Got notification for pid:8631 srvid:0
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508071718:cmd_notify  :rtstop.c:0198:Removing resources allocated for process [ORATMS]
N:NDRX:2:67aded1f:08309:7fb5b7d24900:000:20251124:094508071721:rom_pid_hash:pmodel.c:0445:Removing pid 8631 from pidhash
N:NDRX:3:67aded1f:08309:7fb5b7d24900:000:20251124:094508071735:cmd_notify  :rtstop.c:0220:cmd_notify returns with status 0

1) What is this process "ORATMS", is it binary built by builtms?

2) Can you please increase debug log for it then capture full log from the failed startup and attach it here? The log level can be increased by adding such line:

app.ini

...
[@debug]
...
ORATMS=ndrx=5 file=/path/to/log/file
...

3) Have you restarted the server application instance after changing the limits?
If you want fully cleanup current IPC resources and try start again, please use

$ xadmin down

you can also clean up all the queues by:

$ xadmin qrmall ,

4) Can you please give us current system limits:

sudo sysctl -a | grep msgsize_max

5) Is other binaries starting OK?

(1-2/2)