Table of Contents
- 1. Overview
- 2. Module installation procedure
- 3. SNMP Module
- 4. SNMP Structure details
- 4.1. OID structure
- 4.2. Common notes
- 4.3. T_CLIENT class notes
- 4.4. T_DOMAIN class notes
- 4.5. T_MACHINE class notes
- 4.6. T_QUEUE class notes
- 4.7. T_SERVER class notes
- 4.8. T_SERVICE class notes
- 4.9. T_SVCGRP class notes
- 4.10. T_BRCON class notes
- 4.11. T_USERDATA class notes
- 4.12. Data aggregation
- 4.13. Traps
- 4.14. Multiple Enduro/X node monitoring on the same server (reporting to same snmpd), Option 1
- 4.15. Multiple Enduro/X node monitoring on the same server (reporting to same snmpd), Option 2
- 5. Conclusions
This document describes administrative task required for Enduro/X Enterprise Edition (EE) module. Enterprise Edition of the Enduro/X provides advanced features for the Enduro/X Core.
Module is installed as standard release package in for of RPM, DEB or TAR.GZ, depending on the platform used.
Enterprise Edition module depends on Enduro/X Core and Enduro/X Python modules. These modules shall be installed and configured according to the following guides:
- Enduro/X Core, install according the Enduro/X Administration Manual.
- Enduro/X Python, install according the Enduro/X Python Getting Started Tutorial.
To use Enduro/X Python product for own applications, license shall be acquired separately.
Commands which shall be executed from "root" user, are prefixed with "sudo". If sudo is not installed on the server where installation is done, or it is not operational, then login to root and perform the actions
Such as:
$ su - root # ...
$ sudo rpm -i endurox-ee-8.0.1-1.el8_gnu.x86_64.rpm
$ sudo dpkg -i endurox-ee-8.0.1-1.ubuntu22_04_gnu.x86_64.deb
Enduro/X SNMP module works as SNMP AgentX extension, meaning that on platform where module is standard, SNMP daemon shall be operational and it shall supported AgentX protocol. This guide is based on Net-SNMP package, is it is available on all platforms supported by Enduro/X.
The module by it self provides following features:
- Monitoring of the all tm_mib(5) data items published by Enduro/X.
- Additional aggregates for the monitored data, for simpler system health assessment.
- Trap generation for the ULOG messages.
Please read the eesnmpagx(8) manpage, for details on how to configure the Enduro/X SNMP Agent by it self (i.e. as part of the Enduro/X application). However the further document will describe how to prepare the Operation System to run the Net-SNMP. Once OS is configured, the eesnmpagx in the ini file configure is just directed to Net-SNMP AgentX port, where the eesnmpagx connects and publishes it's data.
This sections lists platform specific installation instructions. For Linux based operating systems, any special path for MIB files are not needed, as Enduro/X EE install package already locates it to correct place (i.e. /usr/share/endurox/snmp/mibs/ENDUROX-MIB.txt). For other operating systems, it is recommended to symlink this file to location, where given OS keeps the database of SNMP MIB files. MIB files are needed for processes such as snmpwalk or for trap processing.
$ sudo apt update $ sudo apt install snmpd snmp libsnmp-dev $ sudo systemctl enable snmpd $ sudo systemctl start snmpd
$ sudo yum update $ sudo yum install net-snmp net-snmp-utils $ sudo systemctl enable snmpd $ sudo systemctl start snmpd
$ sudo zypper install -y net-snmp $ sudo systemctl enable snmpd $ sudo systemctl start snmpd
On Solaris platform Net-SNMP is installed and enable with the stock OS installation. However, that can be done in following way (if for some reason service is not enabled):
$ sudo svcadm enable net-snmp:default -- Link the MIB file $ sudo ln -s /usr/share/endurox/snmp/mibs/ENDUROX-MIB.txt /opt/freeware/share/snmp/mibs/ENDUROX-MIB.txt
Before using yum/dnf, please install Linux ToolBox (script dnf_aixtoolbox.sh) according to the IBM instructions.
Note that AIX ships with own SNMP Daemon which unfortunately does not support AgentX protocol. This installation provides instructions how to disable AIX SNMP daemon, and enable Net-SNMP instead.
The other path to follow, is to configure the Net-SNMP to work with other port numbers and run it along with the AIX SNMP daemon, however that is out of the scope of this admin guide.
-- Stop and disable IBM SNMP Daemon: $ sudo stopsrc -s snmpd $ sudo perl -i -pe 's/^(?=.*start \/usr\/sbin\/snmpd .*)(?!#)/#/' /etc/rc.tcpip $ sudo stopsrc -s snmpmibd $ sudo perl -i -pe 's/^(?=.*start \/usr\/sbin\/snmpmibd .*)(?!#)/#/' /etc/rc.tcpip -- Install Net-SNMP and start it: $ sudo yum install net-snmp net-snmp-utils $ sudo chmod +x /etc/rc.d/rc2.d/Ssnmpd $ sudo /etc/rc.d/rc2.d/Ssnmpd $ sudo /etc/rc.d/rc2.d/Ssnmpd start -- Link the MIB file $ sudo ln -s /usr/share/endurox/snmp/mibs/ENDUROX-MIB.txt /opt/freeware/share/snmp/mibs/ENDUROX-MIB.txt
This section provides configuration used for snmpd by it self. Setup here will list minimalistic configuration, needed to expose the Enduro/X data to the SNMP network. This can be used as starting point for the configuration.
Following setup may be used as starting point for the SNMP configuration, if v2c security level is sufficient.
$ sudo tee /etc/snmp/snmpd.conf << EOF rocommunity public default -V systemonly rocommunity6 public default -V systemonly view systemonly included .1.3.6.1.4.1.55065.10.2 master agentx # SNMP 5.8 has bug, if port is not specified: # see https://github.com/net-snmp/net-snmp/issues/34 agentaddress udp:localhost:161 # May use bellow, if running version > Net-SNMP 5.8 #agentaddress localhost # And for Solaris stock snmpd, tcp socket is not supported, only Unix domain # sockets work, thus for Solaris, uncomment following two lines # AgentXSocket /var/agentx/master # agentXPerms 777 777 # For Solaris, comment following lines out: # while for other OS, leave these in place. AgentXSocket tcp:127.0.0.1:705 agentXPerms trap EOF
For Solaris OS, AgentXSocket TCP mode is not supported, please use unix_socket setting for the eesnmpagx.
In case if SNMP v3 security configuration must be applied for the SNMP configuration. The community strings are being disabled and new SNMP user is being added. Following example adds user "SNMPv3User" with password "SNMPv3SHAPass", channel is being encrypted with AES key "SNMPv3AESPass".
master agentx # SNMP 5.8 has bug, if port is not specified: # see https://github.com/net-snmp/net-snmp/issues/34 agentaddress udp:localhost:161 # May use bellow, if running version > Net-SNMP 5.8 #agentaddress localhost # And for Solaris stock snmpd, tcp socket is not supported, only Unix domain # sockets work, thus for Solaris, uncomment following two lines # AgentXSocket /var/agentx/master # agentXPerms 777 777 # For Solaris, comment following lines out: # while for other OS, leave these in place. AgentXSocket tcp:127.0.0.1:705 agentXPerms trap # SNMP V3 users, with AES connection encryption: CreateUser SNMPv3User SHA SNMPv3SHAPass AES SNMPv3AESPass # This user may read only the Enduro/X SNMP tables: rouser SNMPv3User priv .1.3.6.1.4.1.55065.10.2 EOF
NOTE: after the first run of the snmpd daemon, the line with clear passwords can be removed (at least tested on Ubuntu Linux 20.04), as they are encrypted and stored internal Net-SNMP configuration files (e.g. /var/lib/snmp/snmpd.conf).
When configuration files are updated, restart is required, that may be done in following way:
-- Linux: $ service snmpd start -- Solaris $ sudo svcadm enable net-snmp:default -- AIX $ sudo /etc/rc.d/rc2.d/Ssnmpd restart -- Solaris $ sudo svcadm restart net-snmp:default -- FreeBSD $ sudo service snmpd restart
Once SNMP is configured, boot it back, by:
If running in v2c mode, use following command to list the SNMP data:
$ snmpwalk -v 2c -c public localhost ENDUROX-MIB::enduroxMachineTable ENDUROX-MIB::exMachineAgNodeid.1.1 = INTEGER: 1 ENDUROX-MIB::exMachineTaLmid.1.1 = INTEGER: 1 ENDUROX-MIB::exMachineTaCuraccessers.1.1 = INTEGER: 12 ENDUROX-MIB::exMachineTaCurconv.1.1 = INTEGER: 0 ENDUROX-MIB::exMachineTaState.1.1 = STRING: "ACT" ...
In case if using SNMP v3 mode, use following command:
$ snmpwalk -v 3 -l authPriv -u SNMPv3User -a sha -A SNMPv3SHAPass -x aes -X SNMPv3AESPass localhost ENDUROX-MIB::enduroxSvcgrpTable ENDUROX-MIB::exSvcgrpAgNodeid.1.1.1."@CCONF" = INTEGER: 1 ENDUROX-MIB::exSvcgrpAgNodeid.1.1.10.".TMIB" = INTEGER: 1 ENDUROX-MIB::exSvcgrpAgNodeid.1.1.10.".TMIB-1" = INTEGER: 1 ENDUROX-MIB::exSvcgrpAgNodeid.1.1.10.".TMIB-1-10" = INTEGER: 1 ENDUROX-MIB::exSvcgrpAgNodeid.1.1.9999."@CPMSVC" = INTEGER: 1 ENDUROX-MIB::exSvcgrpTaLmid.1.1.1."@CCONF" = INTEGER: 1 ENDUROX-MIB::exSvcgrpTaLmid.1.1.10.".TMIB" = INTEGER: 1 ENDUROX-MIB::exSvcgrpTaLmid.1.1.10.".TMIB-1" = INTEGER: 1 ENDUROX-MIB::exSvcgrpTaLmid.1.1.10.".TMIB-1-10" = INTEGER: 1 ENDUROX-MIB::exSvcgrpTaLmid.1.1.9999."@CPMSVC" = INTEGER: 1 ENDUROX-MIB::exSvcgrpTaServicename.1.1.1."@CCONF" = STRING: "@CCONF" ENDUROX-MIB::exSvcgrpTaServicename.1.1.10.".TMIB" = STRING: ".TMIB" ...
NOTE: On IBM IBM AIX platform (7.2 and 7.3), snmpwalk returns errors when working with localhost, however when specifying IPv6 ::1 value there, the snmpwalk was functional. However tests have shown that when listening on IPv4 address (i.e. agentaddress set to IPv4), connecting to AIX from external host works. This note affects cases if, Net-SNMP was started on AIX, without specifying ports in agentaddress configuration flag. If ports were specified standard localhost walks work.
This sections lists example, how to setup SNMP ULOG trap for Enduro/X. Traps are optional, and they are enabled by trap_enable=1 setting for the eesnmpagx(8).
Instructions are provided only for Ubuntu platforms. For other platforms instructions shall be similar (contact the Mavimax support if in doubt). The following installs snmptt package, which for demonstration purposes logs the trap messages to the logfile. However system Administrator might change the snmptt configuration to send e-mails on traps or perform other actions.
$ sudo apt-get install snmptrapd libnet-ip-perl libnet-snmp-perl $ sudo systemctl enable snmptt $ sudo systemctl start snmptt $ sudo systemctl enable snmptrapd $ sudo systemctl start snmptrapd
Update /etc/snmp/snmpd.conf to send traps to the snmptrapd:
$ sudo cat << EOF >> /etc/snmp/snmpd.conf agentXPerms trap trapcommunity public # due to bug Net-SNMP 5.8 bug https://github.com/net-snmp/net-snmp/issues/34 # ports shall be used with addresses #trapsink localhost public trapsink localhost:162 public EOF
To route traps from snmptrapd to snmptt scripts, change the /etc/snmp/snmptrapd.conf (replace content):
traphandle default snmptthandler disableAuthorization yes
Update the /etc/snmp/snmptt.ini, to have more details in trap messages:
sudo sed -r -i '/\[General\]/,/\[/ s/^mode =.*$/mode = daemon/' $SMPTT_CONF_FILE sudo sed -r -i '/\[General\]/,/\[/ s/^net_snmp_perl_enable =.*$/net_snmp_perl_enable = 1/' $SMPTT_CONF_FILE sudo sed -r -i '/\[General\]/,/\[/ s/^mibs_environment =.*$/mibs_environment = ALL/' $SMPTT_CONF_FILE sudo sed -r -i '/\[General\]/,/\[/ s/^translate_log_trap_oid =.*$/translate_log_trap_oid = 2/' $SMPTT_CONF_FILE sudo sed -r -i '/\[General\]/,/\[/ s/^translate_value_oids =.*$/translate_value_oids = 2/' $SMPTT_CONF_FILE sudo sed -r -i '/\[General\]/,/\[/ s/^net_snmp_perl_cache_enable =.*$/net_snmp_perl_cache_enable = 0/' $SMPTT_CONF_FILE sudo sed -r -i '/\[General\]/,/\[/ s/^[#]*mibs_environment =.*$/mibs_environment = ALL/' $SMPTT_CONF_FILE
Generate trap file for snmptt:
$ sudo MIBS=ALL snmpttconvertmib \
--in=/usr/share/snmp/mibs/ENDUROX-MIB.txt \
--out=/etc/snmp/ENDUROX.conf.compaqAdd ENDUROX.conf.compaq trap file to /etc/snmp/snmptt.ini (to have similar configuration as):
$ sudo vi /etc/snmp/snmptt.ini ... [TrapFiles] # A list of snmptt.conf files (this is NOT the snmptrapd.conf file). # The COMPLETE path and filename. Wildcards can also be used. # Ex: # /etc/snmptt/snmptt.conf # /etc/snmptt/snmptt.*.conf snmptt_conf_files = <<END /etc/snmp/snmptt.conf /etc/snmp/ENDUROX.conf.compaq END
may use following script to perform above:
sudo sed -r -i "/\TrapFiles\]/,/\[/ s/^\/etc\/snmp\/snmptt\.conf*$/\/etc\/snmp\/snmptt.conf\n\/etc\/snmp\/ENDUROX.conf.compaq/" /etc/snmp/snmptt.ini
As snmptt uses spool files. For some reason default permissions of spool directory are invalid, thus fix them:
$ sudo chmod a+w /var/spool/snmptt
finally restart all involved processes to enable traps processing:
$ sudo service snmpd restart $ sudo service snmptrapd restart $ sudo service snmptt restart
Once Enduro/X application is booted and if userlog(3) logs are generated, following traps are emitted (example output from /var/log/snmptt/snmptt.log):
Mon Jul 24 15:05:43 2023 ENDUROX-MIB::enduroxUlogNotification Normal "Status Events" 192.168.18.94 - Ulog entry 2 2 info 20230112 23440651 22786 ndrxd INODE_TEST1_FOR2-20230724150518 Mon Jul 24 15:05:43 2023 ENDUROX-MIB::enduroxUlogNotification Normal "Status Events" 192.168.18.94 - Ulog entry 2 2 info 20230112 23440651 22786 ndrxd INODE_TEST2_FOR2-20230724150518 Mon Jul 24 15:05:59 2023 ENDUROX-MIB::enduroxUlogNotification Normal "Status Events" 192.168.18.94 - Ulog entry 2 2 error 20230724 15055910 6347 ndrxd Failed to start server [atmi.sv200], error: 2, No such file or directory Mon Jul 24 15:05:59 2023 ENDUROX-MIB::enduroxUlogNotification Normal "Status Events" 192.168.18.94 - Ulog entry 2 2 error 20230724 15055910 6069 ndrxd Server process [atmi.sv200], srvid 10, pid 6347 died
As information provided by Enduro/X SNMP AgentX is based and sourced from tm_mib(5) data. Before continuing with this section, please check the tm_mib(5) manpage.
Following classes are supported by eesnmpagx(8) process (which is SNMP AgentX sub-agent process):
- T_CLIENT - Client process
- T_DOMAIN - Domain
- T_MACHINE - Machines in domain
- T_QUEUE - IPC Queues
- T_SERVER - XATMI server processes
- T_SERVICE - XATMI service definitions
- T_SVCGRP - XATMI service instances
- T_BRCON - tpbridge connection information
- T_USERDATA - User data class, collected from user service by Enduro/X EE AgentX.
Class T_USERDATA is exclusively used by Enduro/X AgentX, which gives option for applications to expose data over SNMP channel without doing any specific SNMP coding. Additionally every SNMP table includes in it’s index field which denote the Enduro/X cluster node id on which AgentX by it self runs. This gives possibility to separate several Enduro/X runtimes operating on the same server.
Additionally module provides SNMP tables for data aggregation, so that selected groups (by services names, process names, connection counts, etc) are exposed as SNMP table entry, thus providing easier system configuration in target monitoring suite.
When Enterprise Edition is installed, the SNMP MIB file is loaded at /usr/share/endurox/snmp/mibs/ENDUROX-MIB.txt.
following naming convention is used in the ENDUROX-MIB.txt:
- Table names and field names follow the camel-case notation.
- Table names are produced as: "endurox<Class name without leading T_>Table".
- Field names are built as: "ex<Class name without leading T_><field name where TA_XXX is converted to TaXxx>" as found in the tm_mib(5).
- Agent node id field used in index: "ex<Class name without leading T_>AgNodeid".
Technical details how to configure the eesnmpagx(8), please follow the manpage of the given process.
- Mavimax company is registered under the OID: 1.3.6.1.4.1.55065.
- Mavimax products: 1.3.6.1.4.1.55065.10.
- Enduro/X product: 1.3.6.1.4.1.55065.10.2.
- Enduro/X product SNMP tables: 1.3.6.1.4.1.55065.10.2.2.
- Enduro/X product SNMP notifications (notifications and notification fields): 1.3.6.1.4.1.55065.10.2.3.
- Enduro/X product SNMP notifications (traps): 1.3.6.1.4.1.55065.10.2.3.0.
- Enduro/X product SNMP notifications (notification field tables): 1.3.6.1.4.1.55065.10.2.3.1.
- T_CLIENT data table (SNMP enduroxClientTable): 1.3.6.1.4.1.55065.10.2.2.10.
- T_CLIENT aggregator in enduroxClientAggTable table: 1.3.6.1.4.1.55065.10.2.2.11.
- T_DOMAIN data table (SNMP enduroxDomainTable): 1.3.6.1.4.1.55065.10.2.2.20.
- T_MACHINE data table (SNMP enduroxMachineTable): 1.3.6.1.4.1.55065.10.2.2.30.
- T_QUEUE data table (SNMP enduroxQueueTable): 1.3.6.1.4.1.55065.10.2.2.40.
- T_QUEUE aggregator in SNMP enduroxQueueAggTable table: 1.3.6.1.4.1.55065.10.2.2.41.
- T_SERVER data table (SNMP enduroxServerTable): 1.3.6.1.4.1.55065.10.2.2.50.
- T_SERVER aggregator in SNMP enduroxServerAggTable table: 1.3.6.1.4.1.55065.10.2.2.51.
- T_SERVICE data table (SNMP enduroxServiceTable): 1.3.6.1.4.1.55065.10.2.2.60.
- T_SVCGRP data table (SNMP enduroxSvcgrpTable): 1.3.6.1.4.1.55065.10.2.2.70.
- T_SVCGRP aggregator in SNMP enduroxSvcgrpAggTable table: 1.3.6.1.4.1.55065.10.2.2.71.
- T_BRCON data table (SNMP enduroxBrconTable): 1.3.6.1.4.1.55065.10.2.2.80.
- T_BRCON aggregator SNMP enduroxBrconAggTable: 1.3.6.1.4.1.55065.10.2.2.81.
- T_USERDATA data table (SNMP enduroxUserdataTable): 1.3.6.1.4.1.55065.10.2.2.200.
- enduroxUlogNotification ULOG monitoring notification/trap: 1.3.6.1.4.1.55065.10.2.3.0.1.
For configuration of the T_USERDATA user service invocation, see eesnmpagx(8) process description. For exact details of SNMP tables and columns, please refer to /usr/share/endurox/snmp/mibs/ENDUROX-MIB.txt.
- TA_LMID and TA_DOMAINID fields equivalents in SNMP MIB are encoded as Integer32(1..255) instead of the string.
- Table name: enduroxClientTable
- TA_CLIENTID / exClientTaClientid is trimmed down to 113 symbols, due to SNMP index length limitations.
- Table index is composed of: exClientAgNodeid, exClientTaClientid.
- Table name: enduroxDomainTable
- Table index is composed of: exDomainAgNodeid, exDomainTaDomainid.
- Table name: enduroxMachineTable
- Table index is composed of: exMachineAgNodeid, exMachineTaLmid.
- Table name: enduroxQueueTable.
- TA_RQADDR / exQueueAgNodeid is trimmed down to 113 symbols, due to SNMP index length limitations.
- Table index is composed of: exQueueAgNodeid, exQueueTaRqaddr.
- Table name: enduroxServerTable.
- Table index is composed of: exServerAgNodeid, exServerTaSrvid.
- Table name: enduroxServiceTable.
- Table index is composed of: exServiceAgNodeid, exServiceTaServicename.
- Table name: enduroxSvcgrpTable.
- Field value from TA_LASTEXECTIMEUSEC is represented in milliseconds and filed name is changed to exSvcgrpTaLastexectimemsec.
- Field value from TA_MAXEXECTIMEUSEC is represented in milliseconds and filed name is changed to exSvcgrpTaMaxexectimemsec.
- Field value from TA_MINEXECTIMEUSEC is represented in milliseconds and filed name is changed to exSvcgrpTaMinexectimemsec.
- Table index is composed of: exSvcgrpAgNodeid, exSvcgrpTaLmid, exSvcgrpTaSrvid, exSvcgrpTaServicename.
This class name is used as an interface from Enduro/X SNMP AgentX to the user service which provides data to be exposed on the SNMP channel. Configured in eesnmpagx parameter userdatasvc. Data returned by this service must follow the standard protocol as described in tm_mib(5).
- Table name: enduroxUserdataTable.
If enabled by flag enable_agg=1 in [@eesnmpagx] section, following additional tables are provided:
- enduroxClientAggTable
- enduroxQueueAggTable
- enduroxServerAggTable
- enduroxSvcgrpAggTable
- enduroxBrconAggTable
All aggregator tables contain standard fields:
- ex<Class>AggAgNodeid (Integer32(1..255)) Enduro/X cluster node id from which data was collected (i.e. node id where tpadmsv was running).
- ex<Class>AggTaLmid (Integer32(1..255)) Enduro/X cluster node id for which data tpadmsv is reporting. Except enduroxBrconAggTable which uses ex<Class>AggTaExNodeid field name instead.
This table is enabled in case if T_CLIENT is enabled. Table contains following fields:
- exClientAggTaCltname (OCTET STRING (SIZE(1..30))) client process name, as reported in TA_CLTNAME field in tm_mib(5), which is client process executable name.
- exClientAggNumAct (Integer32) Number of running processes.
- exClientAggNumDea (Integer32) Number of dead processes, as reported by cpmsrv(8). For stand-alone processes (not registered in ndrxconfig.xml(5)), this is not accounted.
- Table index is composed of: exClientAggAgNodeid, exClientAggTaLmid, exClientAggTaCltname.
These statistics are render when T_QUEUE class is enabled. Table contains following fields:
- exQueueAggSumTaNqueued (Integer32) Total number of messages currently enqueued in all queues. This value is particularly useful to monitor as it indicates overall system health. Typically (however depending on application specifics) this number is low. Thus growing number could indicate application problems.
- Table index is composed of: exQueueAggAgNodeid, exQueueAggTaLmid.
Table contains XATMI server process aggregations by server name. These statistics are render when T_SERVER class is enabled.
Table contains following fields:
- exServerAggTaServername (OCTET STRING(SIZE(1..78))) server name as defined in name attribute in <server> tag in ndrxconfig.xml.
- exServerAggNumAct (Integer32) Total number of processes running.
- exServerAggNumDea (Integer32) Total number of processes dead.
- exServerAggNumIna (Integer32) Total number of processes inactive (not started or shutdown).
- exServerAggAvgTaGeneration (Integer32) Total number of process startup attempts.
- Table index is composed of: exServerAggAgNodeid, exServerAggTaLmid, exServerAggTaServername.
Table contains service aggregations by service name. These statistics are render when T_SVCGRP class is enabled.
Table contains following fields:
- exSvcgrpAggNumSvcAgx (Integer32) Number of services available on given node. When AgNodeid matches TaLmid, this includes total count of services available including remote services. If AgNodeid does not matches TaLmid, information is reported as count of remote services, as visible from particular AgNodeid.
- exSvcgrpAggNumSvcNode (Integer32) Number of local services on particular AgNodeid node.
- exSvcgrpAggSumTaNcompleted (Integer32) Total number of completed requests.
- exSvcgrpAggSumTaTotsuccnum (Integer32) Total number successfully completed requests.
- exSvcgrpAggSumTaTotsfailnum (Integer32) Total number of failed requests.
- exSvcgrpAggAvgTaLastexectimemsec (Integer32) Average service last response time. This does not include response times for remote services. When reporting for different TaLmid than AgNodeid, this returns -1 (i.e. for remote services).
- exSvcgrpAggAvgTaMaxexectimemsec (Integer32) Average max service execution time. in milliseconds. For remote node services, this is set to -1.
- exSvcgrpAggAvgTaMinexectimemsec (Integer32) Average min service execution time. in milliseconds. For remote node services, this is set to -1.
- exSvcgrpAggMaxTaLastexectimemsec (Integer32) Max last service execution time. in milliseconds. For remote node services, this is set to -1.
- exSvcgrpAggMaxTaMaxexectimemsec (Integer32) Max max service execution time. in milliseconds. For remote node services, this is set to -1.
- exSvcgrpAggMaxTaMinexectimemsec (Integer32) Max min service execution time. in milliseconds. For remote node services, this is set to -1.
- exSvcgrpAggMinTaLastexectimemsec (Integer32) Min last service execution time. in milliseconds. For remote node services, this is set to -1.
- exSvcgrpAggMinTaMaxexectimemsec (Integer32) Min max service execution time. in milliseconds. For remote node services, this is set to -1.
- exSvcgrpAggMinTaMinexectimemsec (Integer32) Min min service execution time. in milliseconds. For remote node services, this is set to -1.
- Table index is composed of: exSvcgrpAggAgNodeid, exSvcgrpAggTaLmid, exSvcgrpAggTaServicename.
Table aggregates connection (between the Enduro/X nodes) information. Table These statistics are render when T_BRCON class is enabled.
Table contains following fields:
- exBrconAggTaExRemnodeid (Integer32(1..255)) Remote Enduro/X cluster node id.
- exBrconAggNumConn (Integer32) Number of activate connections.
- exBrconAggMaxTaExLastsync (Integer32) Number of seconds since last Enduro/X internal clock sync have happened between the nodes.
- exBrconAggMaxTaExRoundtrip (Integer32) Ping time between the nodes, in milliseconds.
- Table index is composed of: exBrconAggAgNodeid, exBrconAggTaExNodeid, exBrconAggTaExRemnodeid
Enduro/X eesnmpagx(8) support following traps:
enduroxUlogNotification - ULOG entry posting
Trap support is enabled in sub-agent by setting parameter trap_enable=1 in the [@eesnmpagx] section.
The notification is triggered by ULOG entries in the Enduro/X system and contains information such as the local date and time of the alert, the process ID, program name, and message text of the alert.
Trap message contains following parameters:
- exUlogAlertAgNodeId (Integer32(1..255)) Enduro/X node id on which eesnmpagx runs.
- exUlogAlertLmid (Integer32(1..255)) Enduro/X node id from which ULOG entry is read.
- exUlogAlertSeverity (Integer32) Severity codes: 1 - fatal, 2 - error, 3 - warning, 4 - info, 5 - debug. Severity level is extracted by searching following case insensitive keywords in ulog message: "fatal" (1), "error" (2), "fail" (2), "died" (2), "warn" (3), "warning" (3): "info" (4), "debug" (5), "trace" (5). Default level if no keyword is found, is 4 - info.
- exUlogAlertDate (Integer32) - Locate Unix date when alert occurred, format YYYYMMDD.
- exUlogAlertTime (Integer32) - Local Unix time when alert occurred, format HHMISS.
- exUlogAlertPid (Integer32) - PID of program which generated ULOG entry.
- exUlogAlertProg (OCTET STRING) - Name of the binary which generated ULOG entry.
- exUlogAlertMessage (OCTET STRING) - ULOG message text.
As SNMP AgentX does not allow the same SNMP (by the OID) object to generated by several programs running on the same server (at-least Net-SNMP snmpd does not allow same OID to be hosted by several AgentX processes). However it is not un-usual when several Enduro/X instances run on the same server, and it might be required that all these instances report their status over the SNMP channel. To overcome this limitation, eesnmpagx process can collect the statistics from different tpadmsv(8) process instances which maybe located on other Enduro/X cluster nodes nodes. The eesnmpagx configuration parameter sourcesvc can be set to list of .TMIB-NNN services from which to collect the information. Information is collected and merged into SNMP tables as described here. Cluster node numbers will appear in all data tables in columns ex<Class>AgNodeid or ex<Class>AggAgNodeid, from that actual Enduro/X node id can be distinguished. For example, having sourcesvc=.TMIB-1, .TMIB-2, .TMIB-3, would indicate that data needs to be collected from nodes 1, 2, 3.
To enable such data collection mode, cluster shall be configured accordingly and required tpbridge(8) links shall be established to connect other nodes to the main sub-agentx node which actually reports to the SNMP Agent.
In case if planning to send traps notification, then eesnmpagx processes shall be started on other nodes too. In such case no classes shall be monitored by these other nodes i.e. setting classes=- must be present for other nodes.
Another option to monitor several instances of Enduro/X on the same host via SNMP, multiple instances of snmpd could be configured & started. Each snmpd instance could listen on different UDP / TCP ports. The snmpd could be event configured to be started from cpmsrv(8).
The admin guide for Enduro/X Enterprise Edition (EE) module provides essential instructions on how to configure and effectively use the module. As of now, the module primarily offers an SNMP (Simple Network Management Protocol) extension, which allows for efficient network management and monitoring capabilities. The guide caters to users across different operating systems, ensuring compatibility with Linux, AIX, FreeBSD, and Solaris.
If you require further information or clarifications regarding the configuration and usage of the Enduro/X Enterprise Edition module, don’t hesitate to contact our dedicated support team.