fpmmm & Zabbix

Wed, 2017-02-22 20:09 — hranitel

Hello,

I installed fpmmm 1.0.1 to my system and try to start it with zabbix-agent (UserParameter=FromDual.MySQL.check,/opt/fpmmm/bin/fpmmm --config=/etc/fpmmm/fpmmm.conf). Trigger fpmmm Agent is down not working, and all data update with 10 minutes interval or doesn’t update at all.

I receive this in fpmmm log:

28964:2017-02-21 12:19:18.812 - ERR :   Agent was running for more than 60 seconds (now: 2017-02-21 12:19:18, mtime: 2017-02-21 12:09:18). Killing it now! (rc=1101).
28964:2017-02-21 12:19:18.812 - ERR :   Kill (-TERM) agent with PID 28383 (our PID is 28964).
28964:2017-02-21 12:19:18.812 - ERR :   Kill (-KILL) agent with PID 28383 (our PID is 28964).
29493:2017-02-21 12:28:02.194 - INFO: FromDual Performance Monitor for MySQL and MariaDB (fpmmm) (1.0.1) run started.
29493:2017-02-21 12:28:02.198 - WARN:   Agent Lock file /var/run/fpmmm/fpmmm.lock already exists.
29493:2017-02-21 12:28:02.199 - ERR :   Agent was running for more than 60 seconds (now: 2017-02-21 12:28:02, mtime: 2017-02-21 12:19:18). Killing it now! (rc=1101).
29493:2017-02-21 12:28:02.199 - ERR :   Kill (-TERM) agent with PID 28964 (our PID is 29493).
29493:2017-02-21 12:28:02.199 - ERR :   Kill (-KILL) agent with PID 28964 (our PID is 29493).

One interesting thing, when i run in console any fpmmm.php command i see additional “
n”:

$ ./fpmmm.php --version

1.0.1

Maybe that is the problem

Best regards, Vadim

tags: 
mpm general fpmmm general

Comments

Hello Vadim

Thank you for your suggestion. I am pretty sure that the empty line is not the problem. But I admit that it is a bug and it will be fixed in the next release (in fact it is already fixed in our internal revision 763). But it is IMHO not critical and will not affect your work.

# ./fpmmm-1.0.1/bin/fpmmm --version

1.0.1

# ./fpmmm-rev763/bin/fpmmm --version
rev763

# ./fpmmm-1.0.1/bin/fpmmm --config=/etc/fpmmm.conf 

1
# ./fpmmm-rev763/bin/fpmmm --config=/etc/fpmmm.conf 
1

What in fact is more relevant is the log file:

 7523:2017-02-24 08:41:38.432 - INFO: FromDual Performance Monitor for MySQL and MariaDB (fpmmm) (1.0.1) run started.
 7523:2017-02-24 08:41:40.941 - INFO: FromDual Performance Monitor for MySQL and MariaDB (fpmmm) run finished (rc=0).

 7723:2017-02-24 08:41:53.136 - INFO: FromDual Performance Monitor for MySQL and MariaDB (fpmmm) (rev763) run started.
 7723:2017-02-24 08:41:55.526 - INFO: FromDual Performance Monitor for MySQL and MariaDB (fpmmm) run finished (rc=0).

It shows that both versions run fine. So the problem must be somewhere else! The symptoms you described looks like the fpmmm agent died without leaving any notice... We have to find, where this is. I thought we have fixed all theses situations but it looks like we did not get all of them!
Can you please set LogLevel to 4 and send us the output of the dying fpmmm agent?

Thanks, Oli

Shinguz comment Fri, 2017-02-24 08:52

Hello, thank you for reply.

  1. About additional “\n” - in myEnv.inc file, in the end 2 empty lines.
  2. Problem was in zabbix-agentd.conf. FPMMM need more time to run. So I add “Timeout=30” variable, to fix the problem

Also, set in php.ini variables_order = “EGPCS” for all server(i don’t have dedicated server for mysql) is not good idea. So i changed first string of fpmmm.php to this #!/usr/bin/php -d variables_order=EGPCS

In debug logs i see something strange: 15604:2017-03-01 21:46:33.251 - DBG : Check connection to zabbix server. 15604:2017-03-01 21:46:33.253 - DBG : /usr/bin/zabbix_sender –zabbix-server XXX.XXX.XXX.XXXX –port 10051 –host ‘my-server’ –key FromDual.server.check –value ‘1’ -vv 15604:2017-03-01 21:46:33.385 - DBG : (ret=2). 15604:2017-03-01 21:46:33.385 - DBG : info from server: “processed: 0; failed: 1; total: 1; seconds spent: 0.000009” 15604:2017-03-01 21:46:33.385 - DBG : sent: 1; skipped: 0; total: 1 15604:2017-03-01 21:46:33.385 - DBG : Partial success (ret=2 / rc=0). 15604:2017-03-01 21:46:33.385 - DBG : Connection to zabbix server seems OK.

key FromDual.server.check is not exist in template ( i use zabbix 3.0 templates). Can you check it?

hranitel comment Fri, 2017-03-03 19:23

Hello hranitel

Thanks for your comments.

to 1) it will be fixed in next release in fact it is already fixed in rev763.

to 2) If the fpmmm/Zabbix agent in total takes more than 30 seconds something (with your network?) is wrong. Enable the LogLevel to 4 and check where the time has gone. We had something already earlier with network issues.

to variables_order: Your suggestion is considered and will be introduced into the next version.

to key FromDual.server.check: Something similar I have seen last week. Is it possible that you configured a SPACE in your fpmmm host or database name? That would explain it. As a workaround use a hostname without a SPACE (bad behavior anyway!) Nevertheless I think this should be considered a bug and must be fixed.

Regards, Shinguz

Shinguz comment Wed, 2017-03-08 17:56

My suggestion is that the fpmmm when user with zabbix agent dot have enough time to complete all cycle. Zabbix show me “Timeout while executing a shell script.” on FromDual.MySQL.check variable.

LOG: https://www.dropbox.com/s/zpojyrwz1va259l/fpmmm.log?dl=0

I don’t have space in hostname. This is my fpmmm.conf:

# Put everything related to fpmmm agent/daemon itself into this section [self]

How long fpmmm should sleep when run as daemon

Interval = 60 PidFile = /var/run/zabbix/fpmmm.pid #PidFile = /tmp/fpmmm.pid

Put everything in here which belongs to all instances (= hosts)

[default]

Type of section:

mysqld for a normal MySQL database

host for a server

#Type = mysqld

LogLevels are:

1 is logging only errors

2 is logging errors and warnings (this is the default)

3 logs some informations messages as well

4 logs everything (for debugging purposes only)

LogLevel = 2 LogFile = /var/log/zabbix/fpmmm.log

Directory where the cache files should be written to:

CacheFileBase = /var/log/zabbix/fpmmm AgentLockFile = /var/log/zabbix/fpmmm.lock

If you agent is not located in the same time zone as your server

For US something like -6.0 to our MaaS Server

For Asia something like +7.0 to our MaaS Server

TimeShift = +1.0

Information for MySQL connections:

Username = fpmmm Password = xxxxxxxxxx MysqlHost = 127.0.0.1 MysqlPort = 3306

Zabbix Server IP address

ZabbixServer = XX.XX.XX.XX

If section is disabled

Disabled = false

Possible modules for databases are:

aria galera innodb master myisam mysql process security slave

Possible modules for servers are:

fpmmm server drbd

Other modules are:

memcached

Usual modules are

Modules = innodb master myisam mysql process security galera

Special parameter which is used for module process (MySQL PID file!)

PidFile = /var/run/mariadb/mariadb.pid

If you want to use Monitoring as a Service:

MaaS = on

Hash =

Methodes are: direct, https, email. Currently only http works

Methode = https

Url = https://support.fromdual.com/maas/receiver.php

Here you could enter the hostname of your machine (uname -n)

#[Server1] # This MUST match Hostname in Zabbix!

#Type = host #Type = mysqld #Modules = fpmmm server

All MySQL databases here

Try to avoid section names with spaces!

Here you could add a random name of your MySQL/MariaDB instance

[Server1] # This MUST match Hostname in Zabbix!

Type = mysqld

MysqlPort = 3306 Modules = process mysql myisam innodb security server fpmmm PidFile = /var/run/mariadb/mariadb.pid

[memcached] # This MUST match Hostname in Zabbix! Disabled = true Type = memcached Modules = memcached

MemcachedHost = 127.0.0.1 MemcachedPort = 11211

hranitel comment Wed, 2017-03-08 21:58

Your log file does not show any reason:

6644:2017-03-08 23:13:02.115 - INFO: FromDual Performance Monitor for MySQL and MariaDB (fpmmm) (1.0.1) run started. 6644:2017-03-08 23:13:02.809 - DBG : /usr/bin/zabbix_sender –zabbix-server

It took only 700 ms in the output. So this cannot be a reason for a timeout of 30 seconds.

I will check with your configuration file.

Regards, Shinguz

Shinguz comment Thu, 2017-03-09 10:05

Hello. Do you check my configuration?

hranitel comment Sun, 2017-04-02 20:34