News

From time to time we see in customer engagements that MySQL Master/Slave replication is set-up doing schema or table level replication filtering. This can be done either on Master or on Slave. If filtering is done on the Master (by the binlog_{do|ignore}_db settings), the binary log becomes incomplete and cannot be used for a proper Point-in-Time-Recovery. Therefore FromDual recommends AGAINST this approach.

The replication filtering rules vary depending on the binary log format (ROW and STATEMENT) See also: How Servers Evaluate Replication Filtering Rules.

For reasons of data consistency between Master and Slave FromDual recommends to use only the binary log format ROW. This is also stated in the MySQL documentation: All changes can be replicated. This is the safest form of replication. Especially dangerous is binary log filtering with binary log format MIXED. This binary log format FromDual strongly discourages users to use.

The binary log format ROW affects only DML statements (UPDATE, INSERT, DELETE, etc.) but NOT DDL statements (CREATE, ALTER, DROP, etc.) and NOT DCL statements (CREATE, GRANT, REVOKE, DROP, etc.). So how are those statements replicated? They are replicated in STATEMENT binary log format even though binlog_format is set to ROW. This has the consequences that the binary log filtering rules of STATEMENT based replication and not the ones of ROW based replication apply when running one of those DDL or DCL statements.

This can easily cause problems. If you are lucky, they will cause the replication to break sooner or later, which you can detect and fix - but they may also cause inconsistencies between Master and Slave which may remain undetected for a long time.

Let us show what happens in 2 similar scenarios:

Scenario A: Filtering on mysql schema

On Slave we set the binary log filter as follows:

replicate_ignore_db = mysql

and verify it:

mysql> SHOW SLAVE STATUS\G ... Replicate_Ignore_DB: mysql ...

The intention of this filter setting is to not replicate user creations or modifications from Master to the Slave.

We verify on the Master, that binlog_format is set to the wanted value:

mysql> SHOW GLOBAL VARIABLES LIKE 'binlog_format'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | binlog_format | ROW | +---------------+-------+

Now we do the following on the Master:

mysql> use mysql mysql> CREATE USER 'inmysql'@'%'; mysql> use test mysql> CREATE USER 'intest'@'%';

and verify the result on the Master:

mysql> SELECT user, host FROM mysql.user; +-------------+-----------+ | user | host | +-------------+-----------+ | inmysql | % | | intest | % | | mysql.sys | localhost | | root | localhost | +-------------+-----------+

and on the Slave:

mysql> SELECT user, host FROM mysql.user; +-------------+-----------+ | user | host | +-------------+-----------+ | intest | % | | mysql.sys | localhost | | root | localhost | +-------------+-----------+

We see, that the user intest was replicated and the user inmysql was not. And we have clearly an unwanted data inconsistency between Master and Slave.

If we want to drop the inmysql user some time later on the Master:

mysql> use myapp; mysql> DROP USER 'inmysql'@'%';

we get the following error message on the Slave and are wondering, why this user or the query appears on the Slave:

mysql> SHOW SLAVE STATUS\G ... Last_SQL_Errno: 1396 Last_SQL_Error: Error 'Operation DROP USER failed for 'inmysql'@'%'' on query. Default database: 'test'. Query: 'DROP USER 'inmysql'@'%'' ...

A similar problem happens when we connect to NO database on the Master as follows and change the users password:

shell> mysql -uroot mysql> SELECT DATABASE(); +------------+ | database() | +------------+ | NULL | +------------+ mysql> ALTER USER 'innone'@'%' IDENTIFIED BY 'secret';

This works perfectly on the Master. But what happens on the Slave:

mysql> SHOW SLAVE STATUS\G ... Last_SQL_Errno: 1396 Last_SQL_Error: Error 'Operation ALTER USER failed for 'innone'@'%'' on query. Default database: ''. Query: 'ALTER USER 'innone'@'%' IDENTIFIED WITH 'mysql_native_password' AS '*14E65567ABDB5135D0CFD9A70B3032C179A49EE7'' ...

The Slave wants to tell us in a complicated way, that the user innone does not exist on the Slave...

Scenario B: Filtering on tmp or similar schema

An other scenario we have seen recently is that the customer is filtering out tables with temporary data located in the tmp schema. Similar scenarios are cache, session or log tables. He did it as follows on the Master:

mysql> use tmp; mysql> TRUNCATE TABLE tmp.test;

As he has learned in FromDual trainings he emptied the table with the TRUNCATE TABLE command instead of a DELETE FROM tmp.test command which is much less efficient than the TRUNCATE TABLE command. What he did not consider is, that the TRUNCATE TABLE command is a DDL command and not a DML command and thus the STATEMENT based replication filtering rules apply. His filtering rules on the Slave were as follows:

mysql> SHOW SLAVE STATUS\G ... Replicate_Ignore_DB: tmp ...

When we do the check on the Master we get an empty set as expected:

mysql> SELECT * FROM tmp.test; Empty set (0.00 sec)

When we add new data on the Master:

mysql> INSERT INTO tmp.test VALUES (NULL, 'new data', CURRENT_TIMESTAMP()); mysql> SELECT * FROM tmp.test; +----+-----------+---------------------+ | id | data | ts | +----+-----------+---------------------+ | 1 | new data | 2017-01-11 18:00:11 | +----+-----------+---------------------+

we get a different result set on the Slave:

mysql> SELECT * FROM tmp.test; +----+-----------+---------------------+ | id | data | ts | +----+-----------+---------------------+ | 1 | old data | 2017-01-11 17:58:55 | +----+-----------+---------------------+

and in addition the replication stops working with the following error:

mysql> SHOW SLAVE STATUS\G ... Last_Errno: 1062 Last_Error: Could not execute Write_rows event on table tmp.test; Duplicate entry '1' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log laptop4_qa57master_binlog.000042, end_log_pos 1572 ...

See also our earlier bug report of a similar topic: Option "replicate_do_db" does not cause "create table" to replicate ('row' log)

Conclusion

Binary log filtering is extremely dangerous when you care about data consistency and thus FromDual recommends to avoid binary log filtering by all means. If you really have to do binary log filtering you should exactly know what you are doing, carefully test your set-up, check your application and your maintenance jobs and also review your future code changes regularly. Otherwise you risk data inconsistencies in your MySQL Master/Slave replication.

Taxonomy upgrade extras: replicationbinary logfilterfilteringrow filteringstatementbinlog_formatrow

FromDual has the pleasure to announce the release of the new version 1.2.4 of the popular Backup and Recovery Manager for MySQL and MariaDB (fromdual_brman).

You can download the FromDual Backup and Recovery Manager from here.

In the inconceivable case that you find a bug in the Backup and Recovery Manager please report it to our Bugtracker or just send us an email.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.2.x to 1.2.4 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.4.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.4 fromdual_brman
Changes in FromDual Backup Manager 1.2.4

This release contains mainly fixes related to the backup type cleanup.

You can verify your current FromDual Backup Manager version with the following command:

shell> fromdual_bman --version
FromDual Backup Manager

• Library myEnv.inc synced from myEnv project.

FromDual Backup Manager Cleanup

• Ignore warning for non empty backup directories in cleanup.
• Check if directory exist before going into recursive lookup in cleanup.
• Function rmdir is also covered by simulate option now in cleanup.
• Backup type cleanup also considers physical backup and catalog now.

Taxonomy upgrade extras: BackupRestoreRecoverypitrfromdual_brmanbrmanreleasebmanrman

FromDual has the pleasure to announce the release of the new version 1.2.4 of the popular Backup and Recovery Manager for MySQL and MariaDB (fromdual_brman).

You can download the FromDual Backup and Recovery Manager from here.

In the inconceivable case that you find a bug in the Backup and Recovery Manager please report it to our Bugtracker or just send us an email.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.2.x to 1.2.4 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.4.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.4 fromdual_brman
Changes in FromDual Backup Manager 1.2.4

This release contains mainly fixes related to the backup type cleanup.

You can verify your current FromDual Backup Manager version with the following command:

shell> fromdual_bman --version
FromDual Backup Manager

• Library myEnv.inc synced from myEnv project.

FromDual Backup Manager Cleanup

• Ignore warning for non empty backup directories in cleanup.
• Check if directory exist before going into recursive lookup in cleanup.
• Function rmdir is also covered by simulate option now in cleanup.
• Backup type cleanup also considers physical backup and catalog now.

Taxonomy upgrade extras: BackupRestoreRecoverypitrfromdual_brmanbrmanreleasebmanrman

FromDual has the pleasure to announce the release of the new version 1.2.4 of the popular Backup and Recovery Manager for MySQL and MariaDB (fromdual_brman).

You can download the FromDual Backup and Recovery Manager from here.

In the inconceivable case that you find a bug in the Backup and Recovery Manager please report it to our Bugtracker or just send us an email.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.2.x to 1.2.4 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.4.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.4 fromdual_brman
Changes in FromDual Backup Manager 1.2.4

This release contains mainly fixes related to the backup type cleanup.

You can verify your current FromDual Backup Manager version with the following command:

shell> fromdual_bman --version
FromDual Backup Manager

• Library myEnv.inc synced from myEnv project.

FromDual Backup Manager Cleanup

• Ignore warning for non empty backup directories in cleanup.
• Check if directory exist before going into recursive lookup in cleanup.
• Function rmdir is also covered by simulate option now in cleanup.
• Backup type cleanup also considers physical backup and catalog now.

Taxonomy upgrade extras: BackupRestoreRecoverypitrfromdual_brman

This one is for all MySQL-DBA's, which are working on macOS. Since the Apple OS has a rather peculiar way of starting and stopping MySQL, compared to Linux, you can run into some issues. These problems occur especially, if you have no access to the GUI.

Preparation

Put skip-grant-tables into the mysqld section of the my.cnf. A my.cnf can be found in /usr/local/mysql/support-files. You MUST work as root for all the following steps.

shell> sudo -s shell> vi /usr/local/mysql/support-files/my-default.cnf ... [mysqld] skip-grant-tables skip-networking ...

Save the configuration file! (In vi this is "[ESC] + :x")

Continue with stopping MySQL:

launchctl unload /Library/LaunchDaemons/com.oracle.oss.mysql.mysqld.plist

Restart MySQL, so skip-grant-tables becomes active:

launchctl load /Library/LaunchDaemons/com.oracle.oss.mysql.mysqld.plist

Reset the password

After MySQL is started again, you can log into the CLI and reset the password:

shell> mysql -u root mysql> FLUSH PRIVILEGES; mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'super-secret-password';

Plan B

If you are not capable of stopping MySQL in a civilised manner, you can use the more rough way. You can send a SIGTERM to the MySQL-Server:

shell> ps -aef | grep mysql | grep -v grep 74 28017 1 0 Fri10AM ?? 5:59.50 /usr/local/mysql/bin/mysqld --user=_mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data --plugin-dir=/usr/local/mysql/lib/plugin --log-error=/usr/local/mysql/data/mysqld.local.err --pid-file=/usr/local/mysql/data/mysqld.local.pid

You should receive one line. The second column from the left is the process id. Use this process id to stop the MySQL-Server.

shell> kill -15 [process id]

In this example, the command would look like this:

shell> kill -15 28017

macOS will restart MySQL, since the process has not stopped correctly. The configuration will be read and the changes to the parameters will become effective. Continue with logging in to the CLI.

Conclusion

No matter how secure your MySQL-Password is, it is a lot more important to secure access to the server it self. If your server is not secured by something that prevents access from the internet, it will only take a few minutes for someone with bad intentions to take over your database or worse, the entire server.

Taxonomy upgrade extras: mysqlserver

This one is for all MySQL-DBA's, which are working on macOS. Since the Apple OS has a rather peculiar way of starting and stopping MySQL, compared to Linux, you can run into some issues. These problems occur especially, if you have no access to the GUI.

Preparation

Put skip-grant-tables into the mysqld section of the my.cnf. A my.cnf can be found in /usr/local/mysql/support-files. You MUST work as root for all the following steps.

shell> sudo -s shell> vi /usr/local/mysql/support-files/my-default.cnf ... [mysqld] skip-grant-tables skip-networking ...

Save the configuration file! (In vi this is "[ESC] + :x")

Continue with stopping MySQL:

launchctl unload /Library/LaunchDaemons/com.oracle.oss.mysql.mysqld.plist

Restart MySQL, so skip-grant-tables becomes active:

launchctl load /Library/LaunchDaemons/com.oracle.oss.mysql.mysqld.plist

Reset the password

After MySQL is started again, you can log into the CLI and reset the password:

shell> mysql -u root mysql> FLUSH PRIVILEGES; mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'super-secret-password';

Plan B

If you are not capable of stopping MySQL in a civilised manner, you can use the more rough way. You can send a SIGTERM to the MySQL-Server:

shell> ps -aef | grep mysql | grep -v grep 74 28017 1 0 Fri10AM ?? 5:59.50 /usr/local/mysql/bin/mysqld --user=_mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data --plugin-dir=/usr/local/mysql/lib/plugin --log-error=/usr/local/mysql/data/mysqld.local.err --pid-file=/usr/local/mysql/data/mysqld.local.pid

You should receive one line. The second column from the left is the process id. Use this process id to stop the MySQL-Server.

shell> kill -15 [process id]

In this example, the command would look like this:

shell> kill -15 28017

macOS will restart MySQL, since the process has not stopped correctly. The configuration will be read and the changes to the parameters will become effective. Continue with logging in to the CLI.

Conclusion

No matter how secure your MySQL-Password is, it is a lot more important to secure access to the server it self. If your server is not secured by something that prevents access from the internet, it will only take a few minutes for someone with bad intentions to take over your database or worse, the entire server.

Taxonomy upgrade extras: mysqlserver

This one is for all MySQL-DBA's, which are working on macOS. Since the Apple OS has a rather peculiar way of starting and stopping MySQL, compared to Linux, you can run into some issues. These problems occur especially, if you have no access to the GUI.

Preparation

Put skip-grant-tables into the mysqld section of the my.cnf. A my.cnf can be found in /usr/local/mysql/support-files. You MUST work as root for all the following steps.

shell> sudo -s shell> vi /usr/local/mysql/support-files/my-default.cnf ... [mysqld] skip-grant-tables skip-networking ...

Save the configuration file! (In vi this is "[ESC] + :x")

Continue with stopping MySQL:

launchctl unload /Library/LaunchDaemons/com.oracle.oss.mysql.mysqld.plist

Restart MySQL, so skip-grant-tables becomes active:

launchctl load /Library/LaunchDaemons/com.oracle.oss.mysql.mysqld.plist

Reset the password

After MySQL is started again, you can log into the CLI and reset the password:

shell> mysql -u root mysql> FLUSH PRIVILEGES; mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'super-secret-password';

Plan B

If you are not capable of stopping MySQL in a civilised manner, you can use the more rough way. You can send a SIGTERM to the MySQL-Server:

shell> ps -aef | grep mysql | grep -v grep 74 28017 1 0 Fri10AM ?? 5:59.50 /usr/local/mysql/bin/mysqld --user=_mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data --plugin-dir=/usr/local/mysql/lib/plugin --log-error=/usr/local/mysql/data/mysqld.local.err --pid-file=/usr/local/mysql/data/mysqld.local.pid

You should receive one line. The second column from the left is the process id. Use this process id to stop the MySQL-Server.

shell> kill -15 [process id]

In this example, the command would look like this:

shell> kill -15 28017

macOS will restart MySQL, since the process has not stopped correctly. The configuration will be read and the changes to the parameters will become effective. Continue with logging in to the CLI.

Conclusion

No matter how secure your MySQL-Password is, it is a lot more important to secure access to the server it self. If your server is not secured by something that prevents access from the internet, it will only take a few minutes for someone with bad intentions to take over your database or worse, the entire server.

Taxonomy upgrade extras: mysqlserver

What is SELinux?

The Security-Enhanced Linux is an extension to the Linux Kernel, made by the NSA (National Security Agency). It implements Mandatory Access Controls (MAC), which allow an administrator to define, how applications and users can access resources on a system.

There is more detail in the SELinux Wki: https://selinuxproject.org/page/FAQ
... and the CentOS documentation: https://wiki.centos.org/HowTos/SELinux

Some distributions have it installed by default, but not active, some have it installed and active and some don't have it installed.

How do I know if SELinux is active? SELinux comes with some new commands. To see the current status of SELinux, use "getenforce" or "sestatus": [root@localhost ~]# getenforce Enforcing

- OR -

[root@localhost ~]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: enforcing Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Max kernel policy version: 28

There are three modes available:

• Enforcing: SELinux is active and enforcing restrictions.
• Permissive: Restrictions are not enforced, but policy violations are reported.
• Disabled: SELinux is turned off.

Changing modes

If you want to change the mode of SELinux, use "setenforce":

setenforce [ Enforcing | Permissive | 1 | 0 ]

Or edit the configuration file under "/etc/selinux/config".

Install semanage

If you want to change SELinux policies in an easy way, you will need the tool "semanage" it can be installed with the following command:

yum install policycoreutils-python

Create a directory MySQL/MariaDB can access

NOTE: I am going to work with MariaDB for this blog, as it can be installed from repository in CentOS.

The easy way to create a new policy, which allows to MySQL or MariaDB to use a directory, is to install "semanage". It is provided with the following package:

yum install policycoreutils-python

Then proceed to create the new directory, where MySQL/MariaDB could store the binary logs, if they should not be in the datadir.

mkdir /var/lib/mysql_binlog/ chown -R mysql:mysql mysql* semanage fcontext -a -t mysqld_db_t "/var/lib/mysql_binlog(/.*)?" restorecon -Rv /var/lib/mysql_binlog

NOTE: You have to give the absolute path to the file or the directory!

If you want to use MySQL/MariaDB on a non-standard port, you also have to allow usage of that port:

semanage port -a -t mysqld_port_t -p tcp 3307

Once you have created the new directory for the binary logs and made sure it is owned by mysql, you need to change the type of the directory you created to the one that allows MySQL/MariDB to use this directory. If you do not do this, you will get a "Permission denied (13)" error.

"semanage" is used to make this change persistent, even when the entire file system relabelled.

I was although unable to change the socket. I am yet unsure what the problem was, as MariaDB did not start or return any error.

Enable MySQL to write to this directory vi /etc/my.cnf ... [mysqld] log-bin=/var/lib/mysql_binlog/binlog ... systemctl restart mariadb Taxonomy upgrade extras: mysqlmariadbcentossecurityselinux

What is SELinux?

The Security-Enhanced Linux is an extension to the Linux Kernel, made by the NSA (National Security Agency). It implements Mandatory Access Controls (MAC), which allow an administrator to define, how applications and users can access resources on a system.

There is more detail in the SELinux Wki: https://selinuxproject.org/page/FAQ
... and the CentOS documentation: https://wiki.centos.org/HowTos/SELinux

Some distributions have it installed by default, but not active, some have it installed and active and some don't have it installed.

How do I know if SELinux is active? SELinux comes with some new commands. To see the current status of SELinux, use "getenforce" or "sestatus": [root@localhost ~]# getenforce Enforcing

- OR -

[root@localhost ~]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: enforcing Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Max kernel policy version: 28

There are three modes available:

• Enforcing: SELinux is active and enforcing restrictions.
• Permissive: Restrictions are not enforced, but policy violations are reported.
• Disabled: SELinux is turned off.

Changing modes

If you want to change the mode of SELinux, use "setenforce":

setenforce [ Enforcing | Permissive | 1 | 0 ]

Or edit the configuration file under "/etc/selinux/config".

Install semanage

If you want to change SELinux policies in an easy way, you will need the tool "semanage" it can be installed with the following command:

yum install policycoreutils-python

Create a directory MySQL/MariaDB can access

NOTE: I am going to work with MariaDB for this blog, as it can be installed from repository in CentOS.

The easy way to create a new policy, which allows to MySQL or MariaDB to use a directory, is to install "semanage". It is provided with the following package:

yum install policycoreutils-python

Then proceed to create the new directory, where MySQL/MariaDB could store the binary logs, if they should not be in the datadir.

mkdir /var/lib/mysql_binlog/ chown -R mysql:mysql mysql* semanage fcontext -a -t mysqld_db_t "/var/lib/mysql_binlog(/.*)?" restorecon -Rv /var/lib/mysql_binlog

NOTE: You have to give the absolute path to the file or the directory!

If you want to use MySQL/MariaDB on a non-standard port, you also have to allow usage of that port:

semanage port -a -t mysqld_port_t -p tcp 3307

Once you have created the new directory for the binary logs and made sure it is owned by mysql, you need to change the type of the directory you created to the one that allows MySQL/MariDB to use this directory. If you do not do this, you will get a "Permission denied (13)" error.

"semanage" is used to make this change persistent, even when the entire file system relabelled.

I was although unable to change the socket. I am yet unsure what the problem was, as MariaDB did not start or return any error.

Enable MySQL to write to this directory vi /etc/my.cnf ... [mysqld] log-bin=/var/lib/mysql_binlog/binlog ... systemctl restart mariadb Taxonomy upgrade extras: mysqlmariadbcentossecurityselinux

What is SELinux?

The Security-Enhanced Linux is an extension to the Linux Kernel, made by the NSA (National Security Agency). It implements Mandatory Access Controls (MAC), which allow an administrator to define, how applications and users can access resources on a system.

There is more detail in the SELinux Wki: https://selinuxproject.org/page/FAQ
... and the CentOS documentation: https://wiki.centos.org/HowTos/SELinux

Some distributions have it installed by default, but not active, some have it installed and active and some don't have it installed.

How do I know if SELinux is active? SELinux comes with some new commands. To see the current status of SELinux, use "getenforce" or "sestatus": [root@localhost ~]# getenforce Enforcing

- OR -

[root@localhost ~]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: enforcing Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Max kernel policy version: 28

There are three modes available:

• Enforcing: SELinux is active and enforcing restrictions.
• Permissive: Restrictions are not enforced, but policy violations are reported.
• Disabled: SELinux is turned off.

Changing modes

If you want to change the mode of SELinux, use "setenforce":

setenforce [ Enforcing | Permissive | 1 | 0 ]

Or edit the configuration file under "/etc/selinux/config".

Install semanage

If you want to change SELinux policies in an easy way, you will need the tool "semanage" it can be installed with the following command:

yum install policycoreutils-python

Create a directory MySQL/MariaDB can access

NOTE: I am going to work with MariaDB for this blog, as it can be installed from repository in CentOS.

The easy way to create a new policy, which allows to MySQL or MariDB to use a directory, is to install "semanage". It is provided with the following package:

yum install policycoreutils-python

Then proceed to create the new directory, where MySQL/MariaDB could store the binary logs, if they should not be in the datadir.

mkdir /var/lib/mysql_binlog/ chown -R mysql:mysql mysql* semanage fcontext -a -t mysqld_db_t "/var/lib/mysql_binlog(/.*)?" restorecon -Rv /var/lib/mysql_binlog

NOTE: You have to give the absolute path to the file or the directory!

If you want to use MySQL/MariaDB on a non-standard port, you also have to allow usage of that port:

semanage port -a -t mysqld_port_t -p tcp 3307

Once you have created the new directory for the binary logs and made sure it is owned by mysql, you need to change the type of the directory you created to the one that allows MySQL/MariDB to use this directory. If you do not do this, you will get a "Permission denied (13)" error.

"semanage" is used to make this change persistent, even when the entire file system relabelled.

I was although unable to change the socket. I am yet unsure what the problem was, as MariaDB did not start or return any error.

Enable MySQL to write to this directory vi /etc/my.cnf ... [mysqld] log-bin=/var/lib/mysql_binlog/binlog ... systemctl restart mariadb Taxonomy upgrade extras: mysqlmariadbcentos

MySQL is well known and widely spread because of its philosophy of Keep it Simple (KISS).

We recently had the discussion that with newer releases also MySQL and MariaDB relational databases becomes more and more complicated.

One indication for this trend is the number of MySQL server system variables and status variables.

In the following tables and graphs we compare the different releases since MySQL version 4.0:

mysql> SHOW GLOBAL VARIABLES; mysql> SHOW GLOBAL VARIABLES LIKE 'innodb%'; mysql> SHOW GLOBAL STATUS; mysql> SHOW GLOBAL STATUS LIKE 'innodb%';
VersionSystemIB Sys.StatusIB Stat.MySQL 4.0.3014322*133**0MySQL 4.1.2518926*164**0MySQL 5.0.962393625242MySQL 5.1.732773629142MySQL 5.5.513176031247MySQL 5.6.3143812034151MySQL 5.7.1549113135351MySQL 8.0.048812436351

* Use SHOW STATUS instead.
** Use SHOW ENGINE INNODB STATUS\G instead.

VersionSystemIB Sys.StatusIB Stat.MariaDB 5.1.443547230144MariaDB 5.2.103978632446MariaDB 5.5.4141910341399MariaDB 10.0.2153714745595MariaDB 10.1.18***589178517127MariaDB 10.2.2****58616448196

*** XtraDB 5.6
****InnoDB 5.7.14???

Taxonomy upgrade extras: mysqlvariablesstatusmariadb

MySQL is well known and widely spread because of its philosophy of Keep it Simple (KISS).

We recently had the discussion that with newer releases also MySQL and MariaDB relational databases becomes more and more complicated.

One indication for this trend is the number of MySQL server system variables and status variables.

In the following tables and graphs we compare the different releases since MySQL version 4.0:

mysql> SHOW GLOBAL VARIABLES; mysql> SHOW GLOBAL VARIABLES LIKE 'innodb%'; mysql> SHOW GLOBAL STATUS; mysql> SHOW GLOBAL STATUS LIKE 'innodb%';
VersionSystemIB Sys.StatusIB Stat.MySQL 4.0.3014322*133**0MySQL 4.1.2518926*164**0MySQL 5.0.962393625242MySQL 5.1.732773629142MySQL 5.5.513176031247MySQL 5.6.3143812034151MySQL 5.7.1549113135351MySQL 8.0.048812436351

* Use SHOW STATUS instead.
** Use SHOW ENGINE INNODB STATUS\G instead.

VersionSystemIB Sys.StatusIB Stat.MariaDB 5.1.443547230144MariaDB 5.2.103978632446MariaDB 5.5.4141910341399MariaDB 10.0.2153714745595MariaDB 10.1.18***589178517127MariaDB 10.2.2****58616448196

*** XtraDB 5.6
****InnoDB 5.7.14???

Taxonomy upgrade extras: mysqlvariablesstatusmariadb

As you probably know MySQL is an Open Source product licensed under the GPL v2. The GPL grants you the right to not just read and understand the code of the product but also to use, modify AND redistribute the code as long as you follow the GPL rules.

This redistribution has happened in the past various times. But in the western hemisphere only 3 of these branches/forks of MySQL are of relevance for the majority of the MySQL users: Galera Cluster for MySQL, MariaDB (Server and Galera Cluster) and Percona Server (and XtraDB Cluster).

Now it happened what has to happen in nature: The different branches/forks start to diverge (following the marketing rule: differentiate yourself from your competitors). The biggest an most important divergence happens now between MySQL and MariaDB.

Recently a customer of FromDual claimed that there is no more progress in the MySQL Server development whereas the MariaDB Server does significant progress. I was wondering a bit how this statement could have been made. So I try to summarize the New Features which have been added since the beginning of the separation starting with MySQL 5.1.

It is important to know, that some parts of MySQL code are directly or in modified form ported to MariaDB whereas some MariaDB features were implemented in MySQL as well. So missing features in MariaDB or improvements in MySQL can possibly make it sooner or later also into MariaDB and vice versa. Further both forks were profiting significantly from old MySQL 6.0 code which was never really announced broadly.

Further to consider: Sun Microsystems acquired MySQL in January 2008 (MySQL 5.1.23 was out then and MySQL 5.2, 5.4 and 6.0 were in the queue) and Sun was acquired by Oracle in January 2010 (MySQL 5.1.43, MySQL 5.5.1 were out, MySQL 5.2, 5.4 and 6.0 were abandoned and MySQL 5.6 was in the queue).

MySQL 5.1 MariaDB 5.1 (link), 5.2 (link) and 5.3 (link)

• Partitioning
• Row-based replication
• Plug-in API
• Event scheduler.
• Server log tables.
• Upgrade program mysql_upgrade.
• Improvements to INFORMATION_SCHEMA.
• XML functions with Xpath support.

MariaDB 5.1

• Storage Engines
  • Aria (Crash-safe MyISAM)
  • XtraDB plug-in (Branch of InnoDB)
  • PBXT (transactional Storage Engine)
  • Federated-X (replacement for Federated).

• Performance
  • Faster CHECKSUM TABLE.
  • Character Set conversion improvement/elimination.
  • Speed-up of complex queries using Aria SE for temporary tables.
  • Optimizer: Table elimination.

• Upgrade from MySQL 5.0 improved.
• Better testing.
• Microseconds precision in PROCESSLIST.

MariaDB 5.2

• Storage Engines
  • OQGRAPH (Graph SE)
  • SphinxSE (Full-text search engine)

• Performance
  • Segmented MyISAM key cache (instances)
  • Group Commit for Aria SE

• Security
  • Pluggable Authentication

• Virtual columns
• Extended user statistics
• Storage Engine specific CREATE TABLE
• Enhancements to INFORMATION_SCHEMA.PLUGINS table

MariaDB 5.3

• Performance
  • Subquery Optimization
    • Semi-join subquery optimizations
    • Non-semi-join optimizations
    • Subquery Cache
    • Subquery is not materialized any more in EXPLAIN
  • Optimization for derived tables and views
    • No early materialization of derived tables
    • Derived Table Merge optimization
    • Derived Table with Keys optimization
    • Fields of mergeable views and derived tables are involved in optimization
  • Disk access optimization
    • Index Condition Pushdown (ICP)
    • Multi-Range-Read optimization (MRR)
  • Join optimizations
    • Block-based Join Algorithms: Block Nested Loop (BNL) for outer joins, Block Hash Joins, Block Index Joins (Batched Key Access (BKA) Joins)
  • Index Merge improvements

• Replication
  • Group Commit for Binary Log
  • Annotation of row-based replication events with the original SQL statement
  • Checksum for binlog events
  • Enhancements for START TRANSACTION WITH CONSISTENT SNAPSHOT
  • Performance improvement for row-based replication for tables with no primary key

• Handler Socket Interface included.
• HANDLER READ works with prepared statements
• Dynamic Column support for Handler Interface
• Microsecond support
• CAST extended
• Windows performance improvements
• New status variables
• Progress reports for some operations
• Enhanced KILL command

MySQL 5.5 (link) MariaDB 5.5 (link)

• InnoDB
  • InnoDB Version 5.5
  • Default storage engine switched to InnoDB.
  • InnoDB fast INDEX DROP/CREATE feature added.
  • Multi-core scalability. Focus on InnoDB, especially locking and memory management.
  • Optimizing InnoDB I/O subsystem to more effective use of available I/O capacity.

• Performance
  • MySQL Thread Pool plug-in (Enterprise)

• Security
  • MySQL Audit plug-in (Enterprise)
  • MySQL pluggable authentication (Enterprise) for LDAP, Kerberos, PAM and Windows login

• Replication
  • Semi-synchronous replication.

• Partitioning
  • 2 new partition types (RANGE COLUMNS, LIST COLUMNS).
  • TRUNCATE PARTITION.

• Proxy Users
• Diagnostic improvements to better access execution an performance information including PERFORMANCE_SCHEMA, expanded SHOW ENGINE INNODB STATUS output and new status variables.
• Supplementary Unicode characters (utf16, utf32, utf8mb4).
• CACHE INDEX and LOAD INDEX INTO CACHE for partitioned MyISAM tables.
• Condition Handling: SIGNAL and RESIGNAL.
• Introduction of Metadata locking to prevent DDL statements from compromising transactions serializability.
• IPv6 Support
• XML enhancement LOAD_XML_INFILE.
• Build chain switched to CMake to ease build on other platforms including Windows.
• Deprecation and remove of features.

• Storage Engines
  • SphinxSE updated to 2.0.4
  • PBXT Storage Engine is deprecated.

• XtraDB
  • MariaDB uses XtraDB 5.5 as compiled in SE and InnoDB 5.5 as plug-in.
  • Extended Keys support for XtraDB

• Performance
  • Thread pool plug-in
  • Non-blocking client API Library

• Replication
  • Updates on P_S tables are not logged to binary log.
  • replicate_* variables are dynamically.
  • Skip_replication option

• LIMIT ROWS EXAMINED
• New status variables for features.
• New plug-in to log SQL level errors.

MySQL 5.6 (link) MariaDB 10.0 (link)

• InnoDB
  • InnoDB Version 5.6
  • InnoDB full-text search.
  • InnoDB transportable tablespace support
  • Different InnoDB pages size implementation (4k, 8k, 16k)
  • Improvement of InnoDB adaptive flushing algorithm to make I/O more efficient.
  • NoSQL style Memcached API to access InnoDB data.
  • InnoDB optimizer persistent statistics.
  • InnoDB read-only transactions.
  • Separating InnoDB UNDO tablespace from system tablespace.
  • Maximum InnoDB transaction log size increased from 4G to 512G.
  • InnoDB read-only capability for read-only media (CD, DVD, etc.)
  • InnoDB table compression.
  • New InnoDB meta data table in INFORMATION_SCHEMA.
  • InnoDB internal performance enhancements.
  • Better InnoDB deadlock detection algorithm. Deadlock can be written to MySQL error log.
  • InnoDB buffer pool state saving and restoring capabilities.
  • InnoDB Monitor dynamially disable/enable.
  • Online and inplace DDL operations for normal and partitioned InnoDB Tables to reduce application downtime.

• Optimizer
  • ORDER BY non-index-column for simple queries and subqueries
  • Disk-Sweep Multi-Range Read (MRR) optimization for secondary index/table access to reduce I/O
  • Index Condition Pushdown (ICP) optimization by pushing down the WHERE filter to the storage engine.
  • EXPLAIN also works for DML statemetns.
  • Optimizing of subqueries in derived tables (FROM (...)) by postponing or indexing deived tables.
  • Implementation of semi-join and materialization strategies to optimize subquery execution.
  • Batched Key Access (BKA) join algorithm to improve join performance during table scanning.
  • Optimizer trace capabilities.

• Performance Schema (P_S)
  • Instrumentation for Statements and stages
  • Configuration of consumers at server startup
  • Summary tables for table and index I/O and for table locks
  • Event filtering by table
  • Various new instrumentation.

• Security
  • Encrypted authentication credentials
  • Stronger encryption for passwords (SHA-256 authentication plugin)
  • MySQL User password expiration.
  • Password validation plugin to check password strength
  • mysql_install_db can create secure root password by default
  • cleartext password is not written to any log file any more.
  • MySQL Firewall (Enterprise)

• Replication
  • Transaction based replication using global transaction identifiers (GTID)
  • Row Image Control to reduce binary log volume.
  • Crash-safe replication with checksumming and verfiying.
  • IO and SQL thread information can be stored in an transactional table inside the DB.
  • MySQL binlog streaming with mysqlbinlog possible.
  • Delayed replication
  • Parallel replication on schema level.

• Partitioning
  • Number of partitions including subpartitions increased to 8192.
  • Exchange partition with a normal table.
  • Explicit selection of specific partiton is possible.
  • Partition lock prunining for DML and DDL statements.

• Condition handling: GET DIAGNOSTICS and SET DIAGNOSTICS
• Server defaults changes.
• Data types TIME, DATETIME and TIMESTAMP with microseconds
• Host cache exposure and connection errors status infromation for finding connection problems.
• Improvement in GIS functions.
• Deprecation and remove of features.

• Storage Engine
  • Cassandra Storage Engine
  • Conncect Storage Engine
  • Squence Storage Engine
  • Better table discovery (Federated-X)
  • Spider Storage Engine
  • TokuDB Storage Engine
  • Mroonga fulltext search Storage Engine

• XtraDB
  • XtraDB Version 5.6
  • Async commit checkpoint in XtraDB and InnoDB
  • Support for atomic writes on FusionIO DirectFS

• Replication
  • Parallel Replication
  • Global Transaction ID (GTID)
  • Multi Source Replication

• Performance
  • Subquery Optimization (EXISTS to IN)
  • Faster UNIQUE KEY generation
  • Shutdown performance improvment for MyISAM/Aria table (adjustable hash size)

• Security
  • Roles
  • MariaDB Audit Plugin

• Optimizer
  • EXPLAIN for DML Statements
  • Engine independent table statistics
  • Histogram based statistics
  • QUERY_RESPONSE_TIME plugin
  • SHOW EXPLAIN for running connections
  • EXPLAIN in the Slow Query Log

• Per thread memory usage statistics
• SHOW PLUGINS SONAME
• SHUTDOWN command
• Killing a query by query id not thread id.
• Return result set of delete rows with DELETE ... RETURNING
• ALTER TABLE IF (NOT) EXISTS
• CREATE OR REPLACE TABLE
• Dynamic columns referenced by name
• Multiple use locks (GET_LOCK) in one connection
• Better error messages
• New regular expressions (PCRE) REGEXP_REPLACE, REGEXP_INSTR, REGEXP_SUBSTR
• Metadata lock information in INFORMATION_SCHEMA
• Priority queue optimzation visibility
• FLUSH TABLE ... FOR EXPORT flushes changes to disk for binary copy
• CURRENT_TIMESTAMP as DEFAULT for DATETIME
• Various features backported from MySQL 5.6

MySQL 5.7 (link) MariaDB 10.1 (link)

• InnoDB
  • InnoDB Version 5.7
  • VAR CHAR size increase can be in-place in some cases.
  • DDL performance improvements for temporary InnoDB tables (CREATE DROP TRUNCATE, ALTER)
  • Active InnoDB temporary table metadata are exposed in table INNODB_TEMP_TABLE_INFO.
  • InnoDB support spatial data type (GIS, DATA_GEOMETRY)
  • Separate tablespace for temporary InnoDB tables.
  • Support for InnoDB Full-text parser plugins was added.
  • Multiple page cleaner threads were added.
  • Regular an paritioned InnoDB tables can be rebuilt using online inplace DDL commands (OPTIMZE, ALTER TABLE FORCE)
  • Automatic detection, support and optimization for Fusion-io NVM file system to support atomic writes.
  • Better support for Transportable Tablespaces to ease backup process.
  • InnoDB Buffer Pool size can be configured dynamically.
  • Multi-threaded page cleaner support for shutdown and recovery phase.
  • InnoDB spatial index support for online in place operation (ADD SPATIAL INDEX)
  • InnoDB sorted index builds to improve bulk loads.
  • Identification of modified tablespaces to increase crash recovery performance.
  • InnoDB UNDO log truncation.
  • InnoDB native partion support.
  • InnoDB general tablespace support for databases with a huge amount of tables.
  • InnoDB data at rest encryption for file-per-table tablespaces.

• Performance
  • EXPLAIN for running connections (FOR CONNECTIONS)
  • Finer Control of optimizer hints.

• Security
  • Old password support has been removed.
  • Autmomatic password expiry policies.
  • Lock and unlock of accounts.
  • SSL and RSA certificate and key file generation.
  • SSL enabled automatically if available.
  • MySQL will be initialized secure by default (= hardened)
  • STRICT_TRANS_TABLES sql_mode is now enabled by default.
  • ONLY_FULL_GROUP_BY sql_mode made more sophisticated to only prohibit non deterministic query.

• Replication
  • Master dump thread was refactored to improve throughput.
  • Replication Master change without STOP SLAVE.
  • Multi-source replication introduced.

• Partitioning
  • HANDLER statment works now on partitioned tables.
  • Index Condition Pushdown (ICP) works for partitioned InnoDB and MyISAM tables.
  • ALTER TABLE EXCHANGE PARTITION WITHOU VALIDATION is possible to improve performance of exchnage.

• Native JSON support
  • Data type JSON.
  • JSON functions: JSON_ARRAY, JSON_MERGE, JSON_OBJECT, JSON_CONTAINS, JSON_CONTAINS_PATH, JSON_EXTRACT, JSON_KEYS, JSON_SEARCH, JSON_APPEND, JSON_ARRAY_APPEND, JSON_ARRAY_INSERT, JSON_INSERT, JSON_QUOTE, JSON_REMOVE, JSON_REPLACE, JSON_SET, JSON_UNQUOTE, JSON_DEPTH, JSON_LENGTH, JSON_TYPE, JSON_VALID

• System and status variables moved from INFORMATION_SCHEMA to PERFORMANCE_SCHEMA.
• Sys Schema created by default.
• Condition handling: GET STACKED DIAGNOSTICS
• Multiple triggers per event are possible now.
• Native logging to syslog possible.
• Generated Column support.
• Database rewriting in mysqlbinlog.
• Control+C in mysql client does not exit any more but interrupts query only.
• New China National Standard GB18030 character set.
• RENAME INDEX is online inplace without a table copy.
• Chinese, Japanese and Korean (CJK) full-text parser implemented (ngram MeCab full-test parser plugins).
• Deprecation and remove of features.

• Literature
  • Complete list of new features in MySQL 5.7

• XtraDB
  • Allow up to 64K pages in InnoDB (old limit was 16K).
  • Defragmenting InnoDB Tablespaces improved which uses OPTIMIZE TABLE to defragment InnoDB tablespaces.
  • XtraDB page compression

• Performance
  • Page compression for FusionIO
  • Do not create .frm files for temporary tables.
  • UNION ALL works without usage of a temporary table.
  • Scalability fixes for Power8.
  • Performance improvementes on simple queries.
  • Performance Schema tables no longer use .frm files.
  • xid cache scalability was significantly improved.

• Replication
  • Optimistic mode of in-order parallel replication
  • domain_id based replication filters
  • Enhanced semisync replication: Wait for at least one slave to acknowledge transaction before committing.
  • Triggers can now be run on the slave for row-based events.
  • Dump Thread Enhancements: Makes multiple slave setups faster by allowing concurrent reading of binary log.
  • Throughput improvements in parallel replication.
  • RESET_MASTER is extended with TO.

• Optimizer
  • ANALYZE statement provides output for how many rows were actually read, etc.
  • EXPLAIN FORMAT=JSON
  • ORDER BY optimization is improved.
  • MAX_STATEMENT_TIME can be used to automatically abort long running queries.

• Security
  • Password validation plug-in API.
  • Simple password check password validation plugin.
  • Cracklib_password_check password validation plugin.
  • Table, Tablespace and Log at-rest encryption (TDE)
  • SET DEFAULT ROLE
  • New columns for the INFORMATION_SCHEMA.APPLICABLE_ROLES table.

• Galera Cluster plug-in becomes standard in MariaDB.
• Wsrep information in INFORMATION_SCHEMA: WSREP_MEMBERSHIP and WSREP_STATUS
• Consistent support for IF EXISTS and IF NOT EXISTS and OR REPLACE for: CREATE DATABASE, CREATE FUNCTION UDF, CREATE ROLE, CREATE SERVER, CREATE USER, CREATE VIEW, DROP ROLE, DROP USER, CREATE EVENT, DROP EVENT, CREATE INDEX, DROP INDEX, CREATE TRIGGER, DROP TRIGGER
• Information Schema plugins can now support SHOW and FLUSH statements.
• GET_LOCK() now supports microseconds in the timeout.
• The number of rows affected by a slow UPDATE or DELETE is now recorded in the slow query log.
• Anonymous Compount Statents blocks are supported.
• SQL standards-compliant behavior when dealing with Primary Keys with Nullable Columns.
• Automatic discovery of PERFORMANCE_SCHEMA tables.
• INFORMATION_SCHEMA.SYSTEM_VARIABLES, enforce_storage_engine, default-tmp-storage-engine, mysql56-temporal-format, Slave_skipped_errors, silent-startup
• New status variables to show the number of grants on different object.
• Set variables per statement: SET STATEMENT
• Support for Spatial Reference systems for the GIS data.
• More functions from the OGC standard added: ST_Boundary, ST_ConvexHull, ST_IsRing, ST_PointOnSurface, ST_Relate
• GIS INFORMATION_SCHEMA tables: GEOMETRY_COLUMNS, SPATIAL_REF_SYS

MySQL 8.0 (link) MariaDB 10.2 (link)

• InnoDB
  • InnoDB Version 8.0
  • AUTO_INCREMENT values are persisted accross server restarts.
  • Index corruption and in-memory corruption detection written persistently to the transaction log.
  • InnoDB Memcached plug-in supports multiple get operations.
  • Deadlock detection can be disabled and leads to a lock timeout to increase performance.
  • Index pages cached in buffer pool are listed in INNODB_CACHED_INDEXES.
  • All InnoDB temporary tables are created in InnoDB shared temporary tablespace.

• JSON
  • Inline path operator ->> added.
  • Column paht operator -> improved.
  • JSON aggregation functions JSON_ARRAYAGG() and JSON_OBJECTAGG() added.

• Security
  • Account management supports roles.
  • Aromicity in User Management DDLs.

• Transactional data dictionary (DD).
• Common Table Expressions (CTE, recursive SQL, Series creation)
• Descending Indexes
• Scaling and Performance of INFORMATION_SCHEMA (1 Mio table problem)
• Deprecation and remove of features.

MySQL 8.0 is currently in a very early stage (DMR) so this list will increase over time!

• XtraDB
  • XtraDB Version 5.6

• Security
  • SHOW CREATE USER
  • CREATE USER and ALTER USER extended for limiting resources and TLS/SSL support.

• Performance
  • Connection creation speed-up by separate thread.

• Optimizer
  • EXPLAIN FORMAT=JSON improved.

• Partition
  • Catchall partion for LIST partions.

• Introduction of Window functions: CUME_DIST, DENSE_RANK, NTILE, PERCENT_RANK, RANK, ROW_NUMBER
• SHOW CREATE USER statement and limiting user resource usage introduced
• Common Table Expression (CTE) WITH clause for recursive queries.
• CHECK CONSTRAINT support.
• Support for DEFAULT with expression.
• BLOB and TEXT can now have default values.
• Virtual computed columns restrictions lifted.
• Supported decimals in DECIMAL increased from 30 to 38.
• Temporary tables can be referred to several times in the same query.
• Multiple triggers for the same event.
• InnoDB/XtraDB 5.7.14 was merged.
• ANALYZE TABLE implemented lock free.
• CONNECT engine supports JDBC table type.
• NO PAD collation support.
• Table cache can auto-partition introduced.
• New Window functions: LEAD, LAG, NTH_VALUE, FIRST_VALUE, LAST_VALUE
• Slave binary log read throttling, delayed replication, and binary log compression implemented.
• JSON functions added
• Oracle style EXECUTE IMMEDIATE.
• PREPARE STATEMENT understand most expressions.
• TRIGGERS enhanced by FOLLOWS/PRECEDES clauses.
• I_S.USER_VARIABLES introduced as plug-in.
• New status information: Com_alter_user, Com_multi, Com_show_create_user.
• New variables: innodb_tmpdir, read_binlog_speed_limit.
• DML flashback introduced on instance, database and table level.
• GeoJSON functions added.

• To come soon
  • MariaDB Column store (ex. InfiniDB)
  • MyRocks?

MariaDB 10.2 is currently in a early stage (beta release) so this list will increase over time...

MySQL 9.0 MariaDB 10.3 (link) and 10.4

No details are known yet. MySQL developer meetingt took place in November 2016.

• Literature:
  • MariaDB.org
  • Launchpad

• Suggested features
  • Hidden columns
  • Long unique constraints
  • SQL based CREATE AGGREGATE FUNCTION
  • New data types: IPv6, UUID, pluggable data-type API
  • Better support for CJK (Chinese, Japanese, and Korean) languages. Include the ngram full-text parser and MeCab full-text parser .
  • Improvement of Spider SE.
  • Support for SEQUENCES
  • Additional PL/SQL parser
  • Support for INTERSECT
  • Support for EXCEPT

MariaDB 10.3 is currently in a very early stage so this list will increase over time!

Please let me know if I got something wrong or forgot any significant feature for theses 2 MySQL branches.

Taxonomy upgrade extras: featuresmariadbmysqlGTIDcomparison

As you probably know MySQL is an Open Source product licensed under the GPL v2. The GPL grants you the right to not just read and understand the code of the product but also to use, modify AND redistribute the code as long as you follow the GPL rules.

This redistribution has happened in the past various times. But in the western hemisphere only 3 of these branches/forks of MySQL are of relevance for the majority of the MySQL users: Galera Cluster for MySQL, MariaDB (Server and Galera Cluster) and Percona Server (and XtraDB Cluster).

Now it happened what has to happen in nature: The different branches/forks start to diverge (following the marketing rule: differentiate yourself from your competitors). The biggest an most important divergence happens now between MySQL and MariaDB.

Recently a customer of FromDual claimed that there is no more progress in the MySQL Server development whereas the MariaDB Server does significant progress. I was wondering a bit how this statement could have been made. So I try to summarize the New Features which have been added since the beginning of the separation starting with MySQL 5.1.

It is important to know, that some parts of MySQL code are directly or in modified form ported to MariaDB whereas some MariaDB features were implemented in MySQL as well. So missing features in MariaDB or improvements in MySQL can possibly make it sooner or later also into MariaDB and vice versa. Further both forks were profiting significantly from old MySQL 6.0 code which was never really announced broadly.

Further to consider: Sun Microsystems acquired MySQL in January 2008 (MySQL 5.1.23 was out then and MySQL 5.2, 5.4 and 6.0 were in the queue) and Sun was acquired by Oracle in January 2010 (MySQL 5.1.43, MySQL 5.5.1 were out, MySQL 5.2, 5.4 and 6.0 were abandoned and MySQL 5.6 was in the queue).

MySQL 5.1 MariaDB 5.1 (link), 5.2 (link) and 5.3 (link)

• Partitioning
• Row-based replication
• Plug-in API
• Event scheduler.
• Server log tables.
• Upgrade program mysql_upgrade.
• Improvements to INFORMATION_SCHEMA.
• XML functions with Xpath support.

MariaDB 5.1

• Storage Engines
  • Aria (Crash-safe MyISAM)
  • XtraDB plug-in (Branch of InnoDB)
  • PBXT (transactional Storage Engine)
  • Federated-X (replacement for Federated).

• Performance
  • Faster CHECKSUM TABLE.
  • Character Set conversion improvement/elimination.
  • Speed-up of complex queries using Aria SE for temporary tables.
  • Optimizer: Table elimination.

• Upgrade from MySQL 5.0 improved.
• Better testing.
• Microseconds precision in PROCESSLIST.

MariaDB 5.2

• Storage Engines
  • OQGRAPH (Graph SE)
  • SphinxSE (Full-text search engine)

• Performance
  • Segmented MyISAM key cache (instances)
  • Group Commit for Aria SE

• Security
  • Pluggable Authentication

• Virtual columns
• Extended user statistics
• Storage Engine specific CREATE TABLE
• Enhancements to INFORMATION_SCHEMA.PLUGINS table

MariaDB 5.3

• Performance
  • Subquery Optimization
    • Semi-join subquery optimizations
    • Non-semi-join optimizations
    • Subquery Cache
    • Subquery is not materialized any more in EXPLAIN
  • Optimization for derived tables and views
    • No early materialization of derived tables
    • Derived Table Merge optimization
    • Derived Table with Keys optimization
    • Fields of mergeable views and derived tables are involved in optimization
  • Disk access optimization
    • Index Condition Pushdown (ICP)
    • Multi-Range-Read optimization (MRR)
  • Join optimizations
    • Block-based Join Algorithms: Block Nested Loop (BNL) for outer joins, Block Hash Joins, Block Index Joins (Batched Key Access (BKA) Joins)
  • Index Merge improvements

• Replication
  • Group Commit for Binary Log
  • Annotation of row-based replication events with the original SQL statement
  • Checksum for binlog events
  • Enhancements for START TRANSACTION WITH CONSISTENT SNAPSHOT
  • Performance improvement for row-based replication for tables with no primary key

• Handler Socket Interface included.
• HANDLER READ works with prepared statements
• Dynamic Column support for Handler Interface
• Microsecond support
• CAST extended
• Windows performance improvements
• New status variables
• Progress reports for some operations
• Enhanced KILL command

MySQL 5.5 (link) MariaDB 5.5 (link)

• InnoDB
  • InnoDB Version 5.5
  • Default storage engine switched to InnoDB.
  • InnoDB fast INDEX DROP/CREATE feature added.
  • Multi-core scalability. Focus on InnoDB, especially locking and memory management.
  • Optimizing InnoDB I/O subsystem to more effective use of available I/O capacity.

• Performance
  • MySQL Thread Pool plug-in (Enterprise)

• Security
  • MySQL Audit plug-in (Enterprise)
  • MySQL pluggable authentication (Enterprise) for LDAP, Kerberos, PAM and Windows login

• Replication
  • Semi-synchronous replication.

• Partitioning
  • 2 new partition types (RANGE COLUMNS, LIST COLUMNS).
  • TRUNCATE PARTITION.

• Proxy Users
• Diagnostic improvements to better access execution an performance information including PERFORMANCE_SCHEMA, expanded SHOW ENGINE INNODB STATUS output and new status variables.
• Supplementary Unicode characters (utf16, utf32, utf8mb4).
• CACHE INDEX and LOAD INDEX INTO CACHE for partitioned MyISAM tables.
• Condition Handling: SIGNAL and RESIGNAL.
• Introduction of Metadata locking to prevent DDL statements from compromising transactions serializability.
• IPv6 Support
• XML enhancement LOAD_XML_INFILE.
• Build chain switched to CMake to ease build on other platforms including Windows.
• Deprecation and remove of features.

• Storage Engines
  • SphinxSE updated to 2.0.4
  • PBXT Storage Engine is deprecated.

• XtraDB
  • MariaDB uses XtraDB 5.5 as compiled in SE and InnoDB 5.5 as plug-in.
  • Extended Keys support for XtraDB

• Performance
  • Thread pool plug-in
  • Non-blocking client API Library

• Replication
  • Updates on P_S tables are not logged to binary log.
  • replicate_* variables are dynamically.
  • Skip_replication option

• LIMIT ROWS EXAMINED
• New status variables for features.
• New plug-in to log SQL level errors.

MySQL 5.6 (link) MariaDB 10.0 (link)

• InnoDB
  • InnoDB Version 5.6
  • InnoDB full-text search.
  • InnoDB transportable tablespace support
  • Different InnoDB pages size implementation (4k, 8k, 16k)
  • Improvement of InnoDB adaptive flushing algorithm to make I/O more efficient.
  • NoSQL style Memcached API to access InnoDB data.
  • InnoDB optimizer persistent statistics.
  • InnoDB read-only transactions.
  • Separating InnoDB UNDO tablespace from system tablespace.
  • Maximum InnoDB transaction log size increased from 4G to 512G.
  • InnoDB read-only capability for read-only media (CD, DVD, etc.)
  • InnoDB table compression.
  • New InnoDB meta data table in INFORMATION_SCHEMA.
  • InnoDB internal performance performance enhancements.
  • Better InnoDB deadlock detection algorithm. Deadlock can be written to MySQL error log.
  • InnoDB buffer pool state saving and restoring capabilities.
  • InnoDB Monitor dynamially disable/enable.
  • Online and inplace DDL operations for normal and partitioned InnoDB Tables to reduce application downtime.

• Optimizer
  • ORDER BY non-index-column for simple queries and subqueries
  • Disk-Sweep Multi-Range Read (MRR) optimization for secondary index/table access to reduce I/O
  • Index Condition Pushdown (ICP) optimization by pushing down the WHERE filter to the storage engine.
  • EXPLAIN also works for DML statemetns.
  • Optimizing of subqueries in derived tables (FROM (...)) by postponing or indexing deived tables.
  • Implementation of semi-join and materialization strategies to optimize subquery execution.
  • Batched Key Access (BKA) join algorithm to improve join performance during table scanning.
  • Optimizer trace capabilities.

• Performance Schema (P_S)
  • Instrumentation for Statements and stages
  • Configuration of consumers at server startup
  • Summary tables for table and index I/O and for table locks
  • Event filtering by table
  • Various new instrumentation.

• Security
  • Encrypted authentication credentials
  • Stronger encryption for passwords (SHA-256 authentication plugin)
  • MySQL User password expiration.
  • Password validation plugin to check password strength
  • mysql_install_db can create secure root password by default
  • cleartext password is not written to any log file any more.
  • MySQL Firewall (Enterprise)

• Replication
  • Transaction based replication using global transaction identifiers (GTID)
  • Row Image Control to reduce binary log volume.
  • Crash-safe replication with checksumming and verfiying.
  • IO and SQL thread information can be stored in an transactional table inside the DB.
  • MySQL binlog streaming with mysqlbinlog possible.
  • Delayes replication
  • Parallel replication on schema level.

• Partitioning
  • Number of partitions including subpartitions increased to 8192.
  • Exchange partition with a normal table.
  • Explicit selection of specific partiton is possible.
  • Partition lock prunining for DML and DDL statements.

• Condition handling: GET DIAGNOSTICS and SET DIAGNOSTICS
• Server defaults changes.
• Data types TIME, DATETIME and TIMESTAMP with microseconds
• Host cache exposure and connection errors status infromation for finding connection problems.
• Improvement in GIS functions.
• Deprecation and remove of features.

• Storage Engine
  • Cassandra Storage Engine
  • Conncect Storage Engine
  • Squence Storage Engine
  • Better table discovery (Federated-X)
  • Spider Storage Engine
  • TokuDB Storage Engine
  • Mroonga fulltext search Storage Engine

• XtraDB
  • XtraDB Version 5.6
  • Async commit checkpoint in XtraDB and InnoDB
  • Support for atomic writes on FusionIO DirectFS

• Replication
  • Parallel Replication
  • Global Transaction ID (GTID)
  • Multi Source Replication

• Performance
  • Subquery Optimization (EXISTS to IN)
  • Faster UNIQUE KEY generation
  • Shutdown performance improvment for MyISAM/Aria table (adjustable hash size)

• Security
  • Roles
  • MariaDB Audit Plugin

• Optimizer
  • EXPLAIN for DML Statements
  • Engine independent table statistics
  • Histogram based statistics
  • QUERY_RESPONSE_TIME plugin
  • SHOW EXPLAIN for running connections
  • EXPLAIN in the Slow Query Log

• Per thread memory usage statistics
• SHOW PLUGINS SONAME
• SHUTDOWN command
• Killing a query by query id not thread id.
• Return result set of delete rows with DELETE ... RETURNING
• ALTER TABLE IF (NOT) EXISTS
• CREATE OR REPLACE TABLE
• Dynamic columns referenced by name
• Multiple use locks (GET_LOCK) in one connection
• Better error messages
• New regular expressions (PCRE) REGEXP_REPLACE, REGEXP_INSTR, REGEXP_SUBSTR
• Metadata lock information in INFORMATION_SCHEMA
• Priority queue optimzation visibility
• FLUSH TABLE ... FOR EXPORT flushes changes to disk for binary copy
• CURRENT_TIMESTAMP as DEFAULT for DATETIME
• Various features backported from MySQL 5.6

MySQL 5.7 (link) MariaDB 10.1 (link)

• InnoDB
  • InnoDB Version 5.7
  • VARCHAR size increase can be in-place in some cases.
  • DDL performance improvements for temporary InnoDB tables (CREATE DROP TRUNCATE, ALTER)
  • Active InnoDB temporary table metadata are exposed in table INNODB_TEMP_TABLE_INFO.
  • InnoDB support spatial data type (GIS, DATA_GEOMETRY)
  • Separate tablespace for temporary InnoDB tables.
  • Support for InnoDB Full-text parser plugins was added.
  • Multiple page cleaner threads were added.
  • Regular an paritioned InnoDB tables can be rebuilt using online inplace DDL commands (OPTIMZE, ALTER TABLE FORCE)
  • Automatic detection, support and optimization for Fusion-io NVM file system to support atomic writes.
  • Better support for Transportable Tablespaces to ease backup process.
  • InnoDB Buffer Pool size can be configured dynamically.
  • Multi-threaded page cleaner support for shutdown and recovery phase.
  • InnoDB spatial index support for online in place operation (ADD SPATIAL INDEX)
  • InnoDB sorted index builds to improve bulk loads.
  • Identification of modified tablespaces to increase crash recovery performance.
  • InnoDB UNDO log truncation.
  • InnoDB native partion support.
  • InnoDB general tablespace support for databases with a huge amount of tables.
  • InnoDB data at rest encryption for file-per-table tablespaces.

• Performance
  • EXPLAIN for running connections (FOR CONNECTIONS)
  • Finer Control of optimizer hints.

• Security
  • Old password support has been removed.
  • Autmomatic password expiry policies.
  • Lock and unlock of accounts.
  • SSL and RSA certificate and key file generation.
  • SSL enabled automatically if available.
  • MySQL will be initialized secure by default (= hardened)
  • STRICT_TRANS_TABLES sql_mode is now enabled by default.
  • ONLY_FULL_GROUP_BY sql_mode made more sophisticated to only prohibit non deterministic query.

• Replication
  • Master dump thread was refactored to improve throughput.
  • Replication Master change without STOP SLAVE.
  • Multi-source replication introduced.

• Partitioning
  • HANDLER statment works now on partitioned tables.
  • Index Condition Pushdown (ICP) works for partitioned InnoDB and MyISAM tables.
  • ALTER TABLE EXCHANGE PARTITION WITHOU VALIDATION is possible to improve performance of exchnage.

• Native JSON support
  • Data type JSON.
  • JSON functions: JSON_ARRAY, JSON_MERGE, JSON_OBJECT, JSON_CONTAINS, JSON_CONTAINS_PATH, JSON_EXTRACT, JSON_KEYS, JSON_SEARCH, JSON_APPEND, JSON_ARRAY_APPEND, JSON_ARRAY_INSERT, JSON_INSERT, JSON_QUOTE, JSON_REMOVE, JSON_REPLACE, JSON_SET, JSON_UNQUOTE, JSON_DEPTH, JSON_LENGTH, JSON_TYPE, JSON_VALID

• System and status variables moved from INFORMATION_SCHEMA to PERFORMANCE_SCHEMA.
• Sys Schema created by default.
• Condition handling: GET STACKED DIAGNOSTICS
• Multiple triggers per event are possible now.
• Native logging to syslog possible.
• Generated Column support.
• Database rewriting in mysqlbinlog.
• Control+C in mysql client does not exit any more but interrupts query only.
• New China National Standard GB18030 character set.
• RENAME INDEX is online inplace without a table copy.
• Chinese, Japanese and Korean (CJK) full-text parser implemented (ngram MeCab full-test parser plugins).
• Deprecation and remove of features.

• XtraDB
  • Allow up to 64K pages in InnoDB (old limit was 16K).
  • Defragmenting InnoDB Tablespaces improved which uses OPTIMIZE TABLE to defragment InnoDB tablespaces.
  • XtraDB page compression

• Performance
  • Page compression for FusionIO
  • Do not create .frm files for temporary tables.
  • UNION ALL works without usage of a temporary table.
  • Scalability fixes for Power8.
  • Performance improvementes on simple queries.
  • Performance Schema tables no longer use .frm files.
  • xid cache scalability was significantly improved.

• Replication
  • Optimistic mode of in-order parallel replication
  • domain_id based replication filters
  • Enhanced semisync replication: Wait for at least one slave to acknowledge transaction before committing.
  • Triggers can now be run on the slave for row-based events.
  • Dump Thread Enhancements: Makes multiple slave setups faster by allowing concurrent reading of binary log.
  • Throughput improvements in parallel replication.
  • RESET_MASTER is extended with TO.

• Optimizer
  • ANALYZE statement provides output for how many rows were actually read, etc.
  • EXPLAIN FORMAT=JSON
  • ORDER BY optimization is improved.
  • MAX_STATEMENT_TIME can be used to automatically abort long running queries.

• Security
  • Password validation plug-in API.
  • Simple password check password validation plugin.
  • Cracklib_password_check password validation plugin.
  • Table, Tablespace and Log at-rest encryption (TDE)
  • SET DEFAULT ROLE
  • New columns for the INFORMATION_SCHEMA.APPLICABLE_ROLES table.

• Galera Cluster plug-in becomes standard in MariaDB.
• Wsrep information in INFORMATION_SCHEMA: WSREP_MEMBERSHIP and WSREP_STATUS
• Consistent support for IF EXISTS and IF NOT EXISTS and OR REPLACE for: CREATE DATABASE, CREATE FUNCTION UDF, CREATE ROLE, CREATE SERVER, CREATE USER, CREATE VIEW, DROP ROLE, DROP USER, CREATE EVENT, DROP EVENT, CREATE INDEX, DROP INDEX, CREATE TRIGGER, DROP TRIGGER
• Information Schema plugins can now support SHOW and FLUSH statements.
• GET_LOCK() now supports microseconds in the timeout.
• The number of rows affected by a slow UPDATE or DELETE is now recorded in the slow query log.
• Anonymous Compount Statents blocks are supported.
• SQL standards-compliant behavior when dealing with Primary Keys with Nullable Columns.
• Automatic discovery of PERFORMANCE_SCHEMA tables.
• INFORMATION_SCHEMA.SYSTEM_VARIABLES, enforce_storage_engine, default-tmp-storage-engine, mysql56-temporal-format, Slave_skipped_errors, silent-startup
• New status variables to show the number of grants on different object.
• Set variables per statement: SET STATEMENT
• Support for Spatial Reference systems for the GIS data.
• More functions from the OGC standard added: ST_Boundary, ST_ConvexHull, ST_IsRing, ST_PointOnSurface, ST_Relate
• GIS INFORMATION_SCHEMA tables: GEOMETRY_COLUMNS, SPATIAL_REF_SYS

MySQL 8.0 (link) MariaDB 10.2 (link)

• InnoDB
  • InnoDB Version 8.0
  • AUTO_INCREMENT values are persisted accross server restarts.
  • Index corruption and in-memory corruption detection written persistently to the transaction log.
  • InnoDB Memcached plug-in supports multiple get operations.
  • Deadlock detection can be disabled and leads to a lock timeout to increase performance.
  • Index pages cached in buffer pool are listed in INNODB_CACHED_INDEXES.
  • All InnoDB temporary tables are created in InnoDB shared temporary tablespace.

• JSON
  • Inline path operator ->> added.
  • Column paht operator -> improved.
  • JSON aggregation functions JSON_ARRAYAGG() and JSON_OBJECTAGG() added.

• Security
  • Account management supports roles.
  • Aromicity in User Management DDLs.

• Transactional data dictionary (DD).
• Common Table Expressions (CTE, recursive SQL, Series creation)
• Descending Indexes
• Scaling and Performance of INFORMATION_SCHEMA (1 Mio table problem)
• Deprecation and remove of features.

MySQL 8.0 is currently in a very early stage (DMR) so this list will increase over time!

• XtraDB
  • XtraDB Version 5.6

• Security
  • SHOW CREATE USER
  • CREATE USER and ALTER USER extended for limiting resources and TLS/SSL support.

• Performance
  • Connection creation speed-up by separate thread.

• Optimizer
  • EXPLAIN FORMAT=JSON improved.

• Partition
  • Catchall partion for LIST partions.

• Introduction of Window functions: CUME_DIST, DENSE_RANK, NTILE, PERCENT_RANK, RANK, ROW_NUMBER
• WITH clause for recursive queries.
• CHECK CONSTRAINT support.
• Support for DEFAULT with expression.
• BLOB and TEXT can now have default values.
• Virtual computed columns restrictions lifted.
• Supported decimals in DECIMAL increased from 30 to 38.
• Multiple triggers for the same event.
• Oracle style EXECUTE IMMEDIATE.
• PREPARE STATEMENT understand most expressions.
• I_S.USER_VARIABLES introduced as plug-in.
• New status information: com_alter_user, com_multi, com_show_create_user.
• New variables: innodb_tmpdir, read_binlog_speed_limit.

• To come soon
  • MariaDB Column store (ex. InfiniDB)
  • MyRocks?

MariaDB 10.2 is currently in a early stage (beta release) so this list will increase over time...

MySQL 8.1 MariaDB 10.3 (link) and 10.4

No details are known yet. MySQL developer meetingt took place in November 2016.

• Literature:
  • MariaDB.org
  • Launchpad

• Suggested features
  • Hidden columns
  • Long unique constraints
  • SQL based CREATE AGGREGATE FUNCTION
  • New data types: IPv6, UUID, pluggable data-type API
  • Better support for CJK (Chinese, Japanese, and Korean) languages. Include the ngram full-text parser and MeCab full-text parser .
  • Improvement of Spider SE.
  • Support for SEQUENCES
  • Additional PL/SQL parser
  • Support for INTERSECT
  • Support for EXCEPT

MariaDB 10.3 is currently in a very early stage so this list will increase over time!

Please let me know if I got something wrong or forgot any significant feature for theses 2 MySQL branches.

Taxonomy upgrade extras: featuresmariadbmysqlnew

FromDual has the pleasure to announce the release of the new version 1.0.0 of its popular Database Performance Monitor for MySQL, MariaDB, Galera Cluster and Percona Server fpmmm.

You can download fpmmm from here.

In the inconceivable case that you find a bug in fpmmm please report it to our Bug-tracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

This release contains various bug fixes and improvements. Further significant functionality has changed and improved. So please read carefully...

Changes in fpmmm v1.0.0 fpmmm agent

• SysV init script added to run fpmmm as daemon.
• SystemD unit file added to run fpmmm as daemon.
• Default location of fpmmm.conf file made unique to /etc/.
• Requirement checks improved and made more restrictive.
• Error Messages improved and made more understandable.
• Library MyEnv.inc synced from MyEnv project.
• zabbix_agentd.init and zabbix_server.init moved from bin to tpl folder.
• fpmmm_server.init renamed to fpmmm_sysv.init.
• PHP default timezone set to Europe/Zurich.
• fpmmm Agent lock file is now touched at every run to show that agent is still alive.
• Get user privileges to do checks.
• Reconnect implemented for fpmmm daemon.
• Angel script added: fpmmm_angel.
• New Type = host added to fpmmm configuration template file.
• Fpmmm daemon writes a PID file.
• Fpmmm can be daemonized now to work as stand-alone system (without zabbix_agent).
• Connection test added for daemon.
• Interval for fpmmm daemon is introduced.
• Log verbosity for some messages adjusted.
• fpmmm daemon reacts on SIGTERM (stop) and SIGHUP (reload). Reload is not implemented yet.
• Default file locations moved to LSB suggestions.

fpmmm agent installer

• Error Messages improved and made more understandable.

InnoDB module

• InnoDB variable innodb_log_file_size of 0 is not possible and thus suppressed.
• Check for lacking PROCESS privilege added.
• InnoDB status Innodb_max_dirty_pages_pct added.

Master module

• Check for lacking REPLICATION CLIENT privilege added.

Memcached module

• Error Messages improved and made more understandable.

Server module

• Error Messages improved and made more understandable.
• Advices to fix problems improved.
• iostat is used to gather I/O metrics.

Mysql module

• Gather processlist information introduced.
• MySQL variable max_sort_length check added.
• MySQL variable storage_engine added

For subscriptions of commercial use of fpmmm please get in contact with us.

Taxonomy upgrade extras: mysqlperformancemonitormonitoringfpmmmmaasperformance monitorrelease

FromDual has the pleasure to announce the release of the new version 1.0.0 of its popular Database Performance Monitor for MySQL, MariaDB, Galera Cluster and Percona Server fpmmm.

You can download fpmmm from here.

In the inconceivable case that you find a bug in fpmmm please report it to our Bug-tracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

This release contains various bug fixes and improvements. Further significant functionality has changed and improved. So please read carefully...

Changes in fpmmm v1.0.0 fpmmm agent

• SysV init script added to run fpmmm as daemon.
• SystemD unit file added to run fpmmm as daemon.
• Default location of fpmmm.conf file made unique to /etc/.
• Requirement checks improved and made more restrictive.
• Error Messages improved and made more understandable.
• Library MyEnv.inc synced from MyEnv project.
• zabbix_agentd.init and zabbix_server.init moved from bin to tpl folder.
• fpmmm_server.init renamed to fpmmm_sysv.init.
• PHP default timezone set to Europe/Zurich.
• fpmmm Agent lock file is now touched at every run to show that agent is still alive.
• Get user privileges to do checks.
• Reconnect implemented for fpmmm daemon.
• Angel script added: fpmmm_angel.
• New Type = host added to fpmmm configuration template file.
• Fpmmm daemon writes a PID file.
• Fpmmm can be daemonized now to work as stand-alone system (without zabbix_agent).
• Connection test added for daemon.
• Interval for fpmmm daemon is introduced.
• Log verbosity for some messages adjusted.
• fpmmm daemon reacts on SIGTERM (stop) and SIGHUP (reload). Reload is not implemented yet.
• Default file locations moved to LSB suggestions.

fpmmm agent installer

• Error Messages improved and made more understandable.

InnoDB module

• InnoDB variable innodb_log_file_size of 0 is not possible and thus suppressed.
• Check for lacking PROCESS privilege added.
• InnoDB status Innodb_max_dirty_pages_pct added.

Master module

• Check for lacking REPLICATION CLIENT privilege added.

Memcached module

• Error Messages improved and made more understandable.

Server module

• Error Messages improved and made more understandable.
• Advices to fix problems improved.
• iostat is used to gather I/O metrics.

Mysql module

• Gather processlist information introduced.
• MySQL variable max_sort_length check added.
• MySQL variable storage_engine added

For subscriptions of commercial use of fpmmm please get in contact with us.

Taxonomy upgrade extras: mysqlperformancemonitormonitoringfpmmmmaasperformance monitormpmrelease

FromDual has the pleasure to announce the release of the new version 1.0.0 of its popular Database Performance Monitor for MySQL, MariaDB, Galera Cluster and Percona Server fpmmm.

You can download fpmmm from here.

In the inconceivable case that you find a bug in fpmmm please report it to our Bug-tracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

This release contains various bug fixes and improvements. Further significant functionality has changed and improved. So please read carefully...

Changes in fpmmm v1.0.0 fpmmm agent

• SysV init script added to run fpmmm as daemon.
• SystemD unit file added to run fpmmm as daemon.
• Default location of fpmmm.conf file made unique to /etc/.
• Requirement checks improved and made more restrictive.
• Error Messages improved and made more understandable.
• Library MyEnv.inc synced from MyEnv project.
• zabbix_agentd.init and zabbix_server.init moved from bin to tpl folder.
• fpmmm_server.init renamed to fpmmm_sysv.init.
• PHP default timezone set to Europe/Zurich.
• fpmmm Agent lock file is now touched at every run to show that agent is still alive.
• Get user privileges to do checks.
• Reconnect implemented for fpmmm daemon.
• Angel script added: fpmmm_angel.
• New Type = host added to fpmmm configuration template file.
• Fpmmm daemon writes a PID file.
• Fpmmm can be daemonized now to work as stand-alone system (without zabbix_agent).
• Connection test added for daemon.
• Interval for fpmmm daemon is introduced.
• Log verbosity for some messages adjusted.
• fpmmm daemon reacts on SIGTERM (stop) and SIGHUP (reload). Reload is not implemented yet.
• Default file locations moved to LSB suggestions.

fpmmm agent installer

• Error Messages improved and made more understandable.

InnoDB module

• InnoDB variable innodb_log_file_size of 0 is not possible and thus suppressed.
• Check for lacking PROCESS privilege added.
• InnoDB status Innodb_max_dirty_pages_pct added.

Master module

• Check for lacking REPLICATION CLIENT privilege added.

Memcached module

• Error Messages improved and made more understandable.

Server module

• Error Messages improved and made more understandable.
• Advices to fix problems improved.
• iostat is used to gather I/O metrics.

Mysql module

• Gather processlist information introduced.
• MySQL variable max_sort_length check added.
• MySQL variable storage_engine added

For subscriptions of commercial use of fpmmm please get in contact with us.

Taxonomy upgrade extras: mysqlperformancemonitormonitoringfpmmmmaasperformance monitormpmrelease

FromDual has the pleasure to announce the release of the new version 1.2.3 of the popular Backup and Recovery Manager for MySQL and MariaDB (fromdual_brman).

You can download the FromDual Backup and Recovery Manager from here.

In the inconceivable case that you find a bug in the Backup and Recovery Manager please report it to our Bugtracker or just send us an email.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.2.x to 1.2.3 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.3.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.3 fromdual_brman
Changes in FromDual Backup Manager 1.2.3

This release contains mainly fixes related to MySQL 5.7 and various minor fixes.

You can verify your current FromDual Backup Manager version with the following command:

shell> fromdual_bman --version
FromDual Backup Manager

• New tests added to test-suite.
• MyEnv library myEnv.inc merged into project.
• Output of overview made nicer for ARCHIVE tables.
• Specified but missing configuration file is caught correctly now (bug #163).
• Return and error codes fixed and cleaned up.
• 1M table creation script added to utl/create_1M_tables.php.
• ORDER BY added to all GROUP BY statements to be also correct in future MySQL releases.
• Added short options -h and -V for version and help to fromdual_bman.php.
• Removed redundant error number in fromdual_bman.inc.
• Function parseConnectString moved from fromdual_bman.inc to myEnv.inc.

FromDual Backup Manager and MySQL 5.7

• Bugs related to MySQL 5.7 version detection fixed.

FromDual Backup Manager Privilege Backup

• Privilege backup was fixed for new MySQL 5.7 SHOW GRANTS behaviour.
• Semicolon (;) was added to privilege backup output.

FromDual Backup Manager and Xtrabackup

• New behaviour of MySQL 5.7 for physical backup fixed.
• Backup manager works with new xtrabackup v2.3.x again. Typo bug fixed.

FromDual Backup Manager and LVM Snapshot Backup

• Started to implement LVM snapshot backup functionality.

FromDual Backup and Recovery Manager Catalog

• Catalog version mismatch error made more clear.

FromDual Backup Manager Binary Log Backup

• Error catch implemented for binary logs removed manually.

FromDual Recovery Manager

• Changed wrong rc in progress_bar.php

Taxonomy upgrade extras: BackupRestoreRecoverypitrfromdual_brmanbrmanreleasebmanrman

FromDual has the pleasure to announce the release of the new version 1.2.3 of the popular Backup and Recovery Manager for MySQL and MariaDB (fromdual_brman).

You can download the FromDual Backup and Recovery Manager from here.

In the inconceivable case that you find a bug in the Backup and Recovery Manager please report it to our Bugtracker or just send us an email.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.2.x to 1.2.3 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.3.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.3 fromdual_brman
Changes in FromDual Backup Manager 1.2.3

This release contains mainly fixes related to MySQL 5.7 and various minor fixes.

You can verify your current FromDual Backup Manager version with the following command:

shell> fromdual_bman --version
FromDual Backup Manager

• New tests added to test-suite.
• MyEnv library myEnv.inc merged into project.
• Output of overview made nicer for ARCHIVE tables.
• Specified but missing configuration file is caught correctly now (bug #163).
• Return and error codes fixed and cleaned up.
• 1M table creation script added to utl/create_1M_tables.php.
• ORDER BY added to all GROUP BY statements to be also correct in future MySQL releases.
• Added short options -h and -V for version and help to fromdual_bman.php.
• Removed redundant error number in fromdual_bman.inc.
• Function parseConnectString moved from fromdual_bman.inc to myEnv.inc.

FromDual Backup Manager and MySQL 5.7

• Bugs related to MySQL 5.7 version detection fixed.

FromDual Backup Manager Privilege Backup

• Privilege backup was fixed for new MySQL 5.7 SHOW GRANTS behaviour.
• Semicolon (;) was added to privilege backup output.

FromDual Backup Manager and Xtrabackup

• New behaviour of MySQL 5.7 for physical backup fixed.
• Backup manager works with new xtrabackup v2.3.x again. Typo bug fixed.

FromDual Backup Manager and LVM Snapshot Backup

• Started to implement LVM snapshot backup functionality.

FromDual Backup and Recovery Manager Catalog

• Catalog version mismatch error made more clear.

FromDual Backup Manager Binary Log Backup

• Error catch implemented for binary logs removed manually.

FromDual Recovery Manager

• Changed wrong rc in progress_bar.php

Taxonomy upgrade extras: BackupRestoreRecoverypitrfromdual_brmanbrmanreleasebmanrman

FromDual has the pleasure to announce the release of the new version 1.2.3 of the popular Backup and Recovery Manager for MySQL and MariaDB (fromdual_brman).

You can download the FromDual Backup and Recovery Manager from here.

In the inconceivable case that you find a bug in the Backup and Recovery Manager please report it to our Bugtracker or just send us an email.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.2.x to 1.2.3 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.3.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.3 fromdual_brman
Changes in FromDual Backup Manager 1.2.3

This release contains mainly fixes related to MySQL 5.7 and various minor fixes.

You can verify your current FromDual Backup Manager version with the following command:

shell> fromdual_bman --version
FromDual Backup Manager

• New tests added to test-suite.
• MyEnv library myEnv.inc merged into project.
• Output of overview made nicer for ARCHIVE tables.
• Specified but missing configuration file is caught correctly now (bug #163).
• Return and error codes fixed and cleaned up.
• 1M table creation script added to utl/create_1M_tables.php.
• ORDER BY added to all GROUP BY statements to be also correct in future MySQL releases.
• Added short options -h and -V for version and help to fromdual_bman.php.
• Removed redundant error number in fromdual_bman.inc.
• Function parseConnectString moved from fromdual_bman.inc to myEnv.inc.

FromDual Backup Manager and MySQL 5.7

• Bugs related to MySQL 5.7 version detection fixed.

FromDual Backup Manager Privilege Backup

• Privilege backup was fixed for new MySQL 5.7 SHOW GRANTS behaviour.
• Semicolon (;) was added to privilege backup output.

FromDual Backup Manager and Xtrabackup

• New behaviour of MySQL 5.7 for physical backup fixed.
• Backup manager works with new xtrabackup v2.3.x again. Typo bug fixed.

FromDual Backup Manager and LVM Snapshot Backup

• Started to implement LVM snapshot backup functionality.

FromDual Backup and Recovery Manager Catalog

• Catalog version mismatch error made more clear.

FromDual Backup Manager Binary Log Backup

• Error catch implemented for binary logs removed manually.

FromDual Recovery Manager

• Changed wrong rc in progress_bar.php

Taxonomy upgrade extras: BackupRestoreRecoverypitrfromdual_brman