FromDual GmbH

How to recover deleted tablespace?

oli.sennhauser@fromdual.com (Oli Sennhauser) — Fri, 14 Nov 2014 22:56:17 +0000

Sometimes, MySQL tablespace file(s) might be deleted by mistake, e.g. delete the shared tablespace (ibdata1) or an individual tablespace (table_name.ibd).

In this post I will show you how to recover those files (on Linux OS) having only one condition, MySQL service should still be running. If MySQL service stopped after deleting that file, this method will not work, so it is extremely important to act as quick as possible to avoid data loss.

The following is a simple table creation (innodb_file_per_table is enabled) and the records count inside that table:

SQL> SHOW CREATE TABLE t<br>G
*************************** 1. row ***************************
 Table: t
Create Table: CREATE TABLE `t` (
 `id` int(11) NOT NULL AUTO_INCREMENT,
 PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=23 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

SQL> SELECT COUNT(*) FROM t;
+----------+
| COUNT(*) |
+----------+
| 22 |
+----------+
1 row in set (0.02 sec)

Now, lets delete the individual tablespace for that table:

shell> rm -rf /var/lib/mysql/test/t.ibd

At this time, we can still select and modify that table!!

SQL> INSERT INTO t VALUES (NULL);
Query OK, 1 row affected (0.00 sec)

SQL> SELECT COUNT(*) FROM t;
+----------+
| COUNT(*) |
+----------+
| 23 |
+----------+
1 row in set (0.00 sec)

To be more accurate, rm does not actually delete the file, rather it removes the directory entry pointing to the file’s inode. The inode - and in consequence the file - will be removed only if this is the last reference, but as long as the MySQL server process has the file opened, there is another reference which is the open file handle (that’s why the “mysqld” server process must still be running).

To list the opened files we can use the Linux command lsof (we filter the output to get only the deleted tablespace information):

shell> lsof |grep t.ibd
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
mysqld 11401 mysql 25uW …

Things you should consider before using GTID

oli.sennhauser@fromdual.com (Oli Sennhauser) — Fri, 14 Nov 2014 16:50:19 +0000

Global Transaction ID (GTID) is one of the major features that were introduced in MySQL 5.6 which provides a lot of benefits. I have talked about the GTID concept, implementation and possible troubleshooting at Percona Live London 2014, you can download the slides from our presentations repository or from my session at Percona Live.

On the other hand, there are some important things you should consider before deploying GTID in production, I’m going to list them here in this blog post.

Migration to GTID replication

It is required to shutdown MySQL service on all servers in the replication setup in order to perform the migration from classic replication (based on binary logs information) to the transaction-based (GTID) replication which means that the migration process requires downtime.

The online migration to GTID replication is not yet available. Facebook and Booking.com provided some MySQL patches for this, but they are not yet contained in Oracle’s binaries. So, if you can’t afford a downtime during the migration process, then you might not be able to make the change.

Non transactionally safe statement will raise errors now

It is required to enable the system variable (enforce_gtid_consistency) on all servers inside the GTID replication setup which prevent executing the non transactionally safe statements (check GTID restrictions) like:

CREATE TABLE .. SELECT.
CREATE TEMPORARY TABLE (inside a transaction).
Statements that update non-transactional tables inside a transaction.

So, you will have to fix your application first if it contains any of the above statements before using GTID replication.

MySQL Performance in GTID

It is required to enable the variables (bin_log and log_slave_updates) on - at least - the slave servers which affects the performance on those slaves negatively. …

Galera Cluster and XA Transactions

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 23 Oct 2014 23:47:03 +0000

A few weeks ago, we received an interesting Galera Cluster support case from one of our customers that the application is not working well and they face a lot of troubles in their Galera Cluster setup.

After some investigations, we found a lot of insert queries in state “query end” and lasting for long time without being completed. Also some other queries which were sleeping for long time having the info of “XA COMMIT”:

SQL> SHOW PROCESSLIST;

27 user host:33214 foodmart Query 14440 sleeping XA COMMIT 0x31302e31312e31332e34372e746d30303336383030303031,0x31302e31312e31332e34372e746d333638,0x

SQL> SHOW ENGINE INNODB STATUS;

TRANSACTIONS
============
---TRANSACTION 2DE71D, ACTIVE 14459 sec
9 lock struct(s), heap size 1248, 1 row lock(s), undo log entries 115
MySQL thread id 27, OS thread handle 0x7fc21a42c700, query id 96187 host host-ip foodmart sleeping
XA COMMIT 0x31302e31312e31332e34372e746d30303336383030303031,0x31302e31312e31332e34372e746d333638,0x41544f4d ONE PHASE

XA means eXtended Architecture and “XA COMMIT” statement is one of the distributed transactions (XA Transactions) commands which are clearly NOT supported in Galera Cluster and one of its limitations because of possible rollback on commit.

The following command can be used to check if XA Transactions are being used by your application or not:

SQL> SHOW GLOBAL STATUS LIKE 'Com_xa%';
 
+-------------------+---------+
| Variable_name | Value |
+-------------------+---------+
| Com_xa_commit | 2828094 |
| Com_xa_end | 2828221 |
| Com_xa_prepare | 0 |
| Com_xa_recover | 2205697 |
| Com_xa_rollback | 42 |
| Com_xa_start | 2828305 |
+-------------------+---------+
6 rows in set (0.00 sec)

There are only two possible solutions for this problem:

Get rid of all XA transactions in the application to get the Galera Cluster work.
Use another HA solution (Active/passive, Master/Slave, … etc) but not Galera Cluster.

Conclusion

XA transactions can not …

GTID Replication talk at Percona Live London 2014

oli.sennhauser@fromdual.com (Oli Sennhauser) — Tue, 21 Oct 2014 14:58:10 +0000

Global Transaction ID (GTID) is a new feature coming with MySQL 5.6 which introduced a new MySQL replication method called Transaction-based Replication that is depend on GTID.

In a few weeks, I will be speaking at Percona Live London 2014 (November 3-4) about “Transaction-based REPLICATION (GTID) - IMPLEMENTATION AND TROUBLESHOOTING”. I’ll talk about how to implement GTID replication in different scenarios and how to troubleshoot most of the common issues in GTID replication.

Anyone interested in learning more about GTID replication or planing to go with GTID replication in production should attend this talk.

Face-to-face meetings are very welcome, please let me know if you are interested so we can schedule a meeting.

Looking forward to seeing you there!!

How to install multiple MySQL instances on a single host using MyEnv?

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 16 Oct 2014 16:36:51 +0000

We have been asked several times by MySQL users about how to install multiple MySQL instances on a single host.
Typically, this is required when testing different MySQL versions or MySQL servers (MySQL server, Percona server and MariaDB server) while no available resources are available.
Sometimes, it is even required to install multiple MySQL instances on a single production server.

In this article, I’ll go through the steps needed to install multiple MySQL instances on a single host (using the tar balls binaries) and how our popular tool MyEnv can make such process so easy.

Prepare MySQL environment

[root@centos-temp ~]# groupadd mysql
[root@centos-temp ~]# useradd -g mysql mysql
[root@centos-temp ~]# su - mysql
[mysql@centos-temp:~]$ mkdir ~/product
[mysql@centos-temp:~]$ mkdir ~/data
[mysql@centos-temp ~]$ cd /downloads
[mysql@centos-temp ~]$ wget http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-5.6.21-linux-glibc2.5-x86_64.tar.gz
[mysql@centos-temp ~]$ cd ~/product
[mysql@centos-temp ~]$ tar xf /downloads/mysql-5.6.21-linux-glibc2.5-x86_64.tar.gz
[mysql@centos-temp ~]$ ln -s mysql-5.6.21-linux-glibc2.5-x86_64 mysql-5.6.21

Install MyEnv

MyEnv can be downloaded from here and the installation steps are listed here.

Install the first instance (named master)

Launch the myenv installer:

[mysql@centos-temp ~]$ ~/product/myenv/bin/installMyEnv.sh
PHP is installed on /usr/bin/php
Starting MyEnv installer: /home/mysql/product/myenv-1.1.2/bin/installMyEnv.php


Configuration file /etc/myenv/myenv.conf does NOT exist.
Copy from template or abort (T, a):

Since this is the first instance, the myenv config file does not exist yet, we take the template (t):

Copy from template or abort (T, a): t
Copy /home/mysql/product/myenv-1.1.2/etc/myenv.conf.template to /etc/myenv/myenv.conf

Then MyEnv will detect that no instances are there, we choose the first option (a) to add a new instance:

No instance exists yet.
An instance is the same as a mysqld process.

What do you want to do next?
o Add a new instance,
o …

Galera Cluster VS PXC VS MariaDB Galera Cluster - Benchmarking

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 07 Aug 2014 15:36:29 +0000

It is not clear for many MySQL users that Percona XtraDB Cluster (PXC) and MariaDB Galera Cluster depend on the same Galera library i.e used in Galera Cluster for MySQL which is provided by Codership team:

Galera Cluster: MySQL Server (by Oracle) + Galera library.
Percona XtraDB Cluster: Percona Server + Galera library.
MariaDB Galera Cluster: MariaDB Server + Galera library.

But the question is, are there any performance differences between the three of them ?

Let’s discover that by doing some simple benchmark to test MySQL write performance in Galera Cluster, PXC and MariaDB Galera Cluster installations.

System Information:

HW configurations (AWS Servers):

Nodes Servers HW configurations:

CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz (# of cores 8,
# of threads 16, HT enabled).
Memory: 16GB RAM.
Storage: HDD 120GB/ 5400RPM.

Load balancer Server HW configurations:

CPU: Intel(R) Xeon(R) CPU E5-2651 v2 @ 1.80GHz (# of cores 4,
# of threads 8, HT enabled).
Memory: 16GB RAM.
Storage: HDD 10GB/ 5400RPM.

Load generator Server HW configurations:

CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz (# of cores 16,
# of threads 32, HT enabled).
Memory: 32GB RAM.
Storage: HDD 10GB/ 5400RPM.

Software configurations:

OS : Red Hat Enterprise Linux Server release 6.5 (Santiago)
Sysbench : 0.5.3
GLB : 1.0.0
Galera Cluster : 5.5.34 and 5.6.16
Percona XtraDB Cluster : 5.5.37 and 5.6.19
MariaDB Galera Cluster : 5.5.38 and 10.0.12
Galera Library : 3.5

Test Information:

The testing environment consists of 5 AWS servers, three servers for a three-node cluster (each node is installed on a single server), one server for the load balancer and the final server for the load generator in which sysbench is installed to send requests to the load balancer from.
Sysbench command: sysbench --num-threads=64 --max-requests=1000 --db-driver=mysql --test=/usr/share/doc/sysbench/tests/db/oltp.lua --mysql-table-engine=InnoDB --mysql-user=dev --mysql-password='test' --mysql-host=load_balancer_ip run .
Table structure which was used …

RE: EMPTY TRANSACTIONS CAN BE DANGEROUS

oli.sennhauser@fromdual.com (Oli Sennhauser) — Mon, 07 Jul 2014 12:45:28 +0200

If you used the combination of Percona tools (pt-table-checksum and pt-table-sync) - as I mentioned - to get the data synchronized after injecting an empty transaction and starting the slave you won’t face that problem. Otherwise, yes this could be dangerous if that slave promoted to be master AND the other slaves didn’t replicate that transaction yet or when you take a database backup from that slave as well.

Replication Troubleshooting - Classic VS GTID

oli.sennhauser@fromdual.com (Oli Sennhauser) — Fri, 04 Jul 2014 15:05:58 +0000

In previous posts, I was talking about how to set up MySQL replication, Classic Replication (based on binary logs information) and Transaction-based Replication (based on GTID). In this article I’ll summarize how to troubleshoot MySQL replication for the most common issues we might face with a simple comparison how can we get them solved in the different replication methods (Classic VS GTID).

There are two main operations we might need to do in a replication setup:

Skip or ignore a statement that causes the replication to stop.
Re-initialize a slave when the Replication is broke and could not be started anymore.

Skip or Ignore statement

Basically, the slave should be always synchronized with its master having the same copy of data, but for some reasons there might be inconsistency between both of them (unsafe statement in SBR, Slave is not read_only and was modified apart of replication queries, .. etc) which causes errors and stops the replication, e.g. if the master inserted a record which was already inserted on the slave (Duplicate entry) or updated/deleted a row which was not exist on the slave, … etc.

To solve this issue, we have to either reverse what we have done on the slave (e.g. delete the inserted rows) if that was made by mistake and is known or we can skip executing those statements on the slave and continue the replication again (I’ll focus on skipping a statement in this post as it needs different interaction in Classic and GTID replication).

Sample error messages (from `SHOW SLAVE STATUS` output):

Last_SQL_Error: Could not execute Write_rows event on table test.t1; Duplicate entry '4' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000304, end_log_pos 285

Last_SQL_Error: Could not execute Update_rows event on table test.t1; Can't find record in 't1', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-bin.000304, end_log_pos 492

Last_SQL_Error: Could not execute Delete_rows event on …

MySQL 5.5 and 5.6 ?!

oli.sennhauser@fromdual.com (Oli Sennhauser) — Fri, 27 Jun 2014 14:04:00 +0200

Just to confirm, the above channel failover steps are valid in Galera Cluster for both MySQL versions 5.5 and 5.6. Enjoy!!

GTID In Action

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 12 Jun 2014 14:09:05 +0000

In a previous post I was talking about How to Setup MySQL Replication using the classic method (based on binary logs information). In this article I’ll go through the transaction-based replication implementation using GTID in different scenarios.

The following topics will be covered in this blog:

What is the concept of GTID protocol?

GTID is a Global Transaction IDentifier which introduced in MySQL 5.6.5. It’s not only unique on the server it was originated but it’s unique among all servers in a replication setup.
GTID also guarantee consistency because once a transaction is committed on a server, any other transaction having the same GTID will be ignored, i.e. a committed transaction on a master will be applied only once on the slaves.

GTID consists of two parts separated by a column {source_id:transactions_id}.

WHERE

source_id: Normally the server’s UUID on which the transaction originates. e.g. “b9b4712a-df64-11e3-b391-60672090eb04” .
transaction_id: A sequence number determining the order of the committed transaction.

The following is the GTID for the third transaction on a server having the uuid “b9b4712a-df64-11e3-b391-60672090eb04”:
b9b4712a-df64-11e3-b391-60672090eb04:3

As a new protocol in MySQL there is a set of new related variables, the following are the most important ones (IMHO):

gtid-mode: ON|OFF to enable or disable GTID, this is not a Boolean variable (0 and 1 are not acceptable).
enforce-gtid-consistency: prevent executing the non transactionally safe statements, like:
- CREATE TABLE .. SELECT.
- CREATE TEMPORARY TABLE (inside a transaction).
- Statements that update nontransactional tables inside a transaction.
gtid_purged: The set of transactions that have …

COMMIT!

oli.sennhauser@fromdual.com (Oli Sennhauser) — Wed, 04 Jun 2014 08:16:14 +0200

Kalasha,

Since autocommit is disabled, did you issue “COMMIT” after inserting those records ? If not, then you have to either enable autocommit or commit your DML statements manually (issue “COMMIT” or execute DDL statement)

Loading "Digest::SHA1" instead

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 08 May 2014 10:50:07 +0200

Obyis,

Try to load “Digest::SHA1” instead of “Digest::SHA” at the beginning of “InnoDbStatus.pm” file by replacing “use Digest::SHA” with “use Digest::SHA1”. It should work after that.

How to Setup MySQL Master/Slave Replication ?

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 24 Apr 2014 15:53:07 +0000

It’s not usual to find an easy source on how to setup MySQL replication, I thought it might be useful at least for the beginners to write a direct and simple howto blog on setting up Master/Slave replication in MySQL using the classic method (binary log information). Check out my post GTID In Action for information about transaction-based replication using GTID.

Before going through the replication setup steps, I think it’s better to explain first how Replication works in MySQL.

MySQL replication mainly consists of three-part process:

The master server records all data changes to its binary logs (binary log events) and send it to the slave using a thread called (Binlog dump thread) once the slave connects to the master.
The slave copies the binary log events sent by the master’s binlog dump thread to its relay logs using a thread called (Slave I/O thread).
The slave applies these changes from the relay logs to its data by replaying (executing) all events using a thread called (Slave SQL thread).

Now, lets go through the setup process which is divided into 3 main sections (assuming you have already installed MySQL on master and slave servers):

Master’s side configuration:

Add the following variables to the MySQL configuration file (my.cnf):

[mysqld]
server-id=1 ## must be unique
log-bin=mysql-bin
binlog_format=ROW

Restart MySQL so that configuration changes take place:
```
shell> /etc/init.d/mysql restart
```

Create a MySQL user to be used by the slave:

SQL> GRANT REPLICATION SLAVE ON *.* TO 'slave_user_name'@'slave_ip' IDENTIFIED BY 's3cret';

Take a full snapshot from the master’s databases:

shell> mysqldump -u root -p --all-databases --flush-privileges --single-transaction --master-data=2 --flush-logs --triggers --routines --events --hex-blob >/path/to/backupdir/full_backup-$TIMESTAMP.sql

Note:

If you have MyISAM tables you should omit the option …

Setting the right GCache size in Galera Cluster

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 17 Apr 2014 13:52:10 +0000

One of our customers had a question related to the right value of Galera Cache size (gcache.size) in Galera Cluster for MySQL which I would like to share with you.

The question was: My maintenance window takes 4 hours for my 5TB DB. How can I avoid an SST ?!

Basically, having too small GCache size will lead to SST (Snapshot State Transfer) instead of IST (Incremental State Transfer), thus we can avoid the SST by setting the GCache to the appropriate value.

To check the current value of the GCache size:

mysql> SHOW GLOBAL VARIABLES LIKE 'wsrep_provider_options'<br>G
Variable_name: wsrep_provider_options
 Value: base_host = 192.168.1.12;
.
.
.
gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0;
.
.
.

The value of GCache size could be changed by adding the following line in the my.cnf file and restarting the node (it could NOT be changed online):

#my.cnf
[mysqld]
wsrep_provider_options="gcache.size=256M"

But the question is how can we calculate the right value for GCache size to cover the maintenance window and at the same time not larger than what it needs?

To answer that question we should first find out how much GCache can handle which could be calculated by the following formula:

Hold time = GCache size / Replication Rate.

Where:

Replication Rate = Amount of replicated data / time.
Amount of replicated data = (wsrep_replicated_bytes + wsrep_received_bytes) after the maintenance window - (wsrep_replicated_bytes + wsrep_received_bytes) before the maintenance window.

The amount of replicated data for the customer’s case = 7200MB.

Now, we can find out how much GCache (default 128M) can handle for the customer’s case:

Hold time = 128MB / (7200MB / 4h) = 128MB / 0.5 MB = 256s.

Then, we can calculate the right GCache size value to handle the maintenance window by the following formula:
GCache = Maintenance window
* Replication Rate = 14400s
* 0.5 MB.
GCache = 7200MB.

In other words, the right GCache size should be equivalent to (or not less than) the amount of …

Impact of General Query Log on MySQL Performance

oli.sennhauser@fromdual.com (Oli Sennhauser) — Tue, 08 Apr 2014 11:19:06 +0000

Sometimes, it is required to enable the General Query Log (which is disabled by default). If the General Query Log is enabled the server writes to this log information when clients connect or disconnect, and each SQL statement received from the client.

The question is, does enabling the General Query Log affects the MySQL performance ?
Also, it is possible to record the output of this log into either file or table in the mysql database (mysql.general_log), what is the performance impact of each one?

Let’s do some simple benchmark for those scenarios to measure the actual impact on the mysql performance.

System Information:

HW configurations:

CPU: Intel® Core™ i7-3520M Processor (4M Cache, up to 3.60 GHz).
- 2 cores, 4 threads, HT enabled.
Memory: 8GB RAM (1600).
Storage: HDD 1TB/ 5400RPM.

Software configurations:

OS: Ubuntu 12.04
MySQL Server: 5.6.17
Sysbench: 0.4.12

Test Information:

Sysbench command: sysbench --num-threads=1 --max-requests=1000 --db-driver=mysql --test=oltp --mysql-table-engine=InnoDB --mysql-user=root run .

Table structure which was used by sysbench tests:

mysql> show create table sbtest.sbtest<br>G

CREATE TABLE `sbtest` (
 `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
 `k` int(10) unsigned NOT NULL DEFAULT '0',
 `c` char(120) NOT NULL DEFAULT '',
 `pad` char(60) NOT NULL DEFAULT '',
 PRIMARY KEY (`id`),
 KEY `k` (`k`)
) ENGINE=InnoDB AUTO_INCREMENT=8574 DEFAULT CHARSET=latin1

Note:

The test had been made against 1,2,4,8,16 and 32 threads, each throughput/response time value for each thread’s number for each test case is generated by the average of ten (10) times execution.

General Query Log Disabled:

To make sure that the General Query Log is disabled:

mysql> show global variables like'general_log';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| general_log | OFF |
+---------------+-------+

Testing results:

General Query Log enabled:

The general query log is a dynamic variable which means that it could be …

What is the log output?

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 20 Mar 2014 13:00:20 +0200

Are you using FILE or TABLE as log output ?
Try “show global variables like’log_output’;”

FLUSH LOGS!

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 20 Mar 2014 12:26:03 +0200

Kalasha,
You might need to execute the SQL statement (FLUSH LOGS;) and then check the log files.

Advantage of pt-online-schema-change

oli.sennhauser@fromdual.com (Oli Sennhauser) — Fri, 21 Feb 2014 19:43:17 +0200

Hi Przemek,

I do agree with you that the replication will be blocked until the slaves finish executing the alter statement, but the table being changed on the slaves themselves wont be blocked during the alter statement the same like the master. I agree also that this could be considered as an advantage of pt-online-schema-change over Online DDL. I’ll add that to the blog. Thanks Przemek for the hint …

Replication in Online DDL

oli.sennhauser@fromdual.com (Oli Sennhauser) — Tue, 18 Feb 2014 18:28:00 +0200

Hi Shlomi

Can you please explain more on how did you produce your results?
Because Online DDL are DDL statements anyway and will be written to the binary log as statements (even if RBR is being used) which means that it will be executed on the slave the same like it was on the master.

By the way, I've tested it in a replication environment and the table being changed on the slave was not blocked during the statement execution and it took approximately the same or even less time than it was on the master.

Also I double checked the online documentation and didn't find any hints about slave blocking with online DDL Limitations of Online DDL.

Online DDL vs pt-online-schema-change

oli.sennhauser@fromdual.com (Oli Sennhauser) — Wed, 12 Feb 2014 18:14:22 +0000

One of the most expensive database operations is performing Data Definition Language (DDL, e.g. CREATE, DROP, ALTER, etc.) statements, specially, the ALTER statements because MySQL blocks the entire table for both reads and writes while modifying the table.

For the huge tables, this might take hours to get the table changed which affects the application, so that, a good planning is required for such operations in order to avoid doing these changes during the peak times. For those people who have 24/7 services or limited maintenance window, DDL on huge tables is a really nightmare.

Percona developed a very good tool called pt-online-schema-change (version 2.2.6 at the time of writing this article) to perform such operations online without blocking/affecting the application and read/write operations to the table being changed is available. Also MySQL made some enhancements for DDL statements and introduced the Online DDL feature in MySQL 5.6.

In this article, I will talk about an overview of both ways (Online DDL & pt-online-schema-change) alongside with an example and which one of them should be used in different scenarios.

pt-online-schema-change

Overview

This tool is developed by Percona to alter tables without locking them during the ALTER operation. Simply, this tool creates a new empty table like the original table with the needed structure change, copy the data from the original table in small chunks to the new table, drop the original table and then rename the new table to the original name. During the copy process all new changes to the original table are being applied to the new one because a trigger is created on the original table which ensure that all new changes will be applied on the new table.

For more information about pt-online-schema-change tool, check out the manual documentation.

Example

Altering a table called “test.test1” by adding an index (name_idx) on column “name”:

[root@gcservera ~]# pt-online-schema-change --execute --alter "add index name_idx …

NDB Alignment

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 12 Dec 2013 23:04:00 +0200

Hi Joffrey,

I think NDB tables use 4-byte alignment not InnoDB:

NDB tables use 4-byte alignment; all NDB data storage is done in multiples of 4 bytes. Thus, a column value that would typically take 15 bytes requires 16 bytes in an NDB table. For example, in NDB tables, the TINYINT, SMALLINT, MEDIUMINT, and INTEGER (INT) column types each require 4 bytes storage per record due to the alignment factor.

Check this manual page for reference.

Thanks,
Abdel-Mawla

Workbench starting/stopping multiple instance set-ups with myenv

oli.sennhauser@fromdual.com (Oli Sennhauser) — Sat, 23 Nov 2013 16:03:08 +0000

Introduction

MySQL Workbench is a very good and free GUI tool provided by Oracle to manage MySQL administration and development tasks. Opening many MySQL connections (same or different instances, remote or local MySQL servers) at the same time is one of its main features. While it’s working fine to perform SQL statements on the different connections opened for multiple instances, but some people are asking if it is available as well to start and stop multiple MySQL instances using MySQL Workbench? if yes, how to configure it to perform such task? and also does that make any conflict with MyEnv tool - if it’s installed - or not? Yes, MySQL Workbench could be configured to start and stop multiple MySQL instances (local or remote) and it does not make any conflict with MyEnv tool.

In this article, I will describe how to configure MySQL Workbench to start and stop multiple MySQL instances and getting benefits from MyEnv scripts in this purpose.

Prerequisites

System information and installed packages:

Operating System: Ubuntu 12.04 (64 bit) .
MySQL Server: Any mysql version (I used MySQL 5.5 tarballs).
Number of MySQL Instances: Two instances are installed (mysql1 & mysql2).
MySQL Workbench: Version 6.0 .
MyEnv: Version 1.0.1.

What is MyEnv?

MyEnv is a set of scripts to run comfortably multiple MySQL, Percona Server or MariaDB database instances on the same server. You can even run multiple database instances with different binary versions. If you have MySQL multiple instance setups, you really should try out MyEnv.

I will not talk more about MyEnv features and its benefits rather, I’d like to mention that if you’re using MyEnv and want to use MySQL Workbench at the same time, you will not face any conflict between them both and you can manage your MySQL instances by either MyEnv or MySQL Workbench. More over, you …

Galera 3.1 is now Released

oli.sennhauser@fromdual.com (Oli Sennhauser) — Wed, 20 Nov 2013 13:02:18 +0200

I just want to mention here that Galera 3.1 GA is now released but the online upgrade from MySQL 5.5 to 5.6 has some problems and a workaround should be made (Bug #1251137) to get this process done.

Hi,

oli.sennhauser@fromdual.com (Oli Sennhauser) — Tue, 19 Nov 2013 22:48:00 +0200

Hi,

The output you just provided could be gotten by having your database’s tables backed up individually and then restore them the same way.
For this purpose, the backup script should be something like:

#!/bin/bash
cd /backup/script/
for x in `cat tables.txt`
do 
echo $x
mysqldump --user=user_name --password='password' database_name $x > /backup/$x.sql

echo 'Table '.$x.' completed ...'
done
echo 'Database backed up successfully ...'

Where the tables.txt file contains all tables names separated by new lines.

And the restore script should be something like:

#!/bin/bash
cd /backup/script/
for x in `cat tables.txt`
do 
echo $x
mysql --user=user_name --password='password' database_name < /backup/$x.sql

echo 'Table '.$x.' completed ...'
done
echo 'Database loaded successfully ...'

OR if you have all tables backed up in only one file, you can monitor the whole process progress by using the "pv" command in the restore as follows:

# pv /backup/database_backup_file.sql | mysql --user=user_name --password='password' database_name
96.8MB 0:00:17 [5.51MB/s] [==> ] 11% ETA 0:02:10

BTW, at the end of each method, you might need to run the mysqlcheck command against the restored database to double check that everything is going fine after the restore.

I hope that helped.

Abdel-Mawla

what exactly to be monitored ?

oli.sennhauser@fromdual.com (Oli Sennhauser) — Fri, 15 Nov 2013 14:36:24 +0200

Hi Srihari,

What exactly you want to monitor in MySQL ?

Xtrabackup in a nutshell

oli.sennhauser@fromdual.com (Oli Sennhauser) — Wed, 06 Nov 2013 14:23:50 +0000

Introduction

No one can deny that one of the most important and daily tasks for DBAs is performing backup and restore operations, we’re not required to perform backup and restore operations only when we want to add new replication slave, when we want to implement disaster recovery procedures or when we want to prepare testing or staging server for the running production system, but even if we’re going to make any changes to the database schema in order to enhance the database performance, it’s recommended to have fresh backup copy before making any live changes, so if backup and restore operations cannot be handled smoothly, we’re going to face many troubles in our daily work. If we’re going to talk about backup and restore operations, Xtrabackup tool will be strongly appeared.

Xtrabackup tool is a free open source tool developed by Percona to perform physical backup and restore operations which is much faster than performing logical backup and restore using the MySQL utilities (mysqldump and mysql), and many other advantages.

Xtrabackup tool has many options and features which are very useful, but in this article, I’ll go through only on how to use this tool to perform simple full, incremental and partial backups and restores, advantages and disadvantages of each method and some important tips.

For more information about Xtrabackup tool, you can browse the manual document from here.

Prerequisites

MySQL server installed.
Download the xtrabackup tool.
Install it …

No such way available

oli.sennhauser@fromdual.com (Oli Sennhauser) — Wed, 30 Oct 2013 17:28:00 +0200

I don’t think that there’s such way in MySQL like MSSQL to track users modifications, but you can check the general log file (if it’s enabled) and search for all DCL statements related to the user in question. In the general log file, MySQL records all commands received by the client along with the issuance timestamp. I hope that helps..

Querying the GLB, draining, removing and adding nodes in CentOS

oli.sennhauser@fromdual.com (Oli Sennhauser) — Sun, 27 Oct 2013 23:26:00 +0200

Thanks Oli for the very helpful post, it really helped me to understand how to use and operate GLB, but I'd like to mention here that the option "-q" works only in Ubuntu, but in CentOS, it should be replaced by "-s" to be used as follows:

- To query the GLB on CentOS, the following command should be used instead:

 echo getinfo | nc -s 1 127.0.0.1 4444

- To drain a node from the GLB, the following command should be used:

 echo 192.168.56.101:3306:0 | nc -s 1 127.0.0.1 4444

- To add or remove a node from the GLB, the following commands should be used:

 echo 192.168.56.103:3306:-1 | nc -s 1 127.0.0.1 4444
 
 echo 192.168.56.103:3306:1 | nc -s 1 127.0.0.1 4444

Upgrade from Galera Cluster 2.x to 3.0

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 24 Oct 2013 21:25:22 +0000

Introduction

Codership announced from weeks ago introducing the Galera Cluster new release 3.0 having many bug fixes, performance enhancements plus the main purpose which is working with MySQL 5.6. In this article, I’ll go through the upgrade steps from Galera 2.x to the new release 3.0, but at the time of writing this article - as mentioned in the Codership release notes - THIS IS A BETA QUALITY RELEASE FOR TESTING PURPOSES. NOT RECOMMENDED FOR PRODUCTION YET.

Important note: a new Galera version (3.1) will be available soon for production and it will be INCOMPATIBLE with this beta version (3.0) but still compatible with 2.x, so again DO NOT go for production using 3.0 and postpone the production upgrade process until 3.1 become available.

Prerequisites

System information

The following are the cluster system information:

Operating System: CentOS release 6.4 (64 bit)
Cluster system consists of 3 cluster nodes (192.168.1.251 “gcservera”,192.168.1.252 “gcserverb” & 192.168.1.253 “gcserverc”)

Installed packages

The following are the packages installed on the three cluster nodes:

MySQL version: mysql-5.5.33_wsrep_23.7.6 (RPM)

Galera provider version:galera-23.2.7 (RPM)

mysql> show global variables like'%version%';
+-------------------------+--------------------------------------------------+
| Variable_name | Value |
+-------------------------+--------------------------------------------------+
| innodb_version | 5.5.33 |
| protocol_version | 10 |
| slave_type_conversions | …

oli.sennhauser@fromdual.com (Oli Sennhauser) — Mon, 01 Jan 0001 00:00:00 +0000

FromDual GmbH

How to recover deleted tablespace?

Things you should consider before using GTID

Table of Contents

Migration to GTID replication

Non transactionally safe statement will raise errors now

MySQL Performance in GTID

Galera Cluster and XA Transactions

Conclusion

GTID Replication talk at Percona Live London 2014

How to install multiple MySQL instances on a single host using MyEnv?

Prepare MySQL environment

Install MyEnv

Install the first instance (named master)

Galera Cluster VS PXC VS MariaDB Galera Cluster - Benchmarking

System Information:

HW configurations (AWS Servers):

Nodes Servers HW configurations:

Load balancer Server HW configurations:

Load generator Server HW configurations:

Software configurations:

Test Information:

RE: EMPTY TRANSACTIONS CAN BE DANGEROUS

Replication Troubleshooting - Classic VS GTID

Skip or Ignore statement

Sample error messages (from SHOW SLAVE STATUS output):

MySQL 5.5 and 5.6 ?!

GTID In Action

What is the concept of GTID protocol?

COMMIT!

Loading "Digest::SHA1" instead

How to Setup MySQL Master/Slave Replication ?

Master’s side configuration:

Setting the right GCache size in Galera Cluster

Impact of General Query Log on MySQL Performance

System Information:

HW configurations:

Software configurations:

Test Information:

General Query Log Disabled:

General Query Log enabled:

What is the log output?

FLUSH LOGS!

Advantage of pt-online-schema-change

Replication in Online DDL

Online DDL vs pt-online-schema-change

pt-online-schema-change

Overview

Example

NDB Alignment

Workbench starting/stopping multiple instance set-ups with myenv

Table of Contents

Introduction

Prerequisites

What is MyEnv?

Galera 3.1 is now Released

Hi,

what exactly to be monitored ?

Xtrabackup in a nutshell

Table of Contents

Introduction

Prerequisites

No such way available

Querying the GLB, draining, removing and adding nodes in CentOS

Upgrade from Galera Cluster 2.x to 3.0

Table of Contents

Introduction

Prerequisites

System information

Installed packages

Sample error messages (from `SHOW SLAVE STATUS` output):