MySQL Tech-Feed (en)

Taxonomy upgrade extras: galeraclusterMySQL Clusterload balancerlvsipvsHigh Availability

Recently we had to set-up a 3-node Galera Cluster with a Load Balancer in front of it. Because Galera Cluster nodes (mysqld) still reply to TCP requests on port 3306 when they are expelled from the Cluster it is not sufficient to just leave it to the Load Balancer to check the port if a Galera node is properly running or not.

We used the wsrep_notify_cmd variable to hook our own script into the Galera Cluster which disables each Node on the Load Balancer when its state changed.

# my.cnf # [mysqld] wsrep_notify_cmd = /usr/local/bin/lvs_control.sh

The whole Galera Cluster Architecture looks as follows:

As Load Balancer we used the IPVS Load Balancer from the Linux Virtual Server (LVS) Project. This Load Balancer was made highly available with keepalived.

Our script to take a Galera Node out of the Load Balancer was the following:

#!/bin/bash -eu # # /etc/mysql/conf.d/wsrep.cnf # # [mysqld] # wsrep_notify_cmd = /usr/local/bin/lvs_control.sh # LOG="/tmp/lvs_control.log" LBIP="192.168.0.89" VIP="192.168.0.99" PORT="3306" LBUSER="galera" LBUSER="root" ETC="/etc/mysql/conf.d/wsrep.cnf" ETC="/home/mysql/data/mysql-5.5-wsrep-23.7-a/my.cnf" MYIP='' WEIGHT="100" DATE=$(date '+%Y-%m-%d %H:%M:%S') echo $DATE >>$LOG regex='^.*=\s*([0-9]+.[0-9]+.[0-9]+.[0-9]+).*' str=$(grep "^wsrep_node_incoming_address" $ETC 2>>$LOG) if [[ $str =~ $regex ]] ; then MYIP=${BASH_REMATCH[1]} else echo "Cannot find IP address in $str" >>$LOG exit 1 fi while [ $# -gt 0 ] ; do case $1 in --status) STATUS=$2 shift ;; --uuid) CLUSTER_UUID=$2 shift ;; --primary) PRIMARY=$2 shift ;; --index) INDEX=$2 shift ;; --members) MEMBERS=$2 shift ;; esac shift done # echo $* >> $LOG echo $STATUS >> $LOG # Undefined means node is shutting down # Synced means node is ready again if [ "$STATUS" != "Synced" ] ; then cmd="ssh $LBUSER@$LBIP 'sudo /sbin/ipvsadm -e -t $VIP:$PORT -r $MYIP -w 0'" else cmd="ssh $LBUSER@$LBIP 'sudo /sbin/ipvsadm -e -t $VIP:$PORT -r $MYIP -w $WEIGHT'" fi echo $cmd >>$LOG eval $cmd >>$LOG 2>&1 echo "ret=$?" >>$LOG exit 0

We assume that the same script can be used with little modifications for the Galera Load Balancer as well.

FromDual is looking for professional, enthusiastic and experienced people who:

• Know MySQL, Percona Server or MariaDB extensively
• Are Familiar with the open source eco-system
• Know how to operate database systems, as a DBA or a DevOps
• Understand what can go wrong in operating a database
• Are happy to work autonomously, remotely and to communicate with IRC, Skype, Mail and Phone
• Are comfortable on Linux systems
• Are team players, keen to contribute to the growth of the company
• Are Comfortable dealing direct with clients and
• Look for new challenges

Job description

We are looking for full-time MySQL support engineers (female or male) to primarily take care of our MySQL support services and help our customers operating their MySQL databases (remote-DBA and emergency interventions).

You are well trained and have good experience in:

• Operating critical highly available MySQL production databases mostly on Linux.
• Running MySQL-Replication in all variants is your daily business.
• The working of the most used MySQL HA set-up's and how to fix them efficiently if problems occur. (If you are already experienced in running Galera Cluster this would be a plus!
• Open Source Technologies (LAMP stack, etc.)
• Bash scripting and you can do some simple programs in at least one popular programming/scripting language (Perl, PHP, ...).

You will be in direct contact with the customers and you need good antennae to listen to them, know how to respond and get the answers to their real problems. You also have to be proactive when something goes wrong and direct the customer back to the right track.

You need to have good communication skills and be an active team player.

To fulfil your job you have to work in European Time Zones. You can organize your working time flexible within certain ranges. Participating in the on call duty is expected. FromDual is a completely virtual company and relocation is not needed (home office). Good English verbally and in writing is a must. Most of our current customers speak German and having German skills is a plus.

Beside being our support engineer we expect you to improve your knowledge and skills and to contribute to improving our monitoring solution, our database controlling solution and our other tools. Further, we expect that you write regular technical articles and give help wherever it is needed or requested...

You should be prepared to work, think and act autonomously most of the time and to teach yourself (using Google, MySQL documentation, testing etc.). If you are ever stuck, your colleagues at FromDual will assist you.

If you need somebody holding your hand all the time, FromDual is not a good choice for you.

Who is FromDual?

FromDual is the leading independent and professional MySQL database consulting and service company in Europe with its Headquarters in Switzerland.

Our customers are mostly located in Europe and range from small start-up companies to some of the top-500 companies of Europe.

You will be joining us at an exciting time. We are growing and we need like-minded people to grow with us, individually and collectively. As our horizons expand, we need our team to expand in its skills, knowledge and expertise.

Applying to join FromDual could be the best decision you make.

How to continue

If you are interested in this opportunity and if you feel you are a good "fit" (we know that there will not be a 100% match!) we would be glad to hear from you.

Please send your true CV with your salary expectation and a list of your open source involvements, blog articles, slides, tweets etc. to jobs@fromdual.com. If you want to know more about this job opportunity or if you want to speak with me, please call me at +41 79 830 09 33 (Oli Sennhauser, CTO). Only candidates, NO head hunters please!

After we received and screened your CV we will invite you to prove your technical skills by taking an exam in operating MySQL. If you pass the exam you will be invited for the final interviews.

This job opportunity is open until May 31st 2013.

Taxonomy upgrade extras: galeramyisaminnodbstorage engine

Switching from MySQL/MyISAM to Galera Cluster requires that all tables (except those from the mysql, information_schema and performance_schema) are using the InnoDB Storage Engine.

For altering the Storage Engine of the tables we wrote a script (alter_engine.pl) long time ago already. Because we have made many of those switches recently we have extended its functionality.

New features

• Recognizes VIEW's and does NOT try to alter their Storage Engine (bug).
• Script is MySQL version aware. Complain if too old MySQL version is used.
• Find tables without a Primary Key.
• Check for too long InnoDB Primary Keys
• Check for FULLTEXT indexes in MySQL 5.1 and 5.5 and write a note if version is older.

Example ./alter_engine.pl User [root] : Password [] : secret Schema from (or all) [test] : all Engine to [InnoDB] : Version is : 5.6.10 MR Version is: 050610 The following tables might not have a Primary Key: +--------------+----------------------+ | table_schema | table_name | +--------------+----------------------+ | test | innodb_table_monitor | | test | log_event | | test | parent | | test | t | +--------------+----------------------+ The tables above not having a Primary Key will negatively affect perfor- mance and data consistency in MySQL Master/Slave replication and Galera Cluster replication. The following tables might have a too long Primary Key for InnoDB (> 767 bytes): +--------------+------------+-------------+ | table_schema | table_name | column_name | +--------------+------------+-------------+ | test | test | data | +--------------+------------+-------------+ The following tables might have a FULLTEXT index (which is only supported in MySQL 5.6 and newer): +--------------+------------+-------------+ | table_schema | table_name | column_name | +--------------+------------+-------------+ | test | test | data | +--------------+------------+-------------+ Output written to /tmp/alter_table_all.sql After reviewing it you can apply it with mysql --user=root --password=secret