Neuigkeiten

This is a chaotic collection of my MySQL Cluster experience…

Table of Contents

• config.ini template
• my.cnf template
• General Rules and/or experience
• MySQL Cluster restore
• Skript for converting tables to NDB (alter_engine.pl)
• MySQL Cluster memory sizing

config.ini template

A generic MySQL Cluster configuration file (config.ini) to start with. It is pretty much what the MySQL Cluster experts recommend right now:

#
# config.ini
#

# This configuration file is fore MySQL Clusters 6.2 and above...

# …

MySQL Cluster is pretty fast. The reason for this is, that it is completely memory based. Nowadays memory is still, in contrary to disk, limited to your systems. Thus, before installing a MySQL Cluster you have to calculate the amount of memory you need.

To say it in advance: You should consider to only use 64-bit Linux system with huge amount (4 - 64 GB) of RAM!

In release 5.1 MySQL Cluster became disk based. Now you have the possibility to swap out some data to disk. How much it is we will probably see a …

Table of Contents

• Result set with temporary sequence
• Determination of optimal length of prefixed indexes
• Using MySQL keywords in table or columm names
• Missing Primary Key Index
• Problems while installing a MySQL 5.5 database
• InnoDB AUTO_INCREMENT at 2^nd position

Result set with temporary sequence

Sometimes you would like to have a result set with something like a rownum. You can do this at least in the following two ways:

a) with a TEMPORARY MEMORY table:

CREATE TEMPORARY TABLE mem (
    seq  INT NOT NULL …

Controlling developers is like herding cats.
Kevin Loney, Oracle DBA Handbook

Oh no, it’s not. It’s much harder than that!
Bruce Pihlamae, long-term Oracle DBA

Do not assume!
Unknown IT specialist

Backups ist was für Warmduscher!
(engl. Backup is for sissies!)
Unkown DBA

Yesterday (the DBA version)

Yesterday,
All those backups seemed a waste of pay.
Now my database has gone away.
Oh I believe in yesterday.

Suddenly,
There’s not half the files there used to be,
And there’s a …

Table of Contents

• max_open_files warning during mysqld startup
• Troubles after NON recommended upgrade path
• Com_* counters not updated in SHOW STATUS
• ERROR 1300 (HY000): Invalid utf8 character string
• MySQL crashes during import
• MySQL workbench gives an openGL error

max_open_files warning during mysqld startup

Problem

[Warning] Changed limits: max_open_files: 1024  max_connections: 100  table_cache: 457

[Warning] Could not increase number of max_open_files to more than 1024 (request: 1070)

Explanation …

Table of Contents

• Search with special characters
• Why is InnoDB disabled?
• How to find MySQL system information?
• What is the difference between MySQL certified server and community server?
• MySQL monitoring
• MySQL backup
• Corrupt MyISAM table
• How to compile MySQL
• Test restore procedure
• Reset a MySQL user password
• Reset the MySQL root user password
• How to enable the InnoDB plugin
• Storage Engines shipped with MariaDB / MySQL
• Compiling MySQL Cluster ndb-test fails
• NDB information schema does not show up
• Hyper …

Often asked but informations are spread around: The limitations of MySQL.

If you know any other MySQL limitations, please let us know.

Table of Contents

• General limitations of MySQL
• Limitations of MySQL 4.1
• Limitations of Joins
• Limitations of the MyISAM storage engine
• Limitations of MySQL 5.0
• Limitations of Joins
• Limitations of the MyISAM storage engine
• Limitations of InnoDB
• Limitations of MySQL 5.1
• Limitations of Joins
• Limitations of Partitions
• Limitations of MySQL Cluster
• Limitations in Galera Cluster …

I really like this new toy called UDF. So I try to provide some more, hopefully useful, functionality.

A list of what I have done up to now you can find here:

• Query and change InnoDB spin_wait_delay: udf_spin_wait_delay.tgz
• Send message to the MySQL error log: udf_log_error.tgz, works also with MySQL 5.1.42

If you have some more suggestions, please let me know. If you need some special features as UDF talk to our consulting services if they can implement it.

Get and set InnoDB spin_wait_delay

mysql> …

Because I do a lot of Performance Tuning gigs I get often in contact with these status variables. In the beginning I had a problem to understand them and now I have a problem to memorize the relation of the name and the meaning. Therefore I wrote this little summary:

Prepare the example

To show you the effect I have worked out a little example:

CREATE TABLE test (
    id    INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY
  , data  VARCHAR(32)
  , ts    TIMESTAMP
  , INDEX (data)
);

INSERT INTO test
VALUES …

This MySQL Performance Tuning Key should give you a guide how to best tune you MySQL database systematically… It should also work similar for other RDBMS.

Also check our MySQL Performance Monitor

For a database configuration tuning only please look first at our MySQL database health check.

If this MySQL Database Health Check does NOT solve your problem our specialized Performance Tuning and Architecture Consultants can help you for sure!

Caution: Some recommendations are dangerous! Dangerous means …

Core files under Linux

When dealing with MySQL crashes it is very useful to get the core files for further debugging. I have collected all the informations I found about it and wrote it together here:

Find core files

# find $HOME -name "core*"
/home/oli/core.6440

# file core
core:      ELF 32-bit LSB core file Intel 80386, version 1 (SYSV), SVR4-style

See who caused the core file:

# strings core.6440 | head
CORE
CORE
mysqld
/home/mysql/product/mysql-5.1.30/bin/mysqld …

As you probably have experienced yet MySQL does not always provide all internal information as you might want to have them and as you are used to have from other RDBMS.

MySQL plans to improve this implementing the/a performance schema and its probably already partly done in MySQL 5.4. But who knows when this will be finished and what it contains at all…

What is not provided to me I want to gather myself… But how? Other RDBMS provide interfaces to attach applications directly to their memory to …

Basic solutions (CLI)

Those solutions are run from the command line (CLI):

• top (man)
• vmstat (man)
• iostat (man), mpstat (man), pidstat in package sysstat
• mytop, a Mytop Introduction
• dstat
• free (man)
• procinfo (man)
• mpstat (man)
• mTop
• InnoTop

Advanced solutions

More advanced MySQL database and host monitoring solutions with graphs and/or history and/or hints are:

Performance Tuning, Benchmarking, Capacity Planning and Monitoring

MySQL Database Health Check for MySQL/MariaDB

MySQL Performance Monitor

MySQL monitoring solutions

MySQL Performance Tuning Key

Wie der MySQL Optimizer schummelt, wenn es um MySQL Cluster geht…

How the MySQL Optimizer with MySQL Cluster is cheating you…

MySQL Benchmarking

Some more benchmarks added

My thoughts about MySQL (Cluster) replication

My wish for the New Year: MySQL DBA’s, please install iostat on your servers! …

Why is is data load with LOAD DATA INFILE so much faster?

Probably the answer to this question is already known. But we want to prove it and by the way learn to deal with oprofile.

For the test MySQL 5.0.28 was used and 100k rows were loaded into a table sales which looks as follows:

CREATE TABLE sales (
    sales_id       INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY
  , product_name   VARCHAR(128) NOT NULL
  , product_price  DECIMAL(8,2) NOT NULL
  , product_amount SMALLINT     NOT NULL
) ENGINE = …

Table of Contents

• What is a Materialized View?
• Implement your own Materialized Views
• Refreshing materialized views
• Hands on
• Create your own Materialized View:
• Refresh Materialized View on demand
• Refresh Materialized View immediate
• Materialized Views with snapshotting functionality
• Some performance benchmarks for our Materialized Views:
• Outlook
• Conclusion
• Literature

What is a Materialized View?

A Materialized View (MV) is the pre-calculated (materialized) result of a query. Unlike a simple VIEW the result …

What is a Federated Table?

A Federated Table is a table which points to a table in an other MySQL database instance (mostly on an other server). It can be seen as a view to this remote database table. Other RDBMS have similar concepts for example database links.

What can I do with a Federated Table?

To show what you can do with a federated table let us assume the following constellation: Two MySQL databases on two different servers. The first one called provider (it provides the data) the second one called …

The following tools we use sometimes for our consulting engagements…

Caution: These tools are NOT for production use! Use with care!

Tools

• allocate.c
• backslashG2table.pl
• general2bench.pl
• memuse.pl
• memwaster.c
• MySQLprofiler
• pointer_size.c
• alter_engine.pl
• commit_demo.pl
• test.sh
• vm_mon.sh
• read_process_memory.c
• read_process_memory.sh
• cluster_initial_test.pl
• mem_map.pl
• csv_converter.pl
• mem_tracker.sh
• filesystem_table.php

Sample Databases

• MySQL world: world.tgz (92 kbyte, runs with version 4.1 (tested …

Table of Contents

• Database Benchmark
• General Benchmarking Tools
• CPU Benchmarking
• Disk benchmarking
• Network Benchmarking
  • Tools
• Application Stress Testing
  • OwnCloud / NextCloud
  • Asterisk

Database Benchmark

• sysbench (filesystems and OS/HW tests), Sysbench source, sysbench v0.5, sysbench scripts
• OSDL Database Test Suite (download)
• DBT2 (TPC-C, OLTP), DBT2 Benchmark Tool for MySQL
• DBT3 (TPC-H, complex and long running queries, data warehouse (DWH))
• TM1 (3^rd party, ODBC, telecom benchmark)
• TPC Benchmarks …

One of the big advantages of MySQL is its concept of Pluggable Storage Engines. This means you can choose the most optimal Storage Engine for your needs. This also has a disadvantage: You have to know what you are doing…

If you need some help deciding which is the right Storage Engine or even strategy for you or if you want us to make an independent and vendor neutral comparison of those Storage Engines consider our FromDual Consulting Services!

MySQL provides the following storage engines:

mysql> …