Data Warehouse on FromDual GmbH

MariaDB 10.2 Window Function Examples

oli.sennhauser@fromdual.com (Oli Sennhauser) — Mon, 18 Apr 2016 22:39:09 +0000

MariaDB 10.2 has introduced some Window Functions for analytical queries.

MariaDB and MySQL Benchmarking

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 11 Mar 2010 23:29:59 +0000

Database Benchmark

sysbench (filesystems and OS/HW tests), Sysbench source, sysbench v0.5, sysbench scripts
OSDL Database Test Suite (download)
DBT2 (TPC-C, OLTP), DBT2 Benchmark Tool for MySQL
DBT3 (TPC-H, complex and long running queries, data warehouse (DWH))
TM1 (3^rd party, ODBC, telecom benchmark)
TPC Benchmarks
TPCC-mysql
db_STRESS
HammerDB
db_bench
Acronis perfkit (benchmark, benchmark-db)
Web3Bench (TiDB, MySQL written in Java/Python)
Finch (Finch is a MySQL benchmark tool developed for software engineers and modern infrastructures.)

General Benchmarking Tools

Apache JMeter: To test performance on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more).
Gatling: Gatling is a highly capable load testing tool. It is designed for ease of use, maintainability and high performance.
Tsung: Open-source multi-protocol distributed load testing tool.
Slamd Java-based Distributed Load Generation Engine application designed for stress testing and performance analysis of network-based applications.
Grinder A Java load testing framework that makes it easy to run a distributed test using many load injector machines.
mysqlslap mysqlslap is a diagnostic program designed to emulate client load for a MySQL server and to report the timing of each stage. It works as if multiple clients are accessing the server.
bm
httperf
apachebench (ab)

CPU Benchmarking

CoreMark

Disk benchmarking

FIO
Filesystem Benchmark: IOzone, Bonnie
Harddisk-Benchmark: Bonnie++, Iometer

# bonnie -s 2047 -S 10000

Bonnie 1.4: File './Bonnie.12330', size: 2146435072, volumes: 1
Writing with putc()... done: 20440 kB/s 89.7 %CPU
Rewriting... done: 18940 kB/s 18.3 %CPU
Writing intelligently... done: 46256 kB/s 28.8 %CPU
Reading with getc()... done: 18771 kB/s 82.1 %CPU
Reading intelligently... done: 32549 kB/s 17.3 %CPU

 ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
 -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --10k (03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
master 1*2047 20440 89.7 46256 28.8 18940 18.3 18771 82.1 32549 17.3 183.6 1.5

with fsync

MySQL pluggable Storage Engines (SE)

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 11 Mar 2010 23:20:51 +0000

One of the big advantages of MySQL is its concept of Pluggable Storage Engines. This means you can choose the most optimal Storage Engine for your needs. This also has a disadvantage: You have to know what you are doing…

We build a data warehouse from the General Query Log

oli.sennhauser@fromdual.com (Oli Sennhauser) — Wed, 31 Jan 2024 16:41:13 +0100

The design of a data warehouse differs from relational design. Data warehouses are often designed according to the concept of the star schema.

Data Warehouse Design

oli.sennhauser@fromdual.com (Oli Sennhauser) — Wed, 16 Jun 2021 23:02:43 +0200

This is my cheat sheet for dimensional modelling design techniques of a data warehouse (DWH) according to Kimball/Ross.

Query performance comparison between MariaDB ColumnStore and other Storage Engines

oli.sennhauser@fromdual.com (Oli Sennhauser) — Thu, 03 Jun 2021 15:53:51 +0200

Storage Engines like InnoDB, Aria and MyISAM are Row Stores. They store rows one after the other in blocks or even directly in a single file (MyISAM). On the other hand a Column Store like MariaDB ColumnStore stores all the same attributes (columns) of the rows together in chunks.

Create a single-node MariaDB ColumnStore test installation

oli.sennhauser@fromdual.com (Oli Sennhauser) — Wed, 19 May 2021 19:37:50 +0200

For a long time I wanted to investigate a bit more into the MariaDB ColumnStore architecture. Now I took the time to start with.

Do not trust other peoples benchmarks!

oli.sennhauser@fromdual.com (Oli Sennhauser) — Tue, 06 Apr 2021 13:26:06 +0200

Because they do NOT reflect your problems.

One of our customers upgraded last month from MariaDB 10.2 to MariaDB 10.5. In the same change he also converted all his data warehouse (DWH)/BI tables from MyISAM to Aria. An all this, naturally, without testing. And it miserably failed! And then we were under heavy time pressure to make things working again…

The battle against Oracle is probably over but has the real war begun yet?

oli.sennhauser@fromdual.com (Oli Sennhauser) — Wed, 20 Jan 2010 09:36:00 +0100

According to different sources from the web the decision about the Oracle - Sun merger has been approved by the European commission soon. So at least in the West it is clear what is going on. Let us see what the East decides… [ 1 ], [ 2 ].