VSZ behaviour with MariaDB MEMORY tables

oli.sennhauser@fromdual.com (Oli Sennhauser) — Tue, 05 Jan 2021 17:08:57 +0000

We recently had the situation that a customer complained about the Oom killer terminating the MariaDB database instance from time to time. The MariaDB database configuration was sized quit OK (about 50% of RAM was used for the database) but they did not have swap configured.

MariaDB and MySQL swap analysis

oli.sennhauser@fromdual.com (Oli Sennhauser) — Tue, 14 Aug 2018 17:51:32 +0000

# free
 total used free shared buff/cache available
Mem: 16106332 3914228 367072 296508 11825032 11441608
Swap: 31250428 182924 31067504

# cat /proc/meminfo | grep ^Swap
SwapCached: 10056 kB
SwapTotal: 31250428 kB
SwapFree: 31067504 kB

# for file in /proc/*/status ; do
 awk '/VmSwap|Name/{
 if ( $2 != "0" )
 printf $2 " " $3
 }END{ print ""}' $file
done | grep kB | sort -k 2 -n -r

mysqld 29988 kB
plasmashell 19096 kB
kwin_x11 13444 kB
Xorg 13328 kB
krunner 11200 kB
knotes 5768 kB
kded5 5016 kB
korgac 4760 kB
kglobalaccel5 3288 kB
kwalletd 3148 kB
kaccess 3148 kB
polkit-kde-auth 2704 kB
xembedsniproxy 2660 kB
akonadi_control 2652 kB
kwalletd5 2616 kB
python 2572 kB
kactivitymanage 2508 kB
klauncher 2352 kB
kdeinit5 2324 kB
ksmserver 2256 kB
python 1480 kB
pulseaudio 1312 kB
sddm-helper 1172 kB
baloo_file 1172 kB
kscreen_backend 1132 kB
mission-control 1104 kB
python 992 kB
TeamDrive.bin 772 kB
sddm 772 kB
akonadiserver 616 kB
systemd 504 kB
at-spi2-registr 488 kB
systemd 472 kB
kwrapper5 468 kB
dbus-launch 400 kB
dbus-launch 396 kB
(sd-pam) 364 kB
dbus-daemon 328 kB
(sd-pam) 300 kB
at-spi-bus-laun 264 kB
obexd 260 kB
ssh-agent 252 kB
dbus-daemon 216 kB
gnome-keyring-d 212 kB
dbus-daemon 200 kB
gpg-agent 104 kB
startkde 100 kB
skypeforlinux 64 kB
start_kdeinit 60 kB
apache2 32 kB
apache2 32 kB
apache2 32 kB
apache2 32 kB
apache2 32 kB
apache2 32 kB

# sudo smem --totals --sort=swap
 PID User Command Swap USS PSS RSS
...
 1500 root /usr/sbin/apache2 -k start 32 5616 9534 23740 
10469 www-data /usr/sbin/apache2 -k start 32 136 1421 9848 
10470 www-data /usr/sbin/apache2 -k start 32 132 1418 9848 
10471 www-data /usr/sbin/apache2 -k start 32 132 1418 9848 
10472 www-data /usr/sbin/apache2 -k start 32 132 1418 9848 
10473 www-data /usr/sbin/apache2 -k start 32 132 1418 9848 
 2111 oli /usr/lib/x86_64-linux-gnu/l 60 40 43 548 
 2291 oli /usr/share/skypeforlinux/sk 60 73920 80765 106312 
 1957 oli /bin/sh /usr/bin/startkde 100 8 54 1576 
 2050 oli /usr/bin/gpg-agent --daemon 104 256 278 1924 
 2056 oli /usr/bin/dbus-daemon --fork 200 1740 1820 4480 
 1951 oli /usr/bin/gnome-keyring-daem 212 704 887 5768 
 2132 oli /usr/bin/dbus-daemon --conf 216 712 806 3924 
 2052 oli /usr/bin/ssh-agent /usr/bin 252 204 209 908 
 2315 oli /usr/lib/bluetooth/obexd 260 1084 1345 7936 
 2127 oli /usr/lib/at-spi2-core/at-sp 264 536 625 5700 
 1228 sddm (sd-pam) 300 1412 1728 3036 
 1476 sddm /usr/bin/dbus-daemon --fork 328 56 143 2748 
 1946 oli (sd-pam) 364 1216 1615 3132 
 1475 sddm dbus-launch --autolaunch 61 396 8 28 2252 
 2055 oli /usr/bin/dbus-launch --exit 400 8 24 2160 
 1223 sddm /lib/systemd/systemd --user 444 432 995 4376 
 1937 oli /lib/systemd/systemd --user 448 404 971 4400 
 2144 oli kwrapper5 /usr/bin/ksmserve 468 4 102 6356 
 2134 oli /usr/lib/at-spi2-core/at-sp 488 268 343 5776 
 2223 oli akonadiserver 616 11196 11401 21988 
 1129 root /usr/bin/sddm 724 716 963 11848 
 2277 oli ./TeamDrive.bin autostart 772 352656 353309 365768 
 2298 oli python /usr/bin/hp-systray 992 4904 7157 15340 
 2162 oli /usr/lib/telepathy/mission- 1104 1196 1540 8640 
 2166 oli /usr/lib/x86_64-linux-gnu/l 1132 696 973 15148 
 1934 root /usr/lib/x86_64-linux-gnu/s 1172 352 802 10996 
 2177 oli /usr/bin/baloo_file 1172 74356 74784 90584 
 2208 oli /usr/bin/pulseaudio --start 1312 4520 4897 10672 
 2299 oli python /usr/bin/hp-systray 1480 3152 5448 11184 
 2146 oli /usr/bin/ksmserver 2256 5024 6013 34104 
 2115 oli kdeinit5: Running... 2324 264 748 15492 
 2116 oli klauncher [kdeinit5] --fd=9 2352 1256 1997 21712 
 2154 oli /usr/bin/kactivitymanagerd 2472 4588 5337 29772 
 2284 oli python /usr/bin/hp-systray 2572 15216 18372 37452 
 1956 oli /usr/bin/kwalletd5 --pam-lo 2616 5272 6454 37400 
 2219 oli /usr/bin/akonadi_control 2652 2048 2450 23964 
 2195 oli /usr/bin/xembedsniproxy 2660 968 1405 23600 
 2190 oli /usr/lib/x86_64-linux-gnu/l 2704 1172 1703 26716 
 1954 oli /usr/bin/kwalletd --pam-log 3144 1180 1601 12784 
 2124 oli /usr/bin/kaccess 3148 708 1110 21276 
 2151 oli /usr/bin/kglobalaccel5 3288 1316 1799 25164 
 2203 oli /usr/bin/korgac -session 10 4760 1384 2093 28576 
 2118 oli kded5 [kdeinit5] 5016 77440 78828 107932 
 2217 oli /usr/bin/knotes -session 10 5768 7048 8467 41744 
 2179 oli /usr/bin/krunner 11200 16276 19166 55272 
 1148 root /usr/lib/xorg/Xorg -noliste 13316 31464 45953 68016 
 2167 oli kwin_x11 -session 106368656 13444 35972 39318 76556 
 2186 oli /usr/bin/plasmashell --shut 19096 90608 94701 135284 
 2224 oli /usr/sbin/mysqld --defaults 29988 148588 148624 151832 
-------------------------------------------------------------------------------
 170 14 150808 4085792 4318525 6436312 

# sudo smem --totals --sort=swap | grep -e mysql -e Command
 PID User Command Swap USS PSS RSS 
 6903 mysql /home/mysql/product/mariadb 0 191000 191071 194964 
16937 mysql /home/mysql/product/mysql-8 0 358812 358865 362364 
17480 mysql /home/mysql/product/mysql-5 0 271420 273310 278692 
28484 root grep -e mysql -e Command 0 352 395 2404 
29071 mysql /bin/sh bin/mysqld_safe --d 0 152 227 1724 
29326 mysql /home/mysql/product/mysql-5 0 506848 508747 514268 
 2224 oli /usr/sbin/mysqld --defaults 29988 148588 148624 151832

Oom on FromDual GmbH

VSZ behaviour with MariaDB MEMORY tables

MariaDB and MySQL swap analysis