Mysql terrible performance
I recently got new hardware, but I have a problem with mysql. It is slower than on the old server. When I ran the sysbench tool on the new server, I got this:
sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --mysql-user=root --mysql-password= --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run
sysbench 0.4.12: multi-threaded system evaluation benchmark
No DB drivers specified, using mysql
Running the test with following options:
Number of threads: 8
Doing OLTP test.
Running mixed OLTP test
Doing read-only test
Using Special distribution (12 iterations, 1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Threads started!
Time limit exceeded, exiting...
(last message repeated 7 times)
Done.
OLTP test statistics:
queries performed:
read: 365694
write: 0
other: 52242
total: 417936
transactions: 26121 (435.14 per sec.)
deadlocks: 0 (0.00 per sec.)
read/write requests: 365694 (6091.93 per sec.)
other operations: 52242 (870.28 per sec.)
Test execution summary:
total time: 60.0293s
total number of events: 26121
total time taken by event execution: 479.9747
per-request statistics:
min: 0.52ms
avg: 18.38ms
max: 117.91ms
approx. 95 percentile: 53.21ms
Threads fairness:
events (avg/stddev): 3265.1250/19.37
execution time (avg/stddev): 59.9968/0.01
The old server created this:
sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --mysql-user=root --mysql-password= --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run
sysbench 0.4.12: multi-threaded system evaluation benchmark
No DB drivers specified, using mysql
Running the test with following options:
Number of threads: 8
Doing OLTP test.
Running mixed OLTP test
Doing read-only test
Using Special distribution (12 iterations, 1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Threads started!
Time limit exceeded, exiting...
(last message repeated 7 times)
Done.
OLTP test statistics:
queries performed:
read: 952294
write: 0
other: 136042
total: 1088336
transactions: 68021 (1133.58 per sec.)
deadlocks: 0 (0.00 per sec.)
read/write requests: 952294 (15870.18 per sec.)
other operations: 136042 (2267.17 per sec.)
Test execution summary:
total time: 60.0052s
total number of events: 68021
total time taken by event execution: 479.5141
per-request statistics:
min: 3.27ms
avg: 7.05ms
max: 31.63ms
approx. 95 percentile: 8.60ms
Threads fairness:
events (avg/stddev): 8502.6250/11.10
execution time (avg/stddev): 59.9393/0.00
The old server has E3-1230 cpu and 16GB of RAM. The new server has E5-1660 v3 and 32GB of RAM. OS is debian wheezy for both and mysql version is 5.5.43. My.cnf below
#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html
# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
# Here is entries for some specific programs
# The following values assume you have at least 32M ram
# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
#
# * Basic Settings
#
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 127.0.0.1
#
# * Fine Tuning
#
key_buffer = 256M
key_buffer_size = 256M
sort_buffer_size = 2M
read_rnd_buffer_size = 4M
innodb_buffer_pool_size = 1G
max_allowed_packet = 32M
thread_stack = 192K
thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover = BACKUP
max_connections = 300
table_cache = 256M
#thread_concurrency = 10
#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 200M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
#slow_query_log_file = /var/log/mysql/mysql-slow.log
#slow_query_log = 1
#long_query_time = 2
#log_queries_not_using_indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1
#log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem
[mysqldump]
quick
quote-names
max_allowed_packet = 16M
[mysql]
#no-auto-rehash # faster start of mysql but no tab completition
[isamchk]
key_buffer = 16M
#
# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/
Any ideas?
source to share
query_cache_size = 200M
Disable the request cache. Even if you decide to keep it, do not set the size larger than 50M due to the cost of trimming.
table_cache = 256M
OH! You don't need a quarter of a billion tables! Go to a few hundred (no suffix).
Do this on both machines, then do delta outputs:
SHOW VARIABLES LIKE 'innodb%';
Some of the defaults have changed over time; it should point to them.
source to share
Agree with Rick that your configuration is probably not optimal, but if you actually use the same mysql configuration on both machines, it doesn't explain the discrepancy. All possible reasons for this have been listed for too long. Assuming you have collected stats in the same way on both systems (cold caches), then the most likely cause is IO related, you can check this with a benchmark comparison tool - I would recommend afio if you don't already have something that you know well what is right for the task. Troubleshooting the problem is even more difficult.
source to share