Table of Contents
The key to safe database management is making regular backups. Depending on your data volume, number of MySQL servers, and database workload, you can use these techniques, alone or in combination: hot backup with MySQL Enterprise Backup; cold backup by copying files while the MySQL server is shut down; physical backup for fast operation (especially for restore); logical backup with mysqldump for smaller data volumes or to record the structure of schema objects.
The mysqlbackup command, part of the MySQL
Enterprise Backup component, lets you back up a running MySQL
instance, including InnoDB and
MyISAM tables, with minimal disruption
to operations while producing a consistent snapshot of the database.
When mysqlbackup is copying
InnoDB tables, reads and writes to both
InnoDB and MyISAM tables can
continue. During the copying of MyISAM tables,
reads (but not writes) to those tables are permitted. MySQL
Enterprise Backup can also create compressed backup files, and back
up subsets of tables and databases. In conjunction with MySQL’s
binary log, users can perform point-in-time recovery. MySQL
Enterprise Backup is part of the MySQL Enterprise subscription. For
more details, see MySQL Enterprise Backup Overview.
If you can shut down your MySQL server, you can make a binary backup
that consists of all files used by InnoDB to
manage its tables. Use the following procedure:
Do a slow shutdown of the MySQL server and make sure that it stops without errors.
Copy all InnoDB data files
(ibdata files and .ibd
files) into a safe place.
Copy all the .frm files for
InnoDB tables to a safe place.
Copy all InnoDB log files
(ib_logfile files) to a safe place.
Copy your my.cnf configuration file or
files to a safe place.
In addition to making binary backups as just described, regularly
make dumps of your tables with mysqldump. A
binary file might be corrupted without you noticing it. Dumped
tables are stored into text files that are human-readable, so
spotting table corruption becomes easier. Also, because the format
is simpler, the chance for serious data corruption is smaller.
mysqldump also has a
--single-transaction option for
making a consistent snapshot without locking out other clients. See
Section 1.3.1, “Establishing a Backup Policy”.
Replication works with InnoDB tables,
so you can use MySQL replication capabilities to keep a copy of your
database at database sites requiring high availability.
To recover your InnoDB database to the present
from the time at which the binary backup was made, you must run your
MySQL server with binary logging turned on, even before taking the
backup. To achieve point-in-time recovery after restoring a backup,
you can apply changes from the binary log that occurred after the
backup was made. See Section 1.5, “Point-in-Time (Incremental) Recovery Using the Binary Log”.
To recover from a crash of your MySQL server, the only requirement
is to restart it. InnoDB automatically checks the
logs and performs a roll-forward of the database to the present.
InnoDB automatically rolls back uncommitted
transactions that were present at the time of the crash. During
recovery, mysqld displays output something like
this:
InnoDB: Database was not shut down normally. InnoDB: Starting recovery from log files... InnoDB: Starting log scan based on checkpoint at InnoDB: log sequence number 0 13674004 InnoDB: Doing recovery: scanned up to log sequence number 0 13739520 InnoDB: Doing recovery: scanned up to log sequence number 0 13805056 InnoDB: Doing recovery: scanned up to log sequence number 0 13870592 InnoDB: Doing recovery: scanned up to log sequence number 0 13936128 ... InnoDB: Doing recovery: scanned up to log sequence number 0 20555264 InnoDB: Doing recovery: scanned up to log sequence number 0 20620800 InnoDB: Doing recovery: scanned up to log sequence number 0 20664692 InnoDB: 1 uncommitted transaction(s) which must be rolled back InnoDB: Starting rollback of uncommitted transactions InnoDB: Rolling back trx no 16745 InnoDB: Rolling back of trx no 16745 completed InnoDB: Rollback of uncommitted transactions completed InnoDB: Starting an apply batch of log records to the database... InnoDB: Apply batch completed InnoDB: Started mysqld: ready for connections
If your database becomes corrupted or disk failure occurs, you must perform the recovery using a backup. In the case of corruption, first find a backup that is not corrupted. After restoring the base backup, do a point-in-time recovery from the binary log files using mysqlbinlog and mysql to restore the changes that occurred after the backup was made.
In some cases of database corruption, it is enough just to dump,
drop, and re-create one or a few corrupt tables. You can use the
CHECK TABLE SQL statement to check
whether a table is corrupt, although CHECK
TABLE naturally cannot detect every possible kind of
corruption. You can use the Tablespace Monitor to check the
integrity of the file space management inside the tablespace files.
In some cases, apparent database page corruption is actually due to
the operating system corrupting its own file cache, and the data on
disk may be okay. It is best first to try restarting your computer.
Doing so may eliminate errors that appeared to be database page
corruption. If MySQL still has trouble starting because of
InnoDB consistency problems, see
Forcing InnoDB Recovery for steps to start the
instance in a diagnostic mode where you can dump the data.
InnoDB
crash recovery consists
of several steps:
Applying the redo log:
Redo log application is the first step and is performed during
initialization, before accepting any connections. If all
changes were flushed from the buffer pool to the tablespaces
(ibdata* and *.ibd
files) at the time of the shutdown or crash, the redo log
application can be skipped. If the redo log files are missing
at startup, InnoDB skips the redo log
application.
Removing redo logs to speed up the recovery process is not
recommended, even if some data loss is acceptable. Removing
redo logs should only be considered an option after a clean
shutdown is performed, with
innodb_fast_shutdown set to
0 or 1.
Rolling back incomplete transactions: Any transactions that were active at the time of crash or fast shutdown. The time it takes to roll back an incomplete transaction can be three or four times the amount of time a transaction is active before it is interrupted, depending on server load.
You cannot cancel transactions that are in the process of
being rolled back. In extreme cases, when rolling back
transactions is expected to take an exceptionally long time,
it may be faster to start InnoDB with an
innodb_force_recovery setting
of 3 or greater. See
Forcing InnoDB Recovery for more
information.
Insert buffer merge: Applying changes from the insert buffer (part of the system tablespace) to leaf pages of secondary indexes, as the index pages are read to the buffer pool.
Purge: Deleting delete-marked records that are no longer visible for any active transaction.
The steps that follow redo log application do not depend on the redo log (other than for logging the writes) and are performed in parallel with normal processing. Of these, only rollback of incomplete transactions is special to crash recovery. The insert buffer merge and the purge are performed during normal processing.
After redo log application, InnoDB attempts to
accept connections as early as possible, to reduce downtime. As
part of crash recovery, InnoDB rolls back any
transactions that were not committed or in XA
PREPARE state when the server crashed. The rollback is
performed by a background thread, executed in parallel with
transactions from new connections. Until the rollback operation is
completed, new connections may encounter locking conflicts with
recovered transactions.
In most situations, even if the MySQL server was killed
unexpectedly in the middle of heavy activity, the recovery process
happens automatically and no action is needed from the DBA. If a
hardware failure or severe system error corrupted
InnoDB data, MySQL might refuse to start. In
that case, see Forcing InnoDB Recovery for the
steps to troubleshoot such an issue.
For information about the binary log and InnoDB
crash recovery, see The Binary Log.
InnoDB implements a checkpoint mechanism known
as “fuzzy” checkpointing. InnoDB
flushes modified database pages from the buffer pool in small
batches. There is no need to flush the buffer pool in one single
batch, which would in practice stop processing of user SQL
statements during the checkpointing process.
During crash recovery, InnoDB looks for a
checkpoint label written to the log files. It knows that all
modifications to the database before the label are present in the
disk image of the database. Then InnoDB scans
the log files forward from the checkpoint, applying the logged
modifications to the database.
InnoDB writes to its log files on a rotating
basis. It also writes checkpoint information to the first log file
at each checkpoint. All committed modifications that make the
database pages in the buffer pool different from the images on
disk must be available in the log files in case
InnoDB has to do a recovery. This means that
when InnoDB starts to reuse a log file, it has
to make sure that the database page images on disk contain the
modifications logged in the log file that
InnoDB is going to reuse. In other words,
InnoDB must create a checkpoint and this often
involves flushing of modified database pages to disk.
The preceding description explains why making your log files very large may reduce disk I/O in checkpointing. It often makes sense to set the total size of the log files as large as the buffer pool or even larger. The disadvantage of using large log files is that crash recovery can take longer because there is more logged information to apply to the database.