Table of Contents
The ultimate purpose of backup data is to help recover from a database issue, or to create a clone of the original database in another location (typically to run report queries or to create a new replication slave). This section describes the procedures to handle those various scenarios.
After a serious database issue, you might need to perform a recovery under severe time pressure. It is critical to confirm in advance:
How long the recovery will take, including any steps to transfer, unpack, and otherwise process the data.
That you have practiced and documented all steps of the recovery process, so that you can do it correctly in one try. If a hardware issue requires restoring the data to a different server, verify all privileges, storage capacity, and so on, on that server ahead of time.
That you have periodically verified the accuracy and completeness of the backup data, so that the system will be up and running properly after being recovered.
Immediately after the backup job completes, the backup files might not be in a consistent state, because data could be inserted, updated, or deleted while the backup is running. These initial backup files are known as the raw backup.
You must update the backup files so that they reflect the state of the database corresponding to a specific InnoDB log sequence number. (The same kind of operation as crash recovery.) When this step is complete, these final files are known as the prepared backup.
During the backup, mysqlbackup copies the
accumulated InnoDB log to a file called
ibbackup_logfile. This log file is used to
“roll forward” the backed-up data files, so that
every page in the data files corresponds to the same log sequence
number of the InnoDB log. This phase also creates new
ib_logfiles that correspond to the data files.
The mysqlbackup option for turning a raw backup
into a prepared backup is apply-log.
You can run this step on the same database server where you did
the backup, or transfer the raw backup files to a different system
first, to limit the CPU and storage overhead on the database
server.
Since the apply-log operation does
not modify any of the original files in the backup, nothing is
lost if the operation fails for some reason (for example,
insufficient disk space). After fixing the problem, you can
safely retry apply-log and
by specifying the --force option,
which allows the data and log files created by the failed
apply-log operation to be
overwritten.
For simple backups (without compression or incremental backup),
you can combine the initial backup and the
apply-log step using the option
backup-and-apply-log.
You can also perform apply-log and
copy-back (which restores the prepared
backup) with a single
copy-back-and-apply-log command.
Example 4.1 Applying the Log to a Backup
This example runs mysqlbackup to roll forward the data files so that the data is ready to be restored:
mysqlbackup --backup-dir=/export/backups/2011-06-21__8-36-58 apply-log
That command creates InnoDB log files
(ib_logfile*) within the backup directory
and applies log records to the InnoDB data files
(ibdata* and *.ibd).
Example 4.2 Applying the Log to a Compressed Backup
If the backup is compressed, as in
Section 3.3.3, “Making a Compressed Backup”, specify the
--uncompress option to
mysqlbackup when applying the log to the
backup:
mysqlbackup --backup-dir=/export/backups/compressed --uncompress apply-log
Example 4.3 Applying an Incremental Backup to a Full Backup
After you take an incremental backup, as in Section 3.3.2, “Making an Incremental Backup”, the changes reflected in those backup files must be applied to a full backup to bring the full backup up-to-date, in the same way that you apply changes from the binary log.
To bring the data files from the full backup up to date, first run the apply log step so that the data files include any changes that occurred while the full backup was running. Then apply the changes from the incremental backup to the data files produced by the full backup:
mysqlbackup --backup-dir=/export/backups/full apply-log mysqlbackup --backup-dir=/export/backups/full \ --incremental-backup-dir=/export/backups/incremental \ apply-incremental-backup
Now the data files in the full-backup
directory are fully up-to-date, as of the time of the
incremental backup.
The mysqlbackup options to perform a restore
operation are copy-back and
copy-back-and-apply-log. The
restoration process requires the database server to be already
shut down (except for restorations of backups created with the
--use-tts option; see
explanations below). The
process copies the data files, logs, and other backed-up files
from the backup directory back to their original locations, and
performs any required post-processing on them. For any restore
operation, the options datadir,
innodb_log_files_in_group,
innodb_log_file_size, and
innodb_data_file_path
must be specified either in the target server's configuration
file, in the file specified by the
--defaults-file option, or
as command-line options.
Example 4.4 Shutting Down and Restoring a Database
mysqladmin --user=root --password shutdown
mysqlbackup --defaults-file=/usr/local/mysql/my.cnf \
--backup-dir=/export/backups/full \
copy-back
The restored data includes the backup_history
table, where MySQL Enterprise Backup records details of each backup. Restoring
this table to its earlier state removes information about any
subsequent backups that you did. This is the correct starting
point for future incremental backups, particularly those using
the --incremental-base option.
When performing a full restore (for example, when the backup
data is used to set up a new MySQL server or used to replace all
data of an existing MySQL server), make sure the target data
directories are all clean, containing no old or unwanted data
files. This might require manual removal of files at the
locations specified by both the
--datadir and
--innodb_data_file_path
options. The same cleanup is not required for restoring backups
created with the--use-tts
option (in which case other requirements described in
Restoring Backups Created with the --use-tts Option apply though), and usually not
necessary for restoring a partial backup.
You can combine the apply-log and the
copy-back operations (as well as a
number of other operations, depending on the kind of backup you
are restoring) into a single step by using the
copy-back-and-apply-log
option instead:
mysqlbackup --defaults-file=/usr/local/mysql/my.cnf \ --backup-dir=/export/backups/full \ copy-back-and-apply-log
Example 4.5 Restoring a Compressed Backup
Restore a compressed backup at
<backupDir> to
<restoreDir> on the server using
copy-back-and-apply-log:
mysqlbackup --defaults-file=<my.cnf> -uroot --backup-dir=<backupDir> --datadir=<restoreDir> \ --uncompress copy-back-and-apply-log
Do the same for a compressed backup image named
<image_name>, using the
--backup-dir option to
specify the temporary directory into which the image will be
extracted:
mysqlbackup --defaults-file=<backupDir>/backup-my.cnf -uroot --backup-image=<image_name> \ --backup-dir=<backupTmpDir> --datadir=<restoreDir> --uncompress copy-back-and-apply-log
See Section 3.3.3, “Making a Compressed Backup” and Section 5.1.7, “Compression Options” for more details on compressed backups.
Example 4.6 Restoring an Encrypted Backup Image
Restore an encrypted backup image named
<image_name> to
<restoreDir> on the server with
copy-back-and-apply-log, using the
encryption key contained in a file named
<keyFile> :
mysqlbackup --defaults-file=<backupDir>/backup-my.cnf --backup-image=<image_name> \ --backup-dir=<backupTmpDir> --datadir=<restoreDir> --decrypt --key-file=<keyFile> copy-back-and-apply-log
See Section 5.1.14, “Encryption Options” for more details on backup encryption and decryption.
Example 4.7 Restoring an Incremental Backup Image
mysqlbackup --defaults-file=<backupDir>/backup-my.cnf -uroot --backup-image=<inc_image_name> \ --incremental-backup-dir=<incBackupTmpDir> --datadir=<restoreDir> --incremental \ copy-back-and-apply-log
In this example, the incremental backup image named
<inc_image_name> was restored to
<restoreDir> on the server (where
the full backup that the incremental backup image was based on
has already been restored). The
--incremental-backup-dir option is
used to specify the temporary directory into which the image
will be extracted (you can use
--backup-dir for the same purpose).
Repeat the step with other incremental backup images that you
have, until the data has been restored to a desired point in
time.
Alternatively you can bring your full backup up-to-date with your incremental backup. First, apply to the full backup any changes that occurred while the backup was running:
$ mysqlbackup --backup-dir=/full-backup/2010-12-08_17-14-11 apply-log
..many lines of output...
101208 17:15:10 mysqlbackup: Full backup prepared for recovery successfully!
101208 17:15:10 mysqlbackup: mysqlbackup completed OK!
Then, we apply the changes from the incremental backup:
$ mysqlbackup --incremental-backup-dir=/incr-backup/2010-12-08_17-14-48
--backup-dir=/full-backup/2010-12-08_17-14-11 apply-incremental-backup
...many lines of output...
101208 17:15:12 mysqlbackup: mysqlbackup completed OK!
Now, the data files in the full backup directory are fully up-to-date, as of the time of the last incremental backup. You can keep updating it with more incremental backups, so it is ready to be restored anytime.
See Section 3.3.2, “Making an Incremental Backup”, and Section 5.1.8, “Incremental Backup Options”, for more details on incremental backups.
Example 4.8 Restoring Selected Tables from a TTS Backup
Selected tables can be restored from a backup created with
transportable
tablespaces (TTS) (that is, a backup created with the
--use-tts option) using the
--include-tables and
--exclude-tables options. The
following command restores all tables in the
“sales” database from the backup, but excludes the
table with the name “hardware” :
mysqlbackup --socket=/tmp/restoreserver.sock --datadir=/logs/restoreserverdata --backup-dir=/logs/backup \ --include-tables='^sales\.' --exclude-tables='^sales\.hardware$' copy-back-and-apply-log
See Restoring Backups Created with the --use-tts Option for the special requirements that apply when you restore selected tables from a TTS backup.
Example 4.9 Restoring a Single-file Backup from Cloud Storage to a MySQL Server
Restore a backup image from cloud storage to
datadir on the server, using the
--backup-dir option to
specify the temporary directory into which the image will be
extracted (see Section 5.1.15, “Cloud Storage Options”
for information on the cloud service options):
mysqlbackup\ --defaults-file=/bkups/backupdir/backupmy.cnf \ --cloud-service=s3 --cloud-aws-region=<aws region> \ --cloud-access-key-id=<aws access key id>--cloud-secret-access-key=< aws secret access key>\ --cloud-bucket=<s3 bucket name>--cloud-object-key=<aws object key>\ --backup-dir=/home/user/dba/s3backuptmpdir \ --datadir=/home/user/dba/datadir \ --backup-image=- \ copy-back-and-apply-log
There are some special requirements for restoring backups created
with the --use-tts option:
The destination server must be running.
Make sure that the required parameters for connecting to the
server (port number, socket name, etc.) are provided as
command-line options for mysqlbackup, or
are specified in the [client] section of
a default file.
The destination server must be using the same page size that was used on the server on which the backup was made.
The innodb_file_per_table option must be enabled on the destination server.
The tables being restored must not exist on the destination server.
A restore fails if the InnoDB file format of a per-table
data file (.ibd file) to be restored
does not match the value of the
innodb_file_format system
variable on the destination server. In that case, use the
--force option
with the restore commands to change temporarily the value of
innodb_file_format
on the server, in order to allow restores of per-table data
files regardless of their format.
Using MySQL Enterprise Backup and the binary
log files included by default in the incremental backups it
creates (except when they are created with the
--use-tts option), you can restore your
database to its state at an arbitrary time
tR in between a full
backup and an incremental backup or in between two incremental
backups. To do so:
Binary logging must be enabled in MySQL before taking the full backup that serves as the base for this restore operation.
Using the full and incremental backups that were created before
tR, restore the
database to a time as close to
tR as possible and
note that time (let us call it
tLB). Make sure the
restored database is in a consistent state. This can be achieved
by, for example, using the
copy-back-and-apply-log command for
backup restore.
Roll forward the database to its state at the desired time
tR using the
binary log
file[s] included in the incremental backup that covers
tR. Use the
mysqlbinlog
utility to extract from the
binary log
file[s] all the SQL activities that happened in between the
tLB (which you noted
in step 3) and tR,
specifying those times with the
--start-datetime and
--stop-datetime options, and pipe the output
to the mysql client, to be replayed on the
server:
mysqlbinlog --start-datetime="tLB" \ --stop-datetime="tR" \ binlog.000005 binlog.000006 binlog.000007 | mysql -u root -p
An alternative is to identify the beginning and endpoint of the extraction not by the start and stop times, but by their corresponding binary log positions:
mysqlbinlog --start-position="binary-log-position-corresponding-to-tLB" \ --stop-position="binary-log-position-corresponding-to-tR" \ binlog.000005 binlog.000006 binlog.000007 | mysql -u root -p
Note that you need to pipe all the binary log files in a single connection to the server. You can also dump all the SQL activities to a single file first, and then pipe or play the file to the mysql client.
For more tips on using the binary log for point-in-time recovery, see Point-in-Time (Incremental) Recovery Using the Binary Log.
A table with a table-specific tablespace (stored in an
.ibd file) can be restored individually without
taking down the MySQL server. This technique is applicable if you
delete or update the table data by mistake, without actually
losing the table itself through a DROP
TABLE, TRUNCATE TABLE, or
DROP DATABASE statement.
If you have a clean backup of an .ibd file, you
can restore it to the MySQL installation from which it originated
as follows:
For MySQL 5.5 and earlier, the table must already exist and not have been dropped or truncated since taking the backup. When an InnoDB table is truncated, or dropped and recreated, it gets a new table ID. Any ID mismatch between the table in the database and the backed-up table can prevent it from being restored. The requirement for matching table IDs is also the reason why you must restore to the same MySQL server from which the backup data came, not another server with a similar set of databases and tables. This restriction does not apply to MySQL 5.6 and later, as long as the restoration is made from one Generally Available (GA) version to another in the same series of MySQL servers.
Prevent write operations for the table to be restored. This prevents users from modifying the table while the restore is in progress.
LOCK TABLES tbl_name WRITE;
Issue this ALTER TABLE statement:
ALTER TABLE tbl_name DISCARD TABLESPACE;
Caution: This deletes the current .ibd
file.
Copy the backup .ibd file back to the
appropriate database directory.
Issue this ALTER TABLE statement:
ALTER TABLE tbl_name IMPORT TABLESPACE;
Release the write lock to complete the restore procedure:
UNLOCK TABLES;
In this context, a clean.ibd file backup means:
There are no uncommitted modifications by transactions in the
.ibd file.
There are no unmerged insert buffer entries in the
.ibd file.
Purge has removed all delete-marked index records from the
.ibd file.
mysqld has flushed all modified pages of
the .ibd file from the buffer pool to the
file.
You can make such a clean backup .ibd file with
the following method:
Stop all activity from the mysqld server and commit all transactions.
Wait until SHOW INNODB STATUS shows that
there are no active transactions in the database, and the main
thread status of InnoDB is Waiting
for server activity. Then you can make a copy of the
.ibd file.
Another method for making a clean copy of an
.ibd file is to use
mysqlbackup:
Use mysqlbackup with the
--only-innodb or
--only-innodb-with-frm option to back up the
InnoDB installation.
Run mysqlbackup ... apply-log to create a
consistent version of the backup database.
Start a second (dummy) mysqld server on the
backup and let it clean up the .ibd files
in the backup. Wait for the cleanup to end.
Shut down the dummy mysqld server.
Take a clean .ibd file from the backup.
Due to the changes to the InnoDB storage engine going from MySQL 5.5 to 5.6, restoring a backup of a MySQL 5.5 database to a MySQL 5.6 server requires some extra steps beyond the general restore and upgrade procedures, the skipping of which will crash the target server. For such a restoration, follow the steps described below.
You can back up a server running one MySQL version and restore on a server running a different MySQL version. After the restore, perform the appropriate upgrade steps as if you are running the new MySQL version for the first time. (Or, if you installed on a server running an older MySQL, perform the appropriate downgrade steps.) For information about upgrading and downgrading, see Upgrading MySQL and Downgrading MySQL.
After upgrading between certain combinations of MySQL versions, you might see error messages about missing or mismatching definitions for system tables. Use the mysql_upgrade command as directed in the upgrade instructions to correct such issues. See mysql_upgrade — Check and Upgrade MySQL Tables for instructions on this command.
Restoring a database to an older MySQL version (i.e., server downgrading) is only supported when the original and the final versions are in the same release series (e.g. going from 5.5.30 to 5.5.29). Downgrading to a lower series (e.g. from 5.6.33 to 5.5.33) might cause server crashes or data corruption.
Back up the data on the MySQL 5.5 server.
Restore the data to the directory you plan to use as the
MySQL 5.6 server's data directory by running an
apply-log and then a
copy-back
operation on the backup.
Restart the MySQL 5.5 server, using the intended data directory for the MySQL 5.6 server as its own data directory.
This is an extra step beyond the normal restore and upgrade procedures, special to the restoration of MySQL 5.5 data to MySQL 5.6 server; with it, the MySQL 5.5 server prepares the data for an upgrade to MySQL 5.6 by performing some clean-up steps on the data, similar to what the server would do during a crash recovery.
Stop the MySQL 5.5 server.
Install the MySQL 5.6 server.
Start the MySQL 5.6 server.
Run upgrade steps as documented in the MySQL reference manual on the restored data. Make sure the mysql_upgrade that comes with MySQL 5.6 is applied.
Check data.
Back up on MySQL 5.1.
Install MySQL 5.5.
Restore on MySQL 5.5.
Run upgrade steps as documented in the MySQL reference manual.
Check data.