Table of Contents
To troubleshoot issues regarding backup and restore with the MySQL Enterprise Backup product, consider the following aspects:
Before troubleshooting any problem, familiarize yourself with the known limits and restrictions on the product, in Appendix A, MySQL Enterprise Backup Limitations.
If the mysqlbackup command encounters problems during operating system calls, it returns the corresponding OS error codes. You might need to consult your operating system documentation for the meaning and solution of these error codes.
The output from the mysqlbackup command is
sent to stderr rather than
stdout. By default, the same output is also
saved to a log file in the backup_dir for
use in error diagnosis. See
Section 5.1.12, “Message Logging Options” for details on
how to configure this logging feature.
Incremental backups require care to specify a sequence of time periods You must record the final LSN value at the end of each backup, and specify that value in the next incremental backup. You must also make sure that the full backup you restore is prepared correctly first, so that it contains all the changes from the sequence of incremental backups.
As the mysqlbackup command proceeds, it
writes progress information into the
mysql.backup_progress table. When the command
finishes the backup operation, it records status information in
the mysql.backup_history table. You can query
these tables to monitor ongoing jobs, see how much time was
needed for various stages, and check if any errors occurred.
The return code of the MySQL Enterprise Backup (mysqlbackup) process is 0 if the backup or restore run succeeds. If the run fails for any reason, the return code is 1.
Sometimes the operating system or the hardware can corrupt a data file page, in a location that does not cause a database error but prevents mysqlbackup from completing:
mysqlbackup: Re-reading page at offset 0 3185082368 in /sqldata/mts/ibdata15 mysqlbackup: Re-reading page at offset 0 3185082368 in /sqldata/mts/ibdata15 mysqlbackup: Error: page at offset 0 3185082368 in /sqldata/mts/ibdata15 seems corrupt!
A corruption problem can have different causes. Here are some suggestions for dealing with it:
The problem can occur if the MySQL server is too busy. Before trying other solutions, you might want to perform the backup again using some non-default settings for the following mysqlbackup options:
--page-reread-time=MS.
Try set the value to, for example, “0.05”,
for faster rereads during checksum failures.
--page-reread-count=retry_limit.
Try set the value to, for example, “1000”,
to allow more rereads during checksum failures before
MySQL Enterprise Backup gives up and throws an error.
Scrambled data in memory can cause the problem even though the data on disk is actually uncorrupted. Reboot the database server and the storage device to see if the problem persists.
If the problem persists after the database server and the storage device have been restarted, you might really have a corruption on your disk. You might consider restoring data from an earlier backup and "roll forward" the recent changes to bring the database back to its current state.
If you want to make MySQL Enterprise Backup finish a backup anyway before you go and investigate the root cause of the issue, you can rewrite the checksum values on the disk by running the innochecksum utility on the server:
innochecksum --no-checksum --write=crc32
The option --no-checksum disable the
verification function of the tool, and the option
--write=crc32 makes
innochecksum rewrite the
checksum values on the disk.
IMPORTANT: Do not treat corruption problems as a minor annoyance. Find out what is wrong with the system that causes the corruption—however, such troubleshooting is beyond the scope of this manual.
The mysql.backup_progress table lets you
monitor backup jobs as they run. The
mysql.backup_history table lets you see the
results of completed jobs. Because these tables are created with
the CSV storage engine, you can query them from SQL, or parse the
text files from an application or script.
To skip updating these tables for a backup operation, use the
--no-history-logging option.
backup_progress Table
Each row in the backup_progress table records a
state change or message from a running backup job. The
backup_progress table has the following
columns:
backup_id
tool_name
error_code
error_message
current_time
current_state
Because the CSV storage engine cannot represent
NULL values directly, the logs use a -1 value
instead, for example in the binlog_pos column
if binary logging is not enabled.
Use the backup_id value to group together the
information for a single backup operation, and to correlate with
the corresponding row in the backup_history
table after the job is finished.
Use the error_code and
error_message values to track the progress of
the job, and detect if a serious error occurs that requires
stopping the backup operation.
Use the current_time and
current_state values to measure how long each
part of the backup operation takes, to help with planning the time
intervals for future backups.
backup_history Table
Each row in the backup_history table records
the details of one completed backup job, produced by the
mysqlbackup command. The
backup_history table has the following columns:
backup_id
tool_name
start_time
end_time
binlog_pos
binlog_file
compression_level
engines
innodb_data_file_path
innodb_file_format
start_lsn
end_lsn
backup_type
backup_format
mysql_data_dir
innodb_data_home_dir
innodb_log_group_home_dir
innodb_log_files_in_group
innodb_log_file_size
backup_destination
lock_time
exit_state
last_error
last_error_code
Use the end_lsn value to automate operations
related to incremental backups. When you take a full or
incremental backup, you specify the end LSN from that backup as
the starting LSN for the next incremental backup.
Use the values that correspond to backup-related configuration
settings, such as mysql_data_dir,
innodb_data_home_dir, and
backup_destination, to confirm that the backups
are using the right source and destination directories.
Use the values exit_state,
last_error, and
last_error_code to evaluate the success or
failure of each backup.
If last_error is 'NO_ERROR',
the backup operation was successful. In case of any errors, you
can retrieve the full list of errors for that backup operation
from the backup_progress table.
Each backup directory includes some files in the
meta subdirectory that detail how the backup
was produced, and what files it contains. The files containing
this information are known collectively as the
manifest.
mysqlbackup produces these files for use by database management tools; it does not consult or modify the manifest files after creating them. Management tools can use the manifest during diagnosis and troubleshooting procedures, for example where the original MySQL instance has been lost entirely and the recovery process is more substantial than copying files back to a working MySQL server.
The files in the manifest include:
backup_create.xml: information about the
backup operation.
backup_content.xml: information about the
files in the backup. This information is only complete and
consistent when the backup operation succeeds. The contents of
this file might be expanded in the future. A management tool
might use this information to confirm which tables are part of
a full backup, or a partial backup performed with the
--databases option. (The
information is not present for partial backups taken with the
--include,
--incremental,
--incremental-with-redo-log-only,
--only-innodb, or
--only-innodb-with-frm options.) A
management tool might compare the checksum recorded in the
manifest for a single-file backup against the checksum for the
file after the single-file backup is unpacked. The file also
contains details of all the plugins defined on the backed-up
server, by which users should make sure the same plugins are
defined in the same manner on the target server for
restoration.
image_files.xml: information about the
files in a single-file backup. (Only produced for backups
taken with the backup-to-image and
backup-dir-to-image options.) A
management tool might use the paths recorded in this file to
plan or automate the unpacking of a single-file backup using
the image-to-backup-dir or
extract options, or to remap the paths of
extracted files with the
--src-entry and
--dst-entry options.