Chapter 3 Backing Up a Database Server

Table of Contents

3.1 Before the First Backup
3.1.1 Collect Database Information
3.1.2 Grant MySQL Privileges to Backup Administrator
3.1.3 Designate a Location for Backup Data
3.2 The Typical Backup / Verify / Restore Cycle
3.2.1 Backing Up an Entire MySQL Instance
3.2.2 Verifying a Backup
3.2.3 Restoring a Database at its Original Location
3.3 Backup Scenarios and Examples
3.3.1 Making a Full Backup
3.3.2 Making an Incremental Backup
3.3.3 Making a Compressed Backup
3.3.4 Making a Partial Backup
3.3.5 Making a Single-File Backup
3.3.6 Making an Optimistic Backup
3.3.7 Making a Back Up of In-Memory Database Data
3.3.8 Making Scheduled Backups

This section describes the different kinds of backups that MySQL Enterprise Backup can create and the techniques for producing them, with examples showing the relevant syntax for the mysqlbackup command. It also includes a full syntax reference for the mysqlbackup command.

3.1 Before the First Backup

The best practices for backups involve planning and strategies. This section outlines some of the preparation needed to put such plans and strategies in place.

3.1.1 Collect Database Information

Before backing up a particular database server for the first time, gather some information and decide on some directory names, as outlined in the following table.

Table 3.1 Information Needed to Back Up a Database

Information to Gather

Where to Find It

How Used

Path to MySQL configuration file

Default system locations, hardcoded application default locations, or from --defaults-file option in mysqld startup script.

This is the preferred way to convey database configuration information to the mysqlbackup command, using the --defaults-file option. When connection and data layout information is available from the configuration file, you can skip most of the other choices listed below.

MySQL port

MySQL configuration file or mysqld startup script.

Used to connect to the database instance during backup operations. Specified via the --port option of mysqlbackup. --port is not needed if available from MySQL configuration file. Not needed when doing an offline (cold) backup, which works directly on the files using OS-level file permissions.

Path to MySQL data directory

MySQL configuration file or mysqld startup script.

Used to retrieve files from the database instance during backup operations, and to copy files back to the database instance during restore operations. Automatically retrieved from database connection for hot and warm backups. Taken from MySQL configuration file for cold backups.

ID and password of privileged MySQL user

You record this during installation of your own databases, or get it from the DBA when backing up databases you do not own. Not needed when doing an offline (cold) backup, which works directly on the files using OS-level file permissions. For cold backups, you log in as an administrative user.

Specified via the --password option of the mysqlbackup. Prompted from the terminal if the --password option is present without the password argument.

Path under which to store backup data

You choose this. See Section 3.1.3, “Designate a Location for Backup Data” for details.

By default, this directory must be empty for mysqlbackup to write data into it, to avoid overwriting old backups or mixing up data from different backups. Use the --with-timestamp option to automatically create a subdirectory with a unique name, when storing multiple sets of backup data under the same main directory.

Owner and permission information for backed-up files (for Linux, Unix, and OS X systems)

In the MySQL data directory.

If you do the backup using a different OS user ID or a different umask setting than applies to the original files, you might need to run commands such as chown and chmod on the backup data. See Section A.1, “Limitations of MySQL Enterprise Backup” for details.

Size of InnoDB redo log files

Calculated from the values of the innodb_log_file_size and innodb_log_files_in_group configuration variables. Use the technique explained for the --incremental-with-redo-log-only option.

Only needed if you perform incremental backups using the --incremental-with-redo-log-only option rather than the --incremental option. The size of the InnoDB redo log and the rate of generation for redo data dictate how often you must perform incremental backups.

Rate at which redo data is generated

Calculated from the values of the InnoDB logical sequence number at different points in time. Use the technique explained for the --incremental-with-redo-log-only option.

Only needed if you perform incremental backups using the --incremental-with-redo-log-only option rather than the --incremental option. The size of the InnoDB redo log and the rate of generation for redo data dictate how often you must perform incremental backups.


3.1.2 Grant MySQL Privileges to Backup Administrator

For most backup operations, the mysqlbackup command connects to the MySQL server through --user and --password options. This user requires certain privileges. You can either create a new user with a minimal set of privileges, or use an administrative account such as the root user.

The minimum privileges for the MySQL user that mysqlbackup connects are:

  • RELOAD on all databases and tables.

  • CREATE, INSERT, DROP, and UPDATE on the tables mysql.backup_progress and mysql.backup_history, and also SELECT on mysql.backup_history.

  • SUPER, to enable and disable logging, and to optimize locking in order to minimize disruption to database processing.

  • REPLICATION CLIENT, to retrieve the binlog position, which is stored with the backup.

To set these privileges for a MySQL user (mysqlbackup in this example) connecting from localhost, issue statements like the following from the mysql client program:

GRANT RELOAD ON *.* TO 'mysqlbackup'@'localhost';
GRANT CREATE, INSERT, DROP, UPDATE ON mysql.backup_progress TO 'mysqlbackup'@'localhost';
GRANT CREATE, INSERT, SELECT, DROP, UPDATE ON mysql.backup_history TO 'mysqlbackup'@'localhost';
GRANT REPLICATION CLIENT ON *.* TO 'mysqlbackup'@'localhost';
GRANT SUPER ON *.* TO 'mysqlbackup'@'localhost';

The following additional privileges are required for using transportable tablespaces (TTS) to back up and restore InnoDB tables:

  • LOCK TABLES and SELECT for backing up tables

  • CREATE and ALTER for restoring tables

To set these privileges, issue a statement like the following from the mysql client program:

GRANT LOCK TABLES, SELECT, CREATE, ALTER ON *.* TO 'mysqlbackup'@'localhost';

3.1.3 Designate a Location for Backup Data

All backup-related operations either create new files or reference existing files underneath a specified directory that holds backup data. Choose this directory in advance, on a file system with sufficient storage. (It could even be remotely mounted from a different server.) You specify the path to this directory with the --backup-dir option for many invocations of the mysqlbackup command.

Once you establish a regular backup schedule with automated jobs, it is preferable to keep each backup within a timestamped subdirectory underneath the main backup directory. To make the mysqlbackup command create these subdirectories automatically, specify the --with-timestamp option each time you run mysqlbackup.

For one-time backup operations, for example when cloning a database to set up a replication slave, you might specify a new directory each time, or specify the --force option of mysqlbackup to overwrite older backup files.

3.2 The Typical Backup / Verify / Restore Cycle

To illustrate the basic steps in making and using a backup, the following examples show how to do a full backup, examine the data files in the backup directory, and then restore the backup to correct an issue with corruption or lost data.

3.2.1 Backing Up an Entire MySQL Instance

In this example, we specify all required options on the command line for illustration purposes. After testing and standardizing the backup procedure, we could move some options to the MySQL configuration file. The options specify connection information for the database and the location to store the backup data. The final option backup specifies the type of operation, because mysqlbackup can perform several kinds of backup, restore, and pack/unpack operations.

For this example, we specify the final option as backup-and-apply-log. This option performs an extra stage after the initial backup, to bring all InnoDB tables up-to-date with any changes that occurred during the backup operation, so that the backup is immediately ready to be restored. For backups of huge or busy databases, you might split up these stages to minimize load on the database server. That is, run mysqlbackup first with the backup option, transfer the backup to another server, then run mysqlbackup with the apply-log option to perform the final processing.

The output echoes all the parameters used by the backup operation, including several that are retrieved automatically using the database connection. The unique ID for this backup job is recorded in special tables that mysqlbackup creates inside the instance, allowing you to monitor long-running backups and view the results of previous backups. The final output section repeats the location of the backup data and provides the LSN values that you might use when you perform an incremental backup next time over the full backup that is just made.

$ ./mysqlbackup --user=root -p --backup-dir=/home/admin/backups backup-and-apply-log

MySQL Enterprise Backup version 3.11.1 Linux-2.6.18-274.el5-i686 [2014/09/04]
Copyright (c) 2003, 2014, Oracle and/or its affiliates. All Rights Reserved.

 mysqlbackup: INFO: Starting with following command line ...
 ./mysqlbackup --user=root -p --backup-dir=/home/admin/backups
        backup-and-apply-log

 mysqlbackup: INFO:
Enter password:
 mysqlbackup: INFO: MySQL server version is '5.6.17-log'.
 mysqlbackup: INFO: Got some server configuration information from running server.

IMPORTANT: Please check that mysqlbackup run completes successfully.
           At the end of a successful 'backup-and-apply-log' run mysqlbackup
           prints "mysqlbackup completed OK!".

140904 12:34:54 mysqlbackup: INFO: MEB logfile created at /home/admin/backups/meta/MEB_2014-09-04.12-34-54_backup_apply_log.log

--------------------------------------------------------------------
                       Server Repository Options:
--------------------------------------------------------------------
  datadir = /var/lib/mysql/
  innodb_data_home_dir =
  innodb_data_file_path = ibdata1:12M:autoextend
  innodb_log_group_home_dir = /var/lib/mysql/
  innodb_log_files_in_group = 2
  innodb_log_file_size = 50331648
  innodb_page_size = 16384
  innodb_checksum_algorithm = innodb
  innodb_undo_directory = /var/lib/mysql/
  innodb_undo_tablespaces = 0
  innodb_undo_logs = 128

--------------------------------------------------------------------
                       Backup Config Options:
--------------------------------------------------------------------
  datadir = /home/admin/backups/datadir
  innodb_data_home_dir = /home/admin/backups/datadir
  innodb_data_file_path = ibdata1:12M:autoextend
  innodb_log_group_home_dir = /home/admin/backups/datadir
  innodb_log_files_in_group = 2
  innodb_log_file_size = 50331648
  innodb_page_size = 16384
  innodb_checksum_algorithm = innodb
  innodb_undo_directory = /home/admin/backups/datadir
  innodb_undo_tablespaces = 0
  innodb_undo_logs = 128

 mysqlbackup: INFO: Unique generated backup id for this is 14098484948398421

 mysqlbackup: INFO: Creating 14 buffers each of size 16777216.
140904 12:34:57 mysqlbackup: INFO: Full Backup operation starts with following threads
        1 read-threads    6 process-threads    1 write-threads
140904 12:34:57 mysqlbackup: INFO: System tablespace file format is Antelope.
140904 12:34:57 mysqlbackup: INFO: Starting to copy all innodb files...
140904 12:34:57 mysqlbackup: INFO: Found checkpoint at lsn 1611237.
140904 12:34:57 mysqlbackup: INFO: Starting log scan from lsn 1610752.
140904 12:34:57 mysqlbackup: INFO: Copying log...
140904 12:34:57 mysqlbackup: INFO: Log copied, lsn 1611237.
140904 12:34:57 mysqlbackup: INFO: Copying /var/lib/mysql/ibdata1 (Antelope file format).
140904 12:34:59 mysqlbackup: INFO: Completing the copy of innodb files.
140904 12:34:59 mysqlbackup: INFO: Starting to copy Binlog files...
140904 12:34:59 mysqlbackup: INFO: Preparing to lock tables: Connected to mysqld server.
140904 12:34:59 mysqlbackup: INFO: Starting to lock all the tables...
140904 12:35:00 mysqlbackup: INFO: All tables are locked and flushed to disk
140904 12:35:00 mysqlbackup: INFO: Opening backup source directory '/var/lib/mysql/'
140904 12:35:00 mysqlbackup: INFO: Starting to backup all non-innodb files in
    subdirectories of '/var/lib/mysql/'
140904 12:35:00 mysqlbackup: INFO: Copying the database directory 'performance_schema'
140904 12:35:01 mysqlbackup: INFO: Completing the copy of all non-innodb files.
140904 12:35:01 mysqlbackup: INFO: Completed the copy of binlog files...
140904 12:35:02 mysqlbackup: INFO: A copied database page was modified at 1611237.
          (This is the highest lsn found on page)
          Scanned log up to lsn 1611237.
          Was able to parse the log up to lsn 1611237.
          Maximum page number for a log record 0
140904 12:35:02 mysqlbackup: INFO: All tables unlocked
140904 12:35:02 mysqlbackup: INFO: All MySQL tables were locked for 2.530 seconds.
140904 12:35:03 mysqlbackup: INFO: Reading all global variables from the server.
140904 12:35:03 mysqlbackup: INFO: Completed reading of all global variables from the server.
140904 12:35:03 mysqlbackup: INFO: Creating server config files server-my.cnf and server-all.cnf in /home/admin/backups
140904 12:35:03 mysqlbackup: INFO: Full Backup operation completed successfully.
140904 12:35:03 mysqlbackup: INFO: Backup created in directory '/home/admin/backups'
140904 12:35:03 mysqlbackup: INFO: MySQL binlog position: filename mysqld-bin.000001, position 120

-------------------------------------------------------------
   Parameters Summary        
-------------------------------------------------------------
   Start LSN                  : 1610752
   End LSN                    : 1611237
-------------------------------------------------------------


 mysqlbackup: INFO: Creating 14 buffers each of size 65536.
140904 12:35:03 mysqlbackup: INFO: Apply-log operation starts with following threads
        1 read-threads    1 process-threads
 mysqlbackup: INFO: Using up to 100 MB of memory.
140904 12:35:03 mysqlbackup: INFO: ibbackup_logfile's creation parameters:
          start lsn 1610752, end lsn 1611237,
          start checkpoint 1611237.
 mysqlbackup: INFO: InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
 mysqlbackup: INFO: InnoDB: Setting log file size to 50331648
 mysqlbackup: INFO: InnoDB: Setting log file size to 50331648
140904 12:35:14 mysqlbackup: INFO: We were able to parse ibbackup_logfile up to
          lsn 1611237.
 mysqlbackup: INFO: Last MySQL binlog file position 0 120, file name mysqld-bin.000001:120
140904 12:35:14 mysqlbackup: INFO: The first data file is '/home/admin/backups/datadir/ibdata1'
          and the new created log files are at '/home/admin/backups/datadir'
140904 12:35:14 mysqlbackup: INFO: Apply-log operation completed successfully.
140904 12:35:14 mysqlbackup: INFO: Full backup prepared for recovery successfully.

mysqlbackup completed OK!

Now the backup subdirectory is created under the backup-dir we specified. The directory name for each new backup is formed from the date and the clock time when the backup run was started, in the local time zone. The backup directory contains the backed-up ibdata files and ibbackup_logfile. Each subdirectory corresponds to a MySQL database, and contains copies of .frm, .MYD, .MYI, and similar files. For an example of the layout of such a backup directory, see Section C.1, “Sample Directory Structure for Full Backup”.

3.2.2 Verifying a Backup

To verify the backup, restore the backup data on a different server and run the MySQL daemon (mysqld) on the new data directory. Then you can execute SHOW statements to verify the database and table structure, and execute queries to verify the number of rows, latest updates, and so on.

This is the same general technique to use when you intend to put the backup data to some other use. For example, you might set up a replication slave by making a backup of the master server, or turn a backup into a new MySQL instance for running report queries.

Note

Always do verification against restored data, rather than running mysqld with datadir pointing to the backup directory. The SQL statements you use to verify the data change the underlying logical sequence number, which would interfere with using the backup directory for subsequent incremental backups.

If you did the backup with the backup-and-apply-log option as in the previous example, the backup data is fully consistent and ready to verify. If you only ran the first stage by using the backup option, run mysqlbackup a second time with the apply-log option before doing this verification. (Typically, you run this second phase on the other server after transferring the backup data there, to minimize the load on the original database server.)

See Chapter 4, Recovering or Restoring a Database for the procedure to restore the database files on a different server.

Running the mysqld daemon on the restored data requires a valid configuration file, which you specify with the --defaults-file option of the mysqld command. You can reuse most of the settings from the original my.cnf file, combined with the backup-my.cnf file in the backup directory, which contains only the small subset of parameters required by mysqlbackup. Create a new configuration file by concatenating those two files into a new one, and use that configuration file on the server where you do the verification. Edit the resulting file to make sure the datadir parameter points to the right location on the verification server. directory. Edit the values for port, socket, and so on if you need to use different connection settings on the verification server.

3.2.3 Restoring a Database at its Original Location

To restore a MySQL instance from a backup:

  • Shut down the database server using your usual technique, such as the command mysqladmin shutdown.

  • Make sure the backup data is fully consistent, by either using the backup-and-apply-log option to perform the backup, or running mysqlbackup with the apply-log option after the initial backup.

  • Use the mysqlbackup command with the copy-back option. This operation copies tables, indexes, metadata, and any other required files back to their original locations as defined by the original MySQL configuration file. For the different combinations of options that you can specify as part of this operation, see Section 5.1.1.3, “Restore an Existing Backup”.

$ mysqlbackup --defaults-file=path_to_my.cnf \
  --datadir=path_to_data_directory \
  --backup-dir=path_to_backup_directory copy-back
...many lines of output...
mysqlbackup: Finished copying backup files.

101208 16:48:13 mysqlbackup: mysqlbackup completed OK!

Now the original database directory is restored from the backup, and you can restart the database server.

3.3 Backup Scenarios and Examples

All of the following tasks and examples make use of the mysqlbackup command. For detailed syntax information, see Chapter 5, mysqlbackup Command Reference.

3.3.1 Making a Full Backup

Most backup strategies start with a complete backup of the MySQL server, from which you can restore all databases and tables. After you do one full backup, you might do incremental backups (which are smaller and faster) for the next several backup jobs. Periodically, you then do another full backup to begin the cycle again.

This section outlines some of the considerations for making this most basic kind of backup. Because a full backup can take longer and produce larger backup files then other kinds of backups, your decisions about speed, capacity, and convenience are especially important for this part of the backup strategy.

For examples showing the commands to make a full backup, see Section 3.2.1, “Backing Up an Entire MySQL Instance”.

Options on Command Line or in Configuration File?

For clarity, the examples in this manual typically show command-line options to demonstrate connection parameters and other information that might be the same for each backup job. For convenience and consistency, you can include these options in the [mysqlbackup] section of the MySQL configuration file that you pass to the mysqlbackup command; mysqlbackup also picks them up from the [mysqld] section if they are present. For example, relying on the port information in the configuration file avoids the need to edit your backup scripts if the database instance switches to a different port.

Output in Single Directory or Timestamped Subdirectories?

For convenience, the --with-timestamp option creates uniquely named subdirectories under the backup directory to hold the output from each backup job. The timestamped subdirectories make it simpler to establish retention periods, for example by removing or archiving backup data past a certain age. This option is NOT set on default.

If you do use a single backup directory (that is, if you omit the --with-timestamp option), either specify a new unique directory name for each backup job, or specify the --force option to overwrite existing backup files.

With the --incremental-base option, as part of each incremental backup command, you specify the directory containing the previous backup. To make the directory names predictable, you might prefer to leave out the --with-timestamp option and instead generate a sequence of directory names as part of your backup script.

Always Full Backup, or Full Backup plus Incremental Backups?

If your InnoDB data volume is small, or if your database is so busy that a high percentage of data changes between backups, you might run a full backup each time. Typically, you can save time and storage space by running periodic full backups, and in between running several incremental backups, as described in Section 3.3.2, “Making an Incremental Backup”.

Use Compression or Not?

Creating a compressed backup can save you considerable storage space and reduce I/O significantly. And with the LZ4 compression method (introduced since release 3.10), the overhead for processing compression is quite low. In cases where database backups are moving from a faster disk system where the active database files sit to a possibly slower storage, compression will often significantly lower the overall backup time. It can result in reduced restoration time as well. In general, we recommend LZ4 compression over no compression for most users, as LZ4-based backups often finish in a shorter time period. However, test out MySQL Enterprise Backup within your environment to determine what is the most efficient approach.

3.3.2 Making an Incremental Backup

An incremental backup only backs up data that changed since the previous backup. This technique provides additional flexibility in designing a backup strategy and reduces required storage for backups.

Incremental backups are typically smaller and take less time than a full backup, making them a good choice for frequent backup jobs. Taking frequent incremental backups ensures you can always restore the database to the same state as a few hours or days in the past, without as much load or storage overhead on the database server as taking frequent full backups.

Note

Because an incremental backup always adds to an existing set of backup files, make at least one full backup before doing any incremental backups.

Incremental backup is enabled through an option to the mysqlbackup command. For straightforward incremental backups, specify the --incremental option. An alternative method uses the --incremental-with-redo-log-only option, requiring additional planning on your part.

You also indicate the point in time of the previous full or incremental backup. For convenience, you can use the --incremental-base option to automatically derive the necessary log sequence number (LSN) from the metadata stored in a previous backup directory. Or, you can specify an explicit LSN value using the --start-lsn option, using the ending LSN from a previous full or incremental backup.

To prepare the backup data to be restored, you combine each incremental backup with an original full backup. Typically, you perform a new full backup after a designated period of time, after which you can discard the older incremental backup data.

When running the apply log step for an incremental backup, you specify the option sequence --incremental apply-log, and the paths to 2 MySQL configuration files, first the .cnf file pointing to the full backup that you are updating, then the .cnf file pointing to the incremental backup data files. If you have taken several incremental backups since the last full backup, you might run several such apply log steps, one after the other, to bring the full backup entirely up to date.

Space Considerations for Incremental Backups

The incremental backup feature is primarily intended for InnoDB tables, or non-InnoDB tables that are read-only or rarely updated. For non-InnoDB files, the entire file is included in an incremental backup if that file changed since the previous backup.

You cannot perform incremental backups with the --compress option.

Incremental backups detect changes at the level of pages in the InnoDB data files, as opposed to table rows; each page that has changed is backed up. Thus, the space and time savings are not exactly proportional to the percentage of changed InnoDB rows or columns.

Examples of Incremental Backups

This example uses the mysqlbackup command to make an incremental backup of a MySQL server, including all databases and tables. We show two alternatives, one using the --incremental-base option and the other using the --start-lsn option.

With the --incremental-base option, you do not have to keep track of LSN values between one backup and the next. Instead, you specify the directory of the previous backup (either full or incremental), and mysqlbackup figures out the starting point for this backup based on the metadata of the earlier one. Because you need a known set of directory names, you might use hardcoded names or generate a sequence of names in your own backup script, rather than using the --with-timestamp option.

$ mysqlbackup --defaults-file=/home/pekka/.my.cnf --incremental \
  --incremental-base=dir:/incr-backup/wednesday \
  --incremental-backup-dir=/incr-backup/thursday \
  backup
...many lines of output...
mysqlbackup: Backup created in directory '/incr-backup/thursday'
mysqlbackup: start_lsn: 2654255717
mysqlbackup: incremental_base_lsn: 2666733462
mysqlbackup: end_lsn: 2666736714

101208 17:14:58 mysqlbackup: mysqlbackup completed OK!

With the --start-lsn option, you do have to record the LSN of the previous backup, but then the location of the previous backup is less significant, so you can use --with-timestamp to create named subdirectories automatically.

$ mysqlbackup --defaults-file=/home/pekka/.my.cnf --incremental \
  --start-lsn=2654255716 \
  --with-timestamp \
  --incremental-backup-dir=/incr-backup \
  backup
...many lines of output...
mysqlbackup: Backup created in directory '/incr-backup/2010-12-08_17-14-48'
mysqlbackup: start_lsn: 2654255717
mysqlbackup: incremental_base_lsn: 2666733462
mysqlbackup: end_lsn: 2666736714

101208 17:14:58 mysqlbackup: mysqlbackup completed OK!

Wherever you use the --incremental option, you can use the --incremental-with-redo-log-only option instead. Because --incremental-with-redo-log-only is more dependent on the precise LSN than the --incremental option is, use the --incremental-base option rather than the --start-lsn option with this kind of incremental backup.

For this alternative kind of incremental backup to work, the volume of changed information must be low enough, and the redo log files must be large enough, that all the changes since the previous incremental backup must be present in the redo log and not overwritten. See the --incremental-with-redo-log-only option description to learn how to verify those requirements.

$ mysqlbackup --defaults-file=/home/pekka/.my.cnf --incremental \
  --incremental-base=dir:/incr-backup/wednesday \
  --incremental-backup-dir=/incr-backup/thursday \
  backup
...many lines of output...
mysqlbackup: Backup created in directory '/incr-backup/thursday'
mysqlbackup: start_lsn: 2654255717
mysqlbackup: incremental_base_lsn: 2666733462
mysqlbackup: end_lsn: 2666736714

101208 17:14:58 mysqlbackup: mysqlbackup completed OK!

See Section C.3, “Sample Directory Structure for Incremental Backup” for a listing of files from a typical incremental backup.

Once again, we apply to the full backup any changes that occurred while the backup was running:

$ mysqlbackup --backup-dir=/full-backup/2010-12-08_17-14-11 apply-log
..many lines of output...
101208 17:15:10  mysqlbackup: Full backup prepared for recovery successfully!

101208 17:15:10 mysqlbackup: mysqlbackup completed OK!

Then, we apply the changes from the incremental backup:

$ mysqlbackup --incremental-backup-dir=/incr-backup/2010-12-08_17-14-48 
  --backup-dir=/full-backup/2010-12-08_17-14-11 apply-incremental-backup
...many lines of output...
101208 17:15:12 mysqlbackup: mysqlbackup completed OK!

Now, the data files in the full backup directory are fully up-to-date, as of the time of the last incremental backup.

This example shows an incremental backup. The last full backup we ran reported that the highest LSN was 2638548215:

mysqlbackup: Was able to parse the log up to lsn 2638548215

We specify that number again in the command here; the incremental backup includes all changes that came after the specified LSN.

$ mysqlbackup --defaults-file=/home/pekka/.my.cnf --incremental \
  --start-lsn=2638548215 \
  --incremental-backup-dir=/incr-backup/2010-12-08_17-14-48 \
  --backup-dir=/full-backup/2010-12-08_17-14-11 \
	backup
...many lines of output...
mysqlbackup: Scanned log up to lsn 2654252454.
mysqlbackup: Was able to parse the log up to lsn 2654252454.
mysqlbackup: Maximum page number for a log record 0
mysqlbackup: Backup contains changes from lsn 2638548216 to lsn 2654252454
101208 17:12:24  mysqlbackup: Incremental backup completed!

Next steps:

  • Make a note of the LSN value in the message at the end of the backup, for example, mysqlbackup: Was able to parse the log up to lsn LSN_number. You specify this value when performing incremental backups of changes that occur after this incremental backup.

  • Apply the incremental backup to the backup files, so that the backup is ready to be restored at any time. You can move the backup data to a different server first, to avoid the CPU and I/O overhead of this operation on the database server itself.

  • On a regular schedule, determined by date or amount of database activity, take further take incremental backups.

  • Optionally, periodically start the cycle over again by taking a full, uncompressed or compressed backup. Typically, this milestone happens when you can archive and clear out your oldest backup data.

3.3.3 Making a Compressed Backup

To save disk space, you can compress InnoDB backup data files by using the --compress option of mysqlbackup. Compression lets you keep more sets of backup data on hand and save on transmission time when sending the backup data to another server. The downside includes the extra CPU overhead during the backup itself and the extra time needed for the restoration process for uncompressing the data.

The backup compression feature works only for InnoDB tables. After the InnoDB tablespace files are compressed during backup, they receive the .ibz extension rather than the usual .ibd extension. To avoid wasting CPU cycles without saving additional disk space, --compress does not attempt to compress already-compressed tables that use the Barracuda file format; such tablespace files keep the usual .ibd extension.

Note

When there is unused space within an InnoDB tablespace file, the entire file is copied during an uncompressed backup. Perform a compressed backup to avoid the storage overhead for the unused space.

You can only use the --compress option for full backups, but not for incremental backups.

You can also select the compression algorithm to use by the --compress-method option and, when using the ZLIB or LZMA compression algorithm, the level of compression by the --compress-level option. See Section 5.1.7, “Compression Options” for details.

This is a sample command for making a compressed backup:

mysqlbackup --defaults-file=/etc/my.cnf --compress --compress-level=5 backup

This is a sample command for making a compressed single-file backup:

mysqlbackup --defaults-file=/etc/my.cnf --compress --compress-level=5 \
  --backup-image=backup.img backup-to-image

Next steps:

  • Make a note of the LSN value in the message at the end of both the full and incremental backups (for example, in the line mysqlbackup: Was able to parse the log up to lsn LSN_number). You specify this value when performing incremental backups of changes that occur after this full backup.

  • Apply the log to the compressed backup files, so that the full backup is ready to be restored at any time. You can move the backup data to a different server first, to avoid the CPU and I/O overhead of performing this operation on the database server.

  • After applying the log, periodically take incremental backups, which are smaller and can be made faster than a full backup.

3.3.4 Making a Partial Backup

Note

To facilitate the creation of partial backups, MySQL Enterprise Backup 3.10 introduces two new options for partial backup: --include-tables and --exclude-tables. The new options are intended for replacing the older options of --include, --databases, --databases-list-file, and --only-innodb-with-frm, which are incompatible with the new options and will be deprecated in the upcoming releases. In the discussions below we assume the new options are used for partial backups. For reference purpose, we have included information on the older options at the end of this section in Making a Partial Backup with Legacy Options.

By default, all the files in the data directory are included in the backup, so that the backup includes data from all MySQL storage engines, any third-party storage engines, and even any non-database files in that directory. This section explains options you can use to selectively back up or exclude data.

There are various ways to create different kinds of partial backup with MySQL Enterprise Backup:

  • Including or excluding specific tables by their names. This uses the --include-tables or --exclude-tables option.

    Each table is checked against the regular expression specified with the --include-tables or --exclude-tables option. If the regular expression matches the fully qualified name of the table (in the form of db_name.table_name), the table is included or excluded for the backup. The regular expression syntax used is the extended form specified in the POSIX 1003.2 standard. The options have been implemented with Henry Spencer's regular expression library.

  • Including some or all InnoDB tables, but not other table types. This uses the --only-innodb option.

  • Leaving out files that are present in the MySQL data directory but not actually part of the MySQL instance. This uses the --only-known-file-types option.

  • Achieving a multiple of selection effects by using a combination of the above mentioned options.

  • Backing up a selection of InnoDB tables using transportable tablespaces (TTS). This uses the --use-tts and the --include-tables or --exclude-tables (or both) options.

For syntax details on all the options involved, see Section 5.1.9, “Partial Backup and Restore Options”.

Important

Typically, a partial backup is more difficult to restore than a full backup, because the backup data might not include the necessary interrelated pieces to constitute a complete MySQL instance. In particular, InnoDB tables have internal IDs and other data values that can only be restored to the same instance, not a different MySQL server. Always fully test the recovery procedure for any partial backups to understand the relevant procedures and restrictions.

Important

Because the InnoDB system tablespace holds metadata about InnoDB tables from all databases in an instance, restoring a partial backup on a server that includes other databases could cause the system to lose track of those InnoDB tables in other databases. Always restore partial backups on a fresh MySQL server instance without any other InnoDB tables that you want to preserve.

The following are some command samples for partial backups.

Including all tables with names starting with emp into the backup:

$ mysqlbackup \
 --host=localhost --user=mysqluser --protocol=TCP --port=3306 \
 --backup-dir=$MEB_BACKUPS_DIR/backupdir \ --include-tables='\.emp' \
 backup

Taking a backup of all tables except tables from the mysql and performance_schema databases:

$ mysqlbackup \
 --host=localhost --user=mysqluser --protocol=TCP --port=3306 \
 --backup-dir=$MEB_BACKUPS_DIR/backupdir \
 --exclude-tables='^(mysql|performance_schema)\.' \
 backup

Taking a backup of all tables in the sales database, but excludes the table with the name hardware

$ mysqlbackup \
 --host=localhost --user=mysqluser --protocol=TCP --port=3306 \
 --backup-dir=$MEB_BACKUPS_DIR/backupdir \
 --include-tables='^sales\.' --exclude-tables='^sales\.hardware$' \
 backup

Backing up all InnoDB tables, but not .frm files:

$ mysqlbackup --defaults-file=/home/pekka/.my.cnf --only-innodb backup

You can also make compressed, single-image, and other kinds of selective backups by adding the appropriate options.

Next Steps:

  • Make a note of the LSN value in the message at the end of both full and incremental backups, for example, mysqlbackup: Was able to parse the log up to lsn LSN_number. You specify this value when performing incremental backups of changes that occur after this full backup.

  • Apply the log to the uncompressed backup files, so that the full backup is ready to be restored at any time. You can move the backup data to a different server first, to avoid the CPU and I/O overhead of performing this operation on the database server.

  • After applying the log, periodically take incremental backups, which are much faster and smaller than a full backup like these ones.

Making a Partial Backup with the Legacy Options

Important

Information in this subsection is only for using the legacy options of --include, --databases, --databases-list-file, and --only-innodb-with-frm, which will be deprecated in the upcoming issues. For creating partial backups, it is strongly recommended that the new options of --include-tables and --exclude-tables be used instead. Note that you cannot combine the legacy and the new partial-backup options in a single command.

MySQL Enterprise Backup can make different kinds of partial backup using the legacy partial-backup options:

For syntax details on all these options, see Legacy Partial Backup Options.

Note

Typically, a partial backup is more difficult to restore than a full backup, because the backup data might not include the necessary interrelated pieces to constitute a complete MySQL instance. In particular, InnoDB tables have internal IDs and other data values that can only be restored to the same instance, not a different MySQL server. Always fully test the recovery procedure for any partial backups to understand the relevant procedures and restrictions.

With its --include option, mysqlbackup can make a backup that includes some InnoDB tables but not others:

  • A partial backup with the --include option always contains the InnoDB system tablespace and all the tables inside it.

  • For the InnoDB tables stored outside the system tablespace, the partial backup includes only those tables whose names match the regular expression specified with the --include option.

This operation requires the tables being left out to be stored in separate table_name.ibd files. To put an InnoDB table outside the system tablespace, create it while the innodb_file_per_table MySQL configuration option is enabled. Each .ibd file holds the data and indexes of one table only.

Those InnoDB tables created with innodb_file_per_table turned off are stored as usual in the InnoDB system tablespace, and cannot be left out of the backup.

For each table with a per-table data file a string of the form db_name.table_name is checked against the regular expression specified with the --include option. If the regular expression matches the complete string db_name.table_name, the table is included in the backup. The regular expression syntax used is the extended form specified in the POSIX 1003.2 standard. On Unix-like systems, quote the regular expression appropriately to prevent interpretation of shell meta-characters. This feature has been implemented with Henry Spencer's regular expression library.

The backup directory produced contains a backup log file and copies of InnoDB data files.

IMPORTANT: Although the mysqlbackup command supports taking partial backups, be careful when restoring a database from a partial backup. mysqlbackup copies also the .frm files of those tables that are not included in the backup, except when you do partial backups using, for example, the --databases option. If you use mysqlbackup with the --include option, before restoring the database, delete from the backup data the .frm files for any tables that are not included in the backup.

IMPORTANT: Because the InnoDB system tablespace holds metadata about InnoDB tables from all databases in an instance, restoring a partial backup on a server that includes other databases could cause the system to lose track of those InnoDB tables in other databases. Always restore partial backups on a fresh MySQL server instance without any other InnoDB tables that you want to preserve.

The --only-innodb and --only-innodb-with-frm options back up InnoDB tables only, skipping those of other storage engines. You might also use them together with the --include option to make selective backup of InnoDB tables while excluding all other files created by other storage engines.

Example 3.1 Making an Uncompressed Partial Backup of InnoDB Tables

In this example, we have configured MySQL so that some InnoDB tables have their own tablespaces. We make a partial backup including only those InnoDB tables in test database whose name starts with ib. The contents of the database directory for test database are shown below. The directory contains a MySQL description file (.frm file) for each of the tables (alex1, alex2, alex3, blobt3, ibstest0, ibstest09, ibtest11a, ibtest11b, ibtest11c, and ibtest11d) in the database. Of these 10 tables six (alex1, alex2, alex3, blobt3, ibstest0, ibstest09) are stored in per-table data files (.ibd files).

$ ls /sqldata/mts/test
alex1.frm  alex2.ibd  blobt3.frm    ibstest0.ibd  ibtest11a.frm  ibtest11d.frm
alex1.ibd  alex3.frm  blobt3.ibd    ibtest09.frm  ibtest11b.frm
alex2.frm  alex3.ibd  ibstest0.frm  ibtest09.ibd  ibtest11c.frm

We run the mysqlbackup with the --include option:

# Back up some InnoDB tables but not any .frm files.
$ mysqlbackup --defaults-file=/home/pekka/.my.cnf --include='test\.ib.*' --only-innodb backup
...many lines of output...
mysqlbackup: Scanned log up to lsn 2666737471.
mysqlbackup: Was able to parse the log up to lsn 2666737471.
mysqlbackup: Maximum page number for a log record 0
101208 17:17:45  mysqlbackup: Full backup completed!

# Back up some InnoDB tables and the .frm files for the backed-up tables only.
$ mysqlbackup --defaults-file=/home/pekka/.my.cnf --include='test\.ib.*' \
  --only-innodb-with-frm=related backup
...many lines of output...
mysqlbackup: Scanned log up to lsn 2666737471.
mysqlbackup: Was able to parse the log up to lsn 2666737471.
mysqlbackup: Maximum page number for a log record 0
101208 17:17:45  mysqlbackup: Full backup completed!

The backup directory contains only backups of ibstest and ibtest09 tables. Other InnoDB tables did not match the include pattern test\.ib.*. Notice, however, that the tables ibtest11a, ibtest11b, ibtest11c, ibtest11d are in the backup even though they are not visible in the directory shown below, because they are stored in the system tablespace (ibdata1 file) which is always included in the backup.

# With the --only-innodb option:
$ ls /sqldata-backup/test
ibstest0.ibd   ibtest09.ibd

# With the --only-innodb-with-frm=related option:
$ ls /sqldata-backup/test
ibstest0.frm   ibtest09.frm
ibstest0.ibd   ibtest09.ibd

Example 3.2 Making a Compressed Partial Backup

We have configured MySQL so that every InnoDB table has its own tablespace. We make a partial backup including only those InnoDB tables whose name starts with alex or blob. The contents of the database directory for test database is shown below.

$ ls /sqldata/mts/test
alex1.frm  alex2.ibd  blobt3.frm    ibstest0.ibd  ibtest11a.frm  ibtest11d.frm
alex1.ibd  alex3.frm  blobt3.ibd    ibtest09.frm  ibtest11b.frm
alex2.frm  alex3.ibd  ibstest0.frm  ibtest09.ibd  ibtest11c.frm

We run mysqlbackup with the --compress and --include options:

$ mysqlbackup --defaults-file=/home/pekka/.my.cnf --compress \
  --include='.*\.(alex|blob).*' --only-innodb backup
...many lines of output...
mysqlbackup: Scanned log up to lsn 2666737471.
mysqlbackup: Was able to parse the log up to lsn 2666737471.
mysqlbackup: Maximum page number for a log record 0

mysqlbackup: Compressed 147 MB of data files to 15 MB (compression 89%).

101208 17:18:04  mysqlbackup: Full backup completed!

The backup directory for the database test is shown below. The .ibz files are compressed per-table data files.

$ ls /sqldata-backup/test
alex1.ibz   alex2.ibz   alex3.ibz   blobt3.ibz

The --databases and --databases-list-file options of the mysqlbackup command let you back up non-InnoDB tables only from selected databases, rather than across the entire MySQL instance. (To filter InnoDB tables, use the --include option instead.) With --databases, you specify a space-separated list of database names, with the entire list enclosed in double quotation marks. With --databases-list-file, you specify the path of a file containing the list of database names, one per line.

Some or all of the database names can be qualified with table names, to only back up selected non-InnoDB tables from those databases.

If you specify this option, make sure you include the same set of databases for every backup (especially incremental backups), so that you do not restore out-of-date versions of any databases.

3.3.5 Making a Single-File Backup

To avoid a large number of backup files to track and keep safe, and to simplify moving backup data around, the mysqlbackup command can create a backup in a single-file format, pack an existing backup into a single file, unpack the single file back to the original backup directory structure, list the contents of a single-file backup, verify the contents of a single-file backup against embedded checksums, or extract a single file or directory tree. For the syntax of the relevant mysqlbackup options, see Section 5.1.10, “Single-File Backup Options”.

Because the single-file backup can be streamed or piped to another process, such as a tape backup or a command such as scp, you can use this technique to put the backup on another storage device or server without significant storage overhead on the original database server. (During preparation of the single-file backup, some small work files are prepared temporarily inside the specified backup directory.)

To create a single-file backup, use the mysqlbackup subcommand backup-to-image.

Example 3.3 Single-File Backup to Absolute Path

This command creates a single backup image in the given absolute path. It still requires --backup-dir, which is used to hold temporary output, status, and metadata files.

mysqlbackup --backup-image=/backups/sales.mbi --backup-dir=/backup-tmp backup-to-image

Example 3.4 Single-File Backup to Relative Path

This command specifies --backup-image with a relative path underneath the backup directory. The resulting single-file backup is created as /backups/sales.mbi.

mysqlbackup --backup-image=sales.mbi --backup-dir=/backups backup-to-image

Example 3.5 Single-File Backup to Standard Output

The following command dumps the backup output to standard output. Again, the --backup-dir directory specified in my.cnf is used as a temporary directory.

mysqlbackup --backup-dir=/backups --backup-image=- backup-to-image > /backup/mybackup.mbi

Example 3.6 Convert Existing Backup Directory to Single Image

The backup-dir directory specified in my.cnf is bundled into the /backup/my.mbi file. The directory can contain anything, not necessarily a backup produced by MySQL Enterprise Backup.

mysqlbackup --backup-image=/backup/my.mbi --backup-dir=/var/mysql/backup backup-dir-to-image

Example 3.7 Extract Existing Image to Backup Directory

The image contents are unpacked into backup-dir.

mysqlbackup --backup-dir=/var/backup --backup-image=/backup/my.mbi image-to-backup-dir

Example 3.8 List Single-File Backup Contents

The image contents are listed with each line indicating a file or directory entry.

mysqlbackup --backup-image=/backup/my.mbi list-image

Example 3.9 Extract Single-File Backup into Current Directory

The following command extracts all contents from a single-file backup into the current working directory.

mysqlbackup --backup-image=/var/my.mbi extract

Example 3.10 Extract Single-File Backup into a Backup Directory

This command behaves like the image-to-backup-dir option, by extracting all contents of a single-file backup into the --backup-dir directory.

mysqlbackup --backup-image=/var/my.mbi --backup-dir=/var/backup extract

Example 3.11 Selective Extract of Single File

The following command extracts the single file meta/comments.txt into the local path./meta/comments.txt.

mysqlbackup --backup-image=/var/my.mbi \
  --src-entry=meta/comments.txt extract

The following command extracts the meta/comments.txt file into a specified path /tmp/mycomments.txt by using the --dst-entry option.

mysqlbackup --backup-image=/var/my.mbi \
  --src-entry=meta/comments.txt \
  --dst-entry=/tmp/mycomments.txt extract

The following command dumps the contents of meta/comments.txt (inside a single-file backup) to standard output.

mysqlbackup --backup-image=/var/my.mbi --src-entry=meta/comments.txt --dst-entry=- extract

Example 3.12 Selective Extract of Single Directory

The following command extracts a single directory meta into a local file system path ./meta. Extracting a directory extracts all its contents, including any subdirectories.

mysqlbackup --backup-image=/backup/my.mbi --src-entry=meta extract

The following command extracts all meta directory contents (all its files and subdirectories) into the directory /tmp/my-meta.

mysqlbackup --backup-image=/backup/my.mbi --src-entry=meta \
  --dst-entry=/tmp/my-meta extract

Example 3.13 Dealing with Absolute Path Names

Since absolute pathnames are extracted to the same paths in local system, it could be a problem if you do not have write permission for that path. You can remap absolute paths as follows:

mysqlbackup --backup-image=/backup/my.mbi --src-entry=/ --dst-entry=/myroot extract
mysqlbackup --backup-image=/backup/my.mbi --src-entry=. extract

The first command extracts all absolute paths to /myroot directory in the local system. The second command extracts all relative paths to the current directory.


3.3.5.1 Streaming the Backup Data to Another Device or Server

To limit the storage overhead on the database server, you can transfer the backup data to a different server without ever storing it locally. You can achieve that with a single-file backup. To send the single-file backup to standard output, use the mysqlbackup command backup-to-image without specifying the --backup-image option. (You can also specify --backup-image=- to make it obvious that the data is sent to stdout.) To stream the data, you use the single-file backup in combination with operating system features such as pipes, ssh, scp, and so on, which take the input from standard output and create an equivalent file on a remote system. You can either store the single-file backup directly on the remote system, or invoke mysqlbackup with the image-to-backup-dir option on the other end to reproduce the directory structure of a regular backup.

Example 3.14 Single-File Backup to a Remote Host

The following command streams the backup as a sinlge-file output to a remote host, where it may be saved directly to a tape device. --backup-dir=/tmp designates the directory for storing temporary files rather than the final output file.

mysqlbackup --backup-image=- --backup-dir=/tmp backup-to-image | \
ssh user@host command arg1 arg2...

For simplicity, all the connection and other necessary options are assumed to be specified in the default configuration file. To have the desired operations run on the remote system, substitute the combination of command, device, and so on that you use as part of your normal archiving procedure, such as dd or tar.


Example 3.15 Single-file Backup to a Remote MySQL Server

The following command streams the backup as a single backup file to be restored on a remote MySQL server:


mysqlbackup  --backup-dir=backup --backup-image=-  --compress backup-to-image | \
ssh <user name>@<remote host name> 'mysqlbackup --backup-dir=backup_tmp --datadir=/data \
--innodb_log_group_home_dir=. \
--innodb_log_files_in_group=<innodb_log_files_in_group_of_backedup_server> \
 --innodb_log_file_size=<innodb_log_file_size_of_backedup_server> \ 
--innodb_data_file_path=<innodb_data_file_path_of_backedup_server> \ 
--uncompress --backup-image=- copy-back-and-apply-log'

Example 3.16 Stream a Backup Directory to a Remote MySQL Server

The following command streams a backup directory as a single backup file to be restored on a remote MySQL server:

mysqlbackup --backup-image=- --backup-dir=/path/to/my/backup backup-dir-to-image | \
ssh <user name>@<remote host name> \ 
'mysqlbackup --backup-dir=backup_tmp --datadir=/data --backup-image=- copy-back-and-apply-log'

3.3.5.2 Backing Up to Tape

Tape drives are affordable, high-capacity storage devices for backup data. MySQL Enterprise Backup can interface with media management software (MMS) such as Oracle Secure Backup (OSB) to drive MySQL backup and restore jobs. The media management software must support Version 2 or higher of the System Backup to Tape (SBT) interface.

For information about doing tape backups in combination with MMS products such as Oracle Secure Backup, see Chapter 9, Using MySQL Enterprise Backup with Media Management Software (MMS) Products.

3.3.5.3 Backing Up to Cloud Storage

MySQL Enterprise Backup supports cloud backup since version 3.10.2.

Only single-file backups can be created on and restored from cloud storage. All mysqlbackup options compatible with single-file operations (including, for example, the --incremental, --compression, --partial, and --encryption options) can be used with cloud backup or restoration.

Currently, Amazon S3 is the only cloud service supported by MySQL Enterprise Backup.

A cloud backup is created using the cloud options for mysqlbackup, which are described in details in Section 5.1.15, “Cloud Storage Options”. This is a sample command for creating a cloud backup:

Example 3.17 Creating a Cloud Backup

mysqlbackup\
--cloud-service=s3 --cloud-aws-region=<aws region> \
--cloud-access-key-id=<aws access key id> --cloud-secret-access-key=< aws secret access key> \
--cloud-bucket=<s3 bucket name> --cloud-object-key=<aws object key> \
--backup-dir=/home/user/dba/s3backuptmpdir \
--backup-image=- \
backup-to-image


Besides backup-to-image, all other mysqlbackup operations for single-file backups (backup-dir-to-image, list-image, validate, image-to-backup-dir, extract, copy-back, and copy-back-and-apply-log) can also be performed using cloud storage.

Example 3.18 Extract an Existing Image from Cloud Storage to Backup Directory

Extract a backup image from cloud storage, using the --backup-dir option to specify the directory into which the image will be extracted:

mysqlbackup\
--cloud-service=s3 --cloud-aws-region=<aws region> \
--cloud-access-key-id=<aws access key id> --cloud-secret-access-key=< aws secret access key> \
--cloud-bucket=<s3 bucket name> --cloud-object-key=<aws object key> \
--backup-dir=/home/user/dba/s3backuptmpdir \
--backup-image=- \
image-to-backup-dir


See Example 4.9, “Restoring a Single-file Backup from Cloud Storage to a MySQL Server” on how to restore a backup image from a cloud storage.

3.3.6 Making an Optimistic Backup

Optimistic backup is a feature introduced in MySQL Enterprise Backup 3.11 for improving performance for backing up and restoring huge databases in which only a small number of tables are modified frequently.

During a hot backup of a huge database (say, in the order of terabytes), huge redo log files could be generated on the server when the backup is in progress. As the redo log files grow faster than they can be processed by mysqlbackup, the backup operation can actually fail when mysqlbackup cannot catch up with the redo log cycles and LSNs get overwritten by the server before they are read by mysqlbackup. Moreover, the apply-log step for preparing a backup for restoration can take a very long time as mysqlbackup has huge ibbackup_logfile files (created from the big redo log files) to apply to the backup. The problems are intensified when the I/O resources available for reading and writing the redo logs are scarce during the backup and restoration processes.

Optimistic backup relieves the problems by dividing the backup process into two internal phases, which are transparent to the users:

  1. Optimistic phase: In this first phase, tables that are unlikely to be modified during the backup process (referred to as the inactive tables below, identified by the user with the optimistic-time option or, by exclusion, with the optimistic-busy-tables option) are backed up without locking the MySQL instance. And because those tables are not expected to be changed before the backup is finished, redo logs, undo logs, and system table spaces are not backed up by mysqlbackup in this phase.

  2. Normal phase: In this second phase, tables that are not backed up in the first phase (referred to as the busy tables below) are being backed up in a manner similar to how they are processed in an ordinary backup: the InnoDB files are copied first, and then the other relevant files and copied or processed with the MySQL instance locked. Also, if it turns out that some of the inactive tables have actually been modified after they were backed up in the optimistic phase, they are copied again in the normal phase. The redo logs, undo logs, and the system table space are also backed up in this phase.

An optimistic backup occurs whenever the optimistic-time or optimistic-busy-tables option is used. For how to use the options, see detailed descriptions for them in Section 5.1.11, “Performance / Scalability / Capacity Options”. If, as expected, the inactive tables identified by the options are not changed a lot during the backup, the overall backup time (time for the backup operation plus time for the apply-log operation) can be reduced significantly comparing with an ordinary backup because the apply-log operation will be much faster. However, if it turns out that the inactive tables identified are modified a lot during the backup process, benefits of performing an optimistic back up will become limited and, in the worst case, an optimistic backup might actually take longer to perform and, for a single-file backup, the size of the backup will be larger when comparing with an ordinary backup. Therefore, users should be careful in identifying which tables are inactive and which are busy when trying to perform an optimistic backup.

Note

An optimistic backup cannot be performed for an incremental backup or a backup using transportable tablespaces (TTS).

The following examples illustrate how to make an optimistic backup.

Example 3.19 Optimistic Backup Using the Option optimistic-time=YYMMDDHHMMSS

In this example, tables that have been modified since the noon of May 16, 2011 are treated as busy tables and backed up in the normal phase of an optimistic backup, and all other tables are backed up in the optimistic phase:

mysqlbackup --defaults-file=/etc/my.cnf --optimistic-time=110516120000 backup

Example 3.20 Optimistic Backup Using the Option optimistic-time=now

In this example, all tables are treated as inactive tables and backed up in the optimistic phase of an optimistic backup:

mysqlbackup --defaults-file=/etc/my.cnf --optimistic-time=now backup

Example 3.21 Optimistic Backup Using the optimistic-busy-tables Option

In this example, tables in mydatabase that are suffixed by mytables- in their names are treated as busy tables and backed up in the normal phase of an optimistic backup, and all other tables are backed up in the optimistic phase:

mysqlbackup --defaults-file=/etc/my.cnf --optimistic-busy-tables='mydatabase\.mytables-.*'  backup

When you use both the optimistic-time and optimistic-busy-tables options and they come into conflict on determining which tables are to be busy tables, optimistic-busy-tables takes precedence over optimistic-time. For example:

Example 3.22 Optimistic and Partial Backup Using both the optimistic-busy-tables and optimistic-time Options

In this example, tables in mydatabase that are suffixed by mytables- in their names are treated as busy tables and backed up in the normal phase, even if they have not been modified since May 16, 2010, the time specified by optimistic-time:

mysqlbackup --defaults-file=/etc/my.cnf --optimistic-busy-tables='mydatabase\.mytables-.*'  \
--optimistic-time=100516 backup


3.3.7 Making a Back Up of In-Memory Database Data

The --exec-when-locked option of the mysqlbackup command lets you specify a command and arguments to run near the end of the backup, while the database is still locked. This command can copy or create additional files in the backup directory. For example, you can use this option to back up MEMORY tables with the mysqldump command, storing the output in the backup directory. To delay any redirection or variable substitution until the command is executed, enclose the entire parameter value within single quotes.

3.3.8 Making Scheduled Backups

Maintaining a regular backup schedule is an important measure for preventing data loss for you MySQL server. This section discusses some simple means for setting up a schedule for running MySQL Enterprise Backup.

For Linux and other Unix-like platforms: you can set up a cron job on your system for scheduled backups. There are two types of cron jobs. To set up a user cron job, which is owned and run by a particular user, do the following:

  • Log on as the user who runs MySQL Enterprise Backup and use the following command to invoke an editor for creating (or modifying) a crontab:

    shell> crontab -e

  • In the editor, add an entry similar to the following one to the crontab, and then save your changes:

    @daily /path-to-mysqlbackup/mysqlbackup -uroot --backup-dir=/path-to-backup-folder/cronbackups --with-timestamp backup  &>/dev/null
    

    This crontab entry invokes mysqlbackup to create a backup under the cronbackups directory at 00:00:00 everyday. Outputs from the stderr and stdout streams are redirected to /dev/null/, so they will not invoke other actions on the part of the Cron server (for example, email notifications to the user).

To set up a system cron job, which is owned and run by root, create a file under the /etc/cron.d folder and put into it a similar crontab entry as the one above, adding the user (root in the following example) before the mysqlbackup command:

@daily root /path-to-mysqlbackup/mysqlbackup -uroot --backup-dir=/path-to-backup-folder/cronbackups --with-timestamp backup  &>/dev/null 

Check your platform's documentation for further details on the different ways to set up cron jobs for various types of schedules.

For Windows platforms: Use the Task Scheduler for the purpose. Check the documentation for your Windows platform for instructions.