This page describes how to export data out of BigQuery.
After you've loaded your data into BigQuery, you can export the data in several formats. BigQuery can export up to 1 GB of data per file, and supports exporting to multiple files.
You can also use Google Cloud Dataflow to read data from BigQuery. For more information about using Cloud Dataflow to read from, and write to, BigQuery, see BigQuery I/O in the Cloud Dataflow documentation.
Before you begin
- Ensure that you have read access to the data you want to export. You need
dataset-level
READERaccess. Alternatively, you can use any BigQuery IAM role that provides read access, such asdataViewer,dataEditor, oruser. - Ensure that you have write access to a Google Cloud Storage bucket. You can export data only to a Cloud Storage bucket.
Exporting data stored in BigQuery
The following examples export data from a public dataset to a Cloud Storage bucket. The data is exported in CSV format.
Web UI
Go to the BigQuery web UI.
Go to the BigQuery web UIIn the navigation, find and click the expansion icon next to Public Datasets, and then expand bigquery-public-data:samples to display its contents.

Find and click the down arrow icon
next to
shakespeare.Select Export table to display the Export to Google Cloud Storage dialog.
Leave the default settings in place for Export format and Compression (CSV and None, respectively).
In the Google Cloud Storage URI textbox, enter a valid URI in the format
gs://[BUCKET_NAME]/[FILENAME.CSV], where[BUCKET_NAME]is your Cloud Storage bucket name, and[FILENAME.CSV]is the name of your destination file.Click OK to export the table. While the job is running, (extracting) appears next to the name of the table in the navigation.
To check on the progress of the job, look near the top of the navigation for Job History for an Extract job.
Command-line
Use the bq extract command.
bq extract [DATASET].[TABLE_NAME] gs://[BUCKET_NAME]/[FILENAME.CSV]
For example, the following command exports the shakespeare table from the
bigquery-public-data:samples dataset into a file named shakespeare.csv
in a Cloud Storage bucket named example-bucket:
bq extract 'bigquery-public-data:samples.shakespeare' gs://example-bucket/shakespeare.csv
The default destination format is CSV. To export into JSON or Avro, use the
destination_format flag:
bq extract --destination_format=NEWLINE_DELIMITED_JSON 'bigquery-public-data:samples.shakespeare' gs://example-bucket/shakespeare.json
API
To export data, create a job and populate the
configuration.extract
object.
Create an extract job that points to the BigQuery source data and the Cloud Storage destination. For information about creating jobs, see Managing Jobs, Datasets, and Projects.
Specify the source table by using the sourceTable configuration object, which comprises the project ID, dataset ID, and table ID.
The destination URI(s) must be fully-qualified, in the format gs://[BUCKET_NAME]/[FILENAME.CSV]. Each URI can contain one '*' wildcard character and it must come after the bucket name.
Specify the data format by setting the configuration.extract.destinationFormat property. For example, to export a JSON file, set this property to the value
NEWLINE_DELIMITED_JSON.To check the job status, call jobs.get([JOB_ID]) with the ID of the job returned by the initial request.
- If
status.state = DONE, the job completed successfully. - If the
status.errorResultproperty is present, the request failed, and that object will include information describing what went wrong. - If
status.errorResultis absent, the job finished successfully, although there might have been some non-fatal errors. Non-fatal errors are listed in the returned job object'sstatus.errorsproperty.
- If
API notes:
As a best practice, generate a unique ID and pass it as
jobReference.jobIdwhen callingjobs.insert()to create a job. This approach is more robust to network failure because the client can poll or retry on the known job ID.Calling
jobs.insert()on a given job ID is idempotent; in other words, you can retry as many times as you like on the same job ID, and at most one of those operations will succeed.
Configuring export options
You can configure two aspects of the exported data: the format, and the compression type.
Destination format
BigQuery supports CSV, JSON and Avro format. Nested or repeated data cannot be exported to CSV, but can be exported to JSON or Avro format.
Web UI
Set the destination format in the Export to Google Storage dialog.
Follow steps 1 through 4 in the Exporting data stored in BigQuery section to display the Export to Google Cloud Storage dialog.
Use the drop-down list next to Export format to select CSV, JSON, or Avro format.
Command-line
Use the bq extract command with the destination_format flag to set
the format:
bq extract --destination_format=[CSV | NEWLINE_DELIMITED_JSON | AVRO] [DATASET].[TABLE_NAME] gs://[BUCKET_NAME]/[FILENAME]
For example, the following command exports the shakespeare table from the
bigquery-public-data:samples dataset into a file named shakespeare.json
in a Cloud Storage bucket named example-bucket in JSON format:
bq extract --destination_format=NEWLINE_DELIMITED_JSON 'bigquery-public-data:samples.shakespeare' gs://example-bucket/shakespeare.json
API
Specify the data format by setting the
configuration.extract.destinationFormat
property. For example, to export a JSON file, set this property to the
value NEWLINE_DELIMITED_JSON.
Compression
BigQuery supports GZIP compression, but the default setting is no compression (NONE).
Web UI
Set the compression type in the Export to Google Storage dialog.
Follow steps 1 through 4 in the Exporting data stored in BigQuery section to display the Export to Google Cloud Storage dialog.
Select a compression type, either
NONEorGZIP, using the selection buttons for Compression.
Command-line
Use the bq extract command with the compression flag to set
the format:
bq extract --compression=[GZIP | NONE] [DATASET].[TABLE_NAME] gs://[BUCKET_NAME]/[FILENAME]
For example, the following command exports the shakespeare table from the
bigquery-public-data:samples dataset into a file named shakespeare.zip
in a Cloud Storage bucket named example-bucket in GZIP format:
bq extract --compression=GZIP 'bigquery-public-data:samples.shakespeare' gs://example-bucket/shakespeare.zip
API
Specify the compression type by setting the
configuration.extract.compression
property. For example, to use GZIP compression, set this property to the
value GZIP.
Avro format
BigQuery expresses Avro formatted data in the following ways:
- The resulting export files are Avro container files.
- Each BigQuery row is represented as an Avro Record. Nested data is represented by nested Record objects.
REQUIREDfields are represented as the corresponding Avro types. For example, a BigQueryINTEGERtype maps to an AvroLONGtype.NULLABLEfields are represented as an Avro Union of the corresponding type and "null".REPEATEDfields are represented as Avro arrays.TIMESTAMPdata types are represented as AvroLONGtypes.
The Avro format can't be used in combination with GZIP compression.
Extract configuration example
The following code example shows the configuration of a job that exports data to a CSV file.
C#
For more on installing and creating a BigQuery client, refer to BigQuery Client Libraries.
Go
For more on installing and creating a BigQuery client, refer to BigQuery Client Libraries.
Java
For more on installing and creating a BigQuery client, refer to BigQuery Client Libraries.
Node.js
For more on installing and creating a BigQuery client, refer to BigQuery Client Libraries.
PHP
For more on installing and creating a BigQuery client, refer to BigQuery Client Libraries.
Python
For more on installing and creating a BigQuery client, refer to BigQuery Client Libraries.
Ruby
For more on installing and creating a BigQuery client, refer to BigQuery Client Libraries.
The destinationUris
property indicates the location(s) and file name(s) where BigQuery should export
your files to. Usually, you'll pass a single value so that BigQuery exports to a
single file, but you can alternately pass one or more wildcard URIs. See the
following section for information on what to specify for the destinationUris
property.
Exporting data into one or more files
The destinationUris
property indicates the location(s) and file name(s) where BigQuery should export
your files.
BigQuery supports a single wildcard operator (*) in each URI. The wildcard can
appear anywhere in the URI except as part of the bucket name. Using the wildcard
operator instructs BigQuery to create multiple sharded files based on the
supplied pattern. The wildcard operator is replaced with a number (starting at
0), left-padded to 12 digits. For example, a URI with a wildcard at the end of
the file name would create files with000000000000 appended to the first file,
000000000001 appended to the second file, and so on.
The following table describes several possible options for the destinationUris
property:
| `destinationUris` options | |
|---|---|
| Single URI |
Use a single URI if you want BigQuery to export your data to a single file. This option is the most common use case, as exported data is generally less than BigQuery's 1 GB per file maximum value. Property definition:
Creates: gs://my-bucket/file-name.json |
| Single wildcard URI |
Use a single wildcard URI if you think your exported data will be larger than BigQuery's 1 GB per file maximum value. BigQuery shards your data into multiple files based on the provided pattern. If you use a wildcard in a URI component other than the file name, be sure the path component does not exist before exporting your data. Property definition:
Creates: gs://my-bucket/file-name-000000000000.json gs://my-bucket/file-name-000000000001.json gs://my-bucket/file-name-000000000002.json ... Property definition:
Creates: gs://my-bucket/path-component-000000000000/file-name.json gs://my-bucket/path-component-000000000001/file-name.json gs://my-bucket/path-component-000000000002/file-name.json ... |
| Multiple wildcard URIs |
Use multiple wildcard URIs if you want to partition the export output. You would use this option if you're running a parallel processing job with a service like Hadoop on Google Cloud Platform. Determine how many workers are available to process the job, and create one URI per worker. BigQuery treats each URI location as a partition, and uses parallel processing to shard your data into multiple files in each location. You can use whatever pattern you'd like in your file name, assuming there is a single wildcard operator in each URI, each URI is unique, and the number of URIs does not exceed the quota policy. When you pass more than one wildcard URI, BigQuery creates a special file at the end of each partition that indicates the "final" file in the set. This file name indicates how many shards BigQuery created. For example, if your wildcard URI is Note that a zero record file might contain more than 0 bytes depending on the data format, such as when exporting data in CSV format with a column header. String pattern:
Property definition: ['gs://my-bucket/file-name-1-*.json', 'gs://my-bucket/file-name-2-*.json', 'gs://my-bucket/file-name-3-*.json'] Creates: This example assumes that BigQuery creates 80 sharded files in each partition. gs://my-bucket/file-name-1-000000000000.json gs://my-bucket/file-name-1-000000000001.json ... gs://my-bucket/file-name-1-000000000080.json gs://my-bucket/file-name-2-000000000000.json gs://my-bucket/file-name-2-000000000001.json ... gs://my-bucket/file-name-2-000000000080.json gs://my-bucket/file-name-3-000000000000.json gs://my-bucket/file-name-3-000000000001.json ... gs://my-bucket/file-name-3-000000000080.json |
Quota policy
The following limits apply for exporting data from BigQuery.
- Daily Limit: 1,000 exports per day, up to 10 TB
- Multiple Wildcard URI Limit: 500 URIs per export
What's next
- To learn more about the BigQuery web UI, see BigQuery Web UI.
- To learn more about the
bqcommand-line tool, see bq Command-Line Tool. - To learn how to create an application using the Google BigQuery API, see Create A Simple Application With the API.