Creating Tasks

This page describes how to create tasks and place them in push queues. When you want to run a task, you must create a new task and place it on a queue. You can explicitly specify the service and handler that processes the task, and optionally pass task-specific data along to the handler.

Creating a new task

To create and enqueue a task, call the taskqueue.add() function. The following code creates a task that targets the service named worker and invokes its handler by setting the url /update-counter:

class EnqueueTaskHandler(webapp2.RequestHandler):
    def post(self):
        amount = int(self.request.get('amount'))

        task = taskqueue.add(
            url='/update_counter',
            target='worker',
            params={'amount': amount})

        self.response.write(
            'Task {} enqueued, ETA {}.'.format(task.name, task.eta))

Alternatively, you can create a Task object and call its add() method.

Specifying the worker service

When a task is popped off its queue, the Task Queue service sends it on to a worker service. Every task has a target and a url, which determine what service and handler will ultimately perform the task.

target

The target specifies the service that will receive the HTTP request to perform the task. It is a string that specifies a service/version/instance in any one of the canonical forms. The most often-used ones are:

service
version.service
instance.version.service

The target string is prepended to the domain name of your app. There are three ways to set the target for a task:

• Explicity declare the target when you construct the task.

• Include a target directive when you define a queue in the queue.yaml, as in the definition of queue-blue above. All tasks added to a queue with a target will use that target, even if a different target was assigned to the task at construction time.

• If no target is specified according to either of the previous two methods, then the task's target is the version of the service that enqueues it. Note that if you enqueue a task from the default service and version in this manner, and the default version changes before the task executes, it will run in the new default version.

url

The url selects one of the handlers in the target service, which will perform the task.

The url should match one of the handler URL patterns in the target service. The url can include query parameters if the tasks's method is GET or PULL. If no url is specified the default URL /_ah/queue/[QUEUE_NAME] is used, where [QUEUE_NAME] is the name of the task's queue.

Passing data to the handler

You can pass data to the handler as query parameters in the task's URL—but only if you set the task's method to GET or PULL.

These three calls are equivalent:

taskqueue.add(method=GET, url='/update-counter?key=blue', target='worker')
taskqueue.add(url='/update-counter', params={'key': 'blue'}, target='worker')
taskqueue.add(url='/update-counter', payload="{'key': 'blue'}", target='worker')

Note:

• The payload is delivered in the body of the HTTP request.
• Do not specify params if you are using the HTTP POST method along with a payload, or if you are using the HTTP GET method and you've included a URL with a query parameters.

Naming a task

When you create a new task, App Engine assigns the task a unique name by default. You can assign your own name to a task by using the name parameter.

taskqueue.add(url='/url/path', name='first-try')

A task name must be unique within a queue. If you try to add another task with the same name to a the queue, the operation will fail. After a task is removed from a queue you can't insert a task with the same name into the queue until 10 days have passed. A task name can contain uppercase and lowercase letters, numbers, underscores, and hyphens. The maximum length for a task name is 500 characters.

Recommended: Let App Engine generate task names automatically.

Adding tasks asynchronously

By default, the Task Queue API calls are synchronous. For most scenarios, synchronous calls work fine. For instance, adding a task is usually a fast operation: the median time to add a task is 5 ms and 1 out of every 1000 tasks can take up to 300 ms. Periodic incidents, such as back-end upgrades, can cause spikes to 1 out of every 1000 tasks taking up to 1 second.

If you are building an application that needs low latency, the Task Queue API provides asynchronous calls that minimize latency.

Consider the case where you need to add 10 tasks to 10 different queues (thus you cannot batch them). In the worst case, calling queue.add() 10 times in a loop could block up to 10 seconds, although it's very rare. Using the asynchronous interface to add tasks to their respective queues in parallel, you can reduce the worst-case latency to 1 second.

If you want to make asynchronous calls to a task queue, use the asynchronous methods provided by the Queue class and an RPC object. Call get_result() on the returned RPC object to force the request to complete. When asynchronously adding tasks in a transaction, you should call get_result() on the RPC object before committing the transaction to ensure that the request has finished .

Enqueuing tasks in Cloud Datastore transactions

You can enqueue a task as part of a Google Cloud Datastore transaction, such that the task is only enqueued—and guaranteed to be enqueued—if the transaction is committed successfully. Tasks added in a transaction are considered to be a part of it and have the same level of isolation and consistency.

An application cannot insert more than five transactional tasks into task queues during a single transaction. Transactional tasks must not have user-specified names.

The following code sample demonstrates how to insert transactional tasks into a push queue as part of a Cloud Datastore transaction:

from google.appengine.api import taskqueue
from google.appengine.ext import ndb

@ndb.transactional
def do_something_in_transaction():
  taskqueue.add(url='/path/to/my/worker', transactional=True)
  #...

do_something_in_transaction()

Using the deferred task library instead of a worker service

Setting up a handler for each distinct task (as described in the previous sections) can be cumbersome, as can serializing and deserializing complex arguments for the task—particularly if you have many diverse but small tasks that you want to run on the queue. The Python SDK includes a library (google.appengine.ext.deferred) exposing a simple function that allows you to bypass all the work of setting up dedicated task handlers and serializing and deserializing your parameters.

To use this library, you need to add the deferred builtin to app.yaml. For more information, see the Built-in Handlers section of the app.yaml reference.

To use the deferred library, simply pass the function and its arguments to deferred.defer():

import logging

from google.appengine.ext import deferred

def do_something_expensive(a, b, c=None):
    logging.info("Doing something expensive!")
    # Do your work here

# Somewhere else
deferred.defer(do_something_expensive, "Hello, world!", 42, True)

The deferred library packages your function call and its arguments, then adds it to the task queue. When the task is executed, the deferred library executes do_something_expensive("Hello, world!", 42, True).

For more information about using the deferred library in Python, refer to Background Work with the Deferred Library.

Working with tasks in a multi-tenant application

By default, push queues use the current namespace as set in the namespace manager at the time the task is created. If your application uses multitenancy, see the Namespaces Python API.

What's next

• Learn how to create task handlers.