Using third-party libraries

The Python runtime in the standard environment includes the Python standard library, the App Engine libraries, and a few bundled third-party packages. In addition to those libraries, you can also use third-party libraries that are pure Python code with no C extensions.

Adding libraries

You can add a third-party library to your app in one of two ways:

• Requesting a bundled third-party library in the app.yaml, or
• Copying the third-party library's pure-Python source code into your application's source code.

Requesting a library

You can request a library by using the libraries: directive in app.yaml.

libraries:
- name: PIL
  version: "1.1.7"
- name: webob
  version: "1.1.1"

Note that:

• The library must be one of the supported runtime-provided third-party libraries.
• When deployed, App Engine will provide the requested libraries to the runtime environment.
• Some libraries must be installed locally.

Installing a third-party library

In order to use a third-party library, copy it into a folder in your project's source directory. The library must be implemented as pure Python code with no C extensions. The code is uploaded to App Engine with your application code, and counts towards file quotas.

To copy a library into your project:

1. Create a directory to store your third-party libraries, such as lib/.

   mkdir lib
   

2. Use pip (version 6 or later) with the -t <directory> flag to copy the libraries into the folder you created in the previous step. For example:

   pip install -t lib/ <library_name>
   

   Using Homebrew Python on Mac OS X?

   Homebrew issues

   OK

3. Create a file named appengine_config.py in the same folder as your app.yaml file.

4. Edit the appengine_config.py file and provide your library directory to the vendor.add() method.

   # appengine_config.py
   from google.appengine.ext import vendor
   
   # Add any libraries install in the "lib" folder.
   vendor.add('lib')
   

The appengine_config.py file above assumes that the current working directory is where the lib folder is located. In some cases, such as unit tests, the current working directory can be different. To avoid errors, you can explicity pass in the full path to the lib folder using:

vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))

Using pip requirements files

pip can read a list of libraries to install from a file, known as a requirements file. Requirements files make it easy to set up a new development environment for your app, and upgrade to new versions of libraries.

A requirements file is a text file with one line per library, listing the package name and version:

Flask==0.10
Markdown==2.5.2
google-api-python-client

To install the libraries from a requirements file, use the -r flag in addition to the -t lib flag:

pip install -t lib -r requirements.txt

Using libraries with the local development server

While several of the runtime-provided libraries are available to your local development environment through the App Engine SDK, the following libraries are platform-dependent and must be installed locally before you can use them with the development server:

• lxml
• matplotlib
• mysqldb
• numpy
• PIL
• crcmod
• pycrypto

You can use the pip command to install all of these packages from the Python package index (PyPI).

sudo pip install lxml==2.3.5

Depending on your platform, you might need to install build support tools and Python sources to install these libraries.

• On Linux, the package manager can provide these prerequisites and can often provide a pre-built version of the library.
• On Windows, installers for pre-built versions are usually available. *On OS X, the Xcode Command Line Tools are required to build some packages.

Using Django or matplotlib

This section provides information you should know when using the Django or matplotlib libraries.

Using Django

Django is a full-featured web application framework for Python. It provides a full stack of interchangable components, including dispatch, views, middleware, and templating components, and many others.

The Django data modeling interface is not compatible with the App Engine datastore. You can use the App Engine data modeling libraries (db or ndb) in your Django applications. However, third-party Django applications that use the Django data modeling interface, most notably Django's Admin application, might not directly work with App Engine.

The Datastore modeling library (DB) is the default. To use Django with the NDB storage API instead, add 'google.appengine.ext.ndb.django_middleware.NdbDjangoMiddleware', to the MIDDLEWARE_CLASSES entry in your Django settings.py file. It's a good idea to insert it in front of any other middleware classes, since some other middleware might make datastore calls and those won't be handled properly if that middleware is invoked before this middleware. You can learn more about Django middleware in the project documentation.

To enable Django in your app, specify the WSGI application and Django library in app.yaml:

...
handlers:
- url: /.*
  script: main.app  # a WSGI application in the main module's global scope

libraries:
- name: django
  version: "1.4"

The DJANGO_SETTINGS_MODULE environment variable must be set to the name of your Django settings module, typically 'settings', before packages are imported.

If your Django settings module is something other than settings.py, set the DJANGO_SETTINGS_MODULE environment variable accordingly either in your app.yaml file:

env_variables:
  DJANGO_SETTINGS_MODULE: 'myapp.settings'

Or in your Python code:

import os
# specify the name of your settings module
os.environ['DJANGO_SETTINGS_MODULE'] = 'myapp.settings'

import django.core.handlers.wsgi
app = django.core.handlers.wsgi.WSGIHandler()

Using matplotlib

Matplotlib is a plotting library that produces graphs and figures in a variety of image formats. On App Engine, the interactive modes of matplotlib are not supported, and a number of other features are also unavailable. This means you cannot use pyplot.show() as many matplotlib tutorials suggest. Instead, you should use pyplot.savefig() to write image data to the output stream, a cStringIO.StringIO instance, or the Google Cloud Storage using the Cloud Storage Client Library.

Matplotlib allows extensive customization through the use of the matplotlibrc configuration file, which should be placed in the application's top-level directory. Alternatively, you can set the MATPLOTLIBRC environment variable to a path relative to your application's directory.

The default backend is AGG, which allows writing files of all supported formats: PNG (the default format), RAW, PS, PDF, SVG and SVGZ. If you make the PIL library available by adding PIL to the libraries section of app.yaml, then the AGG backend will automatically support writing JPEG and TIFF image formats as well.

Matplotlib comes with a number of fonts which are automatically available. You can use custom fonts by uploading them in TTF format along with your application, and setting the TTFPATH environment variable to the path where they are located, relative to your application's directory. For more information, see the app.yaml reference.

A number of matplotlib features are not supported on App Engine. In particular:

• There is no ~/.matplotlib directory. However, there are alternative locations to place the matplotlibrc configuration file, as described above.
• Interactive backends and GUI elements are not supported.
• The EMF, Cairo and GDK backends are not supported.
• There is no caching, and therefore a number of mechanisms will re-calculate or re-download data that would normally be cached. Specific caching mechanisms that have been disabled include font data calculated by matplotlib.font_manager.FontManager.findfont, sample data downloaded by matplotlib.cbook.get_sample_data and financial data downloaded by matplotlib.finance.fetch_historical_yahoo.
  • Because there is no caching, it is not possible to call [matplotlib.cbook.get_sample_data](http://matplotlib.org/api/cbook_api.html#matplotlib.cbook.get_sample_data) with asfileobj=False unless examples.download is set to False.
• All features that invoke external commands have been disabled.
  • Use of fontconfig has been disabled. Fonts are found through the mechanism described above.
  • Use of LaTeX for text rendering is not supported. Setting text.usetex to True will not work.
  • Use of an external PostScript distiller program is not supported. Setting ps.usedistiller to ghostscript or xpdf will not work.
  • Use of an external video encoding program is not supported. The matplotlib.animation.Animation.save method will not work, and therefore, the matplotlib.animation package is not useful.
  • The matplotlib.cbook.report_memory function and matplotlib.cbook.MemoryMonitor class are not supported.
• The matplotlib.test function has been disabled.

What's next

• Learn more about the App Engine Python runtime environment.
• Review the built-in third-party libraries reference.