Showing posts with label OpenGL ES. Show all posts
Showing posts with label OpenGL ES. Show all posts

25 May 2015

Game Performance: Geometry Instancing

Posted by Shanee Nishry, Games Developer Advocate

Imagine a beautiful virtual forest with countless trees, plants and vegetation, or a stadium with countless people in the crowd cheering. If you are heroic you might like the idea of an epic battle between armies.

Rendering a lot of meshes is desired to create a beautiful scene like a forest, a cheering crowd or an army, but doing so is quite costly and reduces the frame rate. Fortunately this is possible using a simple technique called Geometry Instancing.

Geometry instancing can be used in 2D games for rendering a large number of sprites, or in 3D for things like particles, characters and environment.

The NDK code sample More Teapots demoing the content of this article can be found with the ndk inside the samples folder and in the git repository.

Support and Extensions

Geometry instancing is available from OpenGL ES 3.0 and to OpenGL 2.0 devices which support the GL_NV_draw_instanced or GL_EXT_draw_instanced extensions. More information on how to using the extensions is shown in the More Teapots demo.

Overview

Submitting draw calls causes OpenGL to queue commands to be sent to the GPU, this has an expensive overhead which may affect performance. This overhead grows when changing states such as alpha blending function, active shader, textures and buffers.

Geometry Instancing is a technique that combines multiple draws of the same mesh into a single draw call, resulting in reduced overhead and potentially increased performance. This works even when different transformations are required.

The algorithm

To explain how Geometry Instancing works let’s quickly overview traditional drawing.

Traditional Drawing

To a draw a mesh you’d usually prepare a vertex buffer and an index buffer, bind your shader and buffers, set your uniforms such as a World View Projection matrix and make a draw call.

To draw multiple instances using the same mesh you set new uniform values for the transformations and other data and call draw again. This is repeated for every instance.

Drawing with Geometry Instancing

Geometry Instancing reduces CPU overhead by reducing the sequence described above into a single buffer and draw call.

It works by using an additional buffer which contains custom per-instance data needed by your shader, such as transformations, color, light data.

The first change to your workflow is to create the additional buffer on initialization stage.

To put it into code let’s define an example per-instance data that includes a world view projection matrix and a color:

C++

struct PerInstanceData
{
 Mat4x4 WorldViewProj;
 Vector4 Color;
};

You also need to the structure to your shader. The easiest way is by creating a Uniform Block with an array:

GLSL

#define MAX_INSTANCES 512

layout(std140) uniform PerInstanceData {
    struct
    {
        mat4      uMVP;
        vec4      uColor;
    } Data[ MAX_INSTANCES ];
};

Note that uniform blocks have limited sizes. You can find the maximum number of bytes you can use by querying for GL_MAX_UNIFORM_BLOCK_SIZE using glGetIntegerv.

Example:

GLint max_block_size = 0;
glGetIntegerv( GL_MAX_UNIFORM_BLOCK_SIZE, &max_block_size );

Bind the uniform block on the CPU in your program’s initialization stage:

C++

#define MAX_INSTANCES 512
#define BINDING_POINT 1
GLuint shaderProgram; // Compiled shader program

// Bind Uniform Block
GLuint blockIndex = glGetUniformBlockIndex( shaderProgram, "PerInstanceData" );
glUniformBlockBinding( shaderProgram, blockIndex, BINDING_POINT );

And create a corresponding uniform buffer object:

C++

// Create Instance Buffer
GLuint instanceBuffer;

glGenBuffers( 1, &instanceBuffer );
glBindBuffer( GL_UNIFORM_BUFFER, instanceBuffer );
glBindBufferBase( GL_UNIFORM_BUFFER, BINDING_POINT, instanceBuffer );

// Initialize buffer size
glBufferData( GL_UNIFORM_BUFFER, MAX_INSTANCES * sizeof( PerInstanceData ), NULL, GL_DYNAMIC_DRAW );

The next step is to update the instance data every frame to reflect changes to the visible objects you are going to draw. Once you have your new instance buffer you can draw everything with a single call to glDrawElementsInstanced.

You update the instance buffer using glMapBufferRange. This function locks the buffer and retrieves a pointer to the byte data allowing you to copy your per-instance data. Unlock your buffer using glUnmapBuffer when you are done.

Here is a simple example for updating the instance data:

const int NUM_SCENE_OBJECTS = …; // number of objects visible in your scene which share the same mesh

// Bind the buffer
glBindBuffer( GL_UNIFORM_BUFFER, instanceBuffer );

// Retrieve pointer to map the data
PerInstanceData* pBuffer = (PerInstanceData*) glMapBufferRange( GL_UNIFORM_BUFFER, 0,
                NUM_SCENE_OBJECTS * sizeof( PerInstanceData ),
                GL_MAP_WRITE_BIT | GL_MAP_INVALIDATE_RANGE_BIT );

// Iterate the scene objects
for ( int i = 0; i < NUM_SCENE_OBJECTS; ++i )
{
    pBuffer[ i ].WorldViewProj = ... // Copy World View Projection matrix
    pBuffer[ i ].Color = …               // Copy color
}

glUnmapBuffer( GL_UNIFORM_BUFFER ); // Unmap the buffer

And finally you can draw everything with a single call to glDrawElementsInstanced or glDrawArraysInstanced (depending if you are using an index buffer):

glDrawElementsInstanced( GL_TRIANGLES, NUM_INDICES, GL_UNSIGNED_SHORT, 0,
                NUM_SCENE_OBJECTS );

You are almost done! There is just one more step to do. In your shader you need to make use of the new uniform buffer object for your transformations and colors. In your shader main program:

void main()
{
    …
    gl_Position = PerInstanceData.Data[ gl_InstanceID ].uMVP * inPosition;
    outColor = PerInstanceData.Data[ gl_InstanceID ].uColor;
}

You might have noticed the use gl_InstanceID. This is a predefined OpenGL vertex shader variable that tells your program which instance it is currently drawing. Using this variable your shader can properly iterate the instance data and match the correct transformation and color for every vertex.

That’s it! You are now ready to use Geometry Instancing. If you are drawing the same mesh multiple times in a frame make sure to implement Geometry Instancing in your pipeline! This can greatly reduce overhead and improve performance.

22 April 2015

Game Performance: Explicit Uniform Locations

Posted by Shanee Nishry, Games Developer Advocate

Uniforms variables in GLSL are crucial for passing data between the game code on the CPU and the shader program on the graphics card. Unfortunately, up until the availability of OpenGL ES 3.1, using uniforms required some preparation which made the workflow slightly more complicated and wasted time during loading.

Let us examine a simple vertex shader and see how OpenGL ES 3.1 allows us to improve it:

#version 300 es

layout(location = 0) in vec4 vertexPosition;
layout(location = 1) in vec2 vertexUV;

uniform mat4 matWorldViewProjection;

out vec2 outTexCoord;

void main()
{
    outTexCoord = vertexUV;
    gl_Position = matWorldViewProjection * vertexPosition;
}

Note: You might be familiar with this shader from a previous Game Performance article on Layout Qualifiers. Find it here.

We have a single uniform for our world view projection matrix:

uniform mat4 matWorldViewProjection;

The inefficiency appears when you want to assign the uniform value.

You need to use glUniformMatrix4fv or glUniform4f to set the uniform’s value but you also need the handle for the uniform’s location in the program. To get the handle you must call glGetUniformLocation.

GLuint program; // the shader program
float matWorldViewProject[16]; // 4x4 matrix as float array

GLint handle = glGetUniformLocation( program, “matWorldViewProjection” );
glUniformMatrix4fv( handle, 1, false, matWorldViewProject );

That pattern leads to having to call glGetUniformLocation for each uniform in every shader and keeping the handles or worse, calling glGetUniformLocation every frame.

Warning! Never call glGetUniformLocation every frame! Not only is it bad practice but it is slow and bad for your game’s performance. Always call it during initialization and save it somewhere in your code for use in the render loop.

This process is inefficient, it requires you to do more work and costs precious time and performance.

Also take into consideration that you might have multiple shaders with the same uniforms. It would be much better if your code was deterministic and the shader language allowed you to explicitly set the locations of your uniforms so you don’t need to query and manage access handles. This is now possible with Explicit Uniform Locations.

You can set the location for uniforms directly in the shader’s code. They are declared like this

layout(location = index) uniform type name;

For our example shader it would be:

layout(location = 0) uniform mat4 matWorldViewProjection;

This means you never need to use glGetUniformLocation again, resulting in simpler code, initialization process and saved CPU cycles.

This is how the example shader looks after the change. Changes are marked in bold:

#version 310 es

layout(location = 0) in vec4 vertexPosition;
layout(location = 1) in vec2 vertexUV;

layout(location = 0) uniform mat4 matWorldViewProjection;

out vec2 outTexCoord;

void main()
{
    outTexCoord = vertexUV;
    gl_Position = matWorldViewProjection * vertexPosition;
}

As Explicit Uniform Locations are only supported from OpenGL ES 3.1 we also changed the version declaration to 310.

Now all you need to do to set your matWorldViewProjection uniform value is call glUniformMatrix4fv for the handle 0:

const GLint UNIFORM_MAT_WVP = 0; // Uniform location for WorldViewProjection
float matWorldViewProject[16]; // 4x4 matrix as float array

glUniformMatrix4fv( UNIFORM_MAT_WVP, 1, false, matWorldViewProject );

This change is extremely simple and the improvements can be substantial, producing cleaner code, asset pipeline and improved performance. Be sure to make these changes If you are targeting OpenGL ES 3.1 or creating multiple APKs to support a wide range of devices.

To learn more about Explicit Uniform Locations check out the OpenGL wiki page for it which contains valuable information on different layouts and how arrays are represented.

26 March 2015

Game Performance: Layout Qualifiers

Today, we want to share some best practices on using the OpenGL Shading Language (GLSL) that can optimize the performance of your game and simplify your workflow. Specifically, Layout qualifiers make your code more deterministic and increase performance by reducing your work.


Let’s start with a simple vertex shader and change it as we go along.

This basic vertex shader takes position and texture coordinates, transforms the position and outputs the data to the fragment shader:
attribute vec4 vertexPosition;
attribute vec2 vertexUV;

uniform mat4 matWorldViewProjection;

varying vec2 outTexCoord;

void main()
{
  outTexCoord = vertexUV;
  gl_Position = matWorldViewProjection * vertexPosition;
}

Vertex Attribute Index

To draw a mesh on to the screen, you need to create a vertex buffer and fill it with vertex data, including positions and texture coordinates for this example.

In our sample shader, the vertex data may be laid out like this:
struct Vertex
{
  Vector4 Position;
  Vector2 TexCoords;
};
Therefore, we defined our vertex shader attributes like this:
attribute vec4 vertexPosition;
attribute vec2  vertexUV;
To associate the vertex data with the shader attributes, a call to glGetAttribLocation will get the handle of the named attribute. The attribute format is then detailed with a call to glVertexAttribPointer.
GLint handleVertexPos = glGetAttribLocation( myShaderProgram, "vertexPosition" );
glVertexAttribPointer( handleVertexPos, 4, GL_FLOAT, GL_FALSE, 0, 0 );

GLint handleVertexUV = glGetAttribLocation( myShaderProgram, "vertexUV" );
glVertexAttribPointer( handleVertexUV, 2, GL_FLOAT, GL_FALSE, 0, 0 );
But you may have multiple shaders with the vertexPosition attribute and calling glGetAttribLocation for every shader is a waste of performance which increases the loading time of your game.

Using layout qualifiers you can change your vertex shader attributes declaration like this:
layout(location = 0) in vec4 vertexPosition;
layout(location = 1) in vec2 vertexUV;
To do so you also need to tell the shader compiler that your shader is aimed at GL ES version 3.1. This is done by adding a version declaration:
#version 300 es
Let’s see how this affects our shader, changes are marked in bold:
#version 300 es

layout(location = 0) in vec4 vertexPosition;
layout(location = 1) in vec2 vertexUV;

uniform mat4 matWorldViewProjection;

out vec2 outTexCoord;

void main()
{
  outTexCoord = vertexUV;
  gl_Position = matWorldViewProjection * vertexPosition;
}
Note that we also changed outTexCoord from varying to out. The varying keyword is deprecated from version 300 es and requires changing for the shader to work.

Note that Vertex Attribute qualifiers and #version 300 es are supported from OpenGL ES 3.0. The desktop equivalent is supported on OpenGL 3.3 and using #version 330.

Now you know your position attributes always at 0 and your texture coordinates will be at 1 and you can now bind your shader format without using glGetAttribLocation:
const int ATTRIB_POS = 0;
const int ATTRIB_UV   = 1;

glVertexAttribPointer( ATTRIB_POS, 4, GL_FLOAT, GL_FALSE, 0, 0 );
glVertexAttribPointer( ATTRIB_UV, 2, GL_FLOAT, GL_FALSE, 0, 0 );
This simple change leads to a cleaner pipeline, simpler code and saved performance during loading time.

To learn more about performance on Android, check out the Android Performance Patterns series.

Posted by Shanee Nishry, Games Developer Advocate

24 April 2009

Introducing GLSurfaceView

GLSurfaceView is a new API class in Android 1.5. GLSurfaceView makes OpenGL ES applications easier to write by:

  • Providing the glue code to connect OpenGL ES to the View system.
  • Providing the glue code to make OpenGL ES work with the Activity life-cycle.
  • Making it easy to choose an appropriate frame buffer pixel format.
  • Creating and managing a separate rendering thread to enable smooth animation.
  • Providing easy-to-use debugging tools for tracing OpenGL ES API calls and checking for errors.

GLSurfaceView is a good base for building an application that uses OpenGL ES for part or all of its rendering. A 2D or 3D action game would be a good candidate, as would a 2D or 3D data visualization application such as Google Maps StreetView.

The Simplest GLSurfaceView Application

Here's the source code to the simplest possible OpenGL ES application:

package com.example.android.apis.graphics;

import javax.microedition.khronos.egl.EGLConfig;
import javax.microedition.khronos.opengles.GL10;

import android.app.Activity;
import android.opengl.GLSurfaceView;
import android.os.Bundle;

public class ClearActivity extends Activity {
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        mGLView = new GLSurfaceView(this);
        mGLView.setRenderer(new ClearRenderer());
        setContentView(mGLView);
    }

    @Override
    protected void onPause() {
        super.onPause();
        mGLView.onPause();
    }

    @Override
    protected void onResume() {
        super.onResume();
        mGLView.onResume();
    }

    private GLSurfaceView mGLView;
}

class ClearRenderer implements GLSurfaceView.Renderer {
    public void onSurfaceCreated(GL10 gl, EGLConfig config) {
        // Do nothing special.
    }

    public void onSurfaceChanged(GL10 gl, int w, int h) {
        gl.glViewport(0, 0, w, h);
    }

    public void onDrawFrame(GL10 gl) {
        gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
    }
}

This program doesn't do much: it clears the screen to black on every frame. But it is a complete OpenGL application, that correctly implements the Android activity life-cycle. It pauses rendering when the activity is paused, and resumes it when the activity is resumed. You could use this application as the basis for non-interactive demonstration programs. Just add more OpenGL calls to the ClearRenderer.onDrawFrame method. Notice that you don't even need to subclass the GLSurfaceView view.

Note that the GLSurfaceView.Renderer interface has three methods:

The onSurfaceCreated() method is called at the start of rendering, and whenever the OpenGL ES drawing context has to be recreated. (The drawing context is typically lost and recreated when the activity is paused and resumed.) OnSurfaceCreated() is a good place to create long-lived OpenGL resources like textures.

The onSurfaceChanged() method is called when the surface changes size. It's a good place to set your OpenGL viewport. You may also want to set your camera here, if it's a fixed camera that doesn't move around the scene.

The onDrawFrame() method is called every frame, and is responsible for drawing the scene. You would typically start by calling glClear to clear the framebuffer, followed by other OpenGL ES calls to draw the current scene.

How about User Input?

If you want an interactive application (like a game), you will typically subclass GLSurfaceView, because that's an easy way of obtaining input events. Here's a slightly longer example showing how to do that:

package com.google.android.ClearTest;

import javax.microedition.khronos.egl.EGLConfig;
import javax.microedition.khronos.opengles.GL10;

import android.app.Activity;
import android.content.Context;
import android.opengl.GLSurfaceView;
import android.os.Bundle;
import android.view.MotionEvent;

public class ClearActivity extends Activity {
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        mGLView = new ClearGLSurfaceView(this);
        setContentView(mGLView);
    }

    @Override
    protected void onPause() {
        super.onPause();
        mGLView.onPause();
    }

    @Override
    protected void onResume() {
        super.onResume();
        mGLView.onResume();
    }

    private GLSurfaceView mGLView;
}

class ClearGLSurfaceView extends GLSurfaceView {
    public ClearGLSurfaceView(Context context) {
        super(context);
        mRenderer = new ClearRenderer();
        setRenderer(mRenderer);
    }

    public boolean onTouchEvent(final MotionEvent event) {
        queueEvent(new Runnable(){
            public void run() {
                mRenderer.setColor(event.getX() / getWidth(),
                        event.getY() / getHeight(), 1.0f);
            }});
            return true;
        }

        ClearRenderer mRenderer;
}

class ClearRenderer implements GLSurfaceView.Renderer {
    public void onSurfaceCreated(GL10 gl, EGLConfig config) {
        // Do nothing special.
    }

    public void onSurfaceChanged(GL10 gl, int w, int h) {
        gl.glViewport(0, 0, w, h);
    }

    public void onDrawFrame(GL10 gl) {
        gl.glClearColor(mRed, mGreen, mBlue, 1.0f);
        gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
    }

    public void setColor(float r, float g, float b) {
        mRed = r;
        mGreen = g;
        mBlue = b;
    }

    private float mRed;
    private float mGreen;
    private float mBlue;
}

This application clears the screen every frame. When you tap on the screen, it sets the clear color based on the (x,y) coordinates of your touch event. Note the use of queueEvent() in ClearGLSurfaceView.onTouchEvent(). The queueEvent() method is used to safely communicate between the UI thread and the rendering thread. If you prefer you can use some other Java cross-thread communication technique, such as synchronized methods on the Renderer class itself. But queueing events is often the simplest way of dealing with cross-thread communication.

Other GLSurfaceView Samples

Tired of just clearing the screen? You can find more interesting samples in the API Demos sample in the SDK. All the OpenGL ES samples have been converted to use the GLSurfaceView view:

  • GLSurfaceView - a spinning triangle
  • Kube - a cube puzzle demo
  • Translucent GLSurfaceView - shows how to display 3D graphics on a translucent background
  • Textured Triangle - shows how to draw a textured 3D triangle
  • Sprite Text - shows how to draw text into a texture and then composite it into a 3D scene
  • Touch Rotate - shows how to rotate a 3D object in response to user input.

Choosing a Surface

GLSurfaceView helps you choose the type of surface to render to. Different Android devices support different types of surfaces, with no common subset. This makes it tricky problem to choose the best available surface on each device. By default GLSurfaceView tries to find a surface that's as close as possible to a 16-bit RGB frame buffer with a 16-bit depth buffer. Depending upon your application's needs you may want to change this behavior. For example, the Translucent GLSurfaceView sample needs an Alpha channel in order to render translucent data. GLSurfaceView provides an overloaded setEGLSurfaceChooser() method to give the developer control over which surface type is chosen:

setEGLConfigChooser(boolean needDepth)
Choose a config that's closest to R5G6B5 with or without a 16-bit framebuffer
setEGLConfigChooser(int redSize, int greenSize,int blueSize, int alphaSize,int depthSize, int stencilSize)
Choose the config with the fewest number of bits per pixel that has at least as many bits-per-channel as specified in the constructor.
setEGLConfigChooser(EGLConfigChooser configChooser)
Allow total control over choosing a configuration. You pass in your own implementation of EGLConfigChooser, which gets to inspect the device's capabilities and choose a configuration.

Continuous Rendering vs. Render When Dirty

Most 3D applications, such as games or simulations, are continuously animated. But some 3D applications are more reactive: they wait passively until the user does something, and then react to it. For those types of applications, the default GLSurfaceView behavior of continuously redrawing the screen is a waste of time. If you are developing a reactive application, you can call GLSurfaceView.setRenderMode(RENDERMODE_WHEN_DIRTY), which turns off the continuous animation. Then you call GLSurfaceView.requestRender() whenever you want to re-render.

Help With Debugging

GLSurfaceView has a handy built-in feature for debugging OpenGL ES applications: the GLSurfaceView.setDebugFlags() method can be used to enable logging and/or error checking your OpenGL ES calls. Call this method in your GLSurfaceView's constructor, before calling setRenderer():

public ClearGLSurfaceView(Context context) {
    super(context);
    // Turn on error-checking and logging
    setDebugFlags(DEBUG_CHECK_GL_ERROR | DEBUG_LOG_GL_CALLS);
    mRenderer = new ClearRenderer();
    setRenderer(mRenderer);
}

Learn about Android 1.5 and more at Google I/O. Members of the Android team will be there to give a series of in-depth technical sessions and to field your toughest questions.

11 September 2008

Three new Samples: Triangle, SpriteText and Downloader

I've posted three new open source samples to the apps-for-android project: Triangle, SpriteText and Downloader.

The first two samples, Triangle and SpriteText, show techniques that would be useful to anyone using the OpenGL ES 3D graphics APIs to write Android applications. The samples contain several reusable classes that may eventually be incorporated (in some form) into the SDK. Chief among these is the GLView class, which abstracts the OpenGL ES book-keeping code from the rest of the application. GLView helps handle the extra work OpenGL ES applications have to do when the activity is paused and resumed, and when the display goes to sleep and wakes up. In the Pause/Resume case the OpenGL surface has to be recreated. In the display sleep / wake-up case the entire OpenGL context has to be recreated.

Triangle

The first sample, Triangle, shows how to use the GLView class and the OpenGL ES 3D library to display a spinning textured triangle. Think of it as the "hello, world" of OpenGL ES apps. Because it's relatively simple, it's a good place to start when experimenting with the OpenGL ES API.

SpriteText

The second sample, SpriteText, shows how to efficiently display screen-aligned text using the GL11Ext.glDrawTexiOES method. SpriteText contains a reusable LabelMaker class for drawing static text and screen-aligned images, as well as a Projector class for finding the 2D screen coordinates corresponding to a 3D point, and a MatrixTrackingGL class for keeping track of the current transformation matrix. Finally, it shows how to use these classes to display a milliseconds per frame counter. A ms/f counter can be helpful for tuning graphics performance.

Downloader

The third sample, Downloader, shows how to add a downloader activity to your application. The downloader activity runs at the beginning of your application and makes sure that a set of files have been downloaded from a web server to the phone's SD card. Downloader is useful for applications that need more local data than can fit into an .apk file. For example a game could use Downloader to download the game's artwork, sound effects, and level data. The Downloader activity is designed to be a drop-in addition to your application. You customize it by supplying the URL of an XML configuration file which lists the data files that need to be downloaded.