OpenVX Overview

Portable, Power-efficient Vision Processing

OpenVX is an open, royalty-free standard for cross platform acceleration of computer vision applications. OpenVX enables performance and power-optimized computer vision processing, especially important in embedded and real-time use cases such as face, body and gesture tracking, smart video surveillance, advanced driver assistance systems (ADAS), object and scene reconstruction, augmented reality, visual inspection, robotics and more.

OpenVX 1.2 was released on 1st May 2017

OpenVX – Vision Acceleration

Royalty-free open standard API

• Reliably accelerated by hardware vendors
• Tightly defined conformance tests

Targeted at low-power, real-time applications

• Mobile and embedded platforms

Portability across diverse heterogeneous processors

• ISPs, Dedicated hardware, DSPs and DSP arrays, GPUs, Multi-core CPUs …

Doesn’t require high-power CPU/GPU Complex

• Low-power host can setup and manage frame-rate vision processing pipeline

OpenVX Graphs – The Key to Efficiency

OpenVX developers express a graph of image operations (‘Nodes’)

• Nodes can be on any hardware or processor coded in any language

Graph enables implementations to optimize for power and performance

• E.g. Nodes may be fused by the implementation to eliminate memory transfers
• E.g. Processing can be tiled to keep data entirely in local memory/cache

Minimizes host interaction during frame-rate graph execution

• Host processor can setup graph which can then execute almost autonomously

OpenVX Framework Efficiency..

Layered Vision Processing Ecosystem

OpenVX Status

OpenVX 1.1 Specification released 2nd May 2016 at Embedded Vision Summit

• 18 months after OpenVX 1.0 in October 2014
• Expands node functionality AND enhances graph framework

OpenVX 1.0 open source sample implementation and conformance tests

• Will be updated to OpenVX 1.1 in 1H16

Roadmap discussions

• Significantly broaden node functionality – including in-graph neural nets
• OpenVX SC – refining OpenVX for markets requiring API safety certification

OpenVX and OpenCV are Complementary

OpenVX Neural Network Extension

The OpenVX™ working group has release an extension to enable Convolutional Neural Network topologies to be represented as OpenVX graphs and mixed with traditional vision functions.

Neural Network technology has seen recent explosive progress in solving pattern matching tasks in computer vision such as object recognition, face identification, image search, image to text, and is also playing a key part in enabling driver assistance and autonomous driving systems. Convolutional Neural Networks (CNN) are computationally expensive, and so many companies are actively developing mobile and embedded processor architectures to accelerate neural network-based inferencing at high speed and low power. As a result of such rapid progress, the market for embedded neural network processing is in danger of fragmenting, creating barriers for developers seeking to configure and accelerate inferencing engines across multiple platforms.

The OpenVX Neural Network extension specifies an architecture for executing CNN-based inference in OpenVX graphs. The extension defines a multi-dimensional tensor object data structure which can be used to connect neural network layers, represented as OpenVX nodes, to create flexible CNN topologies. OpenVX neural network layers types include convolution, pooling, fully-connected, normalization, soft-max and activation – with nine different activation functions. The extension enables neural network inferencing to be mixed with traditional vision processing operations in the same OpenVX graph.

Today, OpenVX has also released an Import/Export extension that complements the Neural Network extension by defining an API to import and export OpenVX objects, such as: traditional computer vision nodes, data objects of a graph or partial graph, and CNN objects including network weights and biases or complete networks.

The high-level abstraction of OpenVX enables implementers to accelerate a dataflow graph of vision functions across a diverse array of hardware and software acceleration platforms. The inclusion of neural net inferencing functionality in OpenVX enables the same portable, processor independent, expression of functionality with significant freedom and flexibility in how that inferencing is actually accelerated. The OpenVX Neural Net extension is released in provisional form to enable developers and implementers to provide feedback before finalization and industry feedback is welcomed at the OpenVX Forums.

Khronos is coordinating its neural network activities, and expects that NNEF files will be able to represent all aspects of an OpenVX neural network graph, and that OpenVX will enable import of network topologies via NNEF files through the Import/Export extension, once the NEFF format definition is complete.

Convolutional Neural Network topologies can be represented as OpenVX graphs

• Layers are represented as OpenVX nodes
• Layers connected by multi-dimensional tensors objects
• Layers include convolution, normalization, pooling, fully-connected, soft-max
• Also activation layers – with nine different activation functions
• CNN nodes can be mixed with traditional vision nodes

Import/Export Extension

• Efficient handling of network weights/biases or complete networks

The specification is provisional

• Seeking feedback from the deep learning community. Join the discussion.