Showing posts with label TV. Show all posts
Showing posts with label TV. Show all posts

Friday, May 16, 2014

Improving the performance of our JavaScript inheritance model


by Dylan Oudyk

When building the initial architecture for Netflix’s common TV UI in early 2009, we knew we wanted to use inheritance to share common functionality in our codebase. Since the first engineers working on the code were coming from the heavily Java-based Website, they preferred simulating a classical inheritance model: an easy way to extend objects and override functions, but also have the ability to invoke the overridden function. (Since then we’ve moved on to greener pastures, or should we say design patterns, but the classical inheritance model still lives on and remains useful within our codebase.) For example, every view in our UI inherits from a base view with common methods and properties, like show, hide, visible, and parent. Concrete views extend these methods while still retaining the base view behavior.


After searching around and considering a number of approaches, we landed on John Resig’s Simple JavaScript Inheritance. We really liked the syntactic sugar of this._super(), which allows you to invoke the super-class function from within the overriding function.


Resig’s approach allowed us to write simple code like this:

var Human = Class.extend({
    init: function (height, weight){
        this.height = height;
        this.weight = weight;
    }
});
var Mutant = Human.extend({
    init: function (height, weight, abilities){
        this._super(height, weight);
        this.abilities = abilities;
    }
});

Not so super, after all


While his approach was sugary sweet and simple, we’d always been leery of a few aspects, especially considering our performance- and memory-constrained environment on TV devices:
  • When extending an object, it loops over the prototype to find functions that use _super
  • To find _super it decompiles a function into a string and tests that string against a regular expression.
  • Any function using _super is then wrapped in a closure to achieve the super-class chaining.


All that sugar was slowing us down. We found that the _super implementation was about 70% slower in executing overridden functions than a vanilla approach of calling the super-class function directly. The overhead of having to invoke the closure then invoke the overriding function which then invokes the inherited function yields a performance penalty that starts to add up directly onto the execution stack.  Our application not only runs on beefy game consoles like the PS3, PS4 and Xbox 360, but also on single-core Blu-ray players with 600 MHz ARM processors.  A great user experience demands that the code run as fast as possible for the UI to remain responsive.


Not only was _super slowing us down, it was gobbling up memory. We have an automation suite that stress tests the UI by putting it through its paces: browsing the entire grid of content, conducting multiple playbacks, searching, etc. We run it against our builds to ensure our memory footprint remains relatively stable and to catch any memory leaks that might accidentally get introduced. On a PS3, we profiled the memory usage with and without _super.  Our codebase had close to 720 uses of _super, that consumed about 12.2 MB. 12.2 MB is a drop in the bucket in the desktop browser world, but we work in a very constrained environment where 12.2 MB represents a large drop in a small bucket.


Worse still, when we went to move off the now deprecated YUI Compressor, more aggressive JavaScript minifiers like UglifyJS and Google’s Closure Compiler would obfuscate _super causing the regular expression test to fail and blow up our run-time execution.


We knew it was time to find a better way.


All your base are belong to us


We wanted to remove the performance and memory bottlenecks and also unblock ourselves from using a new minifier, but without having to rewrite significant portions of our large, existing codebase.


The _super implementation basically uses the wrapping closure as a complex pointer with references to the overriding function and the inherited function: 




If we could find a way to remove the middle man, we’d be able to have the best of both worlds.



We were able to leverage a lesser known feature in JavaScript, named function expressions, to cut out the expensive work that _super had been doing.


Now when we’re extending an object, we loop over the prototype and add a base property to every function. This base property points to the inherited function.


for (name in source) {
    value = source[name];
    currentValue = receiver[name];
    if (typeof value === 'function' && typeof currentValue === 'function' &&
        value !== currentValue) {
        value.base = currentValue;
    }
}

We use a named function expression within the overriding function to invoke the inherited function.


var Human = Class.extend({
    init: function (height, weight){
        this.height = height;
        this.weight = weight;
    }
});
var Mutant = Human.extend({
    init: function init(height, weight, abilities){
        init.base.call(this, height, weight);
        this.abilities = abilities;
    }
});

var theWolverine = new Mutant('5ft 3in', 300, [
    'adamantium skeleton',
    'heals quickly',
    'claws'
]);

(Please note, that if you need to support Internet explorer versions < 9 this may not be an option for you, but arguments.callee will be available, more details at Named function expressions demystified).


Arguably, we lost a teeny bit of sugar but at significant savings. The base approach is about 45% faster in executing an overridden method than _super. Add to that a significant memory savings.





As can be seen from the graph above, after running our application for one minute the memory savings is close to about 12.2MB.  We could pull the line back to the beginning and the memory savings would be even more because after one minute the application code has long been interpreted, and the classes have been created.


Conclusion
We believe we found a great way to invoke overridden methods with the named function expression approach.  By replacing _super, we saved RAM and CPU cycles.  We’d rather save the RAM for gorgeous artwork and our streaming video buffer.  The saved CPU cycles can be put to work on beautiful transitions.  Overall a change that improves the experience for our users.


References:
John Resig’s Simple JavaScript Inheritance System http://ejohn.org/blog/simple-javascript-inheritance/


Invoking Overridden Methods Performance Tests: http://jsperf.com/fun-with-method-overrides/2

Named Function Expressions: http://kangax.github.io/nfe/

Monday, December 16, 2013

Pioneering application design on TVs & TV-connected devices

Netflix recently launched the latest evolution of our core app for an increasing number of TVs and TV-connected devices. The app represents a unique mixture of platform and user interface design innovations. One of these innovations is that this app leverages both web standard technologies we love (like JavaScript) and a lightweight native graphics framework.

Understanding the motivations for our most recent platform advancement requires some context. In the past we’ve explored many different approaches. In 2009 we implemented a Flash Lite based app. Soon after, in 2010, we shifted to a WebKit based app using predominantly the QtWebKit port. Each app eventually reached a critical point where our innovation goals required us to adapt our technology stack.

Evolution of an application platform


We’ve seen WebKit mature into a full-fledged platform for application development. Advances in HTML5 and CSS3 introduced much needed semantics and styling improvements. JavaScript can utilize WebGL backed canvases, drag and drop, geolocation, and more. Increasingly WebKit will have hooks into device services allowing integration with hardware and data outside the browser sandbox. Improvements in mobile device capabilities have made many of these and future advances desirable.

We released our first QtWebKit app in 2010. Over the next 3 years our engineers shared our innovations and approaches with the WebKit community. Our platform engineers contributed our accelerated compositing implementation. Meanwhile our user interface engineers shared best practices and identified rendering optimizations deep in WebCore internals. In addition, we continue to drive standardization efforts for HTML5 Premium Video Extensions and have adopted them for desktop.

Devices running our core app on TVs and TV-connected devices subject WebKit to unique use cases. We deliver a long-lived, single-page, image and video heavy user interface on hardware with a range of CPU speed and addressable RAM, and varied rendering and network pipelines. The gamut of devices is considerable with significant variations in functionality and performance.

Our technology stack innovation


We strive to provide customers with rich, innovative content discovery and playback experiences. All devices running the new Netflix TV app are now running our own custom JS+native graphics framework. This framework enables us to reach our customer experience goals on the broadest set of TVs and TV-connected devices. We own the feature roadmap and tooling and can innovate with minimal constraints.

Our framework is optimized for fast 2D rendering of images, text and color fills. We render from a tree of graphics objects. The approach offers ease of use over immediate mode rendering contexts such as HTML canvas. Display property changes in these objects are aggregated then applied en masse post user interaction.

A bespoke rendering pipeline enables granular control over surfaces, the bitmap data representation of one or more graphics objects. Our surfaces are similar to accelerated compositing surfaces used by modern browsers. Intelligent surface allocation reduces surface (re)creation costs and the resulting memory fragmentation over time. Additionally we have fine-grained control of image decode activity leading up to surface creation.

As the platform matured it gained a pluggable cinematic effect pipeline with blur, desaturation, masking and tinting. These effects can be implemented very close to the metal, keeping them fast on more devices.

While we’re not running full WebKit, we are heavily leveraging JavaScriptCore. We experimented with V8 and SpiderMonkey (with JIT), yet both were impractical without stable ports for the various chipset architectures in use by device manufacturers.

We also rely on WebKit’s Web Inspector for debugging. Our framework integrates directly with a standalone Node server (and ultimately the Web Inspector) using the public remote debugging protocol. The Elements tab displays a tree of graphics objects. The Sources, Network and Timeline tabs work mostly like you’d expect. Familiar tools help while we debug the app running on a reference framework implementation or development devices.

An A/B test of our app written in this new framework performed better than our existing app. Our future is ours to define and we’re not done having fun.

Join our team


We’re working on exciting new features, constantly improving our platform, and we’re looking for help. Our growing team is looking for experts to join us. If you’d like to apply, take a look here.


Monday, November 18, 2013

Building the New Netflix Experience for TV

by Joubert Nel

We just launched a new Netflix experience for TV and game consoles. The new design is based on our premise that each show or movie has a tone and a narrative that should be conveyed by the UI. To tell a richer story we provide relevant evidence and cinematic art that better explain why we think you should watch a show or movie.




The new user interface required us to question our paradigms about what can be delivered on TV – not only is this UI more demanding of game consoles than any of our previous UIs, but we also wanted budget devices to deliver a richer experience than what was previously possible.

For the first time we needed a single UI that could accept navigation using a TV remote or game controller, as well as voice commands and remotes that direct a mouse cursor on screen.

Before we get into how we developed for performance and built for different input methods, let’s take a look at our UI stack.


UI Stack

My team builds Netflix UIs for the devices in your living room: PlayStation 3, PlayStation 4, Xbox 360, Roku 3, and recent Smart TVs and Blu-ray players.

We deploy UI updates with new A/B tests, support for new locales like the Netherlands, and new features like Profiles. While remaining flexible, we also want to take advantage of as much of the underlying hardware as possible in a cross-platform way.

So, a few years ago we broke our device client code into two parts: an SDK that runs on the metal, and a UI written in JavaScript. The SDK provides a rendering engine, JavaScript runtime, networking, security, video playback, and other platform hooks. Depending on the device, SDK updates range from quarterly to annually to never. The UI, in contrast, can be updated at any time and is downloaded (or retrieved from disk cache) when the user fires up Netflix.

Key, Voice, Pointer

The traditional way for users to control our UI on a game console or TV is via an LRUD input (left/right/up/down) such as a TV remote control or game controller. Additionally, Xbox 360 users should be able to navigate with voice commands and folks with an LG Magic Remote Smart TV must be able to navigate by pointing their remote control at elements on screen. Our new UI is our first to incorporate all three input methods in a single design.

We wanted to build our view components in such a way that their interaction behaviors are encapsulated. This code proximity makes code more maintainable and reusable and the class hierarchy more robust. We needed a consistent way to dispatch the three kinds of user input events to the view hierarchy.

We created a new JavaScript event dispatcher that routes key, pointer, and voice input in a uniform way to views. We needed an incremental solution that didn’t require refactoring the whole codebase, so we designed it to coexist with our legacy key handling and provide a migration path.

We must produce JavaScript builds that only contain code for those methods supported by the target device because reduced code size yields faster code parsing, in turn reducing startup time.

To produce lean builds, we use a text preprocessor to strip out input handling code that is irrelevant to a target platform. The advantage of using a text preprocessor instead of, for example, using mixins to layer in additional appearances and interactions, is that we get much higher levels of code proximity and simplicity.

Performance

Devices in the living room use DirectFB or OpenGL for graphics (or something OpenGL-like) and can use hardware acceleration for animating elements of the UI. Leveraging the GPU is key in creating a smooth experience that is responsive to user input – we’ve done it on WebKit using accelerated compositing (see WebKit in Your Living Room and Building the Netflix UI for Wii U).

The typical implementation of hardware accelerated animation of a rectangle requires width x height x bytes per pixel of memory. In our UI we animate entire scenes when transitioning between them; animating one scene at 1080p would require close to 8MB of memory (1920 x 1080 x 4) but at 720p requires 3.5MB (1280 x 720 x 4). We see devices with as little as 20MB memory allocated to a hardware-accelerated rendering cache. Moreover, other system resources such as main memory, disk cache, and CPU may also be severely constrained as compared to a mobile phone, laptop, or game console. 

How can we squeeze as much performance as possible out of budget devices and add more cinematic animations on game consoles?

We think JavaScript, HTML and CSS are great technologies to build compelling experiences with, such as our HTML 5 player UI. But we wanted more fine-grained control of the graphics layer and wanted optimizations for apps that do not need reflowable content. Our SDK team built a new rendering engine with which we can deliver animations on very resource constrained devices, making it possible to give customers our best UI. We can also enrich the experience with cinematic animations & effects on game consoles.

The second strategy is by grouping devices into performance classes that give us entry points to turn different knobs such as pool sizes, prefetch ranges, effects, animations, and caching, to take advantage of fewer/more resources while maintaining the integrity of the UI design & interaction.


Delivering great experiences

In the coming weeks we will be diving into more details of our JavaScript code base on this blog.

Building the new Netflix experience for TV was a lot of work, but it gave us a chance to be a PlayStation 4 launch partner, productize our biggest A/B test successes of 2013, and delight tens of millions of Netflix customers.

If this excites you and want to help build the future UIs for discovering and watching shows and movies, join our team!

Wednesday, January 18, 2012

WebKit in Your Living Room


Hi, it's Matt Seeley, engineer on the device UI team at Netflix.  My team uses WebKit, JavaScript, HTML5 and CSS3 to build user interfaces for the PlayStation 3, Wii, Blu-ray players, Internet-connected TVs, phones and tablets.

Recently I spoke at the HTML5 Dev Conf about WebKit-based UI development on consumer electronics.  I discussed:
  • Responding to user input quickly while deferring expensive user interface updates
  • Managing main and video memory footprint of a single page application
  • Understanding WebKit's interpretation and response to changes in HTML and CSS 
  • Achieving highest possible animation frame rates using accelerated compositing
Watch the video presentation from the conference:


Slides are also available in PDF

Astute readers will realize that portions of the content are also suitable for mobile and desktop development. It's all about building great user interfaces that take best possible advantage of the device.

Interested in joining our team? We're hiring!





Thursday, September 8, 2011

Netflix WebKit-Based UI for TV Devices

This is Matt McCarthy and Kim Trott, device UI engineering managers at Netflix. Our teams use WebKit, JavaScript, HTML5, and CSS3 to build user interfaces that are delivered to millions of Netflix customers on game consoles, Blu-ray players, and Internet-connected TVs.

We recently spoke at OSCON 2011 about how Netflix develops these user interfaces. We talked about:
  • How we use web technologies to support rapid innovation
  • How we architect our UIs to enable vast variation
  • Strategies to improve performance and take advantage of game consoles’ muscle while still running well on limited hardware
We wanted to share our learnings with a broader audience, so we've posted our slide deck. Hopefully, you'll find it interesting, or (better yet) useful.

Netflix Webkit-Based UI for TV Devices: PowerPoint 2010 | PDF | Slideshare