by Kristofer Baxter
In the past we’ve written about HTML5 Video (HTML5 Video in IE11 on Windows 8.1, and HTML5 Video at Netflix) but we haven't spoken much about how we built the player UI. The UI Engineering team here at Netflix has been supporting HTML5 based playback for a little over a year, and now seems like the right time to discuss some of the strategies and techniques we are using to support video playback without a plugin.
One of our main objectives is to keep Netflix familiar to our members. That means we’re keeping the design of the HTML5 player consistent with our Silverlight experience. Features should be rolled out simultaneously for the two platforms. However, HTML5 users will enter playback faster, can enjoy 1080p content when GPU accelerated, and keep all the functionality they know and love.
| Silverlight UI | HTML5 UI |
 |  |
In order to achieve a similar look and feel, we needed to recreate a few key elements of the Silverlight UI:
- Scale interface to users resolution
- Minimize Startup time via minimal dependency on data
- Ensure High Performance on low end hardware
Scaling interface to users resolution
No matter what resolution the browser window used for playback is, our current playback UI ensures all of the controls maintain the same percentage size on screen. This lets users choose their own dimensions for playing content without the UI getting in the way.
Normally, a modern web application could implement this using CSS vw and vh units. However, we found this approach to be inadequate for our needs. Our player can be displayed in two fashions -- taking over the entire initial containing block of a viewport, or a smaller portion. To solve for this, we implemented a sizing scheme based entirely on font-relative lengths.
In this small example, you can see the scaling implementation in a direct form.
<style>
.netflix-player-wrapper {
font-size: 16px;
}
#netflix-player {
position: absolute;
width: 90%; height: 90%;
left: 5%; top: 5%;
overflow: hidden;
background: #ccc;
font-size: 1em;
}
#player-sizing {
position: absolute;
width: 1em; height: 1em;
visibility: hidden;
font-size: 1em;
}
#ten-percent-height {
position: absolute;
width: 80%; height: 10em;
left: 10%; bottom: 8em;
background: #000;
display: flex;
}
#ten-percent-height > p {
display: block;
margin: 1em;
font-size: 2em;
color: #fff;
}
</style>
<div class="netflix-player-wrapper">
<div id="netflix-player">
<div id="player-sizing"></div>
<div id="ten-percent-height"><p>Text</p></div>
</div>
</div>
<script>
(function () {
var sizingEl = document.getElementById("player-sizing"),
controlWrapperEl = document.getElementById('netflix-player'),
currentEmSize = 1.0;
function resize() {
var wrapperHeight = controlWrapperEl.getBoundingClientRect().height,
sizingHeight = sizingEl.getBoundingClientRect().height,
wrapperOnePercentHeight = wrapperHeight / 100,
offsetSize;
if (sizingHeight > wrapperOnePercentHeight) {
offsetSize = sizingHeight / wrapperOnePercentHeight;
currentEmSize = currentEmSize / offsetSize;
} else if (wrapperOnePercentHeight > sizingHeight) {
offsetSize = wrapperOnePercentHeight / sizingHeight;
currentEmSize = currentEmSize * offsetSize;
}
controlWrapperEl.style.fontSize = currentEmSize + "em";
}
window.addEventListener("resize", resize, false);
resize();
})();
</script>
We implement this resizing functionality on a debounced interval in the player UI. Triggering it on every window resize would be wasteful.
By making an em unit represent 1% height of the "netflix-player" container, we can size all of our onscreen elements in a scaling manner - no matter how or where the netflix-player container is placed in the document.
Minimize Startup time via minimal dependency on data
Browser plugins like Flash and Silverlight can take several seconds to initialize, especially on a freshly booted machine. Now that we no longer need to initialize a plugin to play content, we can begin playback faster. However, we learned a lot about quick video startup in Silverlight, and can borrow techniques we developed to make our HTML5 UI launch content even faster.
When possible, allow playback to begin without title metadata.
If we already know which title the customer has selected to play (like a specific episode or movie), we can just start playback of that title immediately. Once the user has begun to buffer content, the UI can request display metadata. Metadata for the player can be a large payload since it includes episode data (title, synopsis, predicted rating), and is personalized to the user. By delaying the retrieval of metadata, users begin streaming 500 to 1200ms sooner in real-world usage.
For other conditions, such when a customer clicks play on a TV show and we want to start playback at the last episode that they were watching, we retrieve the specific episode the user wants before starting the playback process.
Populate controls which depend on rich data as that data becomes available.
Since we can begin playback before the player UI knows anything except which title to play, the player UI needs to be resilient against missing metadata. We display a minimal number of controls while this data is being requested. These controls include play/pause, exit playback, and full-screen toggling.
We use an eventing framework to let individual components know when data state has changed, so each component can stay decoupled. Here’s an example showing how we handle an event telling us the metadata is now loaded for the title.
function populateStatus() {
if (Metadata.videoIsKnown(ObjectPool.videoId())) {
// Update Status to reflect current playing item.
} else {
// Hide or remove current status
}
}
Metadata.addEventListener(Metadata.knownEvents.METADATA_LOADED, populateStatus);
Ensure High Performance on all hardware
Not everyone has the latest and greatest hardware at their disposal, but this shouldn't prevent all sorts of devices from playing content on Netflix. To this end, we develop using a wide variety of hardware and test using a wide range of representative devices.
We’ve found the issues preventing great performance on low end hardware can mostly be avoided by adhering to the following best practices:
Avoid repaints and reflows whenever possible.
Reflows and repaints while playing content is quite costly to overall performance and battery life. As a result, we batch reads and writes to the DOM wherever possible. This helps us avoid accidental reflows.
Take advantage of getBoundingClientRect to determine the size of object.
This is a very fast way to get the dimensions of an object. However, it isn’t a free operation and results should be cached whenever possible.
Caching the size of objects when dragging, instead of recalculating them every time they are needed, is one such way to reduce the number of calls in quick succession.
function setupPointerData(e) {
pointerEventData.dimensions = {
handleEl: handleEl.getBoundingClientRect(),
wrapperEl: wrapperEl.getBoundingClientRect()
};
pointerEventData.drag = {
start: { value: currentValue, max: currentMax },
pointer: { x: e.pageX, y: e.pageY }
};
}
function pointerDownHandler(e) {
if (handleEl.contains(e.target)) {
if (!dragging) {
setupPointerData(e);
dragging = true;
}
}
}
function pointerMoveHandler(e) {
if (dragging && isValidEventLocation(e)) {
if (!pointerEventData || !pointerEventData.dimensions) {
setupPointerData(e);
}
// Use the handleEl dimensions, wrapperEl dimensions,
// and the event values to change the DOM.
}
}
We have a lot of work planned
We’re working on exciting new features and constantly improving our HTML5 Video UI, and we’re looking for help. Our growing team is looking for experts to join us. If you’d like to apply, take a look here.
