This summer, sports fans around the world hit pause on their daily lives to tune into the 2014 World Cup Brazil™. While the trip to Brazil was not possible for many, the opportunity to view at home (or the office!) and consume complementary content on the second screen was a popular choice.

Viewers cheering on via computers and connected devices were offered a new perspective this year by being able to review multiple camera angles as the games were being played. To provide this multi-screen action review and a wide range of other second screen content to viewers from afar, EVS, a global provider of live video production systems, leveraged Brightcove’s Zencoder cloud-based encoding service to power transcoding for its C-Cast content delivery solution.  C-Cast is a multimedia distribution platform that allows broadcasters to instantly deliver complementary content to viewers on web and mobile devices during live events. Host Broadcasting Services (HBS), the main provider of broadcast and interactive services for the 2014 FIFA World Cup, managed the implementation and integration of C-Cast to deploy second-screen experiences.

“We took second screen content to the next level by allowing broadcasters to deliver multi-camera action review, key highlights browsing and review, stats-related content and player, team and personality tracking,” said Nicolas Bourdon, SVP Marketing at EVS. “This completely enhanced the at-home viewing experience and judging by the stats, viewers were pleased and eager to view the World Cup this way.”

Reaching the magnitude of viewers that the World Cup attracts with a high-quality viewing experience is no easy feat. With the games over and the numbers crunched, we’re happy to share some stats directly from Zencoder on the scale of video that powered the World Cup on the second screen:

Video on the web continues to grow by leaps and bounds. If you want a cheap-but-good analogy, it’s about a tween: it already knows a lot, but still has substantial growth ahead. It’s becoming too cool for plugins like Flash and Silverlight, but a lot of the shiny and exciting APIs on the edge of the spec still need to mature a little.

Last year at WebExpo I gave a talk about the future of video on the web, as well as a brief foray into new and exciting features that were on the way. Some of the things I talked about, like Encrypted Media Extensions, were, for the most part, still a pipe dream with few to no working implementations. Now, just a year later, nearly all of the things discussed in that talk have meaningful implementations in most modern browsers.

I recently gave a talk at Developer-Week about how the future of video on the web has arrived. During the talk we walked through real world examples of each “future” feature and, amazingly, talked about how each of the features could be (cautiously) used in the wild today.

One of those features was WebRTC and using the APIs individually. We built a simple example of using getUserMedia to request a user’s webcam and display it in a video element. To take this a step further, let’s take that example and use it to save, then transcode content directly from the browser.

Simple Example

Before we start on taking things further, let’s take a look at the initial, simpler example I showed during the presentation. All we’ll do here is request a user’s video stream, and show that in a video element on the page. We’ll be using jQuery for the more advanced example, so we’ll go ahead and start using it here.

// Do the vendor prefix dance
navigator.getUserMedia  = navigator.getUserMedia    || navigator.webkitGetUserMedia ||
                          navigator.mozGetUserMedia || navigator.msGetUserMedia;

// Set up an error handler on the callback
var errCallback = function(e) {
  console.log('Did you just reject me?!', e);
};

// Request the user's media
function requestMedia(e) {
  e.preventDefault();

// Use the vendor prefixed getUserMedia we set up above and request just video
  navigator.getUserMedia({video: true, audio: false}, showMedia, errCallback);
}

// Actually show the media
function showMedia(stream) {
  var video = document.getElementById('user-media');
  video.src = window.URL.createObjectURL(stream);

video.onloadedmetadata = function(e) {
    console.log('Locked and loaded.');
  };
}

// Set up a click handler to kick off the process
$(function() {
  $('#get-user-media').click(requestMedia);
});

Now we just need the button and the video element and we’re ready to go.

<p><a id="get-user-media" class="btn btn-primary">Get Media!</a></p>

<video id="user-media" autoplay height="300"></video>

After clicking the “Get Media!” button and allowing the browser access to your camera, the end result should look something like this:

This demo should work Firefox, Chrome, or Opera. Live example!

Now you have access to the webcam through the browser! This example is fun, but pretty useless since all we can do is show someone themselves (mirror app?).

Setup

Note – Before we get started, I need to be clear that the example we’re building only works in Firefox. I’ll go into this more later, but the tl;dr is that Firefox is the only one that’s implemented the MediaRecorder API at this point. If you want to make this work in Chrome as well, there are projects such as RecordRTC and MediaStreamRecorder.

We need a dead simple server side component for this example, but it only needs to do two things:
1. Return a valid AWS policy so we can upload directly from their browser.
2. Submit an encoding job to Zencoder.

I like to use the Express framework for Node for examples like this, but if you’re more comfortable using something else, like Sinatra, feel free to ignore this example and use whatever you’d like. Since we’re more concerned about the client side code, I’m not going to dig into the server side implementation, but feel free to comment if you have questions.

var S3_BUCKET = 'YOUR-S3-BUCKET-NAME';

<p>var express    = require('express');
var path       = require('path');
var logger     = require('morgan');
var bodyParser = require('body-parser');
var crypto     = require('crypto');
var moment     = require('moment');
var AWS        = require('aws-sdk');
var s3         = new AWS.S3({ params: { Bucket: S3_BUCKET }});
var zencoder   = require('zencoder')();

var app = express();

app.set('port', process.env.PORT || 3000);
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded());
app.use(express.static(path.join(__dirname, 'public')));

app.post('/process', function(req, res) {
  // Build up the S3 URL based on the specified S3 Bucket and filename included
  // in the POST request body.
  var input = 'https://'+S3_BUCKET+'.s3.amazonaws.com/'+req.body.filename;
  createJob(input, req.body.email, function(err, data) {
    if (err) { return res.send(500, err); }

    res.send(200, data);

  });
});

app.post('/upload', function(req, res) {
  var cors = createS3Policy();
  res.send(201, { url: 'https://'+S3_BUCKET+'.s3.amazonaws.com/', cors: cors });
});

function createS3Policy() {
  var policy = {
    "expiration": moment().utc().add('days', 1).toISOString(),
    "conditions": [
      { "bucket": S3_BUCKET },
      { "acl":"private" },
      [ "starts-with", "$key", "" ],
      [ "starts-with", "$Content-Type", "" ],
      [ "content-length-range", 0, 5368709120 ]
    ]
  };

  var base64Policy = new Buffer(JSON.stringify(policy)).toString('base64');
  var signature = crypto.createHmac('sha1', AWS.config.credentials.secretAccessKey).update(base64Policy).digest('base64');

  return {
    key: AWS.config.credentials.accessKeyId,
    policy: base64Policy,
    signature: signature
  };
}

function createJob(input, email, cb) {
  var watermark = {
    url: 'https://s3.amazonaws.com/zencoder-demo/blog-posts/videobooth.png',
    x: '-0',
    y: '-0',
    width: '30%'
  };

  zencoder.Job.create({
    input: input,
    notifications: [ email ],
    outputs: [
      { format: 'mp4', watermarks: [watermark] },
      { format: 'webm', watermarks: [watermark] }
    ]
  }, cb);
}

var server = app.listen(app.get('port'), function() {
  console.log('Express server listening on port ' + server.address().port);
});

Just a note, this example should mostly work out of the box, but you’ll need to have AWS configurations already [set up](http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/node-configuring.html), as well as aZENCODERAPIKEY` environment variable. You’ll also need to have CORS configured on the bucket you use. Here’s an example CORS configuration that will work:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>POST</AllowedMethod>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

Recording user media

In the simple example above, we requested a user’s media using the getUserMedia API, so now we need a way to record that content. Luckily, there’s a shiny new API called, appropriately, MediaRecorder. Firefox is the only browser that currently supports it (as of version 25), but there are projects like Whammy that can act as a pseudo-shim for other browsers.

The API is simple, we just need to take the same stream we used for playback in the simple example, and use it to create a new instance of MediaRecorder. Once we have our new recorder, all we have to do is call start() to begin recording, and stop() to, you guessed it, stop.

var recorder = new MediaRecorder(this.stream);
recorder.start(); // You're now recording!
// ...A few seconds later...
recorder.stop();

Getting the recorded media

Ok great, we started and stopped a webcam recording…how do we see it?

You can listen for the ondataavailable event on the instance of MediaRecorder we created to record. When it’s done, it will include a new Blob that you can play back just like you did the original user media.

// We'll keep using the same recorder
recorder.ondataavailable = function(e) {
  var videoBlob = new Blob([e.data], { type: e.data.type });
  var player = document.getElementById('playback-video-el');
  var blobUrl = URL.createObjectURL(videoBlob);
  player.src = blobUrl;
  player.play();
}

If you’ve been following along and building out these examples, right about now you’re probably trying to replay the video and getting frustrated. Sadly, nothing you do “right” is going to work here…autoplay on the video element nor calling play() or setting currentTimeon the ended event is going to do what you want.

This seems to simply be a Firefox issue with playing back these blobs. The ugly, but functional workaround is to simply replace the source on the ended event if you want the video to loop.

player.onended = function() {
  video.pause();
  video.src = blobUrl;
  video.play();
}

This blob you have is a (mostly) functional WebM video! If you create an anchor tag with this blob url as the source, you can right click and save the file locally. However, even locally this file doesn’t seem behave quite right (OS X seems to think it’s an HTML file). Besides, if you wanted, say, an h264 version…

This is where Zencoder fits nicely into the picture. Before we can process it, we need to get the file online so Zencoder can access it. We’ll use one of the API endpoints we created earlier, /upload to grab a signed policy, then use that to POST the file directly to S3 (I’m using jQuery in this example).

function uploadVideo(video) {
  $.post('/upload', { key: "myawesomerecording.webm" }).done(function(data) {
    // The API endpoint we created returns a URL, plus a cors object with a key, policy, and signature.
    formUpload(data.url, data.cors.key, data.cors.policy, data.cors.signature, filename, recording);
  });

  function formUpload(url, accessKey, policy, signature, filename, video) {
    var fd = new FormData();</p>

    fd.append('key', filename);
    fd.append('AWSAccessKeyId', accessKey);
    fd.append('acl', 'private');
    fd.append('policy', policy);
    fd.append('signature', signature);
    fd.append('Content-Type', "video/webm");
    fd.append("file",  video);

    $.ajax({
      type: 'POST',
      url: url,
      data: fd,
      processData: false,
      contentType: false
    }).done(function(data) {
      cb(null);
    }).fail(function(jqxhr, status, err) {
      cb(err);
    });
  }
}

uploadVideo(videoBlob);

Now you’ve got a video on an S3 bucket, so all we have to do is actually process it. If you noticed, we added an email to the /process endpoint earlier so we can get the job notification (including download links for the video) sent directly to us when it’s done.

function process(email, filename) {
  $.post('/process', {
    filename: filename,
    email: email
  }).done(function(data) {
    console.log('All done! you should get an email soon.');
  }).fail(function(jqXHR, error, data) {
    console.log('Awww...sad...something went wrong');
  });
};

process('mmcclure@brightcove.com', "myawesomerecording.webm");

A few seconds later you should get an email congratulating you for your brand new, browser-recorded videos. The links included are temporary, so make sure you download them within 24 hours or change the API endpoint we created to upload the outputs to a bucket you own.

I created a demo to showcase this functionality, including some minor styling and a not-so-fancy interface. I’ve called it VideoBooth, but feel free to clone the project and run with it! You can also play with the working demo on Heroku.

During a live event, however, we don’t have the luxury of simply trying again. Since these events are transcoded in real time, we only have one chance to deliver a perfect transcode to end-users; a blip anywhere along the way can mean an interruption. No matter how reliable the system is, you never know when something like a power supply or network card is going to fail, so redundancy is crucial for high profile events.

To account for this, we recently announced redundant transcoding for live streams. In a nutshell, if redundant_transcode is enabled, and at least one secondary_url is specified, Zencoder will transparently create a new job using any secondary URLs specified in the outputs of the original request. This job has all the same settings of the original, with the important distinction of being run in the nearest transcoding region of an entirely different cloud provider.

Let’s look at an example:

{
  "redundant_transcode": true,
  "live_stream": true,
  "region": "us-virginia",
  "output": [
    {
      "label": "super-important-stream",
      "url": "rtmp://primary.example.com/live/stream",
      "secondary_url": "rtmp://backup.example.com/live/stream",
      "live_stream": true
    },
    {
      "label": "not-as-important-stream",
      "url": "rtmp://primary.example.com/live/stream",
      "live_stream": true
    }
  ]
}

This will return one stream name, but 2 stream URLs, one primary and one redundant.

{
  "stream_name": "as230d982389askdfsdkjf2380ejd93d93dj",
  "outputs": [
    {
      "label": "super-important-stream",
      "url": "rtmp://primary.example.com/live/stream",
      "id": 260281679
    },
    {
      "label": "not-as-important-stream",
      "url": "rtmp://primary.example.com/live/stream",
      "id": 260281680
    }
  ],
  "stream_url": "rtmp://live01.us-va.zencoder.io:1935/live",
  "redundant_job_id": 12345678,
  "redundant_stream_url": "rtmp://backup-endpoint.zencoder.io:1935/live",
  "id": 98091238
}

You can then set up your encoder to stream to both simultaneously. The example below is using Flash Media Live Encoder, but most encoders support a primary and backup stream url with little additional setup.

With this setup, if Amazon East were to go down (which is unlikely, but not impossible), the backup stream would continue happily on Google Compute Engine without any problems. Assuming you’re using a CDN with proper backup urls, such as Akamai, the playback URLs would continue to work as if nothing happened.

For simplicity sake, we only discussed adding a backup URL to a single encoder. This would mitigate any issues downstream of the encoder (like Amazon East getting hit by a meteor), but the stream is still at risk if the encoder is unable to publish for any reason. The backup URL, however, could be set up on a separate encoder, which could be on an entirely different network to be as safe as possible.

Notes

• Since a stream needs to be pushed to both the primary and backup endpoint, this will double the bandwidth necessary to publish a stream.
• The backup job is billed as a separate job at standard live rates.
• Akamai will attempt to align the primary and backup streams before playback, so there may be a short interruption during the transition between streams.

Here is a detailed description of what happened, why, and why it will never happen again.

What happened

The Zencoder stack relies on a PostgreSQL database, among other things. We believe PostgreSQL to be an excellent database – fast, reliable, scalable, and well-designed. Many services rely upon PostgreSQL, and we have run it successfully since Zencoder’s inception.

For reasons we are still investigating, an internal PostgreSQL database process (autovacuum) stalled while running on a single table in our database. This ultimately caused the database to stop accepting new transactions, effectively putting it in “read-only” mode. At this time, we believe the underlying problem may have been a bug in the version of PostgreSQL we were using, but we are still verifying this.

Within minutes, engineers were alerted to the problem and started an operation that should have unfrozen the database. This process takes time, especially on a large database, but we expected that it would finish in relatively short order. Unfortunately, this process stalled, possibly due to the same issue that caused the problem in the first place.

In parallel with this, we considered failing over to a standby server. We have redundant database servers (along with redundancy across the rest of our stack), and if this were a hardware failure, we could have failed to our secondary server within minutes. Unfortunately, the issue was replicated onto the secondary database as well, and so this was not an option.

Eventually, we determined that the operation itself was not working. We decided to take a more drastic step, and stood up a new stack. Jobs started processing again by 12:26am EDT.

What we will do

As a response to this incident, we have already begun multiple layers of improvements.

First, we have upgraded to a newer version of PostgreSQL that is not susceptible to the particular bug we identified.

Second, we are working on our database configuration and monitoring to ensure that the conditions that led to the problem will not happen again.

Beyond that, we are working on indirect improvements, including faster recovery in the face of a catastrophe and additional layers of redundancy, that will minimize the impact of future problems.

(Our full response to the incident is still being determined, since we are still verifying the root cause for the operation’s failure, but we are happy to share more information with customers who are interested as we make progress in the coming days.)

We sincerely apologize for the problem and for the impact that it caused to our customers. We pride ourselves on operating reliably at very high scale, and we will work hard to make sure this never happens again.

Some of our internal servers, which are not accessible by customers, were vulnerable. We fixed this vulnerability as of 8:00PM PST on Monday, Apr 7th.

With today’s announcement, Zencoder is applying the elastic scalability of the cloud to solve encoding challenges for a new segment of the production workflow. Just as Zencoder empowered publishers to rapidly transcode video to support the fragmented Web and mobile ecosystem, the service is now enabling broadcasters and professional content providers to scale their media processing operations in the cloud.

Zencoder’s support for advanced encoding formats is critically important as the OTT and streaming landscape continues to expand. The market is disparate and demands that content providers:

• Create a video rendition for every streaming outlet
• Create files for endpoints for international subsidiaries and distribution partners
• Provide encoded libraries of content to support licensing deals

All of these initiatives can be costly when relying on on-premise encoders. Cloud encoding for these workflows alleviates cost concerns and helps publishers to better plan for capacity.

For our customers in the broadcast realm and within other professional workflows, support for H.265 (HEVC) and MPEG-DASH is particularly important. Because H.265 promises to deliver high-quality video at lower-bitrates – ideal for 4K/Ultra HD – broadcasters are hyper-focused on related support for this standard. Additionally, MPEG-DASH will serve as an international standard that will potentially unify cross-platform workflows. We are very excited to provide premium publishers with seamless encoding solutions that will simplify incredibly complex workflows.

Advanced Encoding Formats are now in beta. Contact us to learn more about participating in this beta program.

Last Tuesday was the 5th meeting, and Robert Scott, the platform engineering manager at Inkling, spoke to us about Inkling’s platform and how they handle video. He walked us through the Inkling service itself from both an editor’s and a user’s perspective, then went into the process they went through before deciding on a transcoding service. The group is usually pretty vocal, so Inkling’s use of signed Cloud Front URLs with HTTP Live Streaming led to a great discussion on the practice.

Excellent presentations from last year included J Sherwani and Faraz Khan from Screenhero, who talked about H.264 vs VP8 for low-latency video streaming, and Jon Gubman from Funny or Die, who followed up with a presentation on Funny or Die’s switch to HTML5 video from Flash and the gotchas they uncovered along the way. Robert Scott started the new year on an excellent note, so we’d like to continue the trend in the upcoming months, but the group needs your help.

This month’s meetup (February 25th) is going to focus on HTTP streaming. The talks will be a bit more of a lightning talk format, with Anton Kast (Video Architect at Yahoo) giving an overview of Dash, and myself (Matt McClure) talking about dynamically generating HLS manifests. One more talk maybe added before the meetup, so keep an eye out.

Interested in talking? Please get in touch! Anyone is welcome to the floor, it doesn’t have to be long (anywhere from 5 to 10 minutes), and can consist of anything from technical demos to presentations (or both). The general rule of thumb is that it should be interesting to engineers that work with video, but other than that we’re open to any proposals.

If you’re interested in coming, please join the Meetup so we can get in touch about updates (and make sure to buy enough food and drink). Hope to see you there!