Over the next few months, you’ll hear about replacing Adobe Flash or Microsoft Silverlight with HTML5-based streaming video delivery, both in the trade press and perhaps from your customers, especially in distance learning, media ministry, and remote collaboration applications. The impetus is the adaption of two HTML5 extensions called the Media Source Extensions (MSE) and Encrypted Media Extensions (EME), which bring HTML5 video playback to relative feature parity with Flash for the first time.
Where your clients will need adaptive and live streams across multiple platforms and devices, with the ability to scale to conditions, MSE will factor. Where digital rights management is an issue—as with media ministries and potentially for-profit education—EME will be relevant.
Although support for MSE and EME isn’t yet universal, those beginning new streaming initiatives should consider these technologies immediately, potentially with “Flash fallback.” Many with existing streaming infrastructures will want to consider switching over sometime in the next 18 to 36 months. This summer’s announcements from Google and Mozilla that they will no longer support Flash (Google began blocking Flash ads Sept. 1) should only add to the urgency to help your clients move away from a fading technology and get up to speed on what’s modern.
Furthermore, it’s not just that Flash/Silverlight/QuickTime are finally headed to long overdue retirement; use cases for streaming are changing in ways that require new underlying technology. As the need grows for streaming across multiple devices with different aspect ratios and bandwidth infrastructure, knowledge of MSE/EME will become a crucial skill for interacting with many of your clients.
With that in mind, here’s a quick 101 to get you started.
What are MSE and EME?
The main benefit of HTML5-based video delivery is that plugins like Flash, Silverlight, and QuickTime are no longer needed. However, the initial iteration of HTML5 video was very basic; single file only, no live streaming, no captioning, and no provision for digital rights management (DRM). Meanwhile, as mobile video became more important, most streaming producers wanted adaptive streaming, rather than single file, so they could deliver high-quality streams to computers on LANs while still supporting mobile with multiple streams that could adapt to changing connection conditions. This made HTML5 delivery appropriate only for very simple projects.
The Media Source Extensions (MSE) allows browsers to deliver adaptive streams, as well as live streams. The Encrypted Media Extensions (EME) allow browsers to communicate directly with licensing servers of digital rights management technologies like Microsoft PlayReady, Google Widevine, and Adobe Primetime DRM (formerly Access), once again taking plugins out of the picture.
How does MSE work?
To operate, MSE needs the video files encoded into a format called DASH, which stands for Dynamic Adaptive Streaming over HTTP. Existing formats like Flash, HTTP Live Streaming (HLS) or Smooth Streaming won’t work with the basic, browser-based players, though some off-the-shelf players like JW Player can make these formats work. Once the files are in DASH format, operation is straightforward; any browser that supports MSE should be able to play the files.
Browser support is obviously key and there are significant gaps. Google has supported MSE/EME for multiple Chrome versions on Windows, Mac, and Android-based browsers. Microsoft supports MSE/EME in Internet Explorer 11, but only on computers running Windows 8, which is only about 11 percent of all Windows computers. Similarly, Apple only supports MSE/EME on Safari 8, which is only available for Yosemite (OS 10.10), though about 50 percent of all Macs are running Yosemite today.
Firefox was the last major browser to support MSE, starting with limited support in version 37. The biggest gap is on the iOS platform, which doesn’t currently support MSE, and Apple hasn’t announced that they will support MSE.
How does EME work?
EME is more challenging. Existing browser support is described above, and there are significant short-term gaps. In addition, each browser and device will support only a single Digital Rights Management (DRM) technology; Google supports Widevine in Chrome and Android, Microsoft supports PlayReady in Windows, Apple supports FairPlay in Safari, and Mozilla will support Adobe Primetime DRM in Firefox. Streaming producers wishing to protect their videos with EME-based DRM will either have to license each technology or support a limited number of platforms.
Adding multiple DRMs to the encoded delivery package won’t be a problem; that’s handled in the specification. In addition, many third-party DRM suppliers like BuyDRM and EZDRM now support multiple DRMs, which will make it simpler for distributors to support multiple DRMs and playback platforms.
That said, though Apple has licensed FairPlay to Netflix and HBO, the company hasn’t announced that they will license FairPlay to other smaller distributors. So it’s unclear if smaller producers will be able to distribute protected video for EME-based playback on Safari. To be clear, this shouldn’t be a problem for non-protected video, but may be for those requiring encryption on their videos.
Why switch to MSE/EME?
Google has eliminated support for Silverlight, and has curtailed the use of Flash advertisements from the Chrome browser, forcing those relying on those plugins to choose a new technology. Beyond mobile, many streaming producers are also seeking playback on the growing range of gaming consoles, streaming devices, and Smart TVs. Many of these platforms play DASH-encoded video, so supporting MSE and DASH may enable producers to reach new video distribution platforms. Certainly, Flash and Silverlight are going away as platforms, so any assets that you encode specifically for those formats might have to be re-encoded for MSE/EME down the road.
In addition, there’s a general perception that HTML5-based video loads faster, plays more smoothly, and consumes less CPU and battery than Flash and Silverlight. Finally, many viewers find plugins like Flash and Silverlight obtrusive and objectionable.
How can I implement MSE/EME with so many gaps in support?
There are multiple third-party developers offering MSE/EME players, including bitmovin bitdash, castLabs DASH Everywhere, JW Player, and Viblast Player, as well as two open-source players, video.js and DASH.js, that also support DASH playback. So you won’t have to develop your MSE/EME playback capabilities from scratch.
Most of these players enable Fallback to Flash or Silverlight on browsers that are not yet MSE/EME compatible, using the same set of DASH encoded files. On Android, these players can also fall back to HLS or single file progressive delivery. So, you may be able to create one set of DASH encoded files and support current and legacy platforms.
Who should consider MSE/EME?
Those starting new streaming projects should definitely consider MSE/EME. Those with existing streaming setups that are advertising-supported should check to make sure their ad servers can support MSE delivery before considering a switch. If your video requires DRM, it’s another red flag, since EME support trails MSE support by a few months on several platforms.
Simple video-on-demand applications will be easier to convert to HTML5, while live applications typically involve a lot of infrastructure that may not be affordably upgradable. Developers creating applications with lots of interactivity and features beyond simple playback should check to be sure that they can duplicate their existing functionality via HTML5-based controls.
Beyond that, those currently streaming should consider switching if a significant percentage of their viewers use the Chrome platform, since it’s MSE/EME compatible today. Conversely, those predominantly serving older Windows-based computers running Internet Explorer can wait, since these platforms will be around for awhile, and likely will never upgrade.
In any relatively new organization with tech-savvy users—whether school, corporation, or church, there may be significant pressure to move away from Flash. For older, non-computer technology businesses with lots of older computers, transitioning to HTML5 may not be that immediate a priority.
