If you recall, in Part 1 of this series, I discussed what StageVideo is, and described some of the remarkable performance improvements that can be found by working with StageVideo. For the 2nd installment of the StageVideo primer, I thought it would be helpful to describe, from a high level, how StageVideo actually works.

In other words, how does StageVideo achieve such enormous performance improvements over the traditional method of hardware acceleration.

Well, the performance improvements stem from two broad areas: trimming overhead, and hardware acceleration.

Trimming Overhead

First, on trimming overhead, a traditional Video object in Flash shares many traits with any other DisplayObject in Flash (such as MovieClip and Sprite objects). That is, Video objects can be moved, faded, transformed in size, color and space, can be transparent, etc. All of the awesomely expressive things that we can do with graphics in Flash, we can also do with Video objects.

The thing is, maintaining that degree of creative flexibility takes a lot of power, because Flash must calculate and render every frame — and this is key — even if you aren't doing anything particularly fancy with your video.

Combine this with the fact that while such creativity is awesome — it is also incredibly rare when we consider video. The vast majority of video on the web isn't like the Whiskas Virtual Cat that we built, — that is, a transparent, or otherwise visually expressive presentation of video — but is instead simply a rectangle that should ideally show as high quality an image as possible, and refresh as quickly as possible.

So, one way in which StageVideo saves on performance, is by getting rid of all that creative flexibility that is inherent with DisplayObjects in Flash. (And, if you are a coder, you'll note when working with StageVideo that it does not inherit from the DisplayObject class.)

Hardware Acceleration

Overhead savings is just one aspect in which StageVideo improves performance. The other is hardware acceleration.

If you do not know, hardware acceleration is a fancy way of saying that Flash is asking the viewer's graphics processing unit (or GPU, aka your graphics card) to help take some of the rendering burden off of the central processing unit (or CPU, aka your processor).

As I touched on in Part 1, there are actually four different levels of hardware accelerated video playback — from none, to full. (Note, that using StageVideo is only part of the equation; in order to maximize the benefits from hardware acceleration, you will also want to utilize H.264 video; more on that in a later post).

In order for a video to end up on the viewer's screen, two things need to happen: first, the video must be decoded (have the 0s and 1s turned into images) and then those images need to be rendered (or 'composited', drawn to the screen). Depending on your code and the video codec you are using, your GPU can actually take on one or both burdens, significantly freeing CPU resources.

We see here two examples, side-by-side. On the left, Flash Player is asking the GPU to help decode the video — that is, to turn the 0s and 1s in the video file, into actual images. Then the GPU sends that picture back to Flash, which composites those images, along with all the other Flash content, and sends it to the screen.

On the right, in contrast, the GPU doesn't send the image data back to Flash — instead, the GPU sends that image directly to the screen in a process often referred to as 'blitting'. So now Flash doesn't need to expend ANY effort to get that video image on the screen.

Because of how the GPU sends the image data to the display, StageVideo content (like Stage3D content, too, if you were wondering) renders below the rest of your Flash content. Meaning, for example, that you can not overlay your StageVideo on top of a MovieClip.

That depth management issue is one of the limitations of working with StageVideo. I will cover that, and other such limitations, in a subsequent post.

For now, though, hopefully you have a better idea of how StageVideo actually works to achieve such impressive performance improvements.

Share and enjoy!

-r