310.684.3047

StageVideo Code & Slides from Adobe MAX 2011

Now that Adobe MAX 2011 is over, I wanted to post the slides and sample code from my session.

The talk was called 'Delivering the Best Flash HD Video', since that was really the main point of the session. But, at a technical level, the talk covered StageVideo, which was introduced earlier this year in Flash Player 10.2.

I'm including the slides mostly for the benefit of the people who attended the session — but you are more than welcome to try to work your way through them on your own.

In them, I introduce StageVideo and demonstrate the massive performance improvements that you can get by employing these new video display options. I explain what hardware acceleration actually means, and how it works for video. I cover why H.264 is StageVideo's best friend, and how to work with it. I address the new ActionScript 3 APIs for StageVideo (with the demos, below) and also explain how OSMF (as well as Strobe Media Playback and Flash Media Playback) allow you to work with StageVideo, without knowing or using any custom code at all.

The sample ActionScript 3 code, below, will hopefully be of additional value to many of you who were not there. This is the 'more complex' demonstration piece from the talk — the one that includes the proper sizing of the video, as well as pan and zoom functionality. This supplements the simple StageVideo sample code, the first demo from the talk, which I posted last week.

When the talk is live on Adobe TV, I will post a link to it. Until then, hopefully you can work your through this fully-commented code file.

This demonstration code is intended to be set as the document class for a Flash CS5.5 FLA — just change the name of the video you want to play.

Share and enjoy!

-r

package
{
 
	/**
	 *	This class illustrates a slightly more complicate StageVideo implementation
	 *	Including Pan and Zoom controls
	 *	And information about the rendering state of the StageVideo instance
	 *
	 *	@langversion ActionScript 3.0
	 *	@playerversion Flash 10.2
	 *
	 *	@author R Blank
	 *	@since  09.09.2011
	 */
 
	//import the MovieClip, because this class is a MovieClip
	import flash.display.MovieClip ;
	//import the Rectangle, so that we can size and position the StageVideo instance
	import flash.geom.Rectangle ;
	//import Point so that we can pan and zoom our stage video
	import flash.geom.Point ;
	//import StageVideoAvailabilityEvent so we can hear 
	//when StageVideo availability changes
	import flash.events.StageVideoAvailabilityEvent;
	//StageVideoEvent provides information to us about video rendering
	import flash.events.StageVideoEvent;
	//import StageVideo so we can use StageVideo
	import flash.media.StageVideo;
	//import StageVideoAvailability to get the values for the
	//StageVideoAvailabilityEvent.available property
	import flash.media.StageVideoAvailability;
	//import Video as a backup for when StageVideo is not available
	import flash.media.Video;
	//import the NetConnection and NetStream to play our video file
	import flash.net.NetConnection;
	import flash.net.NetStream;
	//import the TextField so we can log values visibly 
	import flash.text.TextField ;
	//KeyboardEvent for keyboard controls
	import flash.events.KeyboardEvent ;
	//Keyboard class provides keyboard key values
	import flash.ui.Keyboard ;
 
	public class FullStageVideoExample extends MovieClip {
 
		//--------------------------------------
		// CLASS CONSTANTS
		//--------------------------------------
 
		//the location of our video file
		private const _videoURL : String = "../_media/bunny.mov" ;
		//the value which we will use to iterate pan and zoom controls
		private const _zoomAndPanVal : Number = .05 ;
 
		//--------------------------------------
		//  CONSTRUCTOR
		//--------------------------------------
		public function FullStageVideoExample ( ) {
			//call the _init function to startup
			_init () ;
		}
 
		//--------------------------------------
		//  PRIVATE VARIABLES
		//--------------------------------------
 
		//the Video object to play video when StageVideo is not available
		private var _video : Video ;		
		//the StageVideo object to play fully accelerated video
		private var _stageVideo : StageVideo ;
		//the NetStream and NetConnection objects to play video
		private var _ns : NetStream ;
		private var _nc : NetConnection ;
		//the TextField object to log informationvisibly in the browser
		private var _tf : TextField ;
		//_panX stores the current stage video horizontal pan value
		private var _panX : Number ;
		//_panY stores the current stage video vertical pan value
		private var _panY : Number ;
		//_zoom stores the current stage video zoom value
		private var _zoom : Number ;
 
		//--------------------------------------
		//  PRIVATE & PROTECTED INSTANCE METHODS
		//--------------------------------------
 
		//called by the constructor to startup the player
		private function _init ( ) : void
		{
			//create the video object as back up, in case StageVideo is not available
			_video = new Video ( ) ;
			//create the TextField for logging
			_tf = new TextField ( );
			_tf.height = stage.stageHeight ;
			_tf.width = stage.stageWidth ;
			_tf.mouseWheelEnabled = true ; 
			_tf.multiline = true ; 
			_tf.wordWrap = true ; 
			addChild ( _tf ) ;
			//the NetConnection object over which video is delivered
			_nc = new NetConnection ( ) ;
			//connect _nc to null because we are playing video progressively
			_nc.connect ( null ) ;
			//create the NetStream to deliver video over the NetConnection
			_ns = new NetStream ( _nc ) ;
			//define the NetStream client as 'this' (meaning this class)
			//where _ns will look for the onMetaData and onCuePoint methods
			_ns.client = this ;
			//tell the NetStream to play our video file
			_ns.play ( _videoURL ) ;
			//listen for the StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY event
			//this is dispatched as soon as the event listener is added
			//and when stage video availability changes during SWF playback
			stage.addEventListener(StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY, _onStageVideoAvailability);
 
		}
 
		//called when we wish to start StageVideo playback
		private function _enableStageVideo() : void {
			//if the _video object is in the display list
			if ( _video.parent )
				//remove _video from the display list
				removeChild ( _video ) ;
			//if we do not have a StageVideo object reference 
			//already stored in _stageVideo
			if ( _stageVideo == null ) 
			{
				//set _stageVideo to reference the first of our
				//available StageVideo objects (up to 8 available on desktops)
				_stageVideo = stage.stageVideos [ 0 ] ;
				//listen for the StageVideoEvent.RENDER_STATE
				//which provides us information about how the video is being rendered
				//this is fired when the netstream is attached to the StageVideo instance
				_stageVideo.addEventListener(StageVideoEvent.RENDER_STATE, _onStageVideoStateChange );
				//log the available color spaces
				_tf.appendText ( "_stageVideo.colorSpaces : " + _stageVideo.colorSpaces.toString() + "\n");
 
			}
			//attach our NetStream instance to our StageVideo instance
			_stageVideo.attachNetStream ( _ns ) ;
			//listen for key strokes (which we use for pan and zoom controls)
			stage.addEventListener ( KeyboardEvent.KEY_DOWN , _onKeyDown ) ;
			//reset pan values to 0 (center)
			_panX = 0 ;
			_panY = 0 ;
			//reset zoom value to 1 (actual size)
			_zoom = 1 ;
		}
 
		//called when StageVideo becomes unavailable
		//and we need to use a Video object as back-up
		private function _disableStageVideo() : void {
			stage.removeEventListener ( KeyboardEvent.KEY_DOWN , _onKeyDown ) ;
			//attach our NetStream to our Video object
			_video.attachNetStream ( _ns ) ;
			//add the Video object to the display list
			//(so that it can be seen)
			addChild ( _video ) ;
			//add the _tf TextField back to the display list
			//so that it is on top of the _video object
			addChild ( _tf ) ;
		}
 
 
		//called to change the zoom factor on the StageVideo instance
		//called with true to zoom in
		//            false to zoom out
		private function _zoomIn ( zoomIn : Boolean ) : void
		{
			//if we are zooming in
			if ( zoomIn )
			{
				//if our current zoom factor is less than 16 (the max)
				if ( _zoom < 16 )
				{
					//increment our zoom factor (with a ceiling of 16)
					_zoom = Math.min ( 16 , _zoom + _zoomAndPanVal ) ;
					//set the zoom factor on the StageVideo instance to our _zoom value
					_stageVideo.zoom = new Point ( _zoom , _zoom ) ;
				}
			//otherwise, we are zooming out
			} else {
				//if our current zoom factor is greater than 1 (the minimum)
				if ( _zoom > 1 )
				{
					//decrement our zoom factor (with a floor of 1)
					_zoom = Math.max ( 1 , _zoom - _zoomAndPanVal ) ; 
					//update the zoom of the StageVideo to our zoom factor
					_stageVideo.zoom = new Point ( _zoom , _zoom ) ;
				}
			}
			//log the zoom factor
			_tf.appendText ( "zoomed to " + _zoom + "\n") ;
		}
 
		//called to pan our stage video (which only matters if zoom > 1 )
		//called with "up" to pan up
		//            "down" to pan down
		//            "left" to pan left
		//            "right" to pan right
		private function _pan ( direction : String ) : void
		{
			//if the zoom factor is greater than 1
			if ( _zoom > 1 ) 
			{
				//switch, based on the 'direction' parameter
				switch ( direction )
				{
					case "up" :
						//if _panY is greater than the minimum value
						if ( _panY > -1 )
						{
							//decrement _panY (with a floor of -1)
							_panY = Math.max ( -1 , _panY - _zoomAndPanVal ) ;
						}
						break ;
					case "down" :
						//if _panY is less than the max value
						if ( _panY < 1 )
						{
							//increment _panY (with a ceiling of 1)
							_panY = Math.min ( 1 , _panY + _zoomAndPanVal ) ;
						}
						break ;
					case "left" :
						//if _panX is greater than the minimum value
						if ( _panX > -1 )
						{
							//decrement _panX (with a floor of -1)
							_panX = Math.max ( -1 , _panX - _zoomAndPanVal ) ;
						}
						break ;
					case "right" :
						//if _panX is less than the max value
						if ( _panX < 1 )
						{
							//increment _panX (with a ceiling of 1)
							_panX = Math.min ( 1 , _panX + _zoomAndPanVal ) ;
						}
						break ;
				}
				//log the pan
				_tf.appendText ( "Panning to _panX: " + _panX + " , _panY: " + _panY + "\n" );
				//pan the StageVideo
				_stageVideo.pan = new Point ( _panX , _panY ) ;
			}
		}
 
		//called to reset the pan and zoom on the stage video
		private function _resetPanAndZoom ( ) : void
		{
			//set the pan values to 0
			_panX = 0 ;
			_panY = 0 ;
			//set the zoom factor to 1
			_zoom = 1 ;
			//update the pan value on the StageVideo
			_stageVideo.pan = new Point ( _panX , _panY ) ;
			//update the zoom value on the StageVideo
			_stageVideo.zoom = new Point ( _zoom , _zoom ) ;
		}
 
		//--------------------------------------
		//  Event Handlers
		//--------------------------------------
 
		//this serves as the callback when keys are pressed down
		private function _onKeyDown ( evt : KeyboardEvent ) : void
		{
			//switch based on the code of the pressed key
			switch ( evt.keyCode )
			{
				//if the enter key is pressed
				case Keyboard.ENTER :
					//call to reset the StageVideo display
					_resetPanAndZoom ( ) ;
					break ;
				//the F key
				case 70 :
					//enter full screen playback ('esc' will implicitly exit full screen)
					stage.displayState = "fullScreen" ;
					break ;
				//the 'space' key
				case 32 :
					_tf.text = ""; 
					break ;
				//the +/= key
				case 187 :
					//call to zoom in
					_zoomIn ( true ) ;
					break ;
				//the -/_ key
				case 189 :
					//call to zoom out
					_zoomIn ( false ) ;
					break;
				case Keyboard.UP :
					//call to pan up
					_pan ( "up" ) ;
					break ;
				case Keyboard.DOWN :
					//call to pan down
					_pan ( "down" ) ;
					break;
				case Keyboard.LEFT :
					//call to pan left
					_pan ( "left" ) ;
					break ;
				case Keyboard.RIGHT :
					//call to pan right
					_pan ( "right" ) ;
					break ;
			}
		}
 
		//--------------------------------------
		//  StageVideo Event Handlers
		//--------------------------------------
 
		//our callback function for the
		//StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY event
		private function _onStageVideoAvailability ( evt : StageVideoAvailabilityEvent ) : void {
			//log the availability of StageVideo
			_tf .appendText ( "StageVideo has become " + evt.availability + "\n" );
			//if StageVideo is available
			if ( evt.availability )
				//call _enableStageVideo
				_enableStageVideo ( ) ;
			else 
				//otherwise call _disableStageVideo
				_disableStageVideo ( ) ;
		}
 
		private function _onStageVideoStateChange ( evt : StageVideoEvent ) : void {
			//log the rendering method
			_tf .appendText ( "StageVideo is rendering with " + evt.status + "\n" );
			//size and position our _stageVideo with a new Rectangle
			//sized the native width and height of the video we are playing
			_stageVideo.viewPort = new Rectangle ( 0 , 0 , _stageVideo.videoWidth , _stageVideo.videoHeight ) ;
		}		
		//--------------------------------------
		//  NetStream Event Handlers
		//--------------------------------------
 
		//the NetStream will look for these two methods: onMetaData + onCuePoint
		//on whatever object is identified as the NetStream's client
		//if they do not exist, you will receive a run time error
		//when the NetStream encounters metadata or cuepoints in the video
 
		//onMetaData is called when metadata is encoutered in NetStream playback
		public function onMetaData( info:Object ) : void 
		{ 
			//capture the native video width and height from the metadata
			//and update the width and height of the _video object
			//to match the loaded info
			if ( info.width )
				_video.width = int ( info.width ) ;
			if ( info.height )
				_video.height = int ( info.height ) ;
		}
 
		public function onCuePoint( info:Object ) : void { }
 
	}
 
}

Category: Code & Samples

Tagged: , , , ,

12 Responses

  1. [...] A brief note that the slides and code files from my talk at Adobe MAX 2011 have been posted to Almer/Blank labs. [...]

  2. jeff says:

    Hey mate,

    I set the code as a document class for my (empty) fla and changed the video to a .mov that I have but it doesnt work… No errors or anything…

    (Flash 5.5)

  3. rblank says:

    Did you test in the browser? With what version of the player?

    In my experience, StageVideo does NOT work in test-movie mode. (Annoying, right? But no different from full screen mode, in that sense.)

  4. [...] recently at Adobe MAX). I posted the commented code from that talk (the simple code demo, and the more complex code demo), but didn't go into much detail about StageVideo itself and how it [...]

  5. jeff says:

    Ah ok. I was testing it in test movie mode and no dice. But then I moved everything to my xampp install and running it from localhost, no good either. I get the StageVideo has become unavailable error message. (FP 11)

  6. rblank says:

    Hello Jeff:

    When you say 'error message', do you mean the trace output in the Flash movie itself?

    Well, that almost certainly means that StageVideo is not available on your system. However, in that case, the video object should work as fall-back. I'd like to try to help you through this, if you can give me any more information. What kind of video are you trying to play? Are you sure its in the right place? Any chance of yousendit'ing the stuff to me?

    -r

  7. jeff says:

    Yes apologies I should have been more clear, its in the textbox that says that. I am playing a mov file. I emailed you the info

  8. rblank says:

    Jeff:

    A few notes. To start, if I said I knew what was happening here, I would be very much lying.

    That said, in my initial tests, you html/swf failed to play the video. So I tested your video with my original code, and it played. Hmmm….

    I opened your HTML, and saw that WMODE was set to 'window'. It should be set to 'direct' to ensure StageVideo works on as many platforms as possible. So I updated that.

    Still though, nothing.

    Then I just dragged your SWF on to my browser, and boom — it worked. I reloaded, and it worked again. So playing the video with the SWF worked.

    And now, I can not get playback to fail, even with the HTML, even with WMODE set back to 'window'. After that, each time I test the HTML in the browser, your player plays the video.

    It's all quite weird.

    Now, here is a universal rule for testing with video: use video with sound! Otherwise, you don't know if your video isn't playing at all, or just isn't visible.

    That's a pretty big difference. But now that I can not get it to fail, I'm having a hard time figuring out what your problem actually was.

    Three things you can try:

    1) Switch to testing a video with an audio track

    2) StageVideo can not been seen unless you assign a rectangle to the viewPort property. In my code, I do that in the callback on the StageVideoEvent.RENDER_STATE event. If, for whatever reason, that event doesn't fire properly, maybe you should add a default rectangle. So on line 137, right before you listen for the StageVideoEvent.RENDER_STATE event, you can add this line:

    _stageVideo.viewPort = new Rectangle ( 0 , 0 , 320 , 240 ) ;

    This will help determine what the actual failure is.

    3) Less importantly, Call _disableStageVideo() from within the _init method, on line 116, before adding the listener to the stage for the StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY event. That might help ensure that the video object is used as fall-back. (Shouldn't make a difference, but who knows).

    Let me know how this turns out.

  9. jeff says:

    Hey mate,

    Ya strange things going on. For the HTML I justed used Flash's publish settings and used the generated HTML. (Dont do that too often, usually just swfobject but was figuring it would be ok for this demo; guess not! ;)

    The .mov does have sound but there isnt sound when played within Flash. Adding the rectangle and calling _disableStageVideo() where you said in the code allows the vid to play in Flash but then no video plays in the browser.

    I'm thinking at this point it must be something to do with my Flash IDE. Really appreciate your help! I'll keep hammering on it and see how it goes.

  10. R Blank says:

    Jeff, before spending too much time debugging, your reply made me think of one more thing.

    When you said that the video does have sound, I went back and checked the codec. Turns out your video file uses Linear PCM for the audio codec. Flash doesn't understand that — only mp3 (generally with flv) and AAC (generally with H.264). (Also ADPCM but I've never seen that in video).

    There is a chance that the audio track is confusing how flash parses the data in the video file.

    So, go back to the original code, and try a different video file, that has AAC for the audio codec.

    Please let me know if that works.

    Thx

    -r

  11. jeff says:

    You're right. That must have been it. I got a properly encoded video and that works. Thanks!

Leave a Reply