Oct 6, 2011
Now that Adobe MAX 2011 is over, I wanted to post the slides and sample code from my session.
The talk was called 'Delivering the Best Flash HD Video', since that was really the main point of the session. But, at a technical level, the talk covered StageVideo, which was introduced earlier this year in Flash Player 10.2.
I'm including the slides mostly for the benefit of the people who attended the session — but you are more than welcome to try to work your way through them on your own.
In them, I introduce StageVideo and demonstrate the massive performance improvements that you can get by employing these new video display options. I explain what hardware acceleration actually means, and how it works for video. I cover why H.264 is StageVideo's best friend, and how to work with it. I address the new ActionScript 3 APIs for StageVideo (with the demos, below) and also explain how OSMF (as well as Strobe Media Playback and Flash Media Playback) allow you to work with StageVideo, without knowing or using any custom code at all.
The sample ActionScript 3 code, below, will hopefully be of additional value to many of you who were not there. This is the 'more complex' demonstration piece from the talk — the one that includes the proper sizing of the video, as well as pan and zoom functionality. This supplements the simple StageVideo sample code, the first demo from the talk, which I posted last week.
When the talk is live on Adobe TV, I will post a link to it. Until then, hopefully you can work your through this fully-commented code file.
This demonstration code is intended to be set as the document class for a Flash CS5.5 FLA — just change the name of the video you want to play.
Share and enjoy!
-r
package { /** * This class illustrates a slightly more complicate StageVideo implementation * Including Pan and Zoom controls * And information about the rendering state of the StageVideo instance * * @langversion ActionScript 3.0 * @playerversion Flash 10.2 * * @author R Blank * @since 09.09.2011 */ //import the MovieClip, because this class is a MovieClip import flash.display.MovieClip ; //import the Rectangle, so that we can size and position the StageVideo instance import flash.geom.Rectangle ; //import Point so that we can pan and zoom our stage video import flash.geom.Point ; //import StageVideoAvailabilityEvent so we can hear //when StageVideo availability changes import flash.events.StageVideoAvailabilityEvent; //StageVideoEvent provides information to us about video rendering import flash.events.StageVideoEvent; //import StageVideo so we can use StageVideo import flash.media.StageVideo; //import StageVideoAvailability to get the values for the //StageVideoAvailabilityEvent.available property import flash.media.StageVideoAvailability; //import Video as a backup for when StageVideo is not available import flash.media.Video; //import the NetConnection and NetStream to play our video file import flash.net.NetConnection; import flash.net.NetStream; //import the TextField so we can log values visibly import flash.text.TextField ; //KeyboardEvent for keyboard controls import flash.events.KeyboardEvent ; //Keyboard class provides keyboard key values import flash.ui.Keyboard ; public class FullStageVideoExample extends MovieClip { //-------------------------------------- // CLASS CONSTANTS //-------------------------------------- //the location of our video file private const _videoURL : String = "../_media/bunny.mov" ; //the value which we will use to iterate pan and zoom controls private const _zoomAndPanVal : Number = .05 ; //-------------------------------------- // CONSTRUCTOR //-------------------------------------- public function FullStageVideoExample ( ) { //call the _init function to startup _init () ; } //-------------------------------------- // PRIVATE VARIABLES //-------------------------------------- //the Video object to play video when StageVideo is not available private var _video : Video ; //the StageVideo object to play fully accelerated video private var _stageVideo : StageVideo ; //the NetStream and NetConnection objects to play video private var _ns : NetStream ; private var _nc : NetConnection ; //the TextField object to log informationvisibly in the browser private var _tf : TextField ; //_panX stores the current stage video horizontal pan value private var _panX : Number ; //_panY stores the current stage video vertical pan value private var _panY : Number ; //_zoom stores the current stage video zoom value private var _zoom : Number ; //-------------------------------------- // PRIVATE & PROTECTED INSTANCE METHODS //-------------------------------------- //called by the constructor to startup the player private function _init ( ) : void { //create the video object as back up, in case StageVideo is not available _video = new Video ( ) ; //create the TextField for logging _tf = new TextField ( ); _tf.height = stage.stageHeight ; _tf.width = stage.stageWidth ; _tf.mouseWheelEnabled = true ; _tf.multiline = true ; _tf.wordWrap = true ; addChild ( _tf ) ; //the NetConnection object over which video is delivered _nc = new NetConnection ( ) ; //connect _nc to null because we are playing video progressively _nc.connect ( null ) ; //create the NetStream to deliver video over the NetConnection _ns = new NetStream ( _nc ) ; //define the NetStream client as 'this' (meaning this class) //where _ns will look for the onMetaData and onCuePoint methods _ns.client = this ; //tell the NetStream to play our video file _ns.play ( _videoURL ) ; //listen for the StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY event //this is dispatched as soon as the event listener is added //and when stage video availability changes during SWF playback stage.addEventListener(StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY, _onStageVideoAvailability); } //called when we wish to start StageVideo playback private function _enableStageVideo() : void { //if the _video object is in the display list if ( _video.parent ) //remove _video from the display list removeChild ( _video ) ; //if we do not have a StageVideo object reference //already stored in _stageVideo if ( _stageVideo == null ) { //set _stageVideo to reference the first of our //available StageVideo objects (up to 8 available on desktops) _stageVideo = stage.stageVideos [ 0 ] ; //listen for the StageVideoEvent.RENDER_STATE //which provides us information about how the video is being rendered //this is fired when the netstream is attached to the StageVideo instance _stageVideo.addEventListener(StageVideoEvent.RENDER_STATE, _onStageVideoStateChange ); //log the available color spaces _tf.appendText ( "_stageVideo.colorSpaces : " + _stageVideo.colorSpaces.toString() + "\n"); } //attach our NetStream instance to our StageVideo instance _stageVideo.attachNetStream ( _ns ) ; //listen for key strokes (which we use for pan and zoom controls) stage.addEventListener ( KeyboardEvent.KEY_DOWN , _onKeyDown ) ; //reset pan values to 0 (center) _panX = 0 ; _panY = 0 ; //reset zoom value to 1 (actual size) _zoom = 1 ; } //called when StageVideo becomes unavailable //and we need to use a Video object as back-up private function _disableStageVideo() : void { stage.removeEventListener ( KeyboardEvent.KEY_DOWN , _onKeyDown ) ; //attach our NetStream to our Video object _video.attachNetStream ( _ns ) ; //add the Video object to the display list //(so that it can be seen) addChild ( _video ) ; //add the _tf TextField back to the display list //so that it is on top of the _video object addChild ( _tf ) ; } //called to change the zoom factor on the StageVideo instance //called with true to zoom in // false to zoom out private function _zoomIn ( zoomIn : Boolean ) : void { //if we are zooming in if ( zoomIn ) { //if our current zoom factor is less than 16 (the max) if ( _zoom < 16 ) { //increment our zoom factor (with a ceiling of 16) _zoom = Math.min ( 16 , _zoom + _zoomAndPanVal ) ; //set the zoom factor on the StageVideo instance to our _zoom value _stageVideo.zoom = new Point ( _zoom , _zoom ) ; } //otherwise, we are zooming out } else { //if our current zoom factor is greater than 1 (the minimum) if ( _zoom > 1 ) { //decrement our zoom factor (with a floor of 1) _zoom = Math.max ( 1 , _zoom - _zoomAndPanVal ) ; //update the zoom of the StageVideo to our zoom factor _stageVideo.zoom = new Point ( _zoom , _zoom ) ; } } //log the zoom factor _tf.appendText ( "zoomed to " + _zoom + "\n") ; } //called to pan our stage video (which only matters if zoom > 1 ) //called with "up" to pan up // "down" to pan down // "left" to pan left // "right" to pan right private function _pan ( direction : String ) : void { //if the zoom factor is greater than 1 if ( _zoom > 1 ) { //switch, based on the 'direction' parameter switch ( direction ) { case "up" : //if _panY is greater than the minimum value if ( _panY > -1 ) { //decrement _panY (with a floor of -1) _panY = Math.max ( -1 , _panY - _zoomAndPanVal ) ; } break ; case "down" : //if _panY is less than the max value if ( _panY < 1 ) { //increment _panY (with a ceiling of 1) _panY = Math.min ( 1 , _panY + _zoomAndPanVal ) ; } break ; case "left" : //if _panX is greater than the minimum value if ( _panX > -1 ) { //decrement _panX (with a floor of -1) _panX = Math.max ( -1 , _panX - _zoomAndPanVal ) ; } break ; case "right" : //if _panX is less than the max value if ( _panX < 1 ) { //increment _panX (with a ceiling of 1) _panX = Math.min ( 1 , _panX + _zoomAndPanVal ) ; } break ; } //log the pan _tf.appendText ( "Panning to _panX: " + _panX + " , _panY: " + _panY + "\n" ); //pan the StageVideo _stageVideo.pan = new Point ( _panX , _panY ) ; } } //called to reset the pan and zoom on the stage video private function _resetPanAndZoom ( ) : void { //set the pan values to 0 _panX = 0 ; _panY = 0 ; //set the zoom factor to 1 _zoom = 1 ; //update the pan value on the StageVideo _stageVideo.pan = new Point ( _panX , _panY ) ; //update the zoom value on the StageVideo _stageVideo.zoom = new Point ( _zoom , _zoom ) ; } //-------------------------------------- // Event Handlers //-------------------------------------- //this serves as the callback when keys are pressed down private function _onKeyDown ( evt : KeyboardEvent ) : void { //switch based on the code of the pressed key switch ( evt.keyCode ) { //if the enter key is pressed case Keyboard.ENTER : //call to reset the StageVideo display _resetPanAndZoom ( ) ; break ; //the F key case 70 : //enter full screen playback ('esc' will implicitly exit full screen) stage.displayState = "fullScreen" ; break ; //the 'space' key case 32 : _tf.text = ""; break ; //the +/= key case 187 : //call to zoom in _zoomIn ( true ) ; break ; //the -/_ key case 189 : //call to zoom out _zoomIn ( false ) ; break; case Keyboard.UP : //call to pan up _pan ( "up" ) ; break ; case Keyboard.DOWN : //call to pan down _pan ( "down" ) ; break; case Keyboard.LEFT : //call to pan left _pan ( "left" ) ; break ; case Keyboard.RIGHT : //call to pan right _pan ( "right" ) ; break ; } } //-------------------------------------- // StageVideo Event Handlers //-------------------------------------- //our callback function for the //StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY event private function _onStageVideoAvailability ( evt : StageVideoAvailabilityEvent ) : void { //log the availability of StageVideo _tf .appendText ( "StageVideo has become " + evt.availability + "\n" ); //if StageVideo is available if ( evt.availability ) //call _enableStageVideo _enableStageVideo ( ) ; else //otherwise call _disableStageVideo _disableStageVideo ( ) ; } private function _onStageVideoStateChange ( evt : StageVideoEvent ) : void { //log the rendering method _tf .appendText ( "StageVideo is rendering with " + evt.status + "\n" ); //size and position our _stageVideo with a new Rectangle //sized the native width and height of the video we are playing _stageVideo.viewPort = new Rectangle ( 0 , 0 , _stageVideo.videoWidth , _stageVideo.videoHeight ) ; } //-------------------------------------- // NetStream Event Handlers //-------------------------------------- //the NetStream will look for these two methods: onMetaData + onCuePoint //on whatever object is identified as the NetStream's client //if they do not exist, you will receive a run time error //when the NetStream encounters metadata or cuepoints in the video //onMetaData is called when metadata is encoutered in NetStream playback public function onMetaData( info:Object ) : void { //capture the native video width and height from the metadata //and update the width and height of the _video object //to match the loaded info if ( info.width ) _video.width = int ( info.width ) ; if ( info.height ) _video.height = int ( info.height ) ; } public function onCuePoint( info:Object ) : void { } } }
[...] A brief note that the slides and code files from my talk at Adobe MAX 2011 have been posted to Almer/Blank labs. [...]
Hey mate,
I set the code as a document class for my (empty) fla and changed the video to a .mov that I have but it doesnt work… No errors or anything…
(Flash 5.5)
Did you test in the browser? With what version of the player?
In my experience, StageVideo does NOT work in test-movie mode. (Annoying, right? But no different from full screen mode, in that sense.)
[...] recently at Adobe MAX). I posted the commented code from that talk (the simple code demo, and the more complex code demo), but didn't go into much detail about StageVideo itself and how it [...]
Ah ok. I was testing it in test movie mode and no dice. But then I moved everything to my xampp install and running it from localhost, no good either. I get the StageVideo has become unavailable error message. (FP 11)
Hello Jeff:
When you say 'error message', do you mean the trace output in the Flash movie itself?
Well, that almost certainly means that StageVideo is not available on your system. However, in that case, the video object should work as fall-back. I'd like to try to help you through this, if you can give me any more information. What kind of video are you trying to play? Are you sure its in the right place? Any chance of yousendit'ing the stuff to me?
-r
Yes apologies I should have been more clear, its in the textbox that says that. I am playing a mov file. I emailed you the info
Jeff:
A few notes. To start, if I said I knew what was happening here, I would be very much lying.
That said, in my initial tests, you html/swf failed to play the video. So I tested your video with my original code, and it played. Hmmm….
I opened your HTML, and saw that WMODE was set to 'window'. It should be set to 'direct' to ensure StageVideo works on as many platforms as possible. So I updated that.
Still though, nothing.
Then I just dragged your SWF on to my browser, and boom — it worked. I reloaded, and it worked again. So playing the video with the SWF worked.
And now, I can not get playback to fail, even with the HTML, even with WMODE set back to 'window'. After that, each time I test the HTML in the browser, your player plays the video.
It's all quite weird.
Now, here is a universal rule for testing with video: use video with sound! Otherwise, you don't know if your video isn't playing at all, or just isn't visible.
That's a pretty big difference. But now that I can not get it to fail, I'm having a hard time figuring out what your problem actually was.
Three things you can try:
1) Switch to testing a video with an audio track
2) StageVideo can not been seen unless you assign a rectangle to the viewPort property. In my code, I do that in the callback on the StageVideoEvent.RENDER_STATE event. If, for whatever reason, that event doesn't fire properly, maybe you should add a default rectangle. So on line 137, right before you listen for the StageVideoEvent.RENDER_STATE event, you can add this line:
This will help determine what the actual failure is.
3) Less importantly, Call _disableStageVideo() from within the _init method, on line 116, before adding the listener to the stage for the StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY event. That might help ensure that the video object is used as fall-back. (Shouldn't make a difference, but who knows).
Let me know how this turns out.
Hey mate,
Ya strange things going on. For the HTML I justed used Flash's publish settings and used the generated HTML. (Dont do that too often, usually just swfobject but was figuring it would be ok for this demo; guess not!
The .mov does have sound but there isnt sound when played within Flash. Adding the rectangle and calling _disableStageVideo() where you said in the code allows the vid to play in Flash but then no video plays in the browser.
I'm thinking at this point it must be something to do with my Flash IDE. Really appreciate your help! I'll keep hammering on it and see how it goes.
Jeff, before spending too much time debugging, your reply made me think of one more thing.
When you said that the video does have sound, I went back and checked the codec. Turns out your video file uses Linear PCM for the audio codec. Flash doesn't understand that — only mp3 (generally with flv) and AAC (generally with H.264). (Also ADPCM but I've never seen that in video).
There is a chance that the audio track is confusing how flash parses the data in the video file.
So, go back to the original code, and try a different video file, that has AAC for the audio codec.
Please let me know if that works.
Thx
-r
You're right. That must have been it. I got a properly encoded video and that works. Thanks!
Awesome. Glad to hear it.