While in Amsterdam, Almer/Blank debuted Synthia, the first publicly usable musical instrument to emerge from Project Ludi at Almer/Blank Labs.

Synthia, on the web, is a Flash 10 application that translates pictures into music — infinite classical canons — using a small set of rules that establish key, restricted notes within the key, tempo, instrumentation and the score. Synthia was actually developed as a demonstration app to accompany R Blank’s talk, ‘Hearing Pictures’, but the results were so fun to play with, that we decided to make it a publicly usable web-application, where anyone can upload any picture, have Synthia turn it into music, and then share it with your friends and embed it in any web page.

We will be posting a lot more about Synthia in the coming weeks, and updating her rules and features over the coming months. But, for now, I’d encourage all of you to play around with the Synthia Composer (just upload an image and listen), or just browse the gallery to hear what other pictures sound like.

And, if while playing with Synthia, you have some ideas on how we might commercialize her, share your thoughts and ideas and have a chance to win an iPad.

Share and enjoy!

-r

If you have suggestions for enhancements or more bug fixes please do report them at http://zamfbrowser.riaforge.com  If you don’t want to sign up for RIAForge just send me an email to omar@almerblank.com, I’d really like to hear your suggestions and bug reports!

As I write this, I’m also packing up to head off to Amsterdam in the morning, to debut Synthia at FITC Amsterdam, so I will unfortunately have to be brief.

I will be posting more information in the coming weeks, as we, at Almer/Blank Labs continue plugging away, refining Synthia and adding more features to the site.

Until then, I’ve posted a few notes on how Synthia works and how she came about. But, of course, we know that what you really want to do is play with Synthia. So, go ahead, start composing now, or browse the gallery of other compositions.

And, of course, let all your friends know about Synthia!

Share and enjoy,

-r

Unfortunately, even those courses are relatively new, the code in them no longer works, because OSMF has advanced a few sprints. We’re now at OSMF sprint 9, and the framework continues to shift quite a bit.

Because there are almost no examples on the web of using OSMF with Flash (as opposed to Flex), and I’ve had several people email me asking if I had time to update the code.

And, so, finally I did. At least for the progressive player. So you can download the Flash CS4 source code to build an OSMF player with the Sprint 9 framework from the article on Building progressive video players in Flash with OSMF.

Share and enjoy!

ex. PlaylistMediator

package com.project.view
{
	import org.puremvc.as3.patterns.mediator.Mediator;
 
	public class PlaylistMediator extends Mediator
	{
		static public const NAME:String = "PlaylistMediator";
 
		public function PlaylistMediator( viewComponent:PlaylistView )
		{
			super( NAME, viewComponent );
		}
 
		public function get playlistView():PlaylistView
		{
			return getViewComponent() as PlaylistView;
		}
	}
}

As I’ve worked on PureMVC projects with other developers I’ve had the opportunity to see different approaches at writing application code in a PureMVC setting. The view getter was a pretty common practice, but as I’ve seen the way its used, and abused, I started wondering to myself, why? Why is this method necessary? Well obviously, it is necessary to retrieve the view component, without having to cast the type each time its needed. The real question is, why is it public?

The purpose of the Mediator is to listen for events on the view and send notifications to the Controller, and to provide an API to manipulate the view. And to a lesser extent, to listen to notifications that should update the view. However, I am not a fan of this either, and I try to keep notification handling in Mediators to a minimum. But that is beside the point I am trying to make about the view getter. With an API to the view provided by the Mediator I feel like the view getter should actually be protected or private, instead of public.

ex. PlaylistMediator

package com.project.view
{
	import org.puremvc.as3.patterns.mediator.Mediator;
 
	public class PlaylistMediator extends Mediator
	{
		static public const NAME:String = "PlaylistMediator";
 
		public function PlaylistMediator( viewComponent:PlaylistView )
		{
			super( NAME, viewComponent );
		}
 
		protected function get playlistView():PlaylistView
		{
			return getViewComponent() as PlaylistView;
		}
	}
}

Using protected or private enforces that you should manipulate the view via the Mediator’s API, and also stops promoting code like this:

ex. PlaylistLoadedCommand

package com.project.controller.command
{
	import com.project.view.PlaylistMediator;
 
	import org.puremvc.as3.interfaces.INotification;
	import org.puremvc.as3.patterns.command.SimpleCommand;
 
	public class PlaylistLoadedCommand extends SimpleCommand
	{
		override public function execute( notification:INotification ):void
		{
			var playlist:Array = notification.getBody() as Array;
			var playlistMediator:PlaylistMediator = facade.retrieveMediator( PlaylistMediator.NAME ) as PlaylistMediator;
 
			var playlistItemView:PlaylistItemView;
			for (var i:Number = 0; i < playlist.length; i++)
			{
				playlistItemView		= new PlaylistItemView();
				playlistItemView.data		= playlist[ i ];
				playlistItemView.x		= 0;
				playlistItemView.y		= playlistItemView.height * i;
				playlistMediator.playlistView.addChild( playlistItemView );
			}
		}
	}
}

And instead, promotes code like this:

ex. PlaylistLoadedCommand – Registered to a notification sent when a Proxy is done loading playlist data

package com.project.controller.command
{
	import com.project.view.PlaylistMediator;
 
	import org.puremvc.as3.interfaces.INotification;
	import org.puremvc.as3.patterns.command.SimpleCommand;
 
	public class PlaylistLoadedCommand extends SimpleCommand
	{
		override public function execute( notification:INotification ):void
		{
			var playlist:Array = notification.getBody() as Array;
			var playlistMediator:PlaylistMediator = facade.retrieveMediator( PlaylistMediator.NAME ) as PlaylistMediator;
			playlistMediator.updatePlaylist( playlist );
		}
	}
}

In this example the Command simply calls the updatePlaylist() method on the PlaylistMediator, which encapsulates the logic required to actually update the playlist. The PlaylistMediator can now also offer this functionality to another application system if it is required later in development due to another action. Perhaps when the playlist is done and a refresh call is made, or if the user manually refreshes the playlist.

ex. In PlaylistMediator…

public function updatePlaylist( playlist:Array ):void
{
	var playlistItemView:PlaylistItemView;
	for (var i:Number = 0; i < playlist.length; i++)
	{
		playlistItemView		= new PlaylistItemView();
		playlistItemView.data		= playlist[ i ];
		playlistItemView.x		= 0;
		playlistItemView.y		= playlistItemView.height * i;
 
		playlistView.addChild( playlistItemView );
	}
}

This was just a small example, where I only showed accessing one mediator. There are instances where a notification might need to update two or three views. Instead of writing 10-20 lines in a Command per Mediator that needs to have a view updated, its much neater and easier to read to simply call a method on each Mediator. This has other benefits other than short code. If you are following some code that you are trying to debug it is easier to understand what happens if a notification is handled directly by a Command, since you will be able to see all affected actors, Mediators, Proxies, in a single centralized location. Listening to the notifications within all the mediators makes it more difficult to find all of the places that a single notification effects, as the example illustrates below. The alternative would be to listen to the notifications in each mediator, and have a command that triggers the proxy method. I think its more maintainable and easier to debug if all notifications go directly to a command. It makes more commands, and its more tedious, but in the end its far more maintainable and easier to debug.

package com.project.controller.command
{
	import com.project.view.PlaylistMediator;
	import com.project.view.MenuMediator;
	import com.project.model.proxy.ApplicationDataProxy;
 
	import org.puremvc.as3.interfaces.INotification;
	import org.puremvc.as3.patterns.command.SimpleCommand;
 
	public class PlaylistLoadedCommand extends SimpleCommand
	{
		override public function execute( notification:INotification ):void
		{
			var playlist:Array = notification.getBody() as Array;
 
			var playlistMediator:PlaylistMediator = facade.retrieveMediator( PlaylistMediator.NAME ) as PlaylistMediator;
			playlistMediator.updatePlaylist( playlist );
 
			var menuMediator:MenuMediator = facade.retrieveMediator( MenuMediator.NAME ) as MenuMediator;
			menuMediator.resetMenu();
 
			var applicationDataProxy:ApplicationDataProxy = facade.retrieveMediator( ApplicationDataProxy.NAME ) as ApplicationDataProxy;
			applicationDataProxy.getPlaylistInfo( playlist );
		}
	}
}

It’s 35 minutes across five lessons and takes you through the process of converting the progressive OSMF video player (which we create in the first installment of this course on Building Progressive Video Players with Adobe OSMF, into a dynamic multi-bitrate streaming player. Multi-bitrate streaming occurs when you program your Flash to deliver the highest quality video a viewer can see (dependent on their bandwidth). *Dynamic* multi-bitrate streaming is similar, with the additional feature of having your player constantly meters the bandwidth throughout viewing, to adjust the playback between multiple videos seamlessly, as the viewer’s bandwidth may fluctuate.

Please note: that the first course on progressive video players was built with OSMF Sprint 5. The second course on streaming with OSMF was built with OSMF Sprint 7. I spend the first lesson of the new course updating the progressive player to work with Sprint 7, and I wrote a few notes about the major differences between OSMF 0.5 and 0.7 in a post on my personal blog last month. Also note that, by the time the course was published by Adobe, OSMF had already moved forward to Spring 8.

When I wrote the first course on building a progressive video player with OSMF, I was struck by two things:
1) OSMF is clean — really clean — and it just works well in ways that the AS3 video API does not — for instance, with OSMF setting volume is a matter of talking to the volume property of the MediaPlayer
2) OSMF felt like a bit of overkill — you need a lot of imports and classes (a MediaPlayer, a MediaElement, a NetLoader and a URLResource) just to build the simplest of progressive video players — that would be simple enough, requiring seven lines of code with raw AS3 if you knew it

But, I had a hunch that #2 above was due to the fact that I was using approximately 0.05% of this powerful framework — like trying to use a nuclear bomb when a shovel would do just fine.

Well, when I started learning how to write the code to build dynamic multi-bitrate streaming player, I quickly learned that my hunch was right. The benefits of OSMF really kick in once you do anything beyond the most basic video player.

In the case of streaming, to convert your progressive OSMF video player to a streaming OSMF video player, all you do is change the URL! Seriously, that’s it. OSMF is smart enough to know that when you try to load a video from an RTMP server, you’re trying to stream — and it takes care of the rest.

That was pretty neat. But the benefits really kicked in when I beefed my regular streaming OSMF video player to one that supports dynamic multi-bitrate streaming. All it took was just a few extra lines of code to turn my regular video player into one that supports dynamic multi-bitrate streaming. If you want to learn those lines of code, I’d recommend checking out the course.

As I say, I’m still just barely cracking the surface of OSMF, and I’m really looking forward to digging deeper into its support for layouts, playlists and monetization. But, even with this second course, the massive time, workflow and standardization features of OSMF are already revealing themselves.

Share and enjoy!

-r

The Synesthesizer is a Flash 10 musical synthesizer that relies on synesthesia-inspired translation metaphors.

I’m only a few short months into what is a very long-term endeavor, and I’ve had precious few hours to spend on the project in that time, but I always set the benchmark definition of version 1.0 of the synesthesizer as ‘a system that produces something approximating music’, and even though I haven’t invested nearly the amount of time as I’d like, I believe that the current Synesthesizer meets this definition. And thus, I feel I have Synesthesizer 1.0.

The Synesthesizer is a Flash 10 application that translates pictures into music in real-time. What I’ve included with this post is a rendering of the output of the application — not the Synesthesizer itself. I’ll let you judge the quality for yourself — and please feel free to let me know what you think.

Now, of course, there is far more to do here, but what I found most amazing about this process was how easy it was to get to a point of generating ‘music’ instead of ’sound’. In effect, I apply something along the order of six rules to get to this point. The addition of each rule to the Synesthesizer brings us closer to the generation of music, instead of raw sound.

To get a bit more insight into the Synesthesizer, you should check out my talk, Hearing Pictures, which I’ll be presenting at FITC Amsterdam in February and at FITC Toronto in April.

In this talk, I describe the process I followed to get the Synesthesizer to this point. Using the 1943 Hermann Hesse novel, The Glass Bead Game (also published under the title Magister Ludi, the protagonist of the novel, from whom Project Ludi derives its name), as a starting point, and proceeding through a discussion of synesthesia, the aural illusions of Professor Diana Deutsch, the Experiments in Musical Intelligence by Professor David Cope, the music of Tamarin Monkeys, and many other stepping points, I walk through the thought process required to assume an odd endeavor such as the cross-modal translation of pictures into music.

Share and enjoy!

-r

For a bit more information about these courses, check out R’s post.

I included some detail about this launch here, in my inaugural posting. But I wanted to include a few additional notes here.

The RMI, which is now over four years old, has continued to mature, and now, in addition to our physical facilities in Los Angeles and Toronto, where we host the majority of our live workshops, we’ve established a respectable selection of on-demand titles in our online training catalog.

Because of the addition of a new type of training to our business, the needs of our website have also shifted over time. And we’ve attempted to keep pace, having built, modified and maintained two versions of our site of the past few years.

Now, we are in the process of rolling out the 3rd version of our site. Unlike with the previous two versions of the RMI site, this one will be released over time, with new features scheduled for release throughout 2010. These forums are just the first minor feature release, with more significant enhancements to come shortly after the holidays.

The RMI has traditionally maintained forums with all of our on-demand courses (though, those are temporarily disabled during this transition), specifically for the use of students to ask questions of their instructors in our asynchronous, online training environment.

These new forums are intended for a different purpose. Instead of supporting the learning process of students already enrolled in courses, these new forums are intended to foster more of an active sense of community around our website.

Anyone who has attended our live workshops in Los Angeles or Toronto knows that the RMI strives to provide a unique experience — not just in the topics we select, but also in the comfortable (and somewhat unusual) environments we create to bring you those topics. The forums, while representing a minor feature in the scheme of RMI V3, are an important step in helping us create the kind of environment around our website, that we like to create in our physical facilities.

We’ve chosen a few specific forums to start with, and we’ve chosen them to coincide both with the focus of our online training catalog, and also with the expertise of some of our resident faculty members who will be kind enough to monitor these forums on an on-going basis. So if you have any questions on Adobe Flash, Flex or ActionScript, working with PureMVC, or business practices, please feel free to head on over to the new RMI forums and we’ll do our best to help you out.

Share and enjoy!

-r

The idea originated from a 1943 Hermann Hesse novel, The Glass Bead Game (also published under the title, Magister Ludi, after the protagonist). The novel focuses around the life of Magister Ludi, who is a member of a monastic-style order at a time in the future.

The central activity of the order is the Glass Bead Game — it is their form of meditation. Although it is the central activity of the order, it is rather vaguely defined over the course of the text. Despite the lack of detail, the game has fascinated me since I read the novel. In the game, a glass bead represents an idea — any idea — such as an historical event, a work of music, a painting, a philosophical concept, anything. When it’s your turn, you must play a bead — that is, you must play an idea on the game board. But the idea you play (the bead you place) must be related to the idea (bead) placed by the previous player, thus over the course of an entire game creating a thread uniting widely disparate parts of human knowledge and experience. In this way, being a successful player of the game requires an agile intellect with the capacity to relate any idea to a wide variety of other — seemingly unrelated — ideas. Playing the game well requires fluency in math, music, language, history, art and science.

Ever since I started working with Flash 11 years ago, I’d always sort of envisioned Flash as a mini-Glass Bead Game. The term isn’t really used any more, but when Flash was young, it was considered a multimedia tool. And, Flash took the idea of multimedia to a new level — that is, despite its drawbacks, Flash has, since version 4, made it ludicrously simple to work with information of many different kinds. Flash supports character animation, motion design, text, audio, video, images, 3D data, live data, interactivity — Flash basically supports any type of digital information (this is, more or less, literally true with the Flash Player 10 FileReference powers). So, using Flash, we can establish relationships between different types of information (which is what many of the early Flash luminaries did, in one form or another). Hence, my picture of Flash as a mini-Glass Bead Game.

The fact that, especially early on (until Flash 9), the most successful Flashers also tended to be the most eclectic, only strengthened this notion for me. Flash is really the only technology in the world that could give birth to a significant population of designer/developer hybrids (a.k.a., the deselopers), which speaks to the multi-disciplinary nature of the tool and the unique skill-sets that lead to success within Flash.

So, the idea of Flash as a Glass Bead Game kept percolating in my head for years, until I eventually decided to start trying to formalize the notion. The notion of creating a computer program that could translate anything into anything.

But of course, that’s an insanely complex endeavor. So, I wanted to find a more focused niche that could serve as a starting point.

Then, Flash 9 came out, with its enhanced sound powers. All of a sudden you started seeing real-time audio analysis in Flash movies — sound visualizers translating sound into pictures. Now, of course, sound visualizers have been around for a while, but now I could build them myself, using the ActionScript code I already knew. And the thought occurred to me to create a machine that worked in reverse — translating pictures into music. Still, even with Flash 9’s powers, it was a frustrating go. But, when Flash 10 came out, with the SampleDataEvent, I lost all excuses and picked things back up.

Of course, with the amount of time I’ve been able to spend on the project, it’s not even properly called a ‘hobby’. And so, I’m now six months into what I consider to be a 40 year endeavor, and I have a few little experiments to show for the effort. Those experiments — and the thought that went into them — is what my talk at Wednesday’s LA Flash End-Of-Year Party is about.

But, that’s not what this post is about.

One key limitation of this entire process is that I’m working with digital information. Now, I’m not complaining — the fact that all information is digital is what permits us to execute the cross-modal translations with relative ease (i.e., it’s easier to translate a picture into music, if both the picture and music are stored as 0s and 1s). However, working in digital is also a bit creatively limiting. When composing music on the computer, I’m composing from audio information already stored on the computer — more like playing a synthesizer than a piano. Again, that’s fine, but I thought it would be fun to start experimenting with analog inputs and outputs (which I can read and control from Flash using my MakingThings Controller Board).

About a month ago, I picked up a copy of Handmade Electronic Music: The Art of Hardware Hacking, which describes, in a series of clearly written chapters, how to go about assembling synthesizers from inexpensive, easily available hardware.

I’m not done with the book yet, but in the early chapters, you learn how to build your own microphones — a few different types, including contact mics, compressor mics — even electromagnetic mics (microphones that ‘play’ the sounds of the electromagnetic fields that surround the microphone).

When I rewrote the Hearing Pictures presentation this past weekend I found a neat way to integrate one of these piezo contact microphones into my talk. A contact microphone is one that can pick up vibrations in a surface, but not from the air. So, you can plug the contact mic into an amplifier, then hold the contact mic up to something that is vibrating and you will hear the vibration through the amp. They’re fun to play around with (you can even hear the sound of metal heating up with these mics).

The simple, cheap (<$1.50) contact microphone that I’ve built relies on a Piezo disc, or piezo sensor (~$0.75). These Piezo sensors rely on the Piezoelectric effect (discovered by Pierre and Marie Curie in 1880, but not industrialized as a sensor until the 1950s) to convert pressure, acceleration, strain or force into an electrical signal.

You need a:

• Piezo disc
• An amp
• An audio jack (that fits your amp)
• A little bit of audio cabling

Audio cables are composed of two wires.

Step 1) Take one wire and solder it to the outer plate of the piezo disc, and solder the other end to one hookup in the audio jack.

Step 2) Then take the other wire and solder it to the inner plate of the piezo disc, and then solder the other end to the 2nd hookup in the audio jack.

Step 3) Plug the jack into the amp, and start rubbing your piezo disc against things to see how they sound

That’s it! You don’t even need a battery to power this mic. As you’ll quickly see, you can not pick up any vibrations from the air — so it’s totally unusable as a traditional mic (read the chapter on building a condenser mic if you’re interested in that). But, if you place it on the end of a styrofoam cup, and then speak into the cup, you have a neat Halloween microphone!

If you’re interested to see how I integrated this contact mic into my talk, well then, all I can say is, show up on Wednesday. It’s free, and it’ll be a really cool and interesting talk. (Oh, and there’s free beer — did I mention that?).

Share and enjoy!

-r