Inspired by Mix09 being held in Las Vegas the logical choice of entry was a slot machine game. So my Silverlight entry Spin & Win was created.
You can play the game online at the Mix09 Site and information about the game is linked within the game and is also available from my Mix09 10k Information/Help page.
Keeping the entry inside the 10k limit was a definite challenge. Overall it was a great process, diving deeper into creating a Silverlight game encompassing a whole range of factors to create the final product:
- Animation of controls to create visual effects.
- Coding optimisation techniques to minimise the size of the code.
- Dynamically loading controls and components.
- External links from the application e.g. Social bookmarks and Information/Help on the game.
- Graphic design of the game.
- Interaction with external services.
- State management giving the ability to save and load the state of the game.
Visit the Mix site now and add a vote and I look forward to your comments.
Overview
In this post I will provide an overview of the changes from the original wav media files, to wma files required by the Silverlight version.
The main sections are:
- Introduction – Discusses the background behind the use of audio files in the original WPF version and an overview of the changes required.
- Code Changes – Discusses the changes required to move from win32 api implementation to the MediaElement.
- Audio File Changes – Discusses uses Microsoft Expression Media Encoder 2 to convert the wav files to wma files.
- Summary – Overview of the changes made and the advantages of the change.
As Scott indicated in his first post Introducing BabySmash, he started using techniques that he already knew. As a result the laughs and giggles played when a key is pressed where included as embedded wav files. When moving to the Silverlight environment this approach require some modifications to optimise the download experience and cater for the differences of running cross platform within a browser.
The original audio files used by the WPF application where wav files as in made use of class called Win32Audio, which wrapped calls to the underlying Windows unmanaged win32 api’s.
public class Win32Audio
{
#region WIN32 Const Definitions
public const UInt32 SND_ASYNC = 0x0001;
public const UInt32 SND_LOOP = 0x0008;
public const UInt32 SND_MEMORY = 0x0004;
public const UInt32 SND_NOSTOP = 0x0010;
#endregion
// this is the overload we want to play embedded resource...
public static Dictionary<string, string> cachedWavs = new Dictionary<string, string>();
public static object cachedWavsLock = new object();
[DllImport("winmm.dll")]
public static extern bool PlaySound(byte[] data, IntPtr hMod, UInt32 dwFlags);
[DllImport("winmm.dll", SetLastError = true)]
private static extern bool PlaySound(string pszSound, IntPtr hmod, UInt32 fdwSound);
public void PlayWavResource(string wav)
{
string s = GetWavResource(wav);
PlaySound(s, IntPtr.Zero, SND_ASYNC);
}For the Silverlight version pinvokes to the underlying operating system are not allowed from both a security point and cross platform point of view. The answer was to implement a new class XamlAudio that makes use of the MediaElement class present in both WPF and Silverlight to play the audio files.
public class XamlAudio
{
private MediaElement player;
readonly Queue<string> soundsToPlay = new Queue<string>();
private bool inPlay;
private Dictionary<string, Stream> media = new Dictionary<string, Stream>();
//TODO Add lock and config value for sound queue limit
public void PlayWavResourceYield(string s)
{
if (player == null)
return;
if (inPlay && soundsToPlay.Count < 5)
{
soundsToPlay.Enqueue(s);
return;
}
PlaySound(s);
}The second major difference in the code is the shift from the use of embedded audio files, as indicated in my earlier post Silverlight BabySmash Performance – The Asynchronous Story, the Silverlight version asynchronously downloads the audio files on demand. As result the initial download size for the deployed application is smaller.
The encoding format was also a major difference, changing from the use of wav files to Window Media Audio files (*.wma). To convert the files I used Microsoft Expression Encoder.
The steps I used to convert the files where as follows:
- Download and install Microsoft Expression Encoder 2.
- Open Expression Encoder.
- Import the existing wmv files. Either using File Import menu or by licking on the Import button at the bottom of the window.
- Select the Encode tab.
- In the profile audio group expand the advanced properties.
- Change the properties to a smaller size to optimise the smaller size.
- Codec WMA
- Mode CBR
- Bitrate 20kps
- Sample Rate 44.1kHz
- Bits Per Sample 16
- Channels Mono
- Start the encoding process by click the encode button at the bottom of the window or the Encode File menu item.
- Once the encoding is complete select Output tab in the top right.
- Open the output folder by clicking o the Open Job button in the Job Output Section.
From the the output folder I copied the converted wma files to the ClientBin folder of the web site so that they where easily accessible to the Silverlight application when using a WebClient asynchronous download request.
| File |
WAV Encoding (kbps) |
WAV Files Size (KB) |
WMA Encoding (kbps) |
WMA File Size (KB) |
| babygigl2 |
20 |
6 |
20 |
15 |
| babylaugh |
88 |
14 |
20 |
11 |
| ccgiggle |
88 |
9 |
20 |
10 |
| EditedJackPlaysBabySmash |
352 |
227 |
20 |
25 |
| falling |
128 |
29 |
20 |
14 |
| giggle |
88 |
15 |
20 |
13 |
| laugingmice |
89 |
17 |
20 |
14 |
| rising |
128 |
26 |
20 |
13 |
| scooby2 |
20 |
5 |
20 |
13 |
| smallbumblebee |
705 |
862 |
20 |
31 |
One interesting point from the comparison of the conversion is that for the small files with the same bit rate encoding the WMA files where larger. If further optimisation in terms of files size is required, then this is an area that could be further investigated to determine the exact encoding settings to trade-off audio quality against file size.
- Changes to the C# code to make use of the built in MediaElement to play the audio files.
- Cross platform audio support using wma files instead of wmv files.
- Use of Expression Encoder to convert the source wmv files to a web optimised encoding.
Overview
In this blog entry I will cover some the factors effecting perceived performance and the asynchronous code used to optimise the user experience.
Introduction
When looking at the performance of the BabySmash Silverlight application it was important to look at a number of factors to help improve the perceived performance of the application. These included:
- Initial download size.
- Use of a secondary assembly.
- Use of media files.
So let's start of by profiling the current application see what has been done.
Looking at the HTTP profile of the application there are three distinct phases:
- Synchronous download of the initial html code and the Silverlight application.
- Asynchronous download of secondary components.
- Asynchronous download of media files on demand as the user presses keys.
Initial Download
The default.htm file together with the initial BabySmashWeb.xap combined are around 60kb. This xap file contains all the code necessary to interact with the user, request the additional resources and log information to the central ADO.Net Data Service.
Using this approach the loading phase of the the Silverlight application is very short and the user immediately shown the “Smash Canvas” as quickly a possible.
<div id="silverlightControlHost">
<object data="data:application/x-silverlight-2," type="application/x-silverlight-2" width="100%" height="100%">
<param name="source" value="ClientBin/BabySmashWeb.xap"/>
<param name="onerror" value="onSilverlightError" />
<param name="background" value="white" />
<param name="minRuntimeVersion" value="2.0.31005.0" />
<param name="autoUpgrade" value="true" />
<param name="initParams" value="logKeys=true" />
<a href="http://go.microsoft.com/fwlink/?LinkID=124807" style="text-decoration: none;">
<img src="http://go.microsoft.com/fwlink/?LinkId=108181" alt="Get Microsoft Silverlight" style="border-style: none"/>
</a>
</object>
<iframe style='visibility:hidden;height:0;width:0;border:0px'></iframe>
</div>Secondary Download
Once the application is loaded, the constructor of the main code behind Page class is called
public Page()
{
InitializeComponent();
var controller = new Controller();
controller.Launch();
mainPage = controller.Windows[0];
KeyUp += new KeyEventHandler(Page_KeyUp);
LayoutRoot.Children.Add(mainPage);
}Which calls starts a Controller class, which in turn starts the asynchronous requests for the letters and the initial media file.
public void Launch()
{
CheckForUpdatesAsync();
SetupInitialWindowState();
//here to pre-cache letter
CoolLetter.InitLetterStateAsync();
//Startup sound
audio.PlayWavResourceYield(".Resources.Sounds." + "EditedJackPlaysBabySmash.wav");
}The rest of this section will discuss how the letters stored in the secondary BabySmashEnglishLetters.xap are downloaded and stored for use when the user presses a key. The initial technique for loading the audio files is discussed in the implementation of PlayWavResourceYeild in the Load On Demand section below.
public partial class CoolLetter
{
public CoolLetter()
{
this.InitializeComponent();
}
static readonly Dictionary<string, string> _letters = InitLetters();
public static void InitLetterStateAsync()
{
// No work to do as the private _letters will be initialised above
}The InitLetters method is called by the .Net Framework when the static class is initialised. The method utilises a WebClient class instance to asynchronously download the required secondary assembly
private static Dictionary<string, string> InitLetters()
{
var addressUri = new Uri("BabySmashEnglishLetters.xap", UriKind.Relative);
var letterLoader = new WebClient();
letterLoader.OpenReadCompleted += letterAssemblyLoaded;
letterLoader.OpenReadAsync(addressUri);
return new Dictionary<string, string>();
}Once the asynchronous call is completed, reflection is used to dynamically call the GenerateAlphanumericCharacters method of the AlphaNumericLetterGenerator class.
private static void letterAssemblyLoaded(object sender, OpenReadCompletedEventArgs e)
{
if ((e.Error != null) || e.Cancelled) return;
// Convert the downloaded stream into an assembly
var a = LoadAssemblyFromXap("BabySmashLetters.dll", e.Result);
var generator = a.CreateInstance("BabySmashLetters.AlphaNumericLetterGenerator");
if (generator == null) return;
var generated = generator.GetType().InvokeMember("GenerateAlpanumericCharacters", BindingFlags.Public | BindingFlags.InvokeMethod | BindingFlags.Instance, null, generator,null) as Dictionary<string, string>;
if ( generated == null ) return;
foreach (var pair in generated)
{
_letters.Add(pair.Key, pair.Value);
}
}
Loading the assembly is achieved by extracting the requested assembly from the xap stream as follows.
public static Assembly LoadAssemblyFromXap(string relativeUriString, Stream xapPackageStream)
{
var uri = new Uri(relativeUriString, UriKind.Relative);
var xapPackageSri = new StreamResourceInfo(xapPackageStream, null);
var assemblySri = Application.GetResourceStream(xapPackageSri, uri);
var assemblyPart = new AssemblyPart();
var a = assemblyPart.Load(assemblySri.Stream);
return a;
}Load On Demand
By the time that the third stage of load on demand media files is reached the main application is loaded, the secondary assembly is loading or loaded and the introduction sound file has played. This has resulted in the download of around 160kb of files and kept the time wait time for the user to a minimum.
Loading the external dependencies has also given us the following advantages:
- Allow for the possibility of localising the application to other non-english languages.
- Allow media files to be updated independently of the compiled code.
- Allow the media files to to be sourced from alternative locations e.g. File stored locally on the client computer.
- Reduced bandwidth from 406kb for users that visit the page but do not start using the application.
The process for loading the additional media files is similar to the process to load the assembly secondary xap file but has the additional complexity that the user can be pressing keys very quickly. As a result the application needs to respond very quickly by queuing the key requests then asynchronously downloading and playing the associated sound file.
public void PlayWavResourceYield(string s)
{
if (player == null)
return;
if (inPlay && soundsToPlay.Count < 5)
{
soundsToPlay.Enqueue(s);
return;
}
PlaySound(s);
}
If the application is currently playing a media file then the request is placed on queue for later playback otherwise the sound will be played. If the media file is requested for the first time ten it will be asynchronously downloaded, added to the available media list then played.
private void PlaySound(string resourceName)
{
var mediaFile = GetMediaFileFromResourceName(resourceName);
lock (media)
{
if (media.ContainsKey(mediaFile) == false)
{
var wc = new WebClient();
wc.OpenReadCompleted += delegate(object sender, OpenReadCompletedEventArgs e)
{
lock (media)
{
if (media.ContainsKey(mediaFile) != false) return;
media.Add(mediaFile, e.Result);
player.SetSource(e.Result);
}
};
wc.OpenReadAsync(new Uri(mediaFile, UriKind.Relative));
return;
}
}
var sound = media[mediaFile];
if (sound == null) return;
if (sound.CanSeek) sound.Position = 0;
player.SetSource(sound);
}If the media file is requested again then the version available in the media list will be reused and played to the user.
Summary
In summary when looking Silverlight applications one aim is to improve the perceived user performance by:
- Quick Feedback – Provide quick feedback to the user so that the can interact with application. Small initial xap file allowed the “Smash Canvas” and initial introduction message where played without the overhead of other resources.
- Thinking Asynchronously - Performing actions asynchronously so that the user action is not blocked while necessary processing is occurring. Requesting media on demand allowed content to played as needed.
- Separating The Application – Look at architecting the application so that it can be split into different parts that can be managed independently if required. Using separated code/media files allows for further expansion points and hosting options.
Well Scott Hanselman has delivered his session at PDC NET Framework: Overview and Applications for Babies. And from the tweet streams it looks like he had a full room and fantastic feedback.
As Scott indicated in his earlier blog post he covers a lot of content in a short space of time so I would suggest watching the recorded video of his session TL49 (once it becomes available at http://channel9.msdn.com in the next 24 hours) that is now online from Channel9 or download the WMV file directly for offline viewing.
As an overview Scott’s presentation included different variants of his original WPF full framework BabySmash application expanded to use different platforms & technologies i.e.
- BabySmash Surface PC Demo.
- BabySmash WPF application using MEF.
- ADO.Net Data Service.
- Windows Mobile Running Silverlight 2 BabySmash.
- ASP.Net Charting.
- ASP.NET MVC Charting.
- BabySmash running under Silverlight 2 in the Browser.
- Silverlight 2 Charting.
Scott ended the presentation by sending the address to the Silverlight version out to twitter users. As users hit the keys, the results were sent to the ADO.Net Data Service which then displayed the live results using a Silverlight Chart and Microsoft Virtual earth.
This is where my involvement started. Scott decided to use initial port of BabySmash to Silverlight that I created in his PDC presentation, to deliver on the MMO (Massively Multiplayer Online) request for BabySmash.
So the BabySmash Silverlight MMO PDC version was born so to speak. Now that Scott has publically demonstrated the Silverlight version, I will continue adding blog posts to my existing BabySmash series. In the new posts I will discuss further details of the Silverlight version of BabySmash and the Silverlight Reporting that Scott and I collaborated upon to create the final result for the presentation.
Windows Azure has just been announced at PDC 2008. And information is now available online from http://www.microsoft.com/azure/default.mspx.
Drilling into the features for developers there is a video interview with Steve Marx that outlines Azure from a developer perspective:
- His blog http://blog.smarx.com/default.aspx hosted on Windows Azure.
- Aim to use current skills and technology that they are used to. Built around .Net I.e.
- Use Visual Studio
- .Net 3.5
- IIS 7
- Windows Communication Foundation
- Only difference is that it runs on the cloud or Fabric of components.
- Client side local development experience to test before deployment
- Azure API's
- Debugging
- Ability to emulate Simulate scale via Visualised Compute.
- Synchronised state
- Azure storage available (Blobs, Tables)
- Deployment, VS.Net deploy package and configuration file to a web portal.
- Question about management:
- Framework will restart application is required
- Logging and metering
- Integration with Windows Live Alerts to send information about service
- Ability to deploy new versions seamlessly
- Ability to set metrics "e.g. Queue length, response time" to trigger scale out.
- Ability to scale out on demand if required (IIS and SQL Services).
- Currently in CTP.
- Programming Differences
- Horizontal Scale - Store state centralised so that it can be accessed by any machine in the virtual web farm
- e.g. Web Farm - Save shopping cart
- Using in blob or table helps eliminate bottlenecks of centralised SQL Server for persistence of state.
- Ability to create services that can be leveraged buy other users e.g. Email service gateway.
- Service Discovery, still determine plans about how create Market Place of services over the next Year e.g.
- Find Services
- What do they cost
- Ratings for services
- Ability to integrate services in the development environment
- Technology
- RESTful interface
- Manages & Unmanaged Code
- Can be used from Python, Ruby etc
- Does not have to be web form applications could just be a REST interface to a headless service.
This months web meetup showcased the new features of IE 8 Beta 2 presented by Matt Heller from Microsoft and Firefox 3.1 Beta 1 by Robert O'Callahan and Chris Double from Mozilla.
IE 8 Beta 2
Matt was first up he covered the following:
For those wanting to take the new Beta or old versions of IE for a test drive Microsoft Virtual PC images are available for download.
Firefox 3.1 Beta 1
Next up to present were Robert and Chris. Robert has a great post on his blog about how he saw the meeting. He includes a copy of his slide deck and some demo pages that currently work with the daily build of Firefox 3.1 Beta 1.
The video and svg demonstration were of particular interest as it showed equivalent functionality to what Silverlight 1.0 offered, but without the need for a separate plug-in that needed to be installed. As Robert indicated in a recent post Mozilla has elected to use the Ogg Vorbis/Theora codecs for audio and video. With support for H.264 currently in Flash and announced that H.264 will be available in a future version of Silverlight, it will be interesting to see how widespread the use of Ogg and Theora will become.
For those interested Chris has uploaded a couple videos to YouTube Firefox Video Element Bling and SVG Video Demo that demonstrate the new HTML 5 video and SVG features inside the current Firefox 3.1 Beta 1 available which is for download. As this a Beta release Mozilla suggest setting up a separate profile for testing side by side using the -ProfileManager command line switch.
Take away thoughts
- Support for parts of HTML5 and CSS3 are being released from multiple vendors.
- Hosting of media files for Ogg/Thora may pose issues for users without the equivalent YouTube/Silverlight Streaming services.
- JIT compiled javascript code has the possibility to deliver much faster experience to the end user.
- Much more competition between plug-in's (Flash, Silverlight) and embedded browser functionality coming up.
In this post I will briefly discuss using Google Analytics in conjunction with Silverlight, to track the version installed and your application usage patterns.
One of the original problems with the initial port of BabySmash to Silverlight was loading the XAML storyboard animations for UserControls.
In this post I will cover the changes required from a Silverlight point of view to replace the RoutedEvent in the UserControl.Triggers section with either equivalent XAML or C# code depending on your need.