Second Life of a Hungarian SharePoint Geek

January 9, 2011

Speech enable your web pages using Silverlight, WCF and the Managed Speech API

Filed under: Silverlight, Text-To-Speech, WCF — Tags: , , — Peter Holpar @ 23:58

Integrating speech-related functionality (either speech synthesis or speech recognition) into custom applications is one of the many areas I am interested in since years, but never had enough time to delve into.

In the last few months I read several articles about speech-capabilities of Silverlight. A few days ago I found this article where Brian Lagunas shows us how to create a WCF bridge between Silverlight and server-side Managed Speech API. I liked his idea and decided to take this approach one step further combining it with HTML events. My idea was to enable users to select web page text in the browser, similar like in my former post about IE add-ins, but in this case the application should “read” the selected text instead of simply copying it into a text box.

You can watch this screencast about how to handle HTML events in Silverlight. In my case I had to handle the onmouseup event, when the user releases up the mouse button after selecting the text.

To help debugging, I’ve included a text box, called SelectedText in the page XAML. Beside sending the selected text back to the server for speech conversion, we also copy the selection into this text box. This text box can be removed if you don’t need that anymore.

There is a simple sample sentence on the host page that you can use to test the feature.

image

Of course, in a real page you don’t want the Silverlight application to be visible at all. To hide the Silverlight UI, you cannot use the style="display:none", since it will cause  the Silverlight app not to load. Instead, you should set its height and with to zero as shown below. You can read more about this behavior in this forum thread.

  1.     <div id="silverlightControlHost"> <!– style="display:none" doesn't work –>
  2.         <object data="data:application/x-silverlight-2," type="application/x-silverlight-2" width="600" height="120" > <!– width="0" height="0" –>
  3.           <param name="source" value="ClientBin/SpeechApp.xap"/>
  4.           <param name="onError" value="onSilverlightError" />
  5.           <param name="background" value="white" />
  6.           <param name="minRuntimeVersion" value="4.0.50826.0" />
  7.           <param name="autoUpgrade" value="true" />
  8.           <a href="http://go.microsoft.com/fwlink/?LinkID=149156&v=4.0.50826.0&quot; style="text-decoration:none">
  9.               <img src="http://go.microsoft.com/fwlink/?LinkId=161376&quot; alt="Get Microsoft Silverlight" style="border-style:none"/>
  10.           </a>
  11.         </object><iframe id="_sl_historyFrame" style="visibility:hidden;height:0px;width:0px;border:0px"></iframe></div>

Let’s see the main code of the Silverlight application. In the OnLoad event we subscribe to the onmouseup HTML event. The OnMouseUp method is responsible to handle this event. It first determines the selected text (more about that a bit later), copies it to the text box, and if it is not empty, calls the WCF service to get the speech response the very same way as it is done in Brian’s post.

  1. void OnLoaded(object sender, RoutedEventArgs e)
  2. {
  3.     HtmlDocument doc = HtmlPage.Document;
  4.     doc.AttachEvent("onmouseup", OnMouseUp);
  5. }
  6.  
  7. void OnMouseUp(object sender, HtmlEventArgs e)
  8. {
  9.     String selection = (String)HtmlPage.Window.Invoke("getSelection");
  10.     SelectedText.Text = selection;
  11.  
  12.     if (!String.IsNullOrEmpty(selection))
  13.     {
  14.         SpeechServiceClient client = new SpeechServiceClient("BasicHttpBinding_ISpeechService");
  15.         client.SpeakCompleted += (o, ea) =>
  16.         {
  17.             WavMediaStreamSource audioStream = new WavMediaStreamSource(new MemoryStream(ea.Result));
  18.             AudioPlayer.SetSource(audioStream);
  19.         };
  20.         client.SpeakAsync(SelectedText.Text);
  21.     }
  22. }

Unfortunately, I was not able to get the selected text directly from Silverlight, so I’ve included the following JavaScript method in the hosting page, and call that method to get the selection.

  1. function getSelection()
  2. {
  3.     var currentSelection = document.selection;
  4.     var result = "";
  5.  
  6.     if ((currentSelection != null) && (currentSelection.type == "Text")) {
  7.         range = currentSelection.createRange();
  8.         if (range != null) {
  9.             result = range.text;
  10.         }
  11.     }
  12.  
  13.     return result;
  14. }

You can download the code for this application here. This code is really does not differ too much from the original version expect of the few code snippets illustrated above. Thanks to Brian for the original idea and implementation as well as his permission to make this modification and publish the results.

Advertisements

5 Comments »

  1. Thank you for the illustration.
    One question:
    I want to use it with Silverlight as well but how do i modify SpeakAsync to make it synchronous ? The idea would be to play a text first following another audio file.
    Thanks.

    Comment by Claude — May 6, 2011 @ 00:45

    • Hi Claude,

      You shouldn’t modify SpeakAsync to be synchronous, instead I suggest you to catch the SpeakCompleted event and start to play your second audio there.

      Peter

      Comment by Peter Holpar — May 6, 2011 @ 07:55

  2. Hi Peter,
    Thank you for your response.
    Do you have an example that illustrate how to catch the SpeakCompleted event ?
    I’m a bit confused since it’s already used as
    spSvc.SpeakCompleted += (o, ea) =>
    {
    WavMediaStreamSource audioStream = new WavMediaStreamSource(new MemoryStream(ea.Result));
    AudioPlayer.SetSource(audioStream);

    };
    Thank you.

    Claude

    Comment by Claude — May 6, 2011 @ 22:05

    • Hi Claude,

      The example is exactly that you found. You should alter the current version to either start the second audio file (or you can download it in advance, especially if it is a fixed file) while holding the first audio stream in a member variable, or start playing the first audio while the second download is in progress.

      Peter

      Comment by Peter Holpar — May 7, 2011 @ 21:52

  3. Thanks Peter,

    I’ll try that. Appreciate the info.

    Claude

    Comment by Claude — May 9, 2011 @ 19:31


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: