Recognition in Tablet PC

This article focuses on an important feature of the Tablet PC SDK 1.7. i.e. Recognition. Before we delve into developing a recognition aware application let us see how recognition is implemented in the Tablet PC SDK and what has to be done by the developer to implement recognition features.

Features of the Recognizer

The Tablet PC Platform provides support for ink recognition and this is done using software libraries called 'Recognizers.' In other words recognizers are just code that compute the format of the ink strokes for one or more languages. By the word language we are referring to the multiple languages available namely English, Chinese, German etc. Each recognizer therefore includes a property denoting the languages for which it is capable of interpreting ink strokes.

A recognizer can support one or more languages. But the guidelines of the Tablet PC SDK stipulate that a recognizer should support only one language because the accuracy of results tends to decrease when recognizing multiple languages concurrently. Having a recognizer support multiple languages will lead to a higher margin of error.

Since it is possible for multiple recognizers to be installed in the system, the notion of the default recognizer comes into play. This identifies the recognizer most suited to interpret ink given a specific LCID (locale identifier), which is usually obtained from the operating system's locale setting. That means if the operating system is English then the default recognizer will be in English.

It must be remembered that only the Tablet PC Operating system comes with the default recognizer and this is not available as part of the Tablet PC SDK 1.7 when developing on the Windows XP Professional Operating System. Also proper error handling has to be build into the application to check for existence of the recognizers or else unpredictable results such as non recognition can occur.

Synchronous vs. Asynchronous Recognition

It must be remembered that performing ink recognition functionality is often a computationally intensive task since it requires a lot of processing power to process in the background. This level of high accuracy of results is achievable only with the latest family of processors because the latest processors have just the minimal power to perform high intensive operations at the base level. Hence the Tablet PC Platform supports two kinds of recognition viz. synchronous and asynchronous.

Synchronous recognition occurs when the thread requesting recognition results, blocks until computation is complete. Asynchronous recognition occurs when the thread requesting recognition results is allowed to continue immediately following the request and is later notified that computation is complete from an event. Synchronous and Asynchronous are also called as foreground and background tasks.

There is also a good point to raise about when to use Synchronous and Asynchronous Recognition. But that is an advanced topic and will not be covered in this article. For now we will just focus on how these 2 types of recognitions can be achieved.

Let's create an application...

Let us create a simple application which will show how ink can be entered in a control and then recognized instantly and displayed in another control (asynchronous) or displayed only on the click of a button. (synchronous). Recall that as said earlier that recognizers are build only in Tablet PC OS and hence having Windows XP Tablet PC Edition is required to develop and execute the samples.

Start Visual Studio .NET 2003 and then create a windows form application. Design it as shown in Figure 1.

From the figure you will see that there are 2 regions. One is Synchronous and the other is Asynchronous. The button in the top half will do the conversion of ink text whereas in the bottom half, the conversion of ink to text will be done on the fly as the strokes are written on the screen. For now we will focus on the first half i.e. Synchronous and the remaining will be examined in the next part of the article.

As a starting point for coding we will reference this namespace in our application.

using

We have already seen the features of overlay and how it can be mapped to the handle of an existing control so as to enable inking on that control. We will do that for the textbox in the top half of the screen. Let us name the text box as txtSynchronous.

Put the following lines of code after the definition of the controls by the Windows Form Wizard. This is where the declaration of the controls is done.

//defining the inkoverlay

We are creating an instance of the InkOverlay class and we will use that object for binding the InkOverlay class to the text box txtSynchronous.

Now we have to make use of the Strokes class to capture the strokes that is entered in the text box and then convert it to text.

//defining the strokes for the text box

The recognizer context is one of the classes provided in the Tablet PC SDK 1.7 and its purpose is to perform the recognition operation. This object represents a session of recognizer functionality and the objects associated with it are the ink strokes to recognize, the parameters such as the recognition mode, preferred words, content hints, and any recognition results as they are computed.

//defining the recognizerContext

Put this code after the following lines of code

InitializeComponent

Let us go through the above lines of code and see what they signify. The first part is assigning the txtOverlay object to the handle of the text box control i.e. txtSynchronous and then enabling it. Only if this is enabled capturing of ink will be possible.

The next step is to instantiate the Strokes Object so that the strokes can be captured as it is written on the screen. We will see why this strokes object is so important and how this will be used later.

And the final step of instantiation is determining if the Recognizer exists on the hardware on which the application is running. This is because error handling has to be incorporated into the application. If there are no recognizers available then any code which makes use of the RecognizerContext to capture the strokes will result in a fatal exception and cause the application to crash. Hence this is an essential part of every application written for the Tablet PC. If there are any recognizer available then we instantiate the CreateRecognizerContext and then map the strokes of the RecognizerContext to the Strokes Object which we created earlier.

That is all is required to implement recognition in an application. Now as explained earlier about the synchronous and asynchronous process we will now write code in the clicked event of the button so that we will recognize the ink only on clicking the button.

private

That is all is required to convert the ink to text. The Strokes Class provides the mechanism of identifying the strokes in the Overlay class.

Now let us run the application and see the results. Press F5 to run the application. If you have entered the code correctly as how I have written it then there wont be any compilation error. If you have named the controls differently from what I have written then you must make the changes accordingly in the application so as to compile correctly.

The screens for writing in the text box and the outcome after clicking the button are shown in Figure 2 and 3 respectively.

 

 

We have seen how recognition is build into the Tablet PC SDK 1.7 and how this is used to capture the ink written on applications and convert the same to ink.
In the next article we will focus on the other mode of recognition viz. Asynchronous and see how it can be recognized on the fly.