Managed Client Object Model Internals – Theory of the Client Side

In the former part of my client OM series I wrote about the server side. This time I introduce the most important components of the client side of the managed client object model. The bad news is that it will be not simpler than the server side (even if I have neither place nor time to dig into the details here), but the good news is that in the next part I will show you a practical example of using this theory to extend the client OM, so hopefully it will help your better understanding of the theme.

The Microsoft.SharePoint.Client and Microsoft.SharePoint.Client.Runtime assemblies (in folder C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI) contain the classes of the client side. Both of these assemblies are visible to COM as they are decorated with ComVisible(true).

When working with the managed client object model we start with creating a ClientContext  class (in Microsoft.SharePoint.Client assembly) and use its Web or Site properties to access server side objects. This class is inherited from the abstract ClientRuntimeContext class (in Microsoft.SharePoint.Client.Runtime assembly). In fact, if you check the methods and properties of the ClientRuntimeContext you should note this class is not really SharePoint-specific, instead it provides general communication functionality. The SharePoint specific features are added via the ClientContext  class through its Web and Site property getters that return the value of the Web and Site property of the current RequestContext. More on that what this RequestContext is and how it is used in the client OM a bit later.

BTW, the same is true for the Microsoft.SharePoint.Client and Microsoft.SharePoint.Client.Runtime assemblies. The latter one contains the base communication features, interfaces and abstract classes of the client OM, and the first one adds the SharePoint specific classes into the game.

The client classes we work with in the client OM and that represent their server side counterparts are derived from the abstract ClientObject class or from one of its subclasses. ClientObject implements the IFromJson interface. This interface has two methods: CustomFromJson and FromJson, both of these are to read up the object from the JSON response stream sent by the server. ClientObject adds two extra methods to these ones to support JSON seserialization (InitNonPropertyFieldFromJson and InitOnePropertyFromJson).

ClientObject objects follow the inheritance of the server object as illustrated by these examples:

SecurableObject –> List, ListItem, Web
SPSecurableObject –> SPList, SPListItem, SPWeb

Field –> FieldText, FieldUrl
SPField –> SPFieldText, SPFieldUrl

For objects that represent a kind of collection on the server side, the abstract ClientObjectCollection class extends the ClientObject class by implementing IEnumerable, and the abstract ClientObjectCollection<T> class extends it further with LINQ functionality. Client side examples of the collections are FieldCollection and ListItemCollection classes.

ClientValueObject extends the base IFromJson functionality with methods to write object data into the request XML through the CustomWriteToXml and WriteToXml methods. Examples of the ClientValueObject  are FieldUserValue and CamlQuery.

Other classes that implement IFromJson are for example ClientResult<T>, ClientArrayResult<T>, ClientDictionaryResultHandler<T> and ClientListResultHandler<T> to handle single-value and multiple-value results.

Client OM methods and properties that are mapped to server side methods or properties are decorated with the RemoteAttribute attribute.

The base objects for accessing server side members

The abstract ClientAction class and derived classes (like ClientActionInvokeStaticMethod, ClientActionSetProperty) represents a special type of interaction with the server side objects. Its not  hard to guess the functionality of the classes from their names. For eaxmple, ClientActionInvokeStaticMethod calls a static method, ClientActionSetProperty sets a property on the server side object instance. Constructors of classes that represent static method calls / property setters, have a string property as a first parameter, whereas the first parameter of constructors for instance method calls / property getters is an instance of the client object model object. Second parameter of the constructor is the name of the method / property. For the property setters, the third parameter is the value of the property, for method calls the third parameter is an object array for the method call parameters.

(There are a few special types of ClientAction subclasses, for example, the ClientActionExecutionScopeStart, ClientActionExecutionScopeEnd, ClientActionInstantiateObjectPath and ObjectIdentityQuery that are mainly related to the internal working of the object model through managing execution scope and the object paths.)

Just like the ClientAction class there is another abstract class  ObjectPath that is used to describe the client object “position” within the client context. This class has similar subclasses as ClientAction has, like ObjectPathConstructor, ObjectPathMethod, ObjectPathProperty, ObjectPathStaticMethod, ObjectPathStaticProperty. While the classes derived from ClientAction used to call methods with void return types and set property values, the subclasses of the ObjectPath usually have something to do with return values. Either in the case of a new object created through a constructor, a method call or for a property getter.

The constructor of these methods are also similar to the constructors of ClientAction subclasses. For each of these object there is an extra first parameter of ClientRuntimeContext type, for instance member access (ObjectPathMethod, ObjectPathProperty) there is an additional second parameter of type ObjectPath. Of course, since the  ObjectPathProperty and ObjectPathStaticProperty are used for property getters, there is no parameter for the property value.

Let’s see some examples of accessing server side members from client objects!

When a client object method with return type of void is called the corresponding ClientAction subclass (ClientActionInvokeMethod/ClientActionInvokeStaticMethod) is created and added to the pending request of context. See for example the CheckOut method of the File class:

public void CheckOut()
{
    ClientAction query = new ClientActionInvokeMethod(this, "CheckOut", null);
    base.Context.AddQuery(query);
}

If the method has parameters, then the call is a bit more complicated because we need to check parameters and wrap them into an object array as we can see in the case of the CheckIn method of the File class

public void CheckIn(string comment, CheckinType checkInType)
{
    if ((base.Context.ValidateOnClient && (comment != null)) && (comment.Length > 0x3ff))
    {
        throw ClientUtility.CreateArgumentException("comment");
    }
    ClientAction query = new ClientActionInvokeMethod(this, "CheckIn", new object[] { comment, checkInType });
    base.Context.AddQuery(query);
}

If the method has a return value of complex reference type, the wrapper for the client side method call is typically looks like in the case of the GetCustomListTemplates method of the Site class (sometimes there are parameter validation checks as well):

public ListTemplateCollection GetCustomListTemplates(Web web)
{
    return new ListTemplateCollection(base.Context, new ObjectPathMethod(base.Context, base.Path, "GetCustomListTemplates", new object[] { web }));
}

In this case we simple creates and returns a client side object based on the object path returned by the ObjectPathMethod constructor.

For methods that return a primitive type on the server side, the client object methods return their value as a generic ClientResult<T> like in the case of Recycle method of the ListItem class:

public ClientResult<Guid> Recycle()
{
    ClientAction query = new ClientActionInvokeMethod(this, "Recycle", null);
    base.Context.AddQuery(query);
    ClientResult<Guid> result = new ClientResult<Guid>();
    base.Context.AddQueryIdAndResultObject(query.Id, result);
    base.RemoveFromParentCollection();
    return result;
}

As you can see, this time not only the ClientActionInvokeMethod is added to the to the pending request of context, but through the AddQueryIdAndResultObject method we add the query ID – result pair to the map that is used internally in the object method to track results in the response received from the server. After the response is processed, the return value of the method call can be read using the Value property of the ClientResult<T> instance.

There are even more complicated method calls, like the static OpenBinaryDirect and SaveBinary (this private one is called through the public SaveBinaryDirect method) methods of the File class. These methods works with binary stream data either as parameter or return value and calls server side directly through HTTP GET / PUT requests sent by the WebRequestExecutor (note the missing Remote attribute in the case of these methods).

Let’ see how property setters work. When a static / instance property is to be set, a ClientActionSetStaticProperty / ClientActionSetProperty instance is created and added to the to the pending request of context. For example, the property setter of the Title field in the Field class:

public void set_Title(string value)
{
    base.ObjectData.Properties["Title"] = value;
    if (base.Context != null)
    {
        base.Context.AddQuery(new ClientActionSetProperty(this, "Title", value));
    }
}

In this case, first the local “cache” is refreshed (in ObjectData.Properties), so subsequent local references can return the value, then a ClientActionSetProperty is created and added to the pending request of context.

A getter method resulting a primitive type is even more simple, it check if the property is initialized (an exception is thrown if not) and returns the value:

public string get_Title()
{
    base.CheckUninitializedProperty("Title");
    return (string) base.ObjectData.Properties["Title"];
}

For a complex type property it is a bit more complicated (AllProperties method of Web class):

public PropertyValues get_AllProperties()
{
    object obj2;
    if (base.ObjectData.ClientObjectProperties.TryGetValue("AllProperties", out obj2))
    {
        return (PropertyValues) obj2;
    }
    PropertyValues values = new PropertyValues(base.Context, new ObjectPathProperty(base.Context, base.Path, "AllProperties"));
    base.ObjectData.ClientObjectProperties["AllProperties"] = values;
    return values;
}

If the value is available on the client side, it is returned, otherwise a new ObjectPathProperty is created and “cached” for later usage.

But how these property values are populated?

When you would like to get information about server side objects, you should add them to the context through the Load<T> or the LoadQuery<T> methods (implemented in the ClientRuntimeContext class). If we check of the source of one of these methods, we see:

public IEnumerable<T> LoadQuery<T>(ClientObjectCollection<T> clientObjects) where T: ClientObject
{
    if (clientObjects == null)
    {
        throw new ArgumentNullException("clientObjects");
    }
    ClientQueryableResult<T> result = new ClientQueryableResult<T>();
    ClientQueryInternal query = new ClientQueryInternal(clientObjects, null, false, null);
    query.ChildItemQuery.SelectAllProperties();
    clientObjects.Context.AddQueryIdAndResultObject(query.Id, result);
    clientObjects.Context.AddQuery(query);
    return result;
}

We creates a ClientQueryInternal instance (just another subclass of ClientAction) for the client object passed as parameter to the method, select all of the properties to be get and add the query to the pending request of context.

A special subclass of the ClientObject class is the RequestContext class.

[ScriptType("SP.RequestContext", ServerTypeId="{3747adcd-a3c3-41b9-bfab-4a64dd2f1e0a}")]
internal class RequestContext : ClientObject

Based on the value of the ServerTypeId attribute it is not hard to find the server side class for RequestContext:

[ClientCallableType(Name="RequestContext", ServerTypeId="{3747ADCD-A3C3-41b9-BFAB-4A64DD2F1E0A}", Internal=true), SharePointPermission(SecurityAction.InheritanceDemand, ObjectModel=true), SharePointPermission(SecurityAction.LinkDemand, ObjectModel=true)]
public sealed class SPContext

As you can see, the server side pair of the RequestContext class is the SPContext class, that is the root of the server side object model.

It’s worth checking the GetCurrent method of this class, as there is a nice example for the usage of the ObjectPathStaticProperty class:

public static RequestContext GetCurrent(ClientRuntimeContext Context)
{
    object obj2 = null;
    if (!Context.StaticObjects.TryGetValue("Microsoft$SharePoint$SPContext$Current", out obj2))
    {
        obj2 = new RequestContext(Context, new ObjectPathStaticProperty(Context, "{3747adcd-a3c3-41b9-bfab-4a64dd2f1e0a}", "Current"));
        Context.StaticObjects["Microsoft$SharePoint$SPContext$Current"] = obj2;
    }
    return (RequestContext) obj2;
}

The above method simply calls the getter of the static Current property of the server side object with ServerTypeId="{3747ADCD-A3C3-41b9-BFAB-4A64DD2F1E0A}" (that is SPContext).

The ScriptTypeFactory class has a single method, CreateObjectFromScriptType that returns a new instance of the specified client object based on its script type name using the specified context. For example, for the script type "SP.SecurableObject" it will return a new SecurableObject instance.

I suggest you to study using Reflector, how the Web property of the RequestContext class works through creating a new Web instance and adding it to the ClientObjectProperties Dictionary of the ObjectData property (if it is not there already). Part of this instantiation is creating an ObjectPathProperty based on the current ClientRuntimeContext and the ObjectPath of the parent RequestContext. During this the corresponding ClientActionInstantiateObjectPath query, the ID of the query and the result object (ClientActionInstantiateObjectPathResult type) is added to the PendingRequest property (ClientRequest type) of the ClientRuntimeContext.

What happens when you submit your query to the server?

The internal  ExecuteQuery method of the ClientRequest class first calls the private BuildQuery method to assemble the XML request to be sent to the server. It first creates the root of the Request, then adds the Actions node by iterating through the ClientAction items (see more about and ClientAction its derived classes above) and the ObjectPaths node including the ObjectPath items in the current ClientRuntimeContext (you can read about ObjectPath class and its derived classes above as well).

Next, the ExecuteQuery calls the ExecuteQueryToServer method to send the XML down to the wire to the server and to process the server response. The request is executed through the WebRequestExecutor instance returned by the RequestExecutor property. The WebRequestExecutor is an abstract class, the exact implementation uses the derived SPWebRequestExecutor  class (see CreateWebRequestExecutor method of the DefaultWebRequestExecutorFactory class called by the getter of the RequestExecutor property, and the constructor of the ClientRuntimeContext that instantiate DefaultWebRequestExecutorFactory  to be returned by the WebRequestExecutorFactory property of ClientRuntimeContext). When the response is received in ExecuteQueryToServer method, the ProcessResponse method is called that invokes the ProcessResponseStream method that process the response through a JsonReader class. During that processing it reads up entities as IFromJson, and call their CustomFromJson and FromJson methods to initialize the objects.

Summary

Using the client object model is pretty easy and it is a very comfortable alternative of the former web service based client access, but there are more advanced things if you scratches its surface. In this post (and in the former part)  I tried to show you the base classes that build up the both sides of the client OM framework.

This post might be too theoretical and boring for you (thanks for reading this despite of that) and I admit you will need this stuff rarely. But if you are serious about working with the client OM, for example, extend it with your custom objects and methods then that is the absolute minimum you need to start with.

Although the out of box implementation of the client object model is great for everyday work, there might be cases when you hit walls working with it.  In such cases you can start building your own extensions on the top of the framework described in this post.

In a forthcoming post I will show you a simple example how to do that.