Second Life of a Hungarian SharePoint Geek

June 11, 2014

How to filter the results of JSCOM requests on the server side

Filed under: ECMAScript Client Object Model, SP 2013 — Tags: , — Peter Holpar @ 22:32

In the past I already discussed the internals of the Managed Client Object Model and the ECMAScript Object Model (a.k.a. JSCOM) several times. In this post I illustrate an unofficial way of extending the JSCOM.

WARNING: As I wrote above, the method I describe here is totally unofficial and unsupported. It’s only an experiment that depends on unpublished features of the JSCOM, like internal functions that can be changed or even removed without prior notice. Please, take my words seriously, and do not apply this method in a productive environment.

Assuming you are working with the Managed Client Object Model, you can submit queries, that are evaluated on the server side, and return only results that fulfill the conditions. For example, the request below returns only the filterable, non-hidden fields of a list (sample taken from MSDN):

    list => list.Fields.Where(
    field => field.Hidden == false
        && field.Filterable == true));

The most important benefit of this syntax is that you can reduce the network traffic caused by returning irrelevant data (in this case fields) that we would have to filter on the client side without this feature.

Unfortunately, this feature seems to be missing in the ECMAScript Object Model, the Include keyword supports only limiting the members of the object to be returned (the query below returns only the Title and Id properties of each lists), but does not help to filter the results at all (e.g. you can not filter lists based on their properties).

clientContext.load(collList, ‘Include(Title, Id)’);

Since I’m working recently quite a lot with JSCOM (mainly in context of Project Server 2013), I was curious what happens when we use this feature from the Managed Client Object Model, and how we could inject the functionality into JSCOM.

Note: Yes, I know that REST makes it possible (via the $filter query option) to filter the items returned, however it has its own limitation, like the lack of the batch requests – e.g. aggregating requests on the client side and sending them in batches via executeQueryAsync – that is (at least, IMHO) one of the best features of JSCOM.

So I’ve created a simple .NET console application that submits the following query (first without, next with filtering) to the server:

ClientContext ctx = new ClientContext(“http://sp2013”);
// without filtering
// var listQuery = ctx.Web.Lists;
// with filtering
var listQuery = ctx.Web.Lists.Where(l => l.Title == "Images");
var list = ctx.LoadQuery(listQuery);

and captured the network traffic using Fiddler in both cases. The screenshot below highlights the difference between the two queries:


The equivalent version of the no-filter query in JavaScript:

  1. var context = SP.ClientContext.get_current();
  2. var lists = context.get_web().get_lists();
  4. $(document).ready(function () {
  5.     getList();
  6. });
  8. function getList() {
  9.     context.load(lists);
  10.     context.executeQueryAsync(onGetListSuccess, onGetListFail);
  11. }

As in the case of the .NET version, this query submits the request without the QueryableExpression. If we want to enable filtering in case of JSCOM, we should first find where the Query part of the request is assembled. Using a simple search in the .js files shows that this task is performed in the SP.ClientQueryInternal.prototype.$2s_1 method in the SP.Runtime.js.

The default implementation of this method is illustrated below (code taken from SP.Runtime.debug.js):

  1. $2s_1: function SP_ClientQueryInternal$$2s_1($p0, $p1) {
  2.     $p0.writeStartElement('Query');
  3.     this.$2r_1($p0, $p1);
  4.     $p0.writeEndElement();
  5.     if (this.$n_1) {
  6.         $p0.writeStartElement('ChildItemQuery');
  7.         this.$n_1.$2r_1($p0, $p1);
  8.         $p0.writeEndElement();
  9.     }
  10. }

We should create our custom override of the $2s_1 method that injects the QueryableExpression part of the query into the ChildItemQuery block (see a similar implementation in the $2w_0 method of the SP.ClientObjectPropertyConditionalScope.prototype in SP.Runtime.debug.js). For example (to filter child items with Title Images”):

  1. SP.ClientQueryInternal.prototype.$2s_1 = function SP_ClientQueryInternal$$2s_1($p0, $p1) {
  2.     $p0.writeStartElement('Query');
  3.     this.$2r_1($p0, $p1);
  4.     $p0.writeEndElement();
  5.     if (this.$n_1) {
  6.         $p0.writeStartElement('ChildItemQuery');
  7.         this.$n_1.$2r_1($p0, $p1);
  9.         // custom code
  11.         $p0.writeStartElement(SP.ClientConstants.QueryableExpression);
  12.         $p0.writeStartElement(SP.ClientConstants.where);
  13.         $p0.writeStartElement(SP.ClientConstants.Test);
  14.         $p0.writeStartElement(SP.ClientConstants.Parameters);
  15.         $p0.writeStartElement(SP.ClientConstants.Parameter);
  16.         $p0.writeAttributeString(SP.ClientConstants.Name, "l");
  17.         $p0.writeEndElement();  // Parameters
  18.         $p0.writeEndElement();  // Parameter
  19.         $p0.writeStartElement(SP.ClientConstants.Body);
  20.         $p0.writeStartElement(SP.ClientConstants.equal);
  21.         $p0.writeStartElement(SP.ClientConstants.expressionProperty);
  22.         $p0.writeAttributeString(SP.ClientConstants.Name, "Title");
  23.         $p0.writeStartElement(SP.ClientConstants.expressionParameter);
  24.         $p0.writeAttributeString(SP.ClientConstants.Name, "l");
  25.         $p0.writeEndElement();  // ExpressionParameter
  26.         $p0.writeEndElement();  // ExpressionProperty
  27.         $p0.writeStartElement(SP.ClientConstants.expressionConstant);
  28.         SP.DataConvert.writeValueToXmlElement($p0, "Images");
  29.         $p0.writeEndElement();  // EQ
  30.         $p0.writeEndElement();  // Body
  31.         $p0.writeEndElement();  // ExpressionConstant
  32.         $p0.writeEndElement();  // Test
  33.         $p0.writeStartElement(SP.ClientConstants.Object);
  34.         $p0.writeStartElement(SP.ClientConstants.queryableObject);
  35.         $p0.writeEndElement();  // QueryableObject
  36.         $p0.writeEndElement();  // Object
  37.         $p0.writeEndElement();  // Where
  38.         $p0.writeEndElement();  // QueryableExpression
  40.         // end custom code
  43.         $p0.writeEndElement();
  44.     }
  45. }

If we define this method in our page as illustrated above, all request sent to the server will filter the child items using the condition defined in the override (Title EQ Images”). That is probably not what we would like to achieve. It would be better to limit the filtering for example to the context the request was sent from.

I’ve found, that the calling SP.ClientContext object is available as this.$0_1.

First, I‘ve created a filterBase function that writes the filtering part of the request stream based on the condition, filter name and filter value parameters:

  1. function filterBase($p0, condition, filterName, filterValue) {
  2.     $p0.writeStartElement(SP.ClientConstants.QueryableExpression);
  3.     $p0.writeStartElement(SP.ClientConstants.where);
  4.     $p0.writeStartElement(SP.ClientConstants.Test);
  5.     $p0.writeStartElement(SP.ClientConstants.Parameters);
  6.     $p0.writeStartElement(SP.ClientConstants.Parameter);
  7.     $p0.writeAttributeString(SP.ClientConstants.Name, "l");
  8.     $p0.writeEndElement();  // Parameters
  9.     $p0.writeEndElement();  // Parameter
  10.     $p0.writeStartElement(SP.ClientConstants.Body);
  11.     $p0.writeStartElement(condition);
  12.     $p0.writeStartElement(SP.ClientConstants.expressionProperty);
  13.     $p0.writeAttributeString(SP.ClientConstants.Name, filterName);
  14.     $p0.writeStartElement(SP.ClientConstants.expressionParameter);
  15.     $p0.writeAttributeString(SP.ClientConstants.Name, "l");
  16.     $p0.writeEndElement();  // ExpressionParameter
  17.     $p0.writeEndElement();  // ExpressionProperty
  18.     $p0.writeStartElement(SP.ClientConstants.expressionConstant);
  19.     SP.DataConvert.writeValueToXmlElement($p0, filterValue);
  20.     $p0.writeEndElement();  // EQ
  21.     $p0.writeEndElement();  // Body
  22.     $p0.writeEndElement();  // ExpressionConstant
  23.     $p0.writeEndElement();  // Test
  24.     $p0.writeStartElement(SP.ClientConstants.Object);
  25.     $p0.writeStartElement(SP.ClientConstants.queryableObject);
  26.     $p0.writeEndElement();  // QueryableObject
  27.     $p0.writeEndElement();  // Object
  28.     $p0.writeEndElement();  // Where
  29.     $p0.writeEndElement();  // QueryableExpression
  30. }

To keep our former example condition, we can call filterBase from a new function filterLists:

  1. function filterLists($p0) {
  2.     filterBase($p0, SP.ClientConstants.equal, "Title", "Images");
  3. }

We assign the filtering function in our getLists method to the context:

  1. function getList() {
  2.     context.load(lists);
  3.     context.filter = filterLists;
  4.     context.executeQueryAsync(onGetListSuccess, onGetListFail);
  5. }

In the new version of the of the $2s_1 method override we check if the current context has a filtering method assigned to, and if one is found, it is called:

  1. SP.ClientQueryInternal.prototype.$2s_1 = function SP_ClientQueryInternal$$2s_1($p0, $p1) {
  2.     $p0.writeStartElement('Query');
  3.     this.$2r_1($p0, $p1);
  4.     $p0.writeEndElement();
  5.     if (this.$n_1) {
  6.         $p0.writeStartElement('ChildItemQuery');
  7.         this.$n_1.$2r_1($p0, $p1);
  9.         // custom code
  11.         var context = this.$0_1;
  12.         if (context.filter != undefined) {
  13.             context.filter($p0);
  14.         }
  16.         // end custom code
  19.         $p0.writeEndElement();
  20.     }
  21. }

OK, we are one step further now, but wouldn’t it be nice, if we could query more objects in the same context (and the same batch) and have the option to set different filters for each one?

I’ve found that the object path property of the client object to be queried is available in the $2s_1 method via this.$G_0.

In this case we should override the $2s_1 method like this:

  1. SP.ClientQueryInternal.prototype.$2s_1 = function SP_ClientQueryInternal$$2s_1($p0, $p1) {
  2.     $p0.writeStartElement('Query');
  3.     this.$2r_1($p0, $p1);
  4.     $p0.writeEndElement();
  5.     if (this.$n_1) {
  6.         $p0.writeStartElement('ChildItemQuery');
  7.         this.$n_1.$2r_1($p0, $p1);
  9.         // custom code
  11.         var objectPathProp = this.$G_0;
  12.         if (objectPathProp.filter != undefined) {
  13.             objectPathProp.filter($p0);
  14.         }
  16.         // end custom code
  19.         $p0.writeEndElement();
  20.     }
  21. }

The object path property is available as myObject.$5_0.$e_0 property of the client object (for example, lists.$5_0.$e_0), so we can rewrite our methods as illustrated below. In this case we extend our example with a further object (groups) that should be filtered by the filterGroups method (OwnerTitle property of the group EQ "DevSite Owners"). The getList function was also renamed to getObjects.

  1. var context = SP.ClientContext.get_current();
  2. var lists = context.get_web().get_lists();
  3. var groups = context.get_web().get_siteGroups();
  5. $(document).ready(function () {
  6.     getObjects();
  7. });
  9. function filterLists($p0) {
  10.     filterBase($p0, SP.ClientConstants.equal, "Title", "Images");
  11. }
  13. function filterGroups($p0) {
  14.     filterBase($p0, SP.ClientConstants.equal, "OwnerTitle", "DevSite Owners");
  15. }
  17. function getObjects() {
  18.     context.load(lists);
  19.     context.load(groups);
  21.     lists.$5_0.$e_0.filter = filterLists;
  22.     groups.$5_0.$e_0.filter = filterGroups;
  23.     context.executeQueryAsync(onGetListSuccess, onGetListFail);
  24. }

The following Fiddler screenshot shows the resulting query:


This kind of overrides provides already a quite flexible “framework” for filtering objects on the server side using the JSCOM, making this important feature available for JavaScript developers as well. It would be nice if Microsoft would provide a similar, but official solution to this issue in a forthcoming service pack.

We can see from this experiment as well, that there is much more power available in the XML / JSON communication protocol behind the client object models, than it is made public by this APIs, so there is yet place for improvements. You can find valuable information related to the SharePoint Client Query Protocol here.

Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at

%d bloggers like this: