ECMAScript Client Object Model | Second Life of a Hungarian SharePoint Geek

March 26, 2018

Querying SharePoint list items including their attachment in a single request from the JavaScript Client Object Model

Filed under: CAML, ECMAScript Client Object Model, JSCOM, SP 2013 — Tags: CAML, ECMAScript Client Object Model, JSCOM, SP 2013 — Peter Holpar @ 23:31

Assume there is a SharePoint list including a few hundreds of items, each item might have a few attachments, but there might be items without attachments as well. We need to display only some of the items (filtered dynamically based on specific conditions) for the users on a web page. We use already the JavaScript client object model on that page, so it would be ideal to keep this technology, instead of involving another one (like REST or SharePoint web services) just because this task.

I found several useful pages while looking for a solution for my problem, like this one about using CAML from the client object model, although some of the statements in the sections “Include attachment URLs” and “Limitations” turned to be a bit misleading later.

Few posts and forum answers suggested to query the items first using a CAML query, then read the attachments for each matching item in subsequent requests individually. That would not perform very well if there are a lot of matching item and is generally not a nice solution. Another suggestion I often found is to query the items with a CAML query (just like in the previous case), then query all of the attachments (they are stored in a folder called Attachments n the list with a separate subfolder, like 1, 2, etc. for each list items) in a second query, then match the attachments files to items based on the relative path of the attachments (attachment for the list item with ID 1 are in the folder /Attachments/1, etc.). If you have a lot of items having a few attachments for each item, that might have performance problem as well, and the implementation itself is challenging.

Vadim Gremyachev posted a solution using the managed client object model with a single request only on SharePoint Stack Exchange. In his solution, he uses the following line to create an execute the CAML query:

var items = list.GetItems(CamlQuery.CreateAllItemsQuery());

The static CreateAllItemsQuery method of the CamlQuery object is only an easy way to create a CAML query having the RecursiveAll view scope, it means, content of the subfolders (if there is any) would be searched through either.

public static CamlQuery CreateAllItemsQuery()
{
return new CamlQuery { ViewXml = "<View Scope=\"RecursiveAll\">\r\n <Query>\r\n </Query>\r\n</View>" };
}

As we have a flat list, without any folders, I’ve slightly modified Vadim’s code first:

var siteUrl = "http://YourSharePoint/site/subsite&quot;;
using (var ctx = new ClientContext(siteUrl))
{
    var web = ctx.Web;
    var list = web.Lists.GetByTitle("TitleOfYourList");
    // Vadim's version fo a list with a folder struture
    //var items = list.GetItems(CamlQuery.CreateAllItemsQuery());
    // my version for a flat list without folders
    var items = list.GetItems(new CamlQuery());
    ctx.Load(items, icol => icol.Include(i => i["Title"], i => i.AttachmentFiles));
    ctx.ExecuteQuery();
 
    //Print attachment Urls for list items
    foreach (var item in items)
    {
        if (item.AttachmentFiles.Count > 0)
        {
            Console.WriteLine("{0} attachments:", item["Title"]);
            foreach (var attachment in item.AttachmentFiles)
            {
                Console.WriteLine(attachment.ServerRelativeUrl);
            }
        }
        else
        {
            Console.WriteLine("No attachments were found for list item '{0}' ", item["Title"]);
        }
    } 
}

It resulted in the following request sent to the server, captured by Fiddler:

Note, that only the properties Title and AttachmentFiles were requested in the Include statement, and the ViewXml is empty.

We wanted to reproduce the same request using the JavaScript object model, but haven’t found any solution immediately ready to use, so first made a quick check, which alternative solutions are available in JavaScript, if we were not able to transform the managed code sample to JavaScript. Let’s see what we found.

I found a jQuery-based approach here, that utilizes the GetListItems method of the Lists SharePoint web service:

$("table.ms-listviewtable tr:has(td.ms-vb2)").find("td:first").filter(function() {
    return $(this).text().toLowerCase() == ID;
}).html("<img src='" + url[1] + "' width=150 height=100 />");
 
$(document).ready(function() {
    querySPWebServices();
});
 
function querySPWebServices() {
    var soapEnv = "<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/'&gt; \\par         <soapenv:Body> \\par         <GetListItems xmlns='http://schemas.microsoft.com/sharepoint/soap/'&gt; \\par         <listName>Projects</listName> \\par         <viewFields> \\par         <ViewFields> \\par         <FieldRef Name='Title' /> \\par         <FieldRef Name='Body' /> \\par         <FieldRef Name='ID' /> \\par         <FieldRef Name='Attachments' /> \\par         </ViewFields> \\par         </viewFields> \\par         <query> \\par         <Query /> \\par         </query> \\par         <queryOptions> \\par         <QueryOptions> \\par         <IncludeAttachmentUrls>TRUE</IncludeAttachmentUrls> \\par         </QueryOptions> \\par         </queryOptions> \\par         </GetListItems> \\par         </soapenv:Body> \\par         </soapenv:Envelope>";
 
    $.ajax({
        async: false,
        url: "http://server.name.com/site/_vti_bin/lists.asmx&quot;,
        type: "POST",
        dataType: "xml",
        data: soapEnv,
        complete: processResult,
        contentType: "text/xml; charset=\"utf-8\""
    });
};
 
function processResult(xData, status) {
    $(xData.responseXML).find("z\\:row").each(function() {
        if ($(this).attr("ows_Attachments") != 0) {
            var url = $(this).attr("ows_Attachments").replace(/#/g, "").split(';');
            var ID = $(this).attr("ows_ID");
            $("table.ms-listviewtable tr:has(td.ms-vb2)").find("td:first").filter(function() {
                return $(this).text().toLowerCase() == ID;
            }).html("<img src='" + url[1] + "' width=150 height=100 />");
        };
    });
};

Another solution invoking the GetListItems method as well, but this time using SPServices can be found here:

var p = $().SPServices({
    operation: "GetListItems",
    listName: "listname",
    CAMLQueryOptions: "<QueryOptions>" +
            "<IncludeAttachmentUrls>TRUE</IncludeAttachmentUrls>" +
        "</QueryOptions>"
});
  
p.done(function() {
  
    $(p.responseXML).).SPFilterNode("z:row").each(function() {
      
        var attachments = [];
  
        var att = $(this).attr("ows_Attachments");
        if(att !== "0") {
            attachments = att.split(";#"); // Now you'll have an array of attachment URLs
        } else {
            // att will be "0", indicating that there are no attachments
        }
      
    });
  
});

You could query list items with their attachments via the REST interface as well:

http://YourSharePoint/site/subsite/_api/Web/Lists/GetByTitle('TitleOfYourList')/Items/?$expand=AttachmentFiles

One can use CAML in conjunction with REST by using POST requests and the GetItems method as described here, but as far as I see, the $expand=AttachmentFiles plays not well with the GetItems method. Instead of CAML, you can add the condition, ordering and row limit attributes of your query via $filter, $orderby and $top to the REST query as described here.

After this short overview, let’s try to rewrite our C# code above in JavaScript.

First, I came up with a query like the one below. I requested the very same properties, that we had in the managed client object version: the Title and AttachmentFiles. Note, that I omitted all of the filters I needed in the case of real life list from the CAML query due to the readability as this part is not relevant now in solving the main problem. In contrast to the managed client object code above, I restricted the fields in the CAML query as well via the ViewFields element. That was a mistake, as it turned out later, but I’m a strong believer, that mistakes are there to teach us something new.

var list = web.get_lists().getByTitle("TitleOfYourList");
 
var camlQuery = new SP.CamlQuery();
var query = '<View><ViewFields><FieldRef Name="Title"/><FieldRef Name="AttachmentFiles"/></ViewFields></View>';
camlQuery.set_viewXml(query);
 
var items = list.getItems(camlQuery);
ctx.load(items, 'Include(Title, AttachmentFiles)');

The result was not satisfying, the request failed (screenshot from Fiddler):

ErrorMessage: Value does not fall within the expected range.

ErrorTypeName: System.ArgumentException

ErrorStackTrace:

Microsoft.SharePoint.SPFieldMap.GetColumnNumber(String strFieldName, Boolean bThrow)
   at Microsoft.SharePoint.SPListItemCollection.GetColumnNumber(String groupName, Boolean bThrowException)
   at Microsoft.SharePoint.SPListItem.GetValue(SPField fld, Int32 columnNumber, Boolean bRaw, Boolean bThrowException)
   at Microsoft.SharePoint.SPListItem.get_Attachments()
   at Microsoft.SharePoint.SPAttachmentCollection_Client.<GetEnumerator>d__0.MoveNext()
   at Microsoft.SharePoint.Client.ServerStub.<EnumerateChildItems>d__36.MoveNext()
   at Microsoft.SharePoint.Client.ServerStub.WriteChildItems(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJson(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJsonWithMonitoredScope(JsonWriter writer, Object value, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.ServerStub.SPListItemServerStub.WriteOnePropertyValueAsJson(JsonWriter writer, Object target, ClientQueryProperty field, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJson(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteChildItems(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJson(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJsonWithMonitoredScope(JsonWriter writer, Object value, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.WriteQueryResults(Object obj, ClientObjectQuery objQuery)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.ProcessQuery(XmlElement xe)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.ProcessOne(XmlElement xe)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.ProcessStatements(XmlNode xe)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.Process()

You can see the getter method of the Attachments property of the SPListItem object in the fourth line of the stack trace. Let’s have a quick view, how this method works:

public SPAttachmentCollection get_Attachments()
{
   if (this.m_Attachments == null)
   {
       string pbstrUrlPrefix = null;
       object[,] objArrAttachments = null;
       uint pdwRowCount = 0;
       bool bDoQuery = false;
       if (!this.HasExternalDataSource && (this.ID != 0))
       {
           object obj3;
           object obj2 = this.GetValue("Attachments");
           if (obj2 != null)
           {
               bDoQuery = obj2.ToString() != "0";
           }
           SPSecurity.SetListInHttpContext(HttpContext.Current, this.m_Items.List.InternalName);
           SPWeb web = this.Web;
           web.Request.GetAttachmentsInfo(web.Url, this.m_Items.List.InternalName, Convert.ToInt32(this.ID), bDoQuery, out pbstrUrlPrefix, out pdwRowCount, out obj3);
           if (bDoQuery)
           {
               objArrAttachments = (object[,]) obj3;
           }
       }
       this.m_Attachments = new SPAttachmentCollection(this, pbstrUrlPrefix, pdwRowCount, objArrAttachments);
   }
   return this.m_Attachments;
}

As we’ve learnt from Karine’s post, the Attachments field is a flag indicating if a list item has any attachments or not. Obviously, the getter method depends on this property (see the line object obj2 = this.GetValue("Attachments"); in code above), and we should request this property explicitly in our code.

That is the new version of my query:

var camlQuery = new SP.CamlQuery();
var query = '<View><ViewFields><FieldRef Name="Title"/><FieldRef Name="Attachments"/><FieldRef Name="AttachmentFiles"/></ViewFields></View>';
camlQuery.set_viewXml(query);
 
var items = list.getItems(camlQuery);
ctx.load(items, 'Include(Title, Attachments, AttachmentFiles)');

The request was captured by Fiddler as shown bellow. See the properties included in the Include statement, as well the ViewXml:

This version works already as expected, but comparing the request with the one captured at C#-based version (see above) I found, that one can omit the Attachments field from the Include statement without any problem.

var camlQuery = new SP.CamlQuery();
var query = '<View><ViewFields><FieldRef Name="Title"/><FieldRef Name="Attachments"/><FieldRef Name="AttachmentFiles"/></ViewFields></View>';
camlQuery.set_viewXml(query);
 
var items = list.getItems(camlQuery);
ctx.load(items, 'Include(Title, AttachmentFiles)');

Furthermore, there is no problem, if you omit the ViewFields element entirely from the CAML query (in our case, we needed later other parts of the CAML query, like the Where element to build up the filters, but it is irrelevant now). That is actually the same query we had with the managed client object model sample at the beginning of this post.

var list = web.get_lists().getByTitle("TitleOfYourList");
 
var camlQuery = new SP.CamlQuery();
 
var items = list.getItems(camlQuery);
ctx.load(items, 'Include(Title, AttachmentFiles)');

Note: Using of the ViewFields element might have an effect on the performance of the query on the server side (especially if you have a lot of columns in your list and you need only a few of them) and so on the response time as well, but after the query has been completed on the server, it seems to be irrelevant regarding to the network bandwidth usage.

That is the full version of the code (to be exact, one of the working ones), including displaying the information (this time in the console only) returned by the server:

var ctx = SP.ClientContext.get_current();
var web = ctx.get_web();
 
var list = web.get_lists().getByTitle("TitleOfYourList");
 
var camlQuery = new SP.CamlQuery();
 
var items = list.getItems(camlQuery);
ctx.load(items, 'Include(Title, Attachments, AttachmentFiles)');
 
ctx.executeQueryAsync(
    function () {
        var enumerator = items.getEnumerator();
        while (enumerator.moveNext()) {
            var item = enumerator.get_current();
            console.log(item.get_item('Title'));
            if (item.get_item('Attachments')) {
                var attEnumerator = item.get_attachmentFiles().getEnumerator();
                while (attEnumerator.moveNext()) {
                    var attachment = attEnumerator.get_current();
                    console.log(attachment.get_serverRelativeUrl()); // or get_fileName()
                }
            }
        }
    },
    function (sender, args) {
        console.log('Request failed. ' + args.get_message() + '\n' + args.get_stackTrace());
    }
);

That means, we fulfilled our goal by returning the items and their attachments in a single request/response via the JavaScript client object model.

Note, that after the response returned, we first check if there are any attachments for the item by reading the value of item.get_item(‘Attachments’), then accessing the attachments via the item.get_attachmentFiles() method. You could, however, check the existence of the attachments simply by using the get_attachmentFiles() method, without the Attachments field, like this:

var af = item.get_attachmentFiles();
if (af) {
var attEnumerator = af.getEnumerator();

After extending the CAML query with the filters I need in the Where element, the query returned only the filtered items without any negative effect of the existing functionality, so that part of the task was completed either.

Lessons learned: If you need to read the attachments of your items, you must include the AttachmentFiles field in your Include statement. Including the Attachments field in the Include statement is optional, you need it only if you would like to use it explicitly (see the condition if (item.get_item(‘Attachments’)) in the last code snippet), otherwise you can omit it. If you would like to restrict the fields via the ViewFields element of your CAML query, you should include both of the Attachments and AttachmentFiles fields, but if you don’t restrict the columns by the usage of a ViewFields element, it works as well.

One final note: As so many times before, Fiddler was an invaluable tool to understand how the various versions work, or actually how they don’t work, by comparing the requests sent by the codes down to the wire.

Leave a Comment

October 19, 2015

Displaying Notifications and Status Messages from a SharePoint-based AngularJS Application Including a FormController

Filed under: AngularJS, ECMAScript Client Object Model, SP 2013 — Tags: AngularJS, ECMAScript Client Object Model, SP 2013 — Peter Holpar @ 21:54

Assume the following requirements: We should create a Single Page Application (SPA) (no SharePoint App!) that reads data from SharePoint using the JavaScript client object model, allows the user to edit the values, displays if a field value was changed (if it is “dirty” vs. “pristine”), performs data validations (if it is “valid” vs. “invalid”) and lets the user to save the changes. AngularJS was selected as the presentation framework for the SPA. On data save, we should give the users feedback on the progress (like “Saving changes…”, “Save completed” or “Error during the save operation”) via standard SharePoint notifications and status messages.

Challenge 1: The FormController of the AngularJS framework is based on the form HTML element. That means, if we would like to use the features of the FormController, like dirty / pristine, validation, etc., we should include a form element in our HTML application. However, our SharePoint page is an ASP.NET page, that already contains a form element, and it does not like to include multiple ones.

Solution 1: Although there are tricks to bypass this limitation (like this or this one), I chose another way to go. I’ve included a Page Viewer Web Part that displays a “pure” HTML page that is stored in a document library in SharePoint, as well as any other non-standard artifacts of the application (.js and .css files, etc.). This HTML page – displayed in an IFRAME by the Page Viewer Web Part – contains the form element, that does not interfere with the form element on the ASP.NET page.

You can display a notification by calling the SP.UI.Notify.addNotification method, similarly a status message is displayed via the SP.UI.Status.addStatus method. Both of these Notify and Status classes are defined in the SP.UI namespace in the sp.js (and its debug version in sp.debug.js). This JavaScript file is typically referenced in the standard SharePoint pages, however you should add a reference to it in your custom pages, like in the case of our HTML page. If you forget to add the reference, you will get an error like this one:

TypeError: Unable to get property ‘addNotification’ of undefined or null reference

Challenge 2: There is no notification / status message displayed, even if you add the reference to the sp.js. The reason of the problem is, that the HTML elements required by these methods are defined in the master page of the standard SharePoint sites. Obviously, these elements are not found in our custom page in the IFRAME, so the messages are not displayed.

Solution 2: I’ve found two similar blog posts (this one and this one) describing a similar issue with IFRAME and notification messages in the case of Client App Parts. The first of this two posts states that the problem is the IFRAME itself, that prohibits the communication between the parent page and the IFRAME. Of course, that is wrong. The real reason is the different domain names in the URL of the app part (IFRAME) and the host page, as correctly stated in the second post. If we have the same domain name (and we do have in this case), we do not need the rather complex approach described by the posts(that is still valid for the Client App Parts). Displaying a notification / status message from the script included in the HTML page in the IFRAME in our case is so simple as to prepend the text ‘parent.’ before the method invocation, for example:

var notifyId = parent.SP.UI.Notify.addNotification("Saving…", true);

Of course, in this case you are using the JavaScript and HTML objects on the parent page, so you don’t need to reference the sp.js in your HTML page.

Leave a Comment

July 16, 2015

How to Read Project Properties that are not Available in the Client Object Model?

Filed under: ECMAScript Client Object Model, Project Server, PS 2013, PSI — Tags: ECMAScript Client Object Model, Project Server, PS 2013, PSI — Peter Holpar @ 22:23

Recently I had a development task that at the first sight seemed to be trivial, but it turned out quickly to be rather a challenge. I had to display some basic project information on a page in our Project Web Site (PWS), like project start and finish date, remaining work and percent complete. The web page was built using client-side technologies, like the Client-side object model (CSOM) for Project 2013 and using the AngularJS library, and we did not plan to change the architecture to server side code.

If you check the properties of the PublishedProject (either on the client side in namespace / assembly Microsoft.ProjectServer.Client or on the server side in Microsoft.ProjectServer), you see that it has properties like StartDate and FinishDate, and it inherits its PercentComplete property from the Project base class, however there is no property for RemainingWork or PercentWorkComplete defined, although both of these values are available as fields if you manage a Project Server view (see screenshot below). This information is not available via REST / OData either.

You should know, that in the case of Project Server, the server side OM is simply a wrapper around the PSI, for example, the PercentComplete property in the Project class is defined:

public int PercentComplete
{
get
{
    ProjectDataSet.TaskRow summaryTaskRow = this.SummaryTaskRow;
    if (summaryTaskRow != null && !summaryTaskRow.IsTASK_PCT_COMPNull())
      return summaryTaskRow.TASK_PCT_COMP;
    else
      return 0;
}
}

Client side OMs (either managed or ECMAScript) and REST calls invoke the server side OM, so at the background the good old PSI is still in action.

It seems that the developers of Project Server remained simply not enough time to map all of the fields available via PSI to the object models on the server side and the client side.

You should know either, that the project properties we need are stored as task properties for the project summary task of the current project. In the Project Server database the tasks of the published projects (so the project summary tasks as well) are stored in the [pub].[MSP_TASKS] table. If you run the following query (where ProjectWebApp is the name of the database and the Guid in the [PROJ_UID] filter is the ID of your project), you find some specific field values that may help to identify the summary task record of a project:

SELECT [TASK_UID]
      ,[TASK_PARENT_UID]
      ,[TASK_ID]
      ,[TASK_OUTLINE_NUM]
      ,[TASK_OUTLINE_LEVEL]
      ,[TASK_NAME]
      ,[TASK_START_DATE]
      ,[TASK_FINISH_DATE]
      ,[TASK_PCT_COMP]
      ,[TASK_PCT_WORK_COMP]
      ,[TASK_REM_WORK]
FROM [ProjectWebApp].[pub].[MSP_TASKS]
WHERE [PROJ_UID] = ‘d0ae5086-be7a-e411-9568-005056b45654’

The project summary task record – at least, based on my experimental results – , matches the following conditions:

[TASK_ID] = 0

[TASK_OUTLINE_NUM] = 0

[TASK_OUTLINE_LEVEL] = 0

[TASK_UID] = [TASK_PARENT_UID]

But as said, we need a solution on the client side, and obviously one that does not tamper with the Project Server database. What options are there to achieve the missing information?

The Project class has a property called SummaryTaskId, but if you have this value already, and would like to query the project tasks via REST (for example: http://YourProjServer/PWA/_api/ProjectServer/Projects(‘d0ae5086-be7a-e411-9568-005056b45654’)/Tasks(‘FFAE5086-BE7A-E411-9568-005056B45654’)) or via the client object model, the result is empty. The description of the SummaryTaskId property says: “Gets the GUID for the hidden project summary task”. Yes, it is so hidden, that it simply not included in the Tasks collection of the Project class! The Tasks property of the PublishedProject class is of type PublishedTaskCollection, and on the server side the record for the project summary task is simply filtered out, when initializing the internal Dictionary used for the storage of the Task records. If you don’t believe me, or need more details on that, see the constructor method of Microsoft.ProjectServer.PublishedTaskCollection class below:

internal PublishedTaskCollection()
{
    Func<Dictionary<Guid, PublishedTask>> valueFactory = null;
    if (valueFactory == null)
    {
        valueFactory = () => base.ProjectData.Task.OfType<ProjectDataSet.TaskRow>().Where<ProjectDataSet.TaskRow>(delegate (ProjectDataSet.TaskRow r) {
            if (!r.IsTASK_PARENT_UIDNull())
            {
                return (r.TASK_PARENT_UID != r.TASK_UID);
            }
            return true;
        }).ToDictionary<ProjectDataSet.TaskRow, Guid, PublishedTask>(r => r.TASK_UID, r => this.CreateTask(r));
    }
    this._tasks = new Lazy<Dictionary<Guid, PublishedTask>>(valueFactory);
}

Of course, we get the same, empty result if we would like to filter the tasks for one the special conditions we found in the database (like [TASK_OUTLINE_LEVEL] = 0):
http://YourProjServer/PWA/_api/ProjectServer/Projects(‘d0ae5086-be7a-e411-9568-005056b45654’)/Tasks?$filter=OutlineLevel eq 0

The project reporting data contains the project summary tasks as well, so we could invoke the ProjectData OData endpoint from the client side to query the required information. The problem with this approach is that it would require extra permissions on the reporting data and one cannot limit this permission to the summary tasks of a specific project, to summary tasks, or just to tasks at all. If you grant your users the Access Project Server Reporting Service global permission, they can query all of the reporting data. It is sure not our goal, but you can test it if you wish.

Once you have the ID of the project summary task (for example via the SummaryTaskId property), the task is available via a query like this one:

http://YourProjServer/PWA/_api/ProjectData/Tasks(ProjektID=guid’d0ae5086-be7a-e411-9568-005056b45654′,TaskID=guid’FFAE5086-BE7A-E411-9568-005056B45654′)

When using PSI, we can access the required information via the TASK_REM_WORK and TASK_PCT_WORK_COMP fields in ProjectDataSet.TaskRow, that means, rows in the Task property (type of ProjectDataSet.TaskDataTable) of the ProjectDataSet. The first row in the record set contains the information about the project summary task.

We could create our own extensions for the client object model (wrapping around just this piece of PSI), as I illustrated for the managed, and for the ECMAScript object model as well, but it would require a lot of work, so I ignored this option for now. Instead of this, I’ve created a simple .NET console application utilizing the PSI (see the most important part of the code below). Unfortunately, I have not found a method that returns only a specific task of a specific project, so I had to call the ReadProjectEntities method to read all of the tasks of the project.

_projectClient = new SvcProject.ProjectClient(ENDPOINT_PROJECT, pwaUrl + "/_vti_bin/PSI/ProjectServer.svc");
_projectClient.ClientCredentials.Windows.AllowedImpersonationLevel = System.Security.Principal.TokenImpersonationLevel.Impersonation;
 
Guid projId = Guid.Parse("d0ae5086-be7a-e411-9568-005056b45654");
int taskEntityId = 2;
 
var projEntitiesDS = _projectClient.ReadProjectEntities(projId, taskEntityId, SvcProject.DataStoreEnum.PublishedStore);
var tasksTable = projEntitiesDS.Task;
 
foreach (SvcProject.ProjectDataSet.TaskRow task in tasksTable.Rows)
{
    Console.WriteLine(string.Format("TASK_OUTLINE_NUM: {0}; TASK_PCT_WORK_COMP: {1}; TASK_REM_WORK: {2}", task.TASK_OUTLINE_NUM, task.TASK_PCT_WORK_COMP, task.TASK_REM_WORK));
}

I’ve captured the request and the response using Fiddler:

Then extended my JavaScript code with methods that assemble the request in the same format, submit it to the server, then parse the required fields out of the response.

First, I needed a helper method to format strings:

String.format = (function () {
    // The string containing the format items (e.g. "{0}")
    // will and always has to be the first argument.
    var result = arguments[0];
 
    // start with the second argument (i = 1)
    for (var i = 1; i < arguments.length; i++) {
        // "gm" = RegEx options for Global search (more than one instance)
        // and for Multiline search
        var regEx = new RegExp("\\{" + (i – 1) + "\\}", "gm");
        result = result.replace(regEx, arguments[i]);
    }
 
    return result;
});

In my Angular controller I defined this function to format dates:

$scope.formatDate = function (date) {
    var formattedDate = '';
    if ((typeof date != "undefined") && (date.year() > 1)) {
        formattedDate = String.format("{0}.{1}.{2}", date.year(), date.month() + 1, date.date());
    }
 
    return formattedDate;
}

Next, in the controller we get the ID of the project for the current PWS, and we read project properties that are available via the client object model, and finally the ones, that are available only via PSI:

var promiseWebProps = ProjService.getWebProps($scope);
promiseWebProps.then(function (props) {
    $scope.projectId = props.projectId;
 
    // read the project properties that are available via the client object model
    var promiseProjProp = ProjService.getProjectProps($scope);
    promiseProjProp.then(function (props) {
        $scope.projStartDate = moment(props.projStartDate);
        $scope.projFinishDate = moment(props.projFinishDate);
        $scope.percentComp = props.percentComp;
    }, function (errorMsg) {
        console.log("Error: " + errorMsg);
    });
 
    // read the project properties that are available only via PSI
    var promiseProjPropEx = ProjService.getProjectPropsEx($scope);
    promiseProjPropEx.then(function (propsEx) {
        $scope.remainingWork = Math.round(propsEx.remainingWork / 600) / 100;
        $scope.percentWorkComp = propsEx.percentWorkComp;
    }, function (errorMsg) {
        console.log("Error: " + errorMsg);
    });
 
}, function (errorMsg) {
    console.log("Error: " + errorMsg);
});

As you can see, the value we receive in the remainingWork property should be divided by 600 and 100 to get the value in hours.

In our custom ProjService service I’ve implemented the corresponding methods.

The project ID is stored in the property bag of the PWS in a property called MSPWAPROJUID (see this post about how to read property bags from the client object model):

this.getWebProps = function ($scope) {
    var deferred = $q.defer();
 
    var ctx = SP.ClientContext.get_current();
 
    var web = ctx.get_web();
    var props = web.get_allProperties();
    ctx.load(props);
 
 
    ctx.executeQueryAsync(
        function () {
            var allProps = props.get_fieldValues();
 
            deferred.resolve(
                {
                    projectId: allProps.MSPWAPROJUID
                });
        },
        function (sender, args) {
            deferred.reject('Request failed. ' + args.get_message() + '\n' + args.get_stackTrace());
        }
    );
 
    return deferred.promise;
};

Having the project ID, reading project properties via the client object model should be straightforward as well:

this.getProjectProps = function ($scope) {
    var deferred = $q.defer();
 
    var ctx = SP.ClientContext.get_current();
 
    var projContext = PS.ProjectContext.get_current();
 
    projContext.set_isPageUrl(ctx.get_isPageUrl);
    var proj = projContext.get_projects().getById($scope.projectId);
    projContext.load(proj, "StartDate", "FinishDate", "PercentComplete");
 
    projContext.executeQueryAsync(
        function () {
            deferred.resolve({
                projStartDate: proj.get_startDate(),
                projFinishDate: proj.get_finishDate(),
                percentComp: proj.get_percentComplete()
            });
        },
        function (sender, args) {
            deferred.reject('Request failed. ' + args.get_message() + '\n' + args.get_stackTrace());
        }
    );
 
    return deferred.promise;
};

Reading the ‘extra’ properties via PSI is a bit more complex. First, we assemble the request XML as we captured it with Fiddler when used the console application mentioned above, and post it to the server. Next, we process the response (see the code of the helper method buildXMLFromString farther below), and parse out the necessary properties from the project summary task (that is the Task node having rowOrder = 0) using XPath queries.

this.getProjectPropsEx = function () {
    var deferred = $q.defer();
   
    // assuming your PWA is located at /PWA
    var psiUrl = String.format("{0}//{1}/PWA/_vti_bin/PSI/ProjectServer.svc", window.location.protocol, window.location.host);
   
    $http({
        method: 'POST',
        url: psiUrl,
        data: String.format('<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot; xmlns:xsd="http://www.w3.org/2001/XMLSchema"><ReadProjectEntities xmlns="http://schemas.microsoft.com/office/project/server/webservices/Project/"><projectUid&gt;{0}</projectUid><ProjectEntityType>2</ProjectEntityType><dataStore>PublishedStore</dataStore></ReadProjectEntities></s:Body></s:Envelope>', $scope.projectId),
        headers: {
            "Content-Type": 'text/xml; charset=utf-8',
            "SOAPAction": "http://schemas.microsoft.com/office/project/server/webservices/Project/ReadProjectEntities&quot;
        }
    }).success(function (data) {
        var dataAsXml = buildXMLFromString(data);
        dataAsXml.setProperty('SelectionLanguage', 'XPath');
        dataAsXml.setProperty('SelectionNamespaces', 'xmlns:pds="http://schemas.microsoft.com/office/project/server/webservices/ProjectDataSet/&quot; xmlns:msdata="urn:schemas-microsoft-com:xml-msdata"');
        var projSumTaskNode = dataAsXml.selectSingleNode("//pds:Task[@msdata:rowOrder=0]");
        var remainingWork = projSumTaskNode.selectSingleNode("pds:TASK_REM_WORK").nodeTypedValue;
        var percentWorkComp = projSumTaskNode.selectSingleNode("pds:TASK_PCT_WORK_COMP").nodeTypedValue;
        deferred.resolve(
            {
                remainingWork: remainingWork,
                percentWorkComp: percentWorkComp
            });
    })
    .error(function (data, status) {
        deferred.reject('Request failed. ' + data);
    });
    
    return deferred.promise;
}

These are the helper methods I used for processing the response text as XML:

function createMSXMLDocumentObject() {
    if (typeof (ActiveXObject) != "undefined") {
        // http://blogs.msdn.com/b/xmlteam/archive/2006/10/23/using-the-right-version-of-msxml-in-internet-explorer.aspx
        var progIDs = [
                        "Msxml2.DOMDocument.6.0",
                        "Msxml2.DOMDocument.3.0",
                        "MSXML.DOMDocument"
        ];
        for (var i = 0; i < progIDs.length; i++) {
            try {
                return new ActiveXObject(progIDs[i]);
            } catch (e) { };
        }
    }
 
    return null;
}
 
function buildXMLFromString(text) {
    var xmlDoc;
 
    xmlDoc = createMSXMLDocumentObject();
    if (!xmlDoc) {
        alert("Cannot create XMLDocument object");
        return null;
    }
 
    xmlDoc.loadXML(text);
 
    var errorMsg = null;
    if (xmlDoc.parseError && xmlDoc.parseError.errorCode != 0) {
        errorMsg = "XML Parsing Error: " + xmlDoc.parseError.reason
                    + " at line " + xmlDoc.parseError.line
                    + " at position " + xmlDoc.parseError.linepos;
    }
    else {
        if (xmlDoc.documentElement) {
            if (xmlDoc.documentElement.nodeName == "parsererror") {
                errorMsg = xmlDoc.documentElement.childNodes[0].nodeValue;
            }
        }
        else {
            errorMsg = "XML Parsing Error!";
        }
    }
 
    if (errorMsg) {
        alert(errorMsg);
        return null;
    }
 
    return xmlDoc;
}

Having an HTML template like this one:

<div><span>% complete:</span><span>{{percentComp}}%</span></div>
<div><span>% work complete:</span><span>{{percentWorkComp}}%</span></div>
<div><span>Remaining work:</span><span>{{remainingWork}} Hours</span></div>
<div><span>Project start:</span><span>{{formatDate(projStartDate)}}</span></div>
<div><span>Project finish:</span><span>{{formatDate(projFinishDate)}}</span></div>

the result should be displayed similar to this one:

A drawback of this approach (not to mention the fact that it is pretty hacky) is, that due the ReadProjectEntities method, all of the fields of all of the project tasks should be downloaded to the client, although we need only a few fields of a single task, the project summary task. So it would make sense to implement some kind of caching on the client side, but it is out of the scope of this post. But as long as Microsoft does not provide all the project fields in the client object model, I have not found any better solution that would require a relative small effort.

Leave a Comment

July 6, 2015

Accessing and Manipulating Property Bags via the ECMAScript Client Object Model

Filed under: ECMAScript Client Object Model, JavaScript, SP 2013 — Tags: ECMAScript Client Object Model, JavaScript, SP 2013 — Peter Holpar @ 21:39

Recently I work a lot with web applications implemented on the client side with JavaScript, mostly using the AngularJS library. Rather often should I build the application logic on the values stored in the property bags of the web objects, so I decided to sum up the experience I made in this field.

Note: A similar blog entry discussing the same topic can be found here. It might be useful to read that one as well, but I include additional info in my entry as well.

Note 2: The code samples in my post are borrowed from our custom AngularJS service, but the bulk of them should be reusable for any kind of JavaScript solution.

The first example shows how to retrieve the property values:

this.readSettings = function ($scope) {
    var deferred = $q.defer();
 
    var ctx = SP.ClientContext.get_current();
 
    var web = ctx.get_web();
    var props = web.get_allProperties();
    ctx.load(props);
 
    ctx.executeQueryAsync(
        function () {
            // you receive an error if the property is not defined for the web:
            // The property or field has not been initialized. It has not been requested or the request has not been executed. It may need to be explicitly requested.
            var yourProperty = props.get_item("YourProperty");
            // or alternativelyyou can use
            //var allProps = props.get_fieldValues();
            //var yourProperty = allProps.YourProperty;
            // in this second case you can  check, if the property is defined … 
            //if (typeof (yourProperty) == "undefined") {
            //    console.log("The property is undefined");
            //}
            //…or iterate through all the properties
            //for (var property in allProps) {
            //    if (allProps.hasOwnProperty(property)) {
            //        console.log(String.format("{0}: {1}", property, allProps[property]));
            //    }
            //}
 
            deferred.resolve(
                {
                    projectId: projectId
                });
        },
        function (sender, args) {
            deferred.reject('Request failed. ' + args.get_message() + '\n' + args.get_stackTrace());
        }
    );
 
    return deferred.promise;
};

If you trace the network traffic with Fiddler, you will capture the following (or similar) request-response:

As you can see, all of the properties from the bag are returned by the server, they are in the form of simple property name – property value pairs (see response for Query with Id=6).

When you already have the response from the server, there are two different methods to get the value of a specific property. There is a very important difference between the two versions (get_item method vs. get_fieldValues method). If the property is undefined in the property bag, you receive an error, if you would like to read the value via props.get_item("YourProperty"):

The property or field has not been initialized. It has not been requested or the request has not been executed. It may need to be explicitly requested.

However, if you access the property via the get_fieldValues method, you don’t get the exception, even if there is no such property in the property bag. In this case the property remain simply undefined, so you can check its existence via

var allProps = props.get_fieldValues();
if (typeof (allProps.YourProperty) == "undefined") {
console.log("The property is undefined");
}

In this case the properties from the bag are available as simple JavaScript properties from code:

You can dump (or process) all properties defined in the bag via this code (taken from this thread):

var allProps = props.get_fieldValues();
for (var property in allProps) {
if (allProps.hasOwnProperty(property)) {
console.log(String.format("{0}: {1}", property, allProps[property]));
}
}

A sample output of the script in the debugging console:

Unfortunately, I have not found any way to limit the scope of properties returned by the query to specific properties. All of the properties in the property bag are returned by the request. The GetProperty method of the SPWeb class, that on the server side makes it possible to access the value of a single property, is not implemented in the client object model.

If you try this one:

var props = web.get_allProperties();
ctx.load(props, "Include(YourProperty)");

you will receive an exception complaining about the invalid request, as you call the executeQueryAsync method.

Saving the value back to the web properties is a straightforward operation:

this.saveSettings = function ($scope) {
    var deferred = $q.defer();
 
    var ctx = new SP.ClientContext();
 
    var web = ctx.get_web();
    var props = web.get_allProperties();        
    props.set_item("YourProperty", $scope.propValue);
    web.update();
 
    ctx.executeQueryAsync(
        function () {
            deferred.resolve();
        },
        function (sender, args) {
            deferred.reject('Request failed. ' + args.get_message() + '\n' + args.get_stackTrace());
        }
    );
 
    return deferred.promise;
}

Comments (1)

June 11, 2014

How to filter the results of JSCOM requests on the server side

Filed under: ECMAScript Client Object Model, SP 2013 — Tags: ECMAScript Client Object Model, SP 2013 — Peter Holpar @ 22:32

In the past I already discussed the internals of the Managed Client Object Model and the ECMAScript Object Model (a.k.a. JSCOM) several times. In this post I illustrate an unofficial way of extending the JSCOM.

WARNING: As I wrote above, the method I describe here is totally unofficial and unsupported. It’s only an experiment that depends on unpublished features of the JSCOM, like internal functions that can be changed or even removed without prior notice. Please, take my words seriously, and do not apply this method in a productive environment.

Assuming you are working with the Managed Client Object Model, you can submit queries, that are evaluated on the server side, and return only results that fulfill the conditions. For example, the request below returns only the filterable, non-hidden fields of a list (sample taken from MSDN):

clientContext.Load(oList,
    list => list.Fields.Where(
    field => field.Hidden == false
        && field.Filterable == true));
clientContext.ExecuteQuery();

The most important benefit of this syntax is that you can reduce the network traffic caused by returning irrelevant data (in this case fields) that we would have to filter on the client side without this feature.

Unfortunately, this feature seems to be missing in the ECMAScript Object Model, the Include keyword supports only limiting the members of the object to be returned (the query below returns only the Title and Id properties of each lists), but does not help to filter the results at all (e.g. you can not filter lists based on their properties).

clientContext.load(collList, ‘Include(Title, Id)’);

Since I’m working recently quite a lot with JSCOM (mainly in context of Project Server 2013), I was curious what happens when we use this feature from the Managed Client Object Model, and how we could inject the functionality into JSCOM.

Note: Yes, I know that REST makes it possible (via the $filter query option) to filter the items returned, however it has its own limitation, like the lack of the batch requests – e.g. aggregating requests on the client side and sending them in batches via executeQueryAsync – that is (at least, IMHO) one of the best features of JSCOM.

So I’ve created a simple .NET console application that submits the following query (first without, next with filtering) to the server:

ClientContext ctx = new ClientContext(“http://sp2013”);
// without filtering
// var listQuery = ctx.Web.Lists;
// with filtering
var listQuery = ctx.Web.Lists.Where(l => l.Title == "Images");
var list = ctx.LoadQuery(listQuery);
ctx.ExecuteQuery();

and captured the network traffic using Fiddler in both cases. The screenshot below highlights the difference between the two queries:

The equivalent version of the no-filter query in JavaScript:

var context = SP.ClientContext.get_current();
var lists = context.get_web().get_lists();
 
$(document).ready(function () {
    getList();
});
 
function getList() {
    context.load(lists);
    context.executeQueryAsync(onGetListSuccess, onGetListFail);
}

As in the case of the .NET version, this query submits the request without the QueryableExpression. If we want to enable filtering in case of JSCOM, we should first find where the Query part of the request is assembled. Using a simple search in the .js files shows that this task is performed in the SP.ClientQueryInternal.prototype.$2s_1 method in the SP.Runtime.js.

The default implementation of this method is illustrated below (code taken from SP.Runtime.debug.js):

$2s_1: function SP_ClientQueryInternal$$2s_1($p0, $p1) {
    $p0.writeStartElement('Query');
    this.$2r_1($p0, $p1);
    $p0.writeEndElement();
    if (this.$n_1) {
        $p0.writeStartElement('ChildItemQuery');
        this.$n_1.$2r_1($p0, $p1);
        $p0.writeEndElement();
    }
}

We should create our custom override of the $2s_1 method that injects the QueryableExpression part of the query into the ChildItemQuery block (see a similar implementation in the $2w_0 method of the SP.ClientObjectPropertyConditionalScope.prototype in SP.Runtime.debug.js). For example (to filter child items with Title “Images”):

SP.ClientQueryInternal.prototype.$2s_1 = function SP_ClientQueryInternal$$2s_1($p0, $p1) {
    $p0.writeStartElement('Query');
    this.$2r_1($p0, $p1);
    $p0.writeEndElement();
    if (this.$n_1) {
        $p0.writeStartElement('ChildItemQuery');
        this.$n_1.$2r_1($p0, $p1);
 
        // custom code
        
        $p0.writeStartElement(SP.ClientConstants.QueryableExpression);
        $p0.writeStartElement(SP.ClientConstants.where);
        $p0.writeStartElement(SP.ClientConstants.Test);
        $p0.writeStartElement(SP.ClientConstants.Parameters);
        $p0.writeStartElement(SP.ClientConstants.Parameter);
        $p0.writeAttributeString(SP.ClientConstants.Name, "l");
        $p0.writeEndElement();  // Parameters
        $p0.writeEndElement();  // Parameter
        $p0.writeStartElement(SP.ClientConstants.Body);
        $p0.writeStartElement(SP.ClientConstants.equal);
        $p0.writeStartElement(SP.ClientConstants.expressionProperty);
        $p0.writeAttributeString(SP.ClientConstants.Name, "Title");
        $p0.writeStartElement(SP.ClientConstants.expressionParameter);
        $p0.writeAttributeString(SP.ClientConstants.Name, "l");
        $p0.writeEndElement();  // ExpressionParameter
        $p0.writeEndElement();  // ExpressionProperty
        $p0.writeStartElement(SP.ClientConstants.expressionConstant);
        SP.DataConvert.writeValueToXmlElement($p0, "Images");
        $p0.writeEndElement();  // EQ
        $p0.writeEndElement();  // Body
        $p0.writeEndElement();  // ExpressionConstant
        $p0.writeEndElement();  // Test
        $p0.writeStartElement(SP.ClientConstants.Object);
        $p0.writeStartElement(SP.ClientConstants.queryableObject);
        $p0.writeEndElement();  // QueryableObject
        $p0.writeEndElement();  // Object
        $p0.writeEndElement();  // Where
        $p0.writeEndElement();  // QueryableExpression
 
        // end custom code
 
 
        $p0.writeEndElement();
    }
}

If we define this method in our page as illustrated above, all request sent to the server will filter the child items using the condition defined in the override (Title EQ “Images”). That is probably not what we would like to achieve. It would be better to limit the filtering for example to the context the request was sent from.

I’ve found, that the calling SP.ClientContext object is available as this.$0_1.

First, I‘ve created a filterBase function that writes the filtering part of the request stream based on the condition, filter name and filter value parameters:

function filterBase($p0, condition, filterName, filterValue) {
    $p0.writeStartElement(SP.ClientConstants.QueryableExpression);
    $p0.writeStartElement(SP.ClientConstants.where);
    $p0.writeStartElement(SP.ClientConstants.Test);
    $p0.writeStartElement(SP.ClientConstants.Parameters);
    $p0.writeStartElement(SP.ClientConstants.Parameter);
    $p0.writeAttributeString(SP.ClientConstants.Name, "l");
    $p0.writeEndElement();  // Parameters
    $p0.writeEndElement();  // Parameter
    $p0.writeStartElement(SP.ClientConstants.Body);
    $p0.writeStartElement(condition);
    $p0.writeStartElement(SP.ClientConstants.expressionProperty);
    $p0.writeAttributeString(SP.ClientConstants.Name, filterName);
    $p0.writeStartElement(SP.ClientConstants.expressionParameter);
    $p0.writeAttributeString(SP.ClientConstants.Name, "l");
    $p0.writeEndElement();  // ExpressionParameter
    $p0.writeEndElement();  // ExpressionProperty
    $p0.writeStartElement(SP.ClientConstants.expressionConstant);
    SP.DataConvert.writeValueToXmlElement($p0, filterValue);
    $p0.writeEndElement();  // EQ
    $p0.writeEndElement();  // Body
    $p0.writeEndElement();  // ExpressionConstant
    $p0.writeEndElement();  // Test
    $p0.writeStartElement(SP.ClientConstants.Object);
    $p0.writeStartElement(SP.ClientConstants.queryableObject);
    $p0.writeEndElement();  // QueryableObject
    $p0.writeEndElement();  // Object
    $p0.writeEndElement();  // Where
    $p0.writeEndElement();  // QueryableExpression
}

To keep our former example condition, we can call filterBase from a new function filterLists:

function filterLists($p0) {
    filterBase($p0, SP.ClientConstants.equal, "Title", "Images");
}

We assign the filtering function in our getLists method to the context:

function getList() {
    context.load(lists);
    context.filter = filterLists;
    context.executeQueryAsync(onGetListSuccess, onGetListFail);
}

In the new version of the of the $2s_1 method override we check if the current context has a filtering method assigned to, and if one is found, it is called:

SP.ClientQueryInternal.prototype.$2s_1 = function SP_ClientQueryInternal$$2s_1($p0, $p1) {
    $p0.writeStartElement('Query');
    this.$2r_1($p0, $p1);
    $p0.writeEndElement();
    if (this.$n_1) {
        $p0.writeStartElement('ChildItemQuery');
        this.$n_1.$2r_1($p0, $p1);
 
        // custom code
 
        var context = this.$0_1;
        if (context.filter != undefined) {
            context.filter($p0);
        }
 
        // end custom code
 
 
        $p0.writeEndElement();
    }
}

OK, we are one step further now, but wouldn’t it be nice, if we could query more objects in the same context (and the same batch) and have the option to set different filters for each one?

I’ve found that the object path property of the client object to be queried is available in the $2s_1 method via this.$G_0.

In this case we should override the $2s_1 method like this:

SP.ClientQueryInternal.prototype.$2s_1 = function SP_ClientQueryInternal$$2s_1($p0, $p1) {
    $p0.writeStartElement('Query');
    this.$2r_1($p0, $p1);
    $p0.writeEndElement();
    if (this.$n_1) {
        $p0.writeStartElement('ChildItemQuery');
        this.$n_1.$2r_1($p0, $p1);
 
        // custom code
 
        var objectPathProp = this.$G_0;
        if (objectPathProp.filter != undefined) {
            objectPathProp.filter($p0);
        }
 
        // end custom code
 
 
        $p0.writeEndElement();
    }
}

The object path property is available as myObject.$5_0.$e_0 property of the client object (for example, lists.$5_0.$e_0), so we can rewrite our methods as illustrated below. In this case we extend our example with a further object (groups) that should be filtered by the filterGroups method (OwnerTitle property of the group EQ "DevSite Owners"). The getList function was also renamed to getObjects.

var context = SP.ClientContext.get_current();
var lists = context.get_web().get_lists();
var groups = context.get_web().get_siteGroups();
 
$(document).ready(function () {
    getObjects();
});
 
function filterLists($p0) {
    filterBase($p0, SP.ClientConstants.equal, "Title", "Images");
}
 
function filterGroups($p0) {
    filterBase($p0, SP.ClientConstants.equal, "OwnerTitle", "DevSite Owners");
}
 
function getObjects() {
    context.load(lists);
    context.load(groups);
 
    lists.$5_0.$e_0.filter = filterLists;
    groups.$5_0.$e_0.filter = filterGroups;
    context.executeQueryAsync(onGetListSuccess, onGetListFail);
}

The following Fiddler screenshot shows the resulting query:

This kind of overrides provides already a quite flexible “framework” for filtering objects on the server side using the JSCOM, making this important feature available for JavaScript developers as well. It would be nice if Microsoft would provide a similar, but official solution to this issue in a forthcoming service pack.

We can see from this experiment as well, that there is much more power available in the XML / JSON communication protocol behind the client object models, than it is made public by this APIs, so there is yet place for improvements. You can find valuable information related to the SharePoint Client Query Protocol here.

Leave a Comment

April 8, 2014

Mysterious “File Not Found” Error When Working With the ECMAScript Client Object Model

Filed under: ECMAScript Client Object Model, Internet Explorer, SP 2010 — Tags: ECMAScript Client Object Model, Internet Explorer, SP 2010 — Peter Holpar @ 20:38

Recently I was debugging a very simple JavaScript in Internet Explorer 9 with the F12 Developer Tools, when received a “File Not Found” error. In the script I tried to open a site via

webToCheck = context.get_site().openWeb(‘/Subweb’);

and it seemed as the site did not exist. However, the same site could be opened from the Managed Client Object Model.

I launched Fiddler to check what happens in the background and was surprised to see another web site (let’s call it SubwebWrong) in the request. This web site really did not exist, so the response of the server was reasonable.

But where did that site name came from? I found a former entry in the Watch window of the Developer Tools, that referred to this site:

webToCheck = context.get_site().openWeb(‘/SubwebWrong’);

So it seems that this command was executed automatically by IE, that caused the script to malfunction. Pretty strange behavior, indeed.

During debugging I received an erratic “Access denied. You do not have permission to perform this action or access this resource.” error as well. It happened typically after working for a longer time on the same page. I assume that the reason is that the form digest on the page was timed out.

Leave a Comment

July 29, 2013

ECMAScript Client Object Model Internals – Creating custom client OM extensions

Filed under: ECMAScript Client Object Model, SP 2010 — Tags: ECMAScript Client Object Model, SP 2010 — Peter Holpar @ 18:26

About two years ago I published a sample about the extensibility of the Managed Client Object Model of SharePoint 2010. To understand the theory behind the code in the current post you should read that article and my other posts around the client OM first.

Since that I’ve got the question several times, if a similar extension would be available for the JavaScript / ECMAScript object model. Due to the increasing popularity of the JavaScript based solutions around SharePoint, this topic became more interesting in the past months.

The answer for the previous question is yes, it is definitely possible to extend the out-of-the-box client JS libraries, and it is not even so complicate as one might think. To tell the truth, I had more troubles with loading the .js files in the correct order (see the issue and the solution later) as with implementing the library extensions themself. In this post I show you an extension built on the previous sample. As a prerequisite, you should deploy and configure (in web.config) the server-side components from that post.

Next, I’ve created a new .js file called CustomScripts.js (see code below) and deployed it to 14\TEMPLATE\LAYOUTS\Custom.

// registering our custom namespace
Type.registerNamespace('SS');
 
// ———————————————-
// SS.CustomRequestContext class
// ———————————————-
SS.CustomClientContext = function (serverRelativeUrl) {
    SS.CustomClientContext.initializeBase(this, [(SP.ScriptUtility.isNullOrUndefined(serverRelativeUrl)) ? SP.PageContextInfo.get_webServerRelativeUrl() : serverRelativeUrl]);
}
SS.CustomClientContext.get_current = function () {
    if (!SS.CustomClientContext.$1S_1) {
        SS.CustomClientContext.$1S_1 = new SS.CustomClientContext(SP.PageContextInfo.get_webServerRelativeUrl());
    }
    return SS.CustomClientContext.$1S_1;
}
SS.CustomClientContext.prototype = {
    get_customClientObject: function () {
        var $v_0 = SS.CustomRequestContext.getCurrent(this).get_customClientObject();
        return $v_0;
    }
}
 
// ———————————————-
// SS.CustomRequestContext class
// ———————————————-
SS.CustomRequestContext = function (Context, ObjectPath) {
    SS.CustomRequestContext.initializeBase(this, [Context, ObjectPath]);
}
SS.CustomRequestContext.getCurrent = function ($p0) {
    var $v_0 = $p0.get_staticObjects()['ClientExtension$Server$SSCustomContext$Current'];
    if ((!$v_0)) {
        $v_0 = new SS.CustomRequestContext($p0, new SP.ObjectPathStaticProperty($p0, '{DF694817-22BA-4952-A1E9-84C6E69709A8}', 'Current'));
        $p0.get_staticObjects()['ClientExtension$Server$SSCustomContext$Current'] = $v_0;
    }
    return ($v_0);
}
SS.CustomRequestContext.prototype = {
    get_customClientObject: function () {
        var $v_0 = ((this.get_objectData().get_clientObjectProperties()['CustomClientObject']));
        if (SP.ScriptUtility.isUndefined($v_0)) {
            $v_0 = new SS.CustomClientObject(this.get_context(), new SP.ObjectPathProperty(this.get_context(), this.get_path(), 'CustomClientObject'));
            this.get_objectData().get_clientObjectProperties()['CustomClientObject'] = $v_0;
        }
        return $v_0;
    }
}
 
// ———————————————-
// SS.CustomClientObject class
// ———————————————-
SS.CustomClientObject = function (Context, ObjectPath) {
    SS.CustomClientObject.initializeBase(this, [Context, ObjectPath]);
}
SS.CustomClientObject.prototype = {
    getMessage: function (name) {
 
        var $v_0;
        var $v_1 = new SP.ClientActionInvokeMethod(this, 'GetMessage', [name]);
        this.get_context().addQuery($v_1);
        $v_0 = new SP.StringResult();
        this.get_context().addQueryIdAndResultObject($v_1.get_id(), $v_0);
        this.removeFromParentCollection();
       
        return $v_0;
    }
}
 
// registering our custom classes
SS.CustomClientContext.registerClass('SS.CustomClientContext', SP.ClientContext);
SS.CustomRequestContext.registerClass('SS.CustomRequestContext', SP.ClientObject);
SS.CustomClientObject.registerClass('SS.CustomClientObject', SP.ClientObject);
 
// notify waiting jobs
NotifyScriptLoadedAndExecuteWaitingJobs("CustomScripts.js");

If you understand the internals of the managed client OM from my former posts and had already a look at the source code in the standard SharePoint client OM .js files, the code above should be pretty straightforward. It’s nothing more than inheriting our custom JS classes from the standard SharePoint JS classes, and implementing properties and methods as it is done in the standard libraries (almost copy/paste).

For testing the new library, I’ve created a new web part page, and added a Content Editor Web Part (CEWP) with the code below:

<script type="text/javascript">
 
// wait the standard client OM scripts to be loaded first
ExecuteOrDelayUntilScriptLoaded(loadScript, 'sp.js'); 
 
// load our custom script next
function loadScript() {
    var script = document.createElement("script");
    script.src = "/_layouts/Custom/CustomScripts.js";
    script.type = "text/javascript";
    document.getElementsByTagName("head")[0].appendChild(script);
}
 
// when the script is loaded, execute the custom OM methods
ExecuteOrDelayUntilScriptLoaded(startScript, 'CustomScripts.js'); 
 
var result;
 
function startScript() {
  try {
    var context = SS.CustomClientContext.get_current();
    var obj = context.get_customClientObject();
    result = obj.getMessage('Joe');
    context.executeQueryAsync(Function.createDelegate(this, this.success), Function.createDelegate(this, this.failed));
  }
  catch (e) {
    alert("Error: " + e);
  }
}
 
function failed(sender, args) {
    alert("Operation failed: " + args.get_message());
}
 
function success(sender, args) {
    alert(result.get_value());
}
 
</script>

The standard SharePoint client OM .js files (like SP.Runtime.js first and SP.js next) are loaded by the web part page automatically. These files must be loaded first. Our extension script (CustomScripts.js) is built upon objects in these standard files, so it must be loaded after loading the standard libraries is finished. Loading the files in the wrong order (the standard OM files must be completely loaded before loading our extension!) causes mysterious errors. I solved this requirement through the combination of ExecuteOrDelayUntilScriptLoaded methods and NotifyScriptLoadedAndExecuteWaitingJobs method (see at the end of CustomScripts.js above) and the custom loadScript method.

After this preparation, the client OM extension objects and methods can be used as their standard OM counterparts. Of course, in this case we must start the script with our custom context (SS.CustomClientContext), and not the standard SharePoint client object model context (SP.ClientContext).

The image below illustrates the expected output of the script:

Leave a Comment

January 9, 2013

Using IE as a local Host for SharePoint ECMAScript Client Object Model, the Office 365 version

Filed under: ECMAScript Client Object Model, JavaScript, jQuery, Office 365, SP 2013 — Tags: ECMAScript Client Object Model, JavaScript, jQuery, Office 365, SP 2013 — Peter Holpar @ 13:30

In my recent post I wrote about how you can use the JavaScript Client Object Model (JSCOM) against a remote SharePoint 2010 server from a local HTML page. In the current post my goal is to demonstrate a similar technique, but in this case against the Office 365 Developer Preview, that is the cloud-based version of SharePoint 2013.

NOTE: Again, be aware that the methods described here don’t use the public, documented interfaces, so it is probably not supported by MS, and suggested to be used only as an experiment. It is what can be done, and not a best practice at all. There is no guarantee that it will work for you, especially after MS updates the version of Office 365. I publish this results as I believe one can learn a few tricks around the internals of JSCOM and communication between client and server.

In the previous post we authenticated our requests using the standard Windows-integrated authentication method. In the case of Office 365 that is rather different, as it requires claim-based authentication (see these articles on CodeProject for an introduction on the theme, or this guide from Microsoft for a deep dive). You can find excellent examples for such authentication against Office 365 when using the Managed Client Object Model, including this code sample on MSDN. In the code from Sundararajan Narasiman and Wictor Wilén the Windows Identity Foundation (WIF) classes have been used. Doug Ware, another fellow MVP, published a similar solution, but without involving WIF into the game. You can even find an example for PHP, but for JavaScript I found no solution on the web.

The detailed process of the authentication is very well described on MSDN, so I don’t rehash all the steps here, just provide a quick overview to enable better understanding of the JavaScript code sample below.

1. We get the token from the security token service (STS) of MS Online.

2. "Login" to the actual O365 site using the token provided by STS in the former step. As a result of this step, we have the required cookies (FedAuth and rtFA) to be used automatically in the next steps. These cookies are set by Set-Cookie headers of the response and cached and reused by the browser for later requests targeting the same site.

3. Get the digest from the Sites web service and refresh the one stored in the local page.

4. Execute the JSCOM request (after setting the full URL)

As you can see, the last two steps are identical to the steps we performed in the case of on-premise SharePoint in the last post.

And here is the actual code to demonstrate the theory in practice. Don’t forget to set the URLs for JavaScript file references to match your site name, as well as the values of the JavaScript variables, like usr, pwd and siteFullUrl. BTW, it seems that one can access the SharePoint JavaScript files in the LAYOUTS folder without authentication, at least, I get no authentication prompt when I try to download one.

Code Snippet

<script type="text/ecmascript" src="http://code.jquery.com/jquery-1.8.3.min.js"></script>
<script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/1033/init.js"></script>
<script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/MicrosoftAjax.js"></script>
<script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/CUI.js"></script>
<script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/1033/Core.js"></script>
<script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/SP.Core.js"></script>
<script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/SP.Runtime.js"></script>
<script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/SP.js"></script>
 
<input type="hidden" name="__REQUESTDIGEST" id="__REQUESTDIGEST" value="0x753C251974CACCF3B030F7FF1358D0E0229B6DE0B0D363A0272EDBF69FBE4225A2107BE0998E236C248D2116D0A47B0D1849248B558F420AB09BDE06CFCFDB56,07 Jan 2013 13:30:54 -0000" />
 
<script language="ecmascript" type="text/ecmascript">
    var tokenReq = '<?xml version="1.0" encoding="utf-8"?>';
    tokenReq += '<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot; xmlns:xsd="http://www.w3.org/2001/XMLSchema&quot; xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">&#039;;
    tokenReq += '  <soap:Body>';
    tokenReq += '    <GetUpdatedFormDigestInformation xmlns="http://schemas.microsoft.com/sharepoint/soap/&quot; />';
    tokenReq += '  </soap:Body>';
    tokenReq += '</soap:Envelope>';
 
    // you should set these values according your actual request
    var usr =  'username@yourdomain.onmicrosoft.com';
    var pwd = 'password';
    var siteFullUrl = "https://yourdomain-my.sharepoint.com&quot;;
 
    var loginUrl = siteFullUrl + "/_forms/default.aspx?wa=wsignin1.0";
    var authReq =   '<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope&quot; xmlns:a="http://www.w3.org/2005/08/addressing&quot; xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">&#039;
    authReq +=      '  <s:Header>'
    authReq +=      '    <a:Action s:mustUnderstand="1">http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</a:Action>&#039;
    authReq +=      '    <a:ReplyTo>'
    authReq +=      '      <a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>&#039;
    authReq +=      '    </a:ReplyTo>'
    authReq +=      '    <a:To s:mustUnderstand="1">https://login.microsoftonline.com/extSTS.srf</a:To>&#039;
    authReq +=      '    <o:Security s:mustUnderstand="1" xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">&#039;
    authReq +=      '      <o:UsernameToken>'
    authReq +=      '        <o:Username>' + usr + '</o:Username>'
    authReq +=      '        <o:Password>' + pwd + '</o:Password>'
    authReq +=      '      </o:UsernameToken>'
    authReq +=      '    </o:Security>'
    authReq +=      '  </s:Header>'
    authReq +=      '  <s:Body>'
    authReq +=      '    <t:RequestSecurityToken xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust"><wsp:AppliesTo xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">&#039;
    authReq +=      '      <a:EndpointReference>'
    authReq +=      '        <a:Address>' + loginUrl + '</a:Address>'
    authReq +=      '      </a:EndpointReference>'
    authReq +=      '      </wsp:AppliesTo>'
    authReq +=      '      <t:KeyType>http://schemas.xmlsoap.org/ws/2005/05/identity/NoProofKey</t:KeyType>&#039;
    authReq +=      '      <t:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</t:RequestType>&#039;
    authReq +=      '      <t:TokenType>urn:oasis:names:tc:SAML:1.0:assertion</t:TokenType>'
    authReq +=      '    </t:RequestSecurityToken>'
    authReq +=      '  </s:Body>'
    authReq +=      '</s:Envelope>';
    
    var lists;
 
    function startScript() {
      getToken();
    }
 
    // Step 1: we get the token from the STS
    function getToken()
    {
        $.support.cors = true; // enable cross-domain query
        $.ajax({
            type: 'POST',
            data: authReq,
            crossDomain: true, // had no effect, see support.cors above
            contentType: 'application/soap+xml; charset=utf-8',
            url: 'https://login.microsoftonline.com/extSTS.srf&#039;,         
            dataType: 'xml',
            complete: function (result) {
                // extract the token from the response data
                // var token = $(result.responseXML).find("wsse\\:BinarySecurityToken").text(); // responseXML is undefined, we should work with responseText, because Content-Type: application/soap+xml; charset=utf-8
                var token = $(result.responseText).find("BinarySecurityToken").text();
                getFedAuthCookies(token);
            },
            error: function(XMLHttpRequest, textStatus, errorThrown) {
                alert(errorThrown);
                }
        });
    }
 
    // Step 2: "login" using the token provided by STS in step 1
    function getFedAuthCookies(token)
    {
        $.support.cors = true; // enable cross-domain query
        $.ajax({
            type: 'POST',
            data: token,
            crossDomain: true, // had no effect, see support.cors above
            contentType: 'application/x-www-form-urlencoded',
            url: loginUrl,         
         // dataType: 'html', // default is OK: Intelligent Guess (xml, json, script, or html)
            complete: function (result) {  
                refreshDigest();
            },
            error: function(XMLHttpRequest, textStatus, errorThrown) {
                alert(errorThrown);
            }
        });
    }
 
    // Step 3: get the digest from the Sites web service and refresh the one stored locally
    function refreshDigest()
    {
        $.support.cors = true; // enable cross-domain query
        $.ajax({
                type: 'POST',
                data: tokenReq,
                crossDomain: true, // had no effect, see support.cors above
                contentType: 'text/xml; charset="utf-8"',
                url: siteFullUrl + '/_vti_bin/sites.asmx',
                headers: {
                    'SOAPAction': 'http://schemas.microsoft.com/sharepoint/soap/GetUpdatedFormDigestInformation&#039;,
                    'X-RequestForceAuthentication': 'true'
                },
                dataType: 'xml',
                complete: function (result) {  
                    $('#__REQUESTDIGEST').val($(result.responseXML).find("DigestValue").text()); 
                    sendJSCOMReq();
                },
                error: function(XMLHttpRequest, textStatus, errorThrown) {
                    alert(errorThrown);
                }
        });
    }
 
    // Step 4: execute the JSCOM request (after setting the full URL)
    function sendJSCOMReq() {
        try {
 
            var spPageContextInfo = {webServerRelativeUrl: "\u002fTestappforSharePoint", webAbsoluteUrl: "https:\u002f\u002fyour.sharepoint.com\u002fTestappforSharePoint", siteAbsoluteUrl: "https:\u002f\u002fyourdomain-f7079688f25f20.sharepoint.com", serverRequestPath: "\u002fTestappforSharePoint\u002fPages\u002fDefault.aspx", layoutsUrl: "_layouts\u002f15", webTitle: "Test app for SharePoint", webTemplate: "17", tenantAppVersion: "0", webLogoUrl: "\u002f_layouts\u002f15\u002fimages\u002fsiteIcon.png?rev=23", webLanguage: 1033, currentLanguage: 1033, currentUICultureName: "en-US", currentCultureName: "en-US", clientServerTimeDelta: new Date("2013-01-07T13:57:14.0337474Z") – new Date(), siteClientTag: "0$$15.0.4433.1011", crossDomainPhotosEnabled:true, webUIVersion:15, webPermMasks:{High:2147483647,Low:4294967295}, pagePersonalizationScope:1,userId:11, systemUserKey:"i:0h.f|membership|1003bffd844f8d57@live.com", alertsEnabled:true, siteServerRelativeUrl: "\u002f", allowSilverlightPrompt:'True'};
 
 
            var siteRelativeUrl = "/";
            var context = new SP.ClientContext(siteRelativeUrl);
            context.$1P_0 = siteFullUrl;
 
            var web = context.get_web();
            lists = web.get_lists();
 
            context.load(lists);
            context.executeQueryAsync(Function.createDelegate(this, this.onQuerySucceeded), Function.createDelegate(this, this.onQueryFailed));  
        } catch (err) {
            var msg = "There was an error on this page.\n";
            msg += "Error description: " + err.message + "\n";
            alert(msg);
        }
    }
 
    // Step 5 (success): process response
    function onQuerySucceeded(sender, args) {
        var count = lists.get_count();
        var listTitles = "Number of lists: " + count + ":\n";
        for(var i=0;i<count; i++)
        {
            var list = lists.get_item(i);
            listTitles += "  " + list.get_title() + "\n";
        }
        alert(listTitles);
    }
 
    // Step 5 (failure): display error
    function onQueryFailed(sender, args) {
      alert("Request failed: "+ args.get_message());
    }
 
    // start the custom script execution after the scripts and page are loaded
    SP.SOD.executeOrDelayUntilScriptLoaded(function () {
        $(document).ready(startScript);
    }, "sp.js");
 
</script>

You might be wondering, how it is possible to come up with a solution like that. As a first step, I downloaded a few of the managed client object model / WIF samples mentioned above (thank you guys for sharing, you all made my life easier!), and created a simple test console application using them. I analyzed the network traffic (even it is HTTPS!) using Fiddler. Then I tried to understand (based on my knowledge about the authentication process), what happened and why (request data and headers, response data, cookies, etc.). Last (and probably the longest) step was an iteration of trial and error, when I was to reproduce the same network traffic using JavaScript / jQuery objects, step-by-step analyzing the results, comparing them to the original measurements captured for the test console application. So it took some time, but at the end I was quite happy with the results.

Comments (4)

January 8, 2013

Using IE as a local Host for SharePoint ECMAScript Client Object Model

Filed under: ECMAScript Client Object Model, JavaScript, jQuery, SP 2010 — Tags: ECMAScript Client Object Model, JavaScript, jQuery, SP 2010 — Peter Holpar @ 22:52

In the recent weeks I got an idea of a specific client-side HTML application (HTML file located on the local hard drive) that communicate with the SharePoint server. More on that idea later, as I hopefully achieve some results, but for now I would like to share a few interesting byproducts of my research.

This time I start with a few questions: what can we do, if we need a client application that manipulates SharePoint objects on a computer that doesn’t have (or can’t have) the managed client object model installed on it? Let’s say, you don’t have even the necessary .NET runtime version, or you don’t have Visual Studio, you have no former C# / PowerShell experience (or at least, you would not like to bother with them as you need a quickly editable application), so assume all you know is HTML and the good old JavaScript. You can’t deploy files to the SharePoint site, you don’t have even page edit permissions, so the standard way of injecting JavaScript through the Content Editor Web Part (CEWP) is not available to you. Using the REST API from your HTML pages through web requests could be an option, but it might be simply not powerful enough (especially in SharePoint 2010 version) to achieve your goals.

Using the ECMAScript Client Object Model (aka JavaScript Client Object Model or JSCOM) would be ideal to this task, but it has a significant out-of-the-box limitation: it was designed to work in the SharePoint site context it interacts with, that means, you can use only relative URLs when instantiating the SP.ClientContext object, or get the current context from the SharePoint page the script is used on, none of these is possible when working with a local HTML file.

In the last years I already delved quite deep into the client object model (both the managed and the JavaScript versions), and learned that even the out-of-the-box asynchronous calls can be altered to synchronous, so we should never give up.

In this post I show you a possible workaround for the problem in the case of an on-premise SharePoint 2010 that authenticates users through standard windows-integrated authentication. In the next post I plan to illustrate an even more complex solution for the case of the Office 365 Preview (that is a cloud-based SharePoint 2013).

As always, Fiddler proved again to be an invaluable tool analyzing the network traffic caused by the managed OM and trying to simulate the same for ECMAScript.

NOTE: As in the case of the former hacks, be aware that the methods described here don’t use the public, documented interfaces, so it is probably not supported by MS, and suggested to be used only as an experiment. There is no guarantee that it will work for you, especially if our environments are at different SP / cumulative update level.

In the progress of my work, I faced two major (and several minor) issues:

1. How to enforce using of absolute site URLs in JSCOM instead of the relative ones? It was the easier problem to solve as you see it soon.

2. How to achieve the right request digest token to get authorization to access the server side object from the JavaScript code.

Prerequisites to start the work with JSCOM in the local HTML files:

References to the SharePoint JavaScript libraries (and jQuery if you would like to work with that). These ones are included “automatically” when working with standard SharePoint pages, but in this case we should reference them (in the correct order!) as well. We could start our custom scripts only after our page and these libraries are already loaded.
A defined JavaScript variable (see more about that below) called _spPageContextInfo. That is in our case only a dummy placeholder (can be copied from the source of a SharePoint page) that the scripts depend on.
A HTML input field (by default it is a hidden one) called __REQUESTDIGEST that contains the authentication digest for the page. It can be copied from the source of a SharePoint page as well, but it has an expiration time. It is no problem in the case of a web page, as it is refreshed on the server site on page reloads, but it is not the case in the local file, so we should get always an actual one (see problem 2 above).

Regarding Issue 1: After a short investigation I found that the URL the client requests are sent to is determined by the $1P_0 property of the SP.ClientContext object. We should set our custom value right after initializing the context.

var siteRelativeUrl = "/";
var siteFullUrl = "http://intranet.contoso.com";
var context = new SP.ClientContext(siteRelativeUrl); // we set a dummy relative path that always exists
context.$1P_0 = siteFullUrl;

Setting the absolute URL is not enough for the successful request. Next step was to solve the issue of the request digest as without that our request would be rejected. It took me a while, but at the end it turned to be the same token as the one returned by the GetUpdatedFormDigestInformation method of the Sites web service. So all we have to do is to call this method with the properly formatted request, and replace the value of the __REQUESTDIGEST field before calling the actual JSCOM code.

On of the minor issues was, how to send POST requests to another domain (issue with cross-domain scripting), in this case to access the Sites web service. For GET requests there is JSONP (you can set crossDomain: true for your AJAX request in jQuery), but for POST, it is not available, crossDomain had simply no effect during my tests. In this situation I received an “Invalid argument.” error thrown in MicrosoftAjax.js when setting headers for the web requests.

Fortunately I found this jQuery option that solved this problem for me:

$.support.cors = true;

NOTE: When you open your HTML page not from the file system, but rather through another web server, you might be faced with the following prompt:

Choosing the default value (that is “No”) results in an “Access is denied.” error in MicrosoftAjax.js.

NOTE: For the sake of simplicity, I’ve uploaded a jQuery version (v1.8.2 to be exact, as the version number may be important) to my SharePoint server. In the real life one can reference for example http://code.jquery.com/jquery-latest.min.js instead of this.

Finally, here is the full source code of a “minimized” page that displays the number and the name of the lists on a remote SharePoint site. It is assumed that you are online (access to SP) and are (or can be) authenticated by the SharePoint server, having minimum read permissions on the site:

Code Snippet

<script type="text/ecmascript" src="http://intranet.contoso.com/_layouts/jquery/jquery.min.js"></script>
<script type="text/ecmascript" src="http://intranet.contoso.com/_layouts/1033/init.js"></script>
<script type="text/ecmascript" src="http://intranet.contoso.com/_layouts/MicrosoftAjax.js"></script>
<script type="text/ecmascript" src="http://intranet.contoso.com/_layouts/CUI.js"></script>
<script type="text/ecmascript" src="http://intranet.contoso.com/_layouts/1033/Core.js"></script>
<script type="text/ecmascript" src="http://intranet.contoso.com/_layouts/SP.Core.js"></script>
<script type="text/ecmascript" src="http://intranet.contoso.com/_layouts/SP.Runtime.js"></script>
<script type="text/ecmascript" src="http://intranet.contoso.com/_layouts/SP.js"></script>
 
<input type="hidden" name="__REQUESTDIGEST" id="__REQUESTDIGEST" value="0x0B5280FBC219C9A7E41746D9EA6AC24E65CB936560A6856AA4C38997F114401AED3254D7DDFACBE03C2028F689D0E67F55239E8FA679CE15F3EBC7A0C6280A34,05 Jan 2013 21:34:32 -0000" />
 
<script language="ecmascript" type="text/ecmascript">
 
    // define token request XML
    var tokenReq = '<?xml version="1.0" encoding="utf-8"?>';
    tokenReq += '<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot; xmlns:xsd="http://www.w3.org/2001/XMLSchema&quot; xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">&#039;;
    tokenReq += '  <soap:Body>';
    tokenReq += '    <GetUpdatedFormDigestInformation xmlns="http://schemas.microsoft.com/sharepoint/soap/&quot; />';
    tokenReq += '  </soap:Body>';
    tokenReq += '</soap:Envelope>';
 
    var siteFullUrl = "http://intranet.contoso.com&quot;;
    var lists;
 
    function startScript() {
      refreshDigest();
    }
 
    function refreshDigest() {
        $.support.cors = true; // enable cross-domain query
        $.ajax({
            type: 'POST',
            data: tokenReq,
            crossDomain: true,  // had no effect, see support.cors above
            contentType: 'text/xml; charset="utf-8"',
            url: siteFullUrl + '/_vti_bin/sites.asmx',
            headers: {
                'SOAPAction': 'http://schemas.microsoft.com/sharepoint/soap/GetUpdatedFormDigestInformation&#039;,
                'X-RequestForceAuthentication': 'true'
            },   
            dataType: 'xml',
            complete: function (result) {  
                $('#__REQUESTDIGEST').val($(result.responseXML).find("DigestValue").text()); 
                sendJSCOMReq();
            }
        });
    }
 
    function sendJSCOMReq()
    {
        var _spPageContextInfo = {webServerRelativeUrl: "\u002f", webLanguage: 1033, currentLanguage: 1033, webUIVersion:4,pageListId:"{7bbd4c55-f832-40e2-8e2a-243455c3b2ba}",pageItemId:1,userId:1073741823, alertsEnabled:true, siteServerRelativeUrl: "\u002f", allowSilverlightPrompt:'True'};
 
        var siteRelativeUrl = "/";
        var context = new SP.ClientContext(siteRelativeUrl); // we set a dummy relative path that always exists
        context.$1P_0 = siteFullUrl;
 
        var web = context.get_web();
        lists = web.get_lists();
 
        context.load(lists);
        context.executeQueryAsync(Function.createDelegate(this, this.onQuerySucceeded), Function.createDelegate(this, this.onQueryFailed));        
    }
 
    function onQuerySucceeded(sender, args) {
        var count = lists.get_count();
        var listTitles = "Number of lists: " + count + ":\n";
        for(var i=0;i<count; i++)
        {
            var list = lists.get_item(i);
            listTitles += "  " + list.get_title() + "\n";
        }
        alert(listTitles);
    }
 
    function onQueryFailed(sender, args) {
      alert("Request failed: "+ args.get_message());
    }
 
    // start the custom script execution after the scripts and page are loaded
    SP.SOD.executeOrDelayUntilScriptLoaded(function () {
        $(document).ready(startScript);
    }, "sp.js");
 
</script>

After solving these problems I had a bad feeling. What happens if I authenticate myself as a standard user with low privileges, then stole the digest of a site owner included in a SharePoint page using a network traffic analyzer tool and try to send my request with the token of the other user? My experiments show that this issue is handled by SharePoint, as I received an error stating the token was not valid and I had to get a new one from the server.

In the next post I take another step forward to show you how to achieve the same using the Office 365 Preview version.

Leave a Comment

October 16, 2011

Creating more advanced conditions for your ECB menus through jQuery and synchronous Client Object Model / WCF Data Services calls

Filed under: ECB menu, ECMAScript Client Object Model, jQuery, SP 2010, WCF Data Services — Tags: ECB menu, ECMAScript Client Object Model, jQuery, SP 2010, WCF Data Services — Peter Holpar @ 22:00

In my last post I showed you how to alter the Edit Control Block (ECB) menu of SharePoint 2010 based on simple conditions. As I described, if you need to use more advanced conditions, for example based on list item field values or other information not available on the client side, you should apply a few tricks or even hacks.

For these advanced solutions it is useful to provide the context to our allowDeletion method through the ctx parameter, so we add this one to the parameter list of the method.

It means that our original condition in method AddListMenuItems is changed from

if (allowDeletion(currentItemID))

if (allowDeletion(ctx, currentItemID))

In the simplest case, the information is already there at the page on client side, you simply have to find the way to get it. Checking the HTML source of the page or using the Developer Tools in the Tools menu of Internet Explorer (you can press F12 as a shortcut key) usually helps you to find the right track.

For example, if you need the title of the item, that is included in the LinkTitle field (as we have an ECB menu linked to the title) for each items. When looking up the right item, we should use the ID of the item (passed as the itemID parameter) and the ID of our list view (available in ctx.ctxId).

Note: Although it should be evident, your page must load the jQuery library before you can use jQuery methods. Similarly, the ECMAScript Client Object Model library must be loaded before you reference its objects in methods later in the posts.

So if we would like to allow deletion only for items having title beginning with ‘V’ then we need to apply a method like this:

function allowDeletion(ctx, itemID) {
 
    var title = jQuery('div[Field="LinkTitle"][CTXName="ctx' + ctx.ctxId + '"][id="' + itemID + '"]').text();
 
    return (title.toUpperCase().startsWith("V"));
 
}

Next step is to see how to get the information if it is not available on the client side, but included in the list items on the server side. My first idea was to use the ECMAScript Client Object Model, however it turned out quickly that is not the best choice, as the AddListMenuItems method does not wait for the asynchronous reply supported by the client OM to be returned. Although I created a dirty workaround for that, I will show you that later.

Another alternative is to use REST protocol through the WCF Data Services.

As mentioned, our call must be synchronous, so we have to use jQuery.ajax specifying async: false. The next code snippet shows an example using the same condition applied above, that is title of the item must be started with ‘V’ to enable deletion.

function allowDeletion(ctx, itemID) {
 
    itemUrl = ctx.HttpRoot + "/_vti_bin/listdata.svc/" + ctx.ListTitle +"(" + itemID + ")?$select=Title"    
 
    var title = "";
 
    jQuery.ajax(
    {
        type: 'GET',
        url: itemUrl,
        dataType: 'json',
        success: function (result) {
            if (result.isOk != false) {
                title = result.d.Title;
            }
        },
        data: {}, 
        async: false
    });
 
    return (title.toUpperCase().startsWith("V"));
 
}

Note: When working with jQuery and WCF Data Services, it is useful to know about the parsererror issue with field values containing apostrophe (single quote) and how to fix it.

It is important to note that you are not restricted to the actual list only. With a bit of additional complexity you can query related lists as well using $expand and the lookup field / user field IDs in the current list. If you are unsure how to compose the URL for you request, I suggest you to try to create the filter first in C# using the LINQ syntax, then use Fiddler to capture the request or apply this simple trick from Sahil Malik.

If network bandwidth and speed are limited, then waiting for the synchronous result will cause issues via blocking the UI thread of the browser. If that is the case you should consider “pre-caching” the data required for checks on page load through a single request. If the item count of the list is limited, then you can cache all of the data (in our case, the IDs and the related titles of the items is needed), in the case of a larger list, you should get only the items displayed in the current page of the view. To get the IDs of the items on the page you should run a jQuery select similar to the one we used to get the title in the first example, and submit the REST request using a complex condition.

Last, I would like to show you my workaround for calling ECMAScript Client Object Model synchronously. As you might now, it is officially / theoretically not possible. I found that it can be done technologically, although I had to spend a few hours Fiddlering and digging into the internals of the ECMAScript Client OM (SP.Runtime.debug.js), comparing the JavaScript methods to the ones in the managed Client OM classes using Reflector. If you demand more information about it I can give you more details, but now I publish it “as is”.

To build up the request I use the “traditional” ECMAScript Client Object Model (that, of course, should be already loaded before our script), but before sending it I get the built-up request in XML format from the internal methods of the OM, and send the request through jQuery.ajax in synchronous mode, just as like the case of WCF Data Services. The JSON result is “loaded” into JavaScript objects.

Be aware that the method described here don’t use the public, documented interfaces, so it is probably not supported by MS, and suggested to be used only as an experiment. There is no guarantee that it will work for you, especially if our environments are at different SP / cumulative update level.

function allowDeletion(ctx, itemID) {
 
    var title = "";
 
    try {
        var context = SP.ClientContext.get_current();
        var web = context.get_web();
        var selectedListId = SP.ListOperation.Selection.getSelectedList();
        var selectedListItem = web.get_lists().getById(selectedListId).getItemById(itemID);
        context.load(selectedListItem, "Title");
 
        // start hacking
        var pendingRequest = context.get_pendingRequest();
        var webRequest = pendingRequest.get_webRequest();
 
        // get the request XML
        var body = pendingRequest.$24_0().toString();
 
        // get the URL of client.svc
        var url = webRequest.get_url();
 
        // "initialize" request
        SP.ClientRequest.$1T(webRequest);
 
        // we should add digest later to the request as an HTTP header (see below)
        var digest = webRequest.get_headers()['X-RequestDigest'];
 
        jQuery.ajax(
        {
            type: "POST",
            data: body,
            url: ctx.HttpRoot + url,
            success: function (result) {
                if (result.isOk != false) {
                    title = result[result.length – 1].Title;
                }
            },
            headers: {
                "x-requestdigest": digest
            },
            contentType: "application/x-www-form-urlencoded",
            async: false
        });
 
    }
    catch (e) {
        alert("Error: " + e);
    }
 
    return (title.toUpperCase().startsWith("V"));
 
}

BTW, the example above achieves the same, it allows deletion only for items having title beginning with ‘V’. Similarly to the former WCF DS example, the synchronous request through the network blocks the UI thread, and might require pre-caching in case of a slow network.

Leave a Comment

Older Posts »

Second Life of a Hungarian SharePoint Geek

March 26, 2018

Querying SharePoint list items including their attachment in a single request from the JavaScript Client Object Model

October 19, 2015

Displaying Notifications and Status Messages from a SharePoint-based AngularJS Application Including a FormController

July 16, 2015

How to Read Project Properties that are not Available in the Client Object Model?

July 6, 2015

Accessing and Manipulating Property Bags via the ECMAScript Client Object Model

June 11, 2014

How to filter the results of JSCOM requests on the server side

April 8, 2014

Mysterious “File Not Found” Error When Working With the ECMAScript Client Object Model

July 29, 2013

ECMAScript Client Object Model Internals – Creating custom client OM extensions

January 9, 2013

Using IE as a local Host for SharePoint ECMAScript Client Object Model, the Office 365 version

January 8, 2013

Using IE as a local Host for SharePoint ECMAScript Client Object Model

October 16, 2011

Creating more advanced conditions for your ECB menus through jQuery and synchronous Client Object Model / WCF Data Services calls

Archives

Top Posts & Pages

Recent Posts

March 26, 2018

October 19, 2015

July 16, 2015

July 6, 2015

June 11, 2014

April 8, 2014

July 29, 2013

January 9, 2013

January 8, 2013

October 16, 2011

Archives

Categories

Top Posts & Pages

Recent Posts