Using IE as a local Host for SharePoint ECMAScript Client Object Model, the Office 365 version

In my recent post I wrote about how you can use the JavaScript Client Object Model (JSCOM) against a remote SharePoint 2010 server from a local HTML page. In the current post my goal is to demonstrate a similar technique, but in this case against the Office 365 Developer Preview, that is the cloud-based version of SharePoint 2013.

NOTE: Again, be aware that the methods described here don’t use the public, documented interfaces, so it is probably not supported by MS, and suggested to be used only as an experiment. It is what can be done, and not a best practice at all. There is no guarantee that it will work for you, especially after MS updates the version of Office 365. I publish this results as I believe one can learn a few tricks around the internals of JSCOM and communication between client and server.

In the previous post we authenticated our requests using the standard Windows-integrated authentication method. In the case of Office 365 that is rather different, as it requires claim-based authentication (see these articles on  CodeProject for an introduction on the theme, or this guide from Microsoft for a deep dive). You can find excellent examples for such authentication against Office 365 when using the Managed Client Object Model, including this code sample on MSDN. In the code from Sundararajan Narasiman and Wictor Wilén the Windows Identity Foundation (WIF) classes have been used. Doug Ware, another fellow MVP, published a similar solution, but without involving WIF into the game. You can even find an example for PHP, but for JavaScript I found no solution on the web.

The detailed process of the authentication is very well described on MSDN, so I don’t rehash all the steps here, just provide a quick overview to enable better understanding of the JavaScript code sample below.

1. We get the token from the security token service (STS) of MS Online.

2. "Login" to the actual O365 site using the token provided by STS in the former step. As a result of this step, we have the required cookies (FedAuth and rtFA) to be used automatically in the next steps. These cookies are set by Set-Cookie headers of the response and cached and reused by the browser for later requests targeting the same site.

3. Get the digest from the Sites web service and refresh the one stored in the local page.

4. Execute the JSCOM request (after setting the full URL)

As you can see, the last two steps are identical to the steps we performed in the case of on-premise SharePoint in the last post.

And here is the actual code to demonstrate the theory in practice. Don’t forget to set the URLs for JavaScript file references to match your site name, as well as the values of the JavaScript variables, like usr, pwd and siteFullUrl. BTW, it seems that one can access the SharePoint JavaScript files in the LAYOUTS folder without authentication, at least, I get no authentication prompt when I try to download one.

Code Snippet

1. <script type="text/ecmascript" src="http://code.jquery.com/jquery-1.8.3.min.js"></script>
2. <script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/1033/init.js"></script>
3. <script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/MicrosoftAjax.js"></script>
4. <script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/CUI.js"></script>
5. <script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/1033/Core.js"></script>
6. <script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/SP.Core.js"></script>
7. <script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/SP.Runtime.js"></script>
8. <script type="text/ecmascript" src="https://yourdomain-my.sharepoint.com/_layouts/SP.js"></script>
9.  
10. <input type="hidden" name="__REQUESTDIGEST" id="__REQUESTDIGEST" value="0x753C251974CACCF3B030F7FF1358D0E0229B6DE0B0D363A0272EDBF69FBE4225A2107BE0998E236C248D2116D0A47B0D1849248B558F420AB09BDE06CFCFDB56,07 Jan 2013 13:30:54 -0000" />
11.  
12. <script language="ecmascript" type="text/ecmascript">
13.     var tokenReq = '<?xml version="1.0" encoding="utf-8"?>';
14.     tokenReq += '<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot; xmlns:xsd="http://www.w3.org/2001/XMLSchema&quot; xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">&#039;;
15.     tokenReq += '  <soap:Body>';
16.     tokenReq += '    <GetUpdatedFormDigestInformation xmlns="http://schemas.microsoft.com/sharepoint/soap/&quot; />';
17.     tokenReq += '  </soap:Body>';
18.     tokenReq += '</soap:Envelope>';
19.  
20.     // you should set these values according your actual request
21.     var usr =  'username@yourdomain.onmicrosoft.com';
22.     var pwd = 'password';
23.     var siteFullUrl = "https://yourdomain-my.sharepoint.com&quot;;
24.  
25.     var loginUrl = siteFullUrl + "/_forms/default.aspx?wa=wsignin1.0";
26.     var authReq =   '<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope&quot; xmlns:a="http://www.w3.org/2005/08/addressing&quot; xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">&#039;
27.     authReq +=      '  <s:Header>'
28.     authReq +=      '    <a:Action s:mustUnderstand="1">http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</a:Action>&#039;
29.     authReq +=      '    <a:ReplyTo>'
30.     authReq +=      '      <a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>&#039;
31.     authReq +=      '    </a:ReplyTo>'
32.     authReq +=      '    <a:To s:mustUnderstand="1">https://login.microsoftonline.com/extSTS.srf</a:To>&#039;
33.     authReq +=      '    <o:Security s:mustUnderstand="1" xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">&#039;
34.     authReq +=      '      <o:UsernameToken>'
35.     authReq +=      '        <o:Username>' + usr + '</o:Username>'
36.     authReq +=      '        <o:Password>' + pwd + '</o:Password>'
37.     authReq +=      '      </o:UsernameToken>'
38.     authReq +=      '    </o:Security>'
39.     authReq +=      '  </s:Header>'
40.     authReq +=      '  <s:Body>'
41.     authReq +=      '    <t:RequestSecurityToken xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust"><wsp:AppliesTo xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">&#039;
42.     authReq +=      '      <a:EndpointReference>'
43.     authReq +=      '        <a:Address>' + loginUrl + '</a:Address>'
44.     authReq +=      '      </a:EndpointReference>'
45.     authReq +=      '      </wsp:AppliesTo>'
46.     authReq +=      '      <t:KeyType>http://schemas.xmlsoap.org/ws/2005/05/identity/NoProofKey</t:KeyType>&#039;
47.     authReq +=      '      <t:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</t:RequestType>&#039;
48.     authReq +=      '      <t:TokenType>urn:oasis:names:tc:SAML:1.0:assertion</t:TokenType>'
49.     authReq +=      '    </t:RequestSecurityToken>'
50.     authReq +=      '  </s:Body>'
51.     authReq +=      '</s:Envelope>';
52.     
53.     var lists;
54.  
55.     function startScript() {
56.       getToken();
57.     }
58.  
59.     // Step 1: we get the token from the STS
60.     function getToken()
61.     {
62.         $.support.cors = true; // enable cross-domain query
63.         $.ajax({
64.             type: 'POST',
65.             data: authReq,
66.             crossDomain: true, // had no effect, see support.cors above
67.             contentType: 'application/soap+xml; charset=utf-8',
68.             url: 'https://login.microsoftonline.com/extSTS.srf&#039;,         
69.             dataType: 'xml',
70.             complete: function (result) {
71.                 // extract the token from the response data
72.                 // var token = $(result.responseXML).find("wsse\\:BinarySecurityToken").text(); // responseXML is undefined, we should work with responseText, because Content-Type: application/soap+xml; charset=utf-8
73.                 var token = $(result.responseText).find("BinarySecurityToken").text();
74.                 getFedAuthCookies(token);
75.             },
76.             error: function(XMLHttpRequest, textStatus, errorThrown) {
77.                 alert(errorThrown);
78.                 }
79.         });
80.     }
81.  
82.     // Step 2: "login" using the token provided by STS in step 1
83.     function getFedAuthCookies(token)
84.     {
85.         $.support.cors = true; // enable cross-domain query
86.         $.ajax({
87.             type: 'POST',
88.             data: token,
89.             crossDomain: true, // had no effect, see support.cors above
90.             contentType: 'application/x-www-form-urlencoded',
91.             url: loginUrl,         
92.          // dataType: 'html', // default is OK: Intelligent Guess (xml, json, script, or html)
93.             complete: function (result) {  
94.                 refreshDigest();
95.             },
96.             error: function(XMLHttpRequest, textStatus, errorThrown) {
97.                 alert(errorThrown);
98.             }
99.         });
100.     }
101.  
102.     // Step 3: get the digest from the Sites web service and refresh the one stored locally
103.     function refreshDigest()
104.     {
105.         $.support.cors = true; // enable cross-domain query
106.         $.ajax({
107.                 type: 'POST',
108.                 data: tokenReq,
109.                 crossDomain: true, // had no effect, see support.cors above
110.                 contentType: 'text/xml; charset="utf-8"',
111.                 url: siteFullUrl + '/_vti_bin/sites.asmx',
112.                 headers: {
113.                     'SOAPAction': 'http://schemas.microsoft.com/sharepoint/soap/GetUpdatedFormDigestInformation&#039;,
114.                     'X-RequestForceAuthentication': 'true'
115.                 },
116.                 dataType: 'xml',
117.                 complete: function (result) {  
118.                     $('#__REQUESTDIGEST').val($(result.responseXML).find("DigestValue").text());
119.                     sendJSCOMReq();
120.                 },
121.                 error: function(XMLHttpRequest, textStatus, errorThrown) {
122.                     alert(errorThrown);
123.                 }
124.         });
125.     }
126.  
127.     // Step 4: execute the JSCOM request (after setting the full URL)
128.     function sendJSCOMReq() {
129.         try {
130.  
131.             var spPageContextInfo = {webServerRelativeUrl: "\u002fTestappforSharePoint", webAbsoluteUrl: "https:\u002f\u002fyour.sharepoint.com\u002fTestappforSharePoint", siteAbsoluteUrl: "https:\u002f\u002fyourdomain-f7079688f25f20.sharepoint.com", serverRequestPath: "\u002fTestappforSharePoint\u002fPages\u002fDefault.aspx", layoutsUrl: "_layouts\u002f15", webTitle: "Test app for SharePoint", webTemplate: "17", tenantAppVersion: "0", webLogoUrl: "\u002f_layouts\u002f15\u002fimages\u002fsiteIcon.png?rev=23", webLanguage: 1033, currentLanguage: 1033, currentUICultureName: "en-US", currentCultureName: "en-US", clientServerTimeDelta: new Date("2013-01-07T13:57:14.0337474Z") – new Date(), siteClientTag: "0$$15.0.4433.1011", crossDomainPhotosEnabled:true, webUIVersion:15, webPermMasks:{High:2147483647,Low:4294967295}, pagePersonalizationScope:1,userId:11, systemUserKey:"i:0h.f|membership|1003bffd844f8d57@live.com", alertsEnabled:true, siteServerRelativeUrl: "\u002f", allowSilverlightPrompt:'True'};
132.  
133.  
134.             var siteRelativeUrl = "/";
135.             var context = new SP.ClientContext(siteRelativeUrl);
136.             context.$1P_0 = siteFullUrl;
137.  
138.             var web = context.get_web();
139.             lists = web.get_lists();
140.  
141.             context.load(lists);
142.             context.executeQueryAsync(Function.createDelegate(this, this.onQuerySucceeded), Function.createDelegate(this, this.onQueryFailed));  
143.         } catch (err) {
144.             var msg = "There was an error on this page.\n";
145.             msg += "Error description: " + err.message + "\n";
146.             alert(msg);
147.         }
148.     }
149.  
150.     // Step 5 (success): process response
151.     function onQuerySucceeded(sender, args) {
152.         var count = lists.get_count();
153.         var listTitles = "Number of lists: " + count + ":\n";
154.         for(var i=0;i<count; i++)
155.         {
156.             var list = lists.get_item(i);
157.             listTitles += "  " + list.get_title() + "\n";
158.         }
159.         alert(listTitles);
160.     }
161.  
162.     // Step 5 (failure): display error
163.     function onQueryFailed(sender, args) {
164.       alert("Request failed: "+ args.get_message());
165.     }
166.  
167.     // start the custom script execution after the scripts and page are loaded
168.     SP.SOD.executeOrDelayUntilScriptLoaded(function () {
169.         $(document).ready(startScript);
170.     }, "sp.js");
171.  
172. </script>

You might be wondering, how it is possible to come up with a solution like that. As a first step, I downloaded a few of the managed client object model / WIF samples mentioned above (thank you guys for sharing, you all made my life easier!), and created a simple test console application using them. I analyzed the network traffic (even it is HTTPS!) using Fiddler. Then I tried to understand (based on my knowledge about the authentication process), what happened and why (request data and headers, response data, cookies, etc.). Last (and probably the longest) step was an iteration of trial and error, when I was to reproduce the same network traffic using JavaScript / jQuery objects, step-by-step analyzing the results, comparing them to the original measurements captured for the test console application. So it took some time, but at the end I was quite happy with the results.