Second Life of a Hungarian SharePoint Geek

April 7, 2014

How to Use JavaScript to Delete Short-Term Locks from Documents Opened from SharePoint?

After I find a way to delete the short-term lock using PowerShell, I decided to check how Word clears the lock it put on the file, and try to simulate the same network traffic using JavaScript. See again this post regarding the details of the background communication when opening a document from a SharePoint library, but if you are really hardcore, you will find this documentation to be useful too.

As usual I utilized Fiddler to capture the network traffic between Word and SharePoint, and found the following steps:

First, Word calls _vti_bin/_vti_aut/author.dll of FrontPage Server Extensions (FSE) via HTTP POST and sends a package similar to this one (assuming the URL of your document is


The decoded version of the content:


Next to the getDocsMetaInfo method you can see the encoded version number of FSE (, as well the the url_list parameter that contains both the document and library URLs. (I  found later that it’s enough to include only the document URL, since we need only the metadata of the document, but not of the library).

The response of the server should be similar to this one (assumed that the file is already locked by the user):

Content-type: application/x-vermeer-rpc

<html><head><title>vermeer RPC packet</title></head>
<li>TR|23 Aug 2012 11:00:23 -0000
<li>TR|28 Mar 2014 09:32:06 -0000

<li>SR|Division X
<li>TR|23 Aug 2012 11:00:23 -0000
<li>SW|, SITE-351-479

<li>TR|28 Mar 2014 08:32:06 -0000
<li>TR|28 Mar 2014 09:32:06 -0000

<li>SW|RSS-Feed for the List ‘library’.
<li>TR|07 Feb 2008 12:22:51 -0000
<li>SW|Editor: Library
<li>TW|28 Mar 2014 08:36:18 -0000
<li>TR|19 Mar 2014 13:51:30 -0000
<li>TR|06 Mar 2013 13:19:58 -0000

Regarding the format and content of the response, see this article. If we omit the library URL from the url_list parameter as stated above, the second part of the response, related to the library metadata won’t be included as well.

In this case, the most important information is in the vti_sourcecontrollockid, but vti_sourcecontrolcheckedoutby and vti_sourcecontrollockexpires contains useful information either.

In the second step, the SOAPAction of the CellStorage web service (/_vti_bin/cellstorage.svc/CellStorageService) is invoked, and a content like the one below is sent:

Content-ID: <>
Content-Transfer-Encoding: 8bit
Content-Type: application/xop+xml;charset=utf-8;type="text/xml; charset=utf-8"

<s:Envelope xmlns:s=""><s:Body><RequestVersion Version="2" MinorVersion="0" xmlns=""/><RequestCollection CorrelationId="{35E42C96-FE02-41FE-B4D8-F7DEC43AF784}" xmlns=""><Request Url="" RequestToken="1"><SubRequest Type="ExclusiveLock" SubRequestToken="1"><SubRequestData ExclusiveLockRequestType="ReleaseLock" ExclusiveLockID="{3F9C6295-354F-449A-B38B-C7BA192E3EA2}"/></SubRequest></Request></RequestCollection></s:Body></s:Envelope>

See the two important parameters (document URL and lock ID) above in the request.

In case of success, the server should respond with a package like this one:

Content-ID: <;
Content-Transfer-Encoding: 8bit
Content-Type: application/xop+xml;charset=utf-8;type="text/xml"

<s:Envelope xmlns:s=""><s:Body><ResponseVersion Version="2" MinorVersion="0" xmlns=""/><ResponseCollection WebUrl="; xmlns=""><Response Url="; RequestToken="1" HealthScore="0"><SubResponse SubRequestToken="1" ErrorCode="Success" HResult="0"><SubResponseData/></SubResponse>


Having this request and response formats we can start creating our own custom JavaScript lock releaser.

I embedded the my JavaScript code into a SharePoint page via the following HTML snippet:

<script type="text/javascript" src="/_layouts/Unlock.js"></script>

<input id="documentUrl" type="text" style="width:600px" value=""/&gt;
<input type="button" id="submitRequest" onclick="getLockForDoc()" value="Check lock">

Let’s see the functionality in the Unlock.js. I defined the following helper function in JavaScript:

  1. // define String.format function
  2. if (!String.prototype.format) {
  3.     String.prototype.format = function () {
  4.         var args = arguments;
  5.         return this.replace(/{(\d+)}/g, function (match, number) {
  6.             return typeof args[number] != 'undefined'
  7.               ? args[number]
  8.               : match
  9.             ;
  10.         });
  11.     };
  12. }

Then introduced Properties to make parsing the FSE response easier. I had to create my own unescapeEx function to be able to decode the encoded value of the vti_sourcecontrollockid (that is {3F9C6295-354F-449A-B38B-C7BA192E3EA2} instead of the standard %7B3F9C6295-354F-449A-B38B-C7BA192E3EA2%7D), as the standard JavaScript methods (like unescape, decodeURI  or decodeURIComponent) had issues with that. In this function we replace the encoded chars with their decoded values based on the decimal code using a RegEx replace.

  1. function Properties(source) {
  2.     this.source = source;
  3.     this.offspring = [];
  5.     this.getPropValue = function (propName) {
  6.         var propValue = "";
  7.         var propNameFullLine = '<li>' + propName + '\n';
  8.         var startPos = this.source.indexOf(propNameFullLine);
  9.         if (startPos > -1) {
  10.             var endPos = this.source.substr(startPos + propNameFullLine.length).indexOf('\n');
  11.             if (endPos > 0) {
  12.                 propValue = this.source.substr(startPos + propNameFullLine.length, endPos);
  13.                 // trim leading chars
  14.                 //
  15.                 var strDummy = '<li>SW|';
  16.                 propValue = unescapeEx(propValue.substr(strDummy.length));
  17.             }
  18.         }
  20.         return propValue;
  21.     }
  23.     // this function is intended to be accessible from the object itself (e.g. private)
  24.     var unescapeEx = function (value) {
  25.         var result = value.replace(/&#([0-9]|[1-9][0-9]|[[01][0-9][0-9]|2[0-4][0-9]|25[0-5]);/g, function (str, match) { return String.fromCharCode(match); });
  27.         return result;
  28.     }
  29. }

We get the lock status using the getLockForDoc method. Note, that the URL of the author.dll is hardcoded in this case, so you should update it.

  1. function getLockForDoc() {
  3.     var docUrl = $('#documentUrl').val();
  4.     var escapedDocUrl = encodeURI(docUrl);
  6.     $.ajax({
  7.         url: ';,
  8.         type: 'POST',
  9.         contentType: 'application/x-www-form-urlencoded',
  10.         headers: {
  11.             'MIME-Version': '1.0',
  12.             'User-Agent': 'MSFrontPage/14.0',
  13.             'Accept': 'auth/sicily',
  14.             'X-Vermeer-Content-Type': 'application/x-www-form-urlencoded'
  15.         },
  16.         data: 'method=getDocsMetaInfo%3a14%2e0%2e0%2e6009&url%5flist=%5b' + escapedDocUrl + '%5d&listHiddenDocs=false&listLinkInfo=false',
  17.         complete: function (result) {
  18.             if ((result.readyState == 4) && (result.status == 200)) {
  19.                 var rawResponse = result.responseText;
  20.                 var startPos = rawResponse.indexOf('<li>meta_info=\n<ul>');
  21.                 if (startPos > 0) {
  22.                     var endPos = rawResponse.substr(startPos).indexOf('</ul>');
  23.                     if (endPos > 0) {
  24.                         var props = new Properties(rawResponse.substr(startPos, endPos));
  25.                         var lockId = props.getPropValue('vti_sourcecontrollockid');
  26.                         var checkedOutBy = props.getPropValue('vti_sourcecontrolcheckedoutby');
  27.                         var lockExpires = props.getPropValue('vti_sourcecontrollockexpires');
  28.                         if (lockId == "") {
  29.                             alert("File is not locked.");
  30.                         }
  31.                         else {
  32.                             if (confirm(String.format("File is locked by '{0}' until '{1}', LockId = '{2}'.\r\nUnlocking the file can cause problems if the user is still editing the file and would like to save it later.\r\nDo you want to clear the lock?", checkedOutBy, lockExpires, lockId))) {
  33.                                 releaseLock(escapedDocUrl, lockId);
  34.                             }
  35.                         }
  36.                     }
  37.                 }
  38.             }
  39.         }
  40.     });
  41. }

Finally, the lock is released by calling the releaseLock method. The URL is hardcoded in this case as well, so please fix it if you would like to test. The error checking when processing the response is rather simple, you can improve it if you wish.

  1. // message template to unlock a document
  2. var releseLockReq = '\r\n';
  3. releseLockReq += '–urn:uuid:8cfcbb22-dd52-4889-b29d-9ff2dcf909b2\r\n';
  4. releseLockReq += 'Content-ID: <>\r\n';
  5. releseLockReq += 'Content-Transfer-Encoding: 8bit\r\n';
  6. releseLockReq += 'Content-Type: application/xop+xml;charset=utf-8;type="text/xml; charset=utf-8"\r\n';
  7. releseLockReq += '\r\n';
  8. releseLockReq += '<s:Envelope xmlns:s=""><s:Body><RequestVersion Version="2" MinorVersion="0" xmlns=""/><RequestCollection CorrelationId="{35E42C96-FE02-41FE-B4D8-F7DEC43AF784}" xmlns=""><Request Url="{0}" RequestToken="1"><SubRequest Type="ExclusiveLock" SubRequestToken="1"><SubRequestData ExclusiveLockRequestType="ReleaseLock" ExclusiveLockID="{1}"/></SubRequest></Request></RequestCollection></s:Body></s:Envelope>\r\n';
  9. releseLockReq += '–urn:uuid:8cfcbb22-dd52-4889-b29d-9ff2dcf909b2–\r\n';
  11. function releaseLock(escapedDocUrl, lockId) {
  12.     $.ajax({
  13.         url: ';,
  14.         type: 'POST',
  15.         contentType: 'multipart/related; type="application/xop+xml"; boundary="urn:uuid:8cfcbb22-dd52-4889-b29d-9ff2dcf909b2"; start="<>"; start-Info="text/xml; charset=utf-8"',
  16.         headers: {
  17.             'MIME-Version': '1.0',
  18.             'User-Agent': 'Microsoft Office Upload Center 2010 (14.0.6124) Windows NT 6.1',
  19.             'SOAPAction': ';
  20.         },
  21.         data: String.format(releseLockReq, escapedDocUrl, lockId),
  22.         complete: function (result) {
  23.             var succeeded = false;
  24.             if ((result.readyState == 4) && (result.status == 200)) {
  25.                 var rawResponse = result.responseText;
  26.                 // check the result the primitive way
  27.                 succeeded = (rawResponse.indexOf('ErrorCode="Success"') > -1);
  28.             }
  29.             if (succeeded) {
  30.                 alert('Lock is released.');
  31.             } else {
  32.                 alert('An error has occured during the request. Lock is not released, try again later.');
  33.             }
  34.         }
  35.     });
  36. }

The screenshots below illustrate the functionality of the scripts. First, the lock information is displayed.


We receive the feedback after successfully releasing the lock.


Note, that this method works only as long as the document is checked out by the current user, otherwise an error will be returned.

It might make not to much sense to use JavaScript to release the lock, since one can achieve the same using Word alone (via simply opening and closing the document). However, I have further plans with the script, and my goal was to get a deeper insight into the internals of the FSE communication protocol and the related stuff. Hopefully others find something useful in the sample as well.


March 5, 2014

November 7, 2013

Recurring authentication prompt when editing task list in datasheet view

Filed under: Fiddler, Security, SP 2010, Web service — Tags: , , , — Peter Holpar @ 23:35

The other day we received a complaint from a user, stating he cannot edit a specific list in the Datasheet view, although he had no such problem with other lists. Whenever he would have liked to edit a task list in SharePoint using the Datasheet view he was prompted for his credentials repeatedly, even though he had write permissions on the list and was able edit the same items using the standard web forms. The problem occurred not immediately when he switched to the Datasheet view, but only when he was to insert data copied from an Excel sheet or was to edit the data in the view otherwise. When he clicked Cancel in the authentication dialog, IE became unresponsive and must have been restarted. Other users had no such problem.

My first intention was that it may be caused by using wrong (e.g. 64-bit) version of Internet Explorer or some issue with the local Office installation. This theory was proved to be wrong after the issue was reproduced by the same user on a workstation where the other users were able to edit the list.

As a next try, I captured the network traffic by Fiddler for both the problematic user and for another user, who had no issue with the editing. Analyzing the results I found that in the background the Datasheet view calls the Webs and Lists SharePoint web services. The only difference I found between the traces was that in the case of the problematic user there was two 401 – Unauthorized HTTP response when calling the Lists WS. The first 401 response was simply to force authentication of the client application, and could be found in the normal case as well. The other request that resulted in the second 401 response contained the following body:

<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:xsi="; xmlns:xsd="; xmlns:soap=""><soap:Body&gt; <GetListItems xmlns=""&gt; <listName>UserInfo</listName><query><Query><Where><Membership Type="SPGroup" ID="3"><FieldRef Name="ID" /></Membership></Where><OrderBy><FieldRef Name="Department"/></OrderBy></Query></query><viewFields><ViewFields><FieldRef Name="ID"/><FieldRef Name="Department"/></ViewFields></viewFields><rowLimit>10000</rowLimit><queryOptions><QueryOptions><ViewAttributes Scope="RecursiveAll"/></QueryOptions></queryOptions></GetListItems> </soap:Body></soap:Envelope>

From this request it is obvious that a list called UserInfo is queried, however there was no list with that name on the site. One might think that it is about the hidden user information list of SharePoint, but it is not the case. In fact it is a virtual list that refers to the members of a group in the case of a Person or Group field. At that point it was already obvious, that the Assigned To field of the task list was configured to enable selection of users from a specific group (in this case group with ID = 3), and the problematic user was neither member of the group nor a site collection administrator, so he had no permission to query the membership of the group.


The solution for the problem was to enable non-group members to see the members of the group as displayed below:


May 14, 2010

Accessing internal Exchange web services from an external application through ISA server using forms-based authentication

Filed under: Exchange, FBA, ISA, Web service — Tags: , , , — Peter Holpar @ 23:14

I think it should be a quite common request to access the Exchange Server of a company from an application. In our case, the Exchange Server is version 2007, and it is published through ISA Server 2006 (but it might be a ForeFront TMG 2010 as well) for external access. The authentication method is forms-based through HTTPS channel.

Most common way to access Exchange Server 2007 programmatically is Exchange Web Services (EWS). The authentication scenario outlined above requires a few tricks, as you cannot use the standard NetworkCredential directly for authentication.

FBA requires you to post the credentials (like user name and password) to the authentication URL and the response contains the cookie(s) that must be added to forthcoming request for authentication.

My example is based on the code found here:

Access the Exchange store via WebDAV with Form-Based-Authentication turned on [Updated]

There are a few difference, for example, in our case not the owaauth.dll of OWA is used for authentication, but the CookieAuth.dll of ISA.

I’ve got the simple EWS sample for my post from this article:

Using Exchange Web Services 2007: The Basics

As I’ve mentioned above, in my scenario I cannot use NetworkCredential for the web service. Instead, I should get the FBA cookie and add it to the CookieContainer of the web service proxy.

  1. ExchangeServiceBinding.ExchangeServiceBinding esb = new ExchangeServiceBinding.ExchangeServiceBinding();
  2. esb.Url = ";;
  3. esb.Credentials = CredentialCache.DefaultCredentials;
  4. CookieCollection cookies = DoFormbasedAuthentication(esb.Url, new NetworkCredential("user", "password", "doamain"));
  6. FindItemType fit = new FindItemType();
  7. fit.ItemShape = new ItemResponseShapeType { BaseShape = DefaultShapeNamesType.Default };
  8. fit.ParentFolderIds = new DistinguishedFolderIdType[]
  9. {
  10.     new DistinguishedFolderIdType
  11.     {
  12.         Mailbox = new EmailAddressType{ EmailAddress=""},
  13.         Id = DistinguishedFolderIdNameType.calendar
  14.     }
  15. };
  16. // add cookies to WS
  17. esb.CookieContainer = new CookieContainer();
  18. esb.CookieContainer.Add(cookies);
  20. fit.Traversal = ItemQueryTraversalType.Shallow;
  22. FindItemResponseType firt = esb.FindItem(fit);

In the DoFormbasedAuthentication method we get the cookie required for FBA authentication.

  1. private CookieCollection DoFormbasedAuthentication(String url, NetworkCredential credential)
  2. {
  3.     String server;
  4.     HttpWebRequest request;
  5.     HttpWebResponse response;
  6.     byte[] body;
  7.     Stream stream;
  9.     try
  10.     {
  11.         Uri uri = new Uri(url);
  12.         // Get the server portion of the requested uri and appen the authentication dll from Exchange
  13.         server = String.Format("{0}://{1}/{2}", uri.Scheme, uri.DnsSafeHost, "CookieAuth.dll?Logon");
  15.         // Initiate a new WebRequest to the given URI.
  16.         request = (HttpWebRequest)System.Net.WebRequest.Create(server);
  18.         request.Method = "POST";
  19.         request.CookieContainer = new CookieContainer();
  20.         request.ContentType = "application/x-www-form-urlencoded";
  22.         // Prepare the body of the request
  23.         body = Encoding.UTF8.GetBytes(string.Format("curl=Z2F&flags=0&forcedownlevel=0&formdir=6&trusted=4&username={1}\\{2}&password={3}",
  24.             uri, credential.Domain, credential.UserName, credential.Password));
  26.         request.ContentLength = body.Length;
  28.         // Send the request to the server
  29.         stream = request.GetRequestStream();
  30.         stream.Write(body, 0, body.Length);
  31.         stream.Close();
  33.         // Get the response
  34.         response = (HttpWebResponse)request.GetResponse();
  36.         // Check if the login was successful
  37.         if (response.Cookies.Count < 1) throw
  38.             new AuthenticationException("Failed to login to ISA. Be sure your user name, domain name and password are correct.");
  40.         return response.Cookies;
  41.     }
  42.     catch (AuthenticationException ex)
  43.     {
  44.         throw;
  45.     }
  46.     catch (Exception ex)
  47.     {
  48.         throw new Exception("Failed to login to OWA. The following error occured: " + ex.Message, ex);
  49.     }
  50. }

You can see that I have included several form parameter beyond the credentials when assembling the request body. These parameters are hidden HTTP form fields or other form input fields from the source of the authentication page. You can verify that if you try to access the page using a browser and checking its source.


To add  some really SharePoint specific content to this post either, you might need similar tricks when accessing SharePoint web services (like the other EWS: Excel Web Services) on a FBA site. In this case you have to use the Login method of the Authentication web service to get the cookies you need as illustrated in this article:

How to: Set Various Credentials

January 26, 2010

Updating multi value fields using web service call and batch update

Filed under: SharePoint, Web service — Tags: , — Peter Holpar @ 19:00

In my recent post I showed some examples about updating lookup (SPFieldLookup) and user (SPFieldUser) fields. The examples in that post worked with single value fields.

Updating multi value fields through web service calls or using batch updates is a bit different and a rather undocumented area of SharePoint development, so let’s see some examples about that.

I’ve altered the Tasks list we worked with in the past article to allow its Location field to have multiple value, furthermore I’ve added a multi value user field called Owners, and a multi value choice field called Choices with options Value1, Value2 and Value3. It means the value of the Location field will be of type SPFieldLookupValueCollection, the value of the Owners field will be of type SPFieldUserValueCollection, and the value of the Choices field will be of type SPFieldMultiChoiceValue.

The following code example shows how to update these fields using a batch that updates the list item having ID = 3 by calling the ProcessBatchData method.

  1. SPList list = web.Lists["Tasks"];
  3. String batch = String.Format("<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
  4.     "<ows:Batch OnError=\"Continue\"><Method ID='1'>" +
  5.     "<SetList>{0}</SetList>" +
  6.     "<SetVar Name='Cmd'>Save</SetVar>" +
  7.     "<SetVar Name='ID'>3</SetVar>" +
  8.     "<SetVar Name='urn:schemas-microsoft-com:office:office#Location'>1;#;#2</SetVar>" +
  9.     "<SetVar Name='urn:schemas-microsoft-com:office:office#Owners'>1;#;#7<</SetVar>" +
  10.     "<SetVar Name='urn:schemas-microsoft-com:office:office#Choices'>Value1;#Value2</SetVar>" +
  11.     "</Method></ows:Batch>", list.ID);
  13. String batchResult = list.ParentWeb.ProcessBatchData(batch);

The main point is that you should separate IDs in lookup and user fields with “;#;#”, but you should use only a simple “;#” in the case of the multi choice field. That is because we can omit the lookup value of the field value, and you should only specify the lookup ID. As we discussed in the former post, you can specify an arbitrary lookup value if you wish, like “1;#some value;#2;#other value” or “1;#user1;#7;#user7” it has no effect to the update.

An important note about an issue I’ve experienced when playing with the code. As you may have already noticed I prefer using apostrophes to double quotation marks as they make it possible to create more readable code. If you replaces the quotes in the XML header tag as well as shown below:

<?xml version=’1.0′ encoding=’UTF-8′?>

You will get an exception that states:

Value does not fall within the expected range.

At first one can think it is because of a misspelled field name but the real reason is quite different. So keep using quotes in the header.

The following code illustrates the very same concept using UpdateListItems method of the Lists web service, the only difference is that in this case we update the list item having ID = 2.

  1. MyListService.Credentials = CredentialCache.DefaultCredentials;
  2. MyListService.Url = "http://yourserver/_vti_bin/lists.asmx&quot;;
  4. XmlDocument updateRequest = new XmlDocument();
  6. String updateBatch = "<Batch OnError='Continue'>" +
  7.                         "<Method ID='1' Cmd='Update'>" +
  8.                         "<Field Name='ID'>2</Field>" +
  9.                         "<Field Name='Location'>1;#;#2</Field>" +
  10.                         "<Field Name='Owners'>1;#;#7</Field>" +
  11.                         "<Field Name='Choices'>Value1;#Value2</Field>" +
  12.                         "</Method>" +
  13.                         "</Batch>";
  15. updateRequest.LoadXml(updateBatch);
  17. XmlNode deleteResult = MyListService.UpdateListItems("Tasks", updateRequest.DocumentElement);

September 21, 2009

Deleting documents or folders using the built-in web services

Filed under: Code, SharePoint, Web service — Tags: , , — Peter Holpar @ 03:39

Deleting documents or folders using the built-in Lists web service is not trivial. To call the UpdateListItems method to do this you need to pass the ID of the document (or folder) and the FileRef property of the document (or folder). If you don’t know the latter one, you have to get it first, for example, using the GetListItems method.

I wrapped this functionality into a single method, that assumes you have a member instance for the Lists web service called ListService, and accepts three parameters: the list name, the ID of the item and the FileRef of the item. If the FileRef is empty, then the method tries to get it from the server.

The method returns the error code as an integer.

Here comes the code, I hope it will help somebody to do the deletion in SharePoint document libraries:

private int? WSDelete(String listName, int id, String fileRef)
// in this example we use the default
ListService.Credentials =
// if fileRef is empty then we need further information
if (String.IsNullOrEmpty(fileRef))

// get the document / folder with
the specified ID
xmlQuery = new XmlDocument();

Name='ID' /><Value
// we need the ID and the name
xmlViewFields = new XmlDocument();

xmlViewFields.LoadXml("<ViewFields><FieldRef Name='ID'
/><FieldRef Name='FileRef' /></ViewFields>"
// check the documents / folders
in subfolders too
xmlQueryOptions = new

Scope='RecursiveAll' /></QueryOptions>"
XmlNode getResult =
ListService.GetListItems(listName, null, xmlQuery, xmlViewFields, null,
xmlQueryOptions, null);
XmlNamespaceManager nsmgr = new


nsmgr.AddNamespace("rs", "urn:schemas-microsoft-com:rowset");
XmlNode getNode =
getResult.SelectSingleNode("rs:data/z:row", nsmgr);
if (getNode !=


XmlAttribute fileRefAttr =

// should not be null, but we check
(fileRefAttr !=


fileRef =

int pos =

if (pos >


fileRef = fileRef.Substring(pos +


// build the delete request based on the information we
XmlDocument deleteRequest = new XmlDocument();
String deleteBatch = String.Format("<Batch


"<Method ID='1' Cmd='Delete'>"

"<Field Name='ID'>{0}</Field>"

"<Field Name='FileRef'>{1}</Field>"


"</Batch>", id, fileRef);
XmlNode deleteResult = ListService.UpdateListItems("Shared
, deleteRequest.DocumentElement);
XmlNamespaceManager nsmgr2 = new

nsmgr2.AddNamespace("default", "");

int? errorCode = null;
XmlNode errorCodeNode =
if (errorCodeNode != null)

String errorCodeString =

int hexPos =
(hexPos > -1)


errorCodeString = errorCodeString.Substring(hexPos +

errorCode =
return errorCode;

June 11, 2009

Columns missing when using the Lists.GetListItems SharePoint web service

Filed under: SharePoint, Web service — Tags: , — Peter Holpar @ 03:21

Note: This is a repost, the original one was published on SharePoint Blogs on June 11, 2009

More people complained about that the result of the Lists.GetListItems method call does not contain the empty fields, so the resulting XML cannot be load into a DataSet:

A dirty workaround for that issue may be to adding the missing attributes to the XML from code before trying to load into the DataSet.

In the following code I illustrate this approach. In this code I first get the columns of the default view of the list using the List.GetListAndView method, then retrieve the data using the Lists.GetListItems method, iterate through all the rows and columns, and if an attribute is missing for a column, add an empty attribute.

  1. String listName = "YourListName";
  2. listService.Credentials = new NetworkCredential("user", "password", "domain");
  3. XmlNamespaceManager nsmgr;
  5. // get info about the default view
  6. // the 2nd parameter is null -> it is the default view
  7. XmlNode listView = listService.GetListAndView(listName, null);
  8. nsmgr = new XmlNamespaceManager(listView.OwnerDocument.NameTable);
  9. nsmgr.AddNamespace("a", ";);
  11. List<String> fieldNames = new List<string>();
  12. foreach (XmlNode field in listView.SelectNodes("a:List/a:Fields/a:Field", nsmgr))
  13. {
  14.     XmlAttribute attr = field.Attributes["Name"];
  15.     // it should not be null, but we check it
  16.     if (attr != null)
  17.     {
  18.         // we store all fields in a collection for later use
  19.         fieldNames.Add(attr.Value);
  20.     }
  21. }
  23. // get data from list
  24. XmlNode items = listService.GetListItems(listName, null, null, null, null, null, null);
  25. nsmgr = new XmlNamespaceManager(items.OwnerDocument.NameTable);
  26. nsmgr.AddNamespace("z", "#RowsetSchema");
  27. nsmgr.AddNamespace("rs","urn:schemas-microsoft-com:rowset");
  28. XmlNodeList itemNodeList = items.SelectNodes("rs:data/z:row", nsmgr);
  29. foreach(XmlNode itemNode in itemNodeList)
  30. {
  31.   foreach (String fieldName in fieldNames)
  32.   {
  33.     String wsFieldName = "ows_" + fieldName;
  34.     XmlAttribute attr = itemNode.Attributes[wsFieldName];
  35.     // if the attribute is missing, we should add it
  36.     if (attr == null)
  37.     {
  38.       attr = itemNode.OwnerDocument.CreateAttribute(wsFieldName);
  39.       itemNode.Attributes.Append(attr);
  40.     }
  41.   }
  42. }
  43. DataSet listDataSet = new DataSet();
  44. // read the result into a data set
  45. XmlTextReader readerListDataSet = new XmlTextReader(items.OuterXml, XmlNodeType.Document, null);
  46. listDataSet.ReadXml(readerListDataSet);

After this kind of preparation of XML it can be loaded into the DataSet.

You should know that this approach may not scale and perform well in case of a large amount of data, and I consider it a dirty workaround, but I don’t know currently other solution for this request.

Blog at